Fibrechannel Analyse
From Lolly's Wiki
Kategorie:Solaris Kategorie:Brocade
Fibrechannel Analyse unter Solaris
Kommandos : Solaris
luxadm
luxadm -e port
Gibt die Hardwarepfade der vorhandened Fibrechannelports und deren Status aus:
# luxadm -e port
/devices/pci@79,0/pci10de,378@b/pci1077,143@0/fp@0,0:devctl CONNECTED
/devices/pci@79,0/pci10de,378@b/pci1077,143@0,1/fp@0,0:devctl NOT CONNECTED
/devices/pci@79,0/pci10de,376@e/pci1077,143@0/fp@0,0:devctl CONNECTED
/devices/pci@79,0/pci10de,376@e/pci1077,143@0,1/fp@0,0:devctl NOT CONNECTED
2 Dualport Karten: /devices/pci@79,0/pci10de,378@b/pci1077,143@0 und ...,1 /devices/pci@79,0/pci10de,376@e/pci1077,143@0 und ...,1
# prtdiag -v | head -1
System Configuration: Sun Microsystems Sun Fire X4440
Aus der Seite Sun x86 Platforms: Matrix of Recognized Device Paths (Doc ID 1277396.1) (Oracle Support Login benötigt):
Sun Fire x4440 (Tucana) PCI: PCIe SLOT0 /pci@0,0/pci10de,375@f/pci1000,3150@0 // with PCI Express 8-Port SAS/SATA HBA PCIe SLOT0 /pci@0,0/pci10de,375@f/ // without PCI Express 8-Port SAS/SATA HBA PCIe SLOT1 /pci@0,0/pci10de,376@e/ PCIe SLOT2 /pci@7c,0/pci10de,377@f/ PCIe SLOT3 /pci@0,0/pci10de,377@a/ PCIe SLOT4 /pci@7c,0/pci10de,376@e/ PCIe SLOT5 /pci@7c,0/pci10de,378@b/ (7c can be renamed something else depending on BIOS/OS version)
Also stecken unsere Karten in Slot 4 und 5.
luxadm -e dump_map <HW_path>
Gibt die Tabelle der bekannten Geräte an einem Port aus
# luxadm -e dump_map /devices/pci@79,0/pci10de,378@b/pci1077,143@0/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 30200 0 202600a0b86e10e4 200600a0b86e10e4 0x0 (Disk device)
1 30600 0 202700a0b86e10e4 200600a0b86e10e4 0x0 (Disk device)
2 10100 0 203400a0b85bb030 200400a0b85bb030 0x0 (Disk device)
3 10500 0 203500a0b85bb030 200400a0b85bb030 0x0 (Disk device)
4 10200 0 202600a0b86e103c 200600a0b86e103c 0x0 (Disk device)
5 11400 0 202700a0b86e103c 200600a0b86e103c 0x0 (Disk device)
6 30100 0 203200a0b85aeb2d 200200a0b85aeb2d 0x0 (Disk device)
7 30500 0 203300a0b85aeb2d 200200a0b85aeb2d 0x0 (Disk device)
8 10800 0 2100001b32902d45 2000001b32902d45 0x1f (Unknown Type,Host Bus Adapter)
Erklärung der interessanten Spalten:
- Port_ID <Switch_ID><Switchport><??>
Es sind also offensichtlich 2 Switches in der Fabric an Port /devices/pci@79,0/pci10de,378@b/pci1077,143@0/fp@0,0:devctl und zwar mit der ID 1 und mit der ID 3. Switch ID 1 Port 1 und 5 : Node WWN 200400a0b85bb030 Port 2 und 14 : Node WWN 200600a0b86e103c Port 8 : Node WWN 2000001b32902d45 (Wir selbst) Switch ID 3 Port 1 und 5 : Node WWN 200200a0b85aeb2d Port 2 und 6 : Node WWN 200600a0b86e10e4 Wir hängen also mit 2 Storages auf dem Switch mit der ID 1 und haben eine Verbindung zu einem Switch mit der ID 3 an dem 2 weitere Storages hängen.
- Node WWN
Wir sehen hier 4 Disk Devices mit jeweils 2 Einträgen (Gleiche Node WWN)
- Port WWN
Dies ist die Port WWN der an den Switch angeschlossenen Geräte (unter 8 finden wir uns selbst). Pro Storage sehen wir hier 2 Port WWNs, also 2 Pfade über unseren einen Hostport.
Daher nachher 4 Pfade (2 Pro Hostport) beim #mpathadm list lu.
- Type
Disk Device: Storage Host Bus Adapter: FC-Karte
luxadm probe
Auflistung aller erkannten Fibrechanneldevices
#> luxadm probe
Found Fibre Channel device(s):
Node WWN:200600a0b86e10e4 Device Type:Disk device
Logical Path:/dev/rdsk/c8t600A0B80006E10E40000DC1C52E8B751d0s2
...
luxadm display <Diskpath|WWN>
#> luxadm display /dev/rdsk/c8t600A0B80006E10E40000DC1C52E8B751d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c8t600A0B80006E10E40000DC1C52E8B751d0s2
Vendor: SUN
Product ID: STK6580_6780
Revision: 0784
Serial Num: SP01068442
Unformatted capacity: 204800.000 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x300
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c8t600A0B80006E10E40000DC1C52E8B751d0s2
/devices/scsi_vhci/disk@g600a0b80006e10e40000dc1c52e8b751:c,raw
Controller /dev/cfg/c4
Device Address 202600a0b86e10e4,5
Host controller port WWN 2100001b328a417f
Class primary
State ONLINE
Controller /dev/cfg/c4
Device Address 202700a0b86e10e4,5
Host controller port WWN 2100001b328a417f
Class secondary
State STANDBY
Controller /dev/cfg/c6
Device Address 201600a0b86e10e4,5
Host controller port WWN 2100001b32904445
Class primary
State ONLINE
Controller /dev/cfg/c6
Device Address 201700a0b86e10e4,5
Host controller port WWN 2100001b32904445
Class secondary
State STANDBY
- Vendor: SUN
Hersteller
- Product ID: STK6580_6780
Also ein StorageTek 6580/6780
- Revision: 0784
Grobe Firmwarepeilung (Firmware Version: 07.84.47.10)
Siehe hier #lsscs list array <array_name>
- Serial Num: SP01068442
Praktisch, wenn man mit NetApps arbeitet, um die LUNs zuzuordnen.
- Unformatted capacity: 204800.000 MBytes
Immer gut zu wissen
- Write Cache: Enabled
Die Batterie im Storage sollte also OK sein ;-)
- Path(s):
Rawdevicepath Hardwaredevicepath Jetzt folgen immer pro Pfad zu diesem Device ein Block aus Controller (siehe unten) Device Address <Port WWN vom Device>,<LUN ID> Class <primary|secondary> (siehe unten) State <Online|Standby|Oflline>
Zuweisung Controller zum FC-Port über:
# ls -al /dev/cfg/c6
lrwxrwxrwx 1 root root 60 Sep 3 2009 /dev/cfg/c6 -> ../../devices/pci@79,0/pci10de,376@e/pci1077,143@0/fp@0,0:fc
Man sieht den Hardwarepfad von #luxadm -e port
Class:
Via ALUA (Asymmetric Logical Unit Access) teilt das Device dem Host mit, über welche Pfade der Host primär auf die LUN zugreifen soll.
fcinfo
fcinfo hba-port
Gibt ein paar Infos über Hersteller, Modell, Firmware, Port und Node WWN, Current Speed, ... aus
#> fcinfo hba-port
HBA Port WWN: 2100001b328a417f
OS Device Name: /dev/cfg/c4
Manufacturer: QLogic Corp.
Model: 375-3356-02
Firmware Version: 05.06.00
FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
Serial Number: 0402R00-0918701860
Driver Name: qlc
Driver Version: 20110825-3.06
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b328a417f
HBA Port WWN: 2101001b32aa417f
OS Device Name: /dev/cfg/c5
Manufacturer: QLogic Corp.
Model: 375-3356-02
Firmware Version: 05.06.00
FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
Serial Number: 0402R00-0918701860
Driver Name: qlc
Driver Version: 20110825-3.06
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 2001001b32aa417f
HBA Port WWN: 2100001b32904445
OS Device Name: /dev/cfg/c6
Manufacturer: QLogic Corp.
Model: 375-3356-02
Firmware Version: 05.06.00
FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
Serial Number: 0402R00-0918701887
Driver Name: qlc
Driver Version: 20110825-3.06
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b32904445
HBA Port WWN: 2101001b32b04445
OS Device Name: /dev/cfg/c7
Manufacturer: QLogic Corp.
Model: 375-3356-02
Firmware Version: 05.06.00
FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
Serial Number: 0402R00-0918701887
Driver Name: qlc
Driver Version: 20110825-3.06
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 2001001b32b04445
fcinfo remote-port --port <HBA Port WWN> --linkstat
# fcinfo remote-port --port 2100001b32904445 --linkstat
Remote Port WWN: 201600a0b86e103c
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200600a0b86e103c
Link Error Statistics:
Link Failure Count: 3
Loss of Sync Count: 3
Loss of Signal Count: 2
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 201700a0b86e103c
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200600a0b86e103c
Link Error Statistics:
Link Failure Count: 4
Loss of Sync Count: 261
Loss of Signal Count: 4
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 202200a0b85aeb2d
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200200a0b85aeb2d
Link Error Statistics:
Link Failure Count: 2
Loss of Sync Count: 1
Loss of Signal Count: 2
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 202300a0b85aeb2d
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200200a0b85aeb2d
Link Error Statistics:
Link Failure Count: 2
Loss of Sync Count: 1
Loss of Signal Count: 2
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 201600a0b86e10e4
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200600a0b86e10e4
Link Error Statistics:
Link Failure Count: 3
Loss of Sync Count: 1
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 201700a0b86e10e4
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200600a0b86e10e4
Link Error Statistics:
Link Failure Count: 3
Loss of Sync Count: 1
Loss of Signal Count: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 202400a0b85bb030
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200400a0b85bb030
Link Error Statistics:
Link Failure Count: 2
Loss of Sync Count: 1
Loss of Signal Count: 2
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
Remote Port WWN: 202500a0b85bb030
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200400a0b85bb030
Link Error Statistics:
Link Failure Count: 2
Loss of Sync Count: 1
Loss of Signal Count: 3
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 0
Invalid CRC Count: 0
mpathadm
mpathadm list lu
cfgadm
cfgadm -al -o show_FCP_dev [<controller>]
# cfgadm -al -o show_FCP_dev | grep unusable
c8::21000024ff2d49a2,0 disk connected configured unusable
c8::21000024ff2d49a2,1 disk connected configured unusable
c8::21000024ff2d49a2,2 disk connected configured unusable
c8::21000024ff2d49a2,3 disk connected configured unusable
c8::21000024ff2d49a2,4 disk connected configured unusable
c8::21000024ff2d49a2,5 disk connected configured unusable
c8::21000024ff2d49a2,6 disk connected configured unusable
c8::21000024ff2d49a2,7 disk connected configured unusable
c8::21000024ff2d49a2,8 disk connected configured unusable
c8::21000024ff2d49a2,9 disk connected configured unusable
c8::21000024ff2d49a2,10 disk connected configured unusable
c9::203400a0b839c421,31 disk connected configured unusable
c9::203400a0b84913d2,31 disk connected configured unusable
c9::203500a0b839c421,31 disk connected configured unusable
c9::203500a0b84913d2,31 disk connected configured unusable
cfgadm -c unconfigure -o unusable_SCSI_LUN <unusable device>
# cfgadm -c unconfigure -o unusable_SCSI_LUN c8::21000024ff2d49a2
cfgadm -o force_update -c configure <controller>
Rescan LUNs. Be careful! Does a forcelip!
# cfgadm -o force_update -c configure c10
Kommandos : Common Array Manager
lsscs
Ist unter Solaris in /opt/SUNWsefms/bin
lsscs list array
lsscs list array <array_name>
lsscs list -a <array_name> fcport
Kommandos : Brocade
Switch-Kommandos
switchshow
san-sw_11:admin> switchshow
switchName: san-sw_11
switchType: 71.2
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 1
switchId: fffc01
switchWwn: 10:00:00:05:33:df:43:5a
zoning: ON (Fabric1)
switchBeacon: OFF
Index Port Address Media Speed State Proto
==============================================
0 0 010000 id N8 No_Light FC
1 1 010100 id N8 Online FC E-Port 10:00:00:05:33:df:bd:b9 "san-sw_21" (downstream)
2 2 010200 id N8 Online FC F-Port 21:00:00:24:ff:05:74:e4
3 3 010300 id N8 Online FC F-Port 50:0a:09:81:8d:32:5d:c4
4 4 010400 id N8 No_Light FC
5 5 010500 id N8 Online FC E-Port 10:00:00:05:33:df:bd:b9 "san-sw_21"
6 6 010600 id N4 Online FC F-Port 20:06:00:a0:b8:32:38:17
7 7 010700 id N4 Online FC F-Port 20:07:00:a0:b8:32:38:17
8 8 010800 id N4 Online FC F-Port 21:00:00:1b:32:91:4c:ed
9 9 010900 id N4 Online FC F-Port 21:00:00:1b:32:98:05:1a
10 10 010a00 id N8 Online FC F-Port 21:00:00:24:ff:4a:d3:bc
11 11 010b00 id N8 No_Light FC
12 12 010c00 id N8 No_Light FC
13 13 010d00 id N8 No_Light FC
14 14 010e00 id N8 No_Light FC
15 15 010f00 id N8 No_Light FC
16 16 011000 -- N8 No_Module FC (No POD License) Disabled
17 17 011100 -- N8 No_Module FC (No POD License) Disabled
18 18 011200 -- N8 No_Module FC (No POD License) Disabled
19 19 011300 -- N8 No_Module FC (No POD License) Disabled
20 20 011400 -- N8 No_Module FC (No POD License) Disabled
21 21 011500 -- N8 No_Module FC (No POD License) Disabled
22 22 011600 -- N8 No_Module FC (No POD License) Disabled
23 23 011700 -- N8 No_Module FC (No POD License) Disabled
Was sagt uns das?
- Dies ist der "Principal" (alle andere sind "Subordinate") der Fabric "Fabric1" (switchRole:, zoning:)
- Der Switch ist gezoned (zoning:)
- SwitchID ist "fffc01"
- Es ist ein 24-Port Switch
- Es gibt einen doppelten ISL (InterSwitchLink) zu einem anderen Switch E-Port (san-sw_21)
- 6 Ports sind mit SFPs bestückt, aber nicht belegt (0,4,11-15)
- 8 Ports haben keine Lizenz und auch kein SFP (No_Module)
- 9 Ports sind belegt
san-sw_11:root> fabricshow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
-------------------------------------------------------------------------
1: fffc01 10:00:00:05:33:df:43:5a 192.168.1.117 0.0.0.0 >"san-sw_11"
2: fffc02 10:00:00:05:33:df:bd:b9 192.168.1.119 0.0.0.0 "san-sw_21"
The Fabric has 2 switches
Port-Kommandos
portstatsshow
portstatsclear
Zone-Kommandos
zoneshow
alicreate
alishow
Backup der Switchconfig per Script
Put the backup host ssh-pub-key on the switches
fcsw1:root> cat >/root/.ssh/authorized_keys <<EOF
> ssh-dss AAAAB3NzaC1...
...
...
lF8qsgtTD8cc= root@host
> EOF
Generate ssh-key on the switches
fcsw1:root> ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
2a:23:33:...:69:bc:25:a5:f9 root@fcsw1
The key's randomart image is:
+--[ RSA 2048]----+
| |
| ... |
| |
+-----------------+
Copy the key to your backup users ~/.ssh/authorized_keys on backup host
fcsw1:root> cat /root/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAA...
...
KHnw1T1NaQ== root@fcsw1
Now the script on the backup host
# cat /opt/bin/backup_brocade_config
#!/bin/bash
SWITCHES="
172.30.40.50
172.30.40.51
"
LOCALUSER="backupuser"
BACKUPDIR="brocade_backup"
BACKUPHOST="172.30.40.10"
DATE="$(date '+%Y%m%d-%H%M%S')"
for switch in ${SWITCHES} ; do
printf "Backing up ${switch} to ~${LOCALUSER}/${BACKUPDIR}/${switch}_config_${date}.txt... "
ssh root@${switch} /fabos/link_sbin/configupload -all -p scp ${BACKUPHOST},${LOCALUSER},${BACKUPDIR}/${switch}_config_${DATE}.txt
done
Script zum parsen einer configupload-Datei
#!/usr/bin/nawk -f
BEGIN{
vendor["001438"]="Hewlett-Packard";
vendor["00a098"]="NetApp";
vendor["0024ff"]="Qlogic";
vendor["001b32"]="Qlogic";
vendor["0000c9"]="Emulex";
vendor["00e002"]="CROSSROADS SYSTEMS, INC.";
}
/\[Zoning\]/,/^$/ {
if(/^cfg./){
split($0,cfgparts,":");
gsub(/^cfg./,"",cfgparts[1]);
cfg[cfgparts[1]]=cfgparts[2];
}
else if(/^zone./) {
zonename=$0;
gsub(/:.*$/,"",zonename);
gsub(/^zone./,"",zonename);
zonemembers=$0;
gsub(/^[^:]*:/,"",zonemembers);
zone[zonename]=zonemembers;
}
else if(/^alias./) {
aliasname=$0;
gsub(/:.*$/,"",aliasname);
gsub(/^alias./,"",aliasname);
aliasmembers=$0;
gsub(/^[^:]*:/,"",aliasmembers);
alias[aliasname]=aliasmembers;
if(length(aliasname)>longestalias){
longestalias=length(aliasname);
}
}
else if(/^enable:/) {
cfgenabled=$0;
gsub(/^enable:/,"",cfgenabled);
}
}
END {
print "Config:",cfgenabled;
split(cfg[cfgenabled],active_zones,";");
for(active_zone in active_zones) {
split(zone[active_zones[active_zone]],zone_members,";");
asort(zone_members);
print "Zone",active_zones[active_zone],"(",length(zone_members),"Members ):";
for(zone_member in zone_members){
member=zone_members[zone_member];
if(alias[member]!=""){
member=alias[member];
}
WWN=member;
gsub(/:/,"",WWN);
if(WWN ~ /^5/){start=2;}else{start=5;}
vendor_id=substr(WWN,start,6);
printf " Member: %s\t",member;
if(alias[zone_members[zone_member]]!=""){
format=sprintf("%%s%%%ds\t",longestalias-length(zone_members[zone_member]));
printf format,zone_members[zone_member]," ";
}
printf "%s\n",vendor[vendor_id];
}
}
print "Create config:";
printf "cfgdelete \"%s\"\n",cfgenabled;
for(active_zone in active_zones) {
split(zone[active_zones[active_zone]],zone_members,";");
asort(zone_members);
for(zone_member in zone_members){
member=zone_members[zone_member];
if(alias[member]!=""){
printf "alicreate \"%s\",\"%s\"\n",member,alias[member];
alias[member]="";
}
}
printf "zonecreate \"%s\",\"%s\"\n",active_zones[active_zone],zone[active_zones[active_zone]];
if(!secondelement){
secondelement=1;
printf "cfgcreate";
} else {
printf "cfgadd ";
}
printf " \"%s\",\"%s\"\n",cfgenabled,active_zones[active_zone];
}
printf "cfgsave\ncfgenable \"%s\"\n",cfgenabled;
}
