NetApp and Solaris
From Lolly's Wiki
Solaris NetApp Just some unsorted lines...
Settings in Solaris
Settings for MPxIO over FC:
/kernel/drv/ssd.conf
###### START changes by host_config #####
ssd-config-list="NETAPP LUN", "physical-block-size:4096, retries-busy:30, retries-reset:30, retries-notready:300, retriestimeout:10, throttle-max:64, throttle-min:8";
###### END changes by host_config ####
Check it out
# iostat -Er | /opt/sfw/bin/gawk 'BEGIN{command="echo ::ssd_state | mdb -k"; while(command|getline){if(/^un [0-9]+:/ && $NF != "0"){ssd=$2;gsub(/:$/,"",ssd);while(!/^}/){command|getline;if(/un_phy_blocksize/){un_phy_blocksize[ssd]=strtonum($NF);}}}};close(command);}/ssd/{ssd=$1;gsub(/^ssd/,"",ssd);getline;split($0,vendor,",");printf "ssd: %s\tun_phy_blocksize: %d\t%s\t%s\n",ssd,un_phy_blocksize[ssd],vendor[1],vendor[4];}'
Alignment and ZFS
First read ZFS zpools create misaligned I/O in Solaris 11 and Solaris 10 Update 8 and later (407376).
If you have 4k as block size in your storage use ashift=12 (alignment shift exponent).
Status of alignment
# ssh filer01 "priv set -q diag ; lun show -v all; priv set"
-------------------------------------------------------------------------------
LUN for ZFS
-------------------------------------------------------------------------------
/vol/ZoneLUNs/Zone01.lun 50g (53687091200) (r/w, online, mapped)
Serial#: 800KP+EpO-33
Share: none
Space Reservation: enabled
Multiprotocol Type: solaris_efi
Maps: SUN_SERVER01_SERVER02=40
Occupied Size: 46.2g (49583595520)
Creation Time: Wed Jan 7 11:37:58 CET 2015
---> Alignment: partial-writes
Cluster Shared Volume Information: 0x0
Space_alloc: disabled
report-physical-size: enabled
Read-Only: disabled
-------------------------------------------------------------------------------
LUN for Oracle Database
-------------------------------------------------------------------------------
/vol/TEMP201/TEMP201 25g (26843545600) (r/w, online, mapped)
Serial#: 800KP+EpO-2t
Share: none
Space Reservation: enabled
Multiprotocol Type: solaris_efi
Maps: KSP_SUN_SRV06_SRV07=35
Occupied Size: 21.6g (23195856896)
Creation Time: Fri Jul 4 11:02:34 CEST 2014
---> Alignment: misaligned
Cluster Shared Volume Information: 0x0
Space_alloc: disabled
report-physical-size: enabled
Read-Only: disabled
...
Or use "lun alignment show":
# ssh filer01 "priv set -q diag ; lun alignment show; priv set"
-------------------------------------------------------------------------------
LUN for ZFS
Wide spread reads. I think the ashift is not correct.
-------------------------------------------------------------------------------
/vol/ZoneLUNs/Zone01.lun
Multiprotocol type: solaris_efi
Alignment: partial-writes
Write alignment histogram percentage: 5, 5, 4, 6, 4, 6, 14, 5
Read alignment histogram percentage: 8, 7, 10, 7, 7, 8, 36, 5
Partial writes percentage: 47
Partial reads percentage: 9
-------------------------------------------------------------------------------
LUN for Oracle Database
-------------------------------------------------------------------------------
/vol/TEMP201/TEMP201
Multiprotocol type: solaris_efi
Alignment: misaligned
Write alignment histogram percentage: 0, 0, 0, 0, 0, 0, 99, 0
Read alignment histogram percentage: 0, 0, 8, 0, 0, 0, 77, 0
Partial writes percentage: 0
Partial reads percentage: 14
Or "stats show lun":
stats show -e lun:/vol/TEMP201:.*_align_histo.*
ashift=12? Why 12?
# echo "2^12" | bc -l
4096
OK... 4k... I see.
What ashift do I have?
# zdb | egrep ' name|ashift'
name: 'apache_pool'
ashift: 9
name: 'mysql_pool'
ashift: 9
...
Create ZPools on NetApp LUNs with this syntax
# zpool create -o ashift=12 <mypool> mirror <vdev1> <vdev2>
