IBM System Storage™ DS4000 Linux RDAC multipath driver package version 09.00.A5.18 FOR USE IN LINUX 2.4 KERNEL OPERATING SYSTEM ENVIRONMENTS ONLY. Note: For Linux 2.6 kernel operating system environments, please refer to the IBM DS4000 Linux RDAC package version 09.01.B5.xx instead. NOTE TO SERVICE – Reference RETAIN # N/A (C) Copyright International Business Machines Corporation 1999, 2006. All rights reserved. US Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Note: Before using this information and the product it supports, read the general information in section 6.0 "Trademarks and Notices" in this document. Last Update Date: 07/28/2006 Note: The IBM System Storage DS4000 Storage Manager version 9 Installation and Support Guide for Windows 2000/Server 2003, Netware, ESX Server, and Linux publication or the IBM System Storage DS4000 Storage Manager version 9 Installation and Support Guide for AIX, HP-UX, Solaris, and Linux on POWER publication, which is included in the Docs directory in any of the DS4000 Storage Manager CDs, is also available on IBM's Support Web Site as a downloadable Portable Document Format (PDF) file. These publications provide useful information on the installation of all DS4000 Storage Manager host software packages including the RDAC package. Important: For this release, please refer to the "IBM DS4000 Storage Manager Version 9 Installation and Support Guide for AIX, HP-UX, Solaris and Linux on POWER" located at http://www.ibm.com/servers/storage/support/disk/ for all installation and support notes pertaining to Linux on pSeries. Follow these instructions to get to the proper manual - Click the link for your DS4000 storage server. When the page opens, click the Download tab. Then, click the link for Current recommended Firmware and Storage Manager. The tables on the resulting page contain links to the DS4000 Storage Manager publications. Products Supported: ---------------------------------------------------------------- | New Model | Old Model | Machine Type | Model | |------------|-----------|--------------|------------------------| | DS4800 | N/A | 1815 | 82A, 82H, 84A, 84H, | | | | | 88A, 88H, 80A, 80H | |------------|-----------|--------------|------------------------| | DS4700 | N/A | 1814 | 70A, 70H, 72A, 72H, | |------------|-----------|--------------|------------------------| | DS4500 | FAStT 900 | 1742 | 90X, 90U | |------------|-----------|--------------|------------------------| | DS4400 | FAStT 700 | 1742 | 1RX, 1RU | |------------|-----------|--------------|------------------------| | DS4300 | FAStT 600 | 1722 | 60U, 60X (Standard/dual| | | | | cntl or Turbo Option) | | | | | 60J, 60K | |------------|-----------|--------------|------------------------| | DS4200 | N/A | 1814 | 7VA, 7VH | |------------|-----------|--------------|------------------------| | DS4100 | FAStT 100 | 1724 | 100 (Standard/Dual cntl| | | | | Option) | ---------------------------------------------------------------- CONTENTS --------- 1.0 Overview 2.0 Installation and Setup Instructions 3.0 Configuration Information 4.0 Unattended Mode 5.0 Web Sites and Support Phone Number 6.0 Trademarks and Notices 7.0 Disclaimer 1.0 Overview: 1.1 Description The Linux RDAC provides redundant failover/failback support for the logical drives in the DS4000 storage subsystem that are mapped to the Linux host server. The Linux host server must have Fibre Channel (FC) connections to the host ports of both controllers A and B of the DS4000 storage subsystem. It is provided as a redundant path failover/failback driver alternative to the Linux FC host bus adapter failover device driver. There are two versions of the Linux RDAC. The version (09.00.A5.18) that this README is packaged with is for Linux 2.4 kernels only like Redhat EL 3 and SuSe SLES 8. For Linux 2.6 kernel environment, please refer to the Linux RDAC package version 09.01.B5.XX instead. Please refer to the Dependencies section for the required code levels and Linux kernel versions. In addition, please refer to the http://www.ibm.com/servers/storage/support/disk/ IBM System Storage™ Disk Storage Systems Technical Support web site for the latest supported DS4000 storage subsystem models, host adapters, device drivers, Linux kernel versions and updated readme. Note: Deviations from the listed code levels are not supported. Please contact IBM Support or representatives for help if required. 1.2 Limitations 1. This version of DS4000 Linux RDAC will not support any Linux 2.6 kernels like SuSe SLES 9 or Redhat 4 Linux operating system on Intel/AMD architecture or Linux on POWER(LoP) servers. Please refer to the Linux RDAC package version 09.01.B5.XX instead. 2. This version of DS4000 Linux RDAC does not support SuSE SLES 8 Linux on POWER (LoP) servers. 3. The Linux RDAC driver cannot co-exist with a Fibre Channel HBA-level multipath failover/failback driver such as the 6.06.63-fo or 7.07.61-fo Linux device driver for the IBM DS4000 fc2-133 or FC 4Gbps single and dual port Host Bus Adapters driver on servers with Intel or AMD architecture processors or the 4.31j_M SLES 8 RDAC driver on IBM LoP servers. Note: You might have to modify the driver make file for it to be compiled in the non-failover mode. 4. SteelEye's™ LifeKeeper® clustering is supported on LoP, Intel or AMD architecture processors servers. However, it is currently supported with a maximum of 8 luns only. Please use the following additional instructions when setting up the LifeKeeper® cluster in servers with Intel or AMD architecture processors or LoP servers. Also refer to the publications from Steeleye Technology Inc for more information on the cluster configuration/setup. a. Recompile the Fibre Channel Host Bus Adapter driver using the following setting. - Change the setting QLA2X_PERFORMANCE to 0 in the qla_settings.h file. b. Change the following setting in the /etc/default/LifeKeeper file. LCMHBEATTIME=5 LCMNUMHBEATS=16 c. Reboot the Linux host after completing steps a and b. 5. All limitations which apply to the current DS4000 Storage Server when configured with Linux servers also apply to this Linux RDAC release. 6. Auto Logical Drive Transfer(ADT/AVT) mode, is automatically enabled in the Linux storage partitioning host type. This mode causes contention when an RDAc is installed. If using the "Linux" host type, it has to be disabled using the script that is bundled in this Linux RDAC web package or in the \Scripts directory of this DS4000 Storage Manager version 9 support for Linux CD. The name of the script file is DisableAVT_Linux.scr. IMPORTANT: It is recommended, especially in an LoP RDAC server environment, to use the "LNXCL" or "LNXCLVMWARE" host type which has ADT/AVT already disabled instead of using the following steps to disable the ADT/AVT mode in your "Linux" host type partition. Open the DS4000 Storage Manager Enterprise Management window, - Highlight the subsystem - Select Tools - Select Execute script. - A script editing window opens. In this window - Select File - Select Load Script - Give the full path name for the script file (eg . /scripts/DisableAVT_Linux.scr) and click OK - Select Tools - Select Verify and Execute Note: This is assumed that the "Linux" host type is selected for the Storage Partitioning partition that the Linux host server HBA port is defined in. This "Linux" host type has ADT/AVT enabled as default. Please refer to the IBM System Storage DS4000 Storage Manager version 9 Installation and Support Guide for appropriate OS or the Enterprise Window Online help for more information on how to execute a script. 7. The Linux SCSI layer does not support skipped (sparse) LUNs. If the mapped LUNs are not contiguous, the Linux kernel will not scan the rest of the LUNs. Therefore, the LUNs after the skipped LUN will not be available to the host server. The LUNs should always be mapped using consecutive LUN numbers. The SMutil hot_add utility can be used to scan for sparse LUNs. However, after the system is rebooted, the LUNs after the skipped LUN may not be available until the hot-add utility is performed. 8. Fibre channel configuration. If a host server has multiple HBA ports and each HBA port sees both controllers (via a un-zoned switch), the Linux RDAC driver may return I/O errors during controller failover. The recommendation is to use multiple unconnected FC switches to zone the FC switch into multiple zones so that each HBA port sees only one controller in a DS4000 storage subsystem. 9. The RDAC driver reports I/O failures immediately after all paths are failed. When the RDAC driver detects that all paths to a DS4000 storage subsystem are failed, it will report I/O failure immediately. This behavior is different from the IBM Fibre Channel (FC) HBA failover device driver. The FC HBA failover device driver will wait for a certain time out/retry period before reporting an I/O failure to the host application. There is no work-around. 10. The virtual HBA driver can not be loaded if there is no storage array attached. The work-around is to connect the host server to the DS4000 storage subsystem and run the "mppBusRescan" utility, which is previously known as the "hot_add" utility. Note: If no LUN is mapped to the host server partition on the DS4000 storage subsystem, "mppBusRescan"/"hot_add" will not load the mpp_Vhba module. If the server does not recognize any new mapped LUNs after running the "mppBusRescan" utility a few times, consider rebooting the server. 11. The cluster support feature in the Linux RDAC driver is only available for controller 05.4x.xx.xx or later firmware. If a SCSI-2 reserve/release command is addressed to a LUN on an array running a firmware version lower than 05.40, a check condition 05-02-00 (key-asc-ascq) (illegal request, invalid command operation code) will be returned. LoP servers do not support clustering with this Linux RDAC. 12. (For LoP servers only) The "modprobe" command is not supported with this Linux RDAC release on Rhel3 with update 4 when the Linux RDAC driver is installed. Use "insmod" to remove and recover the device driver stack. On LoP pSeries servers with more than 3 processors, using the modprobe command may result in a hung server or panic's. 13. (For LoP Server only) SAN Boot on RHEL3 is restricted to initial root installation on Fibre Channel LUN ID 0. In addition, SAN boot is not supported for JS20 blades with RHEL3 update 5 (RHELu5) in this release due to an error in writing the partition table (BZ 16838.) 14. (For LoP Server only) JS20 blades must not have the following Fibre Channel Expansion card FRU number installed in certain JS20 blade model numbers. FRU number 13N2208 (or Option PN 13N2203) and JS20 8842-21X MT/MN are not compatible. This combination is not supported for Linux on Power releases. Possible intermittent problems are Fibre Channel Expansion card NVRAM corruption symptoms. (CMVC 279060.) 15. Universal Access Luns (UTM LUNs) should not be assigned to LUN ID 0, especially with RDAC's installed. Placing an Universal Access Lun at LUN id 0, may lead to loss of LUN recognition of next sequence LUN ids or partial list of Virtual Luns reported with the RDAC driver. These UTM LUNs should be assigned LUN ID 31. 16. (For LoP Server only) Utilizing standard Emulex installation script (lpfc-install) on SAN booted systems with RDAC installed will remove the currently used lpfc module, causing loss of access to root filesystem. Installation script uses usage count of lpfc module to determine if it is appropriate to remove the module. Installing RDAC driver will cause usage count of lpfc module to go to zero. Work-around Perform the following steps a. Boot system without RDAC driver installed. b. Update Emulex driver and reboot. c. Install RDAC driver and reboot 1.3 Enhancements - New DS4700 Storage Subsystem models 70A/H and 72A/H, which support the attachment of DS4000 EXP710 and EXP810 Storage Expansion Enclosures. - New DS4800 Storage Subsystem Model 80A/H, which provides the same functionality as the DS4500 Storage Subsystem, with better performance. - New 4Gbps Fibre Channel drives in the DS4000 EXP810 Storage Expansion Enclosure. - New DS4200 Express Storage Subsystem models 7VA/H, which support the attachment of DS4000 EXP420 Storage Expansion Enclosures. - New 500G SATA Enhanced Disk Drive Modules (E-DDMs) in the DS4000 DS4700 storage subsystems and the EXP810 Storage Expansion Enclosures. - New 500G SATA Enhanced Value Disk Drive Modules (EV-DDMs) in the DS4000 DS4200 storage subsystem and the EXP420 Storage Expansion Enclosures. 1.4 Level of Recommendations and Prerequisites for the Update: N/A 1.5 Dependencies: The following code levels must be used. Deviations from the listed code levels are not supported. 1.5.1 Linux on POWER (LoP) pSeries servers a. DS4000 storage subsystem Controller firmware and NVSRAM: 1. Storage Controller Firmware versions: a. DS4800: FW_DS4800_06152400 and FW_DS4800_06168800 b. DS4700: FW_DS4700_06168800 c. DS4500: FW_06122700_06100700 d. DS4400: FW_06122700_06100700 e. DS4300 Standard: SNAP_288X_06122700 f. DS4300 Turbo : SNAP_288X_06122700 g. DS4200: FW_DS4200_06168800 h. DS4100 Standard: SNAP_282X_06122700 2. Storage Controller NVSRAM versions ATTENTION: The DS4000 storage subsystem controller firmware version 06.12.xx.xx or higher uses the FC/SATA intermix premium key file to enable the FC/SATA intermix functionality. It does not rely on a certain version of the NVSRAM file like the controller firmware version 06.10.xx.xx. Do not apply this version 06.12.xx.xx or higher of the firmware until a FC/SATA intermix premium feature key file is generated and available. a. DS4800: N1815D480R916V13 (with fw version 06.15.xx.xx and 06.16.xx.xx) b. DS4700: N1814D470R916V17 c. DS4500: N1742F900R912V08 d. DS4400: N1742F700R912V07 e. DS4300 Standard: N1722F600R912V07 (for EXP700/EXP710 ONLY attachment or when FC/SATA intermix premium feature is enabled) f. DS4300 Turbo : N1722F600R912V07 (for EXP700/EXP710 ONLY attachment or when FC/SATA intermix premium feature is enabled) g. DS4200: N1814D420R916V01 h. DS4100 Standard: N1724F100R912V07 i. DS4300 Standard: N1722F600R28enc4 (for EXP100 ONLY attachment) i. DS4300 Turbo : N1722F600R28enc4 (for EXP100 ONLY attachment) ATTENTION: DO NOT LOAD THIS NVSRAM FILE - N1722F600R18enc4 - INTO ANY OF THE DS4300 STORAGE SUBSYSTEMS THAT HAVE DRIVES IN THE DS4300 CHASSIS DRIVE SLOTS. THIS WILL CAUSE THE DS4300 storage subsystem NOT RECOGNIZING ANY OF THE DRIVES IN THE DS4300 CHASSIS. THIS COULD RESULT IN THE DEGRADATION OF THE DEFINED RAID ARRAY OR A LOSS OF ACCESS TO DATA. IF THIS NVSRAM FILE IS LOADED INTO A DS4300 storage subsystem WITHOUT ANY DRIVE EXPANSION ENCLOSURES ATTACHED, IT WILL PREVENT THE storage subsystem FROM COMPLETING THE BOOT PROCESS THAT IS INITIATED TO ACTIVATE THE NEW NVSRAM SETTINGS, WHICH MIGHT RESULT IN LOST OF MANAGEMENT ACCESS TO the DS4300 storage subsystem. THERE IS NO SAFETY MECHANISM TO PREVENT THE UNINTENTIONAL LOADING OF THIS NVSRAM FILE. IN ADDITION, DO NOT USE THIS NVSRAM FILE IN THE DS4300 STORAGE SUBSYSTEMS WITH FC/SATA INTERMIX PREMIUM FEATURE ENABLE. b. Supported Fibre Channel Host Bus Adapters firmware and device driver versions for IBM Feature Code 6228, 6239, or 5716(6239-CR). HBA Firmware Minimum Levels for RHEL3-u4 and RHEL3-u5 ppc64 Linux(LoP): FC 5716: F-code 1.90A4 FC 6239: F-code 1.81x1 FC 6228: F-code 3.91A1 pSeries adapter code can be downloaded from: http://techsupport.services.ibm.com/server/mdownload2/download.html HBA non-rdac device drivers: (Emulex lpfc comes with the Kernel) 7.3.3, elxlinuxapps-2.1a24-7.3.2-1-2 pSeries Emulex "Application Helper Module" and the "Application Kit" for the 7.3.2 device driver can be downloaded from: http://www.emulex.com/ts/docoem/ibm/index.htm - For JS20 Fibre Channel Expansion card, the supported device driver is 7.07.04 (non-failover version). c. Linux Kernel version - For LoP-Redhat 3-u7: 2.4.21-40.EL Note: This Linux RDAC driver is not supported in Linux on Power edition of SLES 8 Linux operating systems. d. IBM DS4000 Storage Manager Host software These IBM DS4000 Storage Manager host software packages are for LoP servers running LoP version of Redhat EL 3 update 4 only (RHEL3-u4). These DS4000 Storage Manager host software packages are not available at this time for installation on LoP servers running the SLES 8 or SLES 9 Linux operating systems. For these LoP servers running the SLES 8 or SLES 9, the DS4000 Storage Manager version 9.15 Client program must be installed in out-of-band management configurations on management stations with supported operating systems (i.e. Microsoft Windows, Intel/AMD architecture Linux or AIX servers.) 1. IBM DS4000 Storage Manager Runtime version: 09.16.A5.00-1 SMruntime-LINUX-09.16.A5.00-1.ppc64.rpm 2. IBM DS4000 Storage Manager Client version: 09.16.G5.62-1 SMclient-LINUX-09.16.G5.62-1.noarch.rpm 3. IBM DS4000 Storage Manager Utilities version: 09.16.A5.02-1 SMutil-LINUX-09.16.A5.02-1.ppc64.rpm 4. IBM DS4000 Storage Manager Agent version: 09.16.A5.02-1 SMagent-LINUX-09.16.A5.02-1.ppc64.rpm OR 5. IBM DS4000 Installer wizard: 09.16.A5.62 SMIA-LINUXPPC-09.16.A5.62.bin 1.5.2 For Linux servers other than the LoP servers. (Intel 32bit architecture (Pentium III, Pentium 4, Zeon ...), Intel 64bit architecture (Itanium), Intel EM64T architecture or AMD 64bit (Opteron ...) a. DS4000 storage subsystem Controller firmware and NVSRAM: 1. Storage Controller Firmware versions: a. DS4800: FW_DS4800_06152400 and FW_DS4800_06168800 b. DS4700: FW_DS4700_06168800 c. DS4500: FW_06122700_06100700 d. DS4400: FW_06122700_06100700 e. DS4300 Standard: SNAP_288X_06122700 f. DS4300 Turbo : SNAP_288X_06122700 g. DS4200: FW_DS4200_06168800 h. DS4100 Standard: SNAP_282X_06122700 2. Storage Controller NVSRAM versions ATTENTION: The DS4000 storage subsystem controller firmware version 06.12.xx.xx or higher uses the FC/SATA intermix premium key file to enable the FC/SATA intermix functionality. It does not rely on a certain version of the NVSRAM file like the controller firmware version 06.10.xx.xx. Do not apply this version 06.12.xx.xx or higher of the firmware until a FC/SATA intermix premium feature key file is generated and available. a. DS4800: N1815D480R916V13 (with fw version 06.15.xx.xx and 06.16.xx.xx) b. DS4700: N1814D470R916V17 c. DS4500: N1742F900R912V08 d. DS4400: N1742F700R912V07 e. DS4300 Standard: N1722F600R912V07 (for EXP700/EXP710 ONLY attachment or when FC/SATA intermix premium feature is enabled) f. DS4300 Turbo : N1722F600R912V07 (for EXP700/EXP710 ONLY attachment or when FC/SATA intermix premium feature is enabled) g. DS4200: N1814D420R916V01 h. DS4100 Standard: N1724F100R912V07 i. DS4300 Standard: N1722F600R28enc4 (for EXP100 ONLY attachment) i. DS4300 Turbo : N1722F600R28enc4 (for EXP100 ONLY attachment) ATTENTION: DO NOT LOAD THIS NVSRAM FILE - N1722F600R18enc4 - INTO ANY OF THE DS4300 STORAGE SUBSYSTEMS THAT HAVE DRIVES IN THE DS4300 CHASSIS DRIVE SLOTS. THIS WILL CAUSE THE DS4300 storage subsystem NOT RECOGNIZING ANY OF THE DRIVES IN THE DS4300 CHASSIS. THIS COULD RESULT IN THE DEGRADATION OF THE DEFINED RAID ARRAY OR A LOSS OF ACCESS TO DATA. IF THIS NVSRAM FILE IS LOADED INTO A DS4300 storage subsystem WITHOUT ANY DRIVE EXPANSION ENCLOSURES ATTACHED, IT WILL PREVENT THE storage subsystem FROM COMPLETING THE BOOT PROCESS THAT IS INITIATED TO ACTIVATE THE NEW NVSRAM SETTINGS, WHICH MIGHT RESULT IN LOST OF MANAGEMENT ACCESS TO the DS4300 storage subsystem. THERE IS NO SAFETY MECHANISM TO PREVENT THE UNINTENTIONAL LOADING OF THIS NVSRAM FILE. IN ADDITION, DO NOT USE THIS NVSRAM FILE IN THE DS4300 STORAGE SUBSYSTEMS WITH FC/SATA INTERMIX PREMIUM FEATURE ENABLE. b. IBM DS4000/FAStT, FC-2, FC2-133 single and dual port and FC 4Gbps single and dual port Fibre Channel Host Bus Adapters - 7.07.61 non-failover (SLES 8 and RHEL3) Note: The FC HBA device driver must be non-failover version. c. Linux Kernel version - For Redhat EL 3.0-u6 X86_64 : 2.4.21-40.EL up smp - For Redhat EL 3.0-u6 (32-Bit) : 2.4.21-40.EL up smp - For Redhat EL 3.0-u6 (IA64) : 2.4.21-40.EL up smp - For SuSe SLES 8 (32-bit) : 2.4.21-295 up smp - For SuSe SLES 8 (IA64) : 2.4.21-295 up smp d. QLogic SANsurfer (aka FAStT MSJ) 2.0.30b81 e. IBM DS4000 Storage Manager Host software - For 32bit Intel architecture (IA-32) servers or AMD64 and EM64T servers 1. IBM DS4000 Storage Manager Runtime version: 09.16.A5.00-1 SMruntime-LINUX-09.16.A5.00-1.i586.rpm 2. IBM DS4000 Storage Manager Client version: 09.16.G5.62-1 SMclient-LINUX-09.16.G5.62-1.noarch.rpm 3. IBM DS4000 Storage Manager Utilities version:09.16.A5.02-1 SMutil-LINUX-09.16.A5.02-1.i386.rpm 4. IBM DS4000 Storage Manager Agent version: 09.16.A5.02-1 SMagent-LINUX-09.16.A5.02-1.i386.rpm OR 5. IBM DS4000 Installer wizard: 09.16.A5.62 SMIA-LINUX-09.16.A5.62.bin - For 64bit Intel architecture (IA-64) servers only 1. IBM DS4000 Storage Manager Runtime version: 09.16.A5.00-1 SMruntime-LINUX-09.16.A5.00-1.ia64.rpm 2. IBM DS4000 Storage Manager Client version: 09.16.G5.62-1 SMclient-LINUX-09.16.G5.62-1.noarch.rpm 3. IBM DS4000 Storage Manager Utilities version:09.16.A5.02-1 SMutil-LINUX-09.16.A5.02-1.ia64.rpm 4. IBM DS4000 Storage Manager Agent version: 09.16.A5.02-1 SMagent-LINUX-09.16.A5.02-1.ia64.rpm OR 5. IBM DS4000 Installer wizard: 09.16.A5.62 SMIA-LINUX64-09.16.A5.62.bin Refer to the http://www.ibm.com/servers/storage/support/disk/ IBM System Storage™ Disk Storage Systems Technical Support web site for the latest update to the supported host adapters, device driver levels, Linux kernel versions, DS4000 controller firmware versions and readme. 2.0 Installation and Setup Instructions ----------------------------------------- 2.1 Prerequisites 1. The Linux RDAC driver is released as a source-code package in the format of a gunzip compressed tar file. To unpack the driver source package, type the following command in the Linux console window: tar -zxvf rdac-LINUX-xx.xx.xx.xx-source.tar.gz where xx.xx.xx.xx is the release version of the RDAC driver. The source files will uncompress to the "linuxrdac" directory. 2. The Host server must have the non-failover Fibre Channel HBA device driver properly built and installed before the Linux RDAC driver installation. Refer to the FC HBA device driver README or the FC HBA User Guide for instructions on installing the non-failover version of the device driver. The driver source tree is included in the package if you need to build it from the source tree. 3. Although the Host server can have different FC HBAs from multiple vendors or different FC HBA models from the same vendors, only one model of FC HBAs can be connected to the IBM DS4000 storage subsystems. 4. Check for the following entries in the appropriate file for a given Linux OS distribution. a. On Intel/AMD processor architecture Redhat AS 2.1 and EL 3.0 servers, it is assumed that there is a scsi_hostadapter entry for the physical HBA driver(s in the /etc/modules.conf file. For example, if the system has the IBM FC2-133 FC HBA, there should be a scsi_hostadapter entry such as alias scsi_hostadapterx qla2300 b. On Intel/AMD processor architecture SuSe SLES 8 servers, it is assumed that the low-level HBA driver names are included in the INITRD_MODULES string in the /etc/sysconfig/kernel file. For example, if the system has the IBM FC2-133 FC HBA, INITRD_MODULES string should include the word "qla2300". c. (For LoP servers only) An example for an LoP server entry: alias scsi_hostadapter1 lpfc 5. (For LoP server only) Editing the /etc/modules.conf file. The /etc/modules.conf file must be edited to include the appropriate following "options" entries. Each "option" driver type must be entered on a separate line. a. options scsi_mod max_scsi_luns=255 <-- this is required in order to be able to <-- see more than 1 device per FAStT b. options lpfc lpfc_nodev_tmo=60 lpfc_inq_pqb_filter=0x01 - lpfc_nodev_tmo=60 <-- This is a required setting for RDAC fail-over - lpfc_inq_pqb_filter=0x01 <--- This parameter was added to fix an issue with the SCSI midlayer in some Linux distributions. The problem can be exhibited when LUNs are reported in /proc/scsi/scsi that do not actually exist (duplicate LUN number 0 on all attached DS4000 storage subsystem, for example). Example: options lpfc lpfc_nodev_tmo=60 lpfc_inq_pqb_filter=0x01 5. The kernel source tree for the Linux kernel version to be built against should already be installed before RDAC driver package installation. 2.2 Build and Installation 3.2.1. This step is for SUSE kernel distribution only. For Redhat kernel distribution, proceed to step 3.2.2. a. Install the kernel-source from the SUSE distribution. b. create a soft link to the kernel source. ln -sf /usr/src/ /usr/src/linux c. To ensure kernel version synchronization between the driver and running kernel, type the following in the Linux console window. Press Enter after each command. - cd /usr/src/linux - make mrproper (completely clean the kernel tree) - make menuconfig - make cloneconfig or cp /boot/vmlinuz.config .config (copy the new config) - make oldconfig (update configuration using .config) - make dep (rebuild the dependencies) - make modules (build the modules-not require on newer kernel version) 3.2.2. RDAC Driver Building for SuSe and Redhat. a. Change to the "linuxrdac" directory. b. To remove the old driver modules in that directory, type the following and press Enter: make clean c. To compile all driver modules and utilities in a multiple CPUs server (SMP kernel), type the following and press Enter: make 3.2.3. RDAC Driver Installation for SuSe and Redhat. a. To copy driver modules to the kernel module tree and build the new RAMdisk image (mpp.img) which includes RDAC driver modules and all driver modules that are needed during boot time. Type the following and press Enter: make install b. Follow the instructions display at the end of the build process to add a new boot menu option which uses /boot/mpp.img as the initial RAMdisk image. IMPORTANT: If your system is a Redhat AS 2.1 or ES 3.0, please note edit the /sbin/mkinitrd script in the following area to change 3000 to 15000: if ['uname -m' = "ia64"]; then IMAGESIZE=400 else IMAGESIZE=15000 fi The following instructions may work in certain Redhat kernel version - add "ramdisk_size=15000" as a kernel boot parameter in the new boot option, similar to the following grub.conf example (Note that it may vary in accordance with different system setup configuration): title Red Hat Linux Advanced Server-up (2.4.9-e.27smp) root (hd0,6) kernel /vmlinuz-2.4.9-e.27smp ro root=/dev/hda7 ramdisk_size=15000 initrd /mpp.img (For LoP servers only) An example of the edited /etc/yaboot.conf file for LoP server is as followed: image=/vmlinux-2.4.21-27.EL label=MPP read-only initrd=mpp-2.4.21-27.EL.img append="root=LABEL=/" ramdisk=15000 2.3 RDAC Driver Post-Installation a. Reboot the system using the new boot menu option. b. To verify that sd_mod, sg, mpp_Upper, qla2300 ( or lpfc for LoP), and mpp_Vhba were loaded after the reboot, type the following and press Enter: /sbin/lsmod c. To verify that the RDAC driver discovered the available physical Luns and created virtual Luns for them, type the following and press Enter: ls -lR /proc/mpp d. You can now issue I/Os to the Luns. e. If you make any changes to MPP configuration file (/etc/mpp.conf) or persistent binding file (/var/mpp/devicemapping), please run mppUpdate to re-build RAMdisk image to include the new file so that the new configuration file (or persistent binding file) can be used on the next system reboot. f. To dynamically re-load the driver stack (scsi_mod, sd_mod, sg, mpp_Upper, , mpp_Vhba) without rebooting the system, you need to first comment out all "scsi_hostadapter" entries in /etc/module.conf. Then, issue "modprobe -r mpp_Upper" to unload the driver stack and follow by "modprobe mpp_Upper" to re-load the driver stack. See Recommendation. Recommendation: Reboot the system whenever a need to unload the driver stack arises. The use of modprobe with the RDAC driver stack or the use of rmmod to remove all the drivers in the RDAC driver stack, in order, is not recommended/ supported. g. (For LoP servers only) Installation of the Emulex Application Kit, ElxLinuxApps- 2.0a20-7.1.14-1, on RHEL3 servers, automatically installs HBAnyware. HBAnyware interferes with the unloading of the lpfcdfc and lpfc modules. The work around for this is to either stop HBAnyware (/usr/sbin/HBAnyware stop) or uninstall the Application kit. See the Emulex Application Kit Readme for further instructions. h. The file /etc/syslog.conf should have entries for kern.debug, kern.notice, kern.info and kern.warning in order to capture all debug and error messages for MPP driver. 2.4 Disable AVT Open the DS4000 Storage Manager Enterprise Management window, - Highlight the subsystem - Select Tools - Select Execute script. - A script editing window opens. In this window - Select File - Select Load Script - Give the full path name for the script file (eg . /usr/scripts/DisableAVT_Linux.scr) and click OK - Select Tools - Select Verify and Execute Please refer to the IBM System Storage DS4000 Storage Manager version 9 Installation and Support Guide for appropriate OS or the Enterprise Window Online help for more information on how to execute a script. 2.5 Persistent Naming Linux SCSI device names have the possibility of changing when the host system reboots. We recommend using a utility such as devlabel to create user-defined device names that will map devices based on a unique identifier, called a UUID. As of this writing, devlabel is available as part of the Red Hat Enterprise Linux 3 distribution, or online at: http://www.lerhaupt.com/devlabel/devlabel.html. 2.6 RDAC driver unload The unload order for the MPP driver stack is as follows: 1. rmmod mppVhba 2. start "rmmod mppUpper" as a background process Note: If this command in not run before the next command in LoP server, one will gets a response of "ERROR: Module lpfcdd is in use" and the lpfcdd driver can not be removed. 3. rmmod the installed FC HBA driver Depending on the installed FC HBA in the server, this can be qla2xxx followed by qla2300 or mptscsih followed by mptbase or lpfcdd or lpfc. 2.7 RDAC Driver Package Uninstall and Reinstall/Update To un-install the RDAC driver package, type the following and press Enter: make uninstall Note: To Reinstall/Update the RDAC driver package, you need to issue the command "make uninstall" followed by the "make install" command under the /linuxrdac directory in the new RDAC driver source code bundle. 2.8 Kernel Update After applying the kernel update and booting into the new kernel, please do the following things to build MPP Initrd image for the new kernel: a. Change directory to Linuxrdac source code directory. b. do "make uninstall". c. Follow driver building and installation procedure described in Section 3.2.2 and Section 3.2.3. 2.9 Initial Ramdisk image (initrd) Initial RamDisk Image (initrd image) is automatically created when the driver is installed using "make install". Boot loader Configuration file needs to have entry for this newly created image. The initrd image is located in boot partition with file name :'mpp-`uname -r`.img. { Example : mpp-2.4.21-9.ELsmp.img } For a Driver Update, If the system already has an old entry for MPP, the system administrator has to just modify the existing MPP entry accordingly in the boot loader configuration file. In most of the cases, no change is required if the kernel version is same. To create new 'initrd image' user has to run 'mppUpdate' which will overwrite old image file with the new image file. If 3rd party drivers needed to be added to initrd image, edit the /etc/sysconfig/kernel ( SUSE ) or /etc/modules.conf (Redhat) with the 3rd party driver entries. Run 'mppUpdate' to create new ramdisk image. 2.10 Adding and Deleting mapped logical drives (LUNs) Linux RDAC supports re-scanning to recognize a newly mapped LUN without rebooting the server. The utility program is packed with the Linux RDAC driver. It can be invoked by using either "hot_add" or "mppBusRescan" command. "hot_add" is a symbolic link to mppBusRescan. There are man pages for both commands. However, the Linux RDAC driver doesn't support LUN deletion. One has to reboot the server after deleting the mapped logical drives. 2.11 Proc entries To verify that the RDAC driver discovered the available physical LUNs and created virtual LUNs for them, type the following and press Enter: ls -lR /proc/mpp The linux system provides a /proc file system that is a special memory-only, software-created filesystem used by the kernel to export driver information to the user space. The proc pseudo- filesystem is mounted to /proc. The linux SCSI subsystem's proc filesystem directory is /proc/scsi. By convention each HBA driver creates a sub directory under /proc/scsi. The linux RDAC driver's proc entries are: 1. /proc/mpp Entry for the RDAC driver itself 2. /proc/scsi/mpp/ The entry for the MPP virtual host adapter. This can differ from system to system and is assigned by the scsi middle-layer. 3. /proc/mpp/ There is an entry for each array that is visible to the host. 4. /proc/mpp//controllerA /proc/mpp//controllerB These are the entries for the 2 targets on each array seen by the host. Each storage array has 2 controllers. 5. /proc/mpp//controller/ Here the low level driver can be any one of qlogic or emulex HBAs. The HCT# is the Host#, channel#, Target#, where Host# is the Host No of the low-level driver and is assigned by the scsi mid-layer. Channel# depends on whether the HBA card is single or dual channeled. Target# is the Target # assigned to that controller on that array and is assigned by the Low-level HBA driver. 6. /proc/mpp//controller//LUN # Lastly are the Volumes or LUN # of Volumes as mapped on host partition on the storage array and seen through that path or Initiater(Host)-Target combination. 2.12 MAN pages There are 4 MAN pages that can provide more information man RDAC - This man page talks about the RDAC(MPP) driver and the RDAC(MPP) configurable parameters. man mppUtil - This man page tells you how to use the mppUtil - the utility for the RDAC(MPP) driver. man mppBusRescan - This man page describes the hot_add (a.k.a mppBusRescan) script that is available to add newly mapped luns and newly added storage arrays. man mppUpdate - This man page instructs you as to how you can use the persistant device mapping facility available with the RDAC driver and how to change the configuration values for the RDAC driver and build a new RDAC ramdisk. 3.0 Configuration Information ------------------------------- 3.1 Configuration installation N/A 3.2 Configuration Settings N/A 3.3 Hardware Status and Information N/A 4.0 Unattended Mode ------------------------------- N/A 5.0 WEB Sites and Support Phone Number ---------------- 5.1 IBM System Storage™ Disk Storage Systems Technical Support web site: http://www.ibm.com/servers/storage/support/disk/ 5.2 IBM System Storage™ Marketing Web Site: http://www.ibm.com/servers/storage/disk 5.3 If you have any questions about this update, or problem applying the update go to the following HelpCenter World Telephone Numbers URL: http://www.ibm.com/planetwide IMPORTANT: You should download the latest version of the DS4000 Storage Manager host software, the DS4000 storage subsystem controller firmware, the DS4000 drive expansion enclosure ESM firmware and the drive firmware at the time of the initial installation and when product updates become available. To be notified of important product updates, you must first register at the IBM Support and Download Web site: www-1.ibm.com/servers/storage/disk/ 6.0 Trademarks and Notices ---------------------------- The following terms are trademarks of the IBM Corporation in the United States or other countries or both: IBM FAStT System Storage™ the e-business logo xSeries HelpCenter Linux is a registered trademark of Linus Torvalds. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United states, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. STEELEYE are TRADEMARKS and LIFEKEEPER is a registered trademark of STEELEYE TECHNOLOGY, INC. 7.0 Disclaimer -------------- 7.1 THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. IBM DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND MERCHANTABILITY WITH RESPECT TO THE INFORMATION IN THIS DOCUMENT. BY FURNISHING THIS DOCUMENT, IBM GRANTS NO LICENSES TO ANY PATENTS OR COPYRIGHTS. 7.2 Note to U.S. Government Users -- Documentation related to restricted rights -- Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corporation. ries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United states, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. STEELEYE are TRADEMARKS and LIFEKEEPER is a registered trademark of STEELEYE TECHNOLOGY, INC. 7.0 Disclaimer -------------- 7.1 THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. IBM DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND MERCHANTABILITY WITH RESPECT TO THE INFORMATION IN THIS DOCUMENT. BY FURNISHING THIS DOCUMENT, IBM GRANTS NO LICENSES TO ANY PATENTS OR COPYRIGHTS. 7.2 Note to U.S. Government Users -- Documentation related to restricted rights -- Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corporation. with IBM Corporation.