KGPSA-CA PCI Fibre Channel adapter
Table of Contents
- 1 General description
- 2 Loading KGPSA storage adapter
- 3 Configuration parameters
- 3.1 host_bus_location
- 3.2 wwid
- 3.3 container
- 3.4 media_type
- 3.5 removable
- 3.6 geometry
- 3.7 use_io_file_buffering
- 3.8 io_queue_depth
- 3.9 min_n_of_threads
- 4 Mapping to host resources
- 4.1 Fabric virtualization mode
- 4.2 Pass Through mode
- 4.2.1 Pass Through mode establishing sequence
- 4.2.2 Building PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
- 4.2.3 Installation of PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
- 4.2.4 Adding PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup
- 4.2.5 Configuration of KGPSA-CA in pass through mode
- 4.2.5.1 FCMGR utility description
- 4.2.5.2 Configuration steps using FCMGR utility
- 4.3 Fabric presentation mode
- 4.3.1 FC HBA interfaces finding
- 4.3.2 Charon Configuration
- 4.3.2.1 By port name
- 4.3.2.2 By host id name
- 4.3.3 Configuration parameters
- 4.3.3.1 interface
- 4.3.3.2 trace_level
- 4.3.3.3 io_model
- 4.3.3.4 port_name
- 4.3.3.5 node_name
- 4.3.3.6 link_up_delay
- 4.3.3.7 link_poll_period
- 4.3.3.8 emu_fabric
- 4.3.3.9 lun_exclude
- 4.3.3.10 lun_include
- 4.3.3.11 scsi_version
- 4.3.3.12 device_id
- 4.3.3.13 vendor_id
- 4.3.3.14 revision_id
- 4.3.3.15 scsi_serialize_io
- 4.3.3.16 scsi_max_queue_depth
General description
CHARON-AXP supports emulation of DEC-KGPSA-CA PCI Fibre Channel adapter.
Every instance of KGPSA-CA works in one of the two following modes:
Fabric virtualization mode (creating virtual fabric in combination with virtual FC-3 Storage Controller). This is default mode.
Pass Through mode (using a specific CHARON PCI Pass Through driver)
Fabric presentation mode (using Linux FC HBA directly)
Loading KGPSA storage adapter
Syntax for loading KGPSA-CA storage adapter:
|
Example:
|
Please note:
In AlphaStation 400 configuration use the following syntax for KGPSA-CA storage adapter loading:
load KGPSA FGA irq_bus=isa
The adapter instance name ("FGA" in the example above) is used then for parametrization, for example:
|
Numbers in the square brackets represent KGPSA-CA units. They can be in the range 0..32766, but no more than 255 units can be configured on a single controller.
By default KGPSA-CA adapter uses first available PCI slot. If instead some particular slot is needed, refer to this section for details of specific placement of PCI peripherals on CHARON-AXP PCI bus.
Configuration parameters
The KGPSA-CA PCI FC adapter emulation has the following configuration parameters:
host_bus_location
Parameter | host_bus_location |
|---|---|
Type | Text String |
Value | KGPSA-CA PCI Fibre Channel adapter in Pass Through mode only ! Establish connection between virtual DEC-KGPSA-CA PCI FC adapter and physical EMULEX LightPulse PCI/PCI-X/PCIe FC adapter (KGPSA-CA PCI Fibre Channel adapter in Pass Through mode) Syntax:
Example:
|
wwid
Parameter | wwid[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Text String |
Value | Sets WWID for emulated KGPSA adapter unit. Syntax:
Example:
|
container
Parameter | container[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Text String |
Possible values of the parameter are strings in one of the following forms:
1. Physical disk
"/dev/sd<L>", where "L" is letter, for example "/dev/sdb"
"/dev/disk/by-id/..." - addressing by the disk ID, for example "/dev/disk/by-id/ata-ST1000DM003-9YN162_S1D01QJ4"
"/dev/disk/by-label/..." - addressing by the disk label, for example "/dev/disk/by-label/MyStorage"
"/dev/disk/by-uuid/..." - addressing by the disk UUID, for example "/dev/disk/by-uuid/0e808a2f-cdd3-4944-a245-f729ffd73882"
Please note that existing data on such a disk may be destroyed, depending on how it is used in the emulator.
"/dev/sd<L>" addressing is not persistent, so it is strongly recommended to use "/dev/disk/by-id/wwn-*" syntax instead to refer the disk by its WWID - especially in the enviroments utilizing FC and SAN storages.
These disks must not be formatted by the host OS.
Example: set FGA container[0]="/dev/disk/by-id/ata-ST1000DM003-9YN162_S1D01QJ4"
It is also possible to use not a whole disk, but previously created partitions on it. In this case the syntax is the following: "/dev/sd<L><N>" where N is the number of partition to be used.
Example: set FGA container[0]="/dev/sdc3"
2. Loop (virtual block) devices
"/dev/loop<N>"
Example: set FGA container[100]="/dev/loop0"
3. Multipath disk
"/dev/dm-<N>"
"/dev/mapper/mpath<N>"
"/dev/mapper/disk<N>"
Be careful not to destroy all the information from the disk dedicated to CHARON-AXP by mistake. These disks must not be formatted by the host OS.
Example: set FGA container[200]="/dev/dm-0"
4. File representing a physical disk of the Alpha system (disk image)
[<drive>":\"<path-name>”\”]<file-name>[“.vdisk”]
These files can be created from scratch with "mkdskcmd" utility. Data and OS disk backups are transferred from the original system via tapes or network and restored into these container files.
Mapping may also include the full path (recommended), for example: "/my_disks/my_boot_disk.vdisk"
Example: set FGA container[401]="my_dka401.vdisk"
This parameter is initially not set, thus creating NO storage elements on the controller.
media_type
Parameter | media_type[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Text String |
Value | Instructs CHARON-AXP to use the supplied value as the PRODUCT field in the FC INQUIRY data returned to a software running on virtual Alpha system in response to FC INQUIRY command. If not specified, CHARON-AXP attempts to guess the FC INQUIRY data based on virtual FC device type and underlying container (which is specified in the corresponding container configuration parameter). Initially is not specified. Example:
|
removable
Parameter | removable[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Boolean |
Value | When set to "true", the removable configuration parameter instructs CHARON-AXP to report the corresponding virtual FC device as removable. By default the removable configuration parameter is set to "false". Example:
|
geometry
Parameter | geometry[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Text String |
Value | This formatted string value specifies the explicit geometry of the disk storage element. The string format is <X>”/”<Y>[“/”<Z>] or <X>”,”<Y>[“,”<Z>][“,”<B>] where: X corresponds to the number of sectors per track Y corresponds to the number of tracks per cylinder Z corresponds the number of cylinders on the unit. If omitted, Z is calculated based on X, Y and the total number of sectors on the unit that reflects the size of the disk storage element. This is an optional parameter. B corresponds to the total size of the disk (in blocks) reported to the guest OS. If omitted it is calculated automatically. This is an optional parameter. If this parameter is not set, CHARON-AXP will configure the geometry based on the most probable disk type. Initially not set. Example:
It is possible to specify each parameter independently of another one. The following syntax is used for that: |
use_io_file_buffering
Parameter | use_io_file_buffering[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Boolean |
Value | When set to "true", instructs CHARON-AXP to enable host operating system I/O cache on read/write operations. Note that this caching has a significant effect only in case of mapping to disk containers, not physical drives. When enabled, host operating system I/O cache may significantly improve I/O performance of the virtual system. At the same time maintaining I/O cache requires additional host resources (CPU and memory) which may negatively affect overall performance of the virtual system. Initially is set to "false". Example:
|
io_queue_depth
Parameter | io_queue_depth[N] N is 0..32766 (no more than 255 units) |
|---|---|
Type | Numeric |
Value | Specifies KGPSA I/O requests (read or write) for a given unit in a range 2..128 Setting this parameter enables KGPSA instance to run up-to the specified numbers of I/O requests (read or write) for unit N in parallel, thus improving the performance. The default value set by controller is optimal for most of the cases. It may be needed to enlarge this number if guest OS I/O queue for a certain unit contains too much pending entries. In this case the value should be equal to an average size of the queue, collected statistically. Please do not set this parameter without clear understanding of the purpose. By default parallel execution of I/O requests is disabled. Example:
|
min_n_of_threads
Parameter | min_n_of_threads |
|---|---|
Type | Numeric |
Value | Instructs KGPSA I/O to reserve a given number of working threads in a range 1..64, thus improving the performance. All units of KGPSA instance share the I/O threads. The default value is equal to number of units plus 2. For optimization it is possible to set this parameter to sum of the "io_queue_depth" parameters for each unit plus 2. This assumption seems optimal for most of the cases. Please do not set this parameter without clear understanding the purpose. Example:
|
When a disk image connected to an emulated KGPSA-CA controller is dismounted by OpenVMS, it is disconnected from CHARON-AXP and can be manipulated. It can be replaced with a different disk image if it keeps the same name. This capability may be useful when designing back-up and restore procedures. When copying CHARON-AXP disk images while CHARON-AXP is running, please take care to minimize the risk of overloading a heavily loaded CHARON-AXP host system. For example, using a sequential series of simple ftp binary copies is less resource intensive and thus less disruptive than multiple, simultaneous copies.
Empty disk images are created with the "mkdskcmd" utility.
CHARON-AXP is able to boot from disk images of any OpenVMS/Alpha and Tru64 version.
Mapping to host resources
Fabric virtualization mode
In this mode KGPSA-CA PCI FC adapter can be directly mapped to physical disks (both local and iSCSI) and disk images as shown in the following example:
|
See the KGPSA-CA PCI Fibre Channel adapter - Configuration parameters section for details.
Pass Through mode
The CHARON PCI Pass Through mode allows connection between virtual DEC-KGPSA-CA PCI FC adapter and physical EMULEX LightPulse PCI/PCI-X/PCIe FC adapter plugged into host’s PCI/PCI-X/PCIe slot.
Syntax:
|
Example:
|
The following is a list of EMULEX LightPulse PCI/PCI-X/PCIe FC adapters supported by CHARON-AXP PCI Pass Through driver and suitable for emulation of KGPSA-CA PCI FC adapter in CHARON PCI Pass Through mode:
Supported | Not Supported | Not tested |
|---|---|---|
LP8000 | LPe1150 (FC2142SR, A8002A) | LPe11000 |
Pass Through mode establishing sequence
To establish "pass through" mode do the following:
Install the EMULEX LightPulse PCI/PCI-X/PCIe FC adapter (see above for a list of supported models) to some spare PCI/PCI-X/PCIe slot of the host system
Build PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
Install PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
Add PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup
Map KGPSA-CA adapter(-s) to EMULEX LightPulse PCI/PCI-X/PCIe FC adapter instance(-s) in CHARON-AXP configuration file
If kernel of the host system has been upgraded or reinstalled all the steps of the PPT KGPSA driver installation must be repeated
Building PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
To build PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC do the following:
Step | Description |
|---|---|
1 | Make sure that the required building tools and include files are installed. If they are absent install them:
The kernel version must match the version of the installed kernel-headers (i.e. this packages must have same versions. It can be verified via Check that the "kernel", the "kernel-devel" and the "kernel-headers" have the same version, and ensure that system is booted from this kernel version (not from some older one and etc) with " |
2 | Open xterm and change the default directory to "
|
3 | Issue "make clean; make" commands to build kernel object:
It is prohibited to use a module built on a certain version of kernel on another one. |
4 | Check that there are no compilation errors and the file " |
Installation of PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
To install PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC do the following:
Step | Description |
|---|---|
1 | Unload standard "lpfc" driver; to do that issue the following command:
|
2 | Load "
|
3 | Issue "dmesg" command and check that no error appeared during the driver loading, also check that the driver has found all KGPSA devices.
|
Adding PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup
To add PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup do the following:
Step | Description |
|---|---|
1 | Disable auto-loading of Linux standard "lpfc" driver on boot. To do that add "blacklist lpfc" to the black list file " |
2 | Copy the KGPSA-CA kernel module to the location of Linux kernel modules, for example: The particular path may be different, depending on the kernel version and Linux distribution. |
3 | Enable auto load of the module:
|
4 | Regenerate new "initramfs" image with "mkinitrd":
The particular path may be different, depending on the kernel version and Linux distribution. |
Configuration of KGPSA-CA in pass through mode
FCMGR utility description
To configure KGPSA-CA adapter in pass through mode a special SRM console utility "FCMGR" is used (it has the same functionality as the "WWIDMGR" utility of the native Alpha hardware).
It provides the following functionality:
Command | Description |
|---|---|
| Scans connected SAN using FC adapters, discovers FC ports, storage controllers, logical units and then builds volatile FC database. The "/verbose" qualifier enables FC communication trace on console (for diagnostic and troubleshooting). |
| Displays corresponding part of volatile FC database. |
| Fills the environment variables wwid0..wwid3 and n1..n4 to identify path(s) to logical unit with the specified UDID. These variables are later used by "INIT" to create device database entries and by OpenVMS/Tru64 to get access to boot and dump disks. This command does NOT make any change to other environment variables. |
| Fills the environment variables wwid0..wwid3 and n1..n4 to identify path(s) to logical unit with the specified WWID. These variables are later used by "INIT" to create device database entries and by OpenVMS/Tru64 to get access to boot and dump disks. This command does NOT make any change to other environment variables. This parameter is useful if UDID is absent. Please note: Only right part of the displayed WWID is used for specification, for example:
|
| Clears environment variables wwid0..wwid3 and n1..n4, which automatically disable (but do NOT delete) device database entries representing FC attached devices. This command does NOT affect volatile FC database. |
Example of usage:
|
Configuration steps using FCMGR utility
Once the configuration steps described above are done, start the CHARON VM and wait for the P00>>> prompt.
Please refer to the following example with two FC adapters PGA and PGB defined:
|
Is this only one line for all WWIDs, of so what is the limit ? (line length limit)
or is it possible to add one WWID per line ?
(same for lun_include)