Table of Contents
General description
CHARON-AXP supports emulation of DEC-KGPSA-CA PCI Fibre Channel adapter.
Every instance of KGPSA-CA works in one of the two following modes:
- Fabric virtualization mode (creating virtual fabric in combination with virtual FC-3 Storage Controller). This is default mode.
- Pass Through mode (using a specific CHARON PCI Pass Through driver)
- Fabric presentation mode (using Linux FC HBA directly)
Loading KGPSA storage adapter
Syntax for loading KGPSA-CA storage adapter:
load KGPSA <name> |
Example:
load KGPSA FGA |
In AlphaStation 400 configuration use the following syntax for KGPSA-CA storage adapter loading:
load KGPSA FGA irq_bus=isa |
The adapter instance name ("FGA" in the example above) is used then for parametrization, for example:
set FGA container[100]="/my_disks/vms_distributive.vdisk" |
Numbers in the square brackets represent KGPSA-CA units. They can be in the range 0..32766, but no more than 255 units can be configured on a single controller.
By default KGPSA-CA adapter uses first available PCI slot. If instead some particular slot is needed, refer to this section for details of specific placement of PCI peripherals on CHARON-AXP PCI bus.
Configuration parameters
The KGPSA-CA PCI FC adapter emulation has the following configuration parameters:
host_bus_location
| Parameter | host_bus_location | ||
|---|---|---|---|
| Type | Text String | ||
| Value |
Establish connection between virtual DEC-KGPSA-CA PCI FC adapter and physical EMULEX LightPulse PCI/PCI-X/PCIe FC adapter (KGPSA-CA PCI Fibre Channel adapter#Pass Through mode) Syntax:
Example:
|
wwid
| Parameter | wwid[N] N is 0..32766 (no more than 255 units) | ||
|---|---|---|---|
| Type | Text String | ||
| Value | Sets WWID for emulated KGPSA adapter unit. Syntax:
Example:
|
container
| Parameter | container[N] N is 0..32766 (no more than 255 units) | |||||
|---|---|---|---|---|---|---|
| Type | Text String | |||||
| Value | Possible values of the parameter are strings in one of the following forms:
This parameter is initially not set, thus creating NO storage elements on the controller. |
media_type
| Parameter | media_type[N] N is 0..32766 (no more than 255 units) | |
|---|---|---|
| Type | Text String | |
| Value | Instructs CHARON-AXP to use the supplied value as the PRODUCT field in the FC INQUIRY data returned to a software running on virtual HP Alpha system in response to FC INQUIRY command. If not specified, CHARON-AXP attempts to guess the FC INQUIRY data based on virtual FC device type and underlying container (which is specified in the corresponding container configuration parameter). Initially is not specified. Example:
|
removable
| Parameter | removable[N] N is 0..32766 (no more than 255 units) | |
|---|---|---|
| Type | Boolean | |
| Value | When set to "true", the removable configuration parameter instructs CHARON-AXP to report the corresponding virtual FC device as removable. By default the removable configuration parameter is set to "false". Example:
|
geometry
| Parameter | geometry[N] N is 0..32766 (no more than 255 units) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | Text String | ||||||||||||
| Value | This formatted string value specifies the explicit geometry of the disk storage element. The string format is <X>”/”<Y>[“/”<Z>] or <X>”,”<Y>[“,”<Z>][“,”<B>] where:
If this parameter is not set, CHARON-AXP will configure the geometry based on the most probable disk type. Initially not set. Example:
It is possiblle to specify each parameter independently of another one. The following syntax is used for that:
|
use_io_file_buffering
| Parameter | use_io_file_buffering[N] N is 0..32766 (no more than 255 units) | |
|---|---|---|
| Type | Boolean | |
| Value | When set to "true", instructs CHARON-AXP to enable host operating system I/O cache on read/write operations.
When enabled, host operating system I/O cache may significantly improve I/O performance of the virtual system. At the same time maintaining I/O cache requires additional host resources (CPU and memory) which may negatively affect overall performance of the virtual system. Initially is set to "false". Example:
|
Note that tio_queue_depth
| Parameter | io_queue_depth[N] N is 0..32766 (no more than 255 units) | |
|---|---|---|
| Type | Numeric | |
| Value | Specifies KGPSA I/O requests (read or write) for a given unit in a range 2..128 Setting this parameter enables KGPSA instance to run up-to the specified numbers of I/O requests (read or write) for unit N in parallel, thus improving the performance. The default value set by controller is optimal for most of the cases. It may be needed to enlarge this number if guest OS I/O queue for a certain unit contains too much pending entries. In this case the value should be equal to an average size of the queue, collected statistically. Please do not set this parameter without clear understanding of the purpose. By default parallel execution of I/O requests is disabled. Example:
|
min_n_of_threads
| Parameter | min_n_of_threads | |
|---|---|---|
| Type | Numeric | |
| Value | Instructs KGPSA I/O to reserve a given number of working threads in a range 1..64, thus improving the performance. All units of KGPSA instance share the I/O threads. The default value is equal to number of units plus 2. For optimization it is possible to set this parameter to sum of the "io_queue_depth" parameters for each unit plus 2. This assumption seems optimal for most of the cases. Please do not set this parameter without clear understanding the purpose. Example:
|
Empty disk images are created with the "mkdskcmd" utility.
CHARON-AXP is able to boot from disk images of any OpenVMS/Alpha and Tru64 version.
Mapping to host resources
Fabric virtualization mode
In this mode KGPSA-CA PCI FC adapter can be directly mapped to physical disks (both local and iSCSI) and disk images as shown in the following example:
|
See the "KGPSA-CA PCI Fibre Channel adapter#Configuration parameters" section for details.
Pass Through mode
The CHARON PCI Pass Through mode allows connection between virtual DEC-KGPSA-CA PCI FC adapter and physical EMULEX LightPulse PCI/PCI-X/PCIe FC adapter plugged into host’s PCI/PCI-X/PCIe slot.
Syntax:
|
Example:
load KGPSA FGA host_bus_location="/dev/kgpsa0" |
The following is a list of EMULEX LightPulse PCI/PCI-X/PCIe FC adapters supported by CHARON-AXP PCI Pass Through driver and suitable for emulation of KGPSA-CA PCI FC adapter in CHARON PCI Pass Through mode:
Supported | Not Supported | Not tested |
|---|---|---|
LP8000 | LPe1150 (FC2142SR, A8002A) | LPe11000 |
Pass Through mode establishing sequence
To establish "pass through" mode do the following:
- Install the EMULEX LightPulse PCI/PCI-X/PCIe FC adapter (see above for a list of supported models) to some spare PCI/PCI-X/PCIe slot of the host system
- Build PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
- Install PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
- Add PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup
- Map KGPSA-CA adapter(-s) to EMULEX LightPulse PCI/PCI-X/PCIe FC adapter instance(-s) in CHARON-AXP configuration file
If kernel of the host system has been upgraded or reinstalled all the steps of the PPT KGPSA driver installation must be repeated
Building PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
To build PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC do the following:
| Step | Description | |
|---|---|---|
| 1 | Make sure that the required building tools and include files are installed. If they are absent install them:
The kernel version must match the version of the installed kernel-headers (i.e. this packages must have same versions. It can be verified via Check that the "kernel", "kernel-devel" and the "kernel-headers" have the same version, and ensure that system is booted from this kernel version (not from some older one and etc) with " | |
| 2 | Open xterm and change the default directory to "
| |
| 3 | Issue "make clean; make" commands to build kernel object:
| |
| 4 | Check that there are no compilation errors and the file "ppt_kgpsa.ko" has been built |
Installation of PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC
To install PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC do the following:
| Step | Description | |
|---|---|---|
| 1 | Unload standard "lpfc" driver; to do that issue the following command:
| |
| 2 | Load "
| |
| 3 | Issue "dmesg" command and check that no error appeared during the driver loading, also check that the driver has found all KGPSA devices:
|
Adding PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup
To add PPT driver for EMULEX LightPulse PCI/PCI-X/PCIe FC to Linux startup do the following:
| Step | Description | |
|---|---|---|
| 1 | Disable auto-loading of Linux standard "lpfc" driver on boot. To do that add "blacklist lpfc" to the black list file "/etc/modprobe.d/blacklist.conf" | |
| 2 | Copy the KGPSA-CA kernel module to the location of Linux kernel modules, for example:
| |
| 3 | Enable auto load of the module:
| |
| 4 | Regenerate new "initramfs" image with "mkinitrd":
|
Configuration of KGPSA-CA in pass through mode
FCMGR utility description
To configure KGPSA-CA adapter in pass through mode a special SRM console utility "FCMGR" is used (it has the same functionality as the "WWIDMGR" utility of the native HP Alpha hardware).
It provides the following functionality:
| Command | Description |
|---|---|
| Scans connected SAN using FC adapters, discovers FC ports, storage controllers, logical units and then builds volatile FC database. The "/verbose" qualifier enables FC communication trace on console (for diagnostic and troubleshooting). |
| Displays corresponding part of volatile FC database. |
| Fills the environment variables wwid0..wwid3 and n1..n4 to identify path(s) to logical unit with the specified UDID. These variables are later used by "INIT" to create device database entries and by OpenVMS/Tru64 to get access to boot and dump disks. This command does NOT make any change to other environment variables. |
fc set {boot | dump} wwid <X> | Fills the environment variables wwid0..wwid3 and n1..n4 to identify path(s) to logical unit with the specified WWID. These variables are later used by "INIT" to create device database entries and by OpenVMS/Tru64 to get access to boot and dump disks. This command does NOT make any change to other environment variables. This parameter is useful if UDID is absent. Only right part of the displayed WWID is used for specification, for example:
|
| Clears environment variables wwid0..wwid3 and n1..n4, which automatically disable (but do NOT delete) device database entries representing FC attached devices. This command does NOT affect volatile FC database. |
Example of usage:
|
Configuration steps using FCMGR utility
Once the configuration steps described above are done, start the CHARON VM and wait for the P00>>> prompt.
Please refer to the following example with two FC adapters PGA and PGB defined:
|
The next step is to configure paths for the FC storage:
|
Fabric presentation mode
The CHARON-AXP FC Fabric presentation mode allows to use Linux FC HBA directly. When using this mode, there is no need to load KGPSA adapter(s) as it was described before. The following syntax has to be used instead:
Example:
load kgpsa_generic_storage PGA interface="host3" |
Below please find step-by-step explanation on how to configure "kgpsa_generic_storage" instance.
This mode is available only for CHARON-AXP on RHEL and CentOS 7.x (and later versions) platforms
FC HBA interfaces finding
First we need to find Linux FC HBA for mapping.
List available adapters
# lspci | grep Fibre 03:00.0 Fibre Channel: Emulex Corporation Lancer-X: LightPulse Fibre Channel Host Adapter (rev 30) 03:00.1 Fibre Channel: Emulex Corporation Lancer-X: LightPulse Fibre Channel Host Adapter (rev 30) 0a:04.0 Fibre Channel: Emulex Corporation LP8000 Fibre Channel Host Adapter (rev 02) |
So, 2 HBAs, 3 ports are available.
Load HBA's kernel modules (drivers)
# lspci -vvnn -s 03:00.0 | grep modules Kernel modules: lpfc # modprobe -v lpfc |
Discover the Linux SCSI hosts the FC HBAs are mapped to
# ll /sys/class/fc_host/ .. lrwxrwxrwx 1 root root 0 окт 22 14:29 host3 -> ../../devices/pci0000:00/0000:00:01.0/0000:03:00.0/host3/fc_host/host3 lrwxrwxrwx 1 root root 0 окт 22 14:29 host4 -> ../../devices/pci0000:00/0000:00:01.0/0000:03:00.1/host4/fc_host/host4 lrwxrwxrwx 1 root root 0 окт 22 14:29 host6 -> ../../devices/pci0000:00/0000:00:1e.0/0000:0a:04.0/host6/fc_host/host6 |
So, the available ports are mapped to hosts 3, 4 and 6.
Discover the connected FC HBAs (ports)
# lsscsi .. [2:0:0:0] disk ATA Hitachi HDS72101 A610 /dev/sda [3:0:0:0] storage HP MSA CONTROLLER 7.20 - [3:0:0:2] disk HP MSA VOLUME 7.20 /dev/sdb ... [5:0:0:2] disk Generic- USB3.0 CRW-MS/HG 1.00 /dev/sdi [6:0:0:0] storage HP MSA CONTROLLER 7.20 - |
So the connected hosts are 3 and 6 ("MSA CONTROLLER").
Discover the HBAs port/node names
# cat /sys/class/fc_host/host3/port_name 0x10000090faa00a31 # cat /sys/class/fc_host/host4/port_name 0x10000090faa00a32 # cat /sys/class/fc_host/host6/port_name 0x10000000c923ea51 |
So the names are:
| Host | Mapping | Port name |
|---|---|---|
| 3 | Lancer-X, port 0 ("3:0:0:0") | 0x10000090faa00a31 |
| 4 | Lancer-X, port 1 ("3:0:0:1") | 0x10000090faa00a32 |
| 6 | LP8000 single port | 0x10000000c923ea51 |
Charon Configuration
2 ways of specification in Charon configuration file are available:
By port name
Specify Linux port name "0x10000090faa00a31" in Charon configuration file:
load kgpsa_generic_storage PGA interface="0x10000090faa00a31" |
The configuration given with port name is independent on hardware change, modules load order, etc
By host id name
load kgpsa_generic_storage PGA interface="fc_host3" |
or
load kgpsa_generic_storage PGA interface="host3" |
The configuration given with host id name might change on hardware change, modules load order, etc
Configuration parameters
Never use the following parameters (except the "interface") without well defined reason. Most of the parameters are implemented for solving possible problems and tracing at customer sites, so they might harm if not used appropriately.
interface
Name of FC, SCSI host or adapter's port/node name, defining a connection to Linux FC HBA, for example:
| Mapping | Description |
|---|---|
fc_host6 | Emulation over Linux SG (FC) host 6 |
host6 | Emulation over Linux SG (FC) host 6 |
2000-0000-C923-EA51 | Emulation over Linux SG (FC) host given with port/node name |
0x20000000C923EA51 | Emulation over Linux SG (FC) host given with port/node name |
wwn-0x20000000C923EA51 | Emulation over Linux SG (FC) host given with port/node name |
Example:
load kgpsa_generic_storage PGA interface="host3" |
trace_level
| Value | Description |
|---|---|
0 | none (default) |
1 | All failures and suspects |
| 2 | SCSI, ELS/CT request flow |
3 | MB/IOCB request flow |
Example:
load kgpsa_generic_storage PGA trace_level=1 |
io_model
I/O model is used for request processing.
This parameter is needed for debugging puproses only, so it is strongly recommended not to specify any value without particular instructions from Stromasys.
| Value | Description |
|---|---|
sync | synchronous |
async | asynchronous (default) |
Example:
load kgpsa_generic_storage PGA io_model="sync" |
port_name
The port name in format: %04x-%04x-%04x-%04x. Option to fake guest's port name.
This parameter is needed for debugging puproses only, so it is strongly recommended not to specify any value without particular instructions from Stromasys.
Example:
load kgpsa_generic_storage PGA port_name=2000-0000-C923-EA51 |
By default this value is obtained from the specified hardware port (see above).
node_name
The node name in format: %04x-%04x-%04x-%04x. Option to fake guest's node name.
This parameter is needed for debugging puproses only, so it is strongly recommended not to specify any value without particular instructions from Stromasys.
Example:
load kgpsa_generic_storage PGA node_name=2000-0000-C923-EA51 |
By default this value is obtained from the specified hardware port (see above).
link_up_delay
The delay in seconds, port suspend reporting the link up waiting for Linux FC infrastructure creation. Default is 120 seconds.
When FC wire is removed and then plugged in, Linux rebuilds all the FC objects with new devices names, etc. Fabric presentation mode senses the link up and down events, but optionally it may delay reporting the event (for the given amount of seconds) to let Linux fully initialize FC. If delay is 0, guest OS may start initialization too early and thus fail to find some or all objects.
Example:
load kgpsa_generic_storage PGA link_up_delay=80 |
link_poll_period
The link check poill interval in seconds Default is 1 second.
When FC wire is removed and then plugged in, Linux rebuilds all the FC objects with new devices names, etc. Fabric presentation mode senses the link up and down events, but optionally it may delay reporting the event (for the given amount of seconds) to let Linux fully initialize FC. If delay is 0, guest OS may start initialization too early and thus fail to find some or all objects.
Example:
load kgpsa_generic_storage PGA link_poll_period=2 |
emu_fabric
The way the emulation deals with fabric:
| Value | Description |
|---|---|
auto | Delegates to Linux if supported, otherwise emulates |
no | Delegates to Linux (real physical fabric port connected too) |
| yes | Emulates |
Qlogic does not map SG devices for ELS/CT traffic. To work around this, Fabric presentation mode implements emulation of the whole non FCP (scsi) communication over fabric. If fabric is emulated, guest OS will not see non FCP target ports (for example initiators) and will not be able to communicate with ELS/CT over fabric.
Example:
load kgpsa_generic_storage PGA emu_fabric="no" |
lun_exclude
The list of the LUNs to be excluded. If empty (default), nothing is excluded.
No multi line is possible for this parameter; the maximum number of symbols inside the double quotes is 255. Use "lun_include" parameter instead if a lot of LUNs must be excluded to specify only the included LUNs.
Example:
load kgpsa_generic_storage PGA lun_exclude="6008-05F3-0005-2950-C758-BCC2-E88B-0007,6008-05F3-0005-2950-8B03-E26B-E231-0005" |
lun_include
The list of the LUNs to be included. If empty (default), all LUNs are included. No multi line is possible for this parameter; the maximum number of symbols inside the double quotes is 255. Use "lun_exclude" parameter instead if a lot of LUNs must be included to specify only the excluded LUNs.
Example:
load kgpsa_generic_storage PGA lun_include="6008-05F3-0005-2950-C758-BCC2-E88B-0007,6008-05F3-0005-2950-8B03-E26B-E231-0005" |
scsi_version
This approach of FC emulation downgrades all attached devices to the specifically defined version of SCSI. This is done to make legacy guest operating systems happy with attached modern devices.
| Value (ANSI SCSI version) | Description |
|---|---|
any | Any version |
v1 | SCSI-1 |
| v2 | SCSI-2 |
| v3 | SCSI-3 |
| v3_SPC1 | SCSI-3, primary command level 1 |
| v3_SPC2 | SCSI-3, primary command level 2 (default) |
| v3_SPC3 | SCSI-3, primary command level 3 |
| v3_SPC4 | SCSI-3, primary command level 4 |
Refer to your guest OS documentation for more details and public resources for general specification of SCSI.
Example:
load kgpsa_generic_storage PGA scsi_version="v3_SPC3" |
device_id
A fake device id (if not "lp8000" is needed), the default is "0xF800" ("lp8000").
This parameter is needed for some specific cases only, so it is strongly recommended not to specify any value without particular instructions from Stromasys.
Example:
load kgpsa_generic_storage PGA device_id=0xFA00 |
vendor_id
A fake vendor id (if not "lp8000" is needed), the default is "0x10DF" ("lp8000").
This parameter is needed for some specific cases only, so it is strongly recommended not to specify any value without particular instructions from Stromasys.
Example:
load kgpsa_generic_storage PGA vendor_id=0x10EA |
revision_id
A fake revision id (if not "lp8000" is needed), the default is "0x02" ("lp8000").
This parameter is needed for some specific cases only, so it is strongly recommended not to specify any value without particular instructions from Stromasys.
Example:
load kgpsa_generic_storage PGA revision_id=0x01 |