Description

AlphaServer supports up to a 3 nodes shared SCSI cluster for OpenVMS cluster or 4 nodes for TruCluster. That is, up to 3 or 4 AlphaServers can be daisy-chained to a set of shared SCSI devices. With Charon-AXP, we do this in a virtual way. The underlining hardware is not limited to KZPBA and HSZ controllers. A virtualized shared SCSI can run on any shared storage, include iSCSI and Fibre Channel. Here we describe how to set up a shared SCSI environment.

Step-by-step guide

Requirement

The shared SCSI can be built on a single PC server multiple Charon instances environment, or a multiple PC server environment.

The requirement of configuring a shared SCSI on a single PC server multiple Charon instances for shared SCSI is basically the same as non-shared SCSI. A PC server with sufficient cores, memory, network ports and disk space to run all the instances is good enough.

The requirement of configuring a shared SCSI on multiple PC servers is a little different. We still need sufficient cores, memory, network ports and disk space. In addition to these, we need to have:

A true shared storage, for example iSCSI or Fibre Channel. Local attached disks and vdisks cannot be shared
An additional network port for the virtual SCSI traffic
That additional network ports of all the systems are connected to form a small network with their own IP like 192.168.13.3, 192.168.13.4 and 192.168.13.5

Configuration in Charon is done in two parts - the virtual SCSI controller; and the SCSI disks. The sharing is done in the SCSI controller setting.

Configuring the shared SCSI controller

Virtual shared SCSI buses are talked through network using TCP/IP. In Charon configuration, we have to specify the IP address and port (ie. an IP socket) for each virtual SCSI controller, so that other virtual SCSI controllers on the same bus can talk to it.

The shared SCSI is defined like this:

set PKA port[600]=17016 host[600]="192.168.13.3:16017"

Meaning:

PKA <== The SCSI bus is PKA

port[600] <== The device using SCSI id 6 is another SCSI controller

=17016 <== The local port of PKA on this server that talk to SCSI id 6, 17=SCSI id 7, 0=PKA, 16=SCSI id 6 of the remote system

host[600] <== This is to configure the IP socket of remote system which use SCSI id 6

=”192.168.13.3 <== The remote system is at 192.168.13.3. If it is a single PC server multiple instances configuration, we can use “localhost”

:16017” <== The port that the remote system use is 16017, it will be defined in the remote system with “set port[700]=16017” which follows the same way above

Here we have an example of a 3-node cluster, A, B and C. Each node got 3 shared SCSI controller PKA, PKB and PKC, and are running on 3 different PC servers. Below shows how they are configured according the above rule. In OpenVMS cluster, the SCSI traffic will use LUN 0. In TruCluster, the SCSI traffic will use LUN 7.

Node A – IP address is 192.168.13.2

Node B – IP address is 192.168.13.3

Node C – IP address is 192.168.13.4

Node A, SCSI controller configuration

SCSI ids of the emulated SCSI controller are all 7

load KZPBA PKA scsi_id=7

set PKA port[600]=17016 host[600]="192.168.13.3:16017" <== point to node B

set PKA port[500]=17015 host[500]="192.168.13.4:15017" <== point to node C
(use [607] and [507] if this is TruCluster)

load KZPBA PKB scsi_id=7

set PKB port[600]=17116 host[600]="192.168.13.3:16117" <== point to node B

set PKB port[500]=17115 host[500]="192.168.13.4:15117" <== point to node C
(use [607] and [507] if this is TruCluster)

load KZPBA PKC scsi_id=7

set PKC port[600]=17216 host[600]="192.168.13.3:16217" <== point to node B

set PKC port[500]=17215 host[500]="192.168.13.4:15217" <== point to node C
(use [607] and [507] if this is TruCluster)

Node B, SCSI controller configuration

SCSI ids of the emulated SCSI controller are all 6

load KZPBA PKA scsi_id=6

set PKA port[700]=16017 host[700]="192.168.13.2:17016" <== point to node A

set PKA port[500]=16015 host[500]="192.168.13.4:15016" <== point to node B
(use [707] and [507] if this is TruCluster)

load KZPBA PKB scsi_id=6

set PKB port[700]=16117 host[700]="192.168.13.2:17116" <== point to node A

set PKB port[500]=16115 host[500]="192.168.13.4:15116" <== point to node B
(use [707] and [507] if this is TruCluster)

load KZPBA PKC scsi_id=6

set PKC port[700]=16217 host[700]="192.168.13.2:17216" <== point to node A

set PKC port[500]=16215 host[500]="192.168.13.4:15216" <== point to node B
(use [707] and [507] if this is TruCluster)

Node C, SCSI controller configuration

SCSI ids of the emulated SCSI controller are all 5

load KZPBA PKA scsi_id=5

set PKA port[700]=15017 host[700]="192.168.13.2:17015" <== point to node A

set PKA port[600]=15016 host[600]="192.168.13.3:16015" <== point to node C
(use [707] and [607] if this is TruCluster)

load KZPBA PKB scsi_id=5

set PKB port[700]=15117 host[700]="192.168.13.2:17115" <== point to node A

set PKB port[600]=15116 host[600]="192.168.13.3:16115" <== point to node C
(use [707] and [607] if this is TruCluster)

load KZPBA PKC scsi_id=5

set PKC port[700]=15217 host[700]="192.168.13.2:17215" <== point to node A

set PKC port[600]=15216 host[600]="192.168.13.3:16215" <== point to node C
(use [707] and [607] if this is TruCluster)

A picture will make this easy to understand.

Configuring the storage

The configuration of multiple PC servers running Charon that use shared SCSI is a normal configuration. It preserves high availability like OpenVMS cluster / TruCluster running on the original AlphaServer, and at the same time enjoy the flexibility of Charon. Therefore, we CANNOT use Charon vdisk or locally attached physical drives as the shared SCSI device in this configuration. We can only use physical disk from either iSCSI or Fibre Channel.

The key point to configure shared SCSI disks in this configuration is to make sure the same DK device on different nodes point to the same LUN. Note that when the disks are presented to PC servers, the PhysicalDrive number of the same LUN may not be the same among different PC servers. It must be careful on that. In above, DKA0 is LUN 7, DKA100 is LUN 8. This is the same across all three hosts even the PhysicalDrive number in PC server B is different from the other two.

To make this easy to manage, it is highly recommended to persistent device naming. Please reference this from Charon-AXP user manual.

Node A, on PC server A –

set PKA container[0]=”\\.\PhysicalDrive(DevID=6008-05F3-0005-2950-BF8E-0B86-A0C7-0001)” <= LUN 7
set PKA container[100]=”\\.\PhysicalDrive(iScsiTarget=iqn.2008-04:iscsi.charon-target-test1,LUN= 8)” <== LUN 8

Node B, on PC server B –

set PKA container[0]=”\\.\PhysicalDrive(DevID=6008-05F3-0005-2950-BF8E-0B86-A0C7-0001)” <= LUN 7
set PKA container[100]=”\\.\PhysicalDrive(iScsiTarget=iqn.2008-04:iscsi.charon-target-test1,LUN= 8)” <== LUN 8

Node C, on PC server C –

set PKA container[0]=”\\.\PhysicalDrive(DevID=6008-05F3-0005-2950-BF8E-0B86-A0C7-0001)” <= LUN 7
set PKA container[100]=”\\.\PhysicalDrive(iScsiTarget=iqn.2008-04:iscsi.charon-target-test1,LUN= 8)” <== LUN 8

Special note

After the configuration, the following (or similar) will be seen in the SRM console:

?ka600.6.0.1.1          ?KA600
?kb600.6.0.3.1          ?KB600
?kc600.6.0.4.1          ?KC600

The ?ka, ?kb and ?kc refer to cluster interconnect. This is to distinguish from other SCSI devices. In addition, if either PC server in the cluster configuration is not booted, some errors will be seen in the Charon log file like:

20201218:234941:ERROR:2:00000411:scsi_host.(2401):  (107) Transport endpoint is not connected:  PKA_6_0: Failed to connect to localhost:10167, port 10167

20201218:234955:INFO :0:00000413:qlogic_isp(4554):  PKA: Logical Unit 0 has been enabled for target mode operation with credit of up to 16 commands

These messages are normal and expected.

Product Documentation and Knowledge Base

Building shared cluster with Charon-AXP