General Information
- Linux Versions and NetworkManager
- Interface MTU Considerations
Host to Guest Communication Considerations
External Communication Considerations
Guest to Guest Layer 2 Communication Considerations
Asymmetric Routing Considerations
Cloud Instance and IP Forwarding
Interface Configuration Basics
- Basic File-based Interface Configuration without NetworkManager
- Basic Interface Configuration with NetworkManager
Further Information

General Information

This section provides some basic information about networking questions that are likely to affect Charon-SSP when running in the cloud.

Linux Versions and NetworkManager

There are significant differences regarding the NetworkManager in the different Linux versions (RHEL 7, 8, 9 and derivatives). This section provides an overview of some important differences.

There are two basic network configuration systems in the relevant Linux systems:

The network service with the network configuration based on ifcfg-files in /etc/sysconfig/network-scripts. This requires the network-scripts package.
The NetworkManager with its own configuration file syntax. Persistent configuration files are stored in /etc/NetworkManager/system-connections. The NetworkManager has a plugin (ifcfg-rh) to handle ifcfg-files. This plugin does not support all configuration options of the network-scripts system (e.g., tunnel and tap interfaces are not supported).

Linux 7.x:

The network-scripts and NetworkManager methods coexist. It is possible to disable the NetworkManager completely or only for certain interfaces (parameter NM_CONTROLLED=no in the ifcfg-file).
The default NetworkManager plugin is the ifcfg-rh plugin.
Interfaces managed by the Charon-SSP Manager must have an ifcfg-file and be removed from NetworkManager control (unmanaged interfaces).

Linux 8.x:

The network-scripts package is deprecated. It is not installed by default, but available in the Linux package repositories.
The default NetworkManager plugin configuration is ifcfg-rh, keyfile. The keyfile plugin is responsible for handling the native NetworkManager configuration file syntax.
If virtual bridge configurations including TAP interfaces are configured using ifcfg-files, the network-scripts package is required. Otherwise, the TAP interfaces cannot be activated (missing support in the ifcfg-rh plugin). Alternatively, such interfaces can be configured as native NetworkManager connections.
There is an ifup command which by default points to nm-ifup. Once the network-scripts package is installed, it points to the ifup command contained in this package.
The loopback interface (lo) cannot be managed by the NetworkManager.
Interfaces managed by the Charon-SSP Manager must be under NetworkManager control (managed interfaces).

Linux 9.x:

The network-scripts package is no longer available in the Linux package repositories.
The default Networkmanager plugin configuration is keyfile, ifcfg-rh.
Existing ifcfg-files can still be read and written, but only if supported by the ifcfg-rh plugin.
A new nmcli command option (nmcli connection migrate <con-name>) helps with the conversion of ifcfg-configuration files to native NetworkManager connection profiles. However, this command only works for connections supported by the ifcfg-rh plugin. This means, for example, that TAP interfaces that were previously configured via ifcfg-files must now be recreated using nmcli commands or another NetworkManager configuration tool. Before using the migration command, take a backup copy of the content of /etc/sysconfig/network-scripts.
By default, there is no ifup command. If it is needed, the NetworkManager variant of the command can be installed (NetworkManager-initscripts-updown).
The loopback interface (lo) cannot be managed by the NetworkManager in versions before 9.2.
Interfaces managed by the Charon-SSP Manager must be under NetworkManager control (managed interfaces).

Additional information about the ifcfg-rh plugin:

The ifcfg-rh plugin is used by the NetworkManager to read/write the traditional ifcfg-files in /etc/sysconfig/network-scripts. Each NetworkManager connection corresponds to one ifcfg-file. The plugin does not support all the connection types supported by the original network-scripts package. The plugin currently supports Ethernet, Wi-Fi, InfiniBand, VLAN, Bond, Bridge, and Team connections. This means that, for example, TYPE=Tap is not supported and cannot be handled by the NetworkManager in the ifcfg-file format. In Linux 7.x and Linux 8.x, the network-scripts package can be used to support the ifcfg-file format. In Linux 9.x, this package is no longer available. Thus, unsupported connection types must be manually recreated.

Interface MTU Considerations

When configuring a dedicated network interface for an emulator, ensure that the MTU of the Charon host interface used is not smaller than the MTU used by the legacy guest operating system. Failing to do so will cause network problems. For further information, please refer to the chapter Interface MTU Considerations in this guide.

Host to Guest Communication Considerations

There are several ways a communication between the host operating system and the guest Solaris system can be implemented. For example:

Internal virtual bridge on the host system:
Such a bridge has several TAP interfaces. The host and the guest systems are connected to this bridge and can communicate directly to one another using L3 and L2 protocols. The bridge uses its own IP subnet that can be defined by the user. For Charon-SSP, setting up such a configuration is supported by the Charon-SSP Manager (leave the default gateway field empty for the bridge interface). Several hosts configured with guest systems and such an internal bridge can communicate across the cloud-internal LAN and the host systems can route the private IP subnets of the bridges between themselves. L2 protocols are not possible if routing across the cloud LAN is used.
Communication via the cloud-internal subnet LAN:
In this case, a second interface is added to the Charon host system. The second interface is then assigned to the emulated guest system. After configuring the interface correctly, the host and guest can communicate across the cloud-internal LAN using IP. L2 protocols or any protocols that require changing the MAC address to something different than the MAC address assigned to the second interface by the cloud provider will not work.
To connect the guest system to the LAN, the following basic configuration steps must be performed:
- Add the additional interface to the Charon host system.
- Create a configuration for the additional interface. This can be done via configuration files on Linux 7.x. For Linux 8.x nmcli commands, the nmtui utility, or (for SSP) the Charon Manager can be used.
- Make a note of the private IP address assigned to the second interface by the cloud provider, and remove it from the Linux configuration (if it has been configured).
- Assign the interface to the emulated system. This can be done by modifying the emulator configuration file or by using the Charon-SSP Manager.
- Set the MAC address of the emulated system to the same value as the one used on the host system Ethernet interface. For Charon-SSP, this configuration can be implemented using the Charon Manager.
- On the guest operating system, configure the private IP address that was previously assigned to the second interface on Linux and configure the appropriate default route for the LAN.

Please note:

The section Dedicated NIC for Guest System provides some hints on how to configure the second interface in the different situations. Please refer to your cloud-provider's documentation for up-to-date comprehensive information.
If Layer 2 communication between guests on different Charon hosts is required, a bridged tunnel solution must be set up between the two Charon host systems.

External Communication Considerations

In addition to allowing SSH access to the host system for management purposes, it may be necessary to enable Internet communication to the host and guest system or connect host and guest to the customer's network.

Please note: Charon hosts based on Charon-SSP AL (Automatic Licensing) marketplace images and using the public license servers always need either direct Internet access or Internet access via NAT from a NAT gateway in the same cloud as the Charon host to access the license server.

Recommended way to connect the Charon host and Solaris guest systems to the customer network:

To ensure data traffic between the Charon host and guest systems and the customer network is encrypted, it is strongly recommended to use a VPN connection. An example of a simple VPN connection based on an SSH tunnel is described in SSH VPN - Connecting Charon Host and Guest to Customer Network. This connection is based on a bridge between Charon host and guest system and (via an encrypted SSH tunnel) the remote end-point in the customer network. The connection supports L3 and L2 protocols.

Cloud providers usually also provides a VPN gateway instance that can be added to the customer cloud network to connect the cloud network to the customer network (for a charge).

Recommended way to connect the guest system to the Internet:

The Internet connection can be implemented across the VPN to the customer network. In this case, the customer can allow the guest Solaris system to access the Internet exactly following the security policies defined by the customer.

Access to the Internet from subnets or guest systems with only private IP addresses:

Access to the Internet for subnets with only private IP addresses is possible across a gateway instance providing VPN access to the customer network and allowing (NATted) Internet access via this path. Alternatively, a NAT gateway in the cloud can be used to map the private addresses to public addresses. The NAT gateway can be implemented on a Charon host system, a dedicated customer-operated gateway, or it can often be provided by the cloud provider for a charge.

Please note: a Charon-SSP AL host system that use the public license servers always needs either direct Internet access or Internet access via NAT from a NAT gateway in the same cloud as the Charon host to access the public license server.

Direct guest system access to the Internet:

This not a recommended standard solution for security reasons. However, should it be required, two interfaces with public IP addresses can be assigned to the Charon host.
One of these interfaces is then dedicated to the guest system which uses the private interface address and the MAC address assigned to the Charon host by the cloud provider (see also Dedicated NIC for Guest System).

Guest to Guest Layer 2 Communication Considerations

Should L2 protocols be required between two guest systems on different host systems, a bridge/tunnel solution similar to the one described in SSH VPN - Connecting Charon Host and Guest to Customer Network must be set up between the two host systems to allow the L2 traffic to pass.

Asymmetric Routing Considerations

This section applies to the case where several interfaces are configured on an instance and they all have IP addresses configured on the Linux level.

When you add a secondary NIC to a Linux instance, a new interface (that is, an Ethernet device) is added to the instance and automatically recognized by the OS. Depending on the cloud-provider, DHCP may not be active for the secondary VNIC, and you must configure the interface with a static IP address and add any routes that are relevant for the new interface.

Connectivity problems caused by asymmetric routing arise if traffic arrives through one interface and, when the service replies, the reply packets (with the incoming interface's IP address as the source address) go out the other interface. Policy-based routing is required to ensure that packets are sent out via the interface configured with the same IP address that is used as the source IP address in the packet, and to find the correct default gateway (if needed).

Please note:

The steps below show a simple non-persistent example (can be made persistent if run at every system startup). Please refer to your Linux documentation for details.
The actual steps depend on your configuration and may vary slightly depending on the specific cloud environment and Linux version. Please always refer to your cloud provider's documentation.

Assumptions:

The Linux instance has a primary Ethernet interface (eth0) with address 10.2.0.8/24 and a public IP address (PubIP0).
The Linux instance has a secondary Ethernet interface (eth1) with address 10.2.0.9/24 which also has a public IP address (PubIP1).
All firewalls on the operating system and cloud level are set to allow ICMP traffic to both interfaces.

Problem description:

A ping from an external host to the public IP address PubIP0 of the primary Ethernet interface works.
A ping from an external host to the public IP address PubIP1 of the secondary Ethernet interface fails.
A network trace on the cloud host shows that the ICMP packets to PubIP1 arrive at the cloud instance on eth1 as expected, but there is no answer.

When adding a second IP interface on the Charon-SSP host, the routing problems described above can occur. They can be solved by creating a second routing table and adding a routing policy as shown in the following example which uses the data provided in the assumptions above:

ip route add 10.2.0.9/32 dev eth1 table 99
ip route add default via 10.2.0.1 dev eth1 table 99
ip route add 10.2.0.0/24 dev eth1 table 99
ip rule add from 10.2.0.9 lookup 99

The example has the following effect:

It creates a non-default routing table (table ID 99) and adds the routes required for the secondary interface to this table. In particular, any primary or alias IP address assigned to the interface must be added.
It then defines a routing policy that any traffic with the source address of the secondary Ethernet interface must use the non-default routing table. This forces traffic sent to the IP address of the secondary Ethernet interface to also leave the system via this interface.

You can verify the configuration using the commands:

ip route show table 99
ip rule show all

Once you found a configuration solving your problem, you can make the configuration permanent by adding it to a startup script.

Please refer to the Linux man pages for ip rule and ip route for more information.

Additional information for Charon-SSP marketplace images: the home directory of the sshuser contains a script named active_sec_network.sh. This script is only an example that illustrates how to create a systemd service to activate necessary routes and rules during system boot (instead of using steps 7 and 8 above). Do not use this script without carefully adapting it to your requirements - failing to do so, may make your system unreachable.

Cloud Instance and IP Forwarding

If a Charon cloud instance is to forward IP packages between its interfaces (act as a router), in addition to configuring IP forwarding on Linux (/sbin/sysctl -w net.ipv4.ip_forward=1), an additional configuration step is required in the configuration of the cloud instance. This configuration has different names in the different cloud environments.

Source/Destination checking on AWS and OCI must be disabled for all relevant interfaces of the instance.
IP forwarding on Azure must be enabled for all relevant interfaces of the instance.
IP forwarding on GCP must be enabled for an instance when it is created.
IP spoofing must be enabled for all relevant interfaces of the instance in the IBM cloud.

Without this configuration, the cloud providers block packets that do not contain the IP address of the cloud instance interface in either the source or destination field.

Interface Configuration Basics

This section shows some basic approaches on how to configure the network interfaces on a Charon host for use by the guest system. That is, the interface should be activated at boot, but without an IP address. The IP address assigned by the Cloud provider can then be used by the guest system.

It is by no means a complete documentation but should provide a starting point. Further information can be found in the documentation of your Charon Linux host and the documentation of your cloud provider. Please refer to them for any additional information beyond the basic examples below.

Please note: in some cases the base marketplace Linux image from which the cloud instance was launched, comes with utilities that automatically configure the interface addresses based on the cloud instance metadata. If this is the case, some additional steps may be needed to remove the IP address from an interface that is to be dedicated to an emulator. Please refer to the Network Interface Management section in this guide for additional information concerning such cases.

The examples show possible configuration steps on

Linux systems with file-based network configuration (mostly Linux 7.x), and
Linux system with NetworkManager-based network configuration (mostly Linux 8.x and higher)

Please note: the interface names used in the following section are for illustrative purposes only. Please familiarize yourself with the interface naming conventions used in your cloud environment.

Expected result of the example:

The system should still be reachable via eth0.
Interface eth1 should be up without having an IP address configured.

Basic File-based Interface Configuration without NetworkManager

This configuration applies to systems with a file-based network configuration where the NetworkManager is either not active, or where network interfaces should be excluded from NetworkManager control (e.g., to be managed by the Charon Manager). The NetworkManager is disabled by default in older Charon-SSP marketplace images that are based on Centos 7.

Please note:

The sample configuration assumes a CentOS 7 system and that the interface is configured outside the control of the NetworkManager.
Should the NetworkManager be active, the plugins ifcfg-rh and ibft must be enabled in section main of the NetworkManager configuration file /etc/NetworkManager/NetworkManager.conf.
After the initial creation of the ifcfg-file, the interface can be managed by the Charon-SSP Manager.

To make the second interface usable for the Charon guest system, perform the following steps:

Add a second interface to your instance as described in the cloud-specific Getting Started guide and your cloud provider's documentation.
Log into the instance and become the root user (use: sudo -i)
Identify the names of the two Ethernet interfaces:
# ip link show
Create an interface configuration file for the second interface (the file for the first one should exist). Example (use correct interface name for your configuration):
# cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth1
Edit this file to match the characteristics of eth1 (use correct interface name for your configuration). The private IP address used for this interface will be assigned to the Solaris guest. Therefore, configure the Linux Interface without IP address, similar to the example below.

Please note:
On some cloud platforms, the automatic cloud-specific configuration prevents the entries in the ifcfg-file to take effect (for example on GCP). Please refer to your cloud-provider's documentation and the Network Management section in the Getting Started Guide of your version for additional information.

BOOTPROTO=none
DEVICE=eth1
NAME=eth1
ONBOOT=yes
TYPE=Ethernet
USERCTL=no NM_CONTROLLED=no
Restart the network:
# systemctl restart networkPlease note: Should there be an error when executing this command, kill the DHCP client process and retry the command.

Basic Interface Configuration with NetworkManager

This configuration applies to systems where the NetworkManager is active and network interfaces are under NetworkManager control. The NetworkManager is enabled by default in newer Charon-SSP marketplace images that are based on Rocky Linux 8.x.

Please note:

The interface names used in the following section are for illustrative purposes only. Please familiarize yourself with the interface naming conventions used in your cloud environment.
The sample configuration assumes a Rocky Linux 8.x system and that the interfaces are under the control of the NetworkManager.
On some cloud platforms, the automatic cloud-specific configuration prevents the operating system configuration to take effect (for example on GCP). Please refer to your cloud-provider's documentation and the Network Management section in the Getting Started Guide of your version for additional information.

In such environments, you have different options to configure network interfaces for use by the guest system. The main options are the following:

On a Charon-SSP system, use the Charon Manager Network Settings utility
On a Linux system with a graphical user interface, use the provided graphical network management tools. This is typically not available in cloud environments.
On a Linux system without a graphical user interface, use the nmtui utility or nmcli commands.
Manually create and modify ifcfg-files in /etc/sysconfig/network-scripts.

The following sections show samples for options 1 and 3.

Charon-SSP Manager Network Settings

The Charon-SSP Manager provides basic network configuration options.

To access them, start the Charon Manager and open the menu option:
Tools > Network Settings
To configure a host system for use by the emulator perform the following steps:
- Select the correct interface.
- In the IP setting field select None.
- Click on Apply.

Using the nmtui Utility

The nmtui utility provides a method to configure the network settings for the NetworkManager in a text-based environment without having to know the nmcli commands. It is provided via the NetworkManager-tui package.

The following basic example shows how to remove the IP address from the interface and how to reactivate the interface afterwards.

Start the tool as the root user: # nmtui
Use the up/down and left/right arrows to navigate.
Select Edit a connection and press RETURN.
Select the interface you want to configure.
Select Edit on the right side and press RETURN.
To make the interface come up without an IP address at boot, set the IP configuration to disabled (pressing RETURN on the value field will open a menu), and enable the automatic connection.
Select OK and press RETURN.
Navigate back to the main screen.
Select Activate a connection and press RETURN
Select the interface you want to reactivate and select Deactivate on the right. Press RETURN.
Repeat the steps for the Activate option to reactivate the interface.
Navigate back to the main screen and end the session.

Using nmcli Commands

To configure the interface dedicated to the emulator such that it receives no IP address but is activated at start, you could use command similar to the following:

1. Identify the NetworkManager connection to configure. The interface may have been automatically activated by the NetworkManager. In the example, it is "Wired connection 1" on device eth1.

# nmcli conn show
NAME                UUID                                  TYPE      DEVICE
System eth0         5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03  ethernet  eth0
Wired connection 1  027a1c2b-3397-37fb-a6e2-f2e02eb59992  ethernet  eth1

If there is no connection for the interface yet, check if the device is visible using the command nmcli dev status or ip link show.

2. For an existing connection:

a) Configure an appropriate name for the connection if required:

# nmcli conn mod "Wired connection 1" con-name eth1

b) Set the IP configuration such that no IP address is assigned:

# nmcli conn mod eth1 ipv4.method disable ipv6.method ignore

c) Configure automatic interface activation at boot:

# nmcli conn mod eth1 connection.autoconnect yes

3. If no connection for the second interface exists:

Add a new connection (with automatic interface activation, without IP address):

# nmcli conn add con-name eth1 type ethernet ifname eth1 autoconnect yes ipv4.method disable ipv6.method ignore

4. (Re-)Activate the connection:

# nmcli con up eth1

Further Information

The following sections provide additional information:

Network Interface Management (adding additional NICs to the Charon host instance)
Interface MTU considerations
Configuration example: SSH VPN tunnel to connect Charon host and guest to remote systems or customer networks
Configuration example: dedicated NIC for the Solaris guest system

Contents

General Information

Linux Versions and NetworkManager

Interface MTU Considerations

Host to Guest Communication Considerations

External Communication Considerations

Guest to Guest Layer 2 Communication Considerations

Asymmetric Routing Considerations

Cloud Instance and IP Forwarding

Interface Configuration Basics

Basic File-based Interface Configuration without NetworkManager

Basic Interface Configuration with NetworkManager

Charon-SSP Manager Network Settings

Using the nmtui Utility

Using nmcli Commands

Further Information

Browser not supported