Contents

General Information

This section provides some basic information about networking questions that are likely to affect Charon when running in the cloud.

If the information in this chapter is not sufficient, please refer to the additional Charon documentation provided by Stromasys and the documentation provided by your cloud provider for up-to-date and comprehensive information.

Linux Versions and NetworkManager

Interface MTU Considerations

When configuring a dedicated network interface for an emulator, ensure that the MTU of the Charon host interface used is not smaller than the MTU used by the legacy guest operating system. Failing to do so will cause network problems. For further information, please refer to the chapter Interface MTU Considerations in this guide.

Host to Guest Communication Considerations

There are several ways a communication between the host operating system and the legacy guest operating system can be implemented. For example:

1. Internal virtual bridge on the host system:
   Such a bridge has several TAP interfaces. The host and the guest systems are connected to this bridge and can communicate directly with one another using L3 and L2 protocols. The bridge uses its own IP subnet that can be defined by the user. For Charon-SSP, setting up such a configuration is supported by the Charon-SSP Manager (leave the default gateway field empty for the bridge interface). Several hosts configured with guest systems and such an internal bridge can communicate across the cloud-internal LAN and the host systems can route the private IP subnets of the bridges between themselves. L2 protocols are not possible if routing across the cloud LAN is used.
2. Communication via the cloud-internal subnet LAN:
   In this case, a second interface is added to the Charon host system. The second interface is then assigned to the emulated guest system. After configuring the interface correctly, the host and guest can communicate across the cloud-internal LAN using IP. Please note: L2 protocols or any protocols that require changing the MAC address to something different than the MAC address assigned to the second interface by the cloud provider will not work.
   To connect the guest system to the LAN, the following basic configuration steps must be performed:
   • Add the additional interface to the Charon host system.
   • Make a note of the private IP address assigned to the second interface by the cloud provider, and remove it from the Linux configuration (if it has been configured).
   • Create a configuration for the additional interface such that it is activated at boot but does not have an IP address on the Linux level. This can be done via configuration files on Linux 7.x. For Linux 8.x nmcli commands, the nmtui utility, or (for SSP) the Charon Manager can be used.
   • Assign the interface to the emulated system. This can be done by modifying the emulator configuration file or by using the Charon-SSP Manager.
   • Set the MAC address of the emulated system to the same value as the one used on the host system Ethernet interface. For Charon-SSP, this configuration can be implemented using the Charon Manager.
   • On the guest operating system, configure the private IP address that was previously assigned to the second interface on Linux and configure the appropriate default route for the LAN.

Please note:

• The section Dedicated NIC for Guest System provides some hints on how to configure the second interface in the different situations. Please refer to your cloud-provider's documentation for up-to-date comprehensive information.
• If Layer 2 communication between guests on different Charon hosts is required, a bridged overlay solution must be set up between the two Charon host systems. Direct L2 or non-IP L3 communication across the cloud LAN is not possible.

External Communication Considerations

In addition to allowing SSH access to the host system for management purposes, it may be necessary to enable Internet communication to the host and guest system or connect host and guest to the customer's network.

Please note: Charon hosts based on Charon AL (Automatic Licensing) marketplace images and using the public license servers always need either direct Internet access or Internet access via NAT from a NAT gateway in the same cloud as the Charon host to access the license server.

Recommended way to connect the Charon host and legacy guest systems (e.g., Solaris, HP-UX) to the customer network:

To ensure data traffic between the Charon host and guest systems and the customer network is encrypted, it is strongly recommended to use a VPN connection. An example of a simple VPN connection based on an SSH tunnel is described in SSH VPN - Connecting Charon Host and Guest to Customer Network. This connection is based on a bridge between Charon host and guest system and (via an encrypted SSH tunnel) the remote end-point in the customer network. The connection supports L3 and L2 protocols.

Cloud providers usually also provides a VPN gateway instance that can be added to the customer cloud network to connect the cloud network to the customer network (for a charge).

Recommended way to connect the guest system to the Internet:

The Internet connection can be implemented across the VPN to the customer network. In this case, the customer can allow the legacy guest operating system to access the Internet exactly following the security policies defined by the customer.

Access to the Internet from subnets or guest operating systems with only private IP addresses:

Access to the Internet for subnets with only private IP addresses is possible across a gateway instance providing VPN access to the customer network and allowing (NATted) Internet access via this path. Alternatively, a NAT gateway in the cloud can be used to map the private addresses to public addresses. The NAT gateway can be implemented on a Charon host system, a dedicated customer-operated gateway, or it can often be provided by the cloud provider for a charge.

Please note: a Charon AL host system that uses the public license servers always needs either direct Internet access or Internet access via NAT from a NAT gateway in the same cloud as the Charon host to access the public license server.

Direct guest system access to the Internet:

This not a recommended standard solution for security reasons. However, should it be required, two interfaces with public IP addresses can be assigned to the Charon host.
One of these interfaces is then dedicated to the guest system which uses the private interface address and the MAC address assigned to the Charon host by the cloud provider  (see also Dedicated NIC for Guest System).

Guest to Guest Layer 2 Communication Considerations

Should L2 protocols be required between two guest systems on different host systems, a bridged overlay solution must be set up between the two host systems to allow the L2 traffic to pass. Such a solution could be, for example, a simple SSH tunnel similar to the one described in SSH VPN - Connecting Charon Host and Guest to Customer Network, a VXLAN overlay network based on native Linux features, or a commercial solution. In all cases, it always requires thorough testing to verify that it fulfills the customer requirements.

Asymmetric Routing Considerations

This section applies to the case where several interfaces are configured on an instance and they all have IP addresses configured on the Linux level.

When you add a secondary NIC to a Linux instance, a new interface (that is, an Ethernet device) is added to the instance and automatically recognized by the OS. Depending on the cloud-provider, DHCP may not be active for the secondary VNIC, and you must configure the interface with a static IP address and add any routes that are relevant for the new interface.

Connectivity problems caused by asymmetric routing arise if traffic arrives through one interface and, when the service replies, the reply packets (with the incoming interface's IP address as the source address) go out the other interface. Policy-based routing is required to ensure that packets are sent out via the interface configured with the same IP address that is used as the source IP address in the packet, and to find the correct default gateway (if needed).

Please note:

• The steps below show a simple non-persistent example (can be made persistent via a startup script, or via a persistent network configuration). Please refer to your Linux documentation for details.
  
• The actual steps depend on your configuration and may vary slightly depending on the specific cloud environment and Linux version. Please always refer to your cloud provider's documentation.

Assumptions:

• The Linux instance has a primary Ethernet interface (eth0) with address 10.2.0.8/24 and a public IP address (PubIP0). The default route points to this interface.
• The Linux instance has a secondary Ethernet interface (eth1) with address 10.2.0.9/24 which also has a public IP address (PubIP1).
• All firewalls on the operating system and cloud level are set to allow ICMP traffic to both interfaces.

Problem description:

• A ping from an external host to the public IP address PubIP0 of the primary Ethernet interface works.
• A ping from an external host to the public IP address PubIP1 of the secondary Ethernet interface fails.
• A network trace on the cloud host shows that the ICMP packets to PubIP1 arrive at the cloud instance on eth1 as expected, but there is no answer. The reason is that the answer follows the default route via the primary interface, and this traffic is blocked by the cloud provider.

When adding a second IP interface to the Charon host, the routing problems described above can occur. They can be solved by creating a second routing table and adding a routing policy as shown in the following example which uses the data provided in the assumptions above:

ip route add 10.2.0.9/32 dev eth1 table 99
ip route add default via 10.2.0.1 dev eth1 table 99
ip route add 10.2.0.0/24 dev eth1 table 99
ip rule add from 10.2.0.9 lookup 99

The example has the following effect:

• It creates a non-default routing table (table ID 99) and adds the routes required for the secondary interface to this table. In particular, any primary or alias IP address assigned to the interface must be added.
• It then defines a routing policy that any traffic with the source address of the secondary Ethernet interface must use the non-default routing table. This forces traffic sent to the IP address of the secondary Ethernet interface to also leave the system via this interface.

You can verify the configuration using the commands:

• ip route show table 99
• ip rule show all

Once you found a configuration solving your problem, you can make the configuration permanent by adding it to a startup script.

Please refer to the Linux man pages for ip rule and ip route for more information.

Additional information for Charon-SSP marketplace images: the home directory of the sshuser contains a script named active_sec_network.sh. This script is only an example that illustrates how to create a systemd service to activate necessary routes and rules during system boot (instead of using steps 7 and 8 above). Do not use this script without carefully adapting it to your requirements - failing to do so, may make your system unreachable.

Cloud Instance and IP Forwarding

If a Charon cloud instance is to forward IP packages between its interfaces (act as a router), in addition to configuring IP forwarding on Linux (/sbin/sysctl -w net.ipv4.ip_forward=1), an additional configuration step is required in the configuration of the cloud instance. This configuration has different names in the different cloud environments.

• Source/Destination checking on AWS and OCI must be disabled for all relevant interfaces of the instance.
• IP forwarding on Azure must be enabled for all relevant interfaces of the instance.
• IP forwarding on GCP must be enabled for an instance when it is created.
• IP spoofing must be enabled for all relevant interfaces of the instance in the IBM cloud.

Without this configuration, the cloud providers block packets that do not contain the IP address of the cloud instance interface in either the source or destination field.

Interface Configuration Basics

This section shows some basic approaches on how to configure the network interfaces on a Charon host for use by the guest system.That is, the interface should be activated at boot, but without an IP address. The IP address assigned by the Cloud provider can then be used by the guest system.

It is by no means a complete documentation but should provide a starting point. Further information can be found in the documentation of your Charon Linux host and the documentation of your cloud provider. Please refer to them for any additional information beyond the basic examples below.

The examples show possible configuration steps on

• Linux systems with file-based network configuration (mostly Linux 7.x), and
• Linux system with NetworkManager-based network configuration (mostly Linux 8.x and later)

Please note: the interface names used in the following section are for illustrative purposes only. Please familiarize yourself with the interface naming conventions used in your cloud environment. 

Expected result of the example:

1. The system should still be reachable via eth0.
2. Interface eth1 should be up without having an IP address configured.

Basic File-based Interface Configuration without NetworkManager

This configuration applies to systems with a file-based network configuration where the NetworkManager is either not active, or where network interfaces should be excluded from NetworkManager control (e.g., to be managed by the Charon Manager). The NetworkManager is disabled by default in older Charon-SSP marketplace images that are based on Centos 7.

Please note:

• The sample configuration assumes a CentOS 7 system and that the interface is configured outside the control of the NetworkManager.
• Should the NetworkManager be active, the plugin ifcfg-rh must be enabled in section main of the NetworkManager configuration file /etc/NetworkManager/NetworkManager.conf. It enables the NetworkManager to read and write ifcfg-files.
• After the initial creation of the ifcfg-file, the interface can be managed by the Charon-SSP Manager.
• For the full feature-set of the file-based network configuration, the network-scripts package is required.

To make the second interface usable for the Charon guest system, perform the following steps:

1. Add a second interface to your instance as described in the cloud-specific Getting Started guide and your cloud provider's documentation.
   
   
2. Log into the instance and become the root user (use: sudo -i)
   
   
3. Identify the names of the two Ethernet interfaces:
   # ip link show
   
   
4. Create an interface configuration file for the second interface.
   1. A file for the first interface may exist depending on the default of the cloud environment. In this case, you can copy  Example (use correct interface name for your configuration):
      # cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth1


   2. If there is no file that can be copied, you must create the ifcfg-file for the new interface manually.
      
      
5. Edit this file to match the characteristics of eth1 (use correct interface name for your configuration). The private IP address used for this interface will be assigned to the Solaris guest. Therefore, configure the Linux Interface without IP address, similar to the example below.
   
   BOOTPROTO=none
DEVICE=eth1
NAME=eth1
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
NM_CONTROLLED=no 
   
   Please note:
   On some cloud platforms, the automatic cloud-specific configuration prevents the entries in the ifcfg-file to take effect (for example on GCP). Please refer to your cloud-provider's documentation and the Network Interface Management section for additional information.       


6. Restart the network:
   # systemctl restart network
Please note: Should there be an error when executing this command, kill the DHCP client process and retry the command.

Basic Interface Configuration with NetworkManager

This configuration applies to systems where the NetworkManager is active and network interfaces are under NetworkManager control. The NetworkManager is enabled by default in newer Charon-SSP marketplace images that are based on Rocky Linux 8.x.

Please note:

• The interface names used in the following section are for illustrative purposes only. Please familiarize yourself with the interface naming conventions used in your cloud environment. 

• The sample configuration assumes a Rocky Linux 8.x system and that the interfaces are under the control of the NetworkManager.

• On some cloud platforms, the automatic cloud-specific configuration prevents the operating system configuration to take effect (for example on GCP). Please refer to your cloud-provider's documentation for additional information.

In such environments, you have different options to configure network interfaces for use by the guest system. The main options are the following:

1. On a Charon-SSP system, use the Charon Manager Network Settings utility. For this, the interfaces must be under the control of the NetworkManager.
2. On a Linux system with a graphical user interface, use the provided graphical network management tools. This is typically not available in cloud environments.
3. On a Linux system without a graphical user interface, use the nmtui utility or nmcli commands.
4. Manually create and modify ifcfg-files in /etc/sysconfig/network-scripts. This requires the installation of the deprecated network-scripts package in Linux 8.x. In Linux 9.x this method only works for interface types supported by the NetworkManager ifcfg-rh plugin which does not have the full network-scripts functionality.

The following sections show samples for options 1 and 3.

Charon-SSP Manager Network Settings

The Charon-SSP Manager provides basic network configuration options.

• To access them, start the Charon Manager and open the menu option:
  Tools > Network Settings
• To configure a host system for use by the emulator perform the following steps:
  • Select the correct interface.
  • In the IP setting field select None.
  • Click on Apply.

Please note:

• For RHEL 7 and derivatives, the interfaces to be managed by the Charon Manager must be removed from NetworkManager control and an ifcfg-file must exist.
• For RHEL 8 and later (and derivatives), the interfaces to be managed by the Charon Manager must be under NetworkManager control.

Using the nmtui Utility

The nmtui utility provides a method to configure the network settings for the NetworkManager in a text-based environment without having to know the nmcli commands. It is provided via the NetworkManager-tui package.

The following basic example shows how to remove the IP address from the interface and how to reactivate the interface afterwards.

• Start the tool as the root user: # nmtui
• Use the up/down and left/right arrows to navigate.
• Select Edit a connection and press RETURN.
• Select the interface you want to configure.
• Select Edit on the right side and press RETURN.
• To make the interface come up without an IP address at boot, set the IP configuration to disabled (pressing RETURN on the value field will open a menu), and enable the automatic connection.
• Select OK and press RETURN.
• Navigate back to the main screen.
• Select Activate a connection and press RETURN
• Select the interface you want to reactivate and select Deactivate on the right. Press RETURN.
• Repeat the steps for the Activate option to reactivate the interface.
• Navigate back to the main screen and end the session.

Using nmcli Commands

To configure the interface dedicated to the emulator such that it receives no IP address but is activated at start, you could use command similar to the following:

1. Identify the NetworkManager connection to configure. The interface may have been automatically activated by the NetworkManager. In the example, it is "Wired connection 1" on device eth1.

# nmcli conn show
NAME                UUID                                  TYPE      DEVICE
System eth0         5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03  ethernet  eth0
Wired connection 1  027a1c2b-3397-37fb-a6e2-f2e02eb59992  ethernet  eth1

If there is no connection for the interface yet, check if the device is visible using the command nmcli dev status or ip link show.

2. For an existing connection:

a) Configure an appropriate name for the connection if required:

# nmcli conn mod "Wired connection 1" con-name eth1

b) Set the IP configuration such that no IP address is assigned:

# nmcli conn mod eth1 ipv4.method manual ipv4.address 0.0.0.0

c) Configure automatic interface activation at boot:

# nmcli conn mod eth1 connection.autoconnect yes

3. If no connection for the second interface exists:

Add a new connection (with automatic interface activation, without IP address):

# nmcli conn add con-name eth1 type ethernet ifname eth1 autoconnect yes ipv4.method manual ipv4.addresses 0.0.0.0

4. (Re-)Activate the connection:

# nmcli con up eth1

Further Information

The following sections provide additional information: