...

Please note: The steps below only provide a basic overview. The exact tasks required will vary depending on your network design. Please refer to the AWS GCP documentation for details.

...

When an instance is created, a default Ethernet interface is attached to the system. This is the primary default network interface . You can create additional network interfaces and attach them to an instance. Please note: If an instance has only one Ethernet interface, a public IP address can be assigned to the interface automatically. However, this automatically assigned address will be removed by AWS if a second interface is added to the instance and the instance is stopped and restarted. Be careful not to lose connectivity to your instance when changing the network configuration. Refer to the section about Elastic IP Addresses for additional information.is mandatory. During the creation of the instance, you can add additional network interfaces.

General Information

The rules for Google cloud instances with respect to network interfaces are strict:

• Interfaces can only be added during instance creation.
• Each network interface configured in a single instance must be attached to a different VPC network.
• The additional VPC networks that the multiple interfaces will attach to must exist before an instance is created. See Using VPC Networks for instructions on creating additional VPC networks.
• You cannot delete a network interface without deleting the instance.

Therefore the required VPCs and subnets must exist before the instance is created.

To create additional VPCs (if required), perform the steps below.

Creating a New Network Interface

The following steps are required to create a new network interface that can later be added to an instance:

...

...

This opens the interface creation window.

Image Removed

On this screen,

• enter a description,
• select the subnet the interface should be on (select the subnet to which your instance is to be connected),
• allow AWS to automatically assign a private IP address or set a custom one from the subnet IP range, and
• associate the interface with a security group (often the same as for the instance).

Click on Create when you are done. The new interface will appear in the overview list. There you can assign a name to the interface. Check that the interface is in state available.

...

Attaching the Interface to your Instance

After creating a network interface, you have to assign it to the instance where it will be used.

...

The right-click opens the context menu. Select Attach.

This will open the window to enter the necessary instance information.

...

Select your instance from the drop-down list and click on Attach.

The state of your interface will change from available to in-use.

...

 You can also attach/detach existing interfaces from the instance overview screen. Select your instance and then Actions > Networking > Attach or Detach network interface.

Assigning an Elastic IP Address to the Network Interface

Please note:

• The public IP address assigned to your instance by default when it starts, is not persistent. You will receive a new address when the instance is stopped and started again. Also this address will be removed after adding a second interface to the instance and restarting the instance.
• An Elastic IP address is a persistent, public IPv4 address to be used for one of your network interfaces or instances. You can associate an Elastic IP address with any instance or network interface in your account.
• The advantage of associating the Elastic IP address with the network interface instead of directly with the instance is that you can move the network interface with its attributes easily from one instance to another.
• The initial automatically assigned public IP address will be removed as soon as you restart the instance after adding a network interface with an Elastic IP address to your instance. Do not restart your instance before you are sure you can reach it via the Elastic IP address. The automatically assigned public IP address will also be disabled if you assign an Elastic IP address to the primary Ethernet interface of the instance.

...

The table below describes the steps required to add an Elastic IP address to a network interface.

...

Image Removed

This will list the already created Elastic IP addresses.

...

In the overview list, click on Allocate new address if you need to allocate a new address. It is also possible to assign an existing address to an interface. However, each address can only be used for one instance.

Image Removed

This will open the address allocation window.

In the address allocation window, select the Amazon pool (or your own pool of public addresses), and click on Allocate.

Image Removed

The new address will be shown in the list.

...

Select the address. Then select Actions > Associate Elastic IP address. A window to enter the required options opens.

Image Removed

In the window,

...

Create VPCs and Subnets for Instance

Step 1: Open the VPC network section by clicking on the Navigation menu, then selecting VPC network, and clicking on VPC networks - as illustrated below.

Image Added

This will open the VPC overview page with the already existing VPCs. If all required VPCs and subnets already exist, continue with creating the new VM instance. Otherwise, continue with step 2.

Step 2: If you need to create a new VPC, click on CREATE VPC NETWORK at the top of the VPC overview list.

Image Added

This opens the VPC configuration window.

Step 3: Create VPC and subnets.

In the VPC configuration window, enter

• the VPC name, and
• the subnet name, region and address.

Image Added

Click on Create at the bottom of the window to create the VPC.

The new VPC should appear in the VPC overview list. Selecting the VPC in the overview list will open the detail information window. Example:

Image Added

Step 4: Create firewall rules for the VPC.

With the detail information open, click on Firewall. This will allow you to define the required firewall rules for the VPC.

An example of a small set of firewall rules that allow incoming SSH and ICMP is shown below:

Image Added

Adding Additional NICs to an Instance

Additional NICs are added during instance creation. Perform the following steps in the instance creation window:

• Open the advanced settings at the bottom of the VM creation window by clicking on Management, security, disks,... at the bottom of the page.
• Select Networking from the advanced settings section.
• Click on Add network interface.
• Select the correct subnet (created before).
• Set the information about internal and external IP address (static or ephemeral) as required.

Image Added

After adding all the required information, click on Done.

The second interface is now visible in the details page of the VM instance:

Image Added

Assigning a Static IP Address to a Network Interface

During the creation of a VM instance, when you add the default and optional additional NICs, you can determine if the IP addresses assigned to a NIC are static (persistent across restarts) or ephemeral (non-persistent across restarts). The process to add a static IP requires reserving the IP address. The public IP address may also have to be created first.

If you choose to add a static private IP address to an interface, you will get the following window to reserve a static private IP address:

Image Added

If you choose to add a static public IP address to an interface, you will get the following window to create (if needed) and reserve an address:

Image Added

You can also manage external IP addresses from the VPC network management section (Navigation menu > VPC network > External IP addresses):

Image Added

Detaching a Network Interface from an Instance

You can detach cannot delete a network interface from your instance in two ways:

1. Select your instance in the instance list and use the menu Actions > Networking > Detach Network Interface. Or,
2. Select your network interface in the network interface list and use the menu Actions > Detach.

Take care that this step will not make your instance unreachable.

Please note: the primary network interface cannot be detached.

Interface Naming on Linux Hosts with Enhanced Networking

When looking at the instance from the AWS management console, the interface names are eth0, eth1, etc.

On instances without support for enhanced networking the Linux interface names are also eth0, eth1, etc.

However, on instances with support for enhanced networking, the names on the AWS level differ from those on the Linux level:

• The first (primary) interface is called ensX (where X is an integer denoting the interface number; example: ens5).
• When a second interface is added to a running instance, it may initially be called eth0.
  However, the command ethtool -i eth0 shows that the enhanced network driver (ena) will be used for this interface. This interface will change its name to ensY (where Y is X+1) after restarting the instance. This means that any configuration file created for this interface must use the final name of the interface instead of eth0. Otherwise, the instance may become unreachable after a restart because there is no valid interface configuration (NetworkManager is not enabled on Charon-SSP AWS, so a configuration file must exist to configure the interface properly).
  Please note: this numbering sequence may change in the future. It is based on the PCI slot on which the Ethernet controller is presented and which is incremented by one for each additional Ethernet interface added. On the Charon host system, the slot can be verified with the following command:
  # lspci -vv | grep -A20 Ethernet

 To avoid confusion before the instance can be restarted, the new interface can be renamed to its final name using the command 
     ip link set eth0 name ensY && ip link set ensY up

...

Address Assignment Information

Each VPC is assigned a block of private IP addresses. This block can be split by the user to form several IP subnets. Routing between such subnets is automatically enabled.

When an E2C instance is launched into the default VPC and a public subnet, the default behavior is as follows:

• If the instance has only one network interface, it is automatically assigned a private IP address from the address range assigned to the public subnet and a public IP address. This network interface is the primary network interface. It is called eth0 on the AWS level (please refer to the interface naming section to learn about the interface names presented to the operating system).
• If the instance has more than one network interface, it is automatically assigned a private IP address for each of the network interfaces - but no public IP address.

The default behavior can be modified, for example:

• Manually assigning a private IP address from the subnet range.
• Enabling or disabling the automatic assignment of a private IP address to deviate from the subnet setting.
• Manually assigning a public IP address from the AWS range or the customer range.

Please note: Public IP addresses are not directly visible to the instance. The instance operating system always works with the private address. For external connections, the private address is mapped to the public IP address via NAT.

Reserved addresses (important, if manual address assignment is used):

The following address range is reserved to allow AWS to query meta-data about instance configuration: without deleting the instance it is attached to. So if you do not need a network anymore, but do not want to delete the instance, you can only disable it from the operating system level.

Address Assignment Information

General information

Each VM instance interface can have one primary internal IP address, one or more secondary IP addresses, and one external IP address.

Addresses can be static (persistent) or ephemeral (on-persistent):

• Ephemeral external IP addresses:
  • For VM instances, the ephemeral external IP address is also released if you stop the instance. After you restart the instance, it is assigned a new ephemeral external IP address.
• Static external IP addresses:
  • Static external IP address can be reserved and thereby assigned a project indefinitely until they are explicitly released. You can reserve a new static external IP address or promote an existing ephemeral external IP address to a static external IP address.
• Ephemeral internal IP addresses:
  • Ephemeral internal IP addresses remain attached to VM instances until the instance is deleted.
• Static internal IP addresses:
  • For VM instances, static internal IP addresses remain attached to stopped instances until they are removed.

Address Ranges

When creating a VPC and its subnets, subnet address ranges are assigned to these subnets. There are some restriction regarding permitted address ranges:

Restricted address ranges:

Restricted ranges include Google public IP addresses and commonly reserved RFC ranges, as described below. These ranges cannot be used for subnet ranges.

• Public IP addresses for Google APIs and services, including Google Cloud netblocks: You can find a link to these IP addresses in this Google FAQ.
• 199.36.153.4/30 and 199.36.153.8/30: private Google access-specific virtual IP addresses
• 0.0.0.0/8: Current (local) network RFC 1122
• 127.0.0.0/8: Local host RFC 1122
• 169.254.0.0/16

...

The following addresses are reserved in each subnet and cannot be used for E2C instances (shown in the example below for network 10.1.1.0/24):

• 10.1.1.0: the network address
• 10.1.1.1: reserved by AWS for the VPC router
• 10.1.1.2: reserved by AWS in any subnet; the second host address in the base VPC network range is the DNS server for the VPC.
• 10.1.1.3: reserved by AWS for future use
• 10.1.1.255: network broadcast address; AWS networks do not use broadcasts.

Please note: An automatically assigned public IP address is released (and not re-assigned) by AWS for example if

• a second interface is added to the instance and the instance is then stopped and restarted,
• an Elastic IP is associated with the the instance,
• an Elastic IP address is associated with the primary interface of the instance.

See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html for details.

...

• : Link-local RFC 3927
• 224.0.0.0/4: Multicast RFC 5771
• 255.255.255.255/32: Limited broadcast destination address RFC 8190 and RFC 919

Reserved subnet addresses:

Every subnet has four reserved IP addresses in its primary IP range. There are no reserved IP addresses in the secondary IP ranges.

• Network: first address in the primary IP range for the subnet 10.1.2.0 in 10.1.2.0/24
• Default gateway: Second address in the primary IP range for the subnet 10.1.2.1 in 10.1.2.0/24
• Second-to-last address: second-to-last address in the primary IP range for the subnet that is reserved by Google Cloud for potential future use 10.1.2.254 in 10.1.2.0/24
• Broadcast: last address in the primary IP range for the subnet 10.1.2.255 in 10.1.2.0/24

Please note:

• The default gateway does not respond to ping.
• The default gateway does not decrement TTL headers (used for traceroute).
• Only IPv4 unicast traffic is supported.

Interface Configuration on Linux

By default, Google cloud tools installed on the Linux instance automatically start the attached network interfaces and configure them using DHCP.

Should this be undesirable, for example, because a NIC is to be dedicated to the Solaris guest system, this automatic configuration can be suppressed by disabling the setup in the file /etc/default/instance_configs.cfg.

Important information:

• Currently, the Charon-SSP marketplace images are based on CentOS 7.
• NetworkManager is disabled by default in these images.
• If you disable the automatic interface setup as shown above, you must make sure that the correct ifcfg-files for every interface exist in /etc/sysconfig/network-config. Failure to do so, can make your instance unreachable after the next network restart.
• If you use a RHEL/CentOS 8 image as the base image for your Charon host, the interface must be controlled by the NetworkManager. You can set up the appropriate configuration by editing the interface configuration files or using nmcli commands.

To disable automatic interface configuration by the cloud tools, edit the file and set the parameter setup to false as shown in the example below:

# vi /etc/default/instance_configs.cfg
[NetworkInterfaces]
dhclient_script = /sbin/google-dhclient-script
dhcp_command =
ip_forwarding = true
setup = false

After restarting the network (systemctl restart network), the configuration as defined in the ifcfg-files should be set for the interfaces. On RHEL/CentOS 8 systems restart the NetworkManager instead (systemctl restart NetworkManager).

Additional GCP-specific Information

IP Interface Netmask

Please note: The latest images provided by Stromasys use a /24 netmask for additional NICs. Therefore, the following information no longer applies to instances created with these images.

However, other base images used to create an instance, may use a netmask of /32 for additional NICs on the VM instance. This means that only ARP requests for the default gateway are answered by the Google metadata server. In such cases, when providing a dedicated NIC to the Solaris guest system, that is, the internal IP address of the interface is not configured on the Linux level, but on the Solaris level, please note the following points:

• The netmask on Solaris has to be set to a value that includes the default gateway (e.g., /24). Otherwise, Solaris will return an error when setting the default gateway (network unreachable).
• If Solaris should communicate with systems on the same subnet, it needs a static ARP entry for these systems (arp -s <target-ip> <target-mac>). This is because the ARP requests sent by Solaris for the MAC addresses of these systems will not be answered by the Google metadata server and they will not be forwarded to the target system.

Routing between VPCs

If a VM instance has more than one NIC, each NIC must be in a different VPC. Routing between VPCs is not enabled by default. It has to be enabled through a mutual VPC peering configuration as shown in the sample below:

Image Added

The example shows one rule for each routing direction between the two VPCs.

If this is not enabled, host and guest system can only communicate via the external IP addresses, not via the internal IP addresses.

Network Interface MTU

The VPC network has a maximum transmission unit (MTU) of 1460 bytes. Interfaces should be configured to this value to avoid the increased latency and packet overhead caused by fragmentation. Client applications that communicate with GCP instances over UDP should have a maximum payload of 1432 bytes to avoid fragmentation.