Contents

Cloud-Routing Introduction

The configuration of IP routing in and between cloud LANs and between cloud LANs and on-premises networks requires additional steps when compared to IP routing in an on-premises LAN.

Unfortunately, the steps in the different cloud environments differ due to the different network implementations of the cloud providers. And cloud LAN implementations in general are quite different from physical LANs in an on-premises network.

The most important differences to bear in mind for all clouds are the following:

Only IP unicast traffic is supported.
Broadcast traffic and multicast traffic are not supported.
The ARP protocol is simulated to satisfy the requirements of the cloud instance operating system, but does not use the usual broadcast mechanism.
The MAC address assigned to an interface by the cloud provider must not be changed. Only this one MAC address is allowed to be visible outside the instance.
Only IP addresses assigned to an interface by the cloud provider are allowed in the cloud LAN.
Typical L2 traffic or non-IP protocols are not allowed.

The following sections will show some of the most important cloud-specific configuration steps required for a custom routing configuration.

Common Steps to Configure Routing on a Linux Instance

The following steps are required on a Linux host to configure custom routing independent of were the Linux instance is installed.

If the Linux instance is to act as a router, IP forwarding must be enabled:

Non-persistent change: # sysctl -w net.ipv4.ip_forward=1

Persistent change: add net.ipv4.ip_forward=1 to a file in /etc/sysctl.d.

Add the custom routes to the routing table as required:

Non-persistent: # ip route add <network>/<netmask> via <gatewayIP>
Persistent (NetworkManager method): # nmcli connection modify <connection-name> +ipv4.routes "<network>/<netmask><gatewayIP>"
Persistent (network-scripts method):
- Create a file in /etc/sysconfig/network-scripts named route-<interface-name>
- Add a line for each static route in the form of
  <network>/<netmask> via <gatewayIP> dev <interface-name>
Add a default gateway if not learned automatically by DHCP.

Azure Routing Specifics

Enable IP Forwarding for the Instance

Even if forwarding is enabled on a Linux instance in the Azure cloud, forwarding will not work unless it is explicitly configured for the instance. This is required any time the Linux instance is to accept packets that are not directed towards one of the IP addresses assigned to it by the cloud provider.

This change must be made on every affected NIC of the instance. Follow the steps below:

Menu path: Virtual Machine > Select instance > Networking > Select NIC > IP Configurations > use slider to enable IP Forwarding > Save
Repeat for additional NICs.

Add a Route Table for Custom Routes the the Instance Subnet

Routes not being part of the Azure vNet range used must be added to the vNet environment via a route table. Using certain managed services (e.g., VPN gateways), such routes are added automatically. A route table can be thought of as being a way to configure the routing table of the cloud LAN default gateway.

To create a new route table use the following steps from the Azure services overview:

+Create > search for Route Table > press Create in the Route Table selection

To add routes to a route table use the following steps:

Select the Route Table from the resource overview > Routes > Add

To add the route table to a subnet, perform the following steps:

Virtual Machine > Select instance > Networking > Virtual network / subnet > Subnets > click on subnet > select correct Route Table > Save

To check if the routes are active, perform the following steps on the running cloud instance:

Virtual Machine > Select instance > Networking > Select NIC > Effective Routes

Other Influences on Network Operation

ARP operation

Even though each NIC has its own MAC address from a Linux perspective, any ARP request for other systems on the same LAN will be answered with the MAC address of the default gateway. This can be confusing during troubleshooting as it is not possible from the ARP entries to see which MAC address a target system has. Also, static ARP entries become meaningless.

Default gateway behavior

The Azure cloud LAN default gateway does not answer to pings. This can cause problems for some guest systems using a dedicated NIC and relying on ICMP requests being answered by the gateway.

Potential Routing Problems on Guest when Using a Dedicated NIC

The Azure cloud LAN default gateway does not respond to pings. This may cause problems for guest operating systems that use ICMP as a mechanism of "Dead Gateway Detection".

On example for this is HP-UX: once it marks the default gateway as unusable, any additional traffic that would require the default gateway fails.

Possible workaround for HP-UX:

Check if dead gateway detection is active:
# ndd -get /dev/ip ip_ire_gw_probe
a 1 indicates that the feature is active
Reboot and immediately disable dead gateway detection (before the first gateway check occurs):
# ndd -set /dev/ip ip_ire_gw_probe 0
Test connectivity.
If the command solved the problem, you can make it permanent by editing /etc/rc.config.d/nddconf and adding the following lines:
TRANSPORT_NAME[0]=ipNDD_NAME[0]=ip_ire_gw_probeNDD_VALUE[0]=0

Possible workaround for other guest operating systems (or if the above is not an option for HP-UX):

Add static routes for the custom network routes to the guest OS in addition to the default route.
The static routes should point to the correct next hop. In some cases, it is also possible to point them to the default gateway (tested on HP-UX 11.11).

Browser not supported