Network Interface Configuration

This chapter provides a primer on configuring multiple network interfaces after installing and initializing a CipherTrust Manager physical appliance or private cloud Virtual CipherTrust Manager, and bonding these interfaces to achieve redundancy and performance improvements.

Caution

Network interface configuration on public clouds should be done using the cloud provider's tools.

The CipherTrust Manager uses GNOME NetworkManager and its CLI tool nmcli to configure network interfaces. For in-depth details regarding these tools, refer to GNOME official documentation:

GNOME NetworkManager

GNOME nmcli tool

Caution

Modifying a remote network interface over SSH is risky. The remote connection will stop responding if the IP address settings are incorrectly configured, resulting in the remote machine being unreachable.

Typical Network Configuration Tasks

This section provides an example of typical network configuration steps using nmcli.

To get started with nmcli

1. Use nmcli to list the available network devices (also called network interfaces or NICs). The devices listed in this example are truncated for brevity.

   $ nmcli device
   
   DEVICE       TYPE      STATE         CONNECTION
   kylo0        bridge    connected     kylo0
   ens3         ethernet  connected     ens3
   ens4         ethernet  disconnected  --
   veth05d02c4  ethernet  unmanaged     --
   ...

   Copy

   The output of this command may show over 30 devices. However, only a few require explanation:

   • kylo0 is used for internal communication by CipherTrust Manager services. It should never be altered in any way.

   • ens3 is an active device with a NetworkManager connection profile named ens3.

   • ens4 is an inactive device with no NetworkManager connection profile defined.

   • veth05d02c4 is an unmanaged device and should be ignored because NetworkManager cannot be used to control or configure the device.

   As a general rule, device names starting with eth or en are devices an administrator can configure.

   Consider giving connections meaningful names based on the context of their use. For example, if ens3 is responsible for web traffic and ens4 is responsible for database traffic, then naming the connections web and db provides better context for how they are used.

2. Next, use nmcli to see an active device's live values. In this example the only active device so far is ens3.

   $ nmcli device show ens3
   
   GENERAL.DEVICE:            ens3
   GENERAL.TYPE:              ethernet
   GENERAL.HWADDR:            00:50:56:99:3F:54
   GENERAL.MTU:               1500
   GENERAL.STATE:             100 (connected)
   GENERAL.CONNECTION:        ens3
   GENERAL.CON-PATH:          /org/freedesktop/NetworkManager/ActiveConnection/8
   WIRED-PROPERTIES.CARRIER:  on
   IP4.ADDRESS[1]:            10.121.105.97/22
   IP4.GATEWAY:               10.121.104.1
   IP4.DNS[1]:                10.121.8.7
   IP4.DNS[2]:                172.16.2.13
   IP4.DNS[3]:                172.16.2.12
   IP6.ADDRESS[1]:            fe80::bd7e:b93f:7e66:4b92/64
   IP6.GATEWAY:

   Copy

   The output shows that IPv4 has an address and accompanying values and that IPv6 only has a link-local address.

3. Next, use nmcli to see the configured network devices that NetworkManager calls connections. Notice that the connections listed in this example do not include device ens4 because it is not yet configured.

   $ nmcli conn
   
   NAME   UUID                                  TYPE            DEVICE
   ens3   9d86421b-7032-48eb-ac5a-3c84d285d01e  802-3-ethernet  ens3
   kylo0  78765519-b051-4d85-a9bd-3a681ce3c9cf  bridge          kylo0

   Copy

   A connection NAME and a DEVICE name are the same in this example. However, the connection name does not need to match the device name and may be any string, such as "Wired connection 1".

4. Next, use nmcli to create a connection configuration for device ens4 using DHCP for IPv4 and no IPv6 configuration. When ipv4.method and/or ipv6.method are not explicitly provided, then NetworkManager uses auto by default.

   $ nmcli conn add type ethernet con-name ens4 ifname ens4 -- ipv4.method auto ipv6.method ignore

   Copy

   If the network interface name begins with eth then there is a known issue where the network interface name may be associated with a different MAC address after a reboot. In this case it is highly recommended to bind the connection to the MAC address instead of the network interface name when creating the connection profile.

   $ nmcli conn add type ethernet con-name db ifname '' -- ethernet.mac-address 00:50:56:99:3F:55 ipv4.method auto ipv6.method ignore

   Copy

5. Next, use nmcli to confirm the connection is created with the correct configuration for device ens4. The configuration and active values in this example are truncated for brevity.

   $ nmcli conn show ens4
   
   connection.id:                          ens4
   connection.uuid:                        d797d28c-fe8a-49ab-8181-271870d6cfc6
   connection.interface-name:              ens4
   connection.type:                        802-3-ethernet
   ...
   ipv4.method:                            auto
   ...
   ipv6.method:                            ignore
   ...
   IP4.ADDRESS[1]:                         10.121.105.113/22
   IP4.GATEWAY:                            10.121.104.1
   IP4.DNS[1]:                             10.121.8.7
   IP4.DNS[2]:                             172.16.2.13
   IP4.DNS[3]:                             172.16.2.12
   ...

   Copy

   The output format uses lowercase key names to indicate configuration values (e.g. ipv4.method) and uppercase key names to indicate live values (e.g. IP4.ADDRESS[1]). From this output we can see the newly configured device's IP address obtained via DHCP is 10.121.105.113.

6. Next, try connecting to ens4's address from a browser and confirm that CipherTrust Manager UI loads. If the UI does not load, try pinging the address to confirm the IP address can be reached. If neither works, then double check the connection values are as expected.

7. Now that a connection is created for ens4, if it needs to be altered, then the modify sub-command may be used. Use nmcli to modify device ens4's connection to use a static IP address instead of DHCP.

   $ nmcli conn modify ens4 ipv4.method manual ipv4.addresses 10.121.105.18/22 ipv4.gateway 10.121.104.1 ipv4.dns 8.8.8.8,8.8.4.4
   
   nmcli conn show ens4 | grep IP4.ADDRESS
   
   IP4.ADDRESS[1]:                         10.121.105.113/22

   Copy

8. Notice that the IP4.ADDRESS[1] field listed in the second command still has the original IP address obtained via DHCP. In order to activate the modification, the connection must be restarted using the up sub-command:

   $ nmcli conn up ens4
   
   Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/15)
   
   $ nmcli conn show ens4 | grep IP4.ADDRESS
   
   IP4.ADDRESS[1]:                         10.121.105.18/22

   Copy

Setting IPv4 or IPv6 Protocol for a Network Device

By default, NetworkManager uses auto for ipv4.method and ipv6.method, creating both IPv4 and IPv6 addresses for a CipherTrust Manager network device. You can explicitly provide ipv4.method and/or ipv6.method to disable creation of either type of address.

We suggest explicitly disabling IPv6 unless it is required for your network. That way, there are fewer IP addresses exposed for CipherTrust Manager network devices, and therefore fewer ways for potential attackers to reach CipherTrust Manager.

You can specify IPv4 or IPv6 when first adding a network connection, or when modifying a network connection.

Disabling IPv6 for a network interface does not disable the link local address. As traffic to and from link local addresses cannot be propagated through a router, an IPv6 link local address generally does not present security concerns with external attackers. If you still need to disable the link local address, contact customer support.

To Add a New Connection with IPv4 Only

1. In the nmcli connection add command, specify ipv4.method auto ipv6.method ignore.

   nmcli conn add type ethernet con-name <connection-name> ifname <connection-interface-name> -- ipv4.method auto ipv6.method ignore

   Copy

2. View the connection details to ensure IPv6 is disabled. ipv6.method is set to ignore for correct configuration. For example:

   nmcli con show <connection-name>
   connection.id:                          eth0
   [...]
   ipv6.method:                            ignore
   [...]
   IP6.ADDRESS[1]:                         fe80::c9c:4cff:fe6a:1267/64

   Copy

To Modify an Existing Connection to Disable IPv6

1. Modify the IPv6 configuration for the connection.

   nmcli con modify <connection-name> ipv6.method ignore

   Copy

2. Apply the configuration changes to the connection

   nmcli con up <connection-name>

   Copy

3. View the connection details to ensure IPv6 is disabled. ipv6.method is set to ignore for correct configuration.

   nmcli con show <connection-name>
   connection.id:                          eth0
   [...]
   ipv6.method:                            ignore
   [...]
   IP6.ADDRESS[1]:                         fe80::c9c:4cff:fe6a:1267/64

   Copy

Configuring Static Routes

To configure a static route:

1. Modify the NetworkManager connection to use a static route.

   Run the command:

   $ nmcli conn modify <connection name> +ipv4.routes "<static IP address> <gateway>"

   Copy

   Example:

   $ nmcli conn modify ens3 +ipv4.routes "192.168.122.0/24 172.30.1.1"

   Copy

2. If you wish to configure DNS servers manually, modify the NetworkManager connection to use specific DNS servers.

   Run the commands:

   $ nmcli conn modify <network device> ipv4.ignore-auto-dns yes
   $ nmcli conn modify <network device> ipv4.dns  "<DNS servers>"

   Copy

   Example:

   $ nmcli conn modify <network device> ipv4.dns  "8.8.8.8, 4.4.4.4"

   Copy

3. Apply the configuration change.

   Run the command:

   $ nmcli conn up <connection name>

   Copy

   Example:

   $ nmcli conn up <connection name>
   
   Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/10)

   Copy

4. Verify that the route is active.

   $ ip route
   
   default via 172.30.1.1 dev ens3  proto static
   
   172.30.0.128/26 dev kylo0  proto kernel  scope link  src 172.30.0.129
   
   172.30.1.0/24 dev ens3  proto kernel  scope link  src 172.30.1.71
   
   192.168.122.0/24 via 172.30.1.1 dev ens3  proto static

   Copy

   The output shows that the route is now active.

Bonding Network Interfaces

A bonded network interface provides redundancy and performance improvements by aggregating two or more network interfaces to form a single logical interface.

The official Linux kernel HOWTO documents this feature in detail.

The nmcli's reference documentation covers details specific to NetworkManager configuration.

Note

Bonding network interfaces is intended for physical appliances, for example, CipherTrust Manager k570. It is generally not relevant to virtual CipherTrust Manager instances, as bonding is not useful in a virtual environment.

1. Start by using nmcli to create a bonded connection for fail-over only (the most basic type of bond).

   $ nmcli conn add type bond con-name bond0 ifname bond0 -- bond.options mode=active-backup
   Connection 'bond0' (c6661f6b-5d68-4ffa-9ae8-63f0e4224cb4) successfully added.
   $ nmcli conn show bond0
   bond.options:                           mode=active-backup

   Copy

   Alternatively, for certain networking environments such as in a virtual environment, the default setting of fail_over_mac=0 might not be optimal and setting fail_over_mac=1 can be an option. Please refer to the Linux Kernel HOWTO for more information. The command to create the bonded connection would then be:

   $ nmcli conn add type bond con-name bond0 ifname bond0 -- bond.options mode=active-backup,fail_over_mac=1
   Connection 'bond0' (c6661f6b-5d68-4ffa-9ae8-63f0e4224cb4) successfully added.
   $ nmcli conn show bond0
   ...
   bond.options:                           mode=active-backup,fail_over_mac=1

   Copy

2. Next, use nmcli to list devices and add some of them to the bond.

   $ nmcli device
   DEVICE       TYPE      STATE         CONNECTION
   ...
   ens3         ethernet  connected     ens3
   ens4         ethernet  disconnected  --
   ...

   Copy

3. Create the bond connections for ens3 and ens4 by specifying bond0 as their master and bring them up. Notice that ens3 has an existing connection profile, so delete it first to ensure only the bond connection profile is used.

   $ nmcli conn delete ens3
   Connection 'ens3' (f4e38090-6d3c-4612-b5b5-2d4c6a0c361d) successfully deleted.
   $ nmcli conn add type ethernet con-name ens3-bond0 ifname ens3 master bond0
   Connection 'ens3-bond0' (3d87c66e-a128-4e4f-897a-b31cf1b5fdbf) successfully added.
   $ nmcli conn add type ethernet con-name ens4-bond0 ifname ens4 master bond0
   Connection 'ens4-bond0' (b488d1f4-4a73-43ee-8a7f-9e992a850903) successfully added.
   $ nmcli conn up ens3-bond0
   Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7)
   $ nmcli conn up ens4-bond0
   Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8)

   Copy

4. Next, use nmcli to bring up the bonded interface and confirm that it received an IP address.

   $ nmcli conn show bond0
   ...
   connection.type:                        bond
   ...
   IP4.ADDRESS[1]:                         10.121.105.113/22

   Copy

5. Verify the bonded interface continues to receive and transmit data when either of the enslaved network interfaces are down (e.g. by removing the cable, disabling the port in the switch, etc.).

6. It is recommended to restart the OS to verify that the bond persist across reboots. If it does not then check if a connection profile exist for any of the interfaces that compose this bond and have an equal or higher connection priority. Either delete the duplicate connection profile or increase the priority of the bonded connections so it has precedence.

Configuring NIC Bonding with Static MAC Address

For CipherTrust Manager appliances, NIC bonding can result in alternating MAC address after reboot. If a static MAC address is preferred, it can be set on the virtual bonding network interface to ensure it never changes after reboot. The MAC address can, for example, be taken from the first NIC such as ens3.

After the NIC bonding is setup, a static MAC address can be set with the following command:

$ nmcli connection modify bond0 802-3-ethernet.cloned-mac-address "00:0C:29:56:F8:A2"

Note

This feature should be used together with fail_over_mac=0 on the bonded network interface, the default setting.

Virtual LAN (VLAN)

Virtual Local Area Network (VLAN) is a mechanism to virtually isolate systems located on the same physical network. If you want to configure your CipherTrust Manager network adapters to use VLAN, we recommend having supporting network equipment and also configuring VLAN on the network layer.

VLAN can be configured with any VLAN ID and it is also possible to mix non-VLAN and multiple VLANs on the same network adapter. When mixing non-VLAN and multiple VLANs be careful to keep subnets and routing correct for the machine to be able to route traffic using the correct VLAN ID.

All examples are illustrated using a static IP address but DHCP can be used assuming there is a VLAN aware DHCP server on the network.

Switch a network adapter to VLAN only

This example disables any non-VLAN configuration on the selected network adapter and enables VLAN ID 10 with a static IP address. With this configuration, this machine can only talk to other machines using the same VLAN ID.

Delete any existing connections to make sure no stale configuration is left on the connections. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.

$ nmcli conn delete ens3.10
$ nmcli conn delete ens3

Target setup:

• ens3 with no network configuration
• ens3.10 with VLAN ID 10 and a static IP address 192.168.1.2

Setup commands:

$ nmcli conn add type vlan con-name ens3.10 dev ens3 id 10 ip4 192.168.1.2/24

Any traffic to the subnet 192.168.1.0/24 uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.

Mixing non-VLAN and VLAN on same network adapter

This example enables both non-VLAN and VLAN on the same network adapter with different subnets. With this configuration, this machine can talk to other machines without VLAN on one subnet and with VLAN on another subnet.

Delete any existing connections to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.

$ nmcli conn delete ens3.10
$ nmcli conn delete ens3

Target setup:

• ens3 with no VLAN and with static IP address 192.168.1.2
• ens3.10 with VLAN ID 10 and a static IP address 192.168.2.2

Setup commands:

$ nmcli conn add type ethernet con-name ens3 ifname ens3 ip4 192.168.1.2/24
$ nmcli conn add type vlan con-name ens3.10 dev ens3 id 10 ip4 192.168.2.2/24

Any traffic to the subnet 192.168.1.0/24 does not use VLAN and any traffic to the subnet 192.168.2.0/24 uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.

Multiple VLANs on same network adapter

It is also possible to have multiple VLANs on the same network adapter. This example disables any non-VLAN configuration on the selected network adapter and enables VLAN ID 10 and VLAN ID 20 with static IP addresses with different subnets. With this configuration this machine can talk to two different VLANs using two different subnets.

Delete any existing connection to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.

$ nmcli conn delete ens3.10
$ nmcli conn delete ens3.20
$ nmcli conn delete ens3

Target setup:

• ens3 with no network configuration
• ens3.10 with VLAN ID 10 and a static IP address 192.168.1.2
• ens3.20 with VLAN ID 20 and a static IP address 192.168.2.2

Setup commands:

$ nmcli conn add type vlan con-name ens3.10 dev ens3 id 10 ip4 192.168.1.2/24
$ nmcli conn add type vlan con-name ens3.20 dev ens3 id 20 ip4 192.168.2.2/24

Any traffic to the subnet 192.168.1.0/24 uses VLAN 10 and any traffic to the subnet 192.168.2.0/24 uses VLAN 20 unless there are other subnets and routes configured on this machine using the same subnet.

VLAN with bonding

VLAN can also be used together with bonding. The principle is very similar to the normal non-bonding case with the difference that the bonded network adapter is used as the device. Consult Bonding Network Interfaces for more information on setting up bonding.

Switch a network adapter to VLAN only

This example disables any non-VLAN configuration on the selected bonded network adapter and enables VLAN ID 10 with a static IP address. With this configuration this machine can only talk to other machines using the same VLAN ID.

Delete any existing connection to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.

$ nmcli conn delete bond0.10
$ nmcli conn delete ens3-bond0
$ nmcli conn delete ens4-bond0
$ nmcli conn delete bond0

Target setup:

• bond0 with no network configuration
• bond0.10 with VLAN ID 10 and a static IP address 192.168.1.2

Setup commands:

$ nmcli conn add type bond con-name bond0 ifname bond0 -- bond.options mode=active-backup ipv4.method disabled ipv6.method ignore
$ nmcli conn add type ethernet con-name ens3-bond0 ifname ens3 master bond0
$ nmcli conn add type ethernet con-name ens4-bond0 ifname ens4 master bond0
$ nmcli conn add type vlan con-name bond0.10 dev bond0 id 10 ip4 192.168.1.2/24

Any traffic to the subnet 192.168.1.0/24 uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.

Mixing non-VLAN and VLAN on same network adapter

This example enables both non-VLAN and VLAN on the same network adapter with different subnets. With this configuration this machine can talk to other machines without VLAN on one subnet and with VLAN on another subnet.

Delete any existing connection to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.

$ nmcli conn delete bond0.10
$ nmcli conn delete ens3-bond0
$ nmcli conn delete ens4-bond0
$ nmcli conn delete bond0

Target setup:

• bond0 with no VLAN and with static IP address 192.168.1.2
• bond0.10 with VLAN ID 10 and a static IP address 192.168.2.2

Setup commands:

$ nmcli conn add type bond con-name bond0 ifname bond0 ip4 192.168.1.2/24 -- bond.options mode=active-backup
$ nmcli conn add type ethernet con-name ens3-bond0 ifname ens3 master bond0
$ nmcli conn add type ethernet con-name ens4-bond0 ifname ens4 master bond0
$ nmcli conn add type vlan con-name bond0.10 dev bond0 id 10 ip4 192.168.2.2/24

Any traffic to the subnet 192.168.1.0/24 does not use VLAN and any traffic to the subnet 192.168.2.0/24 uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.