Network Interface Configuration
This chapter provides a primer on configuring multiple network interfaces after installing and initializing a CipherTrust Manager physical appliance or private cloud Virtual CipherTrust Manager, and bonding these interfaces to achieve redundancy and performance improvements.
Caution
Network interface configuration on public clouds should be done using the cloud provider's tools.
The CipherTrust Manager uses GNOME NetworkManager and its CLI tool nmcli
to configure network interfaces. For in-depth details regarding these tools, refer to GNOME official documentation:
Caution
Modifying a remote network interface over SSH is risky. The remote connection will stop responding if the IP address settings are incorrectly configured, resulting in the remote machine being unreachable.
Typical Network Configuration Tasks
This section provides an example of typical network configuration steps using nmcli
.
To get started with nmcli
Use
nmcli
to list the available network devices (also called network interfaces or NICs). The devices listed in this example are truncated for brevity.$ nmcli device DEVICE TYPE STATE CONNECTION kylo0 bridge connected kylo0 ens3 ethernet connected ens3 ens4 ethernet disconnected -- veth05d02c4 ethernet unmanaged -- ...
The output of this command may show over 30 devices. However, only a few require explanation:
kylo0
is used for internal communication by CipherTrust Manager services. It should never be altered in any way.ens3
is an active device with a NetworkManager connection profile namedens3
.ens4
is an inactive device with no NetworkManager connection profile defined.veth05d02c4
is an unmanaged device and should be ignored because NetworkManager cannot be used to control or configure the device.
As a general rule, device names starting with
eth
oren
are devices an administrator can configure.Consider giving connections meaningful names based on the context of their use. For example, if
ens3
is responsible for web traffic andens4
is responsible for database traffic, then naming the connectionsweb
anddb
provides better context for how they are used.Next, use
nmcli
to see an active device's live values. In this example the only active device so far is ens3.$ nmcli device show ens3 GENERAL.DEVICE: ens3 GENERAL.TYPE: ethernet GENERAL.HWADDR: 00:50:56:99:3F:54 GENERAL.MTU: 1500 GENERAL.STATE: 100 (connected) GENERAL.CONNECTION: ens3 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/8 WIRED-PROPERTIES.CARRIER: on IP4.ADDRESS[1]: 10.121.105.97/22 IP4.GATEWAY: 10.121.104.1 IP4.DNS[1]: 10.121.8.7 IP4.DNS[2]: 172.16.2.13 IP4.DNS[3]: 172.16.2.12 IP6.ADDRESS[1]: fe80::bd7e:b93f:7e66:4b92/64 IP6.GATEWAY:
The output shows that IPv4 has an address and accompanying values and that IPv6 only has a link-local address.
Next, use
nmcli
to see the configured network devices that NetworkManager calls connections. Notice that the connections listed in this example do not include deviceens4
because it is not yet configured.$ nmcli conn NAME UUID TYPE DEVICE ens3 9d86421b-7032-48eb-ac5a-3c84d285d01e 802-3-ethernet ens3 kylo0 78765519-b051-4d85-a9bd-3a681ce3c9cf bridge kylo0
A connection NAME and a DEVICE name are the same in this example. However, the connection name does not need to match the device name and may be any string, such as "Wired connection 1".
Next, use
nmcli
to create a connection configuration for deviceens4
using DHCP for IPv4 and no IPv6 configuration. Whenipv4.method
and/oripv6.method
are not explicitly provided, then NetworkManager usesauto
by default.$ nmcli conn add type ethernet con-name ens4 ifname ens4 -- ipv4.method auto ipv6.method ignore
If the network interface name begins with
eth
then there is a known issue where the network interface name may be associated with a different MAC address after a reboot. In this case it is highly recommended to bind the connection to the MAC address instead of the network interface name when creating the connection profile.$ nmcli conn add type ethernet con-name db ifname '' -- ethernet.mac-address 00:50:56:99:3F:55 ipv4.method auto ipv6.method ignore
Next, use
nmcli
to confirm the connection is created with the correct configuration for deviceens4
. The configuration and active values in this example are truncated for brevity.$ nmcli conn show ens4 connection.id: ens4 connection.uuid: d797d28c-fe8a-49ab-8181-271870d6cfc6 connection.interface-name: ens4 connection.type: 802-3-ethernet ... ipv4.method: auto ... ipv6.method: ignore ... IP4.ADDRESS[1]: 10.121.105.113/22 IP4.GATEWAY: 10.121.104.1 IP4.DNS[1]: 10.121.8.7 IP4.DNS[2]: 172.16.2.13 IP4.DNS[3]: 172.16.2.12 ...
The output format uses lowercase key names to indicate configuration values (e.g.
ipv4.method
) and uppercase key names to indicate live values (e.g.IP4.ADDRESS[1]
). From this output we can see the newly configured device's IP address obtained via DHCP is 10.121.105.113.Next, try connecting to
ens4
's address from a browser and confirm that CipherTrust Manager UI loads. If the UI does not load, try pinging the address to confirm the IP address can be reached. If neither works, then double check the connection values are as expected.Now that a connection is created for
ens4
, if it needs to be altered, then themodify
sub-command may be used. Usenmcli
to modify deviceens4
's connection to use a static IP address instead of DHCP.$ nmcli conn modify ens4 ipv4.method manual ipv4.addresses 10.121.105.18/22 ipv4.gateway 10.121.104.1 ipv4.dns 8.8.8.8,8.8.4.4 nmcli conn show ens4 | grep IP4.ADDRESS IP4.ADDRESS[1]: 10.121.105.113/22
Notice that the IP4.ADDRESS[1] field listed in the second command still has the original IP address obtained via DHCP. In order to activate the modification, the connection must be restarted using the
up
sub-command:$ nmcli conn up ens4 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/15) $ nmcli conn show ens4 | grep IP4.ADDRESS IP4.ADDRESS[1]: 10.121.105.18/22
Setting IPv4 or IPv6 Protocol for a Network Device
By default, NetworkManager uses auto
for ipv4.method
and ipv6.method
, creating both IPv4 and IPv6 addresses for a CipherTrust Manager network device. You can explicitly provide ipv4.method
and/or ipv6.method
to disable creation of either type of address.
We suggest explicitly disabling IPv6 unless it is required for your network. That way, there are fewer IP addresses exposed for CipherTrust Manager network devices, and therefore fewer ways for potential attackers to reach CipherTrust Manager.
You can specify IPv4 or IPv6 when first adding a network connection, or when modifying a network connection.
Disabling IPv6 for a network interface does not disable the link local address. As traffic to and from link local addresses cannot be propagated through a router, an IPv6 link local address generally does not present security concerns with external attackers. If you still need to disable the link local address, contact customer support.
To Add a New Connection with IPv4 Only
In the
nmcli connection add
command, specifyipv4.method auto ipv6.method ignore
.nmcli conn add type ethernet con-name <connection-name> ifname <connection-interface-name> -- ipv4.method auto ipv6.method ignore
View the connection details to ensure IPv6 is disabled.
ipv6.method
is set toignore
for correct configuration. For example:nmcli con show <connection-name> connection.id: eth0 [...] ipv6.method: ignore [...] IP6.ADDRESS[1]: fe80::c9c:4cff:fe6a:1267/64
To Modify an Existing Connection to Disable IPv6
Modify the IPv6 configuration for the connection.
nmcli con modify <connection-name> ipv6.method ignore
Apply the configuration changes to the connection
nmcli con up <connection-name>
View the connection details to ensure IPv6 is disabled.
ipv6.method
is set toignore
for correct configuration.nmcli con show <connection-name> connection.id: eth0 [...] ipv6.method: ignore [...] IP6.ADDRESS[1]: fe80::c9c:4cff:fe6a:1267/64
Configuring Static Routes
To configure a static route:
Modify the NetworkManager connection to use a static route.
Run the command:
$ nmcli conn modify <connection name> +ipv4.routes "<static IP address> <gateway>"
Example:
$ nmcli conn modify ens3 +ipv4.routes "192.168.122.0/24 172.30.1.1"
If you wish to configure DNS servers manually, modify the NetworkManager connection to use specific DNS servers.
Run the commands:
$ nmcli conn modify <network device> ipv4.ignore-auto-dns yes $ nmcli conn modify <network device> ipv4.dns "<DNS servers>"
Example:
$ nmcli conn modify <network device> ipv4.dns "8.8.8.8, 4.4.4.4"
Apply the configuration change.
Run the command:
$ nmcli conn up <connection name>
Example:
$ nmcli conn up <connection name> Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/10)
Verify that the route is active.
$ ip route default via 172.30.1.1 dev ens3 proto static 172.30.0.128/26 dev kylo0 proto kernel scope link src 172.30.0.129 172.30.1.0/24 dev ens3 proto kernel scope link src 172.30.1.71 192.168.122.0/24 via 172.30.1.1 dev ens3 proto static
The output shows that the route is now active.
Bonding Network Interfaces
A bonded network interface provides redundancy and performance improvements by aggregating two or more network interfaces to form a single logical interface.
The official Linux kernel HOWTO documents this feature in detail.
The nmcli's reference documentation covers details specific to NetworkManager configuration.
Note
Bonding network interfaces is intended for physical appliances, for example, CipherTrust Manager k570. It is generally not relevant to virtual CipherTrust Manager instances, as bonding is not useful in a virtual environment.
Start by using
nmcli
to create a bonded connection for fail-over only (the most basic type of bond).$ nmcli conn add type bond con-name bond0 ifname bond0 -- bond.options mode=active-backup Connection 'bond0' (c6661f6b-5d68-4ffa-9ae8-63f0e4224cb4) successfully added. $ nmcli conn show bond0 bond.options: mode=active-backup
Alternatively, for certain networking environments such as in a virtual environment, the default setting of
fail_over_mac=0
might not be optimal and settingfail_over_mac=1
can be an option. Please refer to the Linux Kernel HOWTO for more information. The command to create the bonded connection would then be:$ nmcli conn add type bond con-name bond0 ifname bond0 -- bond.options mode=active-backup,fail_over_mac=1 Connection 'bond0' (c6661f6b-5d68-4ffa-9ae8-63f0e4224cb4) successfully added. $ nmcli conn show bond0 ... bond.options: mode=active-backup,fail_over_mac=1
Next, use
nmcli
to list devices and add some of them to the bond.$ nmcli device DEVICE TYPE STATE CONNECTION ... ens3 ethernet connected ens3 ens4 ethernet disconnected -- ...
Create the bond connections for
ens3
andens4
by specifyingbond0
as their master and bring them up. Notice thatens3
has an existing connection profile, so delete it first to ensure only the bond connection profile is used.$ nmcli conn delete ens3 Connection 'ens3' (f4e38090-6d3c-4612-b5b5-2d4c6a0c361d) successfully deleted. $ nmcli conn add type ethernet con-name ens3-bond0 ifname ens3 master bond0 Connection 'ens3-bond0' (3d87c66e-a128-4e4f-897a-b31cf1b5fdbf) successfully added. $ nmcli conn add type ethernet con-name ens4-bond0 ifname ens4 master bond0 Connection 'ens4-bond0' (b488d1f4-4a73-43ee-8a7f-9e992a850903) successfully added. $ nmcli conn up ens3-bond0 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/7) $ nmcli conn up ens4-bond0 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/8)
Next, use
nmcli
to bring up the bonded interface and confirm that it received an IP address.$ nmcli conn show bond0 ... connection.type: bond ... IP4.ADDRESS[1]: 10.121.105.113/22
Verify the bonded interface continues to receive and transmit data when either of the enslaved network interfaces are down (e.g. by removing the cable, disabling the port in the switch, etc.).
It is recommended to restart the OS to verify that the bond persist across reboots. If it does not then check if a connection profile exist for any of the interfaces that compose this bond and have an equal or higher connection priority. Either delete the duplicate connection profile or increase the priority of the bonded connections so it has precedence.
Configuring NIC Bonding with Static MAC Address
For CipherTrust Manager appliances, NIC bonding can result in alternating MAC address after reboot. If a static MAC address is preferred, it can be set on the virtual bonding network interface to ensure it never changes after reboot. The MAC address can, for example, be taken from the first NIC such as ens3
.
After the NIC bonding is setup, a static MAC address can be set with the following command:
$ nmcli connection modify bond0 802-3-ethernet.cloned-mac-address "00:0C:29:56:F8:A2"
Note
This feature should be used together with fail_over_mac=0
on the bonded network interface, the default setting.
Virtual LAN (VLAN)
Virtual Local Area Network (VLAN) is a mechanism to virtually isolate systems located on the same physical network. If you want to configure your CipherTrust Manager network adapters to use VLAN, we recommend having supporting network equipment and also configuring VLAN on the network layer.
VLAN can be configured with any VLAN ID and it is also possible to mix non-VLAN and multiple VLANs on the same network adapter. When mixing non-VLAN and multiple VLANs be careful to keep subnets and routing correct for the machine to be able to route traffic using the correct VLAN ID.
All examples are illustrated using a static IP address but DHCP can be used assuming there is a VLAN aware DHCP server on the network.
Switch a network adapter to VLAN only
This example disables any non-VLAN configuration on the selected network adapter and enables VLAN ID 10 with a static IP address. With this configuration, this machine can only talk to other machines using the same VLAN ID.
Delete any existing connections to make sure no stale configuration is left on the connections. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.
$ nmcli conn delete ens3.10
$ nmcli conn delete ens3
Target setup:
ens3
with no network configurationens3.10
with VLAN ID 10 and a static IP address192.168.1.2
Setup commands:
$ nmcli conn add type vlan con-name ens3.10 dev ens3 id 10 ip4 192.168.1.2/24
Any traffic to the subnet 192.168.1.0/24
uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.
Mixing non-VLAN and VLAN on same network adapter
This example enables both non-VLAN and VLAN on the same network adapter with different subnets. With this configuration, this machine can talk to other machines without VLAN on one subnet and with VLAN on another subnet.
Delete any existing connections to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.
$ nmcli conn delete ens3.10
$ nmcli conn delete ens3
Target setup:
ens3
with no VLAN and with static IP address192.168.1.2
ens3.10
with VLAN ID 10 and a static IP address192.168.2.2
Setup commands:
$ nmcli conn add type ethernet con-name ens3 ifname ens3 ip4 192.168.1.2/24
$ nmcli conn add type vlan con-name ens3.10 dev ens3 id 10 ip4 192.168.2.2/24
Any traffic to the subnet 192.168.1.0/24
does not use VLAN and any traffic to the subnet 192.168.2.0/24
uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.
Multiple VLANs on same network adapter
It is also possible to have multiple VLANs on the same network adapter. This example disables any non-VLAN configuration on the selected network adapter and enables VLAN ID 10 and VLAN ID 20 with static IP addresses with different subnets. With this configuration this machine can talk to two different VLANs using two different subnets.
Delete any existing connection to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.
$ nmcli conn delete ens3.10
$ nmcli conn delete ens3.20
$ nmcli conn delete ens3
Target setup:
ens3
with no network configurationens3.10
with VLAN ID 10 and a static IP address192.168.1.2
ens3.20
with VLAN ID 20 and a static IP address192.168.2.2
Setup commands:
$ nmcli conn add type vlan con-name ens3.10 dev ens3 id 10 ip4 192.168.1.2/24
$ nmcli conn add type vlan con-name ens3.20 dev ens3 id 20 ip4 192.168.2.2/24
Any traffic to the subnet 192.168.1.0/24
uses VLAN 10 and any traffic to the subnet 192.168.2.0/24
uses VLAN 20 unless there are other subnets and routes configured on this machine using the same subnet.
VLAN with bonding
VLAN can also be used together with bonding. The principle is very similar to the normal non-bonding case with the difference that the bonded network adapter is used as the device. Consult Bonding Network Interfaces for more information on setting up bonding.
Switch a network adapter to VLAN only
This example disables any non-VLAN configuration on the selected bonded network adapter and enables VLAN ID 10 with a static IP address. With this configuration this machine can only talk to other machines using the same VLAN ID.
Delete any existing connection to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.
$ nmcli conn delete bond0.10
$ nmcli conn delete ens3-bond0
$ nmcli conn delete ens4-bond0
$ nmcli conn delete bond0
Target setup:
bond0
with no network configurationbond0.10
with VLAN ID 10 and a static IP address192.168.1.2
Setup commands:
$ nmcli conn add type bond con-name bond0 ifname bond0 -- bond.options mode=active-backup ipv4.method disabled ipv6.method ignore
$ nmcli conn add type ethernet con-name ens3-bond0 ifname ens3 master bond0
$ nmcli conn add type ethernet con-name ens4-bond0 ifname ens4 master bond0
$ nmcli conn add type vlan con-name bond0.10 dev bond0 id 10 ip4 192.168.1.2/24
Any traffic to the subnet 192.168.1.0/24
uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.
Mixing non-VLAN and VLAN on same network adapter
This example enables both non-VLAN and VLAN on the same network adapter with different subnets. With this configuration this machine can talk to other machines without VLAN on one subnet and with VLAN on another subnet.
Delete any existing connection to make sure no stale configuration is left on the connection. If the connections are not already configured, the commands fail as expected with 'Error: unknown connection'.
$ nmcli conn delete bond0.10
$ nmcli conn delete ens3-bond0
$ nmcli conn delete ens4-bond0
$ nmcli conn delete bond0
Target setup:
bond0
with no VLAN and with static IP address192.168.1.2
bond0.10
with VLAN ID 10 and a static IP address192.168.2.2
Setup commands:
$ nmcli conn add type bond con-name bond0 ifname bond0 ip4 192.168.1.2/24 -- bond.options mode=active-backup
$ nmcli conn add type ethernet con-name ens3-bond0 ifname ens3 master bond0
$ nmcli conn add type ethernet con-name ens4-bond0 ifname ens4 master bond0
$ nmcli conn add type vlan con-name bond0.10 dev bond0 id 10 ip4 192.168.2.2/24
Any traffic to the subnet 192.168.1.0/24
does not use VLAN and any traffic to the subnet 192.168.2.0/24
uses VLAN 10 unless there are other subnets and routes configured on this machine using the same subnet.