Networking

Solaris 11 Single Interface

Report physical links and availability:

$ dladm show-phys | egrep "LINK|1000"

Create the network interface, VLAN and IP address,

$ ipadm create-ip net0
$ dladm create-vlan -l net0 -v 183 vlan183

$ ipadm create-addr -T static -a 192.168.183.50/24 net0/v4

To remove this configuration,


$ ipadm delete-addr net0/v4
$ dladm delete-vlan vlan183

$ ipadm delete-ip net0

Solaris 11 IPMP on 10Gbps

Identify the pair of 10Gpbs interfaces (E.g. Links net4 and net5):

$ dladm show-phys | egrep "LINK|1000"

Create the network interfaces, configure jumbo frames:

$ ipadm create-ip net4 ; ipadm create-ip net5
$ dladm set-linkprop -p mtu=9000 net4

$ dladm set-linkprop -p mtu=9000 net5

Add the VLANs:

$ dladm create-vlan -l net4 -v 183 vlan183
$ dladm create-vlan -l net5 -v 183 net183005

$ dladm show-link

Create the IPMP group using the first interface, assign our IP, add another interface:

$ ipadm create-ipmp -i net4 ipmp0

$ ipadm create-addr -T static -a 192.168.183.50/24 ipmp0/v4
$ ipadm add-ipmp -i net5 ipmp0

To remove this configuration, first remove IPMP then interfaces and VLANs:

$ ipadm delete-addr ipmp0/v4

$ ipadm remove-ipmp -i net4 -i net5 ipmp0
$ ipadm delete-ipmp ipmp0
$ ipadm delete-ip net4 ; ipadm delete-ip net5

$ dladm delete-vlan net183005 ; dladm delete-vlan vlan183

Solaris 11 Network Configuration Review

Review the configuration as follows, and confirm access using ping and SSH:

$ ipadm

$ dladm show-phys

$ dladm show-link

$ netstat -nrv

Solaris 10 IPMP

This is an example of configuring IPMP on a pair of interfaces for sysadmin, and another for production traffic.

Identify the interface that was enabled during the Solaris installation, and the others that are connected to switches.

$ ifconfig -a
$ for i in `dladm show-dev | grep "link: unknown" | awk '{print $1}'`; do
  ifconfig $i plumb
  done
$ for i in `dladm show-dev | grep down | awk '{print $1}'`; do
  ifconfig $i unplumb
  done
$ dladm show-dev | grep " up" | grep 1000
In my case, four interfaces are available, and I used "ifconfig" to determine that e1000g0 and e1000g7 are on the sysadmin VLAN, eg

$ ifconfig e1000g7 192.168.10.61 netmask 255.255.255.0 broadcast 192.168.10.255 up
$ ping 192.168.10.1 ; ifconfig e1000g7 down

Add the entries to /etc/hosts, for example,

192.168.10.61     myserver-mgt loghost
192.168.10.62     myserver-e1000g0
192.168.10.63     myserver-e1000g7
192.168.20.24     myserver
192.168.20.25     myserver-e1000g3
192.168.20.26     myserver-e1000g6

Create the IPMP sysadmin interface files, for example,

$ cat /etc/hostname.e1000g0
group mgt
set myserver-mgt netmask + broadcast + up
addif myserver-e1000g0 netmask + broadcast + deprecated -failover up
$ cat /etc/hostname.e1000g7
group mgt
set myserver-e1000g7 netmask + broadcast + deprecated -failover up
$ cat /etc/hostname.e1000g3
group prod
set myserver netmask + broadcast + up
addif myserver-e1000g3 netmask + broadcast + deprecated -failover up
$ cat /etc/hostname.e1000g6
group prod
set myserver-e1000g6 netmask + broadcast + deprecated -failover up

Add the default router, plus a persistent route for another subnet, and update netmasks,

$ echo "192.168.20.1" > /etc/defaultrouter
$ route -p add -net 192.168.50.0 -gateway 192.168.10.1
$ echo "192.168.10.0  255.255.255.0" > /etc/netmasks
$ echo "192.168.20.0  255.255.255.0" >> /etc/netmasks

Restart the network service, be sure to check netmask and broadcast

$ svcadm restart network/physical
$ svcadm refresh ssh
$ ifconfig -a

Verify all is well with ping and ssh. Test failover as follows (see /var/adm/messages):

$ if_mpadm -d igb0
$ if_mpadm -r igb0

LACP

Occassionally you have the need to aggregate a set of interfaces to accomodate a larger volume of inbound traffic. A backup and recovery host for example. This is how you aggregate a set of interfaces in Solaris:

Remove the unaggregated interface (if applies),

 ifconfig bge0 down && ifconfig bge0 unplumb
 netstat -rn
 mv /etc/hostname.bge0 /etc/hostname.bge0.ORIG 
Configure aggregated interface,
 
 dladm show-link
 dladm create-aggr -d bge0 1
 dladm add-aggr -d bge1 1
 dladm modify-aggr -l active -T short 1
 
Bring the interface online,
 
 dladm show-aggr -L
 uname -n > /etc/hostname.aggr1
 ifconfig aggr1 plumb 192.168.10.10 netmask 255.255.255.0 up
 
Review,
 
 dladm show-aggr
 dladm show-aggr -L
 dladm show-dev
 dladm show-dev -s
 dladm show-link
 dladm show-linkprop
 dladm show-link aggr1
 netstat -nr
 arp -a
 ping -s 192.162.10.10
 ping -s somehost

For 10Ge, enable jumbo frames,
 
 echo `uname -n` mtu 900 > /etc/hostname.aggr183001 
For systems with many vCPUs, add to /etc/system,
 
 set ip:ip_squeue_fanout=1
 set ip:tcp_squeue_wput=1 
For nxge interfaces, edit /platform/sun4v/kernel/drv/nxge.conf, setting "accept_jumbo = 1". Do a reconfiguration reboot. 

Test using iperf or something similar.