Results

Supermicro P3TDL3

Netperf Test

The following table indicates the Netperf results:

test	UDP Mbits/sec	TCP Mbits/sec
Single test on single interface	95.70	94.13
Two tests on single Interface	47.86/47.86	46.99/47.15
Two tests on two interfaces	95.70/95.70	94.13/94.14

Netpipe Test

The following graph shows three sets of results. A single interface running the Netpipe benchmark (i.e. test 1 above). The same interface running two Netpipe benchmarks to two different nodes. (i.e. test 2 above). The vertical line at the end of the graph is the jump back to full bandwidth as one of the shared tests finished first.

Netpipe results for single and shared interface.

The following graph is Netpipe run on two separate interfaces connected with two separate receivers (e.g. test 3 above).

Netpipe results for two separate interfaces

Tyan 2762

Netperf Test

The following table indicates the Netperf results (* = invalid result, program seemed to have some math error when computing rate):

test	UDP Mbits/sec	TCP Mbits/sec
Single test on single interface	95.72	94.15
Two tests on single Interface	*	47.26/46.89
Two tests on two interfaces	95.73/95.73	94.16/94.16

Netpipe Test

The following graph shows three sets of results. A single interface running the Netpipe benchmark (i.e. test 1 above). The same interface running two Netpipe benchmarks to two different nodes. (i.e. test 2 above). The vertical line at the end of the graph is the jump back to full bandwidth as one of the shared tests finished first.

Netpipe results for single and shared interface.

The following graph is Netpipe run on two separate interfaces connected with two separate receivers (e.g. test 3 above).

Netpipe results for two separate interfaces

Intel SE7500WV2

Netperf Test

The following table indicates the Netperf results:

test	UDP Mbits/sec	TCP Mbits/sec
Single test on single interface	95.72	94.15
Two tests on single Interface	47.87/47.86	47.07/47.07
Two tests on two interfaces	95.70/95.71	94.15/94.12

Netpipe Test

The following graph shows three sets of results. A single interface running the Netpipe benchmark (single Interface). The same interface running two Netpipe benchmarks to two different nodes. (i.e. there were three nodes in the test.) The vertical line at the end of the graph is the jump back to full bandwidth as one of the shared tests finished first.

Netpipe results for single and shared interface.

The following graph is Netpipe run on two separate interfaces connected with two separate receivers (e.g. test 3 above).

Netpipe results for two separate interfaces

The Effect of Two Interfaces on MPI jobs

As many motherboards have at lest two 100/100 Ethernet interfaces, it is interesting to see the effect of using both of these interfaces as separate networks for an MPI job. This test is not channel bonding were two interfaces are combined to form a single interface. To tun these tests, a small cluster with four dual processor nodes (PIII-800) and dual Ethernet (EEPro100) was used. The systems was configured to have two separate networks (192.168.0.0, 192.168.1.0). The /etc/hosts file was altered to show the same systems but though a different net. For instance:

norfolk1 192.168.0.1

norfolk2 192.168.0.2

norfolk3 192.168.0.3

norfolk4 192.168.0.4

norfolk1a 192.168.1.1

norfolk2a 192.168.1.2

norfolk3a 192.168.1.3

norfolk4a 192.168.1.4

The machines file uses for MPICH where as follows. For the a four CPU test (single interface):

norfolk1

norfolk2

norfolk3

norfolk4

For an eight CPU test (single interface):

norfolk1

norfolk2

norfolk3

norfolk4

For an eight CPU test (dual interfaces)

norfolk1

norfolk2

norfolk3

norfolk4

norfolk1a

norfolk2a

norfolk3a

norfolk4a

Running this test using LAM is a bit more difficult as multiple LAM daemons must be run on the same node. The total Mop/s for each are given below. (* = test did not run because it require square number of CPUs)

test	1 CPU	4 CPUs	8 CPUs Shared Interface	8 CPUs Dual Interfaces
BT	77.66	270.55	*	*
CG	54.80	116.62	80.91	132.94
EP	2.93	11.52	21.88	21.95
FT	124.49	111.53	136.37	152.39
IS	7.77	4.01	4.85	5.27
LU	100.20	358.64	399.63	422.81
SP	71.22	210.49	287.07	306.74
MG	51.96	158.93	*	*

The speed-up are given below (speedup=parallel Mops/sequential Mops) is provide below.

test	1 CPU	4 CPUs	8 CPUs Shared Interface	8 CPUs Dual Interfaces
BT	1	3.48	*	*
CG	1	2.13	1.48	2.43
EP	1	3.93	7.47	7.49
FT	1	.9	1.10	1.22
IS	1	.52	.62	.68
LU	1	3.58	3.99	4.22
SP	1	2.96	4.03	4.31
MG	1	3.06	*	*