Remote access: Speed tests
Doing remote data collection and transferring images to your home
institution obviously requires fast network connection. But how fast is
the 'fast' and is my connection speed sufficient? The tests below may
help you to benchmark your connection and compare it against the
connection from the University of Michigan that we consider just about
The first test we recommend to run is connecting to the ANL Network
Diagnostic Tools (NDT). NDT is a web-based server that can be accessed
either using a web browser or using a web100clt command-line tool
(also known as "ndt-client"). The later can be downloaded from
Argonne offers a number of servers to diagnose the effect of the laboratory
|| -- behind the ANL firewall, 1Gbps
|These NDT speed servers can be used with web100clt if it is installed on your computer:
||web100clt -n prfsnr.aps.anl.gov
|| -- behind the ANL firewall & the APS divisional firewall, 10Gbps
||web100clt -n 22.214.171.124
|| -- Residing on the GMCA ScienceDMZ subnet, 10Gbps
Please note that web browser tests may not work with all web browsers
and in some case may require installing Java Run Time (JRE) browser
plugin. Navigate to one of the above links and start the test. After
the test is complete, click on Statistics.
Here are typical results for the University of Michigan collected back
in 2012. You should expect your speed being not lower than that for
successful remote data collection and transfering
WEB100 Enabled Statistics:
Checking for Middleboxes . . . . . . . . . . . . . . . . . . Done
checking for firewalls . . . . . . . . . . . . . . . . . . . Done
running 10s outbound test (client-to-server [C2S]) . . . . . 46.09Mb/s
running 10s inbound test (server-to-client [S2C]) . . . . . . 50.80Mb/s
------ Web100 Detailed Analysis ------
10 Gbps 10 GigEthernet/OC-192 link found.
Link set to Full Duplex mode
No network congestion discovered.
Good network cable(s) found
Normal duplex operation found.
Web100 reports the Round trip time = 20.06 msec; the Packet size = 1448 Bytes; and
No packet loss was observed.
S2C throughput test: Packet queuing detected: 0.03%
This connection is receiver limited 50.51% of the time.
Increasing the client's receive buffer (128.0 KB) will improve performance
This connection is sender limited 48.73% of the time.
Increasing the NDT server's send buffer (127.0 KB) will improve performance
This connection is network limited 0.77% of the time.
If your numbers are significantly lower, that may become a show stopper for
remote access. It is important to understand where the problem may occur. Such
task is not trivial because in most cases there is no a single bottleneck, but
the delays add up non-linearly. In other words, you may have a descent speed
between A and B and between B and C, but slow between A and C in spite of the
fact that the connection between A and C may be routed via B. Still, it is a
good idea to check the speed with some other NDT servers. Here are a few:
In case you cannot perform the NDT tests, they may be substituted by a
simpler Traceroute test (on Windows system you need to use tracert.exe).
Again, in order to be able to successfully collect data remotely and
sft/scp the images to your institution in real time, the times should
not be considerably longer than in the test below:
[sstepanov1@watson ~]$ traceroute ndt.anl.gov
traceroute to ndt.anl.gov (126.96.36.199), 30 hops max, 38 byte packets
1 188.8.131.52 (184.108.40.206) 1.154 ms 1.199 ms 1.221 ms
2 d-lifsci1-seb.r-seb.umnet.umich.edu (220.127.116.11) 0.666 ms 0.268 ms 0.253 ms
3 l3-bseb-rseb.r-bin-seb.umnet.umich.edu (18.104.22.168) 0.262 ms 0.244 ms 0.238 ms
4 v-bin-seb-inet-aa2.merit-aa2.umnet.umich.edu (22.214.171.124) 0.621 ms 0.705 ms 0.533 ms
5 tenge0-0-0-0x76.nw-chi1.mich.net (126.96.36.199) 7.970 ms 7.804 ms 7.763 ms
6 188.8.131.52 (184.108.40.206) 18.450 ms 18.527 ms 18.483 ms
7 sdn-sdsc-anl.anchor.anl.gov (220.127.116.11) 19.425 ms 19.620 ms 19.264 ms
8 fwzsw-banana-gwv685.anchor.anl.gov (18.104.22.168) 19.311 ms 19.308 ms 19.268 ms
9 * * *
Note that in some organizations traceroute is blocked by the firewall.
The test below is to compare downloading speeds of the same 88MB file
from two different ANL servers. Server-1 resides the ANL public subnet and
Server-2 on the GM/CA subnet that is behind an additional APS firewall. The
rationale for this test is that we have seen cases when the speed of outside
connection to the public ANL servers was fast while the APS firewall was overloaded
and then the connection to the GM/CA computers was much slower. Below are typical
results for the University of Michigan (you can also use a web browser instead
17:52:44 (5.39 MB/s) ....
16:46:14 (1.49 MB/s) ...
A part of the difference in speeds between mirror.anl.gov and www.gmca.anl.gov
is because the later is not a partcularly fast computer, but the APS firewall may
have its share in the lower speed too. Therefore, it is important to run the NDT
servers with different ANL servers.
Finally, if you already have an SSH access to the GM/CA computers, you
may test the real data transfer speeds. Make a directory, e.g. TEST with twenty
or so data frames produced by the detector and copy them to your institution.
Below are typical speeds for the University of Michigan obtained with 32MB
Rayonix-300 CCD frames:
[sstepanov1@watson ~]$ scp -rp sstepanov1@bl3ws5:TEST .
BS_test_0.0001 100% 32MB 5.3MB/s 00:06
BS_test_0.0002 100% 32MB 5.3MB/s 00:06
BS_test_0.0003 100% 32MB 5.3MB/s 00:06
BS_test_0.0004 100% 32MB 5.3MB/s 00:06
BS_test_0.0005 100% 32MB 5.3MB/s 00:06
BS_test_0.0006 100% 32MB 5.3MB/s 00:06
BS_test_0.0007 100% 32MB 6.4MB/s 00:05
BS_test_0.0008 100% 32MB 6.4MB/s 00:05
BS_test_0.0009 100% 32MB 6.4MB/s 00:05
BS_test_0.0010 100% 32MB 5.3MB/s 00:06
BS_test_0.0011 100% 32MB 6.4MB/s 00:05
BS_test_0.0012 100% 32MB 5.3MB/s 00:06
BS_test_0.0013 100% 32MB 5.3MB/s 00:06
BS_test_0.0014 100% 32MB 6.4MB/s 00:05
BS_test_0.0015 100% 32MB 5.3MB/s 00:06
BS_test_0.0016 100% 32MB 5.3MB/s 00:06
BS_test_0.0017 100% 32MB 6.4MB/s 00:05
BS_test_0.0018 100% 32MB 6.4MB/s 00:05
BS_test_0.0019 100% 32MB 6.4MB/s 00:05
BS_test_0.0020 100% 32MB 5.3MB/s 00:06
To estimate required speeds, one should consider that when data
collection is running permanently, the Rayonix frames are typically
acquired at the rate of 22fps (frames per minute), which corresponds to
32MB x 22/60 = 11.73MB/s. With Pilatus3-6m and Eiger-16m typical rates
are much higher, 50fps (frames per second), which corresponds to 6MB x
50 = 300MB/s and 18MB x 50 = 900MB/s respectively. However, given
typical overhead of crystal mounting, centering and screening, average
daily data rates are ~200GB/day with Rayonix-300, ~1000GB/day with
Pilatus3-6m and ~2000GB/day with Eiger-16m. This amounts to required
average speeds of 2MB/s, 10MB/s and 20MB/s with Rayonix, Pilatus and
Eiger respectively. In addition, please note that the Eiger detector
produces data in the HDF5 format, which are then automatically
converted to CBF. If you intent to transfer both formats, expected
amount of data and the required speeds for Eiger need to be doubled.
Also, with some experiments the amounts of data may be up to several
times higher than average. We strongly recommend using
Globus for transferring Eiger
and Pilatus data.
Improving the speeds
It may be possible to improve network speeds by tuning the TCP
parameters on your computer, although the administrative access to your
computer and some computer administrating skills may be required.
Please read the fasterdata.es.net guide.