Brief Guide to the Use of FDT

http://www.phenix.bnl.gov/phenix/WWW/simulation/fdtGuide.html

Contact Person: Charles F. Maguire
Creation Date: March 28, 2007

Introduction

This site provides a brief guide to the use of the Fast Data Transfer (FDT) software which is used for transferring files between one IP address and another. The main documentation for this software is at a CERN site http://monalisa.cern.ch/FDT/index.html. My intention here is simply to provide a quick introduction, using sample commands which have worked successfully for me. I have been using this software only since March 27, 2007. It was introduced to me by one of my high energy physics colleagues at Vanderbilt.

Obtaining the 1.6 Java Software Environment

The software uses Java classes, and one must have the latest version (1.6) of Java available. Java is not part of the rcas farm at RCF, but it is on rcf2 as an older 1.4 version. It is easy enough to have your own 1.6 version of Java. Here are the instructions for installing it in your own directory on the rcas farm at RCF. Of course, PHENIX and RCF should make a decision on whether they want to support this technology, such that everyone does not have their own set of Java binaries (260 MBytes) for this work.

  1. Create a working directory at RCF such as I did on /phenix/data11/maguire/fdt, except don't use the /phenix/data11/ disk.

  2. Copy into your working directory the three files fdt.csh, fdt.jar, and jdk-6-linux-i586.bin, where this last is a "self-extracting" executable from Sun for the 1.6 Java version.

  3. Execute the command ./jdk-6-linux-i586.bin which will initially produce a lot of Sun company warnings. At the end of this list, type yes that you agree to abide by the warnings.

  4. After you give the yes response, then the directory jdk1.6.0 should be made, and then populated with 15 files or subdirectories.

  5. Create the softlink jdk -> jdk1.6.0/ as I have done.

  6. Edit the file fdt.csh to replace the instances of /phenix/data11/maguire with whatever you have chosen. Of course, if there was already an existing version of 1.6 Java, you wouldn't have to do any of these steps.

  7. Do a source fdt.csh command so that your binary path will pick up this new version of Java. You should see an output like

    rcas2064 % source fdt.csh

    java version "1.6.0"
    Java(TM) SE Runtime Environment (build 1.6.0-b105)
    Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing)

  8. Repeat these steps on you local computer node if necessary, such that you have 1.6 Java working there too.
Of course, you could put the two setenv commands which you see in the fdt.csh file into your .cshrc start up c-shell file such that you don't have to give the command explicitly each time. (Notes to Vanderbilt group: I will put these commands in our own phnxVandy_setup.csh start-up shell on both VUPAC and ACCRE, such that you don't have do the work in steps 3 to 8 yourself on the Vanderbilt computers. You will need a copy of the fdt.jar file and you should look at the sample commands in the fdt.csh file.)

Sample Operation

Pushing data from client to server

As a first example, I show the transfer of one Run4 PRDF file from a disk area on /phenix/data11/maguire to a local disk on my desktop workstation at Vanderbilt. The first step is to give the source fdt.csh command both of the Java environment directories (rcas node and vpac23 node) which was set up previously. Of course if 1.6 Java is already present, then there is no need to give this source command.

The next step is to give a Java command which starts a server process running. In this example command (all commands in this document are given as commented lines in the fdt.csh file), the server runs on the vpac23 node, as follows

[maguirc@vpac23 fdt]$ java -jar fdt.jar -S -bs 4096K -limit 100M

The explanation of the options is

Of course you should look at the home documentation site for more explanations than what I am giving here.

After you give the server command, a server process should start listening for a client request. You should see something like:

FDT [ 0.5.4-200703061259 ] STARTED ...

Mar 28, 2007 5:36:26 PM lia.util.net.common.p 
INFO: FDT started in server mode
READY
Mar 28, 2007 5:36:27 PM lia.util.net.copy.FDTServer a
INFO: FDTServer start listening on port: 54321

Next go to the other machine, in my case the rcas2064 node. On this node, assuming that you already gave the source fdt.csh command, you would give the client command such as

rcas2064 % java -jar fdt.jar -c vpac23.phy.vanderbilt.edu
-P 10 -d /rhic1/maguire/fdt/
/phenix/data11/maguire/fdt/Run4PRDF/EVENTDATAxxx_P01-0000120845-0000.PRDFF

For purpose of better clarity in this document, I have broken the command into 3 lines, but the command would really be all on one line. The parts of this command are explained as follows:

After you give the above client command, you should see output on both the client and the server machines. On the client machine you will see something like: 

FDT [ 0.5.4-200703061259 ] STARTED ...

Mar 28, 2007 6:36:29 PM lia.util.net.common.p 
INFO: FDT started in client mode
Mar 28, 2007 6:36:29 PM lia.util.net.copy.transport.ControlChannel b
INFO: NEW CONTROL stream for 1376ec32-c8f5-46af-b96b-1dd8de07f37f initialized
Mar 28, 2007 6:36:31 PM lia.util.net.copy.h t
INFO:  Started DiskReaderTasks for the following partions [ 15 ] for FDTSession: 1376ec32-c8f5-46af-b96b-1dd8de07f37f
Mar 28, 2007 6:36:31 PM lia.util.net.copy.transport.a a
INFO: Requested window size -1. Using window size: 8192
 <-- [ Wed Mar 28 18:36:39 EDT 2007 ] Disk Read 7.984 MB/s Avg Disk Read Rate 7.984 MB/s -->
 <-- [ Wed Mar 28 18:36:44 EDT 2007 ] Disk Read 7.171 MB/s Avg Disk Read Rate 7.577 MB/s -->
 <-- [ Wed Mar 28 18:36:49 EDT 2007 ] Disk Read 8.765 MB/s Avg Disk Read Rate 7.973 MB/s -->
 <-- [ Wed Mar 28 18:36:54 EDT 2007 ] Disk Read 9.562 MB/s Avg Disk Read Rate 8.371 MB/s -->

....
There will be status outputs every 5 seconds showing how fast the transfer is proceeding.

On the server machine you should see the corresponding status messages, such as (note the one hour time difference between Nashville and Brookhaven): 

Mar 28, 2007 5:36:29 PM lia.util.net.copy.transport.ControlChannel b
INFO: NEW CONTROL stream for 1376ec32-c8f5-46af-b96b-1dd8de07f37f initialized
 --> [ Wed Mar 28 17:36:32 CDT 2007 ] Disk Write Rate 0 MB/s Avg Disk Write Rate 0 MB/s <--
Mar 28, 2007 5:36:34 PM lia.util.net.copy.disk.a run
INFO: DiskWriterTask [ 833 ] STARTED = true / lia.util.net.copy.disk.a@1bb60c3
 --> [ Wed Mar 28 17:36:37 CDT 2007 ] Disk Write Rate 7.994 MB/s Avg Disk Write Rate 3.998 MB/s <--
 --> [ Wed Mar 28 17:36:42 CDT 2007 ] Disk Write Rate 14.391 MB/s Avg Disk Write Rate 7.462 MB/s <--
 --> [ Wed Mar 28 17:36:47 CDT 2007 ] Disk Write Rate 7.197 MB/s Avg Disk Write Rate 7.396 MB/s <--
 --> [ Wed Mar 28 17:36:52 CDT 2007 ] Disk Write Rate 9.592 MB/s Avg Disk Write Rate 7.835 MB/s <--
 --> [ Wed Mar 28 17:36:57 CDT 2007 ] Disk Write Rate 9.594 MB/s Avg Disk Write Rate 8.128 MB/s <--
These status messages will continue on each machine until the transfer has completed.

Pulling data from server to client

The next example simply reverses the process. The source file is on the server side (vpac23) and it is copied to the client machine. On the server side you would repeat the same server command as before

[maguirc@vpac23 fdt]$ java -jar fdt.jar -S -bs 4096K -limit 100M

On the client machine you would give this command, again all on one line

rcas2064 % java -jar fdt.jar -c vpac23.phy.vanderbilt.edu -pull
-P 10 -d /phenix/data11/maguire/fdt/Run4PRDF/Run4PRDFCopy
/rhic1/maguire/fdt/EVENTDATAxxx_P01-0000120845-0000.PRDFF

Notice that there is an extra option -pull being specified here. The destination directory is changed to a copy subdirectory on /phenix/data11, while the source file on the server side is the file which was copied in the previous example.

Recursive operation for multiple file transfer from client machine to server machine

The last example shows how to copy all the files in a given directory, in this case from a directory on the client machine to a directory on the server machine. As before, start the server process first. Then on the client machine give the following command:

rcas2064 % java -jar fdt.jar -c vpac23.phy.vanderbilt.edu
-P 10 -d /rhic1/maguire/fdt/Run4PRDF
-r /phenixdata11/maguire/fdt/Run4PRDF

 The -r /phenix/data11/maguire/fdt/Run4PRDF option means copy all the files which are in this subdirectory. These files will be copied to the server machine in the destination directory.

More Command Line Options

You can find out the whole set of command line options by giving a help command java -jar fdt.jar -h . You will see that some of the options apply to both the server and the client machines, while other options apply to only one side.

A particular option which I prefer to use on the server side is

java -jar fdt.jar -S -bs 4096K -f rftpexp01.rhic.bnl.gov -limit 100M

The -f rftpexp01.rhic.bnl.gov means that the server will recognize only clients which are coming from the rftpexp01.rhic.bnl.gov machine. This will not work at RCF, I believe, since one is using one of the rcas nodes as the actual client, and not the rftpexp01.rhic.bnl.gov server itself.

Cron Jobs for Database Copying

It should be possible to set up daily cron jobs on the server and the client machines to send a copy of the database update file from RCF to the local site. I haven't tried that yet. For that matter, one could omit the -S option in the server command, so that the server was always ready to accept input. However, I think that then you would want to specify a given client node for security reasons.