DataTransportation

UserPreferences

Proposed system for observer data transportation

Large amounts of data from array detectors need to be moved from the observatory to home institutions. We want these data to be write-verified on reliable media with a long shelf life. The Internet has insufficient bandwidth, tapes are slow and error-prone, and CDs have insufficient data capacity for large imaging arrays. We propose an upgraded data handling system for high capacity transportable storage media.

Requirements

• Easy to use
• Readable by Red Hat Linux Enterprise OS (for OCIW)
• Media smaller than a laptop computer
• Media no more fragile than a laptop computer
• Save a night's data in less than 15 minutes

Possibilities

• DVD disks. These are inexpensive and can be read by any operating system. Write times are under 1/2 hour, which is outside the 15 minute goal but might be OK. Capacity is about 4.2 GB, which is well below what IMACS is capable of producing in a single night (about 20GB).
• DAT tapes. Moderate cost, moderate capacity, slow-to-moderate speed. Available now.
• Removable hard drives. High capacity but more fragile than DVD or CD solutions. USB, Firewire, direct IDE or serial ATA interfaces available.
• NAS (networked attached storage). This is a turnkey version of removable hard disk backup.

Discussion

DVDs would be sufficient for most observing runs using the current instrumentation. DVDs are convenient, durable, and inexpensive. We propose to install a DVD writer on the two observer's computers in the Magellan control rooms.

Higher capacity and speed are needed for high-data-volume instruments. For this, hard drive technology seems to be the best option. One can buy hard drives with USB and Firewire connections and I know that at least USB would work on the observer's workstation (I don't know if they have Firewire ports). USB 1.0 is slow so the computers should have USB 2.0 (does it?).

A better option would be NAS storage, suggested by David Osip. The traditional NAS configuration is a hot-swappable disk farm used for creating off-site backups of large data sets. David's suggestion prompted me to look again and I found a new product, a packaged portable drive with an ethernet interface (http://www.ximeta.com/products/netdisk.html).

This device plugs into a CAT5 network port and appears as a network disk. We'll probably have to activate Samba on the observer's Linux workstation, if it isn't already running and reformat the drive to FAT32 (it comes as NTFS). Disadvantages: at 2-3 lbs it's heavy (although some of that is in the power brick, which need not travel), and Windows XP cannot create FAT32 partitions bigger than 32 GB (although XP will read larger partitions if you can create them). The 32 GB limit is probably only a nuisance.