Sneakernet

A USB flash drive is physically much smaller than a floppy disk, but it has a capacity thousands of times larger.

Memory cards are a popular physical medium for transferring files, and have become smaller in size as technology has advanced

Sneakernet is an informal term describing the transfer of electronic information, especially computer files, by physically moving removable media such as magnetic tape, floppy disks, compact discs, USB flash drives (thumb drives, USB stick) or external hard drives from one computer to another, usually in lieu of transmitting the information over a computer network. The term, a tongue-in-cheek play on Ethernet, refers to the use of someone wearing sneakers as the transport mechanism for the data.

Sneakernet is also used to refer to long ping times and the tradeoff between latency and bandwidth.

Summary and background

Sneakernets are in use throughout the computer world. Sneakernets may be used when computer networks are prohibitively expensive for the owner to maintain, in high-security environments where manual inspection (for re-classification of information) is necessary, where information needs to be shared between networks with different levels of security clearance, when data transfer is impractical due to bandwidth limitations, when a guest laptop is incompatible with the local network, or simply when two computers are not powered up at the same time or lack the correct interconnecting cabling. Because sneakernets take advantage of physical media, different security measures must be taken into account for the transfer of sensitive information.

This form of data transfer is also used for peer-to-peer (or friend-to-friend) file sharing and has grown in popularity in metropolitan areas and college communities, sometimes for the purpose of distributing works of authorship. The ease of this system has been facilitated by the availability of USB external hard drives, USB flash drives and portable music players.

The United States Postal Service also offers a Media Mail service for compact discs among other items. This provides a viable mode of transport for long distance sneakernet use. In fact, when mailing a sufficiently large hard drive or a spindle of DVDs, the throughput (amount of data per unit time) may compete favorably with other methods of data transfer.

Theory

From an information theory standpoint, sneakernets can achieve tremendous throughput, but they suffer from high latency (see Comparison of latency and throughput). The throughput of the network is directly proportional to the size of the transmitted file(s). Latency is based on the amount of time it takes to fully process the request for information. Latency would include the time it takes to write the storage media and the time to travel from point A to point B.

For example: Alice requests Bob to send her a DVD (4.7 GB) worth of information. Over the Internet, the latency for the file request may be milliseconds, but at a modest broadband download speed of 50 kB/s it may take up to a day to complete the transfer. On the other hand, Bob could burn a DVD and deliver it to Alice in an hour. The latency was an hour, but the throughput of the transfer is roughly equal to a transfer rate of 1305 kB/s.

The theoretical capacity of an Airbus A380 filled with microSD cards is 91,000,000 Terabytes, resulting in a 5.284 PB/s flight from New York to Los Angeles.

Similarly, as of 2010 the highest capacity backup tape format available is LTO-5, with a capacity of 1500 GB. If a tape of this capacity were sent by overnight mail and were to arrive around 20 hours after it was sent, the effective data rate would be 166 Mbit/s. With networking technology, this magnitude of speed over this distance would be very difficult to attain without a costly dedicated connection as one would likely need to use several hops and have a connection that is not oversubscribed.

Sneakernets may also be used in tandem with computer network data transfer to increase data security. For example, a file or collection of files may be encrypted and sent over the Internet while the encryption key is printed and hand delivered or mailed. This method greatly reduces the possibility of an individual intercepting both the key and encrypted data.

Another way sneakernets are used together with network transfers is to provide an initial full backup as a base for incremental backups. In the case of a large (several terabyte) dataset that grows by just a few megabytes a day, the initial seeding of the data to be backed up would require an excessively long time upload over a network. One solution is to make a local copy of the data which is then physically relocated to the remote location and topped up with daily incremental backups over a network.

Usage examples

The May 2011 raid of Osama Bin Laden's compound revealed that he used a series of USB thumb drives to store his email drafts. A courier of his would then take the saved emails to a nearby Internet cafe and send them out to the desired recipients.

In September 2009, Durban company Unlimited IT reportedly pitted a carrier pigeon against South African ISP Telkom to transfer 4 GB of data 60 miles (97 km) from Howick to Durban. The pigeon, carrying the data on a memory stick, arrived in one hour eight minutes, with the data taking another hour to read from the memory stick. During the same two-hour period, only about 4.2% of the data had been transferred over the ADSL link. A similar experiment was conducted in England in September 2010; the "pigeonnet" also proved superior. In November 2009 the Australian comedy/current-affairs television program Hungry Beast repeated this experiment. The experiment had the team transfer a 700 MB file via three delivery methods to determine which was the fastest; A carrier pigeon with a microSD card, a car carrying a USB Stick, or a Telstra (Australia's largest telecom provider) ADSL line. The data was to be transferred a distance of 132 km by road. The pigeon won the race with a time of approximately 1 hour 5 minutes, the car came in second at 2 hours 10 minutes, while the internet transfer did not finish, having dropped out a second time and not coming back.

Google has used a sneakernet to transport large datasets, such as the 120 TB of data from the Hubble Space Telescope.

The SETI@home project uses a sneakernet to overcome bandwidth limitations: data recorded by the radio telescope in Arecibo, Puerto Rico is stored on magnetic tapes which are then shipped to Berkeley, California for processing. In 2005, Jim Gray reported sending hard drives and even "metal boxes with processors" to transport large amounts of data by postal mail.

Many film editing and visual effects companies transfer large film-scans using hard drives shipped via courier (to reduce bandwidth bills, and to reserve bandwidth for more time-critical transfers).

Wizzy Digital Courier provides Internet access to schools with poor or no network connectivity by implementing UUCP on USB memory sticks. This allows offline cached email transport and scoops of web pages that back-fill a web cache.

When home broadband access was less common, many people downloaded large files over their workplace networks and took them home by sneakernet. Today, as home broadband is more common, sometimes technical workers at institutions with congested WAN links do the reverse: downloading data at home in the evening and carrying the files to work on USB flash drives.

In the early demoscene, the primary method of exchanging data was using snail mail to exchange floppies between groups. Each group usually had at least one person designated as a swapper, who would exchange news, data and productions with swappers from other groups this way. The best swappers were known to send and receive over 100 mails a month.

Online DVD rental services such as GameFly, or those formerly provided by Netflix, are effectively sneakernets. They deliver data, stored on DVDs and occasionally other forms of digital media, via regular postal mail.

Petroleum seismic surveys routinely record field data which are many terabytes in size. A cluster computer is required to process these data, and may take a year or more, during which time the field crew will wish to work in other areas. The field data are generally hand-carried on tape, and increasingly on hard disk inserts, to the processing centre.

Data analytics teams in the financial services sector often use sneakernets to transfer sensitive corporate information and information obtained from data mining, such as ledger entries, customer data, and financial statistics. There are several reasons for this: firstly, sneakernets can generally provide very high security (and possibly more importantly, they are perceived to be secure) due to the impossibility of a MITM Attack or packet sniffing, even if the receiving or sending computational device is compromised via malware; secondly, the volumes of data concerned are often extremely high; and thirdly, setting up secure network links between the client business and the analytics team's facilities is often either impossible or an extremely convoluted process.

Very Long Baseline Interferometry performed using the Very Long Baseline Array ships hard drives to a data reduction site in Socorro, New Mexico. They refer to their data transfer mechanism as "HDOA" (Hard Drives On Airplane).

The Rigsum Sherig Collection project uses a sneakernet to distribute offline educational resources, including Kiwix and Khan Academy on a Stick, to hundreds of schools and other educational institutional in the Kingdom of Bhutan. Many of the schools in Bhutan have computers or IT labs, but no Internet connection (or a very slow one). The sneakernet, facilitated by teachers, distributes about 25 GB of free, open-source educational software to the schools, often using external hard disks.

North Korean dissidents have been known to smuggle flash drive filled with western movies and television shows, largely in an effort to inspire a cultural revolution.

El Paquete Semanal is a roughly 1TB compilation of media, distributed weekly throughout Cuba via portable hard drives and USB memory sticks.

In 2015 Amazon Web Services launched Amazon Snowball, a 50 lb (23 kg), 50 TB device for transporting data to the AWS cloud.

In media

Non-fiction

Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

—

The original version of this quotation came much earlier; the very first problem in Tanenbaum's 1981 textbook Computer Networks asks the student to calculate the throughput of a St. Bernard carrying floppy disks (in 1981 the standard 8 inch floppy held 241 kilobytes of data per side, though various formatting and data blocking techniques could get that as high as 800 kilobytes per side, while the 5.25 inch floppy held 180 kilobytes per side). The first USENET citation is July 16, 1985, and it was widely considered an old joke already, possibly dating from the 1970s. Other alleged speakers included Tom Reidel, Warren Jackson, or Bob Sutterfield. The station wagon transporting magnetic tapes is the canonical version, but variants using trucks or Boeing 747s or C-5s and later storage technologies such as CD-ROMs, DVDs, Blu-Rays, or SD Cards have frequently appeared.

Fiction

The Terry Pratchett novel Going Postal includes a contest between a horse-drawn mail coach and the "Grand Trunk Clacks" (a semaphore line) to see which is faster to transmit the contents of a book to a remote destination.

The "valuable data file" has become a common MacGuffin in action films and television programs, and indeed the motif of the "valuable letter or documents" (pre-electronic information storage technology) dates back hundreds of years.

• The film Live Free or Die Hard depicts a digital thief attempting to download 500 TB of financial data to (a) suitcase sized package(s).

• The 1995 film Johnny Mnemonic, based on the short story by William Gibson, stars Keanu Reeves as a digital courier with 320 GB of corporate data transported in his head. Gibson's 2007 novel Spook Country also featured sneakernets, with iPods being the storage device used to clandestinely move information.

• In the episode "Amen" of The Newsroom, associate producer Maggie Jordan, after being told "it's 2011, we don't run film manually", is told that the video is taking too long to render on the local machine. It cannot feasibly be transferred over the network in raw form, and must be run to the newsroom manually on a USB drive.

• In the James Bond film Skyfall, MI6 sets up a sneakernet due to security concerns.

Similar concepts

• Delay-tolerant networks, such as the Haggle project at Cambridge University
• IP over Avian Carriers (RFC 1149), an April Fools' Day RFC describing the transmission of messages via homing pigeon
• Drew Houston, the inventor of the Dropbox cloud storage and file synchronization system once referred to his invention as "sneakernet minus the removable media".

See also

• Darknet
• Data Mule
• Jargon File
• Meatspace (tongue-in-cheek implication of the real world as a similar analogue to cyberspace writ large)
• Pod slurping
• Repurposing
• USB dead drop
• Sideloading

External links

• "FedEx Bandwidth" from xkcd
• "Don't Copy That Floppy" in Internet Archive
• Entry for "sneakernet" in the Jargon File
• "Buzzwords Galore and Bandwidth that May Rival Your Stationwagon"