Hadoop

The talking heads and politicians have finally begun to catch on about Hadoop. In today's WSJ:

The NSA's advances have come in the form of programs developed on the West Coast—a central one was known by the quirky name Hadoop—that enable intelligence agencies to cheaply amplify computing power, U.S. and industry officials said. ... A computing and software revolution, launched in Silicon Valley a few years earlier, made sifting all that data easier. That was particularly true with the development of Hadoop, a piece of free software that lets users distribute big-data projects across hundreds or thousands of computers. ... The NSA also became an early adopter. At a 2009 conference on so-called cloud computing, an NSA official said the agency was developing a new system by linking its various databases and using Hadoop software to analyze them, according to comments reported by the trade publication InformationWeek. ... Mr. Garrett now runs RTRG's successor program, which was moved to the Defense Advanced Research Projects Agency and renamed Nexus 7. That effort has been using Hadoop and similar software to help manage large masses of data. One of the pieces of software, called Accumulo, was developed by the NSA using technology from Google, said a person briefed on the program.

Hadoop and its predecessor MapReduce have been around for about 10 years, It's always entertaining to see how "cutting edge" the main stream media and the politicians are.