Spying 2.0: The Open Source Paradigm

If analysts and agents were encouraged to post personal blogs and wikis on Intelink -- linking to their favorite analyst reports or the news bulletins they considered important -- then mob intelligence would take over. In the traditional cold-war spy bureaucracy, an analyst's report lived or died by the whims of the hierarchy. If he was in the right place on the totem pole, his report on Soviet missiles could be pushed up higher; if a supervisor chose to ignore it, the report essentially vanished. Blogs and wikis, in contrast, work democratically. Pieces of intel would receive attention merely because other analysts found them interesting. This grass-roots process, (Calvin Andrus, chief technology officer of the Center for Mission Innovation at the C.I.A.) argued, suited the modern intelligence challenge of sifting through thousands of disparate clues: if a fact or observation struck a chord with enough analysts, it would snowball into popularity, no matter what their supervisors thought. ...

(T)he best Internet search engines, including Google, all use "link analysis" to measure the authority of documents. When you type the search "Afghanistan" into Google, it finds every page that includes that word. Then it ranks the pages in part by how many links point to the page -- based on the idea that if many bloggers and sites have linked to a page, it must be more useful than others.

This, Burton (former D.I.A. analyst) pointed out, is precisely the problem with Intelink. It has no links, no social information to help sort out which intel is significant and which isn't. When an analyst's report is posted online, it does not include links to other reports, even ones it cites. There's no easy way for agents to link to a report or post a comment about it. Searching Intelink thus resembles searching the Internet before blogs and Google came along -- a lot of disconnected information, hard to sort through. If spies were encouraged to blog on Intelink, Burton reasoned, their profuse linking could mend that situation.

"Imagine having tools that could spot emerging patterns for you and guide you to documents that might be the missing pieces of evidence you're looking for," Burton wrote in his Galileo paper. ...

(T)op-secret information is becoming less useful than it used to be. "The intelligence business was initially, if not inherently, about secrets -- running risks and expending a lot of money to acquire secrets," (Thomas Fingar, the head of analysis for the D.N.I.) said, with the idea that "if you limit how many people see it, it will be more secure, and you will be able to get more of it. But that's now appropriate for a small and shrinking percentage of information." The time is past for analysts to act like "monastic scholars in a cave someplace," he added, laboring for weeks or months in isolation to produce a report.

Fingar says that more value can be generated by analysts sharing bits of "open source" information -- the nonclassified material in the broad world, like foreign newspapers, newsletters and blogs.