IP Source

Sunday, 5 February 2012

Twitter archive goes to Library of Congress

(Ed. I'm a bit tardy on this, the actual announcement came December 7th of 2011. But I wanted to see what fallout would inevitably happen as all those 'tweets' became public domain. I keep reminding people that anything that they place on the Internet IS public domain, whether you intended it to or not. If you want real security, you will have to use a specific technology designed for the purpose you intend, i.e., PKI, AES, TOR, PGP, VPN, etc. I also emphasize to users that when they write an email, keep in mind that the world has a copy....so write your words CAREFULLY as if they could be displayed on roadside billboard (actually this HAS happened a couple of time in the US, much to the consternation of both writer and recipient, not only embarassment but job loss as well). Needless to say, my 'Tweets' are receive ONLY).
 
The Announcement....
 
The Library of Congress and Twitter have signed an agreement that will see an archive of every public Tweet ever sent handed over to the library's repository of historical documents.
 
"We have an agreement with Twitter where they have a bunch of servers with their historic archive of tweets, everything that was sent out and declared to be public," said Bill Lefurgy, the digital initiatives program manager at the library's national digital information infrastructure and preservation program. The archives don't contain tweets that users have protected, but everything else — billions and billions of tweets — are there.
 
Using new technical processes it has developed, Twitter is moving a large quantity of electronic data from one electronic source to another. "They've had to do some pretty nifty experimentation and invention to develop the tools and a process to be able to move all of that data over to us," Lefurgy said. The Library of Congress has long been the repository of important, historical documents and the Twitter library, as a whole, is something historic in itself.
 
"We were excited to be involved with acquiring the Twitter archives because it's a unique record of our time," Lefurgy said. "It's also a unique way of communication. It's not so much that people are going to be interested in what you or I had for lunch, which some people like to say on Twitter." Researchers will be able to look at the Twitter archive as a complete set of data, which they could then data-mine for interesting information. "There have been studies involved with what are the moods of the public at various times of the day in reaction to certain kinds of news events," Lefurgy said. "There's all these interesting kinds of mixing and matching that can be done using the tweets as a big set of data."
 
One benefit for the Library of Congress in receiving this large data set is that it's been forced to stretch itself technologically. "It's been difficult at times," Lefurgy said. "But we firmly believe that we have to do this kind of thing because we anticipate that we'll be bringing in large data sets again into the future. We don't know specifically what, but certainly there's no sign of data getting smaller or less complicated or less interesting."
 
The library's Twitter partnership comes amid a renewed push by the Obama administration and the National Archives and Records Administration for federal agencies to better archive their own social media postings and emails as potential government records.