Earlier this year, our lawsuit against the Central Intelligence Agency resulted in the release of 13 million pages of historical documents online - the CREST archives. That’s a lot to go through, so to help kick off CIA Week we put together a bot that tweets out documents created on this day in history: @TodayInCIA.
The titles range from the intriguing to the unintentionally hilarious, with a lot of (UNTITLED), (CLASSIFIED), and (SANITIZED) thrown in for good measure.
Some sample entries from MuckRock’s birthday, February 11:
- R&D PLANNING REPORT
- UNUSUALLY CONFIGURED ANTENNAS AT FACILITIES NEAR TALDOM AND NIKOLAYEV, USSR
- INTRODUCTION TO THE KRUG INSTALLATION SERIES
- TYPES OF LASERS
Check out the latest updates here:
For now, we’re tweeting out a new document every hour, on the hour, from nine to five Eastern time, but I’m working on tweaking the code to tweet 24/7 soon.
Speaking of code, the bot and underlying data are all open source. Right now, it’s a very hacky pile of CSVs that are uploaded manually, but I’m working on integrating it into a fork of Ben Nuttall’s Raspberry Pi OTD Twitter Bot, which he graciously allowed. See that work in progress here, or make a pull request if you have ideas and want to implement them.
Future planned features include automatic screenshots of the first page, making sure that it avoids tweeting very similar titles soon after one another, and other enhancements that make it easier to help highlight particularly interesting documents. Also, getting rid of me manually uploading CSVs.
Huge thanks goes to MuckRock contributor Emma Best and the awesome team at Data.World who helped wrangle all the data. You can dig more deeply into the CREST database metadata at Data.World, and we’ll have a few more fun looks at CREST later this week thanks to that effort.
Image via CIA Flickr
