[liberationtech] DHS Social Media Scrapes

[Andrew Lewis]

Keyword searching in most cases.

Depending on the business unit, I'd be surprised if it was anything more then a horrendously written app or a commercial product with an obscene markup. 



Sent from my iPad

On Feb 9, 2012, at 12:34 PM, caroline budge <budge.caroline at gmail.com> wrote:

> Hello Lib Tech community! 
> 
> I was referred to the community by a professor in the CAST department at Goldsmiths University in London. I am currently on a MA Digital Journalist course there where I am investigating the recent incident where two british tourists got turned away from LAX because the DHS had detected a tweet that said they were going to 'destroy america' --which is british slang for "party in america'. 
> 
> The DHS has contracted a security company called General Dynamics to carry out a majority of their work scraping the internet. 
> 
> Does anyone in the community have a good idea at what sort of methods they are using to do so. I have a very basic understanding of programming (I've been scraping twitter using python recently), but I was wondering if the community had a better idea of how they are going about performing these scrapes. 
> 
> I hope that this is clear and that someone can help me write about the methods they are using to collate user data in real time. 
> 
> Thank you, 
> 
> Carrie 
> 
> 
> _______________________________________________
> liberationtech mailing list
> liberationtech at lists.stanford.edu
> 
> Should you need to change your subscription options, please go to:
> 
> https://mailman.stanford.edu/mailman/listinfo/liberationtech
> 
> If you would like to receive a daily digest, click "yes" (once you click above) next to "would you like to receive list mail batched in a daily digest?"
> 
> You will need the user name and password you receive from the list moderator in monthly reminders.
> 
> Should you need immediate assistance, please contact the list moderator.
> 
> Please don't forget to follow us on http://twitter.com/#!/Liberationtech