[liberationtech] DHS Social Media Scrapes

[Carrie Budge]

Great, thank you for the help Andrew. 

I've learnt since writing earlier that the FBI has just put out a RFP for developers to create an app so that they can scrape social networks too. Delightful. 

 

//////////////////////
carrie budge

On 9 Feb 2012, at 17:05, Andrew Lewis <andrew at pdqvpn.com> wrote:

> Keyword searching in most cases.
> 
> Depending on the business unit, I'd be surprised if it was anything more then a horrendously written app or a commercial product with an obscene markup. 
> 
> 
> 
> Sent from my iPad
> 
> On Feb 9, 2012, at 12:34 PM, caroline budge <budge.caroline at gmail.com> wrote:
> 
>> Hello Lib Tech community! 
>> 
>> I was referred to the community by a professor in the CAST department at Goldsmiths University in London. I am currently on a MA Digital Journalist course there where I am investigating the recent incident where two british tourists got turned away from LAX because the DHS had detected a tweet that said they were going to 'destroy america' --which is british slang for "party in america'. 
>> 
>> The DHS has contracted a security company called General Dynamics to carry out a majority of their work scraping the internet. 
>> 
>> Does anyone in the community have a good idea at what sort of methods they are using to do so. I have a very basic understanding of programming (I've been scraping twitter using python recently), but I was wondering if the community had a better idea of how they are going about performing these scrapes. 
>> 
>> I hope that this is clear and that someone can help me write about the methods they are using to collate user data in real time. 
>> 
>> Thank you, 
>> 
>> Carrie 
>> 
>> 
>> _______________________________________________
>> liberationtech mailing list
>> liberationtech at lists.stanford.edu
>> 
>> Should you need to change your subscription options, please go to:
>> 
>> https://mailman.stanford.edu/mailman/listinfo/liberationtech
>> 
>> If you would like to receive a daily digest, click "yes" (once you click above) next to "would you like to receive list mail batched in a daily digest?"
>> 
>> You will need the user name and password you receive from the list moderator in monthly reminders.
>> 
>> Should you need immediate assistance, please contact the list moderator.
>> 
>> Please don't forget to follow us on http://twitter.com/#!/Liberationtech