[liberationtech] DHS Social Media Scrapes

[caroline budge]

Hello Lib Tech community! 

I was referred to the community by a professor in the CAST department at Goldsmiths University in London. I am currently on a MA Digital Journalist course there where I am investigating the recent incident where two british tourists got turned away from LAX because the DHS had detected a tweet that said they were going to 'destroy america' --which is british slang for "party in america'. 

The DHS has contracted a security company called General Dynamics to carry out a majority of their work scraping the internet. 

Does anyone in the community have a good idea at what sort of methods they are using to do so. I have a very basic understanding of programming (I've been scraping twitter using python recently), but I was wondering if the community had a better idea of how they are going about performing these scrapes. 

I hope that this is clear and that someone can help me write about the methods they are using to collate user data in real time. 

Thank you, 

Carrie