[liberationtech] XKeyScore code authenticity - genuine [was: messing with XKeyScore]

[Nathan Andrew Fain]

The points are concise and valid arguments. The lack of efficient
regular expressions may give hint to the immaturity of this code. So
do does the half hazard rules protecting five eyes users. Apparently
as someone (terrorist or not) accessing Tor from the five eyes nations
you are filtered out. If you also happen to have looked at the linux
journal there is no filtering for five eyes so you would have been
tagged. This inconsistency shows how foot loose things were/are at the
NSA.

This would fit with what Diane Roark said in her interview with
Frontline [1]. She protested the removal of strict filtering for US
Citizens and was buffed by every side. In that light it is clear the
US made the entire cache free game for one or many engineers without
anyone looking over their shoulder. A ship early fix later policy many
engineers love so much.

In the end, with their budget, a 50% reduction in efficiency from a
regex would just justify a few hundred more distributed computing
nodes. The only person in the chain that may question this would be a
pro-surveillance tax payer that knows regular expressions. So it may
not indicate the age of the filters at all.

http://www.pbs.org/wgbh/pages/frontline/government-elections-politics/united-states-of-secrets/the-frontline-interview-diane-roark/

On 06/07/2014 19:11, coderman wrote:
> the theme of messing with XKeyScore is amusing[0], but more to the 
> point i was asked to respond to some concerns of authenticity made
> in a different post:
> 
> "Validating XKeyScore code" 
> http://blog.erratasec.com/2014/07/validating-xkeyscore-code.html
> 
> i'm trying to keep this feedback technical, as i don't like much
> of Graham's reasoning. (i do however approve of his use of "Great
> Man" in the Voldemort sense, in reference to Cowboy Alexander[1])
> 
> his claim that "we believe the code partly fake and that it came
> from the Snowden treasure trove." should be ammended:
> 
> "we believe the code deprecated, and that it came from the Snowden
> archives"
> 
> onward!
> 
> ---
> 
> first segment of summary, by point:
> 
> 
> # Point 1) "The signatures are old (2011 to 2012), so it fits
> within the Snowden timeframe, and is unlikely to be a recent
> leak." - agreed.
> 
> 
> # Point 2) "The code is weird, as if they are snippets combined
> from training manuals rather than operational code. That would mean
> it is “fake”." - false; the code is valid and deprecated (can be
> used as example) rather than false. the technical detail. as a
> programmer, i know that a regexp rule like: ''' extractors: {{ 
> bridges[] = 
> /bridge\s([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}):?([0-9]{2,4}?[^0-9])/;
>
> 
}}
> ''' is both written by a novice regexp'er, and also took them a bit
> of time. more than they'd spend on an example.
> 
> for another example, ''' for (size_t i=0; i < bridges.size(); ++i)
> { std::string address = bridges[i][0] + ":" + bridges[i][1]; 
> DB[SCHEMA_OLD]["tor_bridge"] = address; DB.apply(); 
> DB[SCHEMA_NEW]["tor_ip"] = bridges[i][0]; 
> DB[SCHEMA_NEW]["tor_port_or"] = bridges[i][1]; 
> DB[SCHEMA_NEW]["tor_flags"] = FLAGS; DB.apply(); } '''
> 
> why two commits here to backend changes? as a programmer i
> understand why this is done, but as a purely fictitious example the
> double commit is pointless noise.
> 
> 
> # Point 3) "The story makes claims about the source that are
> verifiably false, leading us to believe that they may have
> falsified the origin of this source code." - false
> 
> how does limited misunderstanding arcane technicalities invalidate
> the entirety? if this were true, Robert Graham would be a complete
> imbecile, rather than  technically competent and occasionally
> wrong.
> 
> 
> 
> # Point 4) "The code is so domain specific that it probably is, in
> some fashion, related to real XKeyScore code – if fake, it's not
> completely so." - false. as stated above, these rules are
> deprecated rather than fictitiously constructed. (and perhaps
> referenced in training materials for utilizing the particular
> language engines demonstrated)
> 
> as explained above, and i will go into more detail later (i wager
> i have more big data experience and DPI experience than Mr. Graham
> the DPI expert does in this domain alone[2] ;)
> 
> 
> 
> last but not least, this speaks to the need for greater technical 
> expertise to be applied to the leaked archives.  if anything, the 
> nature of domain specific details discussed here show that not
> just generalists, but an army of specialists, will ultimately be
> needed to properly parse and protect based upon the archives as yet
> revealed.
> 
> 
> best regards,
> 
> 
> 
> ---
> 
> 0.  "" for those with "Jam Eschelon Day" nostalgia ;) ^- see whole
> thread from "messing with XKeyScore"
> 
> 1. "The character assassination of Keith Alexander" '... People
> have criticized calling him a "great man". I'm quoting the Harry
> Potter movie here people, where the guy who sells Harry's[sic] wand
> points out that Voldermort was a great wizard,[sic] a great and
> terrible wizard' 
> http://blog.erratasec.com/2014/06/the-character-assassination-of-keith.html
>
>  2. "XKeyScore: it's not attacking Tor" '... I am an expert in deep
> packet inspection (DPI). I've written a system vaguely similar to
> this XKeyScore system here: (ferret). I find the conclusions in
> this story completely unwarranted, though the technical information
> cited by this story is pretty good. I suggest future stories about
> the NSA's deep packet inspection actually consult with engineers
> who've written DPI code before making wild claims.' 
> http://blog.erratasec.com/2014/07/xkeyscore-its-not-attacking-tor.html
>