[liberationtech] Mechanisms of intercepting service provider internal connectivity

[Andrés Leopoldo Pacheco Sanfuentes]

Another application for the "deep packet inspection" technique..
On Jun 9, 2013 6:32 PM, "Gregory Maxwell" <greg at xiph.org> wrote:

> On Fri, Jun 7, 2013 at 6:47 AM, Eugen Leitl <eugen at leitl.org> wrote:
> > but the ability to assemble intelligence out of taps on providers'
> internal connections
> > would require reverse engineering the ever changing protocols of all of
> those providers.
>
> This is somewhat less difficult than some people think.
>
> Various equipment manufacturers have implemented passive monitoring
> support on their interfaces specifically for these applications.  You
> configure the interface to go into UP/UP state and to listen in a half
> duplex manner.  This way you get the compatibility advantage of using
> standard network equipment to implement the interception, and so it
> will likely speak the same link-layer protocols the device being
> intercepted speaks.
>
> (E.g. here is some of the relevant documentation for Juniper:
> http://kb.juniper.net/InfoCenter/index?page=content&id=KB23036 and
>
> https://www.juniper.net/techpubs/en_US/junos11.2/topics/concept/flowmonitoring-passive-overview-solutions.html
> )
>
> A lot of the mechanisms— the protocols, techniques, equipment
> features— for mass surveillance are easily visible to the public but
> the things visible to the public are all technical minutia dealing
> with the practical engineering challenges (Like the one you raise
> here— how the heck do you keep up with the ever changing layer 1/2
> protocols used by service providers) that most people wouldn't even
> think to ask about.
>
> Using commodity hardware gets you compatibility, lower costs, and fast
> deployment. Even though budgets for massive surveillance no doubt
> allow for all kinds of specialized hardware— you can get more of it
> faster if you use commodity stuff with a few tweaks where you can.
>
> Here's another tidbit in public docs:
>
> Another challenge in implementing massive surveillance is the sheer
> volumes of traffic involved.  People do seem to be aware of this one,
> but they argue that it makes the task impossible.... but there are few
> technical challenges which can't be solved by the suitable application
> of ingenuity and money. (_Lots_ of money: but keep in mind "defense"
> spending is just on another order of magnitude from regular spending.
> How much does a fighter jet cost? A one time use smart munition?  How
> much more valuable is a good network tap than these devices? In light
> of that— how much is a fair defense industry price for one?)
>
> One way that the traffic volume problems gets solved is to realize
> that the vast majority of traffic is uninteresting.  If you can
> rapidly filter the traffic you can throw out the 99% of uninteresting
> stuff and capture all of the rest.  Filtering is, of course,
> computationally expensive—  but it turns out that the power of
> 'commodity' technology can come to the rescue again:   The same
> standard networking equipment that you need to speak the L1/L2
> protocols on your optical taps also has built in line-rate packet
> filtering with scalability to millions of filter criteria (at no extra
> cost! :) ).
>
> The filtering in these devices has not historically been DPI grade:
> you can do stateless range/prefix matches on the packet headers, not
> free-form regex (although this is changing and the latest generation
> of hardware is more powerful— the need for NAT everywhere, if nothing
> else, is mandating it).  But, if you can update those filters very
> fast— say, in under 50ms— then it doesn't matter that the filters
> aren't very powerful:   Configure the filters to catch all known
> interesting hosts, the beginning of every new connection, and some
> small fraction (say, 1:10000 of all packets) and then feed that data
> to analysis systems which trigger updates to the filters when they
> spot something interesting.  They only need to be powerful enough to
> limit a terabit of traffic to tens of gigabits, and that level of
> filtering can be accomplished just on 5-tuples..
>
> You can go even further, then, by having two sets of filters with a
> delay line— say implemented using the >100ms of delay-product packet
> buffers in high end commodity networking hardware— in between them.
> The first set of filters catches enough so that your analysis systems
> can identify and track interesting flows, and by the time the traffic
> makes it through the delay line the second set of filters has been
> updated to capture the entirety of the interesting flow.  ... though
> the persistence of traffic and the delay created by the TCP three way
> handshake make going this far not terribly necessary.
>
> Of course, using filtering in this way would require a protocol
> between the network elements and the analysis systems so that they
> could rapidly and dynamically 'task' the filters like you task
> surveillance satellites... And it sure would be convenient if the
> protocol was standardized so you could get many kinds of devices
> speaking it. ... something like:
> http://tools.ietf.org/html/draft-cavuto-dtcp-03
> (and here is one vendor's helpful documentation on applying it,
>
> https://www.juniper.net/techpubs/en_US/junos/information-products/topic-collections/nce/lawful-intercept-flow-tap/lawful-intercept-using-flow-tap.pdf
> )
>
> I've been continually amazed at how poorly the public has been doing
> at figuring out the mechanisms used for this stuff— You don't need
> some insider to tell you how it works, you could have just looked up
> the authors of packet interception standards and looked for people
> people who worked for major providers and advertise TS/SCI clearance
> on their resumes— then put together the pieces.  People have observed
> that building high-tech infrastructure makes keeping secrets very
> costly but then fail to notice all of the completely open building
> blocks than just require the suitable application of big money.
>
> You're obviously not going to find smoking guns— things like
> production system core-dumps showing that which newspapers were being
> targeted for total surveillance— because the people building this
> stuff are not idiots, and it's not customary to publish that
> information generally. ... but the underlying technology and
> documentation needed to build this infrastructure, or at least the
> parts built out of commodity hardware. You absolutely can find that if
> you bother looking.
>
> But at the end of the day the technical details behind how it works
> really don't matter much.  Just assume that a hostile party is at
> least passively monitoring all network links that aren't completely in
> your control and you'll be making the right kind of security decisions
> regardless of the technical minutia.
> --
> Too many emails? Unsubscribe, change to digest, or change password by
> emailing moderator at companys at stanford.edu or changing your settings at
> https://mailman.stanford.edu/mailman/listinfo/liberationtech
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/liberationtech/attachments/20130610/5c511878/attachment.html>