[liberationtech] The worrisome trend toward liability in networking technology

[Brett Solomon]

Hi

Apologies first for the long email. I'm interested in this discussion and
hope it helps to progress our sector's approach to managing the export, use
and service of surveillance technology. One key takeaway for me here is
that there's no silver bullet. It's going to take numerous strategies to
protect users - good companies implementing best practice, the tech sector
self-regulating, the open source community showing leadership, legislators
enforcing law and users being educated. To rely upon or reject one approach
would be detrimental to what I think is a shared goal - to prevent
surveillance technology being used to abuse people's rights.

I'm glad to see Collin's and others' interest in legislation. Decades of
human rights protection across numerous sectors has relied upon hard law in
order save the vulnerable from the powerful. Producers, and the abusers
they enable, have always argued against intervention as a threat to
innovation and sovereignty - they say 'we can manage it ourselves
internally'. The UDHR, the ICC, the Landmine Treaty are all examples of
third party frameworks that have been net positive gains.

As to the well-trodden and discarded argument that if we don't
do/sell/buy/license it, someone else will, I believe we should stop the
abuse where it takes place and where we have some agency. We may not be
able to prevent homegrown technology, nor control the national
infrastructure or influence companies like Huawei, but we can hamper
harmful technology sales with new norms and laws in the US and EU - the
exact place where, according to Ben Wagner, almost all such surveillance
systems are currently coming from.

Blue Coat should be prosecuted if they breached US Sanction export
controls, which on my reading prohibits the use of this technology from
falling into the hands of Syrian end users, no matter how many resellers
are in between. But if they'd sold to a non-sanctioned rights-abusing
country, they'd still have done something wrong, but get off without
liability. Here is a gap where we I believe we certainly need new laws.
And if a product is being used to abuse rights (eg detain, torture or
kill), I do think that the company should, and be required to, stop
auto-updates and ultimately disable the technology they've created. As
should those in the open source community play their role to help prevent
abuse - its one standard we should uphold whether proprietary or not.

I don't think opposing regulation per se because it necessitates a good/bad
list of countries is reason enough to reject regulation.Theoretically,
regulation can be product or service based as opposed to politically
determined. It's criteria can be set on concepts beyond intent, and instead
focus on usage or risk of usage for abuse. I think a sensible approach is
one based on licensing and transparency requirements across all countries
(and not necessarily bans) - similar to that of dangerous chemicals. Part
of this is 'knowing your customer' approach as set out by the EFF in their
excellent standards.

As I mentioned, it's also about civil society action, and we at Access (
www.accessnow.org) have launched a campaign against the four Western tech
companies *Area* (Italian), *NetApp* (American), *Qosmos* (French), and *
Utimaco* (German) engaged in Syria's latest surveillance plan. 10,000
people signed it since Thursday - and as a result of pressure from many
quarters at least two companies have wavered (wait, 3 as of this morning).
You can add your name here to make it all 4:
https://www.accessnow.org/page/s/stop-syrian-surveillance). But civil
society cannot run a campaign against a company every time this happens. We
need coordinated and systemic change - which brings me back to legislation
and regulation. For those who are interested, I think Access and the EFF
are going to have a call later in the week to discuss this issue and
explore the options, and others who are interested should join. Will send
out details soon.

Finally, instead of waiting for governments to establish laws requiring
transparency from companies like these, let's start crowd sourcing a list
of companies that are selling these technologies. I will start an email
chain about that separately.

Thanks for reading this far!

Brett


On Wed, Nov 9, 2011 at 12:37 PM, Jillian York
<jyork at cyber.law.harvard.edu>wrote:

> Hi Collin,
>
> I'm not sure I can speak for EFF just yet on this front...to be frank,
> this is something we're still mulling over.  On a personal level, I would
> say that regulations are probably--for better or worse--inevitable.  I say
> "for better or worse" because I have grave concerns about their
> implementation.  On the one hand, something *like* EFF's "Know your
> customer" standards could be implemented as a licensing scheme; on the
> other hand, we could end up with GOFA, where countries are split into
> "good" and "bad" lists--and, depending on how that's implemented, it could
> mean our allies (e.g., Bahrain, Saudi) are "good", despite their use of
> surveillance technology and their torture of detainees.
>
> I know that doesn't sound too optimistic, but I am very interested in
> hearing from others as to how regulations might be enacted.
>
> Best,
> Jillian
>
>
>
> On Tue, Nov 8, 2011 at 6:28 PM, Collin Anderson <collin at averysmallbird.com
> > wrote:
>
>> Jillian,
>>
>> A number of individuals, including myself, are interested in using the
>> opening presented by Sen. Mark Kirk's public comments regarding Western
>> involvement with censorship regime to encourage specific legislation
>> regarding the regulation of the export of surveillance equipment. Speaking
>> in both the contexts of yourself as an activist and as a representative of
>> EFF, do you believe there is a future in legislation, or that industry
>> self-regulation backed by standard sanctions is the best we can achieve?
>>
>>
>> On Mon, Nov 7, 2011 at 6:03 PM, Jillian York <jyork at cyber.law.harvard.edu
>> > wrote:
>>
>>> *It could, however, be a useful tactic for getting the message out to
>>> the public that such monitoring is going on.  But, I would worry that it
>>> confuses people - if their takeaway is that they think "Blue Coat" is what
>>> enables the monitoring, and that stopping Blue Coat seriously hampers
>>> surveillance they have been handed an incorrect mental model of how
>>> Internet surveillance works.*
>>>
>>> To tack on to that (because I agree, and because I forgot to say it in
>>> my longer email): Right now, Syrian friends do see Blue Coat very much as
>>> an American export.  In the sense that all other US "net freedom" policy is
>>> moot when American companies are doing this, there's yet another reason for
>>> such tactics.  We're not exactly "winning hearts and minds" with this stuff.
>>>
>>>
>>> On Mon, Nov 7, 2011 at 2:56 PM, Josh <jdsaxe at gmail.com> wrote:
>>>
>>>> Thanks Daniel, I think you make important points.
>>>>
>>>> Just to corroborate what you say, the UNIX command
>>>>
>>>> *tcpdump -A > log &; tail -f log | grep "protest"*
>>>>
>>>> Would monitor network traffic for mentions of the word "protest."  Give
>>>> a developer a couple hours and they could write you a python script that
>>>> could do much more.
>>>>
>>>> I think in this discussion perhaps it's important to remember that
>>>> governments get their surveillance power from their control of the physical
>>>> medium of the Internet, the software applications being talked about here
>>>> are the icing on the cake.  Certainly software that parses application
>>>> protocols and reconstructs what the user's web browser looks like could be
>>>> helpful for an intelligence analyst working for the Syrian government, but
>>>> governments can also hire analysts to sift through more voluminous, noisy
>>>> data from more primitive tools.
>>>>
>>>> In either case the problem of educating activists to avoid surveillance
>>>> remains.  If they are going to ensure the security of their communications
>>>> to some reasonable degree of certainty activists need to understand the
>>>> ways in which their IP traffic are exposed to interlopers, and they need to
>>>> know strategies for circumventing surveillance, as you say.
>>>>
>>>> So that said, I think you're probably right about regulating filtering
>>>> software as a red herring.  I don't think such a tactic, for those of us in
>>>> countries like the U.S., will be particularly effective in actually
>>>> stopping oppressive regimes from spying on their citizens, even if it
>>>> succeeds in depriving these governments of specific monitoring software.
>>>>
>>>> It could, however, be a useful tactic for getting the message out to
>>>> the public that such monitoring is going on.  But, I would worry that it
>>>> confuses people - if their takeaway is that they think "Blue Coat" is what
>>>> enables the monitoring, and that stopping Blue Coat seriously hampers
>>>> surveillance they have been handed an incorrect mental model of how
>>>> Internet surveillance works.
>>>>
>>>> Josh
>>>>
>>>> On Mon, Nov 7, 2011 at 4:18 PM, Daniel Colascione <
>>>> dan.colascione at gmail.com> wrote:
>>>>
>>>>> I've read with interest the recent discussion on this list of the
>>>>> ramifications of the use of western filtering technology by oppressive
>>>>> regimes. While the problem warrants serious discussion, I feel that
>>>>> the direction of remedies is veering dangerously and I'd like to adopt
>>>>> a bit of a contrary position on the nature of an appropriate response:
>>>>>
>>>>> We're all outraged by the use by oppressive regimes of western
>>>>> filtering and network monitoring technology. But we should pause
>>>>> before acting on this outrage: the situation is more subtle than might
>>>>> be immediately apparent.  Oppressive regimes don't censor for a
>>>>> profit: the filtering and monitoring are existential and ideological
>>>>> imperatives. While these regimes (like all rational actors) seek to
>>>>> minimize costs, they would continue to censor even at greatly
>>>>> increased prices. Of course, there is some price at which these
>>>>> regimes would discontinue censorship: but, as the failure of
>>>>> conventional economic sanctions to discourage other behavior has
>>>>> shown, oppressive regimes are nowhere near being inconvenienced enough
>>>>> to change their behavior.
>>>>>
>>>>> If the efforts discussed here were successful and oppressive regimes
>>>>> were somehow cut off from western filtering technology, we wouldn't
>>>>> see an end to Internet censorship: Instead, we would see regimes
>>>>> invest in domestically-developed solutions, which are actually less
>>>>> expensive than one might think: anyone can create professional-quality
>>>>> software for free, and hardware can be manufactured with a little
>>>>> capital investment. Iran has already made headway in this area. All
>>>>> the sanctions, codes of conduct, lawsuits, and so on we've been
>>>>> discussing here amount to an increase in the cost of censorship, and
>>>>> oppressive regimes simply aren't cost-bound at this point. Now, all
>>>>> things being equal, the world becomes a better place as censorship
>>>>> becomes more expensive. If all things were equal, I would support the
>>>>> ideas being raised on this list.
>>>>>
>>>>> But not all things are equal: the measures under discussion have
>>>>> serious externalities that warrant discussion, and they could affect
>>>>> the entire technology industry.
>>>>>
>>>>> 1) How do we decide what technology is subject to regulation? Network
>>>>> technology is quintessentially dual-use: anyone who has administered a
>>>>> network knows that the same features that allow us to block outbound
>>>>> attacks, accelerate the web via transparent caching, scan our servers
>>>>> for vulnerabilities, detect cross-site scripting attacks, and scan our
>>>>> email for spam can also be used for malicious ends. Cluster bombs and
>>>>> Saturday night specials have no legitimate purpose, but proxies,
>>>>> firewalls, and logging tools have many.
>>>>>
>>>>> The only reasonable way to draw a line between filtering and general
>>>>> network technology is to use intent, which is a notoriously vague
>>>>> standard. If marketing censorship technology were to become attached
>>>>> to serious liability issues, vendors would develop software with
>>>>> precisely the same features and market them with "wink, wink, nudge,
>>>>> nudge" references to benign network management.
>>>>>
>>>>> 2) Where does responsibility end? Brett Solomon suggests that we
>>>>> attach liability to technology, but the concept isn't entirely clear.
>>>>>
>>>>> Consider Blue Coat: the exact mechanism by which Syria acquired Blue
>>>>> Coat's technology remains unclear. If Blue Coat knowingly sold this
>>>>> technology to Syria, it ought to be prosecuted under applicable
>>>>> laws. But if it turns out that Syria indirectly obtained Blue Coat's
>>>>> technology through the gray market or outright piracy, I feel it'd be
>>>>> difficult to attach legal or moral blame to the company.
>>>>>
>>>>> Let's suppose we do hold Blue Coat liable for Syria's use of its
>>>>> products because it provided automated software updates or other
>>>>> routine and free support: to avoid future liability, companies would
>>>>> be forced to consult blacklists and prevent certain IP address ranges
>>>>> from downloading updates, manuals, and other materials. (Or they could
>>>>> be required to go through a more robust process to verify the
>>>>> nationality of anyone requesting support.)
>>>>>
>>>>> These restrictions would be wholly ineffective --- any marginally
>>>>> competent network administrator could obtain support via third parties
>>>>> and proxy servers, but every networking firm would need to bear the
>>>>> expense of implementing these restrictions. Would we propose that
>>>>> companies use a more robust process to verify the nationalities of
>>>>> those requesting support?
>>>>>
>>>>> After the failure of these measures, what would be next? Would we
>>>>> compel networking vendors to include "kill switches" that could be
>>>>> used to remotely disable network technology remotely if it's found in
>>>>> an unapproved context?  Should devices refuse to function without
>>>>> continuous approval of the original manufacturer?
>>>>>
>>>>> Never mind the financial cost of these measures: instead, imagine the
>>>>> potential for misuse. What happens when bad actors inevitably
>>>>> commandeer these safeguards to cause mischief? What happens when
>>>>> governments force companies to cut off support for (and perhaps
>>>>> disable outright) network equipment in a rival nation?
>>>>>
>>>>> 3) What about open source software: do we subject it to the same
>>>>> regulations, effectively destroying the tradition of openness that
>>>>> fosters its development?  Or do we exempt open source software from
>>>>> regulation, allowing oppressive regimes to access filtering technology
>>>>> almost as easily as before, but penalizing companies that try to
>>>>> innovate with new networking technology they'd like to keep
>>>>> proprietary. Should downloading Squid or Nessus warrant a background
>>>>> check? The result of such a scheme would be a retroactive,
>>>>> self-inflicted defeat in the crypto wars of the 1990s.
>>>>>
>>>>> When we ask for regulation of filtering technology, we're asking for
>>>>> the very dystopia we're all trying to avoid. Brett Solomon's proposal
>>>>> for "human rights by design" is in fact "oppression by
>>>>> design". Because software cannot judge the moral orientation of the
>>>>> bytes it processes, the only mechanism with which we can enforce
>>>>> enforce human rights is authoritative third party control, which can
>>>>> be used to do more evil than it can ever do good.
>>>>>
>>>>> Look: it makes sense to enforce simple, clear measures to disrupt
>>>>> censorship and surveillance: we already effectively prohibit direct
>>>>> transfer of technology to oppressive regimes, as anyone who has dealt
>>>>> with OFAC can attest. But the increasingly strenuous measures proposed
>>>>> here, e.g. kill switches and licensing, come with diminishing returns
>>>>> and serious costs to our budgets and liberties, and any "victory"
>>>>> would be Pyrrhic.
>>>>>
>>>>> Most participants in this discussion have analogized filtering
>>>>> software to weapons technology, but the better analogy is drug
>>>>> prohibition: both drugs and filtering technology can be produced
>>>>> without massive capital investment; both can be produced clandestinely
>>>>> (albeit at higher cost relative to commercial production); both are in
>>>>> high demand; both are easily moved on black and gray markets; both
>>>>> often have perfectly legitimate uses impossible to distinguish from
>>>>> illicit ones; both may very well have important adverse effects on
>>>>> society.  Most importantly, in both cases, attempts to crack down on
>>>>> trade simply moves the trade underground while simultaneously causing
>>>>> collateral damage to society as a whole.
>>>>>
>>>>> I may attract some flames for the sentiment, but I believe that
>>>>> outrage over the use of western filtering technology is a red
>>>>> herring. Our resources are better spent on 1) circumventing of the
>>>>> filtering that will inevitably be put in place, and 2) activism that
>>>>> will reduce the demand for censorship and surveillance in the first
>>>>> place.
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> liberationtech mailing list
>>>>> liberationtech at lists.stanford.edu
>>>>>
>>>>> Should you need to change your subscription options, please go to:
>>>>>
>>>>> https://mailman.stanford.edu/mailman/listinfo/liberationtech
>>>>>
>>>>> If you would like to receive a daily digest, click "yes" (once you
>>>>> click above) next to "would you like to receive list mail batched in a
>>>>> daily digest?"
>>>>>
>>>>> You will need the user name and password you receive from the list
>>>>> moderator in monthly reminders.
>>>>>
>>>>> Should you need immediate assistance, please contact the list
>>>>> moderator.
>>>>>
>>>>> Please don't forget to follow us on
>>>>> http://twitter.com/#!/Liberationtech
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> liberationtech mailing list
>>>> liberationtech at lists.stanford.edu
>>>>
>>>> Should you need to change your subscription options, please go to:
>>>>
>>>> https://mailman.stanford.edu/mailman/listinfo/liberationtech
>>>>
>>>> If you would like to receive a daily digest, click "yes" (once you
>>>> click above) next to "would you like to receive list mail batched in a
>>>> daily digest?"
>>>>
>>>> You will need the user name and password you receive from the list
>>>> moderator in monthly reminders.
>>>>
>>>> Should you need immediate assistance, please contact the list moderator.
>>>>
>>>> Please don't forget to follow us on
>>>> http://twitter.com/#!/Liberationtech
>>>>
>>>
>>>
>>>
>>> --
>>> jilliancyork.com | @jilliancyork | tel: +1-857-891-4244
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> liberationtech mailing list
>>> liberationtech at lists.stanford.edu
>>>
>>> Should you need to change your subscription options, please go to:
>>>
>>> https://mailman.stanford.edu/mailman/listinfo/liberationtech
>>>
>>> If you would like to receive a daily digest, click "yes" (once you click
>>> above) next to "would you like to receive list mail batched in a daily
>>> digest?"
>>>
>>> You will need the user name and password you receive from the list
>>> moderator in monthly reminders.
>>>
>>> Should you need immediate assistance, please contact the list moderator.
>>>
>>> Please don't forget to follow us on http://twitter.com/#!/Liberationtech
>>>
>>
>>
>>
>> --
>> *Collin David Anderson*
>> averysmallbird.com | @cda | Washington, D.C.
>>
>>
>
>
> --
> jilliancyork.com | @jilliancyork | tel: +1-857-891-4244
>
>
>
>
> _______________________________________________
> liberationtech mailing list
> liberationtech at lists.stanford.edu
>
> Should you need to change your subscription options, please go to:
>
> https://mailman.stanford.edu/mailman/listinfo/liberationtech
>
> If you would like to receive a daily digest, click "yes" (once you click
> above) next to "would you like to receive list mail batched in a daily
> digest?"
>
> You will need the user name and password you receive from the list
> moderator in monthly reminders.
>
> Should you need immediate assistance, please contact the list moderator.
>
> Please don't forget to follow us on http://twitter.com/#!/Liberationtech
>



-- 
Brett Solomon
Executive Director | Access
accessnow.org | rightscon.org
+1 917 969 6077 | skype: brettsolomon | @accessnow

<http://www.europarl.europa.eu/parliament/public/staticDisplay.do?language=en&id=42>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/liberationtech/attachments/20111114/9c28c901/attachment.html>