[liberationtech] Fwd: Avaaz in "grave danger" due to GMail spam filters

[Rich Kulawiec]

On Tue, Aug 20, 2013 at 12:27:24PM -0400, Matt Holland wrote:
> Rich: We actually do run our email lists in-house, sent from our own MTA's,
> with appropriate SPF records, DKIM signature, list-precedence headers, etc.
> etc. Our message to members was focused on getting into a particular "tab"
> at Gmail though; I think if we were having problems with those basic
> list-management issues we'd be more likely to see our messages being marked
> spam or just dropped outright.

First, it's good that you're listening in here.

Second, Gmail is a poorly-run email service.  That's somewhat surprising
to me, actually: I expect much better out of them.  But it really is
quite mediocre, and I therefore recommend against it for anyone who's
actually serious about email.  (Then again, it's not the *worst* Google
service: their horrible mangling of Usenet into "Google Groups", a disaster
from its inception to the present, holds that honor.)

Third, to generalize that comment: it's not worth worrying about delivery
to Gmail/Yahoo/Hotmail/AOL.  They're either (a) crap or (b) well on their
way to being crap.  I can't fix this.  You can't fix this.  I'm pretty
sure they can't fix it or just plain don't want to fix it.  So the solution
to this isn't to turn yourselves inside-out trying to jump through Gmail's
hoops or Yahoo's hoops so that they'll accept your mail: the solution is
to tell everyone that freemail is worth what they're paying for it.
(Arguably, given recent events: it's worth less.)  To borrow from the
previous paragraph, anyone who is serious about email should get
a real email account.  Those four providers have spent the last decade [1]
proving that they can't furnish one.

Fourth, I've taken the time to evaluate -- at a cursory level -- your
mailing list operation.  Here are my findings and recommendations.
I'm sure they're incomplete (hence "cursory").

1. SPF is snake-oil, as should have been obvious to everywhere when
it was introduced with this grandiose and ludicrous claim:

	"Spam as a technical problem is solved by SPF."

So: don't bother.

(DKIM?  DKIM shows some *potential*.  I am as yet unconvinced of its
anti-spam value, since my spamtraps receive spam all day every day that
passes DKIM validation.  Some say that DKIM has anti-forgery value, but
(a) the Internet clearly does not consider email forgery an important
problem and (b) even if it did, the problem is currently insoluble even
if DKIM is globally deployed and works perfectly.)

2. You're using Google to handle your incoming email.  Not a good choice:
see comments above.

3. You have working "postmaster" and "abuse" addresses that are
answered in a timely manner by a real live human being.  Excellent.
You're thus in compliance with the applicable portions of
RFC 5321 and RFC 2142, and you're doing what every single responsible,
ethical, and competent operation on this planet should do.

4. You're not in compliance with section 6 of RFC 2142 because
your mailing list does not support a -request address.  This is not
only mandatory, but it's been a best practice for 30-ish years.
Thus *this* mailing list supports:

	liberationtech-request at lists.stanford.edu

because it darn well should.

5. You also don't appear to be in compliance with the long-standing
convention and best practice of -owner, which is analogous to -request,
except that (a) -request may or may not be a person but (b) -owner
is always a person.  Thus the -owner address is the one to use when
the automation behind -request isn't behaving: it provides a way
for subscribers and non-subscribers alike to initiate a conversation
with the person(s) operating any particular list.

6. You're not in compliance with RFC 2919 or RFC 2369.  Again,
using *this* list as an example, these headers are present:

	List-Id: liberationtech <liberationtech.lists.stanford.edu>
	List-Unsubscribe: <https://mailman.stanford.edu/mailman/options/liberationtech>, <mailto:liberationtech-request at lists.stanford.edu?subject=unsubscribe>
	List-Archive: <http://mailman.stanford.edu/pipermail/liberationtech>
	List-Post: <mailto:liberationtech at lists.stanford.edu>
	List-Help: <mailto:liberationtech-request at lists.stanford.edu?subject=help>
	List-Subscribe: <https://mailman.stanford.edu/mailman/listinfo/liberationtech>, <mailto:liberationtech-request at lists.stanford.edu?subject=subscribe>

7. List messages are sent from an unreplyable address.  That's not
only an extremely bad idea, it's very rude.  It is the email equivalent
of sticking your fingers in your ears and saying "LA LA LA LA I can't
hear you" when the entire rest of the Internet is trying to tell you
that you've got a problem or are causing a problem.  All email should
always be sent from a replyable address, period.

8. You do not appear to use web bugs in your mailing list messages.
A wise choice: web bugs are malware, they're invasive and abusive, 
and they actively degrade the security of recipients...which is
a pretty crappy way to treat one's audience.

9. Your mailing list traffic does not wrap lines properly -- yet
doing so is a basic email courtesy.  You do, however, use paragraph
breaks in a sensible manner, which helps readability.

10. And the kicker: you are not using COI (confirmed opt-in, or
closed-loop opt-in, take your pick) on your mailing list,
therefore you are spamming.  Period, full stop, this is not open
for debate or question.

See, for example (and this doesn't cover the whole thing, but...)

	http://www.spamhaus.org/whitepapers/permissionpass/

The problem you now face is that because you haven't been doing COI
since Day One your list is now full of crap.  Oh, sure, there are some
legitimate subscribers on it, but there are also typos (yes, people
typo their own addresses all day every day, this is common knowledge),
there are spamtraps, there are dead addresses, there are repurposed
addresses...as I said, your list is now full of crap.  And every time you
send out a message to that list full of crap, you're stacking evidence
on the pile that says "Avaaz is spamming".


The fix for *most* of this is simple and easy.  First, get off Google
and host your own email.  Any combination of (Linux/BSD) with (sendmail,
postfix, exim, courier) will do.  Second, install Mailman, which will
solve several of the above problems in its stock/default configuration.
Third, dump SPF.  Keep DKIM if you want, just don't expect it to do
much good.

Then comes the hard part.  If you want to stop spamming, you're going
to have to run your list through a COI pass, which will take time
and effort.  There is no evading this, no easy way around it, no trick:
either you do it or you don't.  If you do, then you'll be able to produce
proof-on-demand of the verified provenance of every subscriber -- which
is something that is part of baseline ethical and competent mailing
list operational practice.  If you don't, then you will keep spamming,
you will keep getting (correctly) blocked/blacklisted, and your problems
will continue to get worse.

(The latter is precisely what has happened to others who have faced this
decision and chosen...poorly.)

Bottom line: you've made a rather large mistake.  You now have the
opportunity to fix it.

---rsk

[1] I would be remiss if I allowed that blanket statement to cover AOL
in toto.  AOL made massive improvements during Carl Hutzler's tenure.
He and his coworkers fixed a lot that was broken, and were well on their
way to moving AOL from the bottom of the pile to (perhaps) the top.
Their work was much appreciated by those of us *outside* AOL who
enjoyed the tremendous reduction in AOL-sourced abuse.  AOL rewarded
this hard-working, competent, diligent team by dismissing them a
couple of years ago and, predictably, the slide downward started
almost immediately.  AOL has not yet quite reached the bottom of
the barrel (currently occupied by Yahoo and Hotmail) but I think
in another year or two its journey will be complete.