[H-GEN] People's thoughts on Greylisting

[Bruce Campbell]

On Sat, 26 Nov 2005, Greg Black wrote:

> On 2005-11-25, Bruce Campbell wrote:
>>
>> Have people used it or encountered it?  What are your thoughts on the
>> method or implementations that you have seen ?
>
> The original paper actually lists two significant issues with
> greylisting that, to the best of my knowledge, have not been
> addressed:
>
>  1. Some large ISPs use a rotating pool of servers to handle
>     outgoing email, so that it's possible for an email to time
>     out from the sender's perspective before the original host
>     happens to be called upon to attempt a second delivery.

Although I have not done this, the key to solving this appears to be 
keying on a logical IP range (eg, a /24), rather than on a single IP 
address.  This catches most instances of mail clustering.

IMHO, implementors of mail clustering should note which nodes encountered 
temporary errors, and retry the delivery attempt from that node within a 
'reasonable' interval.

>  2. Some mailing lists (in particular ezmlm, which is still
>     widely-used but not in active development) use a variation
>     on VERPs that breaks the greylisting proposal.

The implementation that I am using on my personal domains gets around this 
problem by keying on the sender domain, and not the full sender address. 
This works because all VERP tricks that I've seen are being performed on 
the LHS of the sending address, not the RHS (the domain).

>     Suggesting that mailing list traffic is not
>     timely anyway and that additional delays in delivery don't
>     matter doesn't fit with my needs either.

It is worthwhile noting here that enabling greylisting will mean that the 
first (mailing list) posts to you and your user's mailboxes will be 
delayed.  Once the Greylisting system has seen the mails and added the 
relevant details to the database, future posts go through without any 
delay.

> So, although I know plenty of people are happy with greylisting,
> I'm not going to try it; the problems with it seem like too high
> a price to pay for what would be, at most, a limited return.

Other problems which have been mentioned in other forums are:

 	*) Senders which treat any kind of error as permanent.
 	   Only a problem if you care about the specific mail.
 	*) Senders that do not retry until your record expiry
 	   time has past (slow queue re-run)

On Fri, 25 Nov 2005, Robert Brockway wrote:

> On Sat, 26 Nov 2005, Stephen Thorne wrote:
>
>> I've implemented greylisting, and one of the things I discovered through 
>> error:
>> 
>> a) the primary and the backup mx's should all have graylisting implemented.
>> b) you should only have one mx.
>
> This seems to be a common belief these days based (partly?) on the belief 
> that mail servers in Data Centres are highly reliable and so a backup MX 
> isn't needed.

Actually, I would render both of those points as:

   a) All your MXes should implement the same behaviour.
   b) Assuming that all of the Internet is reachable from a single point is
      a mistake.

> The argument that backup MXs allow more spam through certainly doesn't count 
> in our case - we don't whitelist mail just because it comes from the backup 
> MX.  I've never seen a reason to do this.

The only reason for specifically whitelisting secondary MXes, in the 
case of Greylisting, would be if:

 	1) You're reasonably sure that mail is not going to be bad and you
 	   do not wish to incur the second time penalty (see (a)).
or
 	2) The operators of the secondary MX have Said Words about
 	   your mail building up in their queue[1].

>> Hopefully, once we deploy it on a site with a non-trivial amount of
>> spam, we'll be able to do some statistics collection to see what net
>> effect is has.
>
> Back to the topic...I ran greylisting on my personal domain for a while and 
> initially saw a drop in spam levels but I think they returned to normal 
> fairly quickly (no hard data available).

On my own, recently greylist-enabled servers, there is a slow rise in the 
amount of spam not being relayed through a proper MTA, due to the fact 
that I am not cleaning out the old keys previously used.  Some spammers 
tend to use the same SMTP envelope details from the same (compromised?) 
hosts, thus, a new attempt may match a key created several days ago (and 
not used since), and the spam gets through.

Statistics for my own mailbox over ~2 weeks are:

 	Total Keys: 2800	# Senderdomain + remoteIP + destaddress
 	Total Attempts: 8308	# [2]

 	Successful Keys: 189
 	Number of Mails: 5342		# Average 28 mails / key  [3]
 	Blocked, then passed: 543	# Initial addition of keys to db
 					# Some MTAs have fast retry times,
 					# and try again inside the greylist
 					# interval.

 	Unsuccessful Keys: 2526
 	Number of Attempts: 2478
 	Number of DNS Aborts: 206	# Verification of sender domain

So, enabling greylisting has cut my mail by roughly a third, (hopefully) 
all of which was spam.  Correctly speaking, 2478 items of mail were not 
retried by the same IP address, and may have appeared in later counts.

Since cutting the amount of spam mail in my inbox was the goal of enabling 
greylisting on my servers, the removal of ~170 spams per day from my 
direct inbox view to ~3 spams per day is a success in my book.

--==--
Bruce.

[1] Which in itself is an indicator that you've got a large volume of
     mail.
[2] 8309, 8310, 8311,... stay still and be counted dammit!
[3] Yes, a number of lists that I'm on are rather busy.