Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

MGLEE (4392)

MGLEE
  (email not shown publicly)

If any of this makes any sense yes I probably am that person : anatomy, anat0010, AND, Banbury, bookstack, Bristol, Community, ebookers, Fitz, GDM, ghosts, IFSBM, mondus, naming company, ox.test, Oxford, Oxdigital, spam, statistics.

Journal of MGLEE (4392)

Thursday September 25, 2003
07:52 AM

Distributed DNS blacklists # 2

The best architecture I can think of is using the distributed nature of Usenet to disseminate incremental updates to DNS blacklists using some authenticated structured format. The most appropriate I can think of is signed XML.

This system allows users to access blacklist information anonymously and create local (or regional) blacklist mirrors which are private to a local network (or ISP) or at the least not widely advertised. In any case, if one mirror is taken out, it doesnt affect any of the other mirrors.

The newsgroup would be moderated, trusted maintainers of blacklists given a key with which to create an 'approved' header with their own stamp of trusted approval.

The obvious attacks are post flooding and cancel bots. The former can be defeated with an official cancel bot, the latter by a resurrection bot.

Signing the content of the posts allows users to determine if they trust the assertion that the post content relates to the named blacklist and detect and reject attempts at poisoning the lists.

Its an imperfect solution, but its a start. Ideas ?

Wednesday September 24, 2003
10:51 AM

Distributed DNS blacklists

Now that yet another DNS blacklist (monkeys) has been retired due to a continuing massive denial of service attack perhaps its time to rethink DNS blacklists.

Using DNS to rapidly query a server to check if an IP address is listed in it is a great idea. Its fast, little overhead involved, DNS is a well known and supported protocol. Querying against a single server allows the owners of the list to rapidly ammend the list when needed.BUT it also provides a single point of attack.

Somebody (maybe a spammer ?) has taken it upon themselves to launch continued denial of service attacks on the servers hosting the DNS lists. DNS wasnt designed to withstand such attacks, but surely with all the knowledge that has gone into designing distributed P2P networks there must be another way of distributing DNS blacklists.

Napster got taken out due to its client server architecture, Gnutella continues due to being entirely distributed. You can force a single server to go down through court action, or a DoS attack, but that will only affect a small part of the network, the rest continues unaffected.

So how can we apply this architecture to DNS blacklists ?

continued tomorrow...

Tuesday September 23, 2003
08:36 AM

Bogus Email addresses

Interesting question this morning - Can I block all these viruses being mailed to me by email address alone ? .

My first thought was, if the addesses are obviously bogus strings of random characters that manage to pass simple email address syntax checkers, you could trap them by applying Shannon's Entropy and detecting the randomness. But this may not work in email since many usernames and even domain names can appear to be entirely random, k5031@hotmail.com for example.

I then started thinking about Benford's Law and the entropy of numbers and wondered if that could be applied to emails.

I have a small number of friends who email me frequently and a large number of aquaintances (and newsletters) who email me occasionally. The most anyone would ever email me in a day is 20x, and that is a case of spending way too much time conducting an all day email conversation. Being a boring stay-at-home type, my set of friends is unlikely to change, and any new friend will more than likely start off in the set of acquaintances before being upgraded. Strangers write to me from time to time, frequently its spam, although sometimes it is an aquaintance with a new email address, or a new newsletter I have subscribed too.

Therefore I should be able to deduce an algorithm predicting the frequency with which any given email address sends me email. Armed with this I should be able to construct an allowed message frequency band for email communication. If an email address suddenly jumps outside this allowed frequency band, then something is wrong. Either this is a spam, or a friend (or acquaintance has a virus). Either way I dont want this email in my inbox, but quarantined.

hmmm could work.

Monday September 22, 2003
04:01 AM

Net::Milter

On Friday I was given permission to release the last month's or so work to CPAN. How cool is it to work for an employer that allows you to both take from and give to the open source community.

Net::Milter is a module that acts as the Sendmail end of a Sendmail-Milter conversation. Milter is Sendmail's API for email filters that can test if an email is spam or a virus, or whatever. While Sendmail::Milter exists if you want to write an email filter in Perl, nothing existed that allowed you to query a Milter filter unless you were Sendmail itself. Hence you could only query Milter filters through Sendmail.

Net::Milter changes that, you can now query a Milter filter through an easy Perl interface. Sure the software is only alpha tested at the moment, and as far as I'm aware only works in my development environment with one Milter filter, but I'm certain others will test it and use it. If anyone finds any filters or environments where it doesnt work, tell me and I'll look into it.

Martin