Here is an interesting one-liner that has been on a stickie (a Mac OS thingy) on my desktop for a long time. Maybe it stayed there so long because it wants other people to see it.
I use procmail to filter my mail (I still think it is easier to use than Mail::Audit or whatever else is out there---I have meager filtering needs). I go through my read-mail files (mbox format), pull out lines that start with "From:", sort those and pull out the unique entries, then fix up the line, which is still the entire line, with a perl quickie. It works amazingly well for what I needed.
This is one line despite any formatting
grep -h ^From: read-mail* | sort | uniq | perl -p -e 's/^From:\s+//; s/\s+\(.*?\)//g; s/.*<(.*)>/$1/;' > goodfile
Once I have that list of email addresses, I add them to my list of people who do not end up in my might-be-spam folder.