A digest and compilation of links resulting from a short research I did concerning anti-spam and mailing list management.
It is tackling with:
— legislation (US and Europe)
— mailing list management best practices
— anti spam and anti virus software
— tips, tools, and techniques to be aware of
That's meant to help the team I am working with to define the standards for emailings.
Note that It is a bit "raw" (collection of online references)...
DefinitionsGlossary
LegislationOverview
OnlineToolsAndGuidelines
BounceManagement
E-MailAddressObfuscation
IspRules
IndustryCollaborationInEmailAuthentication
AntiSpamVirusSoftware
Spam: “Any massive flood of drivel which serves to flood a communications channel, reduce the signal-to-noise ratio and annoy the hell out of a large number of people.
The word comes from an old Monty Python skit where some folks in a diner are unable to have a conversation because a group of Vikings at a nearby table keep singing the "Spam" song. (This is a gross oversimplification of the skit, but covers the important point.)
The term became connected with computers in 1985 when somebody harassed one of the original Pern MUSHes by echoing:
SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM SPAM
on all their terminals every few seconds until they booted him.”
Source: http://www.rahul.net/falk/glossary.html#spam
“Spam is unsolicited bulk e-mail (UBE). From the sender's perspective, spam is an extremely efficient and cost-effective way to distribute a message, but to most recipients, spam is just junk e-mail. Spammers typically send a piece of e-mail to a distribution list in the millions, expecting that only a tiny number of readers will respond to their offer.
The term is said to derive from a famous Monty Python sketch ("Well, we have Spam, tomato & Spam, egg & Spam, Egg, bacon & Spam...") that was current when spam first began arriving on the Internet. SPAM is a trademarked Hormel meat product that was well-known in the U.S. Armed Forces during World War II.
According to a report from the Spamhaus Project anti- spam organization, over 90% of all the spam received in North America and Europe originates from only about 200 senders. Most spam falls into the category of unsolicited commercial e-mail (UCE), but the term also encompasses other types of mass mailings, such as e-mail chain letters, personal campaign mailings, messages with virus-laden attachments, and messages containing virus hoaxes, among other possibilities.
Source: http://whatis.techtarget.com/definition/0,,sid9_gci931780,00.html
"Spam is a major annoyance these days. Legislation is the only thing that will bring spam to an end"
Source: http://gmane.org/tmda.php
*UCE* means Unsolicited Commercial Email
*UBE* means Unsolicited Bulk Email
More vocabulary concerning Spam: http://www.rahul.net/falk/glossary.html
More to learn about email spam: http://en.wikipedia.org/wiki/E-mail_spam
Spam is a pricey pest (spam by the numbers):
o $30 billion -- What spam is costing businesses this year in infrastructure expenses
o $113 billion -- What spam will cost businesses worldwide by 2007
o 52% -- Number of businesses who say reducing spam is a key messaging priority
(Source: Radicati Group, May 2003)
back to summary
In the UK, the Advertising Standards Authority has recently introduced some guidelines, making sure advertisers have prior consent, that advertisements should be clearly marked and contain suitable content etc. Sadly, most UCE does not originate from the UK and therefore will rarely adhere to the ASA guidelines. The European Union has recently voted to ban unsolicited email. It is proposed that advertisers will have to have recipients' consent before sending email. The details of this 'opt in' scheme are in the process of being finalized and adopted by all member countries.
In the US, some states have laws against the sending of unsolicited email. New anti-spam legislation is currently being drafted for other parts of the US. However due to the nature of the Internet it is likely that the perpetrators will simply move their servers. Many of the "spammers" will hide their activities by sending emails via unsecured machines they find elsewhere on the Internet. Regardless of legislation, virtually all ISPs have an Acceptable Usage Policy (AUP), which should prohibit the sending of unsolicited email. It therefore makes sense to report a "spammer's" activities to their ISP in the first instance.
Anti-Spam Laws and Authorities Worldwide
Reference: http://www.itu.int/osg/spu/spam/law.html
The US Model:
USA essential reference:
http://www.spamlaws.com/
United States – Federal Trade Commission
On 1 January, 2004, the Can-Spam Act , which stands for “Controlling the Assault of Non-Solicited Pornography and Marketing Act”, came into effect in the United States. This law puts specific requirements on senders of commercial e-mail and places enforcement in the hands of the Federal Trade Commission and State Attorney's General.
European Union - Information Society
The European Commission (EC) has identified four directives that are relevant in regulating Spam. The Directorate General Information Society is responsible for the implementation of the following legislation, while enforcement is a responsibility of each Member State.
Alternate online reference for European rules:
http://www.spamlaws.com/eu.html
Italy - Garante Per La Protezione Dei Dati Personali
http://www.garanteprivacy.it/garante/navig/jsp/index.jsp
Italy has enacted a tough anti-spam law that makes spamming a criminal offence and is punishable by up to three years imprisonment. The Italian Data Protection Authority is an independent agency created to ensure personal data protection and deal with Spam problems.
France - Direction Du Développement Des Médias - Commission Nationale Informatique Libertés (CNIL)
http://www.itu.int/osg/spu/spam/legislation/legislation_france.html
The Direction du Développement des Médias (DDM) - under the authority of the Office of the French Prime Minister - is in charge of regulatory reform in the field of communication and online services. The DDM provides several documents and information regarding anti-spam legislation and activities, and in July 2003 established a Contact Group to fight spam.
The competent enforcement agency is the Commission Nationale de l'Informatique et des Libertés (CNIL), an independent administrative agency which enforces the Data Protection Act enacted in 1978 and other related laws. In July 2002 the Commission created a Spam Mailbox, a reporting mechanism for spam emails, to help combat this scourge.
Another body involved in the fight against spam is the Générale de la Concurrence, de la Consommation et de la Répression des Fraudes (DGCCRF), which deals with fraud and scams perpetrated through spam messages. However, thus far, the DGCCRF has not taken any concrete action in this area.
Spain - Spanish Data Protection Agency:
https://www.agpd.es/index.php
The Agencia Española de Protección de Datos (AEPD) is the independent Spanish Data Protection Authority that was set up in 1992 by the first Spanish Data Protection Act. After the implementation into Spanish Law of the Directive on privacy and electronic communications (2002/58/EC) through the General Telecommunications Act and Services of the Information Society and Electronic Trade Act, the AEPD is the competent body to supervise compliance and enforce the anti-spam legislation in Spain.
United Kingdom - Information Commissioner Office
http://www.informationcommissioner.gov.uk/eventual.aspx?id=5801
The UK Department for Trade and Industry (DTI) implemented the new anti-spam regulation, based on the EU Directive 58/2002 (pdf), with the Privacy and Electronic Communications (EC Directive) Regulation, which came into force on 11 December 2003. The enforcement of this new instruments is under the responsibility of the Information Commissioner, however considering that several issues relating to spam concern also consumer protection and trade, the Office of Fair Trading is also active in this field, in particular on the subject of online scams.
On 2 July 2004 the United Kingdom’s Office of Fair Trading, the United Kingdom’s Information Commissioner, Her Majesty’s Secretary of State for Trade and Industry in the United Kingdom signed with agencies from the United States and Australia, a Memorandum of Understanding for mutual assistance in the enforcement of spam laws.
Portugal - National Communication Authority:
http://www.anacom.pt
The Portuguese legislation against spam was implemented in the wider framework of the national discipline to regulate electronic commerce. The recently approved Decree-Law 7/2004 aims mainly to transpose Directive 200/31/EC, and includes a chapter ("network advertising communications") incorporating the main dispositions of Directive 2002/58/EC relating to unsolicited communications.
Unsolicited communications sent in violation of the law shall be deemed as an offence punishable with a fine that may range from $2500 to $50000, in case of natural persons, or from $3333.34 to $66666.67, in case of legal persons, together with other additional sanctions.
back to summary
Guidelines:
Guideline for proper mailing list management:
http://www.mail-abuse.com/an_listmgntgdlines.html
Other guidelines from the same source:
http://www.mail-abuse.com/library.html
Tutorials on Reading Email Headers:
http://www.stopspam.org/email/headers.html
http://www.doofus.org/spam/lessons/
Understanding email headers:
http://www.by-users.co.uk/faqs/email/headers/#Mail
RFC2822 compliant header :
http://www.faqs.org/rfcs/rfc2822.html
Sites providing various network tools which can help identify the originating ISP:
http://www.samspade.org/
http://www.geektools.com/
Other Look-up tools
http://www.mail-abuse.com/lookup.html
Report spam online tool:
http://www.spamcop.com
Abuse Contact Database which provides the contact address for a large number of domains:
http://www.abuse.net/lookup.phtml
Information on Virus Hoaxes and Chain Mails
http://hoaxbusters.ciac.org/
http://www.vmyths.com/
Virus information (from the major vendors):
http://www.symantec.com/avcenter/
http://www.sophos.com/virusinfo/
http://www.viruslist.com/eng/
http://vil.nai.com/vil/
back to summary
*Bounce (non-delivery report) management:*
A bounce is a notification that your message, for whatever reason, didn't make it to the recipient. Ideally, these bounces take the form of SMTP [Simple Mail Transfer Protocol (RFC821)] codes, defined as a standard in RFC821. Using these codes, ISPs can communicate the reason for the bounce. Not everyone follows this standard, however, and accurate bounce handling may involve some keyword review of the replies.
Distinguish between hard and soft bounce:
A *hard bounce* means either the receiving server purposely rejected the message or the receiving server doesn't exist. Examples of hard bounces are:
* The user doesn't exist at the domain.
* The domain doesn't exist.
* The message was rejected.
A *soft bounce* typically denotes a temporary error with delivery and may be any response other than a hard bounce. Examples of soft bounces are:
* The e-mail server isn't responding.
* The user's mailbox is full.
List management software that also performs bounce management:
(There is an automated bounce handling in Lyris but I don't know how sophisticated it is)
Sympa : http://www.sympa.org
Boogietools : http://www.boogietools.com/products
B-bounce : http://www.bbounce.com
Inxmail : http://www.inxmail.com/products
back to summary
Combating Email Harvester Robots using ISO, Hexadecimal and Mixed Output Email Obfuscation: http://www.seowebsitepromotion.com/obfuscate_email.asp
A very interesting paper:
Why Am I Getting All This Spam?
Concepts:
spam, web sites, receiving, newsgroups, Internet users, preferences, providers, attacks, harvesting, commercial e-mail, amount, privacy, posting, online, spammers.
Summary:
Why Am I Getting All This Spam?
In the summer of 2002, CDT (Center for Democracy and Technology) embarked on a project to attempt to determine the source of spam.
To do so, we set up hundreds of different e-mail addresses, used them for a single purpose, and then waited six months to see what kind of mail those addresses were receiving.
Our analysis indicated that e-mail addresses posted on Web sites or in newsgroups attract the most spam.
In our study, we discovered that most newsgroup-related spam is sent to the address in the message header, even if other e-mail addresses are included in the text of the posting.
For the most part, companies that offered users a choice about receiving commercial e-mails respected that choice.
The second-greatest amount of spam we received was from public postings to USENET newsgroups.
Once again, neither the "human-readable" nor the "HTML-obscured" e-mail addresses received any spam.
We tested two different kinds of opt-out: first, opt-out immediately after opting-in (simulating a consumer changing his/her mind immediately about his/her privacy preferences), and second, opt-out two or more weeks after the initial opt-in (simulating a consumer changing his/her mind after some time).
For the majority of Web sites we encountered no difficulty and found that "opt-outs" were respected within the two-week grace period our methodology provided.
Finally, at one point in the project our mail system began receiving spam messages to addresses that had never been used for any purpose, had been submitted to no one and, in many cases, did not even exist.
Even when an e-mail address has not been posted or shared in any way, it is still possible to receive spam through various "attacks" on a mail server.
back to summary
Best Current Practice for combating Unsolicited Bulk Email published by LINX (UK based ISP consortium):
http://www.linx.net/noncore/bcp/ube-bcp-v2_0.html
back to summary
Industry Collaboration In Email Authentication:
Source: http://truste.org/about/authentication.php
4 organisations and endeavors identified :
SPF (Sender Policy Framework)
Cisco's Identified Internet Mail
Microsoft's sender ID framework
Yahoo domain key
And an interesting article:
I'm Sick and Tired Of Spam (Filters)
Concepts:
filters, spam, ISP, publishers, SpamAssassin, subscribers, newsletter, opt-in, filtering rules, complains, deliveries, junk e-mail, administrators, trigger, catching.
Summary:
Because I have no interest in self-censoring this column, odds are high that some of my e-mail subscribers will not receive it -- or it will end up in their "junk mail" folders.
If you never use such "controversial" words in your opt-in newsletters (that is, e-mail publications that subscribers have asked to receive free, or paid to receive), the odds are much higher that your subscribers will receive your messages.
As an ISP law expert told me, new anti-spam efforts increasingly are "catching more dolphins in the nets along with the tuna."
-- although it's typically affected only a small percentage of an e-newsletter's subscribers.
But it has become a more serious problem recently because of the popularity of a spam solution called SpamAssassin, which is increasing the amount of opt-in (that is, ethical, non-spam) e-mail that is being blocked along with the spam.
What makes it an effective spam-killer is that it features hundreds of filtering rules from which the software analyzes e-mail -- not just subject lines and e-mail headers, but the content of a message -- and determines if it might be spam.
"ISPs are blocking content in a very bone-headed way," he says, looking for keywords that spammers typically use -- but that ethical publishers also may use from time to time.
Some corporate servers simply delete mail that they think is spam, so unless a subscriber complains about not receiving a requested newsletter, the publisher is left in the dark about there being a problem.
back to summary
Anti Virus:
Most popular one for list server management: clamav virus scanner
Anti-Spam Mail Filter:
SpamAssassin
Performs a bunch of tests to see whether the mail looks like it's spam. In addition to the normal pattern checks, this includes *RBL checks* and statistical Bayes classification
*RBL checks:*
An abbreviation for Realtime Blackhole List, a tool for blocking Internet access to known spammers, maintained by the Mail Abuse Prevention System http://www.mail-abuse.org , or MAPS, of Redwood City.