The New York Times yesterday reported that fake reviews are a growing trend. The story concentrates on tourism and product review sites.

Product owners, marketing agencies or individuals can buy reviews online for a small amount of money. If you visit Fiverr for instance, you will notice that you can buy positive reviews for $5 on almost every site imaginable. But Fiverr ist just one of the sites where you find people willing to put up fake reviews on websites.

The ingenious aspect of this is that hiring people to post fake reviews bypasses most of the site’s fake detection security. If you were to do it on your own, you would connect with certain characteristics like the computer’s IP address, browser version or operating system which might be used to identify manipulation, even if proxy servers or virtual private network connections were used. A single cookie could be enough for that.

But with unique users from all over the world, it is not possible to use hard facts to identify fraud.

Cornell researchers recently published a paper about fake review detection. The algorithmic approach looks for strong and slight deceptive indicators in a review to determine whether it is fake or not. Indicators on the other hand are not proof, and it will happen that the algorithm detects legit reviews as fake and vice verse.

Some sites could implement a better procedure to avoid the majority of fake reviews. Amazon for instance could only allow reviews by users who have purchased the product on their site. While that would certainly reduce the number of reviews on site, it would eliminate the majority of fake reviews as well.

Businesses who use these marketing techniques will adapt. They would simply have to do some initial teaching, or review writing of their own, to deceive the algorithm.

One element that has not been mentioned yet, and that has not been addressed in the paper, is the option to write fake reviews with a less than perfect rating. I personally read the negative reviews first on most sites to get an understanding of what’s wrong with a product. Some complaints here are less serious than others. A picky user might complain that the product arrived late, or that the breakfast buffet at the hotel did not have enough carrots on one day. Those might be serious issues for them, but they might not be serious to the majority of potential customers.

My guess is that we will see better fake reviews in the coming years. We will see fake reviews with less-than-perfect ratings, and fake reviews that use the findings of the research paper to avoid detection.

Your take on fake reviews on the Internet? Let me know in the comments.

Wallace denies the charges which carry a jail term of up to 10 years and has been released on $100,000 bail by the authorities.

Prosecutors have said that Wallace earned “substantial revenue” from selling the personal data harvested by his app which propagated by posting itself to the walls of the friends of victims.  The spam was sent, and the personal data harvested between November 2008 and March 2009.

Facebook successfully sued Wallace in 2009 and a federal judge ordered him not to access their service.  Prosecutors are claiming this is an order Wallace ignored and violated on countless occasions.

Facebook is not the only service to have been hit by Wallace.  In 2008 he lost a civil prosecution brought by MySpace for sending junk messages on their network.

Facebook spam and malware apps is clearly a growing problem that isn’t going away.  Malware writers and criminals are taking advantage of the lack of knowledge most computer users have about what the threats to their personal data are, and how to look for them.

It is getting more and more common to see fake videos posted to walls, apparently by friends, with subjects such as “Daddy walked in on her” or “World’s worst hen night prank”.  The simple rule to follow with video is that the video, when clicked directly, will play in the wall view.  If it takes you to another page, even if it looks like Facebook and asks you to click to allow it permission it’s malware.

This is the same for all other malware links on the service.  If you suspect you have already authorised malware on your Facebook account follow these simple instructions to remove them.

1. Click on Account in the top right of the Facebook screen
2. Next click on Privacy Settings
3. Under Apps and Websites click Edit your settings
4. On the next page in the Apps you use section click Edit Settings
5. Here you can revoke permissions for all but the most essential Facebook apps that you like to use

It is also wise never to include your home address, home telephone number or mobile telephone number in your profile as this is information that is most valuable to spammers.  If your friends want to know your personal details they’ll always ask you in a secure direct message and you can tell them directly.

It is also wise to check your general Facebook privacy settings with you can do in Account > Privacy Settings.  Here you can see if your personal information is shared just with your friends, their friends or with everybody on Facebook.  Any information shared publicly will also be visible to search engines and could include sensitive information about you.

It is because Facebook have tightened privacy controls in the last year that we’re seeing more and more malware apps that want permission to access your personal information.  Giving an app permission is the same as making the writers of that app a friend, as they will then have access to all the information about you that you put on the social network, including photographs and status messages.

By far the safest way to protect yourself on Facebook is not to put sensitive and personal information there in the first instance.

A recent study of the University of California, entitled Click Trajectories: End-to-End Analysis of the Spam Value Chain comes to a similar conclusion, albeit from a different point of view.

95% of spam-advertised pharmaceutical, replica and software products are monetized using merchant services from just a handful of banks.

According to the university’s study the most effective approach of taking down botnets is to stop the money flow at the bank level.

Considering that it is only three banks that “provide the payment servicing for over 95% of the spam-advertised goods in [the] study” it is safe to say that payment processing is the biggest bottleneck in botnet operation.

The researches analyzed other possible bottlenecks, domain registrars and hosting companies for instance, but came to the conclusions that this angle was not as effective as the payment processing angle:

For example, while only a small number of individual IP addresses were used to support spam-advertised sites, the supply of hosting resources is vast, with thousands of hosting providers and millions of compromised hosts. The switching cost is also low and new hosts can be provisioned on demand and for low cost.

By contrast, the situation with registrars appears more promising. The supply of registrars is fewer (roughly 900 gTLD registrars are accredited by ICANN as of this writing) and there is evidence that not all registrars are equally permissive of spam-based advertising. Moreover, there have also been individual successful efforts to address malicious use of domain names, both by registries (e.g., CNNIC) and when working with individual registrars (e.g., eNom). Unfortunately, these efforts have been slow, ongoing, and fraught with politics since they require global cooperation to be effective (only individual registrars or registries can take these actions). Indeed, in recent work we have empirically evaluated the efficacy of past registrar-level interventions and found that spammers show great agility in working around such actions. Ultimately, the low cost of a domain name (many can be had for under $1 in bulk) and ease of switching registrars makes such interventions difficult.

When it comes to payment processing and banks, the researchers concluded:

Finally, it is the banking component of the spam value chain that is both the least studied and, we believe, the most critical. Without an effective mechanism to transfer consumer payments, it would be difficult to finance the rest of the spam ecosystem. Moreover, there are only two networks—Visa and Mastercard—that have the consumer footprint in Western countries to reach spam’s principal customers. While there are thousands of banks, the number who are willing to knowingly process what the industry calls “high-risk” transactions is far smaller. This situation is dramatically reflected in Figure 5, which shows that just three banks provide the payment servicing for over 95% of the spam-advertised goods in our study. More importantly, the replacement cost for new banks is high, both in setup fees and more importantly in time and overhead. Acquiring a legitimate merchant account directly with a bank requires coordination with the bank, with the card association, with a payment processor and typically involves a great deal of due diligence and delay (several days or weeks). Even for so-called third-party accounts (whereby a payment processor acts as middleman and “fronts” for the merchant with both the bank and Visa/Mastercard) we have been unable to locate providers willing to provide operating accounts in less than five days, and such providers have significant account “holdbacks” that they reclaim when there are problems.21Thus, unlike the other resources in the spam value chain, we believe payment infrastructure has far fewer alternatives and far higher switching cost.

The study, available here as a pdf document confirms that the most effective way of seriously impact the operation of botnets is at the payment processing level.

Even users who are very careful with their primary email address may end up with spam in their inbox. Think of friends who upload their email contacts list to a service like Facebook, or computers that get hacked.

The following guide shows you how you can block most of the spam before it reaches your inbox. It requires some configuration and maintenance on the other hand if done right.

The technique uses a secondary account as a spam blocker. You basically setup a secondary account at a free hoster, Gmail or Hotmail l for instance and configure forwarding rules for all the emails that you want to receive in your primary account.

Signing Up For A New Email Account

Please note that it may not be necessary to sign up for a new account. Hotmail for instance supports the creation of fully independent email aliases which can be used for the purpose. Lets say we have made the decision to create a new Gmail account. Visit the Gmail sign up page http://mail.google.com/mail/signup/ and create the account.

Fill out all the details on the page until the account has been setup. You should have two email accounts by now.

Configuring the forwarding and spam blocking

It is theoretically possible to use the second account from now on for everything web related. This would mean that you need to access the account regularly to check for new emails. A more comfortable solution would be to configure email forwarding and filters to forward all legit emails to your real email account, and to block everything else that is spam.

Before you can create filters to forward emails in Gmail you need to allow your primary email address to be set as a forwarding email address. Click the settings icon in the upper right corner and select Mail Settings from the options.

Now switch to the Forwarding and POP/IMAP tab and click the Add a forwarding address button.

Here you enter your primary address.

A confirmation message is send to the email address. You basically need to click on the link to accept the forwarding. Now you have two options to create filters to forward messages. The easiest option is to create filters if you already have emails in the account that you want to forward. Click on the Gmail inbox and then a message that is not spam that you want to forward to your primary account. Click the More Options button and there the Filter messages like these link.

This opens a new page with a listing of all emails that match the sender email address. You can modify the settings if you like which is necessary if you want to accept some messages but not all from a sender. You can add filters for several parameters, including subject, words included or not included or recipient.

Click on the Next Step button after the process. Place a checkmark in the Forward it to box, the forward email should be the one that you have added to the Gmail account earlier.

A click on Apply Filter adds the filter to the Gmail address. All new emails that match the filter are from that point on automatically forwarded to your primary email address. You can check the Also apply filter to x conversations below to forward all existing matching emails as well.

You now need to repeat that process for all legit emails. It is a one time process that takes some time to setup. If you do not have emails in your inbox you can create filters via Mail Settings > Filters. You do however need to know the sender address or another characteristic of the emails that you want to filter.

Two options are available for sending emails with the second client. Desktop email users can add the secondary email address to their mail program. Web mail users on the other hand need to load the web mail interface if they want to send emails.

Hotmail

It gets a little bit complicated if you want to apply the method to a secondary Hotmail account. You first need to create a Windows Live Hotmail account, the sign up url is accessible here. Existing Hotmail users can alternatively add an email alias to their account. Log into the account once it has been setup and click on Options > More Options in the upper right corner.

Click Filters and Reporting from the available options and switch from the standard junk mail filter to exclusive. This sends all emails but whitelisted and official Hotmail emails to the junk.

Now click on Safe Senders, select Safe Senders again on the next screen and add email addresses that you consider safe. You may need to look in the junk mailbox at the beginning as the majority of emails will be moved automatically to it.

Go back to options once you are finished. You need to add the forwarding email address in the last step. Click on Email Forwarding in the options menu, select Forward your mail to another email account and add your primary email to the form.

You may want to check the Keep a copy of forwarded messages in your Windows Live Hotmail inbox. A click on Save saves the new forwarding information. All inbox emails are from that moment on forward to your primary email address.

Closing Words

Setting up a secondary email address with whitelist forwarding is an effective but time consuming way of protecting your primary email account from landing in spam databases. It is however sometimes necessary to add new filters to the whitelist from time to time, for instance after signing up for a new service that sends out emails regularly. Account confirmation emails and the like on the other hand do not necessarily need filters setup.

You see, every site that gets blocked is submitted to Google. Google engineers will “study the resulting feedback” and “explore using it as a potential ranking signal” for Google Search.

As a webmaster, I see the danger of it, more than I see the benefit. What’s keeping a company from hiring an army of Indian users to block websites of companies that rank before them in Google Search? It won’t be long before the first service offers appear on webmaster forums that offer a service like that: 100 blocks for $10, 1000 for $75. Oh and before you start wondering, the topic is already discussed in popular Black Hat forums.

Anyway, another interesting question is why it is only offered for Google Chrome. Google could very well have added an option to block sites directly into Google Search.

Possible explanations include that Google may receive additional information from the Chrome browser that they would not receive if the blocking would be available directly on the search engine pages, and that it would be available everywhere providing that it would be linked to a Google account. Another that a Google engineer created the blocking extension during free time.

If you look at user comments over at the Chrome store you notice that many have blocked experts-exchange immediately and not “content farms” such as ehow, about, or Demand Media which are pumping out thousands of articles per day.

The extension lacks several features like syncing or the ability to import and export blocklists. Another negative aspect is that the Personal Blocklist extension only removes or blocks entries from the search engine which means that fewer results appear per page.

The extension can be beneficial purely from a user perspective but it is not the first that does it (see Blacklist Google Search Results In Google Chrome). If the blocking is indeed used by Google as a ranking signal then people will start abusing it for their personal gain.

List of blogs that do not verify comments are traded on Internet marketing forums, services and websites. Webmasters fight a constant battle against spam. When I started Ghacks six years ago it was a small site, and I was able to verify all comments, trackbacks and pingbacks manually.

But a lot has changed since then. More and more tools that automate the commenting have been created, to the point that everyone can use them without problems. All that is needed is a big list of blogs that accept comments and that’s it.

Advanced spammers use proxy lists, virtual private networks, unique comments with variables (for instance the authors name included) and more to improve the chances that the comments are accepted on the spammed sites.

Ghacks today receives more than 900 spam comments per day. That still may not sound as much considering that other blogs may very well receive tens of thousands of comments per day. I moved from manually checking every comment to only checking comments sporadically, mainly because of time constraints.

Ghacks has received its 1 Millionth spam comment in this month, another blog milestone. Akismet did not record spam from the very beginning which means that the actual figure may indeed be a lot higher than the one reported. The one million mark has been reached in the last three years.

I have tried a lot to reduce the amount of spam but have not found one reliable option to block spam before it reaches the blog. Tried a lot but nothing worked, or interfered with regular commenters who started reporting troubles.

Next to the one million spam comments are more than 79k legit comments, a ratio of 1:12.6 which I think is incredible.

Are you a webmaster? How do you cope with spam?

New reports came in shortly after I wrote the article that the Rustock botnet, which has been responsible for as much as 48% of all global spam and which went suddenly and inexplicably silent in December has sprung back into life.

Overall, the level of spam sent worldwide is still down considerably on previous levels and there is still no apparent reason for this.

In an interview with the BBC, Alex Cox of NetWitness said ”As best we can tell, they took a holiday, The people running Rustock are running a business – albeit an illegitimate one – so maybe they needed time off too.”  This was the best guess anyone so far has been able to offer.

Rustock was expected to have sent out 67 billion spam emails yesterday, more than doubling the amount sent worldwide the day before.

While Rustock has restarted its activities, it is too soon to say, according to security experts, whether spem levels will again reach the volumes we saw back in August.

The fall, which you can see in the graph below, shows a steady decline from almost a quarter of a trillion messages every day to just 50 billion now.  The largest drop was seen over the Christmas period when the total volume of spam halved in just a few short days.

Security experts are warning that the lull may not last though they are at a loss to explain why the global spam levels have dropped so far and so regularly in recent months.

While authorities, especially in the US which generates the most spam worldwide, have had great success in the last year closing illegal operations, these were a drop in the ocean overall.  According to a report by the BBC, Botnets are responsible for the majority of spam and the largest of these, Rustock, was at its peak responsible for up to 48% of all global spam.  By December however Rustock was responsible for only 0.5% of global spam.

Around the same time two other global spam botnets also went quiet.

While we have seen global drops in spam before, it is uncommon for a drop to last so long.  It will be interesting to see if the levels rise again or if they will continue to drop for the next few months.

But what can you do if you receive spam? This guide looks at a few possibilities. It will not cover ways to prevent spam in the first place, for that you need to look elsewhere. A few pointers are temporary email addresses and a secondary email address for untrustworthy sites and communications.

Solution 1: Getting rid of the email address

If you do not really need the email address, or have only a few contacts, then you may want to consider ditching the email address that is receiving the spam and creating a new one.

That can be highly problematic because..

• You need to inform contacts of the change.
• You need to change the email on websites and services which can lead to spam to the new email address

Deleting an email address is usually not an option, especially since you cannot guarantee that the new email address will not receive spam as well.

Solution 2: Secondary Email address

A good solution is to create a secondary email address without deleting the first. Communicate the secondary email address to friends and contacts so that they use this new email address to communicate with you Make sure you only use the email for select contacts and not websites.

There are still chances that your email will land in the email pool of spammers. One example are friends who upload their email address book to social networking sites to find friends easier. Another possibility is a compromised computer of a friend or a hacked server on the Internet.

A secondary email address may help but you could also end up with two email addresses that receive double the amount of spam.

Antispam software

So called antispam software can block spam before it lands in the inbox. This reduces the amount of spam the user has to deal with. False positives can be a problem though, nothing’s worse than having to realize that important business emails have landed in the spam folder for the past couple days.

If you make use of antispam software you need to regularly check the spam folders to make sure that no false positives have been placed there.

Select antispam applications offer advanced features. Spamfighter for instance uses language recognition to automatically block emails that are written in select languages (or in all languages except those that are whitelisted by the user).

Spam Filtering and whitelisting

The goal should be to spend as little time as possible dealing with spam. A solid option to deal with spam is to whitelist senders. Blacklisting has the disadvantage that it is a regular task. Every new wave of email spam needs to be blacklisted.

Whitelisting on the other hand is a task that is done once, and then only when new contacts need to be added to the list. This means less work is involved in maintaining the list.

What you should not do

Some spammers add unsubscribe links to their email messages. Never ever use those links. If you do the spammer knows that the email address is valid. While legit companies will remove you from their list if you opt out, spammers will do the opposite since they have now verified that the email address is actually in use.

It goes without saying that you should not reply to spam emails as well as it has the same result.

Verdict

Spam is everywhere and users have to cope with it. Most email addresses will be used by spammers eventually and there is little one can do about it. You can limit the exposure but the chance is high that even careful users will end up with spam in their inbox.

Let us know how you cope with spam in the comments.

The scam, reported by the BBC, tricks users into installing a rogue application that then posts spam messages to all their contacts.  The spam messages then containing links through to malicious websites.

The messages will try to get your attention by using messages such as “OMG, shocking video” and they appear to come legitimately from a friend.

The rogue application takes advantage of the fact that many users don’t properly understand their privacy settings on Facebook and will not know how to deactivate the app later on which, by the way you can do by clicking on “Account” in the top right corner of the window then “Application Settings” and pressing the “x” next to the offending app.

Once a user has installed the app it then posts a message to their profile along the lines of “I just got the dislike button, so now I can dislike all of your dumb posts lol!!!” in order to try and tempt their friends to install the app too.

You should always be careful what apps you install in Facebook and if something looks too good to be true, it probably is!

Fortunately there is a tool for that. The tool? Spamassassin. Spamassassin is a very versatile SPAM tool that is part of the Apache Foundation. Spamassassin uses numerous means to detect SPAM including: DNS and Checksum based SPAM detection as well as Bayesian filtering, external programs, black lists, and online databases. These tools together make for a fairly powerful detection system.

In this article you are going to see how to install and configure Spamassassin to work in conjunction with Postfix to further enhance your email server.

Installing Spamassassin

Obviously the first thing you need to do is install Spamassassin. You will find Spamassassin in the Ubuntu repositories. And since this entire series has been laid on top of a Ubuntu Server installation, that is quite convenient. So, to install Spamassassin, open up your terminal window and issue the following command:

sudo apt-get install spamassassin

There may or may not be some dependencies to install in order for the Spamassassin  installation to complete. Go ahead and OK those. Once this installation is complete you are ready to start configuring.

Configuration

Before we actually get to the configuration it is important to understand the SPAM scoring system. With Spamassassin, messages are tagged as SPAM only when they have enough SPAM-matching characteristics (according to a scoring level). The scoring level is 0-5, however it’s not as simple as saying a 0 means it is 0% SPAM. The system is set up so that every characteristic can add to the overall score. For example a message tested to find a base64 attachment does not have a file name filtered with both bayes+net will add 0.224 to the over all score of the message. When all of the characteristic scores are added up, if they exceed the default score you have set in the configuration file, that message is considered SPAM.

Now that you have a basic understand of how the scoring system works. Let’s start configuring Spamassassin.

The main configuration file is /etc/spamassassin/local.cf. The first option you want to configure is the default score. Look for the line:

# required_score 5.0

The first thing you want to do is uncomment that line (by removing the “#” character) and then changing the score. A score of 5 is pretty high and sure to be SPAM. Understand the more you lower that score the likely you are of missing message messages that are tagged false-positives. A score of 3.5 is a fairly reliable score that will catch a lot of SPAM but not a lot of false positives.

Above this line are a couple of other options that are important. The first is the option to set the option:

report_safe

To 0. This option can be set to either 0 or 1. A zero means that if a message is found to be SPAM the message will not be deleted, but instead the subject line will be rewritten to include a message marking it as SPAM.  This is handy to prevent users from losing important messages to false positives. This also allows you to set a lower score threshold.

To do this first look for the line:

# report_safe 1

Uncomment this line by removing the “#” character and then change the “1″ to “0″ (no quotes).

The next step is to uncomment the line:

# rewrite_header Subject *****SPAM*****

Now you can alter the “*****SPAM*****” section of this line to reflect what you’d prefer it to say. Just make sure it is clear to your users that a message with this rewritten subject line is most likely SPAM.

Now restart the Spamassassin daemon with the command:

sudo /etc/init.d/spamassassin restart

Configure Postfix

The last step is to set up Postfix to use Spamassassin. To do this open up the file /etc/postfix/master.cf and look for the line:

smtp     inet    n   –   –   –   –   smtpd

You need to alter this line to look like:

smtp      inet   n   -   -   -   -   smtpd -o content_filter=spamassassin

Finally, at the end of this file add the following:

spamassassin
unix - n n - - pipe
flags=R
user=spamd
argv=/usr/bin/spamc
-e /usr/sbin/sendmail
-oi -f ${sender} ${recipient}

Now all you need to do is restart Postfix with the command:

sudo /etc/init.d/postfix restart

Your mail server should now be scoring incoming message as SPAM or HAM.

Final thoughts

The mail server is a tricky beast. You have to ensure that users are getting their mail, but you have to make sure they aren’t receive SPAM or viruses. After completing this series of articles, you should have a pretty solid server running that will send out mail that is safe for users eyes.

Most email clients contain some sort of SPAM prevention. On the Linux operating system you can employ such tools as Bogofilter or Spamassassin. Either of these tools are great for stopping SPAM – but what about using them with your favorite email client? With some Linux email clients you have to add extra applications in order to have one of these SPAM filters doing their job with your client. Some clients, however, have built-in tools to save you a little work. KMail belongs to the latter category. So long as you have either/or Bogofilter or Spamassassin installed, you are one step closer to having solid SPAM filtering.

In this article you will learn how to set up reliable SPAM filtering in the KMail mail client.

Bogofilter and Spamassassin

KMail can use either Bogofilter or Spammassassin easily. Read up on either tool and decide which application you want to use. I have had good luck with both, but I do believe Bogofilter is easier to set up. And since KMail will automatically detect either, the choice is yours.

You can install either tool from within the Add/Remove Software utility by following these steps:

1. Open Add/Remove Software utility.
2. Search for either “bogofilter” or “spamassassin” (No quotes).
3. Select either (or both) tools for installation.
4. Click Apply to install.

That’s it. Now you are ready to move on to KMail.

Kmail SPAM Wizard

Figure 1

When you have KMail set up and running click on the Tools menu and select the “Anti-SPAM Wizard” entry. When the Wizard opens the first thing it will do is detect which SPAM tools you have installed (see Figure 1).

Once the wizard has detected your SPAM tools, select the tool you want to use and click Next. For the purpose of this article I will select Bogofilter.

The next step in the wizard will ask you for two configurations:

• Mark detected SPAM as read: I prefer to unset this, because of the next configuration option.
• Which folder to relocate SPAM: By default KMail will move SPAM to the trash folder. I prefer to create a new SPAM folder in order to catch any false-positives.

By moving SPAM to a specific SPAM folder (and not the trash folder) and keeping SPAM unread, you are less likely to miss any email mistakenly marked as SPAM (false-positive).

The final window of the wizard gives you a report on what you have set up. In my case the wizard creates the following filters:

• Bogofilter Check
• Spam handling
• Classify as Spam
• Classify as NOT Spam

Click the Finish button and your KMail client is ready to learn. And learning is the key.

Help Bogofilter learn

Figure 2

Before Bogofilter can become a reliable SPAM filter, it has to learn what should be classified as SPAM and what should be classified as HAM. Here’s how I like to do it. Allow plenty of email to collect in your inbox. As the mail comes in take all SPAM (that is not already marked and moved) and move it to the SPAM folder. Once you have cleared your Inbox of all SPAM, select all mail in that folder and then click the Ham button (see Figure 2).

Now, once you have collect a good amount of SPAM in your SPAM folder open that folder up, select all the SPAM, and click the SPAM button.

You may have to do this trick a few times before Bogofilter has been properly trained. You will know when Bogofilter has become accurate with your SPAM/HAM.

Final thoughts

It couldn’t be much easier to catch and mark SPAM with KMail and Bogofilter. But if you find Bogofilter not as reliable as you would like, run the Wizard and try Spamassassin instead.

One of the aspects of Claws Mail that I like more than that of either Thunderbird or Evolution, is the filter tool. I am sure you know that email filters are one of the best means to manage the enormous amount of email coming down the pipe. You can keep SPAM in the trash, filter specific email to specific folders, and take just about any kind of action you need/want.

For many, the Claws Mail filters might not be as simple to use as the standard fare. But that doesn’t mean they shouldn’t be used.  With this tool, it is just a matter of getting used to a different interface. And that is what this article will help you with.

The tools

There are two ways to create a filter with Claws mail: The easy, less configurable method, or the harder, more configurable method. I will only briefly mention the easy method as that is, well, easy. The second, harder, method is what we will primarily focus on.

The easy method

If you have an email selected, click on the Tools menu and then select “Create Filter Rule”. When this new sub-menu pops up you will see four entries:

• Automatically
• By From
• By To
• By Subject

That will define the condition of the filter automatically for you. You will still have to configure the rest of the filter, but this step automates a portion of the creation.

Now let’s take a look at the full creation of a filter by doing things the hard way. I shouldn’t scare you off by saying “hard way”…it’s not really that difficult.

Filter from scratch

Figure 1

When you go to the Configuration menu you will see the Filtering entry. Click that to open up the Filters window (see Figure 1).

As you can see, for the most part, creating a filter is simple. The only aspect that might not be as user-friendly as you are used to is the Conditions and Actions. For these what you have to do is is click the associated Define buttons.  When you do this you will configure the specifics of either the conditions or the action of the filter. Let’s take a look at configuring the conditions of a filter. Click the Define button associated with the Conditions. When you do this a new window will open up.

Figure 2

Figure 2 shows a condition already created. What you can not see, from the image, are the included options in the drop down lists. With different types of filters you will have different drop down lists available. The different types of criteria you can select include: Header, Age, Phrase, Flag, Color lables, etc. Say you select the Header from Match Criteria. When you do you can then select:

Name: From this drop down you can then select such familiar items as To, From, Reply To, etc.

Header: This is where you decide if your header Contains or Doesn’t Contain. Basically this is a positive or negative match.

Once you have your condition rule set you then click the Add button. If you want you can then create more rules for this one condition. Each condition can have multiple rules and you can move the rules up or down in the chain of command. You can also configure if the rule must meet all or at least one of the rules.

Figure 3

When you are finished with your Condution click OK and it is set. You are now ready to define your Action. Click the Define button associated with Action which will open up a new window (see Figure 3).

This is the easier portion of the setup. What you do is select what action you want to take from the Action dropdown and then select the associated sub-action. The associated action will depend upon which Action you choose. For instance, if you select to Move the mail you will then have to select the Destination folder. If you want to flag the message you will have to select how you want it flagged.

Once you have completed this section click OK. Now, before you finalize the new Filter you have to click the Add button in the main window. If you do not do this your newly created filter is not saved. You will get a warning if you try and then click the Continue Editing and then click the Add button.

Your filter is complete. You can test this by click the Tools menu and selecting Filter All Messages in Folder. If your folder is set up correctly you will see the changes made instantly.

Final thoughts

Claws Mail filtering is a very powerful tool that will allow you to create very helpful filters to keep your Claws Mail organized and free from unwanted email.

The second type are ping and trackback spammers. This is either done by auto-posting articles that others have written without their consent on so called autoblogs. Each post generates at least one pingback to the original site. This can lead to hundreds of pingbacks over time which many webmasters and blog owners publish.

The third and most spammy type are the automated spammers that use specifically designed tools to spam thousands of blogs, forums and every other type of site where content can be posted in hours. Now that we know the different types of spammers we have to discuss how to handle them. This post will mostly deal with automated spammers who post dozens if not hundreds of comments and pingbacks.

Webmasters should use at least one anti-spam tool to block the majority of spam reaching the frontend of the website. WordPress users can for example use Akismet or one of the several other anti-spam plugins that are available. Still, some spam will come through. Today for example a webmaster decided to copy and post more than 300 articles from Ghacks on a website. Each post generated at least one pingback, many multiple pingbacks as the tags and links were kept by that webmaster.

Akismet did not object to these ping and Ghacks ended up with more than 300 accepted and published pings from that blog. Many webmasters would now delete them one by one which takes quite a while. A far better solution is to filter for a common denominator which in this case was the url of the website. All comments, pings and trackbacks that include the domain name of that website were listed giving me the option to mark them all and send them to the spam folder.

There might be situations where this is not possible. Maybe the website or script does not offer the option to bulk moderate comments. There is however another option if the website is making use of a database. This does however require some knowledge about the database as a query has to be run in the administration. Most users will probably use MySql where a basic delete query looks like this:

delete from [table] where [column name] = 'value'

To delete all comments from a specific url in WordPress one would do the following:

delete from wp-comments where comment_author_url ='www.example.com'

It is possible to use other table columns like the commenters IP or email for example. How do you handle mass spam to your website or blog?

Project Honey Pot is a system designed with those who receive spam in mind. What this system basically does is sit on a site and watch for email harvesters. When it finds an email harvester, the honey pot logs information about the harvester into the Project Honey Pot system. This information is then built up into various sets of statistics that are used in court to prosecute spammers. One of the things that makes Project Honey Pot cool is that it shows all this data on their website for the world to see. This allows those curious about their own IPs to check and see if they are considered a spammer. It also offers information on various IPs and statistics such as the average amount of emails sent to the honey pot.

Website owners can do one of three things to help Project Honey Pot catch spammers. A honey pot can be added to any website which will watch for and log any suspicious data on that site. This is for those who have a web host and are willing to install the honey pot script onto their site. Those who don’t have their website hosted or don’t want to install a honey pot can install a QuickLink. When a bot visits a site, they likely visit other sites that the original site is linked to in order to find as many email addresses as possible. A QuickLink is a hidden, secret link that only bots can see and visit. The QuickLink will take the bot to a site that does have a honey pot installed. Another thing that webmasters can do is donate an MX record to the project. What this does is give Project Honey Pot an email address to receive spam. Project Honey Pot will use this email address to see what kind of spam the harvesters are sending among many other statistics. This option is for webmasters who have their own domain name.

Project Honey Pot is a completely free service that survives on donations and tshirt sales through CafePress. They also offer various other services such as a directory where users can look up information about various IPs, including IPs that are known to belong to dictionary attackers.

There might be white hat reasons for running a bot on a website to retrieve all the email messages found. A reason that comes to mind is to locate all emails that are posted on a public website to remove them from that website in order to protect the owner from spam.

Droid Email Seeker is one of the few free email extractors that are available for the Windows operating system. It can be used to quickly scan a website for email messages. It will begin crawling the website at the specified url and follow all inbound links that point to other pages on the same website. It will also ignore links pointing to other websites.

Another option of using the software is to let it search for keywords on Google and check the websites that turn up as results for email addresses. The number of sites can be specified. It is a little bit irritating that the process of scanning a website cannot be stopped from within the application. The only way is to terminate the program in Windows.

The email extractor software supports web proxies which it can verify. All email addresses that are found on the websites that are crawled will be added to a database. This email database can be exported as a csv file at any time. Droid Email Seeker requires the Microsoft .net Framework 2.0 to be installed on the computer system.

With a little preparation this setup will go quickly. Without preparation, it will take some time for Bogofilter to learn SPAM/HAM.

Your preparation

The easiest way to train Bogofilter is to have a collection of SPAM and HAM ready (approximately 200 each for training purposes) for it to use in the training process. These will be placed in SPAM and HAM folders (respectively) to be used once Bogofilter is installed and ready. But before you get to that point, you need to install Bogofilter and the Bogofilter Evolution plugin.

To install the necessary items open up your Add/Remove Software tool. Within this tool do a search for “bogofilter” (no quotes). You should see two particular pieces of software to install: bogofilter and bogofilter-evolution. Mark both of these applications for installation and click Apply to install them.

After they are installed you are ready to set up Bogofilter.

Bogofilter setup

The first thing to do is to enable the Bogofilter plugin in Evolution. Click on the Edit menu and select the Plugins entry. In this new window that opens click the Bogofilter check box to enable to plugin and then close the window.

Figure 1

Now that the plugin is enabled, it’s time to configure it within Evolution. Go back to the Edit menu and select Preferences. When the Preferences window opens up click on the Mail Preferences button and then the Junk tab (see Figure 1).

From within the Junk tab the first thing to do is to select Bogofilter from the Default Junk Plugin dropdown. Now go back to the top and make sure “Check incoming messages for junk” is checked. Close this window after you have completed these steps.

Now you are ready to train Bogofilter.

Training

A lot of people complain that Bogofilter doesn’t work well. The main reason it doesn’t work well is because it hasn’t been trained. Remember those SPAM and HAM emails you collected in the SPAM and HAM folders? Now you are going to use them. First click on the HAM emails. These are all good emails (not SPAM). Select all of these and mark them as Junk. I know it sounds crazy, but it’s necessary. Now click on the Junk folder and you should see all of those emails you just marked as SPAM. Select them all again and mark them as Not Junk. Bogofilter just learned what you consider to not be junk.

Now click on the SPAM folder, select all of these messages, and mark them as Junk.

That’s it. Bogofilter should now be trained properly. And remember the more training you give Bogofilter, the smarter it will be.

Final thoughts

Bogofilter is a great tool for keeping SPAM out of your inbox. And best of all, it doesn’t require you to open up a terminal and issue a bunch of commands in order to get it working properly.

By employing Spamassassin you can set up an outstanding SPAM  filter system that will protect your users. It’s not as simple as double clicking an .exe file to install, but the end results will be worth the time and effort.

Installing the tool

This portion of the task is simple. Open up your Add/Remove utility, do a search for “spamassassin” (no quotes), select the results, and apply the changes. After you have installed the package you need to take a few steps to finish up the installation.

Now you are going to need a local.rc file. You can open up the /etc/mail/spamassassin/local.rc file for hand editing, or you can use this handy web-based tool to set it up for you. This tool was written by Michael Moncur and makes the setup of Spamassassin much easier. Once you have selected all your options click the Generate button and then click the Download button. You will need to place this file in /etc/mail/spamassassin in order for it to be used.

It’s time to start the daemon. Issue the command /etc/init.d/spamassassin start (or /etc/rc.d/init.d/spamassassin start – depending upon your distribution.)

Test the install

Spamassassin comes with a sample file you can run through the filter for testing. In modern installations this file is located in /usr/share/doc/spamassassin-*/sample-spam.txt. To test this issue the command:

spamc -R </usr/share/doc/spamassassin-3.2.5/sample-spam.txt

You should see results like this:

Content analysis details:   (1000.0 points, 5.0 required)

pts rule name              description
—- ———————- ————————————————–
-0.0 NO_RELAYS              Informational: message was not relayed via SMTP
1000 GTUBE                  BODY: Generic Test for Unsolicited Bulk Email
-0.0 NO_RECEIVED            Informational: message has no Received headers

As you can see, this sample scored 1000 points. Only 5 points are required for an email to be considered SPAM. It’s safe to say this is SPAM and safe to say Spamassassin is working.

Using Spamassassin with Procmail

You will need to set up Procmail (or another MTA) in order to deliver mail. What we want to do is set this up globally so you don’t have to deal with each users ~/.procmailrc file. To do this you will edit the /etc/procmailrc file. All you need to do is add the following lines:

DROPPRIVS=yes

:0fw

| /usr/bin/spamc

Spamassassin should now be working.

Training spamassassin

At first you might not notice much of a drop in SPAM. This is because Spamassassin has to have a period of training.  There is a built-in tool for this called sa-learn. What you need to do is create two folders, one for SPAM and one for HAM. In the SPAM folder collect 100 or so emails that are definately SPAM. In the HAM folder collect 100 or so valid emails. Once you have those folders collected issue the following commands:

sa-learn –spam /PATH/TO/SPAM/FOLDER

sa-learn –ham /PATH/TO/HAM/FOLDER

Where /PATH/TO/SPAM/FOLDER and /PATH/TO/HAM/FOLDER are the explicit paths to these folders.

You can also set up a cron job to help Spamassassin train frequently if you need.

Final Thoughts

If SPAM is clogging up your Linux mail server you will be wise to employ a tool like Spamassassin. Your users and your IT staff will thank you for it.

All kinds of protection have been created to prevent this from happening. Some suggest using images instead of text, many obscure their emails in the hope that the bots will not be able to identify the right one and others are not publishing their mail at all.

Tinymail steps in and tries to be a barrier between your email address and the Internet. It does that by linking your email address to a Tinymail webpage which becomes your profile page. Instead of publishing your email on the web you embed the Tinymail code that is created after linking the email to a Tinymail email address.

The Tinymail code is showing part of the real email address and a link to the profile page which is only accessible if the user enters a captcha. If he does that he can take a look at the real email.

The idea itself is nice but it has two flaws in my opinion.The first is that it adds another step to the contact process. That might not be a problem if you desperately are trying to contact one but it is definitely a problem for other forms like providing feedback on a product. Users might give up in frustration and prefer to spend their time elsewhere.

The second is the captcha protection. Captchas have been broken in the past and once it is broken it could become a haven for a email collector who stored all those Tinymail emails in a database waiting for the right moment to decipher them all.

It would be nice if the email link could be deleted again after usage for a certain time, maybe even let it run out automatically after x days where x is a figure that the user can define in the options.

A click on the Calendars tab loads the calendars that are currently active. A click on the Calendar loads the details of that calendar with information about the calendar timezone and addresses. There is also another tab in that view that lets the user share the calender with other users.

A click on the Share This Calendar tab displays a list of all users who have access to this calendar with the option to add new users by pasting their email address into the form field.

The problem arises if a Gmail email address is pasted into that field. Nothing happens until the changes have been saved and the question if the user should be invited if he does not use Google Calendar is denied.

The full name of the user is disclosed int he Share This Calendar tab even if that user has chosen a gmail address that is not made up of his first and last username.

This might not seem like a big deal for many users but this is a honeypot for spammers. All they need to do is enter email addresses to find out the real name of the user to send out personalized spam. You probably would not react to a phishing mail asking you to login to your eBay account if the name would be missing or be wrong but what about if the real name would be there ?