Difference between revisions of "Common CanIt User Tasks"

From Roaring Penguin
Jump to: navigation, search
Line 32: Line 32:
 
The value assigned by these rules can be overridden with Rules->Score Overrides.
 
The value assigned by these rules can be overridden with Rules->Score Overrides.
  
=Stop Good Messages From Being Trapped/Let Through=
+
=Stop Messages From Being Trapped/Let Through=
  
We at Roaring Penguin run what we call the Roaring Penguin Training Network (RPTN). This is a massive database consisting of all of combined judgement of all of our participating members. By participating your CanIt system will benefits from the judgement of millions of votes, creating a very accurate set of general rules on what the content of spam emails look like. That being said, your organization receives a unique sub-set of all of the mail on the internet and so rules that fit the rest of the world may not fit you exactly. This is why it is important that you still train your own system, even though it has so much inherited knowledge. Your votes will override the RPTN votes if they differ allowing you to have a filter that is customized to your needs.
+
We at Roaring Penguin run what we call the Roaring Penguin Training Network (RPTN). This is a massive database consisting of all of combined judgement of all of our participating members. By participating, your CanIt system will benefits from the judgement of millions of votes, creating a very accurate set of general rules on what the content of spam emails look like. That being said, your organization receives a unique sub-set of all of the mail on the internet and so rules that fit the rest of the world may not fit you exactly. This is why it is important that you still train your own system, even though it has so much inherited knowledge. Your votes will override the RPTN votes if they differ allowing you to have a filter that is customized to your needs.
  
Any time that you release a message from your quarantine, this is done automatically; teaching the system that messages which resemble the one you released are good. Likewise, if you reject a pending message it will reinforce the assumptions that the scanner made about its content and will make similar messages score even higher in the future.  
+
Any time that you release a message from your quarantine, this is done automatically; teaching the system that messages which resemble the one you released are good. Likewise, if you reject a pending message it will reinforce the assumptions that the scanner made about its content and will make similar messages even more likely to be caught in the future.  
  
 
If a spam has already made it to your inbox without being trapped, you will likely have the ability to vote it as spam from the CanIt footer that is generally added to the content of the email itself.
 
If a spam has already made it to your inbox without being trapped, you will likely have the ability to vote it as spam from the CanIt footer that is generally added to the content of the email itself.
Line 42: Line 42:
 
If you find that you consistently have the same types of good messages trapped or bad messages let through over an extended period, a special rule may be in order.
 
If you find that you consistently have the same types of good messages trapped or bad messages let through over an extended period, a special rule may be in order.
  
Rules can be created under the Rules category which may or may not be available to you as a user. For more on this, see Blacklisting and Whitlisting, as well as Create a Custom Rule below.
+
Rules can be created under the Rules category which may or may not be available to you as a user. For more on this, see Blacklisting and Whitelisting, as well as Create a Additional Rules below.
  
 
=Blacklisting and Whitelisting=
 
=Blacklisting and Whitelisting=
  
Blacklist and whitelist entries can be requested from the either your inbox or the WebUI in the same manner that you accept or reject mail. Please note that depending on the configuration of your system and its current load, these rules will take some time to be applied. If they don't seem to be working, wait a few hours before contacting your administrator.
+
Blacklists and whitelists are absolute rules. Messages that trigger either of these will not go through any scanning. They will either be automatically discarded in the prior case or automatically accepted in the latter. As such, you need to be careful with them. The exception to this is for whitelisted senders who do not pass the SPF check. This is a system used to check that the mail originated from a machine that is allowed to the sender's address, and if the test fails it is a strong indicator that it was spoofed.
 
 
Blacklists and whitelists are absolute rules. Messages that trigger either of these will not go through any scanning. They will either be automatically discarded in the prior case or automatically accepted in the latter. As such, you need to be careful with them. The exception to this is for whitelisted senders who do not pass the SPF check. This is a system used to check that the mail originated from a machine that is allowed to the sender's address, and if the test fails even whitelisted mail will be scanned.
 
  
 
Blacklisting is often not as useful as you might initially think. Spammers tend to generate new sender addresses for every message they send, and so you will rarely receive two spams from the same sender. Blacklists are most appropriate for things like recurring newsletters and ads from e-retailers or other similar senders whose unsubscribe process is too tedious.
 
Blacklisting is often not as useful as you might initially think. Spammers tend to generate new sender addresses for every message they send, and so you will rarely receive two spams from the same sender. Blacklists are most appropriate for things like recurring newsletters and ads from e-retailers or other similar senders whose unsubscribe process is too tedious.
  
You should also use some caution when creating whitelists as well. Domain whitelisting is rarely advisable, especially for common domains like gmail.com or hotmail.com. Even for senders, however, there is a level of danger to whitelists because spammers are capable of spoofing their address as one of your common contacts, or causing one of these contacts to perpetuate their spam attack. If either of these is the case for a contact that you have whitelisted the spam will make it through unchecked. You are also not able to whitelist your own domain
+
You should also use some caution when creating whitelists as well. Domain whitelisting is rarely advisable, especially for common domains like gmail.com or hotmail.com. Even for senders, however, there is a level of danger because spammers are capable of spoofing their address of one of your common contacts, or can infect one of their machines. If either of these is the case for a contact that you have whitelisted the spam will make it through unchecked. You are also not able to whitelist your own domain.
  
If you are worried about adding absolute rules like these, see Creating a Custom Rule below.
+
If you are worried about adding absolute rules like these, see Creating Additional Rules below.
  
 
==From the Inbox==
 
==From the Inbox==
  
If you receive a spam in your inbox you can select one of the blacklist voting links in the footer. These voting links will also be available in your quarantine notifications if you are set up to receive them.
+
Blacklist and Whitelist entries can be created from the your inbox using the voting links in the footers added to regular mail or from the action list in the notifications ONLY IF the admin has made these available to you.
  
 
Because blacklisting is often useless, as described above, a blacklist entry generated in this manor is automatically given a 30 day expiry. If you would like to make a permanent blacklist you can do so in the WebUI or request one from your administrator.
 
Because blacklisting is often useless, as described above, a blacklist entry generated in this manor is automatically given a 30 day expiry. If you would like to make a permanent blacklist you can do so in the WebUI or request one from your administrator.
Line 64: Line 62:
 
==From the WebUI==
 
==From the WebUI==
  
These can be made in a very similar manor to the inbox by selecting Always Accept or Always Reject from the Home page or Quarantine->Pending. Blacklists created in this manor are also temporary when created in this way.
+
By default you can create these from the Home page of the WebUI using the Accept and Reject List box (this is available by default for all users with web access, but can be disabled as well).
  
Manual blacklisting and whitelisting rules can be created under Rules->Sender, Rules->Domains, and Rules->Networks for the relevant level of scope. You create a rule by entering the address into the text field at the top and clicking Add Rule. Once it is in the list you can specify what to do for that address, when the rule should expire, if ever, and a brief description of why you created the rule. If there are existing rules that you would like to change, that can also be done here. Rules with generated from the inbox or quarantine will have a comment with the IncidentID automatically added to the comments. Once all of the changes are made to your liking, make sure that you click the Submit Changes button at the bottom.
+
You can also create these rules in response to specific incidents if you are using the pulldown lists for action selection (as opposed to radio button) and if you administrator allows these as an option.
  
=Create Custom Rules=
+
Manual blacklisting and whitelisting rules can also be created under Rules->Sender, Rules->Domains, and Rules->Networks (not available to users by default) for the relevant level of scope. You create a rule by entering the address into the text field at the top and clicking Add Rule. It will then ask you to specify what to do for that address/domain/IP, when the rule should expire, and a brief description of why you created the rule.
  
This is something that you may want to - or need to - request from your administrator. Custom rules are able to be specified in Rules->Custom Rules or Rules->Compound Rules. Compound Rules are a more complex version of custom rules that allow for the use of logic operators such as AND and OR. This allows you to make rules only apply when multiple fields are matched. If you don't know how logic operator work, ask your administrator for help.
+
These pages will also allow you to view and modify existing rules. Those that where generated from the inbox or quarantine will have a comment with the Incident ID that the rule was created for automatically added. Once all of the changes are made to your liking, make sure that you click the Submit Changes button at the bottom.
  
==General Rules==
+
=Create Additional Rules=
  
A custom rule is quite simple and can be made to look at a variety of fields such as the sender address, sender domain, subject, or otherwise to varying levels of specificity. You can then select a value to apply to any message that matches this definition. By selecting a positive value, messages that trip the rule will have their score increased by that amount. It is generally advisable that you don't set a broad rule to apply as much as your S-300 value (Preferences->Quarantine Settings->Filter Settings->S-300). This makes it so that a message must still score on at least one other rule before it is quarantined, minimizing the incidence of false-positives. Similarly, applying a negative value will lower the total spam score, making matching messages less likely to get trapped. Again, you don't want to apply too large of a negative score for a broad rule otherwise you risk having spam get through because it tripped the rule. For very specific rules that apply to messages you never want to see, or always want to see, you can go much higher or create a backlist/whitelist as explained above. See Where Does it All Go? above to help you find the right value for your desired result.
+
This is something that you may want to - or need to - request from your administrator. A limited number of rule options will be available to regular Users under the Rules category.
  
 
==Country Rules==
 
==Country Rules==
Line 82: Line 80:
 
==Attachment Extensions==
 
==Attachment Extensions==
  
Rules->MIME Types allows you to create rules for the files that are included as attachments based on their extensions. CanIt will detect not only the immediate file extensions, but also those within certain archives such as .zip. Virus transmission is extremely common with many Microsoft file formats such as .exe, .msi, and .bat. These are all blocked by default and while it is highly advisable that you don't change this, you can apply a rule to allow or at least hold them if you need to. It is much more advisable that you use a file sharing service such as Dropbox and run thorough scans of the files if you need to send or receive them. If you try to send many file extensions to major services like Gmail, they will be rejected on the receiving end anyways.
+
Rules->MIME Types allows you to create rules for types such as "application/html".
 +
 
 +
Rules->Filename Extensions can be used to manage attachments based on their extensions. CanIt will detect not only the immediate file extensions, but also those within archives. Virus transmission is extremely common with many Microsoft file formats such as .exe, .msi, and .bat, so these are blocked by default as well as the rest of [[Blocked_File_Extensions_-_Hosted|these extensions]]. You could override these with your own rules for the same extensions, but that is not advisable.
 +
 
 +
You can add your own by entering the extension without the dot in the text field at the top, clicking Add Rule, then applying an action for whitelisted sender and a general action for all other sender. Be sure to Submit Changes when you are done. If you > to the file extension CanIt will look for that extension only if it is contained within an archive (.zip, .rar, etc.).
 +
 
 +
Other formats such as Microsoft Office documents and PDFs are common carriers of viruses. CanIt does have facilities in place to scan for viruses built into Office Macros as well as to scan for links, however some malicious content may continue to come through using this vector. If you do not anticipate needing these files via email, you may consider blocking them.
 +
 
 +
==Custom/Compound Rules==
 +
 
 +
The most flexible and powerful are Custom Rules and Compound Rules. These let you identify a wide variety of aspects of a message including the Header content, Body content and a wide variety of other fields. You can create arbitrary rules for words, phrases and even pattern matching with regular expressions and add or subtract your desired number of points if they are triggered. Compound Rules are a more complex, they allow for more fields to be searched and the use of logic operators such as AND and OR to match multiple criteria. These rules allow you to do things like adding or removing points if certain words are in the body; add additional score if multiple low scoring tests are hit; block messages that are similar to your domain but not the same.
  
Additionally, other formats such as Microsoft Office documents are common carriers of viruses. These are not generally enforced by default, but if your company doesn't deal with these files very often, they may be for you.  You can add your own by entering the extension without the dot in the text field at the top, clicking Add Rule, then applying an action for whitelisted sender and a general action for all other sender. Be sure to Submit Changes when you are done. As a final note, you can append > to the file extension to have the rule only take effect if it is contained within an archive.
+
Some of these rules can be tricky to manage and you should not hesitate to consult an administrator before trying to make rules of your own.
  
 
:''Continue to: [[Common CanIt User Problems]]''
 
:''Continue to: [[Common CanIt User Problems]]''

Revision as of 15:13, 4 May 2016

Main article: Quick Start Guide

The following are suggestions on how to perform frequent tasks applicable to a CanIt end-user.

Please Note

The options within this guide may or not apply to you, depending on your version of CanIt and how it has been set up by the e-mail administrator. If you can't find something as described but think that you need it, consult your administrator; there may be a reason that you don't have it

Manage Your Quarantine

Quarantined messages can be accepted or rejected. When you perform these actions, the message is trained into our Bayes Database for content analysis which makes the scanner better at identifying spam in the future. You may also be allowed to whitelist or blacklist the Sender or Domain so that mail from them in the future will either be accepted or rejected without your intevention.

From the Inbox

If you are set up to receive notification emails it will contain voting links beside each entry. You may or may not have all of the above options available. By clicking the voting link for each item the changes should automatically be applied. This may require you to log in to the WebUI to register the vote, but if a window opens to acknowledge the vote without asking you to log in, you can simply ignore it. This is unfortunately a behaviour of the mail reader and no changes made in CanIt will prevent these from opening.

You can click on the Subject of the message to be shown its contents in the WebUI if you are able to log in. See below the next section for advanced tools.

From the WebUI

The pending messages in your quarantine will be immediately visible from the Home page as well as being categorized and searchable in the from the Quarantine page. The messages visible on the Home page are messages that were deemed to be suspicious, but not egregiously so and are pulled from the more detailed Quarantine->Pending. You can also select the Spam, Non-Spam or All pages from the Quarantine category to see messages that you have already been rejected or (manually) accepted. Mail that initially scored below the quarantine threshold is not stored in the WebUI except if you have the Archive available.

From Home, Quarantine, or the incident pages there will be a drop-down menu or series of checkboxes that will let you accept or reject any of the trapped messages (except those that have already been resolved; only admins can re-open incidents by default). From Home or Quarantine you will need to click Submit Changes before these will take effect. With drop-down menus you may additionally have the ability to Always Accept, or Always Reject which will create whitelist or blacklist rules respectively.

See Why a Message Was Trapped

From either the Home or Quarantine section you will be able to click on the subject of messages to see the body contents. You can then click on See Incident Details at the top of the page (or the Date column from the list view) to get detailed information on how the message scored. It is helpful to note the Incident ID if you are going to see help from an admin (IDs are case-sensitive).

At the bottom of the incident page is a list of all of the rules that were triggered by the message. The most common reason a message is blocked will be if it triggered the Bayes statistical analysis. This is the probability that it is spam based on content analysis and is represented by the list of coloured words at the very bottom. The overall likelihood is listed next to the spam score as a percentage. The other scores will be above and look something like:

# RULE_NAME      Human-readable description of rule

The value assigned by these rules can be overridden with Rules->Score Overrides.

Stop Messages From Being Trapped/Let Through

We at Roaring Penguin run what we call the Roaring Penguin Training Network (RPTN). This is a massive database consisting of all of combined judgement of all of our participating members. By participating, your CanIt system will benefits from the judgement of millions of votes, creating a very accurate set of general rules on what the content of spam emails look like. That being said, your organization receives a unique sub-set of all of the mail on the internet and so rules that fit the rest of the world may not fit you exactly. This is why it is important that you still train your own system, even though it has so much inherited knowledge. Your votes will override the RPTN votes if they differ allowing you to have a filter that is customized to your needs.

Any time that you release a message from your quarantine, this is done automatically; teaching the system that messages which resemble the one you released are good. Likewise, if you reject a pending message it will reinforce the assumptions that the scanner made about its content and will make similar messages even more likely to be caught in the future.

If a spam has already made it to your inbox without being trapped, you will likely have the ability to vote it as spam from the CanIt footer that is generally added to the content of the email itself.

If you find that you consistently have the same types of good messages trapped or bad messages let through over an extended period, a special rule may be in order.

Rules can be created under the Rules category which may or may not be available to you as a user. For more on this, see Blacklisting and Whitelisting, as well as Create a Additional Rules below.

Blacklisting and Whitelisting

Blacklists and whitelists are absolute rules. Messages that trigger either of these will not go through any scanning. They will either be automatically discarded in the prior case or automatically accepted in the latter. As such, you need to be careful with them. The exception to this is for whitelisted senders who do not pass the SPF check. This is a system used to check that the mail originated from a machine that is allowed to the sender's address, and if the test fails it is a strong indicator that it was spoofed.

Blacklisting is often not as useful as you might initially think. Spammers tend to generate new sender addresses for every message they send, and so you will rarely receive two spams from the same sender. Blacklists are most appropriate for things like recurring newsletters and ads from e-retailers or other similar senders whose unsubscribe process is too tedious.

You should also use some caution when creating whitelists as well. Domain whitelisting is rarely advisable, especially for common domains like gmail.com or hotmail.com. Even for senders, however, there is a level of danger because spammers are capable of spoofing their address of one of your common contacts, or can infect one of their machines. If either of these is the case for a contact that you have whitelisted the spam will make it through unchecked. You are also not able to whitelist your own domain.

If you are worried about adding absolute rules like these, see Creating Additional Rules below.

From the Inbox

Blacklist and Whitelist entries can be created from the your inbox using the voting links in the footers added to regular mail or from the action list in the notifications ONLY IF the admin has made these available to you.

Because blacklisting is often useless, as described above, a blacklist entry generated in this manor is automatically given a 30 day expiry. If you would like to make a permanent blacklist you can do so in the WebUI or request one from your administrator.

From the WebUI

By default you can create these from the Home page of the WebUI using the Accept and Reject List box (this is available by default for all users with web access, but can be disabled as well).

You can also create these rules in response to specific incidents if you are using the pulldown lists for action selection (as opposed to radio button) and if you administrator allows these as an option.

Manual blacklisting and whitelisting rules can also be created under Rules->Sender, Rules->Domains, and Rules->Networks (not available to users by default) for the relevant level of scope. You create a rule by entering the address into the text field at the top and clicking Add Rule. It will then ask you to specify what to do for that address/domain/IP, when the rule should expire, and a brief description of why you created the rule.

These pages will also allow you to view and modify existing rules. Those that where generated from the inbox or quarantine will have a comment with the Incident ID that the rule was created for automatically added. Once all of the changes are made to your liking, make sure that you click the Submit Changes button at the bottom.

Create Additional Rules

This is something that you may want to - or need to - request from your administrator. A limited number of rule options will be available to regular Users under the Rules category.

Country Rules

Country rules can be set in Rules->Countries and they allow you to apply scores in the same way as custom rules specifically to messages originating from a country. It is important to note that these decisions are made based on server locations, not top-level-domains. Mail is often relayed through countries other than those that they originate from and so your points may end up being applied or missed where you might not expect.

Attachment Extensions

Rules->MIME Types allows you to create rules for types such as "application/html".

Rules->Filename Extensions can be used to manage attachments based on their extensions. CanIt will detect not only the immediate file extensions, but also those within archives. Virus transmission is extremely common with many Microsoft file formats such as .exe, .msi, and .bat, so these are blocked by default as well as the rest of these extensions. You could override these with your own rules for the same extensions, but that is not advisable.

You can add your own by entering the extension without the dot in the text field at the top, clicking Add Rule, then applying an action for whitelisted sender and a general action for all other sender. Be sure to Submit Changes when you are done. If you > to the file extension CanIt will look for that extension only if it is contained within an archive (.zip, .rar, etc.).

Other formats such as Microsoft Office documents and PDFs are common carriers of viruses. CanIt does have facilities in place to scan for viruses built into Office Macros as well as to scan for links, however some malicious content may continue to come through using this vector. If you do not anticipate needing these files via email, you may consider blocking them.

Custom/Compound Rules

The most flexible and powerful are Custom Rules and Compound Rules. These let you identify a wide variety of aspects of a message including the Header content, Body content and a wide variety of other fields. You can create arbitrary rules for words, phrases and even pattern matching with regular expressions and add or subtract your desired number of points if they are triggered. Compound Rules are a more complex, they allow for more fields to be searched and the use of logic operators such as AND and OR to match multiple criteria. These rules allow you to do things like adding or removing points if certain words are in the body; add additional score if multiple low scoring tests are hit; block messages that are similar to your domain but not the same.

Some of these rules can be tricky to manage and you should not hesitate to consult an administrator before trying to make rules of your own.

Continue to: Common CanIt User Problems