Introduction
The most mysterious part of sendmail are the address rewriting rules. These rules have been compaired to line noise or the comments Dagwood makes when he smashes his thumb with a hammer. While the syntax of sendmail address rewriting rules is less than user friendly, some might say user hostile, the syntax is actually fairly simple. Not much worse than say perl with its arrays, hashes, and inline subsitutions. The rule syntax has about a dozen meta-symbols and the rules work in well defined ways; un-expected but well defined ways.
This tutorial has three main sections:
- Introduction to the R or Rule line, S or ruleSet line, and sendmail’s address test mode.
- The Left Hand Side and Right Hand Side rule syntax.
- A stratigy for using the comment field to translate the rule syntax into a human readable form.
An Address Rewriting Rule
The core of sendmail’s address rewriting capabilities is based on the idea of rules and rulesets. An individual address rewriting rule is a simple if ... then ... statement. If the address matches the Left Hand Side (LHS) pattern, then it is rewritten into the Right Hand Side (RHS) pattern. These rules are collected into rulesets which can be considered to be sub-routines. Different rulesets have different functions: some are called directly by sendmail, some are called by sendmail’s mailers, some do specific types of address rewriting or address cleanup.
An address rewriting rule has 2 or optionally 3 fields separated by tabs:
| R | LHS | <one or more tabs> | RHS | <optionally one or more tabs> | Optional comment field |
| If | Then | ||||
| Here is an example rule: | |||||
| R | $+ @ foo.com | $1 @ bar.com | user@foo.com -> user@bar.com | ||
Sendmail rules are recursively applied
One thing to keep in mind is that sendmail is a naturally recursive language. When sendmail applies a rule to an address and the address matches the LHS pattern, sendmail will rewrite the address according to the RHS. Then sendmail will apply the same rule again and if the address still matches the LHS pattern, sendmail will again rewrite the address according to the RHS. This is called recursion and will continue untill the address no longer matches the LHS pattern or the same rule is applied 100 times which sendmail detects as an infine loop. Recursion can be turned off with ruleset control meta-symbols.
Spaces and tabs within a rule
Some of the simplest mistakes to make with sendmail rules are with tabs and spaces.
"Picking" and "stuffing" with the mouse
The most common mistake people make is copying a rule by "picking" and "stuffing" a rule with the mouse. "picking" and "stuffing" converts tabs in the copied rule into spaces. If you do this you will get the error:
sendmail.cf: line nnn: invalid rewrite line "R some_LHS some_RHS" (tab expected)
This error will show up in both sendmail’s log file and in address test mode.
You must remember to convert the spaces between the fields back into tabs.
Spaces within a field
Spaces in a rule are un-important to the rule. Conceptually sendmail inserts a space between each ASCII string and meta-symbol in both the LHS and RHS. When you write a rule you can put spaces in a LHS or RHS field to make the rule more readable, or you can leave them out. For example:
| RSomepatternontheLHS | <one or more tabs> | ArewritepatternontheRHS |
| Is the same as: | ||
| R Some pattern on the LHS | <one or more tabs> | A rewrite pattern on the RHS |
Splitting a rule into multiple lines
If a rule becomes to long and you want to split it across two or more lines, you can split a rule between fields or within a field.
If you want to split a rule between fields, you simply start the next field on the the next line. The line must start with one or more tabs:
| R Some pattern on the LHS | |
| <one or more tabs> | A rewrite pattern on the RHS |
| <optionally one or more tabs> | Optional comment field |
If you want to split a rule within a field, you simply start the next line with one or more spaces:
| R | Some pattern on |
| the LHS <one or more tabs> A rewrite pattern | |
| on the RHS |
Sendmail Rulesets
The S line Numbered rulesets Named rulesets Name=number rulesetsTesting A Ruleset With Sendmail’s Address Test Mode, sendmail -bt
Sendmail can be invoked in an iteractive address test mode using the -bt command line option. Address test mode does not collect or deliver any mail nor does it interact with the running SMTP server or queue deamons so it can be run at any time.
When starting sendmail, an alternate sendmail.cf configuration file can be specified with the -C command line option. Additional -d debugging flags can also be specified. To start sendmail in address test mode with an alternate configuration file testrules.cf type:
sendmail -bt -C testrules.cf
You will be dumped into the interaactive test mode:
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter <ruleset> <address> >
To see a complete list of debugging commands available, type ?
You can download the testrules.cf file used in this tutorial.
Testing an address against a ruleset
The most common test to run is to test how a set of rulesets will rewrite a specific address. You enter the sequence of rulesets to test separated by commas. These can be either names and/or numbers. The rulesets are followed by the address to test (rewrite). The output for the default debug levels will show each ruleset being called (input:) and the address after the ruleset has rewritten it (returns:):
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| MyTestRuleset returns: user @ bar . com |
The first input: line is important because it shows how sendmail parses or "tokenizes" the input address. In this case it has broken the address into 5 tokens: user, @, foo, ., com.
Spaces in the input test address are ignored which can be useful when testing a specific ruleset at a higher debugging level.
| > 99,MyTestRuleset user @ foo.com |
| MyTestRuleset input: user @ foo . com |
| MyTestRuleset returns: user @ bar . com |
| MyTestRuleset input: user @ bar . com |
| MyTestRuleset returns: user @ bar . com |
If you see two input:Xs in a row, this indicates that the first ruleset called a second ruleset as a sub-routine call. When the second ruleset returns, the first ruleset then continues rewriting the address.
Displaying a ruleset in address test mode with =S
To see the contents of a ruleset that sendmail has loaded into memory use the =Sruleset command:
| > =SMyTestRuleset |
| R$+ @ foo . com $1 @ bar . com |
The ruleset can be either a name or a nubmer:
| > =S99 |
| R$+ @ foo . com $1 @ bar . com |
Changing Debugging Levels In Address Test Mode
A useful feature of address test mode is the ability to dynamically change the debug flags durring testing with the -dn.l command. n is the type of debugging, 21 for address rewriting. l is the level or ammount of output to show. To turn a debugging flag off set it to level 0, -dn.0.
Show rulesets being called, debug flag -d21.1
The most common debug flag used in address test mode is 21 to show address rewriting. When address test mode is started flag 21 is set to level 1, i.e. -d21.1, which shows the start or input: of each ruleset and rewriting results or returns: of the ruleset. This is a very useful level of debugging in that it gives you a fair amount of information in a relatively small amount of output. 2 line for each ruleset called.| > -d21.1 |
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| MyTestRuleset returns: user @ bar . com |
Show each time the address is rewritten, -d21.4
The next address rewriting level I use is level 4, -d21.4. At this level you see the address each time it is rewritten in a ruleset. This can be useful to follow the sequence of rewrites within a ruleset:
| > -d21.4 |
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| rewritten as: user @ bar . com |
| MyTestRuleset returns: user @ bar . com |
Again this is a useful level of debugging in that it gives you a fair amount of information in a relatively small amount of output. 2 line for each ruleset called and one line for each rule that rewrites the address. What this level does not show is what rules are rewriting the address.
This level also shows the current values of dynamic macros defined with delayed evaluation, $¯o. This is explianed later.
Show each rule being called, -d21.12
The highest debugging level for address rewriting I use is level 12. This level shows each rule being called, rule failures, and address rewrites. This is somewhat like a non-interactive symbolic debugger for sendmail:
| > -d21.12 |
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| -----trying rule: $+ @ foo . com |
| -----rule matches: $1 @ bar . com |
| rewritten as: user @ bar . com |
| -----trying rule: $+ @ foo . com |
| ----- rule fails |
| MyTestRuleset returns: user @ bar . com |
In the ouput the -----trying rule: line shows the LHS of a rule, the -----rule matches: shows the RHS of the rule if it matches, and the ----- rule fails indicates a non-matching rule without show the RHS.
Notice that while the ruleset only has a single rule, the rule is tested twice. This is sendmail’s recursion in action.
While this level of debugging can be useful to find exactly which rule rewrote and address or why a rewrite rule failed, it needs to be used with care because it generates a lot of output. 2 or 3 lines for the application of each rule in a ruleset. A standard test of a couple of standard rulesets can generate between 200 and 600 lines of output. I will discuss how to use this level of debugging output in a (future) address debugging tutorial.
Show matched parts of the address, -d21.15
A higher level of debugging I use in this tutorial is level 15 which shows the parts of the address that are matched by the meta-symbols in the rules. It also shows the line number of each rule:
| > -d21.15 |
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| -----trying rule (line 7): $+ @ foo . com |
| -----rule matches: $1 @ bar . com |
| $1: 0xbffddd80="user" |
| rewritten as: user @ bar . com |
| -----trying rule (line 7): $+ @ foo . com |
| ----- rule fails |
| MyTestRuleset returns: user @ bar . com |
I do not use this level except in explaining how pattern matching works.
Sendmail Address Rewriting Rules
The Left Hand Side, LHS
Generic Pattern Matches
Match zero or more tokens
Testing for domain style and RFC 822 source route addresses An RFC 822 source route addresses has the form: @host1,@host2,@host3:user@final.domain| > =SMatchZeroOrMoreTest1 |
| R$* @ foo . com $* First match = < $1 > , Second match = < $2 > |
| > MatchZeroOrMoreTest1 user@foo.com |
| MatchZeroOrMoreT input: user @ foo . com |
| MatchZeroOrMoreT returns: First match = < user > , Second match = < > |
| > MatchZeroOrMoreTest1 @foo.com:user@bar.com |
| MatchZeroOrMoreT input: @ foo . com : user @ bar . com |
| MatchZeroOrMoreT returns: First match = < > , Second match = < : user @ bar . com > |
| > =SMatchZeroOrMoreTest2 |
| R$* @ $* foo . com Second match = < $2 > |
| > MatchZeroOrMoreTest2 user@foo.com |
| MatchZeroOrMoreT input: user @ foo . com |
| MatchZeroOrMoreT returns: Second match = < > |
| > MatchZeroOrMoreTest2 user@host.foo.com |
| MatchZeroOrMoreT input: user @ host . foo . com |
| MatchZeroOrMoreT returns: Second match = < host . > |
| > MatchZeroOrMoreTest2 user@host.subdomain.foo.com |
| MatchZeroOrMoreT input: user @ host . subdomain . foo . com |
| MatchZeroOrMoreT returns: Second match = < host . subdomain . > |
Match one or more tokens
| R $+ @ foo.com |
| user@foo.com |
| foo.com:user@bar.com |
| first.last@foo.com |
| R $+ @ $+ . foo.com |
| user@foo.com |
| user@host.foo.com |
| user@host.subdomain.foo.com |
| R $+ @ $+ |
| anything @ anything |
| R $+ |
| something |
| (nothing) |
Match exactly one token
| R$- ! $+ |
| uuhost!user |
| uuhost!nexthost!user |
| R$+ @ $- |
| user@host |
| user@host.domain |
| R$+ @ $- . $+ |
| user@host.domain |
| user@host.subdomain.domain |
Match exactly zero token
| R$@ |
| (nothing) |
| something |
Specific Pattern Matches
ASCII Text Macro Matches Delayed Evaluation Macro Matches Class Matches Exclusive Class MatchesHow Does Sendmail Match An Address?
| > -d21.35 |
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| -----trying rule (line 7): $+ @ foo . com |
| ADVANCE rp=$+, ap=user |
| ADVANCE rp=@, ap=@ |
| ADVANCE rp=foo, ap=foo |
| ADVANCE rp=., ap=. |
| ADVANCE rp=com, ap=com |
| -----rule matches: $1 @ bar . com |
| $1: 0xbffddd80="user" |
| rewritten as: user @ bar . com |
| -----trying rule (line 7): $+ @ foo . com |
| ADVANCE rp=$+, ap=user |
| ADVANCE rp=@, ap=@ |
| ADVANCE rp=foo, ap=bar |
| ADVANCE rp=@, ap=bar |
| ADVANCE rp=@, ap=. |
| ADVANCE rp=@, ap=com |
| ADVANCE rp=@, ap=<null> |
| ----- rule fails |
| MyTestRuleset returns: user @ bar . com |
| > -d21.36 |
| > MyTestRuleset user@foo.com |
| MyTestRuleset input: user @ foo . com |
| -----trying rule (line 7): $+ @ foo . com |
| ADVANCE rp=$+, ap=user |
| ADVANCE rp=@, ap=@ |
| ADVANCE rp=foo, ap=foo |
| ADVANCE rp=., ap=. |
| ADVANCE rp=com, ap=com |
| -----rule matches: $1 @ bar . com |
| $1: 0xbffddd80="user" |
| rewritten as: user @ bar . com |
| -----trying rule (line 7): $+ @ foo . com |
| ADVANCE rp=$+, ap=user |
| ADVANCE rp=@, ap=@ |
| ADVANCE rp=foo, ap=bar |
| BACKUP rp=$+, ap=@ |
| ADVANCE rp=@, ap=bar |
| BACKUP rp=$+, ap=bar |
| ADVANCE rp=@, ap=. |
| BACKUP rp=$+, ap=. |
| ADVANCE rp=@, ap=com |
| BACKUP rp=$+, ap=com |
| ADVANCE rp=@, ap=<null> |
| BACKUP rp=$+, ap=<null> |
| ----- rule fails |
| MyTestRuleset returns: user @ bar . com |
The Right Hand Side, RHS
Address Rewriting
ASCII Text
Pattern Subsitution
Database Subsitution
Database rewriting allows sendmail to query different types of databases with a lookup key and rewrite the address based on the information returned by the database. The database meta-symbols include:
| $( dbname | Start the look up in dbname |
| $@ parameter | An additional parameter to be subsitued in the return value |
| $: default | The default rewrite if the lookup key is not found in the database |
| $) | End the database lookup |
Database meta-symbols and database rewriting is covered in a separate (future) tutorial.
Ruleset Control
Calling Another Ruleset
Calling A Mailer
Just another "Harker's Helpful Hint"
RLH