Vortexmind: free your mind

Due to high demand, I’m going to write a straightforward mod_rewrite guide to group together the solutions to the most common problems. Again, this article is NOT a detailed guide about mod_rewrite: if you need such a thing, you should go to Apache’s website which brings a complete document about this issue. I will update this tutorial when I get new info about this “Apache 2 swiss army knife”. But first, let’s just introduce the topic by seeing what mod_rewrite is…

From Wikipedia:

A rewrite engine is a piece of web server software used to modify URLs, for a variety of purposes. Some benefits derived from a rewrite engine are:

• Making website URLs more user friendly
• Making website URLs more search-engine friendly
• Preventing undesired ‘hot linking’
• Not exposing the (web address related) inner workings of a website to users

The Apache HTTP server has a rewrite engine called mod_rewrite, which has been described as “the Swiss Army knife of URL manipulation“.

Before going on, you must know that mod_rewrite works by using regular expressions. From Wikipedia:

A regular expression (abbreviated as regexp or regex, with plural forms regexps, regexes, or regexen) is a string that describes or matches a set of strings, according to certain syntax rules. Regular expressions are used by many text editors and utilities to search and manipulate bodies of text based on certain patterns. Many programming languages support regular expressions for string manipulation. For example, Perl and Tcl have a powerful regular expression engine built directly into their syntax. The set of utilities (including the editor sed and the filter grep) provided by Unix distributions were the first to popularize the concept of regular expressions.

This article is not meant to explain you what regexp are, and how to deal with them. Maybe I will focus on them on another tutorial, but if I did this now I would have thrown out of scope this article! So, I just remind the most important regexp operators used by mod_rewrite (as they’re shown on apache guide)

Text:

• . Any single character
• [chars] Character class: Any character of the class “chars”
• [^chars] Character class: Not a character of the class “chars”
• text1|text2 Alternative: text1 or text2

Quantifiers:

• ? 0 or 1 occurrences of the preceding text
• * 0 or N occurrences of the preceding text (N > 0)
• + 1 or N occurrences of the preceding text (N > 1)

Grouping:

• (text) Grouping of text (used either to set the borders of an alternative as above, or to make backreferences, here the Nth group can be referred to on the RHS of a RewriteRule as $N)

Anchors

• ^ Sart-of-line anchor
• $ End-of-line anchor

Escaping:

• \char escape the given char (for instance, to specify the chars “.[]()” etc.)

Let’s now deal with the most common issues!

Tweaks Index

(Last Updated 28th Feb 2006)

1. Basic URL redirection
2. Add Trailing Slashes
3. Blocking bad referers - No hotlinking
4. Blocking Bad Bots | Fetchers
5. Do not show ‘www’
6. Guessable url
7. Flags Explained
8. Time-Dependent Rewriting
9. Browser Dependent Content
10. On-the-fly Content-Regeneration
11. External Rewriting Engine
12. Sources

1) Basic URL redirection

You have moved a file and you want to redirect all the visit to the new location


<IfModule mod_rewrite>
RewriteEngine on
RewriteRule ^old\.html$ new.html [R=301,L]
</IfModule>


This way, all the request that point to old.html will be pointed to new.html with a permanent redirect.
If the redirect is temporary, use R=302
Be careful with redirects, they are a bad thing when we talk about SEO and SE rankings!

./ Back to Index

2) Add Trailing Slashes

Use this to add trailing slashes to some recurrent url of your site


<IfModule mod_rewrite>
RewriteEngine on
RewriteRule ^myphotos/([0-9]+)$ myphotos/$1/ [R]
</IfModule>


./ Back to Index

3) Blocking bad referers - No hotlinking

See this Tip

./ Back to Index

4) Blocking Bad Bots | Fetchers

See this Tip

./ Back to Index

5) Do not show ‘www’

See this Tip

./ Back to Index

6) Guessable URL

Suppose you see from your logs that a lot of visitors come searching some page they’d think you provide, but is really in another section of your website. For example, suppose visitors come searching for http://www.example.com/tweaks and keep getting a 404 error, when the things they’re searching for are at http://www.example.com/tutorials/. You can route them to the right location!


<IfModule mod_rewrite>
RewriteEngine on
RewriteRule ^tweaks(/)?$ /tutorials/ [R]
</IfModule>


./ Back to Index

7) Flags Explained

You can add some flags to each rewrite rule to tweak its behaviour. The flags are added after each RewriteRule in [ ]
To have a complete reference, see here.

• C (chain): chains the current rule with the next rule
• CO=NAME:VAL:domain[:lifetime[:path]] (cookie):This sets a cookie in the client’s browser
• E=VAR:VAL (env): This forces an environment variable named VAR to be set to the value VAL, where VAL can contain regexp backreferences ($N and %N) which will be expanded. You can use this flag more than once, to set more than one variable
• F (forbidden): This forces the current URL to be forbidden - it immediately sends back a HTTP response of 403 (FORBIDDEN)
• G (gone): This forces the current URL to be gone - it immediately sends back a HTTP response of 410 (GONE)
• L (last): Stop the rewriting process here and don’t apply any more rewrite rules
• N (next): Re-run the rewriting process (starting again with the first rewriting rule). This time, the URL to match is no longer the original URL, but rather the URL returned by the last rewriting rule.Be careful not to create an infinite loop!
• NE (noescape): This flag prevents mod_rewrite from applying the usual URI escaping rules to the result of a rewrite
• NS (nosubreq): This flag forces the rewrite engine to skip a rewrite rule if the current request is an internal sub-request.
• P (proxy): This flag forces the substitution part to be internally sent as a proxy request and immediately (rewrite processing stops here) put through the proxy module.
• PT (passthrough): This flag forces the rewrite engine to set the uri field of the internal request_rec structure to the value of the filename field
• QSA (querystringappend): This flag forces the rewrite engine to append a query string part of the substitution string to the existing string, instead of replacing it
• R[=code] (redirect):Prefix Substitution with http://thishost[:thisport]/ (which makes the new URL a URI) to force a external redirection. If no code is given, a HTTP response of 302 (MOVED TEMPORARILY) will be returned
• S=num (skipnext): This flag forces the rewriting engine to skip the next num rules in sequence, if the current rule matches
• T=MIME-type (mimetype): Force the MIME-type of the target file to be MIME-type. This can be used to set up the content-type based on some conditions

./ Back to Index

8) Time-Dependent Rewrite

Sometimes you may want to apply different rules with respect to time


<IfModule mod_rewrite>
RewriteEngine on
RewriteCond %{TIME_HOUR}%{TIME_MIN} >0700
RewriteCond %{TIME_HOUR}%{TIME_MIN} <1900
RewriteRule ^foo\.html$ foo.day.html
RewriteRule ^foo\.html$ foo.night.html
</IfModule>


./ Back to Index

9) Browser Dependent Content

At least for important top-level pages it is sometimes necessary to provide the optimum of browser dependent content, i.e. one has to provide a maximum version for the latest Netscape variants, a minimum version for the Lynx browsers and a average feature version for all others.


<IfModule mod_rewrite>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.*
RewriteRule ^foo\.html$ foo.NS.html [L]

RewriteRule ^foo\.html$ foo.32.html [L]
</IfModule>


./ Back to Index

10) On-the-fly Content-Regeneration

Here comes a really esoteric feature: Dynamically generated but statically served pages, i.e. pages should be delivered as pure static pages (read from the filesystem and just passed through), but they have to be generated dynamically by the webserver if missing. This way you can have CGI-generated pages which are statically served unless one (or a cronjob) removes the static contents. Then the contents gets refreshed.


<IfModule mod_rewrite>
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-s
RewriteRule ^page\.html$ page.cgi [T=application/x-httpd-cgi,L]
</IfModule>


./ Back to Index

11) External Rewriting Engine

Question? How can we solve the FOO/BAR/QUUX/etc. problem? There seems no solution by the use of mod_rewrite
Response: Use an external RewriteMap, i.e. a program which acts like a RewriteMap. It is run once on startup of Apache receives the requested URLs on STDIN and has to put the resulting (usually rewritten) URL on STDOUT (same order!).


<IfModule mod_rewrite>
RewriteEngine on
RewriteMap quux-map prg:/path/to/map.quux.pl
RewriteRule ^/~quux/(.*)$ /~quux/${quux-map:$1}
</IfModule>


Then you code your personal mapping


#!/path/to/perl

# read URLs one per line from stdin and
# generate substitution URL on stdout
while (<>) {
s|^foo/|bar/|;
print $_;
}


./ Back to Index

Sources

• Url Rewriting Tutorial
• Apache 2.0 mod_rewrite
• Apache 2.0 rewrite guide
• Mod rewrite cheat sheets

./ Back to Index