README
author Mahlon E. Smith <mahlon@martini.nu>
Fri, 04 May 2012 10:25:31 -0700
changeset 25 6ceafe5ffe7f
parent 22 822094314703
child 29 c5d00a24af56
permissions -rw-r--r--
Added tag 0.1.1 for changeset d3b6f9ccbe20
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     1
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     2
Volta
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     3
=====
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     4
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
     5
What is volta?
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     6
--------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     7
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     8
Volta is a high performance, low resource URI rewriter for use with the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     9
Squid caching proxy server (http://www.squid-cache.org/.)  With it, you
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    10
can dynamically alter URI requests that pass through Squid based on
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    11
various criteria.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    12
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    13
It uses a state machine to parse URIs and rules, and a constant database
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    14
to store and access those rules.  It can then either perform conditional
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    15
rewrites internally, or by evaluating Lua scripts.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    16
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    17
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    18
Why is it called "volta"?
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    19
-------------------------
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    20
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    21
It's a type of old Italian music written in triple-time.  Quick!
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    22
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    23
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    24
How fast is it?
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    25
---------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    26
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    27
On a 2Ghz Xeon 5130, it can process a million squid requests against
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    28
10000 rules in less than 8 seconds, using about 800k of ram.  On an
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    29
1.8Ghz Intel E4300, it can do it in 3 seconds.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    30
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    31
Your mileage may vary, but for most all intents and purposes the answer
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    32
is "definitely fast enough."
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    33
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    34
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    35
Configuring squid
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    36
-----------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    37
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    38
You must enable url rewriting from within the squid.conf file.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    39
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    40
	url_rewrite_program /usr/local/bin/volta
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    41
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    42
... and that's it.  You may need some additional customization, like where
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    43
the volta database is stored on disk:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    44
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    45
	url_rewrite_program /usr/local/bin/volta -f /var/db/squid/volta.db
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    46
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    47
Busy servers:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    48
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    49
Make sure rewrite_concurrency is disabled, volta is single threaded.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    50
Instead, just add more volta children.  They are lightweight, so load em
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    51
up.  A proxy at my $DAYJOB is in use by around 450 people, and we get by
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    52
nicely with 10 volta children.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    53
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    54
	url_rewrite_concurrency 0
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    55
	url_rewrite_children 10
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    56
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    57
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    58
Using volta
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    59
-----------
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    60
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    61
See the INSTALL file for instructions on how to compile volta.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    62
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    63
Volta reads its rewrite rules from a local database.  You can create the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    64
rules in a text editor, then convert it to the database like so:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    65
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    66
	% volta -c rules.txt
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    67
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    68
You'll be left with a "volta.db" file in the current directory.  Put it
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    69
wherever you please, and use the -f flag to point to it.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    70
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    71
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    72
Rule file syntax
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    73
----------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    74
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    75
Volta's rule syntax is designed to be easy to parse by humans and
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    76
machines.  Blank lines are skipped, as is any line that starts with the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    77
'#' character, so you can keep the ascii version of your rules well
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    78
documented and in version control.  There is no practical limit on the
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    79
number of rules in this database.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    80
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    81
When compiling the ruleset into the database format, volta detects
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    82
malformed rules and stops if there are any problems, leaving your
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    83
original database intact.  You can change the ruleset at any time while
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    84
volta is running, and the new rules will take affect within about 10
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    85
seconds.  No need to restart squid!
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    86
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    87
There are two types of rules -- positive matches, and negative matches.
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    88
Positive matches cause the rewrite, negative matches intentionally allow
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    89
the original request to pass.  Rule order is consistent, top-down, first
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    90
match wins.  Fields are separated by any amount of whitespace (spaces or
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    91
tabs.)
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    92
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    93
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    94
### Positive matches:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    95
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    96
    First field: the hostname to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    97
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    98
      You can use an exact hostname (www.example.com), or the top level
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    99
      domain (tld) if you want to match everything under a specific host
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   100
      (example.com.)  You can also use a single '*' to match every request,
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   101
      though this essentially bypasses a lot of what makes volta quick, it
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   102
      is included for completeness.  You may have an unlimited amount of
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   103
      rules per hostname.  Hostnames are compared without case sensitivity.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   104
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   105
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   106
    Second field: the path to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   107
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   108
	  This can be an exact match ('/path/to/something.html'), a regular
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   109
	  expression ('\.(jpg|gif|png)$'), or a single '*' to match for any
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   110
	  path. Regular expressions are matched without case sensitivity.  There
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   111
	  is currently no internal support for captures, though you can use
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   112
	  a Lua rule (see below) for more complex processing.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   113
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   114
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   115
    Third field: The redirect code and url to rewrite to.
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   116
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   117
      Any pieces of a url that are omitted are automatically replaced
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   118
      with the original request's element -- the exception is a hostname,
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   119
      which is required.  If you omit a redirect code, the URL rewrite is
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   120
      transparent to the client.  You can attach a 301: or 302: prefix to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   121
      cause a permanent or temporary code to be respectively sent, instead.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   122
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   123
      If you require more complex processing than what volta provides
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   124
      internally, you can also specify a path to a Lua script (prefixed
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   125
      with 'lua:'.)  See the 'Lua rules' section of this README for more
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   126
	  information.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   127
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   128
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   129
### Negative matches:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   130
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   131
    First field: the hostname to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   132
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   133
	  See above -- all the same rules apply.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   134
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   135
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   136
    Second field: the path to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   137
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   138
	  See above -- all the same rules apply.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   139
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   140
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   141
	Third field: the 'negative' marker.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   142
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   143
	  This is simply the '-' character, that signals to volta that this is
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   144
	  a negative matching rule.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   145
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   146
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   147
You can easily test your rules by running volta on the command line, and
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   148
pasting URLs into it.   Boost the debug level (-d4) if you're having any issues.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   149
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   150
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   151
Examples
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   152
--------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   153
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   154
Rewrite all requests to Google to the SSL version:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   155
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   156
    google.com * 302:https://www.google.com
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   157
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   158
	This will redirect the request "http://www.google.com/search?q=test" to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   159
	"https://www.google.com/search?q=test".
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   160
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   161
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   162
Transparently alter all uploaded images on imgur to be my face:  :)
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   163
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   164
	i.imgur.com \.(gif|png|jpg)$ http://www.martini.nu/images/mahlon.jpg
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   165
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   166
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   167
Expand a local, non qualified hostname to a FQDN (useful alongside the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   168
'dns_defnames' squid setting to enforce browser proxy behaviors):
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   169
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   170
	local-example * local-example.company.com
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   171
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   172
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   173
Cause all blog content except for 2011 posts to permanently redirect to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   174
an archival page:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   175
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   176
	martini.nu /blog/2011 -
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   177
	martini.nu /blog 301:martini.nu/content-archived.html
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   178
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   179
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   180
Send all requests to reddit/r/WTF/* through a lua script for further processing.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   181
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   182
	reddit.com /r/wtf lua:/path/to/a/lua-script.lua
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   183
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   184
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   185
Turn off rewriting for specific network segment or IP address:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   186
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   187
	Squid has this ability built in -- see the 'url_rewrite_access' setting.
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   188
	Alternatively, do the checks in lua.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   189
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   190
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   191
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   192
Lua Rules
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   193
---------
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   194
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   195
Volta has an embedded Lua interpreter that you can use to perform all
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   196
kinds of conditional rewrites.  Read more about the syntax of the Lua
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   197
language here: http://www.lua.org/manual/5.1/
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   198
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   199
### Loading a script
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   200
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   201
To use a Lua script, prefix the rewrite target of a volta rule with
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   202
'lua:'.  The rest of the target is then treated as a path to the script.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   203
(You can find an example in the Examples section of this README.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   204
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   205
You can specify a path to either an ascii file, or Lua bytecode. (If
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   206
speed is an absolute premium, I'm seeing around a 25% performance
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   207
increase by using Lua bytecode files.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   208
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   209
You can use different scripts for different rules, or use the same
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   210
script across any number of separate rules.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   211
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   212
There is no need to restart squid when modifying Lua rules.  Changes are
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   213
seen immediately.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   214
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   215
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   216
### Environment
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   217
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   218
* Global variable declarations are disabled, so scripts can't accidently stomp on each other.  All variables must be declared with the 'local' keyword.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   219
* There is a global table called 'shared' you may use if you want to share data between separate scripts, or remember things in-between rule evaluations.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   220
* The details of the request can be found in a table, appropriately named 'request'.  HTTP scheme, host, path, port, method, client_ip, and domain are all available by default from the request table.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   221
* Calling Lua's print() function emits debug information to stderr.  Use a debug level of 2 or higher to see it.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   222
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   223
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   224
### Return value
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   225
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   226
The return value of the script is sent unmodified to squid, which should
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   227
be a URL the request is rewritten to, with an optional redirect code
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   228
prefix (301 or 302.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   229
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   230
Omitting a return value, or returning 'nil' has the same effect as a negative
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   231
rule match -- the original request is allowed through without any rewrite.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   232
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   233
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   234
An extremely simple Lua rule script can be found in the 'examples'
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   235
directory, distributed with volta.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   236
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   237
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   238