README
author Mahlon E. Smith <mahlon@martini.nu>
Tue, 09 Jun 2015 16:36:16 -0700
changeset 33 ba41bfbe87a2
parent 30 5cc836e06759
permissions -rw-r--r--
Updates to support the newer Squid rewrite syntax.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     1
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     2
Volta
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     3
=====
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     4
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
     5
What is volta?
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     6
--------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     7
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     8
Volta is a high performance, low resource URI rewriter for use with the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     9
Squid caching proxy server (http://www.squid-cache.org/.)  With it, you
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    10
can dynamically alter URI requests that pass through Squid based on
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    11
various criteria.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    12
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    13
It uses a state machine to parse URIs and rules, and a constant database
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    14
to store and access those rules.  It can then either perform conditional
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    15
rewrites internally, or by evaluating Lua scripts.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    16
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    17
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    18
Why is it called "volta"?
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    19
-------------------------
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    20
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    21
It's a type of old Italian music written in triple-time.  Quick!
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    22
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    23
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    24
How fast is it?
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    25
---------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    26
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    27
On a 2Ghz Xeon 5130, it can process a million squid requests against
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    28
10000 rules in less than 8 seconds, using about 800k of ram.  On an
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    29
1.8Ghz Intel E4300, it can do it in 3 seconds.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    30
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    31
Your mileage may vary, but for most all intents and purposes the answer
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    32
is "definitely fast enough."
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    33
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    34
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    35
Configuring squid
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    36
-----------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    37
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    38
You must enable url rewriting from within the squid.conf file.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    39
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    40
	url_rewrite_program /usr/local/bin/volta
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    41
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    42
... and that's it.  You may need some additional customization, like where
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    43
the volta database is stored on disk:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    44
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    45
	url_rewrite_program /usr/local/bin/volta -f /var/db/squid/volta.db
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    46
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    47
Busy servers:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    48
29
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    49
While Volta is lightweight enough to simply increase the amount of
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    50
rewriter children, it also supports Squid's rewrite_concurrency format
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    51
if you find that to be more efficient for your environment.  Adjust to
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    52
taste.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    53
33
ba41bfbe87a2 Updates to support the newer Squid rewrite syntax.
Mahlon E. Smith <mahlon@martini.nu>
parents: 30
diff changeset
    54
	url_rewrite_children 5 startup=1 idle=2 concurrency=50
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    55
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    56
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    57
Using volta
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    58
-----------
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    59
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    60
See the INSTALL file for instructions on how to compile volta.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    61
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    62
Volta reads its rewrite rules from a local database.  You can create the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    63
rules in a text editor, then convert it to the database like so:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    64
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    65
	% volta -c rules.txt
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    66
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    67
You'll be left with a "volta.db" file in the current directory.  Put it
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    68
wherever you please, and use the -f flag to point to it.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    69
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    70
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    71
Rule file syntax
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    72
----------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    73
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    74
Volta's rule syntax is designed to be easy to parse by humans and
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    75
machines.  Blank lines are skipped, as is any line that starts with the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    76
'#' character, so you can keep the ascii version of your rules well
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    77
documented and in version control.  There is no practical limit on the
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    78
number of rules in this database.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    79
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    80
When compiling the ruleset into the database format, volta detects
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    81
malformed rules and stops if there are any problems, leaving your
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    82
original database intact.  You can change the ruleset at any time while
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    83
volta is running, and the new rules will take affect within about 10
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    84
seconds.  No need to restart squid!
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    85
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    86
There are two types of rules -- positive matches, and negative matches.
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    87
Positive matches cause the rewrite, negative matches intentionally allow
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    88
the original request to pass.  Rule order is consistent, top-down, first
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    89
match wins.  Fields are separated by any amount of whitespace (spaces or
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    90
tabs.)
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    91
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    92
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    93
### Positive matches:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    94
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    95
    First field: the hostname to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    96
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    97
      You can use an exact hostname (www.example.com), or the top level
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    98
      domain (tld) if you want to match everything under a specific host
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    99
      (example.com.)  You can also use a single '*' to match every request,
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   100
      though this essentially bypasses a lot of what makes volta quick, it
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   101
      is included for completeness.  You may have an unlimited amount of
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   102
      rules per hostname.  Hostnames are compared without case sensitivity.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   103
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   104
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   105
    Second field: the path to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   106
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   107
	  This can be an exact match ('/path/to/something.html'), a regular
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   108
	  expression ('\.(jpg|gif|png)$'), or a single '*' to match for any
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   109
	  path. Regular expressions are matched without case sensitivity.  There
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   110
	  is currently no internal support for captures, though you can use
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   111
	  a Lua rule (see below) for more complex processing.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   112
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   113
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   114
    Third field: The redirect code and url to rewrite to.
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   115
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   116
      Any pieces of a url that are omitted are automatically replaced
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   117
      with the original request's element -- the exception is a hostname,
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   118
      which is required.  If you omit a redirect code, the URL rewrite is
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   119
      transparent to the client.  You can attach a 301: or 302: prefix to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   120
      cause a permanent or temporary code to be respectively sent, instead.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   121
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   122
      If you require more complex processing than what volta provides
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   123
      internally, you can also specify a path to a Lua script (prefixed
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   124
      with 'lua:'.)  See the 'Lua rules' section of this README for more
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   125
	  information.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   126
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   127
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   128
### Negative matches:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   129
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   130
    First field: the hostname to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   131
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   132
	  See above -- all the same rules apply.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   133
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   134
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   135
    Second field: the path to match.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   136
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   137
	  See above -- all the same rules apply.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   138
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   139
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   140
	Third field: the 'negative' marker.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   141
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   142
	  This is simply the '-' character, that signals to volta that this is
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   143
	  a negative matching rule.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   144
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   145
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   146
You can easily test your rules by running volta on the command line, and
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   147
pasting URLs into it.   Boost the debug level (-d4) if you're having any issues.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   148
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   149
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   150
Examples
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   151
--------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   152
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   153
Rewrite all requests to Google to the SSL version:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   154
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   155
    google.com * 302:https://www.google.com
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   156
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   157
	This will redirect the request "http://www.google.com/search?q=test" to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   158
	"https://www.google.com/search?q=test".
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   159
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   160
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   161
Transparently alter all uploaded images on imgur to be my face:  :)
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   162
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   163
	i.imgur.com \.(gif|png|jpg)$ http://www.martini.nu/images/mahlon.jpg
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   164
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   165
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   166
Expand a local, non qualified hostname to a FQDN (useful alongside the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   167
'dns_defnames' squid setting to enforce browser proxy behaviors):
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   168
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   169
	local-example * local-example.company.com
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   170
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   171
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   172
Cause all blog content except for 2011 posts to permanently redirect to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   173
an archival page:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   174
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   175
	martini.nu /blog/2011 -
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   176
	martini.nu /blog 301:martini.nu/content-archived.html
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   177
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   178
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   179
Send all requests to reddit/r/WTF/* through a lua script for further processing.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   180
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   181
	reddit.com /r/wtf lua:/path/to/a/lua-script.lua
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   182
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   183
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   184
Turn off rewriting for specific network segment or IP address:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   185
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   186
	Squid has this ability built in -- see the 'url_rewrite_access' setting.
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   187
	Alternatively, do the checks in lua.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   188
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   189
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   190
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   191
Lua Rules
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   192
---------
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   193
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   194
Volta has an embedded Lua interpreter that you can use to perform all
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   195
kinds of conditional rewrites.  Read more about the syntax of the Lua
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   196
language here: http://www.lua.org/manual/5.1/
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   197
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   198
### Loading a script
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   199
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   200
To use a Lua script, prefix the rewrite target of a volta rule with
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   201
'lua:'.  The rest of the target is then treated as a path to the script.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   202
(You can find an example in the Examples section of this README.)
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   203
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   204
You can specify a path to either an ascii file, or Lua bytecode. (If
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   205
speed is an absolute premium, I'm seeing around a 25% performance
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   206
increase by using Lua bytecode files.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   207
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   208
You can use different scripts for different rules, or use the same
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   209
script across any number of separate rules.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   210
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   211
There is no need to restart squid when modifying Lua rules.  Changes are
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   212
seen immediately.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   213
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   214
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   215
### Environment
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   216
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   217
* Global variable declarations are disabled, so scripts can't accidently stomp on each other.  All variables must be declared with the 'local' keyword.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   218
* There is a global table called 'shared' you may use if you want to share data between separate scripts, or remember things in-between rule evaluations.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   219
* The details of the request can be found in a table, appropriately named 'request'.  HTTP scheme, host, path, port, method, client_ip, and domain are all available by default from the request table.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   220
* Calling Lua's print() function emits debug information to stderr.  Use a debug level of 2 or higher to see it.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   221
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   222
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   223
### Return value
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   224
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   225
The return value of the script is sent unmodified to squid, which should
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   226
be a URL the request is rewritten to, with an optional redirect code
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   227
prefix (301 or 302.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   228
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   229
Omitting a return value, or returning 'nil' has the same effect as a negative
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   230
rule match -- the original request is allowed through without any rewrite.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   231
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   232
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   233
An extremely simple Lua rule script can be found in the 'examples'
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   234
directory, distributed with volta.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   235
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   236