README.md
author Mahlon E. Smith <mahlon@martini.nu>
Mon, 02 Sep 2019 09:27:03 -0700
changeset 37 39ad26216bbc
parent 35 c24dbd004cbc
permissions -rw-r--r--
Bye, Bitbucket.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     1
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     2
Volta
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     3
=====
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
     4
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
     5
What is volta?
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     6
--------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     7
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     8
Volta is a high performance, low resource URI rewriter for use with the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
     9
Squid caching proxy server (http://www.squid-cache.org/.)  With it, you
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    10
can dynamically alter URI requests that pass through Squid based on
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    11
various criteria.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    12
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    13
It uses a state machine to parse URIs and rules, and a constant database
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    14
to store and access those rules.  It can then either perform conditional
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    15
rewrites internally, or by evaluating Lua scripts.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    16
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    17
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    18
Why is it called "volta"?
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    19
-------------------------
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    20
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    21
It's a type of old Italian music written in triple-time.  Quick!
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    22
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    23
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    24
How fast is it?
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    25
---------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    26
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    27
On a 2Ghz Xeon 5130, it can process a million squid requests against
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    28
10000 rules in less than 8 seconds, using about 800k of ram.  On an
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    29
1.8Ghz Intel E4300, it can do it in 3 seconds.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    30
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    31
Your mileage may vary, but for most all intents and purposes the answer
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    32
is "definitely fast enough."
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    33
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    34
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    35
Configuring squid
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    36
-----------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    37
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    38
You must enable url rewriting from within the squid.conf file.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    39
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    40
	url_rewrite_program /usr/local/bin/volta
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    41
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    42
... and that's it.  You may need some additional customization, like where
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    43
the volta database is stored on disk:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    44
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    45
	url_rewrite_program /usr/local/bin/volta -f /var/db/squid/volta.db
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    46
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    47
Busy servers:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    48
29
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    49
While Volta is lightweight enough to simply increase the amount of
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    50
rewriter children, it also supports Squid's rewrite_concurrency format
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    51
if you find that to be more efficient for your environment.  Adjust to
c5d00a24af56 Add support for Squid's 'url_rewriter_concurrency' pipelining.
Mahlon E. Smith <mahlon@laika.com>
parents: 22
diff changeset
    52
taste.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    53
33
ba41bfbe87a2 Updates to support the newer Squid rewrite syntax.
Mahlon E. Smith <mahlon@martini.nu>
parents: 30
diff changeset
    54
	url_rewrite_children 5 startup=1 idle=2 concurrency=50
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    55
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    56
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    57
Using volta
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    58
-----------
10
d07309450285 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu>
parents:
diff changeset
    59
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    60
See the INSTALL file for instructions on how to compile volta.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    61
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    62
Volta reads its rewrite rules from a local database.  You can create the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    63
rules in a text editor, then convert it to the database like so:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    64
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    65
	% volta -c rules.txt
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    66
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    67
You'll be left with a "volta.db" file in the current directory.  Put it
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    68
wherever you please, and use the -f flag to point to it.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    69
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    70
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    71
Rule file syntax
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    72
----------------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    73
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    74
Volta's rule syntax is designed to be easy to parse by humans and
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    75
machines.  Blank lines are skipped, as is any line that starts with the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    76
'#' character, so you can keep the ascii version of your rules well
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    77
documented and in version control.  There is no practical limit on the
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    78
number of rules in this database.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    79
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    80
When compiling the ruleset into the database format, volta detects
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    81
malformed rules and stops if there are any problems, leaving your
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    82
original database intact.  You can change the ruleset at any time while
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    83
volta is running, and the new rules will take affect within about 10
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    84
seconds.  No need to restart squid!
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    85
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    86
There are two types of rules -- positive matches, and negative matches.
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    87
Positive matches cause the rewrite, negative matches intentionally allow
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    88
the original request to pass.  Rule order is consistent, top-down, first
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    89
match wins.  Fields are separated by any amount of whitespace (spaces or
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
    90
tabs.)
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    91
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    92
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    93
### Positive matches:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    94
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
    95
**First field**: *the hostname to match*
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
    96
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
    97
	You can use an exact hostname (www.example.com), or the top level
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
    98
	domain (tld) if you want to match everything under a specific host
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
    99
	(example.com.)  You can also use a single '*' to match every request,
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   100
	though this essentially bypasses a lot of what makes volta quick, it
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   101
	is included for completeness.  You may have an unlimited amount of
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   102
	rules per hostname.  Hostnames are compared without case sensitivity.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   103
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   104
**Second field**: *the path to match*
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   105
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   106
	This can be an exact match ('/path/to/something.html'), a regular
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   107
	expression ('\.(jpg|gif|png)$'), or a single '*' to match for any
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   108
	path. Regular expressions are matched without case sensitivity.  There
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   109
	is currently no internal support for captures, though you can use
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   110
	a Lua rule (see below) for more complex processing.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   111
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   112
**Third field**: *the redirect code and url to rewrite to*
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   113
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   114
	Any pieces of a url that are omitted are automatically replaced
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   115
	with the original request's element -- the exception is a hostname,
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   116
	which is required.  If you omit a redirect code, the URL rewrite is
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   117
	transparent to the client.  You can attach a 301: or 302: prefix to
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   118
	cause a permanent or temporary code to be respectively sent, instead.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   119
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   120
	If you require more complex processing than what volta provides
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   121
	internally, you can also specify a path to a Lua script (prefixed
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   122
	with 'lua:'.)  See the 'Lua rules' section of this README for more
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   123
	information.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   124
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   125
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   126
### Negative matches:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   127
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   128
**First field**: *the hostname to match*
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   129
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   130
See above -- all the same rules apply.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   131
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   132
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   133
**Second field**: *the path to match*
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   134
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   135
See above -- all the same rules apply.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   136
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   137
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   138
**Third field**: *the 'negative' marker*
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   139
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   140
This is simply the '-' character, that signals to volta that this is
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   141
a negative matching rule.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   142
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   143
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   144
You can easily test your rules by running volta on the command line, and
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   145
pasting URLs into it.   Boost the debug level (-d4) if you're having any issues.
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   146
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   147
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   148
Examples
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   149
--------
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   150
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   151
Rewrite all requests to Google to the SSL version:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   152
35
c24dbd004cbc Rename INSTALL and README for Bitbucket formatting.
Mahlon E. Smith <mahlon@martini.nu>
parents: 33
diff changeset
   153
	google.com * 302:https://www.google.com
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   154
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   155
	This will redirect the request "http://www.google.com/search?q=test" to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   156
	"https://www.google.com/search?q=test".
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   157
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   158
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   159
Transparently alter all uploaded images on imgur to be my face:  :)
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   160
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   161
	i.imgur.com \.(gif|png|jpg)$ http://www.martini.nu/images/mahlon.jpg
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   162
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   163
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   164
Expand a local, non qualified hostname to a FQDN (useful alongside the
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   165
'dns_defnames' squid setting to enforce browser proxy behaviors):
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   166
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   167
	local-example * local-example.company.com
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   168
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   169
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   170
Cause all blog content except for 2011 posts to permanently redirect to
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   171
an archival page:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   172
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   173
	martini.nu /blog/2011 -
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   174
	martini.nu /blog 301:martini.nu/content-archived.html
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   175
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   176
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   177
Send all requests to reddit/r/WTF/* through a lua script for further processing.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   178
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   179
	reddit.com /r/wtf lua:/path/to/a/lua-script.lua
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   180
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   181
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   182
Turn off rewriting for specific network segment or IP address:
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   183
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   184
	Squid has this ability built in -- see the 'url_rewrite_access' setting.
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   185
	Alternatively, do the checks in lua.
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   186
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   187
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   188
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   189
Lua Rules
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   190
---------
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   191
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   192
Volta has an embedded Lua interpreter that you can use to perform all
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   193
kinds of conditional rewrites.  Read more about the syntax of the Lua
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   194
language here: http://www.lua.org/manual/5.1/
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   195
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   196
### Loading a script
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   197
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   198
To use a Lua script, prefix the rewrite target of a volta rule with
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   199
'lua:'.  The rest of the target is then treated as a path to the script.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   200
(You can find an example in the Examples section of this README.)
18
d4ce82194b64 - 1st pass at documentation with the README.
Mahlon E. Smith <mahlon@laika.com>
parents: 15
diff changeset
   201
22
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   202
You can specify a path to either an ascii file, or Lua bytecode. (If
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   203
speed is an absolute premium, I'm seeing around a 25% performance
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   204
increase by using Lua bytecode files.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   205
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   206
You can use different scripts for different rules, or use the same
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   207
script across any number of separate rules.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   208
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   209
There is no need to restart squid when modifying Lua rules.  Changes are
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   210
seen immediately.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   211
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   212
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   213
### Environment
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   214
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   215
* Global variable declarations are disabled, so scripts can't accidently stomp on each other.  All variables must be declared with the 'local' keyword.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   216
* There is a global table called 'shared' you may use if you want to share data between separate scripts, or remember things in-between rule evaluations.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   217
* The details of the request can be found in a table, appropriately named 'request'.  HTTP scheme, host, path, port, method, client_ip, and domain are all available by default from the request table.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   218
* Calling Lua's print() function emits debug information to stderr.  Use a debug level of 2 or higher to see it.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   219
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   220
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   221
### Return value
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   222
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   223
The return value of the script is sent unmodified to squid, which should
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   224
be a URL the request is rewritten to, with an optional redirect code
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   225
prefix (301 or 302.)
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   226
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   227
Omitting a return value, or returning 'nil' has the same effect as a negative
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   228
rule match -- the original request is allowed through without any rewrite.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   229
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   230
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   231
An extremely simple Lua rule script can be found in the 'examples'
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   232
directory, distributed with volta.
822094314703 Add the ability to optionally script rewrite logic using Lua.
Mahlon E. Smith <mahlon@martini.nu>
parents: 18
diff changeset
   233