Fri, 04 Nov 2011 20:34:28 -0700 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu> [Fri, 04 Nov 2011 20:34:28 -0700] rev 14
There isn't a fast way to look up ( value exists or null ) for every piece of data that we're handed from squid -- a full table scan must be performed for each request, regardless of any indexing in place. With a decent number of rules, it's much, much slower than I anticipated, making it fairly pointless to even do parsing in ragel for speed. Trading some matching features (IPs, for now) for speed, and aiming at just doing good host/path matching at the moment, using CDB for fast host keying. Ripped out all the SQLite work, replacing it with a first round of CDB lookups. Added a ascii rulefile to cdb converter flag. Churning through the requests again! Also, fixed an off-by-one error with line extending.
Mon, 31 Oct 2011 17:17:07 -0700 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com> [Mon, 31 Oct 2011 17:17:07 -0700] rev 13
1st iteration of volta actually doing something. Process the request, find the best matching rule, and rewrite the request. Without the DB queries, volta was parsing over 750k requests a second. Currently, it's down to 129.5 with 1161 rules in place. Yikes. I may need to re-evaluate some design choices here.
Sun, 23 Oct 2011 22:59:59 -0700 Clean up redundant parser actions via preprocessor macros, add a
Mahlon E. Smith <mahlon@martini.nu> [Sun, 23 Oct 2011 22:59:59 -0700] rev 12
Clean up redundant parser actions via preprocessor macros, add a hostname->tld extractor for doing tld "wildcard" queries. Stop tracking 'parser.c' completely.
Mon, 17 Oct 2011 09:12:00 -0700 Carry the amalgamated sqlite3 around with volta, rather than requiring
Mahlon E. Smith <mahlon@martini.nu> [Mon, 17 Oct 2011 09:12:00 -0700] rev 11
Carry the amalgamated sqlite3 around with volta, rather than requiring it to be installed on the system -- making volta a zero dependency compile. Add some more Makefile hacks to get things consistently building across OSX, FreeBSD, and Ubuntu. (There may be more later, but I'll deal with those then.)
Mon, 17 Oct 2011 09:12:00 -0700 Get the ragel line parser properly tokenizing the input lines. Add a
Mahlon E. Smith <mahlon@martini.nu> [Mon, 17 Oct 2011 09:12:00 -0700] rev 10
Get the ragel line parser properly tokenizing the input lines. Add a "lines per second" timer. General cleanup and memory management.
Wed, 28 Sep 2011 09:04:16 -0700 Ansi-fy getopt function and variable declarations.
Mahlon E. Smith <mahlon@martini.nu> [Wed, 28 Sep 2011 09:04:16 -0700] rev 9
Ansi-fy getopt function and variable declarations.
Wed, 28 Sep 2011 09:04:16 -0700 Add common archive files (LICENCE, README, etc) and a 'release' Makefile
Mahlon E. Smith <mahlon@martini.nu> [Wed, 28 Sep 2011 09:04:16 -0700] rev 8
Add common archive files (LICENCE, README, etc) and a 'release' Makefile target.
Wed, 28 Sep 2011 09:04:16 -0700 Fine tuning for database initialization/upgrades.
Mahlon E. Smith <mahlon@martini.nu> [Wed, 28 Sep 2011 09:04:16 -0700] rev 7
Fine tuning for database initialization/upgrades.
Thu, 22 Sep 2011 21:43:50 -0700 Use pkg-config to find dependency locations, add gdb debug flags.
Mahlon E. Smith <mahlon@martini.nu> [Thu, 22 Sep 2011 21:43:50 -0700] rev 6
Use pkg-config to find dependency locations, add gdb debug flags.
Thu, 22 Sep 2011 21:43:25 -0700 Check the return value of fread() when loading SQL migration files.
Mahlon E. Smith <mahlon@martini.nu> [Thu, 22 Sep 2011 21:43:25 -0700] rev 5
Check the return value of fread() when loading SQL migration files.
Wed, 14 Sep 2011 16:49:28 -0700 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com> [Wed, 14 Sep 2011 16:49:28 -0700] rev 4
Groundwork for automatic database initialization and schema upgrades.
Tue, 13 Sep 2011 22:16:11 -0700 Whoops, lets not commit .o files. Fix .hgignore.
Mahlon E. Smith <mahlon@martini.nu> [Tue, 13 Sep 2011 22:16:11 -0700] rev 3
Whoops, lets not commit .o files. Fix .hgignore.
Tue, 13 Sep 2011 22:13:02 -0700 Ensure that parsing can't be subverted by requests larger than the
Mahlon E. Smith <mahlon@martini.nu> [Tue, 13 Sep 2011 22:13:02 -0700] rev 2
Ensure that parsing can't be subverted by requests larger than the default line buffer maximum size. If a line fits in the stack, great. If not, allocate additional memory for it instead of truncating. Add google-proftools to debug builds. Don't clean parser.c by default, so distributions don't require ragel to build. Add BSD licensing. Move helper functions out to util.c. Rename files to match function names. Start playing with sqlite API.
Sat, 03 Sep 2011 14:12:06 -0700 Dial in the Makefile and command line option parsing. Better debug
Mahlon E. Smith <mahlon@laika.com> [Sat, 03 Sep 2011 14:12:06 -0700] rev 1
Dial in the Makefile and command line option parsing. Better debug output.
(0) +14 tip