Fri, 04 Nov 2011 20:34:28 -0700 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu> [Fri, 04 Nov 2011 20:34:28 -0700] rev 14
There isn't a fast way to look up ( value exists or null ) for every piece of data that we're handed from squid -- a full table scan must be performed for each request, regardless of any indexing in place. With a decent number of rules, it's much, much slower than I anticipated, making it fairly pointless to even do parsing in ragel for speed. Trading some matching features (IPs, for now) for speed, and aiming at just doing good host/path matching at the moment, using CDB for fast host keying. Ripped out all the SQLite work, replacing it with a first round of CDB lookups. Added a ascii rulefile to cdb converter flag. Churning through the requests again! Also, fixed an off-by-one error with line extending.
Mon, 31 Oct 2011 17:17:07 -0700 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com> [Mon, 31 Oct 2011 17:17:07 -0700] rev 13
1st iteration of volta actually doing something. Process the request, find the best matching rule, and rewrite the request. Without the DB queries, volta was parsing over 750k requests a second. Currently, it's down to 129.5 with 1161 rules in place. Yikes. I may need to re-evaluate some design choices here.
(0) -10 -2 +2 +10 tip