db.c
author Mahlon E. Smith <mahlon@martini.nu>
Fri, 04 Nov 2011 20:34:28 -0700
changeset 14 51eb85ae4de4
parent 13 23a242d7b7fa
child 15 2706fc514dea
permissions -rw-r--r--
There isn't a fast way to look up ( value exists or null ) for every piece of data that we're handed from squid -- a full table scan must be performed for each request, regardless of any indexing in place. With a decent number of rules, it's much, much slower than I anticipated, making it fairly pointless to even do parsing in ragel for speed. Trading some matching features (IPs, for now) for speed, and aiming at just doing good host/path matching at the moment, using CDB for fast host keying. Ripped out all the SQLite work, replacing it with a first round of CDB lookups. Added a ascii rulefile to cdb converter flag. Churning through the requests again! Also, fixed an off-by-one error with line extending.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     1
/* vim: set noet nosta sw=4 ts=4 ft=c : */
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     2
/*
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     3
Copyright (c) 2011, Mahlon E. Smith <mahlon@martini.nu>
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     4
All rights reserved.
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     5
Redistribution and use in source and binary forms, with or without
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     6
modification, are permitted provided that the following conditions are met:
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     7
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     8
    * Redistributions of source code must retain the above copyright
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     9
      notice, this list of conditions and the following disclaimer.
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    10
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    11
    * Redistributions in binary form must reproduce the above copyright
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    12
      notice, this list of conditions and the following disclaimer in the
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    13
      documentation and/or other materials provided with the distribution.
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    14
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    15
    * Neither the name of Mahlon E. Smith nor the names of his
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    16
      contributors may be used to endorse or promote products derived
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    17
      from this software without specific prior written permission.
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    18
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    19
THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    20
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    21
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    22
DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    23
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    24
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    25
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    26
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    27
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    28
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    29
*/
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    30
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    31
#include "volta.h"
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    32
#include "db.h"
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    33
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    34
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    35
/*
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    36
 * Open the database specified in the 'v' global struct,
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    37
 * setting the file descriptor.  Returns 0 on success.
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    38
 *
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    39
 */
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    40
short int
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    41
db_attach( void )
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    42
{
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    43
	/* only re-open the db at most every 10 seconds */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    44
	time_t now = time( NULL );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    45
	if ( v.timer.db_lastcheck > 0 ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    46
		if ( now - v.timer.db_lastcheck >= 10 ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    47
			close( v.db_fd );
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    48
		}
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    49
		else {
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    50
			return( 0 );
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    51
		}
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    52
	}
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    53
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    54
	debug( 2, LOC, "(Re)attaching to database '%s'\n", v.dbname );
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    55
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    56
	/* db filename not set? */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    57
	if ( strlen(v.dbname) == 0 ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    58
		debug( 1, LOC, "Error when attaching to database: DB filename unset?\n" );
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    59
		return( -1 );
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    60
	}
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    61
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    62
	if ( (v.db_fd = open( v.dbname, O_RDONLY )) == -1 ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    63
		debug( 1, LOC, "Error when attaching to database: %s\n", strerror(errno) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    64
		return( -1 );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    65
	}
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    66
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    67
	v.timer.db_lastcheck = now;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    68
	return( 0 );
4
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    69
}
5701b7859a31 Groundwork for automatic database initialization and schema upgrades.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    70
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    71
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    72
/*
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    73
 * Given a rule file in ascii specified by the -c command line arg,
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    74
 * convert it to a cdb after checking rules for validity.
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    75
 *
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    76
 */
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    77
unsigned short int
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    78
db_create_new( char *txt )
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    79
{
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    80
	struct cdb_make cdbm;
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    81
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    82
	char buf[ LINE_BUFSIZE*10 ];
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    83
	char tmpfile[25];
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    84
	int  tmp_fd;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    85
	FILE *txt_f = NULL;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    86
	int  linenum = 0, parsed = 0;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    87
	struct db_input *dbline;
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    88
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    89
	/* open temporary file */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    90
	debug( 0, LOC, "Creating/updating database (%s) using rules in \"%s\"\n", v.dbname, txt );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    91
	sprintf( tmpfile, "/tmp/volta-db-%d.tmp", getpid() );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    92
	if ( (tmp_fd = open( tmpfile,
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    93
						 O_RDWR|O_CREAT, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH )) == -1 ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    94
		debug( 0, LOC, "Error writing temporary file: %s\n", strerror(errno) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    95
		return( 1 );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    96
	}
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
    97
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    98
	/* open rules file */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
    99
	if ( (txt_f = fopen( txt, "r" )) == NULL ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   100
		debug( 0, LOC, "Error reading rules file: %s\n", strerror(errno) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   101
		return( 1 );
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   102
	}
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   103
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   104
	/* init struct and start parsing lines */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   105
	cdb_make_start( &cdbm, tmp_fd );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   106
	while ( fgets( buf, LINE_BUFSIZE*10, txt_f ) != NULL ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   107
		linenum++;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   108
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   109
		/* skip blank lines and comments */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   110
		if ( strlen(buf) == 1 || buf[0] == '#' ) continue;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   111
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   112
		/* validate and add! */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   113
		dbline = parse_dbinput( buf );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   114
		if ( dbline == NULL ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   115
			debug( 0, LOC, "Invalid rule (line %d), skipping: %s", linenum, buf );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   116
			continue;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   117
		}
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   118
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   119
		cdb_make_add( &cdbm, dbline->key, dbline->klen, dbline->val, dbline->vlen );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   120
		parsed++;
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   121
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   122
		free( dbline->key );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   123
		free( dbline->val );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   124
		free( dbline );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   125
	}
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   126
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   127
	/* write indexes */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   128
	fclose( txt_f );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   129
	cdb_make_finish( &cdbm );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   130
	close( tmp_fd );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   131
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   132
	/* move cdb into place */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   133
	if ( (rename( tmpfile, v.dbname )) == -1 ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   134
		debug( 1, LOC, "Unable to move temp cdb into place: %s", strerror(errno) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   135
		return( 1 );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   136
	}
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   137
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   138
	debug( 0, LOC, "Added %d rules to %s.\n", parsed, v.dbname );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   139
	return( 0 );
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   140
}
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   141
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   142
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   143
/* Fast single record lookup.
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   144
 * Returns a pointer to the found value or NULL if there is no match.
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   145
 *
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   146
 * The returned pointer should be freed after use.
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   147
 *
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   148
 */
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   149
char *
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   150
find_record( char *key )
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   151
{
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   152
	if ( key == NULL ) return( NULL );
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   153
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   154
	char *val = NULL;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   155
	cdbi_t vlen;
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   156
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   157
	if ( cdb_seek( v.db_fd, key, (int)strlen(key), &vlen) > 0 ) {
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   158
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   159
		if ( (val = malloc( vlen + 1 )) == NULL ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   160
			debug( 5, LOC, "Unable to allocate memory for value storage: %s\n", strerror(errno) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   161
			return( NULL );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   162
		}
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   163
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   164
		cdb_bread( v.db_fd, val, vlen );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   165
		val[vlen] = '\0';
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   166
		debug( 4, LOC, "Match for key '%s': %s\n", key, val );
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   167
	}
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   168
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   169
	return val;
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   170
}
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   171
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   172
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   173
/* 
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   174
 * Search the CDB for all occurences of the given +key+,
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   175
 * populating the +results+ array with pointers to parsed rule structs.
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   176
 *
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   177
 * Returns the number of successful matches.  reset_results()
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   178
 * should be called after the result set is examined.
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   179
 *
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   180
 */
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   181
unsigned int
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   182
find_records( char *key, parsed **results )
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   183
{
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   184
	if ( key == NULL ) return( 0 );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   185
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   186
	struct cdb cdb;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   187
	struct cdb_find cdbf; /* structure to hold current find position */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   188
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   189
	unsigned int match = 0;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   190
	parsed *result     = NULL;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   191
	char *val          = NULL;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   192
	unsigned int vlen, vpos;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   193
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   194
	/* initialize search structs */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   195
	if ( db_attach() == -1 ) return( 0 );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   196
	cdb_init( &cdb, v.db_fd );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   197
	cdb_findinit( &cdbf, &cdb, key, (int)strlen(key) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   198
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   199
	while ( cdb_findnext( &cdbf ) > 0 && match < DB_RESULTS_MAX ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   200
		vpos = cdb_datapos( &cdb );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   201
		vlen = cdb_datalen( &cdb );
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   202
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   203
		/* pull the value from the db */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   204
		if ( (val = calloc( vlen, sizeof(char) )) == NULL ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   205
			debug( 5, LOC, "Unable to allocate memory for DB value storage: %s\n",
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   206
					strerror(errno) );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   207
			return( 0 );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   208
		}
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   209
		cdb_read( &cdb, val, vlen, vpos );
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   210
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   211
		/* if it parses properly, add it to the result set. */
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   212
		result = parse_rule( val );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   213
		if ( result != NULL ) {
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   214
			results[match] = result;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   215
			debug( 4, LOC, "DB match %d for key '%s': %s\n", match+1, key, val );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   216
		}
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   217
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   218
		match++;
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   219
		free( val );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   220
	}
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   221
14
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   222
	cdb_free( &cdb );
51eb85ae4de4 There isn't a fast way to look up ( value exists or null ) for every
Mahlon E. Smith <mahlon@martini.nu>
parents: 13
diff changeset
   223
	return match;
13
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   224
}
23a242d7b7fa 1st iteration of volta actually doing something. Process the request,
Mahlon E. Smith <mahlon@laika.com>
parents: 10
diff changeset
   225