author | mahlon |
Sat, 08 Nov 2008 18:59:05 +0000 | |
branch | ruby-modules |
changeset 1 | 9e127bf6e84f |
child 2 | e5c705047540 |
permissions | -rw-r--r-- |
1
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
1 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
2 |
# Chunker! |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
3 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
4 |
# Mahlon E. Smith <mahlon@martini.nu> |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
5 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
6 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
7 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
8 |
### Namespace for the datablock parser. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
9 |
### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
10 |
module Chunker |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
11 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
12 |
require 'strscan' |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
13 |
require 'stringio' |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
14 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
15 |
# SVN Revision |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
16 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
17 |
SVNRev = %q$Rev$ |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
18 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
19 |
# SVN Id |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
20 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
21 |
SVNId = %q$Id$ |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
22 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
23 |
# Package version |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
24 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
25 |
VERSION = '0.1' |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
26 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
27 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
28 |
### Parser class for __END__ data blocks. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
29 |
### Find each __MARKER__ within the __END__, and put each into a |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
30 |
### DATA_MARKER constant within the namespace that included us. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
31 |
### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
32 |
class DataParser |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
33 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
34 |
# The mark for a DATA block. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
35 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
36 |
END_MARKER = /^__END__\r?\n/ |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
37 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
38 |
# The mark for a 'sub' block. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
39 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
40 |
CHUNK_MARKER = /^__([A-Z\_0-9]+)__\r?\n/ |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
41 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
42 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
43 |
### Constructor: Given a +klass+ and an +io+ to the class file, |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
44 |
### extract the data blocks and install constants. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
45 |
### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
46 |
def initialize( klass, io ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
47 |
io.open if io.closed? |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
48 |
end_string = io.read.split( END_MARKER, 2 ).last |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
49 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
50 |
@klass = klass |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
51 |
@scanner = StringScanner.new( end_string ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
52 |
io.close |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
53 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
54 |
if @scanner.check_until( CHUNK_MARKER ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
55 |
# put each chunk into its own constant |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
56 |
self.extract_blocks |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
57 |
else |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
58 |
# no sub blocks, put the whole mess into DATA_END |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
59 |
@klass.const_set( :DATA_END, StringIO.new( end_string ) ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
60 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
61 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
62 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
63 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
64 |
######### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
65 |
protected |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
66 |
######### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
67 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
68 |
### Parse the current +io+ for data blocks, set contents to |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
69 |
### IO constants in the including class. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
70 |
### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
71 |
def extract_blocks |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
72 |
label = nil |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
73 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
74 |
while @scanner.scan_until( CHUNK_MARKER ) and ! @scanner.eos? |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
75 |
data = '' |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
76 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
77 |
# First pass, __END__ contents (until next marker, instead |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
78 |
# of entire data block.) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
79 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
80 |
if label.nil? |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
81 |
label = 'END' |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
82 |
data = @scanner.pre_match |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
83 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
84 |
@scanner.pos = self.next_position |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
85 |
else |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
86 |
label = @scanner[1] |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
87 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
88 |
if data = @scanner.scan_until( CHUNK_MARKER ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
89 |
# Pull the next marker text out of the data, set up the next pass |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
90 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
91 |
data = data[ 0, data.length - @scanner[0].length ] |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
92 |
@scanner.pos = self.next_position |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
93 |
else |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
94 |
# No additional blocks |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
95 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
96 |
data = @scanner.rest |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
97 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
98 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
99 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
100 |
# Add the IO constant to the class that included me. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
101 |
# |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
102 |
@klass.const_set( "DATA_#{label}".to_sym, StringIO.new( data ) ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
103 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
104 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
105 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
106 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
107 |
### Return the next scanner position for searching. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
108 |
### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
109 |
def next_position |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
110 |
return @scanner.pos - @scanner[0].length |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
111 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
112 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
113 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
114 |
|
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
115 |
### Included hook: Find the file path for how we arrived here, and open |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
116 |
### it as an IO object. __FILE__ won't work, so we find it via caller(). |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
117 |
### Start parsing this file for data blocks. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
118 |
### |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
119 |
def self.included( klass ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
120 |
# klass.instance_eval{ __FILE__ } awww, nope. |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
121 |
io = File.open( caller(1).last.sub(/:.*?$/, ''), 'r' ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
122 |
DataParser.new( klass, io ) |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
123 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
124 |
end |
9e127bf6e84f
Initial commit of chunker, a ruby module to aid with data blocks.
mahlon
parents:
diff
changeset
|
125 |