mahlon/ruby-chunker
1
0
Fork 0
Code Issues Pull requests Activity

* Rename 'markers' to 'token'

Browse source 
* Fix up the Rakefile's gem generation
* Add LICENSE
* Add a real README
This commit is contained in:
mahlon 2008-11-09 00:27:36 +00:00
parent f4051c5a35
commit 194fadda98
5 changed files with 183 additions and 138 deletions
Download patch file Download diff file Expand all files Collapse all files

42
chunker/lib/chunker.rb
View file

@ -1,9 +1,17 @@
#!/usr/bin/ruby
#
# Chunker!
# Chunker: A convenience library for parsing __END__ tokens consistently.
#
# Mahlon E. Smith <mahlon@martini.nu>
# == Version
#
# $Id$
#
# == Author
#
# * Mahlon E. Smith <mahlon@martini.nu>
#
# :include: LICENSE
#
### Namespace for the datablock parser.
###
@ -26,18 +34,18 @@ module Chunker
### Parser class for __END__ data blocks.
### Find each __MARKER__ within the __END__, and put each into a
### DATA_MARKER constant within the namespace that included us.
### Find each __TOKEN__ within the __END__, and put each into a
### DATA_TOKEN constant within the namespace that included us.
###
class DataParser
# The mark for a DATA block.
#
END_MARKER = /^__END__\r?\n/
END_TOKEN = /^__END__\r?\n/
# The mark for a 'sub' block.
#
CHUNK_MARKER = /^__([A-Z\_0-9]+)__\r?\n/
CHUNK_TOKEN = /^__([A-Z\_0-9]+)__\r?\n/
### Constructor: Given a +klass+ and an +io+ to the class file,
@ -45,13 +53,13 @@ module Chunker
###
def initialize( klass, io )
io.open if io.closed?
end_string = io.read.split( END_MARKER, 2 ).last
end_string = io.read.split( END_TOKEN, 2 ).last
@klass = klass
@scanner = StringScanner.new( end_string )
io.close
if @scanner.check_until( CHUNK_MARKER )
if @scanner.check_until( CHUNK_TOKEN )
# put each chunk into its own constant
self.extract_blocks
else
@ -71,10 +79,10 @@ module Chunker
def extract_blocks
label = nil
while @scanner.scan_until( CHUNK_MARKER ) and ! @scanner.eos?
while @scanner.scan_until( CHUNK_TOKEN ) and ! @scanner.eos?
data = ''
# First pass, __END__ contents (until next marker, instead
# First pass, __END__ contents (until next token, instead
# of entire data block.)
#
if label.nil?
@ -85,8 +93,8 @@ module Chunker
else
label = @scanner[1]
if data = @scanner.scan_until( CHUNK_MARKER )
# Pull the next marker text out of the data, set up the next pass
if data = @scanner.scan_until( CHUNK_TOKEN )
# Pull the next token text out of the data, set up the next pass
#
data = data[ 0, data.length - @scanner[0].length ]
@scanner.pos = self.next_position
@ -112,13 +120,15 @@ module Chunker
end
### Included hook: Find the file path for how we arrived here, and open
### it as an IO object. __FILE__ won't work, so we find it via caller().
### Start parsing this file for data blocks.
### Hook included: Find the file path for how we arrived here, and open
### it as an IO object. Parse the IO for data block tokens.
###
def self.included( klass )
# klass.instance_eval{ __FILE__ } awww, nope.
# __FILE__ won't work here, so we find the filename via caller().
#
io = File.open( caller(1).last.sub(/:.*?$/, ''), 'r' )
DataParser.new( klass, io )
end
end