mahlon/symphony-metronome
1
0
Fork 0
Code Issues Pull requests Activity

Commit the pile of initial work. Largely taken from company internal

Browse source 
repos, allowed to be newly released.  Portions already in production,
coverage still needs to be boosted.  Enjoy.

FossilOrigin-Name: 0f17fa483f55467bdf9e8f99dace58e6a90f5a8a7e595bdd79dfda5c92d16b7f
This commit is contained in:
Mahlon E. Smith 2014-04-22 00:21:43 +00:00
commit b18647f6a5
23 changed files with 2639 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

11
.hgignore Normal file
View file

@ -0,0 +1,11 @@
\.orig$
\.rej$
etc/.*\.(conf|yml)$
\.DS_Store
~$
pkg/
^ChangeLog$
^docs/
^coverage/
lib/symphony/metronome/intervalexpression.rb

14
.pryrc Normal file
View file

@ -0,0 +1,14 @@
#!/usr/bin/ruby -*- ruby -*-
require 'pathname'
begin
$LOAD_PATH.unshift( Pathname(__FILE__).dirname + 'lib' )
require 'symphony'
require 'symphony/metronome'
rescue => e
$stderr.puts "Ack! Libraries failed to load: #{e.message}\n\t" +
e.backtrace.join( "\n\t" )
end

1
.rspec Normal file
View file

@ -0,0 +1 @@
-fd -c

6
.rvm.gems Normal file
View file

@ -0,0 +1,6 @@
autotest -v4.4.6
sequel -v4.9.0
sqlite3 -v1.3.9
symphony -v0.6.0
rspec -v3.0.0.beta2
timecop -v0.7.1

32
.rvmrc Normal file
View file

@ -0,0 +1,32 @@
#!/usr/bin/env bash
# This is an RVM Project .rvmrc file, used to automatically load the ruby
# development environment upon cd'ing into the directory
environment_id="2.0.0@metronome"
if [[ -d "${rvm_path:-$HOME/.rvm}/environments" \
&& -s "${rvm_path:-$HOME/.rvm}/environments/$environment_id" ]]; then
echo "Using ${environment_id}"
. "${rvm_path:-$HOME/.rvm}/environments/$environment_id"
if [[ -s "${rvm_path:-$HOME/.rvm}/hooks/after_use" ]]; then
. "${rvm_path:-$HOME/.rvm}/hooks/after_use"
fi
else
# If the environment file has not yet been created, use the RVM CLI to select.
if ! rvm --create use "$environment_id"
then
echo "Failed to create RVM environment '${environment_id}'."
exit 1
fi
fi
filename=".rvm.gems"
if [[ -s "$filename" ]]; then
rvm gemset import "$filename"
fi

9
.simplecov Normal file
View file

@ -0,0 +1,9 @@
# Simplecov config
SimpleCov.start do
add_filter 'spec'
add_filter 'integration'
add_group "Needing tests" do |file|
file.covered_percent < 90
end
end

198
README.rdoc Normal file
View file

@ -0,0 +1,198 @@
= metronome
== Description
Metronome is an interval scheduler and task runner. It can be used
locally as a cron replacement, or as a network-wide job executor.
Events are stored via simple database rows, and optionally managed
via AMQP events. Interval/time values are expressed with intuitive
English phrases, ie.: 'at 2pm', or 'Starting in 20 minutes, run every 10
seconds and then finish in 2 days', or 'execute 12 times during the next
minute'.
It includes an executable under bin/:
metronome-exp::
A simple tester for trying out interval expression parsing.
== Synopsis
Here's an example of a cron clone:
require 'symphony/metronome'
Symphony.load_config
Symphony::Metronome.run do |opts, id|
Thread.new do
pid = fork { exec opts.delete('command') }
Process.waitpid( pid )
end
end
And here's a simplistic AMQP message broadcaster, using existing
Symphony connection information:
require 'symphony/metronome'
Symphony.load_config
Symphony::Metronome.run do |opts, id|
key = opts.delete( :routing_key ) or next
exchange = Symphony::Queue.amqp_exchange
exchange.publish( 'hi from Metronome!', routing_key: key )
end
== Adding Actions
There are two primary components to Metronome -- getting actions into
its database, and performing some task with those actions when the time
is appropriate.
By default, Metronome will start up an AMQP listener, attached to your
Symphony exchange, and wait for new scheduling messages. There are two
events it will take action on:
metronome.create::
Create a new scheduled event. The payload should be a hash. An
'expression' key is required, that provides the interval description.
Anything additional is serialized to 'options', that are passed to the
block when the interval fires. You can populate it with anything
your task requires to execute.
metronome.delete::
The payload is the row ID of the action. Metronome removes it from
the database.
If you'd prefer not to use the AMQP listener, you can put actions into
Metronome using any database methodology you please. When the daemon
starts up or receives a HUP signal, it will re-read and schedule out
upcoming work.
== Options
Metronome uses
Configurability[https://rubygems.org/gems/configurability] to determine
behavior. The configuration is a YAML[http://www.yaml.org/] file. It
shares AMQP configuration with Symphony, and adds metronome specific
controls in the 'metronome' key.
metronome:
splay: 0
listen: true
db: sqlite:///tmp/metronome.db
=== splay
Randomize all start times for actions by this many seconds on either
side of the original execution time. Defaults to none.
=== listen
Start up an AMQP listener using Symphony configuration, for remote
administration of schedule events. Defaults to true.
=== db
A {Sequel}[https://rubygems.org/gems/sequel] connection URI. Currently,
Metronome is tested under SQLite and PostgreSQL. Defaults to a SQLite
file at /tmp/metronome.db.
== Scheduling Examples
Note that Metronome is designed as an interval scheduler, not a
calendaring app. It doesn't have any concepts around phrases like "next
tuesday", or "the 3rd sunday after christmas". If that's what you're
after, check out the {chronic}[http://rubygems.org/gems/chronic]
library instead.
Here are a small set of example expressions. Feel free to use the
+metronome-exp+ utility to get a feel for what Metronome anticipates.
in 30.5 minutes
once an hour
every 15 minutes for 2 days
at 2014-05-01
at 2014-04-01 14:00:25
at 2pm
starting at 2pm once a day
start in 1 hour from now run every 5 seconds end at 11:15pm
every other hour
run every 7th minute for a day
once a day ending in 1 week
run once a minute for an hour starting in 6 days
10 times a minute for 2 days
run 45 times every hour
30 times per day
start at 2010-01-02 run 12 times and end on 2010-01-03
starting in an hour from now run 6 times a minute for 2 hours
beginning a day from now, run 30 times per minute and finish in 2 weeks
execute 12 times during the next 2 minutes
once a minute beginning in 5 minutes
In general, you can use reasonably intuitive phrasings. Capitalization,
whitespace, and punctuation doesn't matter. When describing numbers,
use digit/integer form instead of words, ie: '1 hour' instead of 'one
hour'.
== Installation
gem install symphony-metronome
== Contributing
You can check out the current development source with Mercurial via its
{project page}[http://bitbucket.org/mahlon/symphony-metronome].
After checking out the source, run:
$ rake
This task will run the tests/specs and generate the API documentation.
If you use {rvm}[http://rvm.io/], entering the project directory will
install any required development dependencies.
== License
Copyright (c) 2014, Mahlon E. Smith
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the author/s, nor the names of the project's
contributors may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

143
Rakefile Normal file
View file

@ -0,0 +1,143 @@
#!/usr/bin/env rake
# vim: set nosta noet ts=4 sw=4:
require 'rake/clean'
require 'pathname'
PROJECT = 'metronome'
BASEDIR = Pathname( __FILE__ ).dirname.relative_path_from( Pathname.pwd )
LIBDIR = BASEDIR + 'lib' + 'symphony'
CLOBBER.include( 'coverage' )
$LOAD_PATH.unshift( LIBDIR.to_s )
EXPRESSION_RL = LIBDIR + 'metronome' + 'intervalexpression.rl'
EXPRESSION_RB = LIBDIR + 'metronome' + 'intervalexpression.rb'
if Rake.application.options.trace
$trace = true
$stderr.puts '$trace is enabled'
end
# get the current library version
$version = ( LIBDIR + "#{PROJECT}.rb" ).read.split(/\n/).
select{|line| line =~ /VERSION =/}.first.match(/([\d|.]+)/)[1]
task :default => [ :spec, :docs, :package ]
# Generate the expression parser with Ragel
file EXPRESSION_RL
file EXPRESSION_RB
task EXPRESSION_RB => EXPRESSION_RL do |task|
sh 'ragel', '-R', '-T1', '-Ls', task.prerequisites.first
end
task :spec => EXPRESSION_RB
########################################################################
### P A C K A G I N G
########################################################################
require 'rubygems'
require 'rubygems/package_task'
spec = Gem::Specification.new do |s|
s.email = 'mahlon@martini.nu'
s.homepage = 'http://projects.martini.nu/ruby-modules'
s.authors = [ 'Mahlon E. Smith <mahlon@martini.nu>' ]
s.platform = Gem::Platform::RUBY
s.summary = "A natural language scheduling and task runner."
s.name = 'symphony-' + PROJECT
s.version = $version
s.license = 'BSD'
s.has_rdoc = true
s.require_path = 'lib'
s.bindir = 'bin'
s.files = File.read( __FILE__ ).split( /^__END__/, 2 ).last.split
s.executables = %w[ metronome-exp ]
s.description = <<-EOF
Metronome is a scheduler and task runner. It can be used locally as a
cron replacement, or as a network-wide job executor. Events are stored
via simple database rows, and optionally managed via AMQP events.
Interval/time values are expressed with reasonably intuitive English
phrases, ie.: 'at 2pm', or 'Starting in 20 minutes, run every 10 seconds
and then finish in 2 days'.
EOF
s.required_rubygems_version = '>= 2.0.3'
s.required_ruby_version = '>= 2.0.0'
s.add_dependency 'symphony', '~> 0.6'
s.add_dependency 'sequel', '~> 4'
s.add_dependency 'sqlite3', '~> 1.3'
s.add_development_dependency 'rspec', '~> 3.0'
s.add_development_dependency 'simplecov', '~> 0.8'
s.add_development_dependency 'timecop', '~> 0.7'
end
Gem::PackageTask.new( spec ) do |pkg|
pkg.need_zip = true
pkg.need_tar = true
end
########################################################################
### D O C U M E N T A T I O N
########################################################################
begin
require 'rdoc/task'
desc 'Generate rdoc documentation'
RDoc::Task.new do |rdoc|
rdoc.name = :docs
rdoc.rdoc_dir = 'docs'
rdoc.main = "README.rdoc"
rdoc.rdoc_files = [ 'lib', *FileList['*.rdoc'] ]
end
RDoc::Task.new do |rdoc|
rdoc.name = :doc_coverage
rdoc.options = [ '-C1' ]
end
rescue LoadError
$stderr.puts "Omitting 'docs' tasks, rdoc doesn't seem to be installed."
end
########################################################################
### T E S T I N G
########################################################################
begin
require 'rspec/core/rake_task'
task :test => :spec
desc "Run specs"
RSpec::Core::RakeTask.new do |t|
t.pattern = "spec/**/*_spec.rb"
end
desc "Build a coverage report"
task :coverage do
ENV[ 'COVERAGE' ] = "yep"
Rake::Task[ :spec ].invoke
end
rescue LoadError
$stderr.puts "Omitting testing tasks, rspec doesn't seem to be installed."
end
########################################################################
### M A N I F E S T
########################################################################
__END__
lib/symphony/metronome.rb
lib/symphony/tasks/scheduletask.rb
lib/symphony/metronome/scheduler.rb
lib/symphony/metronome/mixins.rb
lib/symphony/metronome/intervalexpression.rb
lib/symphony/metronome/scheduledevent.rb
data/symphony-metronome/migrations/20140419_initial.rb
README.rdoc

33
bin/metronome-exp Executable file
View file

@ -0,0 +1,33 @@
#!/usr/bin/env ruby
# vim: set nosta noet ts=4 sw=4:
#
# Simplistic interval expression tester.
#
require 'symphony/metronome'
loop do
begin
exp = gets.chomp
next if exp.empty?
begin
parsed = Symphony::Metronome::IntervalExpression.parse( exp )
puts "OK:"
puts "\tvalid | %s" % [ parsed.valid ]
puts "\trecurring | %s" % [ parsed.recurring ]
puts "\tstarting | %s" % [ parsed.starting ]
puts "\tinterval | %s" % [ parsed.recurring ? parsed.interval : '-' ]
puts "\tending | %s" %
[ parsed.ending ? parsed.ending : (parsed.recurring ? 'never' : '-') ]
rescue => err
puts "NOPE: (%s) %s" % [ exp, err.message ]
end
puts
rescue Interrupt
exit 0
end
end

32
data/symphony-metronome/migrations/20140419_initial.rb Normal file
View file

@ -0,0 +1,32 @@
# vim: set nosta noet ts=4 sw=4:
### The initial Metronome DDL.
###
class Initial < Sequel::Migration
def initialize( db )
@db = db
end
def up
create_table( :metronome ) do
case @db.adapter_scheme
when :postgres
serial :id, primary_key: true
timestamptz :created, null: false
text :expression, null: false
text :options
else
Integer :id, auto_increment: true, primary_key: true
DateTime :created, null: false
String :expression, null: false
String :options
end
end
end
def down
drop_table( :metronome )
end
end

79
lib/symphony/metronome.rb Normal file
View file

@ -0,0 +1,79 @@
#!/usr/bin/env ruby
# vim: set nosta noet ts=4 sw=4:
require 'pathname'
require 'symphony' unless defined?( Symphony )
module Symphony::Metronome
extend Loggability,
Configurability
# Library version constant
VERSION = '0.1.0'
# Version-control revision constant
REVISION = %q$Revision$
# The name of the environment variable to check for config file overrides
CONFIG_ENV = 'METRONOME_CONFIG'
# The path to the default config file
DEFAULT_CONFIG_FILE = 'etc/config.yml'
# The data directory that contains migration files.
#
DATADIR = if ENV['METRONOME_DATADIR']
Pathname.new( ENV['METRONOME_DATADIR'] )
elsif Gem.datadir( 'symphony-metronome' )
Pathname.new( Gem.datadir('symphony-metronome') )
else
Pathname.new( __FILE__ ).dirname.parent.parent + 'data/symphony-metronome'
end
# Loggability API -- use symphony's logger
log_to :symphony
# Configurability API
config_key :metronome
### Get the loaded config (a Configurability::Config object)
def self::config
Configurability.loaded_config
end
### Load the specified +config_file+, install the config in all objects with
### Configurability, and call any callbacks registered via #after_configure.
def self::load_config( config_file=nil, defaults=nil )
config_file ||= ENV[ CONFIG_ENV ] || DEFAULT_CONFIG_FILE
defaults ||= Configurability.gather_defaults
config = Configurability::Config.load( config_file, defaults )
config.install
end
# The generic parse exception class.
class TimeParseError < ArgumentError; end
require 'symphony/metronome/scheduler'
require 'symphony/metronome/intervalexpression'
require 'symphony/metronome/scheduledevent'
require 'symphony/tasks/scheduletask'
###############
module_function
###############
### Convenience method for running the scheduler.
###
def run( &block )
raise LocalJumpError, "No block provided." unless block_given?
return Symphony::Metronome::Scheduler.run( &block )
end
end # Symphony::Metronome

590
lib/symphony/metronome/intervalexpression.rl Normal file
View file

@ -0,0 +1,590 @@
# vim: set noet nosta sw=4 ts=4 ft=ragel :
%%{
#
# Generate the actual code like so:
# ragel -R -T1 -Ls inputfile.rl
#
machine interval_expression;
########################################################################
### A C T I O N S
########################################################################
action set_mark { mark = p }
action set_valid { event.instance_variable_set( :@valid, true ) }
action set_invalid { event.instance_variable_set( :@valid, false ) }
action recurring { event.instance_variable_set( :@recurring, true ) }
action start_time {
time = event.send( :extract, mark, p - mark )
event.send( :set_starting, time, :time )
}
action start_interval {
interval = event.send( :extract, mark, p - mark )
event.send( :set_starting, interval, :interval )
}
action execute_time {
time = event.send( :extract, mark, p - mark )
event.send( :set_interval, time, :time )
}
action execute_interval {
interval = event.send( :extract, mark, p - mark )
event.send( :set_interval, interval, :interval )
}
action execute_multiplier {
multiplier = event.send( :extract, mark, p - mark ).sub( / times/, '' )
event.instance_variable_set( :@multiplier, multiplier.to_i )
}
action ending_time {
time = event.send( :extract, mark, p - mark )
event.send( :set_ending, time, :time )
}
action ending_interval {
interval = event.send( :extract, mark, p - mark )
event.send( :set_ending, interval, :interval )
}
########################################################################
### P R E P O S I T I O N S
########################################################################
recur_preposition = ( 'every' | 'each' | 'per' | 'once' ' per'? ) @recurring;
time_preposition = 'at' | 'on';
interval_preposition = 'in';
########################################################################
### K E Y W O R D S
########################################################################
interval_times =
( 'milli'? 'second' | 'minute' | 'hour' | 'day' | 'week' | 'month' | 'year' ) 's'?;
start_identifiers = ( 'start' | 'begin' 'n'? ) 'ing'?;
exec_identifiers = ('run' | 'exec' 'ute'?);
ending_identifiers = ( ('for' | 'until' | 'during') | ('end'|'finish'|'stop'|'complet' 'e'?) 'ing'? );
########################################################################
### T I M E S P E C S
########################################################################
# 1st
# 202nd
# 2015th
# ...
#
ordinals = (
( (digit+ - '1')? '1' 'st' ) |
( digit+?
( '1' digit 'th' ) | # all '11s'
( '2' 'nd' ) |
( '3' 'rd' ) |
( [0456789] 'th' )
)
);
# 2014-05-01
# 2014-05-01 15:00
# 2014-05-01 15:00:30
#
fulldate = digit{4} '-' digit{2} '-' digit{2}
( space+ digit{2} ':' digit{2} ( ':' digit{2} )? )?;
# 10am
# 2:45pm
#
time = digit{1,2} ( ':' digit{2} )? ( 'am' | 'pm' );
# union of the above
date_or_time = fulldate | time;
# 20 seconds
# 5 hours
# 1 hour
# 2.5 hours
# an hour
# a minute
# other minute
#
interval = (
(( 'a' 'n'? | [1-9][0-9]* ( '.' [0-9]+ )? ) | 'other' | ordinals ) space+
)? interval_times;
########################################################################
### A C T I O N C H A I N S
########################################################################
start_time = date_or_time >set_mark %start_time;
start_interval = interval >set_mark %start_interval;
start_expression = ( (time_preposition space+)? start_time ) |
( (interval_preposition space+)? start_interval );
execute_time = date_or_time >set_mark %/execute_time;
execute_interval = interval >set_mark %execute_interval;
execute_multiplier = ( digit+ space+ 'times' )
>set_mark %execute_multiplier @recurring;
execute_expression = (
# regular dates and intervals
( time_preposition space+ execute_time ) |
( ( interval_preposition | recur_preposition ) space+ execute_interval )
) | (
# count + interval (10 times every minute)
execute_multiplier space+ ( recur_preposition space+ )? execute_interval
) |
# count for 'timeboxed' intervals
execute_multiplier;
ending_time = date_or_time >set_mark %ending_time;
ending_interval = interval >set_mark %ending_interval;
ending_expression = ( (time_preposition space+)? ending_time ) |
( (interval_preposition space+)? ending_interval );
########################################################################
### M A C H I N E S
########################################################################
Start = (
start: start_identifiers space+ -> StartTime,
StartTime: start_expression -> final
);
Interval = (
start:
Decorators: ( exec_identifiers space+ )? -> ExecuteTime,
ExecuteTime: execute_expression -> final
);
Ending = (
start: space+ ending_identifiers space+ -> EndingTime,
EndingTime: ending_expression -> final
);
main := (
( (Start space+)? Interval Ending? ) |
( Interval ( space+ Start )? Ending? ) |
( Interval Ending space+ Start )
) %set_valid @!set_invalid;
}%%
require 'symphony' unless defined?( Symphony )
require 'symphony/metronome'
require 'symphony/metronome/mixins'
using Symphony::Metronome::TimeRefinements
### Parse natural English expressions of times and intervals.
###
### in 30 minutes
### once an hour
### every 15 minutes for 2 days
### at 2014-05-01
### at 2014-04-01 14:00:25
### at 2pm
### starting at 2pm once a day
### start in 1 hour from now run every 5 seconds end at 11:15pm
### every other hour
### once a day ending in 1 week
### run once a minute for an hour starting in 6 days
### 10 times a minute for 2 days
### run 45 times every hour
### 30 times per day
### start at 2010-01-02 run 12 times and end on 2010-01-03
### starting in an hour from now run 6 times a minute for 2 hours
### beginning a day from now, run 30 times per minute and finish in 2 weeks
### execute 12 times during the next 2 minutes
###
class Symphony::Metronome::IntervalExpression
include Comparable,
Symphony::Metronome::TimeFunctions
extend Loggability
log_to :symphony
# Ragel accessors are injected as class methods/variables for some reason.
%% write data;
# Words/phrases in the expression that we'll strip/ignore before parsing.
COMMON_DECORATORS = [ 'and', 'then', /\s+from now/, 'the next' ];
########################################################################
### C L A S S M E T H O D S
########################################################################
### Parse a schedule expression +exp+.
###
### Parsing defaults to Time.now(), but if passed a +time+ object,
### all contexual times (2pm) are relative to it. If you know when
### an expression was generated, you can 'reconstitute' an interval
### object this way.
###
def self::parse( exp, time=nil )
# Normalize the expression before parsing
#
exp = exp.downcase.
gsub( /(?:[^[a-z][0-9][\.\-:]\s]+)/, '' ). # . : - a-z 0-9 only
gsub( Regexp.union(COMMON_DECORATORS), '' ). # remove common decorator words
gsub( /\s+/, ' ' ). # collapse whitespace
gsub( /([:\-])+/, '\1' ). # collapse multiple - or : chars
gsub( /\.+$/, '' ) # trailing periods
event = new( exp, time || Time.now )
data = event.instance_variable_get( :@data )
# Ragel interface variables
#
key = ''
mark = 0
%% write init;
eof = pe
%% write exec;
# Attach final time logic and sanity checks.
event.send( :finalize )
return event
end
########################################################################
### I N S T A N C E M E T H O D S
########################################################################
### Instantiate a new TimeExpression, provided an +expression+ string
### that describes when this event will take place in natural english,
### and a +base+ Time to perform calculations against.
###
private_class_method :new
def initialize( expression, base ) # :nodoc:
@exp = expression
@data = expression.to_s.unpack( 'c*' )
@base = base
@valid = false
@recurring = false
@starting = nil
@interval = nil
@multiplier = nil
@ending = nil
end
######
public
######
# Is the schedule expression parsable?
attr_reader :valid
# Does this event repeat?
attr_reader :recurring
# The valid start time for the schedule (for recurring events)
attr_reader :starting
# The valid end time for the schedule (for recurring events)
attr_reader :ending
# The interval to wait before the event should be acted on.
attr_reader :interval
# An optional interval multipler for expressing counts.
attr_reader :multiplier
### If this interval is on a stack somewhere and ready to
### fire, is it okay to do so based on the specified
### expression criteria?
###
### Returns +true+ if it should fire, +false+ if it should not
### but could at a later attempt, and +nil+ if the interval has
### expired.
###
def fire?
now = Time.now
# Interval has expired.
return nil if self.ending && now > self.ending
# Interval is not yet in its current time window.
return false if self.starting - now > 0
# Looking good.
return true
end
### Just return the original event expression.
###
def to_s
return @exp
end
### Inspection string.
###
def inspect
return ( "<%s:0x%08x valid:%s recur:%s expression:%p " +
"starting:%p interval:%p ending:%p>" ) % [
self.class.name,
self.object_id * 2,
self.valid,
self.recurring,
self.to_s,
self.starting,
self.interval,
self.ending
]
end
### Comparable interface, order by interval, 'soonest' first.
###
def <=>( other )
return self.interval <=> other.interval
end
#########
protected
#########
### Given a +start+ and +ending+ scanner position,
### return an ascii representation of the data slice.
###
def extract( start, ending )
slice = @data[ start, ending ]
return '' unless slice
return slice.pack( 'c*' )
end
### Parse and set the starting attribute, given a +time_arg+
### string and the +type+ of string (interval or exact time)
###
def set_starting( time_arg, type )
start = self.get_time( time_arg, type )
@starting = start
# If start time is expressed as a post-conditional (we've
# already got an end time) we need to recalculate the end
# as an offset from the start. The original parsed ending
# arguments should have already been cached when it was
# previously set.
#
if self.ending && self.recurring
self.set_ending( *@ending_args )
end
return @starting
end
### Parse and set the interval attribute, given a +time_arg+
### string and the +type+ of string (interval or exact time)
###
### Perform consistency and sanity checks before returning an
### integer representing the amount of time needed to sleep before
### firing the event.
###
def set_interval( time_arg, type )
interval = nil
if self.starting && type == :time
raise Symphony::Metronome::TimeParseError, "That doesn't make sense, just use 'at [datetime]' instead"
else
interval = self.get_time( time_arg, type )
interval = interval - @base
end
@interval = interval
return @interval
end
### Parse and set the ending attribute, given a +time_arg+
### string and the +type+ of string (interval or exact time)
###
### Perform consistency and sanity checks before returning a
### Time object.
###
def set_ending( time_arg, type )
ending = nil
# Ending dates only make sense for recurring events.
#
if self.recurring
@ending_args = [ time_arg, type ] # squirrel away for post-set starts
# Make the interval an offset of the start time, instead of now.
#
# This is the contextual difference between:
# every minute until 6 hours from now (ending based on NOW)
# and
# starting in a year run every minute for 1 month (ending based on start time)
#
if self.starting && type == :interval
diff = self.parse_interval( time_arg )
ending = self.starting + diff
# (offset from now)
#
else
ending = self.get_time( time_arg, type )
end
# Check the end time is after the start time.
#
if self.starting && ending < self.starting
raise Symphony::Metronome::TimeParseError, "recurring event ends before it begins"
end
else
self.log.debug "Ignoring ending date, event is not recurring."
end
@ending = ending
return @ending
end
### Perform finishing logic and final sanity checks before returning
### a parsed object.
###
def finalize
raise Symphony::Metronome::TimeParseError, "unable to parse expression" unless self.valid
# Ensure start time is populated.
#
unless self.starting
if self.recurring
@starting = @base
else
raise Symphony::Metronome::TimeParseError, "non-deterministic expression" if self.interval.nil?
@starting = @base + self.interval
end
end
# Alter the interval if a multiplier was specified.
#
if self.multiplier
if self.ending
# Regular 'count' style multipler with end date.
# (run 10 times a minute for 2 days)
# Just divide the current interval by the count.
#
if self.interval
@interval = self.interval.to_f / self.multiplier
# Timeboxed multiplier (start [date] run 10 times end [date])
# Evenly spread the interval out over the time window.
#
else
diff = self.ending - self.starting
@interval = diff.to_f / self.multiplier
end
# Regular 'count' style multipler (run 10 times a minute)
# Just divide the current interval by the count.
#
else
raise Symphony::Metronome::TimeParseError, "An end date or interval is required" unless self.interval
@interval = self.interval.to_f / self.multiplier
end
end
end
### Given a +time_arg+ string and a type (:interval or :time),
### dispatch to the appropriate parser.
###
def get_time( time_arg, type )
time = nil
if type == :interval
secs = self.parse_interval( time_arg )
time = @base + secs if secs
end
if type == :time
time = self.parse_time( time_arg )
end
raise Symphony::Metronome::TimeParseError, "unable to parse time" if time.nil?
return time
end
### Parse a +time_arg+ string (anything parsable buy Time.parse())
### into a Time object.
###
def parse_time( time_arg )
time = Time.parse( time_arg, @base ) rescue nil
# Generated date is in the past.
#
if time && @base > time
# Ensure future dates for ambiguous times (2pm)
time = time + 1.day if time_arg.length < 8
# Still in the past, abandon all hope.
raise Symphony::Metronome::TimeParseError, "attempt to schedule in the past" if @base > time
end
self.log.debug "Parsed %p (time) to: %p" % [ time_arg, time ]
return time
end
### Parse a +time_arg+ interval string ("30 seconds") into an
### Integer.
###
def parse_interval( interval_arg )
duration, span = interval_arg.split( /\s+/ )
# catch the 'a' or 'an' case (ex: "an hour")
duration = 1 if duration.index( 'a' ) == 0
# catch the 'other' case, ie: 'every other hour'
duration = 2 if duration == 'other'
# catch the singular case (ex: "hour")
unless span
span = duration
duration = 1
end
use_milliseconds = span.sub!( 'milli', '' )
interval = calculate_seconds( duration.to_f, span.to_sym )
# milliseconds
interval = duration.to_f / 1000 if use_milliseconds
self.log.debug "Parsed %p (interval) to: %p" % [ interval_arg, interval ]
return interval
end
end # class TimeExpression

130
lib/symphony/metronome/mixins.rb Normal file
View file

@ -0,0 +1,130 @@
# -*- ruby -*-
#encoding: utf-8
# vim: set nosta noet ts=4 sw=4:
require 'symphony' unless defined?( Symphony )
require 'symphony/metronome' unless defined?( Symphony::Metronome )
module Symphony::Metronome
# Functions for time calculations
module TimeFunctions
###############
module_function
###############
### Calculate the (approximate) number of seconds that are in +count+ of the
### given +unit+ of time.
###
def calculate_seconds( count, unit )
return case unit
when :seconds, :second
count
when :minutes, :minute
count * 60
when :hours, :hour
count * 3600
when :days, :day
count * 86400
when :weeks, :week
count * 604800
when :fortnights, :fortnight
count * 1209600
when :months, :month
count * 2592000
when :years, :year
count * 31557600
else
raise ArgumentError, "don't know how to calculate seconds in a %p" % [ unit ]
end
end
end # module TimeFunctions
# Refinements to Numeric to add time-related convenience methods
module TimeRefinements
refine Numeric do
### Number of seconds (returns receiver unmodified)
def seconds
return self
end
alias_method :second, :seconds
### Returns number of seconds in <receiver> minutes
def minutes
return TimeFunctions.calculate_seconds( self, :minutes )
end
alias_method :minute, :minutes
### Returns the number of seconds in <receiver> hours
def hours
return TimeFunctions.calculate_seconds( self, :hours )
end
alias_method :hour, :hours
### Returns the number of seconds in <receiver> days
def days
return TimeFunctions.calculate_seconds( self, :day )
end
alias_method :day, :days
### Return the number of seconds in <receiver> weeks
def weeks
return TimeFunctions.calculate_seconds( self, :weeks )
end
alias_method :week, :weeks
### Returns the number of seconds in <receiver> fortnights
def fortnights
return TimeFunctions.calculate_seconds( self, :fortnights )
end
alias_method :fortnight, :fortnights
### Returns the number of seconds in <receiver> months (approximate)
def months
return TimeFunctions.calculate_seconds( self, :months )
end
alias_method :month, :months
### Returns the number of seconds in <receiver> years (approximate)
def years
return TimeFunctions.calculate_seconds( self, :years )
end
alias_method :year, :years
### Returns the Time <receiver> number of seconds before the
### specified +time+. E.g., 2.hours.before( header.expiration )
def before( time )
return time - self
end
### Returns the Time <receiver> number of seconds ago. (e.g.,
### expiration > 2.hours.ago )
def ago
return self.before( ::Time.now )
end
### Returns the Time <receiver> number of seconds after the given +time+.
### E.g., 10.minutes.after( header.expiration )
def after( time )
return time + self
end
### Return a new Time <receiver> number of seconds from now.
def from_now
return self.after( ::Time.now )
end
end # refine Numeric
end # module TimeRefinements
end # module Symphony::Metronome

174
lib/symphony/metronome/scheduledevent.rb Normal file
View file

@ -0,0 +1,174 @@
#!/usr/bin/env ruby
# vim: set nosta noet ts=4 sw=4:
require 'set'
require 'sequel'
require 'sqlite3'
require 'yajl'
require 'symphony/metronome'
Sequel.extension :migration
### A class the represents the relationship between an interval and
### an event.
###
class Symphony::Metronome::ScheduledEvent
extend Loggability, Configurability
include Comparable
log_to :symphony
config_key :metronome
# Configure defaults.
#
CONFIG_DEFAULTS = {
db: 'sqlite:///tmp/metronome.db',
splay: 0
}
class << self
# A Sequel-style DB connection URI.
attr_reader :db
# Adjust recurring intervals by a random window.
attr_reader :splay
end
######################################################################
# C L A S S M E T H O D S
######################################################################
### Configurability API.
###
def self::configure( config=nil )
config = self.defaults.merge( config || {} )
@db = Sequel.connect( config.delete(:db) )
@splay = config.delete( :splay )
# Ensure the database is current.
#
migrations_dir = Symphony::Metronome::DATADIR + 'migrations'
unless Sequel::Migrator.is_current?( self.db, migrations_dir.to_s )
Sequel::Migrator.apply( self.db, migrations_dir.to_s )
end
end
### Return a set of all known events, sorted by date of execution.
### Delete any rows that are invalid expressions.
###
def self::load
now = Time.now
events = SortedSet.new
# Force reset the DB handle.
self.db.disconnect
self.log.debug "Parsing/loading all actions."
self.db[ :metronome ].each do |event|
begin
event = new( event )
events << event
rescue ArgumentError, Symphony::Metronome::TimeParseError => err
self.log.error "%p while parsing \"%s\": %s" % [
err.class,
event[:expression],
err.message
]
self.log.debug " " + err.backtrace.join( "\n " )
self.db[ :metronome ].filter( :id => event[:id] ).delete
end
end
return events
end
######################################################################
# I N S T A N C E M E T H O D S
######################################################################
### Create a new ScheduledEvent object.
###
def initialize( row )
@event = Symphony::Metronome::IntervalExpression.parse( row[:expression], row[:created] )
@options = row.delete( :options )
@id = row.delete( :id )
self.reset_runtime
unless self.class.splay.zero?
splay = Range.new( - self.class.splay, self.class.splay )
@runtime = self.runtime + rand( splay )
end
end
# The parsed interval expression.
attr_reader :event
# The unique ID number of the scheduled event.
attr_reader :id
# The options hash attached to this event.
attr_reader :options
# The exact time that this event will run.
attr_reader :runtime
### Set the datetime that this event should fire next.
###
def reset_runtime
now = Time.now
# Start time is in the future, so it's sufficent to be considered the run time.
#
if self.event.starting >= now
@runtime = self.event.starting
return
end
# Otherwise, the event should already be running (start time has already
# elapsed), so schedule it forward on it's next interval iteration.
#
@runtime = now + self.event.interval
end
### Perform the action attached to the event. Yields the
### deserialized options, the action ID to the supplied block if
### this event is okay to execute.
###
### Automatically remove the event if it has expired.
###
def fire
rv = self.event.fire?
if rv
opts = Yajl.load( self.options )
yield opts, self.id
end
self.delete if rv.nil?
return rv
end
### Permanently remove this event from the database.
###
def delete
self.log.debug "Removing action %p" % [ self.id ]
self.class.db[ :metronome ].filter( :id => self.id ).delete
end
### Comparable interface, order by next run time, soonest first.
###
def <=>( other )
return self.runtime <=> other.runtime
end
end # Symphony::Metronome::ScheduledEvent

156
lib/symphony/metronome/scheduler.rb Normal file
View file

@ -0,0 +1,156 @@
#!/usr/bin/env ruby
# vim: set nosta noet ts=4 sw=4:
require 'symphony'
require 'symphony/metronome'
### Manage the delta queue of events and associated actions.
###
class Symphony::Metronome::Scheduler
extend Loggability, Configurability
include Symphony::SignalHandling
log_to :symphony
config_key :metronome
# Signals the daemon responds to.
SIGNALS = [ :HUP, :INT, :TERM ]
CONFIG_DEFAULTS = {
:listen => true
}
class << self
# Should Metronome register and schedule events via AMQP?
# If +false+, you'll need a separate way to add event actions
# to the database, and manually HUP the daemon.
attr_reader :listen
end
### Configurability API
###
def self::configure( config=nil )
config = self.defaults.merge( config || {} )
@listen = config.delete( :listen )
end
### Create and start an instanced daemon.
###
def self::run( &block )
return new( block )
end
### Actions to perform when creating a new daemon.
###
private_class_method :new
def initialize( block ) #:nodoc:
# Start the queue subscriber for schedule changes.
#
if self.class.listen
Symphony::Metronome::ScheduledEvent.db.disconnect
@child = fork do
$0 = 'Metronome (listener)'
Symphony::Metronome::ScheduleTask.run
end
Process.setpgid( @child, 0 )
end
# Signal handling for the master (this) process.
#
self.set_up_signal_handling
self.set_signal_traps( *SIGNALS )
@queue = Symphony::Metronome::ScheduledEvent.load
@proc = block
# Enter the main loop.
self.start
rescue => err
self.log.error "%p while running: %s" % [ err.class, err.message ]
self.log.debug " " + err.backtrace.join( "\n " )
Process.kill( 'TERM', @child ) if self.class.listen
end
# The sorted set of ScheduledEvent objects.
attr_reader :queue
#########
protected
#########
### Main daemon sleep loop.
###
def start
$0 = "Metronome%s" % [ self.class.listen ? ' (executor)' : '' ]
@running = true
loop do
wait = nil
if ev = self.queue.first
wait = ev.runtime - Time.now
wait = 0 if wait < 0
self.log.info "Next event in %0.3f second(s) (id: %d)..." % [ wait, ev.id ]
else
self.log.warn "No events scheduled. Waiting indefinitely..."
end
self.process_events unless self.wait_for_signals( wait )
break unless @running
end
end
### Dispatch incoming signals to appropriate handlers.
###
def handle_signal( sig )
case sig
when :TERM, :INT
@running = false
Process.kill( sig.to_s, @child ) if self.class.listen
when :HUP
@queue = Symphony::Metronome::ScheduledEvent.load
self.queue.each{|ev| ev.fire(&@proc) if ev.event.recurring }
else
self.log.debug "Unhandled signal: %s" % [ sig ]
end
end
### Process all event that have reached their runtime.
###
def process_events
now = Time.now
self.queue.each do |ev|
next unless now >= ev.runtime
self.queue.delete( ev )
rv = ev.fire( &@proc )
# Reschedule the event and place it back on the queue.
#
if ev.event.recurring
ev.reset_runtime
self.queue.add( ev ) unless rv.nil?
# It was a single run event, torch it!
#
else
ev.delete
end
end
end
end # Symphony::Metronome::Scheduler

81
lib/symphony/tasks/scheduletask.rb Normal file
View file

@ -0,0 +1,81 @@
#!/usr/bin/env ruby
# vim: set nosta noet ts=4 sw=4:
require 'symphony'
require 'symphony/routing'
require 'symphony/metronome'
### Accept metronome scheduling events, translating them
### to DB rows for persistence.
###
class Symphony::Metronome::ScheduleTask < Symphony::Task
include Symphony::Routing
queue_name 'metronome'
timeout 30
### Get a handle to the database.
###
def initialize( * )
@db = Symphony::Metronome::ScheduledEvent.db
@actions = @db[ :metronome ]
super
end
# The Sequel dataset of scheduled event actions.
attr_reader :actions
### Accept a new scheduled event. The payload should be a free
### form hash of options, along with an expression string that
### conforms to IntervalExpression.
###
### {
### :expression => 'run 25 times for an hour',
### :payload => { ... },
### }
###
on 'metronome.create' do |payload, metadata|
raise ArgumentError, 'Invalid payload.' unless payload.is_a?( Hash )
exp = payload.delete( 'expression' )
raise ArgumentError, 'Missing time expression.' unless exp
self.actions.insert(
:created => Time.now,
:expression => exp,
:options => Yajl.dump( payload )
)
self.signal_parent
return true
end
### Delete an existing scheduled event.
### The payload is the id of the action (row) to delete.
###
on 'metronome.delete' do |id, metadata|
self.actions.filter( :id => id.to_i ).delete
self.signal_parent
return true
end
### Tell our parent (the Metronome broadcaster) to re-read its event
### list.
###
def signal_parent
parent = Process.ppid
# Check to make sure we weren't orphaned.
#
if parent == 1
self.log.error "Lost my parent process? Exiting."
exit 1
end
Process.kill( 'HUP', parent )
end
end

45
spec/helpers.rb Normal file
View file

@ -0,0 +1,45 @@
#!/usr/bin/ruby
# coding: utf-8
# vim: set nosta noet ts=4 sw=4:
require 'pathname'
BASEDIR = Pathname( __FILE__ ).dirname.parent
LIBDIR = BASEDIR + 'lib'
$LOAD_PATH.unshift( LIBDIR.to_s )
# SimpleCov test coverage reporting; enable this using the :coverage rake task
require 'simplecov' if ENV['COVERAGE']
require 'timecop'
require 'loggability'
require 'loggability/spechelpers'
require 'configurability'
require 'configurability/behavior'
require 'rspec'
require 'symphony'
require 'symphony/metronome'
Loggability.format_with( :color ) if $stdout.tty?
### RSpec helper functions.
module Loggability::SpecHelpers
end
### Mock with RSpec
RSpec.configure do |config|
config.run_all_when_everything_filtered = true
config.filter_run :focus
# config.order = 'random'
config.expect_with( :rspec )
config.mock_with( :rspec ) do |mock|
mock.syntax = :expect
end
config.include( Loggability::SpecHelpers )
end

477
spec/symphony/metronome/intervalexpression_spec.rb Normal file
View file

@ -0,0 +1,477 @@
# vim: set nosta noet ts=4 sw=4 ft=rspec:
require_relative '../../helpers'
describe Symphony::Metronome::IntervalExpression do
# 2010-01-01 12:00
let( :past ) { Time.at(1262376000) }
before( :each ) do
Timecop.freeze( past )
end
it "can't be instantiated directly" do
expect { described_class.new }.to raise_error( NoMethodError )
end
it "raises an exception if unable to parse the expression" do
expect {
described_class.parse( 'wut!' )
}.to raise_error( Symphony::Metronome::TimeParseError, /unable to parse/ )
end
it "normalizes the expression before attempting to parse it" do
parsed = described_class.parse( '\'";At 2014---01-01 14::00(' )
expect( parsed.to_s ).to eq( 'at 2014-01-01 14:00' )
end
it "can parse the expression, offset from a different time" do
parsed = described_class.parse( 'every 5 seconds ending in an hour' )
expect( parsed.starting ).to eq( past )
expect( parsed.ending ).to eq( past + 3600 )
end
it "is comparable" do
p1 = described_class.parse( 'at 2pm' )
p2 = described_class.parse( 'at 3pm' )
p3 = described_class.parse( 'at 2:00pm' )
expect( p1 ).to be < p2
expect( p2 ).to be > p1
expect( p1 ).to eq( p3 )
end
it "won't allow scheduling dates in the past" do
expect {
described_class.parse( 'on 1999-01-01' )
}.to raise_error( Symphony::Metronome::TimeParseError, /schedule in the past/ )
end
it "doesn't allow intervals of 0" do
expect {
described_class.parse( 'every 0 seconds' )
}.to raise_error( Symphony::Metronome::TimeParseError, /unable to parse/ )
end
context 'exact times and dates' do
it 'at 2pm' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be
expect( parsed.recurring ).to be_falsey
expect( parsed.interval ).to be( 7200.0 )
end
it 'at 2:30pm' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 9000.0 )
end
it "pushes ambiguous times in today's past into tomorrow (at 11am)" do
parsed = described_class.parse( 'at 11am' )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 82800.0 )
end
it 'on 2010-01-02' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 43200.0 )
end
it 'on 2010-01-02 12:00' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 86400.0 )
end
it 'on 2010-01-02 12:00:01' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 86401.0 )
end
it 'correctly timeboxes the expression' do
parsed = described_class.parse( 'at 2pm' )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 7200.0 )
expect( parsed.ending ).to be_nil
expect( parsed.recurring ).to be_falsey
expect( parsed.starting ).to eq( past + 7200 )
end
it 'sets the start time to the exact date' do
parsed = described_class.parse( 'at 2pm' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_falsey
expect( parsed.starting ).to eq( past + 7200 )
expect( parsed.interval ).to be( 7200.0 )
end
end
context 'one-shot intervals' do
it 'in 30 seconds' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_falsey
expect( parsed.interval ).to be( 30.0 )
end
it 'in 30 seconds from now' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 30.0 )
end
it 'in an hour from now' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 3600.0 )
end
it 'in 2.5 hours from now' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 9000.0 )
end
it 'in a minute' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 60.0 )
end
it 'correctly timeboxes the expression' do
parsed = described_class.parse( 'in 30 seconds' )
expect( parsed.valid ).to be_truthy
expect( parsed.interval ).to be( 30.0 )
expect( parsed.ending ).to be_nil
expect( parsed.recurring ).to be_falsey
expect( parsed.starting ).to eq( past + 30 )
end
it 'sets the start time to now if one is not specified' do
parsed = described_class.parse( 'in 5 seconds' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_falsey
expect( parsed.starting ).to eq( past + 5 )
expect( parsed.interval ).to be( 5.0 )
end
it 'raises error for end specifications with non-recurring run times' do
expect {
described_class.parse( 'run at 2010-01-02 end at 2010-03-01' )
}.to raise_error( Symphony::Metronome::TimeParseError, /non-deterministic/ )
end
end
context 'repeating intervals' do
it 'every 500 milliseconds' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 0.5 )
end
it 'every 30 seconds' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 30.0 )
end
it 'once an hour' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 3600.0 )
end
it 'once a minute' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 60.0 )
end
it 'once per week' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 604800.0 )
end
it 'every day' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 86400.0 )
end
it 'every other day' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 172800.0 )
end
it 'every 4th hour' do |example|
parsed = described_class.parse( example.description )
parsed2 = described_class.parse( 'every 4 hours' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 14400.0 )
expect( parsed ).to eq( parsed2 )
end
it 'always sets a start time if one is not specified' do
parsed = described_class.parse( 'every 5 seconds' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past )
expect( parsed.interval ).to be( 5.0 )
end
end
context 'repeating intervals with an expiration date' do
it 'every day ending in 1 week' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 86400.0 )
expect( parsed.ending ).to eq( past + 604800 )
end
it 'once a minute until 6pm' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 60.0 )
expect( parsed.ending ).to eq( past + 3600 * 6 )
end
it 'once a day finishing in a week from now' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 86400.0 )
expect( parsed.ending ).to eq( past + 604800 )
end
it 'once a day completing on 2010-02-01' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 86400.0 )
expect( parsed.ending ).to eq( past + 2635200 )
end
it 'once a day end on 2010-02-01 00:00:10' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.interval ).to be( 86400.0 )
expect( parsed.ending ).to eq( past + 2635210 )
end
it 'always sets a start time if one is not specified' do
parsed = described_class.parse( 'every 5 seconds ending in 1 week' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past )
expect( parsed.interval ).to be( 5.0 )
expect( parsed.ending ).to eq( past + 604800 )
end
end
context 'repeating intervals with only a start time' do
it "won't allow explicit start times with non-recurring run times" do
expect {
described_class.parse( 'start at 2010-02-01 run at 2010-02-01' )
}.to raise_error( Symphony::Metronome::TimeParseError, /use 'at \[datetime\]' instead/ )
end
it 'starting in 5 minutes, run once a second' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 300 )
expect( parsed.interval ).to be( 1.0 )
end
it 'starting in a day execute every 3 minutes' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 86400 )
expect( parsed.interval ).to be( 180.0 )
end
it 'start at 2010-01-02 execute every 1 minute' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 43200 )
expect( parsed.interval ).to be( 60.0 )
end
it 'always sets a start time if one is not specified' do
parsed = described_class.parse( 'every 5 seconds' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past )
expect( parsed.interval ).to be( 5.0 )
end
end
context 'intervals with start and end times' do
it 'beginning in 1 hour from now run every 5 seconds ending on 2010-01-02' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 3600 )
expect( parsed.interval ).to be( 5.0 )
expect( parsed.ending ).to eq( past + 43200 )
end
it 'starting in 1 hour, run every 5 seconds and finish at 3pm' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 3600 )
expect( parsed.interval ).to be( 5.0 )
expect( parsed.ending ).to eq( past + 3600 * 3 )
end
it 'begin in an hour run every 5 seconds and then stop at 3pm' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 3600 )
expect( parsed.interval ).to be( 5.0 )
expect( parsed.ending ).to eq( past + 3600 * 3 )
end
it 'mid-expression starts' do |example|
parsed = described_class.parse( 'every 5 seconds starting in an hour for 3 hours' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 3600 )
expect( parsed.interval ).to be( 5.0 )
expect( parsed.ending ).to eq( past + 3600 * 4 )
end
it 'post-expression starts' do |example|
parsed = described_class.parse( 'every 5 seconds for 3 hours beginning in an hour' )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 3600 )
expect( parsed.interval ).to be( 5.0 )
expect( parsed.ending ).to eq( past + 3600 * 4 )
end
it 'start at 2010-01-02 10:00 and then run each minute for the next 6 days' do |example|
parsed = described_class.parse( example.description )
expect( parsed.valid ).to be_truthy
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 43200 + 36000 )
expect( parsed.interval ).to be( 60.0 )
expect( parsed.ending ).to eq( Time.parse('2010-01-02 10:00') + 86400 * 6 )
end
it 'raises an error if the end time is before the start' do
expect {
described_class.parse( 'starting at 2pm run once a minute end at 1pm' )
}.to raise_error( Symphony::Metronome::TimeParseError, /ends before it begins/ )
end
end
context 'intervals with a count' do
it "won't allow count multipliers without an interval nor an end date" do
expect {
described_class.parse( 'run 10 times' )
}.to raise_error( Symphony::Metronome::TimeParseError, /end date or interval is required/ )
end
it '10 times a minute for 2 days' do |example|
parsed = described_class.parse( example.description )
expect( parsed.multiplier ).to be( 10 )
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past )
expect( parsed.interval ).to be( 6.0 )
expect( parsed.ending ).to eq( past + 86400 * 2 )
end
it 'run 45 times every hour' do |example|
parsed = described_class.parse( example.description )
expect( parsed.multiplier ).to be( 45 )
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past )
expect( parsed.interval ).to be( 80.0 )
expect( parsed.ending ).to be_nil
end
it 'start at 2010-01-02 run 12 times and end on 2010-01-03' do |example|
parsed = described_class.parse( example.description )
expect( parsed.multiplier ).to be( 12 )
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 43200 )
expect( parsed.interval ).to be( 7200.0 )
expect( parsed.ending ).to eq( past + 86400 + 43200 )
end
it 'starting in an hour from now run 6 times a minute for 2 hours' do |example|
parsed = described_class.parse( example.description )
expect( parsed.multiplier ).to be( 6 )
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 3600 )
expect( parsed.interval ).to be( 10.0 )
expect( parsed.ending ).to eq( past + 3600 * 3 )
end
it 'beginning a day from now, run 30 times per minute and finish in 2 weeks' do |example|
parsed = described_class.parse( example.description )
expect( parsed.multiplier ).to be( 30 )
expect( parsed.recurring ).to be_truthy
expect( parsed.starting ).to eq( past + 86400 )
expect( parsed.interval ).to be( 2.0 )
expect( parsed.ending ).to eq( past + 1209600 + 86400 )
end
end
context "when checking if it's okay to run" do
it 'returns true if the interval is within bounds' do
parsed = described_class.parse( 'at 2pm' )
expect( parsed.fire? ).to be_falsey
Timecop.freeze( past + 7200 ) do
expect( parsed.fire? ).to be_truthy
end
end
it 'returns nil if the ending (expiration) date has passed' do
parsed = described_class.parse( 'every minute for an hour' )
Timecop.freeze( past + 3601 ) do
expect( parsed.fire? ).to be_nil
end
end
it 'returns false if the starting window has yet to occur' do
parsed = described_class.parse( 'starting in 2 hours run each minute' )
expect( parsed.fire? ).to be_falsey
end
end
end

59
spec/symphony/metronome/mixins_spec.rb Normal file
View file

@ -0,0 +1,59 @@
#!/usr/bin/env rspec -wfd
# vim: set nosta noet ts=4 sw=4:
require_relative '../../helpers'
using Symphony::Metronome::TimeRefinements
describe Symphony::Metronome, 'mixins' do
describe "numeric constant methods" do
SECONDS_IN_A_MINUTE = 60
SECONDS_IN_AN_HOUR = SECONDS_IN_A_MINUTE * 60
SECONDS_IN_A_DAY = SECONDS_IN_AN_HOUR * 24
SECONDS_IN_A_WEEK = SECONDS_IN_A_DAY * 7
SECONDS_IN_A_FORTNIGHT = SECONDS_IN_A_WEEK * 2
SECONDS_IN_A_MONTH = SECONDS_IN_A_DAY * 30
SECONDS_IN_A_YEAR = Integer( SECONDS_IN_A_DAY * 365.25 )
it "can calculate the number of seconds for various units of time" do
expect( 1.second ).to eq( 1 )
expect( 14.seconds ).to eq( 14 )
expect( 1.minute ).to eq( SECONDS_IN_A_MINUTE )
expect( 18.minutes ).to eq( SECONDS_IN_A_MINUTE * 18 )
expect( 1.hour ).to eq( SECONDS_IN_AN_HOUR )
expect( 723.hours ).to eq( SECONDS_IN_AN_HOUR * 723 )
expect( 1.day ).to eq( SECONDS_IN_A_DAY )
expect( 3.days ).to eq( SECONDS_IN_A_DAY * 3 )
expect( 1.week ).to eq( SECONDS_IN_A_WEEK )
expect( 28.weeks ).to eq( SECONDS_IN_A_WEEK * 28 )
expect( 1.fortnight ).to eq( SECONDS_IN_A_FORTNIGHT )
expect( 31.fortnights ).to eq( SECONDS_IN_A_FORTNIGHT * 31 )
expect( 1.month ).to eq( SECONDS_IN_A_MONTH )
expect( 67.months ).to eq( SECONDS_IN_A_MONTH * 67 )
expect( 1.year ).to eq( SECONDS_IN_A_YEAR )
expect( 13.years ).to eq( SECONDS_IN_A_YEAR * 13 )
end
it "can calculate various time offsets" do
starttime = Time.now
expect( 1.second.after( starttime ) ).to eq( starttime + 1 )
expect( 18.seconds.from_now ).to be_within( 10.seconds ).of( starttime + 18 )
expect( 1.second.before( starttime ) ).to eq( starttime - 1 )
expect( 3.hours.ago ).to be_within( 10.seconds ).of( starttime - 10800 )
end
end
end

154
spec/symphony/metronome/scheduledevent_spec.rb Normal file
View file

@ -0,0 +1,154 @@
#!/usr/bin/env rspec -wfd
# vim: set nosta noet ts=4 sw=4:
require_relative '../../helpers'
describe Symphony::Metronome::ScheduledEvent do
let( :ds ) { described_class.db[:metronome] }
before( :all ) do
described_class.configure( :db => 'sqlite:///tmp/metronome-testing.db' )
end
after( :all ) do
Pathname( '/tmp/metronome-testing.db' ).unlink
end
after( :each ) do
described_class.db[ :metronome ].delete
end
context 'class methods' do
after( :all ) do
Timecop.return
end
# 2010-01-01 12:00
let( :time ) { Time.at(1262376000) }
before( :each ) do
Timecop.travel( time )
end
it 'applies migrations upon initial config' do
migrations = described_class.db[ :schema_migrations ].all
expect( migrations.first[:filename] ).to eq( '20140419_initial.rb' )
end
it 'can load all stored events sorted by next execution time' do
ds.insert(
:created => Time.now,
:expression => 'at 2pm'
)
ds.insert(
:created => Time.now,
:expression => 'at 3pm'
)
ds.insert(
:created => Time.now,
:expression => 'at 1pm'
)
events = described_class.load.to_a
expect( events.length ).to be( 3 )
expect( events.first.event.instance_variable_get(:@exp) ).to eq( 'at 1pm' )
expect( events.last.event.instance_variable_get(:@exp) ).to eq( 'at 3pm' )
end
it 'removes invalid expressions from storage when loading' do
ds.insert(
:created => Time.now,
:expression => 'blippity'
)
ds.insert(
:created => Time.now,
:expression => 'at 3pm'
)
events = described_class.load.to_a
expect( events.length ).to be( 1 )
end
end
context 'an instance' do
let( :time ) { Time.at(1262376000) }
it 'can reschedule itself into the future when recurring (future start)' do
ev = described_class.new(
:created => time,
:expression => 'every 30 seconds'
)
Timecop.travel( time - 3600 ) do
ev.reset_runtime
end
expect( ev.runtime ).to eq( time )
end
it 'can reschedule itself into the future when recurring (past start)' do
ev = described_class.new(
:created => time,
:expression => 'every 30 seconds'
)
Timecop.travel( time + 3600 ) do
ev.reset_runtime
end
expect( ev.runtime ).to be >= time + 3600 + 30
end
it 'removes itself when firing if expired' do
ds.insert(
:created => time,
:expression => 'every 30 seconds for an hour',
:options => ""
)
ev = described_class.new( ds.first )
expect( ev.fire {} ).to be_nil
expect( ds.count ).to eq( 0 )
end
it 'yields a deserialized options hash if okay to fire' do
ev = described_class.new(
:created => time,
:expression => 'every 30 seconds',
:options => '{"excitement_level":12}'
)
res = 0
ev.fire do |opts, id|
res = opts['excitement_level']
end
expect( res ).to be( 12 )
end
it 'randomizes start times if a splay is configured' do
described_class.instance_variable_set( :@splay, 5 )
Timecop.travel( time ) do
100.times do
ev = described_class.new(
:created => time,
:expression => 'every 30 seconds'
)
diff = (( time + 30 ) - ev.runtime ).round
expect( diff ).to be_within( 5 ).of( 0 )
end
end
end
end
end

19
spec/symphony/metronome/scheduler_spec.rb Normal file
View file

@ -0,0 +1,19 @@
#!/usr/bin/env rspec -wfd
# vim: set nosta noet ts=4 sw=4:
require_relative '../../helpers'
describe Symphony::Metronome::Scheduler do
before( :all ) do
described_class.configure
end
it 'spins up an AMQP listener by default' do
# described_class.run {}
# expect( described_class.listen ).to eq( :sd )
end
end

66
spec/symphony/metronome_spec.rb Normal file
View file

@ -0,0 +1,66 @@
#!/usr/bin/env rspec
require_relative '../helpers'
describe Symphony::Metronome do
before( :each ) do
ENV.delete( 'METRONOME_CONFIG' )
end
it "will load a default config file if none is specified" do
config_object = double( "Configurability::Config object" )
expect( Configurability ).to receive( :gather_defaults ).
and_return( {} )
expect( Configurability::Config ).to receive( :load ).
with( described_class::DEFAULT_CONFIG_FILE, {} ).
and_return( config_object )
expect( config_object ).to receive( :install )
described_class.load_config
end
it "will load a config file given in an environment variable if none is specified" do
ENV['METRONOME_CONFIG'] = '/usr/local/etc/config.yml'
config_object = double( "Configurability::Config object" )
expect( Configurability ).to receive( :gather_defaults ).
and_return( {} )
expect( Configurability::Config ).to receive( :load ).
with( '/usr/local/etc/config.yml', {} ).
and_return( config_object )
expect( config_object ).to receive( :install )
described_class.load_config
end
it "will load a config file and install it if one is given" do
config_object = double( "Configurability::Config object" )
expect( Configurability ).to receive( :gather_defaults ).
and_return( {} )
expect( Configurability::Config ).to receive( :load ).
with( 'a/configfile.yml', {} ).
and_return( config_object )
expect( config_object ).to receive( :install )
described_class.load_config( 'a/configfile.yml' )
end
it "will override default values when loading the config if they're given" do
config_object = double( "Configurability::Config object" )
expect( Configurability ).to_not receive( :gather_defaults )
expect( Configurability::Config ).to receive( :load ).
with( 'a/different/configfile.yml', {database: {dbname: 'test'}} ).
and_return( config_object )
expect( config_object ).to receive( :install )
described_class.load_config( 'a/different/configfile.yml', database: {dbname: 'test'} )
end
end

130
spec/symphony/tasks/scheduletask_spec.rb Normal file
View file

@ -0,0 +1,130 @@
#!/usr/bin/env rspec -wfd
# vim: set nosta noet ts=4 sw=4:
require_relative '../../helpers'
describe Symphony::Metronome::ScheduleTask do
let( :time ) { Time.at(1262376000) }
let( :db ) { double('sequel db handle') }
let( :actions ) { double('sequel dataset') }
before( :each ) do
allow( Symphony::Metronome::ScheduledEvent ).to receive( :db ).and_return( db )
allow( db ).to receive( :[] ).with( :metronome ).and_return( actions )
Timecop.freeze( time )
end
context 'creating a new event' do
let( :task ) { described_class.new(nil) }
let( :metadata ) {
{
:delivery_info => double( 'delivery info' ),
:properties => double( 'properties' )
}
}
it 'fails with non-hash payloads' do
expect( metadata[:delivery_info] ).to receive( :routing_key ).
and_return( 'metronome.create' )
expect {
task.work( [], metadata )
}.to raise_error( ArgumentError, 'Invalid payload.' )
end
it 'fails without an expression argument' do
expect( metadata[:delivery_info] ).to receive( :routing_key ).
and_return( 'metronome.create' )
expect {
task.work( {}, metadata )
}.to raise_error( ArgumentError, 'Missing time expression.' )
end
it 'saves the event and seralized options to storage' do
allow( Process ).to receive( :ppid ).and_return( 12000 )
expect( metadata[:delivery_info] ).to receive( :routing_key ).
and_return( 'metronome.create' )
payload = {
'expression' => 'at 2pm',
'excitement_level' => 12
}
expect( actions ).to receive( :insert ).with({
:created => time,
:expression => 'at 2pm',
:options => '{"excitement_level":12}'
})
expect( Process ).to receive( :kill ).with( 'HUP', 12000 )
expect( task.work(payload, metadata) ).to be_truthy
end
it 'exits if it has become an orphaned process' do
expect( metadata[:delivery_info] ).to receive( :routing_key ).
and_return( 'metronome.create' )
payload = {
'expression' => 'at 2pm',
'excitement_level' => 12
}
expect( actions ).to receive( :insert ).with({
:created => time,
:expression => 'at 2pm',
:options => '{"excitement_level":12}'
})
expect( Process ).to_not receive( :kill )
# parent gone! init takes over.
allow( Process ).to receive( :ppid ).and_return( 1 )
expect { task.work( payload, metadata ) }.to raise_error( SystemExit )
end
end
context 'removing an existing event' do
let( :task ) { described_class.new(nil) }
let( :metadata ) {
{
:delivery_info => double( 'delivery info' ),
:properties => double( 'properties' )
}
}
it 'removes rows matching the payload ID' do
allow( Process ).to receive( :ppid ).and_return( 12000 )
expect( metadata[:delivery_info] ).to receive( :routing_key ).
and_return( 'metronome.delete' )
payload = "44"
row = double( 'filtered dataaset' )
expect( row ).to receive( :delete )
expect( actions ).to receive( :filter ).with( :id => 44 ).and_return( row )
expect( Process ).to receive( :kill ).with( 'HUP', 12000 )
task.work( payload, metadata )
end
it 'exits if it has become an orphaned process' do
expect( metadata[:delivery_info] ).to receive( :routing_key ).
and_return( 'metronome.delete' )
payload = "44"
row = double( 'filtered dataaset' )
expect( row ).to receive( :delete )
expect( actions ).to receive( :filter ).with( :id => 44 ).and_return( row )
expect( Process ).to_not receive( :kill )
# parent gone! init takes over.
allow( Process ).to receive( :ppid ).and_return( 1 )
expect { task.work( payload, metadata ) }.to raise_error( SystemExit )
end
end
end