Posted by Jim Morris
on Fri Feb 06 15:04:54 -0800 2009
There is a raging debate in many forums about how to do fixture-like
things. Basically how do you populate a database with test data so you
can run your Specs/Tests/Features.
There are several libraries out there to do this like FactoryGirl,
FixtureReplacement, Machinist, Fixjour etc etc. If you use Rails and
ActiveRecord pick the one you like and be happy ;)
However I am focusing on how to do this for Integration tests that
test the entire stack, using Cucumber/Webrat in Merb and using the Sequel
ORM. I also use these techniques to test Java based web services
directly over HTTP.
I can't use the above mentioned libraries as they rely on
ActiveRecord, and I don't use that ORM in my Merb Apps (and obviously
not in my Java apps).
I also don't have a preference for fixtures over factories, in
fact I like bits of both paradigms.
My problem with fixtures is that they are really hard to maintain, and
can be slow to load the entire database test set if run for every
Scenario. My problem with factories is they use the Models defined in
ActiveRecord to setup the test data set, which can be very slow as
well. What I like about Factories is keeping the data setup close to
the actual tests, and ease of use and maintenance. What I like about
Fixtures is directly loading the database, and bypassing the models
and associations.
I have solved the problem many times, and in different ways for each
project I do. For my Java projects I have written a typical fixture
loader in Java reading from a standard YAML file, so this is no
different from Rails fixtures, just written in Java. I also wrote a
Sequel based ruby script which loaded the database with test data, and
needed to be manually run every time my integration tests were run.
This was acceptable as it was a final test against a remote staging
server before deployment to production. I needed to be able to load
the staging database over an SSH pipe and test over an SSH pipe.
For my Merb/Sequel based
wolfmanblog
project I use a hybrid solution which so far seems to work well for
me, as it keeps the test data really close to the testing code, it is
relatively fast as it loads the database directly rather than through
the models, and is maintainable as it is close to the testing code.
Using Cucumber for
the integration tests, and loading the test data in Givens with steps
specifically setup to load data, and writing a Class that talks
directly to the database via Sequel
seems to work very nicely for me.
Basically I write a little helper in a DBHelper Class, that loads
specific types of data for a given test or set of tests, these helpers
are called from the steps called by the Givens. i use a pretty high
level abstract in the givens rather than defining low level data, so
I say Given a valid user or Given there are 20 Articles, I let the
step specify the actual data written, because the step also has the
Then clauses so I can keep the data in one place so if the user name
is testuser1, I can test for testuser1 in the same place further down
the file in the Then clause for testing the results. A good example
is in wolfmanblog/features/posts/...
# index.feature
Scenario: GET /
Given 8 posts exist
When I go to /
Then the request should succeed
And I should see post 1
And I should see post 2
And I should see post 3
And I should see post 4
And I should not see post 5
Given /^(\d+) posts exist$/ do |n|
@dbhelper.truncate(:posts)
for i in (1..n.to_i) do
@dbhelper.add_post(:id => i, :title => "post #{i}", :body => "body of post #{i}")
end
end
Then /^I should see post (\d+)$/ do |n|
@response.should have_xpath("//h2/a[@href='/posts/#{n}']['post #{n}']")
@response.should have_selector("p:contains('body of post #{n}')")
end
Then /^I should not see post (\d+)$/ do |n|
@response.should_not have_xpath("//h2/a[@href='/posts/#{n}']")
end
The dbhelper is defined in ./features/support/db_helper.rb and
@dbhelper is set in ./features/support/env.rb.
Notice I truncate the database before loading the new data, and I set
the id of the record so I can test later on. I do this rather than use
transactions as I like to be able to look in the database later for
debugging purposes. Also note I use Postgresql which supports
TRUNCATE .. CASCADE which makes it easier to clean up when you have
foreign key constraints all over the place as I do.
The DBHelper class...
class DBHelper
attr_reader :db
def initialize(target=nil, debug=nil)
if target.nil?
@@target= ENV['testtarget'].nil? ? "test" : ENV['testtarget']
else
@@target= target
end
dburl= case @@target
when 'test'
"postgres://morris:test@localhost:5432/sample1_test"
else
raise 'Bad target'
end
@db= Sequel.open dburl
dblog= Logger.new($stdout)
dblog.level= debug ? Logger::INFO : Logger::WARN
@db.logger= dblog
end
def close
@db.disconnect
end
def truncate(table)
@db.execute("TRUNCATE #{table.to_s} CASCADE")
end
def add_user(name, password, salt)
@db[:users] << {:name => name, :crypted_password => password, :salt => salt}
end
def add_post(h)
@db[:posts].insert(h.merge(:created_at => Time.now.iso8601, :updated_at => Time.now.iso8601))
end
end
Before do
@dbhelper= DBHelper.new('test', false)
end
After do
@dbhelper.close
end
I open a new connection to the database in this case, but I could use
the one used by Merb using @db= Sequel::DATABASES.first.
I try to keep the helpers in this file as generic as possible so they
can be used in different Givens, but sometimes you have to be very
specific. Try to encapsulate all database knowledge in this file.
To summarize...
- I put very high level setup commands in the features Given.
- I put the detailed contents in the associated step.
- I actually load the database in the helper.
- I keep the Then tests close to the Given that sets up the data
Posted in
Rails,Merb,Sequel
|
Tags
merb,sequel,fixtures
|
1 comments
Show
Posted by Jim Morris
on Wed Jan 02 13:32:50 -0800 2008
I've been working on a project that is mostly Java for the last many
months, so haven't had much Ruby or Rails stuff to share.
However one thing I found when working on my tests in Java was an
xpath matcher for JUnit 4.0 using the
Hamcrest libraries.
When I dropped back into Ruby to write some cron scripts that process
information from the database and generate xml files I wanted to check
the script with an rspec, and check the xml files it was generating.
To do this I wanted to use a similar matcher to the Hamcrest ones, but
use it in RSpec.
I Googled around and found a simple
example,
but it wasn't very sophisticated and didn't check what I needed so I
upgraded it to do the kind of matches I needed, the results are here.
require 'rexml/document'
require 'rexml/element'
module Spec
module Matchers
class HaveXpath
def initialize(xpath)
@xpath = xpath
end
def matches?(response)
@response = response
doc = response.is_a?(REXML::Document) ? response : REXML::Document.new(@response)
match = REXML::XPath.match(doc, @xpath)
not match.empty?
end
def failure_message
"Did not find expected xpath #{@xpath}"
end
def negative_failure_message
"Did find unexpected xpath #{@xpath}"
end
def description
"match the xpath expression #{@xpath}"
end
end
def have_xpath(xpath)
HaveXpath.new(xpath)
end
class MatchXpath
def initialize(xpath, val)
@xpath = xpath
@val= val
end
def matches?(response)
@response = response
doc = response.is_a?(REXML::Document) ? response : REXML::Document.new(@response)
ok= true
REXML::XPath.each(doc, @xpath) do |e|
@actual_val= case e
when REXML::Attribute
e.to_s
when REXML::Element
e.text
else
e.to_s
end
return false unless @val == @actual_val
end
return ok
end
def failure_message
"The xpath #{@xpath} did not have the value '#{@val}'
It was '#{@actual_val}'"
end
def description
"match the xpath expression #{@xpath} with #{@val}"
end
end
def match_xpath(xpath, val)
MatchXpath.new(xpath, val)
end
class HaveNodes
def initialize(xpath, num)
@xpath= xpath
@num = num
end
def matches?(response)
@response = response
doc = response.is_a?(REXML::Document) ? response : REXML::Document.new(@response)
match = REXML::XPath.match(doc, @xpath)
@num_found= match.size
@num_found == @num
end
def failure_message
"Did not find expected number of nodes #{@num} in xpath #{@xpath}
Found #{@num_found}"
end
def description
"match the number of nodes #{@num}"
end
end
def have_nodes(xpath, num)
HaveNodes.new(xpath, num)
end
end
end
The first matcher HaveXPath was pretty much the original I found on
the net, it simply checks that an XPath exists, I don't use this one.
The next matcher MatchXPath is more like what I was using in Java,
it gets an element from the xpath and checks the value is equal to the
one expected string. I will eventually add regex matching and arrays
of strings or regexs to check against.
The last one HaveNodes I find handy to make sure a given xpath matches the
expected number of nodes.
I even wrote an rspec to check the matchers, and this also is handy to
show the way to use them.
Note that you can pass the matchers a String containing the XML or (much faster)
a REXML::Document.
require 'matchers'
describe "test matchers" do
before(:each) do
@xml= <<-EOFXML
<?xml version='1.0'?>
<claims>
<testnode1>
<day>
<rank order='1' value='0' userid='26' alias='user25'/>
<rank order='2' value='0' userid='93' alias='user92'/>
<rank order='3' value='0' userid='55' alias='user54'/>
<sometext>this is text</sometext>
</day>
</testnode1>
</claims>
EOFXML
@doc= REXML::Document.new(@xml)
end
it "should test xpath" do
@xml.should have_nodes("/claims/*", 1)
@doc.should have_nodes("/claims/*", 1)
@doc.should have_nodes("/claims/testnode1/day/rank", 3)
@doc.should have_xpath("/claims/testnode1/day/rank[@order='1']")
@doc.should_not have_xpath("/claims/testnode1/day/rank[@order='10']")
@doc.should match_xpath("/claims/testnode1/day/rank[1]/@order", "1")
@doc.should match_xpath("/claims/testnode1/day/rank[2]/@value", "0")
@doc.should match_xpath("/claims/testnode1/day/rank[3]/@alias", "user54")
@doc.should match_xpath("/claims/testnode1/day/sometext", "this is text")
end
end
Posted in
Rails,RSpec
|
Tags
rspec,xpath,matcher
|
5 comments
Show
Posted by Jim Morris
on Tue Aug 07 23:53:09 -0700 2007
In my latest web project I potentially have a lot of boolean
preferences, which I use for enabling or disabling various email
notifications to users.
Rather than having to add a migration everytime I want to add a new
preference, I thought I would use the composed_of feature in my model
and compose the boolean preferences from a bitvector. That way I can
simply modify my model to add new preferences rather than add new
columns to the database.
I also wanted this to be easy to add new boolean preferences, so I use
some Macros (I guess you could also call it Meta-Programming) to do
all the repetitive code.
The result is a little class in my Person model called Preferences,
one integer field in my persons database called preferences, and a
composed_of :preferences in the Person model, and of course the
following class in the person.rb model.
class Person < ActiveRecord::Base
class Preferences
@@bits= {
:comment_notifications => 1,
:friendship_notifications => 2,
:event_notifications => 4,
:misc_notifications => 8 }
@@bits.each_key do |a|
attr_reader a
end
def initialize(prefs)
if prefs.nil?
@@bits.each do |a, v|
instance_variable_set("@#{a}", false)
end
@comment_notifications= true
@event_notifications= true
elsif prefs.is_a?(Hash)
@@bits.each do |a, v|
instance_variable_set("@#{a}", false)
end
prefs.each do |k,v|
raise(ArgumentError, "Unknown preference #{k}") unless @@bits.has_key?(k.to_sym)
instance_variable_set("@#{k}", true) if v == '1'
end
else
@@bits.each do |a, v|
instance_variable_set("@#{a}", (prefs & v) != 0 ? true : false)
end
end
end
def preferences
bv= 0
@@bits.each do |a, v|
bv |= instance_variable_get("@#{a}") ? v : 0
end
return bv
end
@@bits.each_key do |a|
alias_method((a.to_s + '?').to_sym, a)
end
end
composed_of :preferences
end
All I need to do to add new preferences is add it to the @@bits
class variable, which is a Hash of the preference name as a symbol and
the bit it sets in the integer (actually the value of the bit, bit0 is
1, bit1 is 2 etc). The rest of the code is derived from the class
variable.
To make things easier I also add a predicate for each preference, so I
can access @person.preferences.comment_notifications? to see if any
comment notifications are required for instance.
One other thing I do in the initialize method is set up defaults for
the preferences. This is only really needed if it is being added as an
after thought, and the column preferences is NULL in the database.
Because composed_of classes are immutable you must always create a
whole new one to update them, so I also allow initialize to be called
with a Hash, which can come straight from the controller. The last
case of initialize is being passed the integer from the database,
expanding it into the various boolean instance variables. The
preferences method does the reverse and converts the boolean
instance variables into the bit vector. Calls to these are all taken
care of by ActiveRecord.
An example of it being called from the controller is...
@person.preferences= Person::Preferences.new(params[:preferences])
presuming you have a bunch of check boxes in your view which are
passed in as part of the preferences hash.
Because I added this later I had one migration to initially add the
new column...
add_column :people, :preferences, :integer
Posted in
Rails
|
Tags
rails,preferences,composed_of,bitvector
|
2 comments
Show
Posted by Jim Morris
on Mon Jul 30 15:04:50 -0700 2007
A question I see asked a lot is how do I paginate acts_as_taggable
(on steroids)?
I haven't seen any answers I liked, so I created my own, which I'm
sure a few people won't like either ;) But it works for me (tm).
I use will_paginate, but this does
not work with custom finds that plugins define themselves, as is the
case with
acts_as_taggable.
If you do use a custom find_by_sql you have to hit the database twice, once to find the total number
of items and then the paginated find.
I have a situation where I generate a
tag cloud
with every page, and part of that tag cloud has already calculated the
number of tags for each classification I use.
I combined this with the paginator gem
to get myself pages without too many hits to the database.
The first thing I do is pass the total count I get from the tag cloud
to the action that renders the index for all those items matching the
cloud... If you look at the article cited above I make this
modification, this is dumbed down a bit for the sake of simplicity...
tags= Post.tag_counts(:order => 'tags.name')
tags.each do |t|
link_to(h(t.name), tagged_post_path(:tag => t.name, :cnt => t.count))
end
This passes the count I have already calculated to the action that
will list the paginated results.
In my controller I do this to get the paginated results using
will_paginate and the Pagination gem...
def tagged
...
size= params[:cnt]
if size
per_page= 10
pager = ::Paginator.new(size, per_page) do |offset, per_page|
Post.find_tagged_with(params[:tag], :limit => per_page, :offset => offset)
end
page= params[:page] || 1
@posts= returning WillPaginate::Collection.new(page, per_page, size) do |p|
p.replace pager.page(page).items
end
else
@posts= Post.find_tagged_with(params[:tag])
end
render :action => 'index'
end
This fits in nicely with the tag cloud I need to calculate, and it
uses will_paginate just like the regular index action does.
UPDATE
I refactored this to be more generally useful, I added the following as a
protected method in application.rb...
def tag_paginator(klass, tag, count=nil, per_page=10, page=1, order='updated_at DESC')
count ||= klass.count_tags(tag)
pager = ::Paginator.new(count, per_page) do |offset, per_page|
klass.find_tagged_with(tag, :order => order, :limit => per_page, :offset => offset)
end
page ||= 1
returning WillPaginate::Collection.new(page, per_page, count) do |p|
p.replace pager.page(page).items
end
end
I call it from one of my other actions like this...
@faqs= tag_paginator(Post, 'FAQ', nil, per_page, params[:page], 'updated_at DESC')
Passing in nil as the third parameter causes the tag_paginator
method to call Post.count_tags which is not part of the
acts_as_taggable methods, I added it to the SingletonMethods module
myself...
module ActiveRecord
module Acts
module Taggable
module SingletonMethods
...
def count_tags(tag)
count_by_sql("select count(*) FROM tags, taggings WHERE " + sanitize_sql(['name = ? AND tags.id = taggings.tag_id AND taggable_type = ?', tag, name]))
end
....
If you don't want to hack acts_as_taggable then simply leave that
part out and call the count_by_sql yourself.
My refactored tagged action from above now looks like this...
def tagged
...
tag= params[:tag]
per_page= 10
size= params[:cnt]
@posts= tag_paginator(Post, tag, size, 10, params[:page])
UPDATE
Johns suggestion works as of today (2/12/2009) so none of the above is needed...
options = Car.find_options_for_tagged_with(params[:tag_name]).merge :page => params[:page] @cars = Car.paginate(options)
or my example from above...
opts= Faq.find_options_for_tagged_with('FAQ')
@faqs= Faq.paginator(opts.merge(:page => params[:page]))
Posted in
Rails
|
Tags
acts_as_taggable,will_paginate
|
20 comments
Show
