Wolfmans Howlings

A programmers Blog about Ruby, Rails and a few other issues

Paginating acts_as_taggable with will_paginate

Posted by Jim Morris Mon, 30 Jul 2007 22:04:50 GMT

A question I see asked a lot is how do I paginate acts_as_taggable (on steroids)?

I haven't seen any answers I liked, so I created my own, which I'm sure a few people won't like either ;) But it works for me (tm).

I use will_paginate, but this does not work with custom finds that plugins define themselves, as is the case with acts_as_taggable.

If you do use a custom find_by_sql you have to hit the database twice, once to find the total number of items and then the paginated find.

I have a situation where I generate a tag cloud with every page, and part of that tag cloud has already calculated the number of tags for each classification I use.

I combined this with the paginator gem to get myself pages without too many hits to the database.

The first thing I do is pass the total count I get from the tag cloud to the action that renders the index for all those items matching the cloud... If you look at the article cited above I make this modification, this is dumbed down a bit for the sake of simplicity...

  tags= Post.tag_counts(:order => 'tags.name')
  tags.each do |t|
    link_to(h(t.name), tagged_post_path(:tag => t.name, :cnt => t.count))
  end

This passes the count I have already calculated to the action that will list the paginated results.

In my controller I do this to get the paginated results using will_paginate and the Pagination gem...

    def tagged
    ...
      # page if we can
      size= params[:cnt]
      if size
        per_page= 10

        # use Paginator gem to do the actual paging
        pager = ::Paginator.new(size, per_page) do |offset, per_page|
          Post.find_tagged_with(params[:tag], :limit => per_page, :offset => offset)
        end

        # default to page 1 if not specified
        page= params[:page] || 1

        # gets a paged array of posts
        @posts= returning WillPaginate::Collection.new(page, per_page, size) do |p|
          p.replace pager.page(page).items
        end
      else
        # fall back if we don't know the size
        @posts= Post.find_tagged_with(params[:tag])
      end

      render :action => 'index'
    end    

This fits in nicely with the tag cloud I need to calculate, and it uses will_paginate just like the regular index action does.

UPDATE

I refactored this to be more generally useful, I added the following as a protected method in application.rb...

  # paginate a call to find_tagged_with
  # klass is the tagged class
  # tag is the tag to find
  # count is the total number of items with that tag, if nil count_tags is called
  # per_page is numbe rof items per page
  # page is the page we are on
  # order is the order to return the items in
  def tag_paginator(klass, tag, count=nil, per_page=10, page=1, order='updated_at DESC')
    count ||= klass.count_tags(tag)
    pager = ::Paginator.new(count, per_page) do |offset, per_page|
      klass.find_tagged_with(tag, :order => order, :limit => per_page, :offset => offset)
    end

    page ||= 1

    returning WillPaginate::Collection.new(page, per_page, count) do |p|
      p.replace pager.page(page).items
    end
  end

I call it from one of my other actions like this...

@faqs= tag_paginator(Post, 'FAQ', nil, per_page, params[:page], 'updated_at DESC')

Passing in nil as the third parameter causes the tag_paginator method to call Post.count_tags which is not part of the acts_as_taggable methods, I added it to the SingletonMethods module myself...

module ActiveRecord
  module Acts #:nodoc:
    module Taggable #:nodoc:
      module SingletonMethods

      ...

        # Return the count of tag tags in this class
       def count_tags(tag)
         count_by_sql("select count(*) FROM tags, taggings WHERE " + sanitize_sql(['name = ? AND tags.id = taggings.tag_id AND taggable_type = ?', tag, name]))
       end

      ....

If you don't want to hack acts_as_taggable then simply leave that part out and call the count_by_sql yourself.

My refactored tagged action from above now looks like this...

def tagged
  ...
  tag= params[:tag]
  per_page= 10
  size= params[:cnt]
  @posts= tag_paginator(Post, tag, size, 10, params[:page])

Posted in  | Tags ,  | 10 comments | no trackbacks

RSpec testing all actions of a controller

Posted by Jim Morris Sat, 28 Jul 2007 21:23:07 GMT

A pattern I find very helpful is to find all the actions in a controller and apply a test to all those actions.

For instance this is useful for automatically testing all actions are protected from unauthorized access when using a login system.

One nice feature of this pattern is that if you add an action to a controller it will automatically be tested. This is less helpful if you use

before_filter :login_required, :except => {...}

as it will automatically be protected, but there are other use cases where this is not the situation. Just as in the except clause above you need to explicitly add any action that does not need to be tested to an exception list, which is supported by this pattern.

Here are the methods I use to test for login accessibility.

module MySpecHelper

  # get all actions for specified controller
  def get_all_actions(cont)
    c= Module.const_get(cont.to_s.pluralize.capitalize + "Controller")
    c.public_instance_methods(false).reject{ |action| ['rescue_action'].include?(action) }
  end

  # test actions fail if not logged in
  # opts[:exclude] contains an array of actions to skip
  # opts[:include] contains an array of actions to add to the test in addition
  # to any found by get_all_actions
  def controller_actions_should_fail_if_not_logged_in(cont, opts={})
    except= opts[:except] || []
    actions_to_test= get_all_actions(cont).reject{ |a| except.include?(a) }
    actions_to_test += opts[:include] if opts[:include]
    actions_to_test.each do |a|
      #puts "... #{a}"
      get a
      response.should_not be_success
      response.should redirect_to('http://test.host/login')
      flash[:warning].should == @login_warning
   end
 end
end

I put this in my spec_helper.rb and include it as shown here:

describe "When Logged out" do
  include MySpecHelper
  controller_name :events

  before(:each) do
    controller.stub!(:current_user).and_return(:false)
    @login_warning= "You need to be logged in to do that"
  end

  # test all actions require login except the ones specified
  # add new_comment as it is not seen by the automatic collector
  it "actions should fail" do
    controller_actions_should_fail_if_not_logged_in(:input, 
                              :except => ['index', 'show', 'tagged'], 
                              :include => ['new_comment'])
  end
end

The get_all_actions method collects all the public un-inherited methods in the given controller, these will consist of all the accessible actions in that controller. I explicitly exclude rescue_action as it is created by RSpec itself and should not be tested. Note it will not see any actions that are in application.rb so you need to add those to the list manually of you want them tested. (See the :include option in the example).

The controller_actions_should_fail_if_not_logged_in could be put in the spec itself rather than the spec_helper, but as I call this from all my controller specs it is more DRY to put it here. This method takes the controller name and an option array of actions names to ignore. This method tests all the actions and makes sure I get the expected result of the filter failing due to not being logged in.

I show an example spec that uses this to test my events controller, it mocks the login calls to say I am not logged in, and then tests them with the exceptions of the actions in this controller that do not require one to be logged in.

This pattern can be extended to test all sorts of things, and is especially useful for testing things where you can add an action and forget to do something in a filter to protect it. Make sure the default is on the side of caution though. IE you need to explicitly except actions rather than include actions.

Another example is something I recently stumbled upon in my RESTful controllers. In many cases it is good to use a verify statement to make sure that the RESTful actions actually can only be called with PUT, POST or DELETE and fail if called with GET. I use this statement in my controllers to enforce this...

# GETs should be safe (see http://www.w3.org/2001/tag/doc/whenToUseGet.html)
verify :method => :put, :only => [ :update ], :add_flash => { :error => "Operation Failed" }, :redirect_to => { :action => :index }
verify :method => :post, :only => [ :create, :new_comment ], :add_flash => { :error => "Operation Failed" }, :redirect_to => { :action => :index }
verify :method => :delete, :only => [ :destroy ], :add_flash => { :error => "Operation Failed" }, :redirect_to => { :action => :index }

I test this in my specs using this in the MySpecHelper Module

  def controller_actions_should_fail_with_get(cont, except=[])
    actions_to_test= get_all_actions(cont).reject{ |a| except.include?(a) }
    actions_to_test.each do |a|
      #puts "... #{a}"
      get a
      response.should redirect_to("http://test.host/#{cont.to_s.pluralize}")
      flash[:error].should == 'Operation Failed'
    end
  end

and an example of its use in a spec...

  it "actions should fail if not post or put" do
    controller_actions_should_fail_with_get(:event, ['index', 'show', 'edit', 'new'])
  end                                                                                     

Now whenever I add an action, the default is that it will fail with a GET, unless I add it to the exclude list in the spec, this will remind me to check if the action required PUT, POST or DELETE instead and to add it to the verify if so or add it to the specs exclude list if not.

These automatic tests keep me honest, especially in the last case where you really don't want a GET to be able to delete something.

I hope this pattern is useful to you.

Posted in ,  | Tags , ,  | 12 comments | no trackbacks

Using RSpec to test HAML helpers

Posted by Jim Morris Sat, 14 Jul 2007 23:40:00 GMT

UPDATED for HAML 2.0 and RSpec 1.1.5 - Changed open to haml_tag, prefix helper. to all rspec calls...

The most recent release of HAML introduced a neat feature that allows you to use HAML-like syntax in your helpers to generate HTML HAML#haml_tag.

A question on the HAML news group asked how to test a helper that uses HAML#haml_tag (used to be open/puts) and thanks to Nathan on that list I finally got RSpec to do it. As shown below.

However a really good point was made that really in RSpec the way to test anything is to use mocks to mock any call to an outside method thus focusing the test on the specific module under test. Generally I agree with that philosophy. But this is way cool so I thought I'd do it anyway, and also as it is a new feature in HAML one may not want to simply trust HAML to generate the correct HTML.

So in my application_helper.rb I have a simple helper...

module ApplicationHelper

 ...

  def display_flash
    for name in [:notice, :warning, :error]
      if flash[name]
        haml_tag :div, flash[name], {:class => name.to_s}
      end
    end
    nil
  end

  ...

end

This is called in my views as...

- display_flash

Notice the - instead of =, this is because the open (and puts) write output directly to the HAML buffer, and so this routine should return nothing. (This is also a very simply case and does not show off the utility of the open/puts methods, I'll show one of those later on).

The RSpec helper test that tests this is as follows...

# File: spec/helpers/application_helper_spec.rb
require File.dirname(__FILE__) + '/../spec_helper'

describe ApplicationHelper do

  before :each do
    helper.extend Haml
    helper.extend Haml::Helpers 
    helper.send :init_haml_helpers
  end

  it "should display flash" do
    for name in [:notice, :warning, :error]
      flash[name]= "flash #{name.to_s} message"
      helper.capture_haml{
        helper.display_flash
      }.should =~ /<div class='#{name.to_s}'>\s*#{flash[name]}\s*<\/div>/
      flash[name]= nil
    end
  end

end

Excellent, a simple test for the HTML generated by my haml helper. NOTE the setup required in the before :each, this sets up the haml helpers in the helpers context

Why would I want to use HAML#open you ask?

Well it makes the helpers look so much tidier IMHO, take this example from my previous post on tag clouds, the re-factored helper now looks like this...

  # display a tag cloud for the given model
  def tag_cloud(model, title= nil)
    m= model.to_s.camelcase.constantize
    plural= model.to_s.capitalize.pluralize
    title ||= plural
    tags= m.tag_counts(:order => 'tags.name')
    return false if tags.empty?
    urlmeth= "tagged_#{model.to_s.pluralize}_path".to_sym
    haml_tag :div, {:class => "tagcloud"} do
      haml_tag :h3, title
      tags.each do |t|
        next if t.name == 'FAQ'
        haml_tag :span, {:style => "font-size:#{calc_size(t.count)}%"} do
          puts link_to(h(t.name), self.send(urlmeth, :tag => t.name))
        end
      end
    end
    return true
  end      

So much cleaner, plus I can return a boolean to indicate if there was anything output or not, which tells me if I need to output an <hr/> or not.

Posted in , ,  | Tags , , ,  | 8 comments | no trackbacks

RSpec testing views for escaped HTML

Posted by Jim Morris Sat, 07 Jul 2007 01:22:00 GMT

For my social networking site snowdogsr.us I decided to escape all user input that gets displayed. I know people like to trick out their profiles with HTML but I want to avoid the various hacks that it allows.

So thinking I had done a good job of using h everywhere I output user input fields, I decided to see if I could actually test this with RSpec view tests.

I recently switched to RSpec for my testing needs, its cool :)

One thing it does is isolate the various things for testing using built in mocking, and views can be entirely tested standalone without accessing a model or a controller.

So how would it do testing for escaped user input I wondered?

Very well actually.

An example is worth a thousand words, so here is my RSpec for my home page.

BTW I found about 4 places where embedded HTML in user input was bleeding through, so it was well worth the effort.

So this goes in spec/views/home/home_spec.rb...

  it "should escape all user input" do
    @place= mock_model(Place, :name => 'place name<b>', :location => 'place location<b>', :tag_list => "place taglist <b>", :rated? => false)
    @event= mock_model(Event, :name => 'event name<b>', :where => 'event where<b>', :tag_list => "event taglist <b>", :date_time => DateTime.now, :hosted_by => 'Event host<b>')
    @post= mock_model(Input, :input => 'input body <b>', :tag_list => "post taglist <b>", :updated_at => DateTime.now, :created_at => DateTime.now, :created_by => 'post created by person<b>', :rated? => false)
    @picture= mock_model(Picture, :public_filename => "filename<b>.png")
    @pictures= [@picture]
    @pet= mock_model(Pet, :name => 'pet name<b>', :owned_by => "pet owner <b>", :breed => 'breed <b>', :description => "pet description <b>", :neutered => true, :gender => 'M<b>', :pictures => @pictures, :owned_by? => false)

    @posts= [@post]
    @events= [@event]
    @places= [@place]
    @top_places= [@place]
    @new_pets= [@pet]

    @comment= mock_model(Comment)
    @comment.stub!(:user).and_return(@user)
    @comment.stub!(:created_at).and_return(DateTime.now)
    @comment.stub!(:comment).and_return('comment body <b>')
    @comments= [@comment]
    @post.should_receive(:comments).and_return(@comments)

    @new_stuff= []
    @new_stuff << {:list => @posts, :title => 'Posts', :link => '#'}
    @new_stuff << {:list => @events, :title => 'Events', :link => '#'}
    @new_stuff << {:list => @places, :title => 'Places', :link => '#'}

    @top= []
    @top << {:list => @top_places, :title => 'Hot Places', :link => '#'}

    assigns[:new_stuff] = @new_stuff
    assigns[:top] = @top
    assigns[:new_pets] = @new_pets

    render "/home/logged_in"   
    #puts excerpt(response.body, "<b>")
    response.should_not have_text(/<b>/)
  end

Its quite complex as the home page renders a lot of summaries of the various lists I have.

First I mock the models that are called, and stub out the calls that are made to them. I force them all to return an embedded <b> which I don't use anyway, and with the new CSS oriented web styles shouldn't be used in HTML anyway.

Then I just test that <b> does not appear anywhere. If I have correctly used h to escape all the inputs then it should be rendered as &lt;b&gt; instead.

The response.should_not have_text(/<b>/) should do that test.

One cool thing is the mocking will tell you if any new inputs (ie calls to model attributes) have been added, or if you have forgotten any. So this should keep you honest in the future if you add new attributes that need escaping.

The

assigns[:new_stuff] = @new_stuff
assigns[:top] = @top
assigns[:new_pets] = @new_pets

Sets the assigns to the variables that my view uses,simulating what the controller would pass in.

The mock_model calls at the top also use a shortcut to define all the attributes that get called, and what they return. You can also explicitly do this...

@post.should_receive(:comments).and_return(@comments)

If you read the RSpec docs you can see that you can also test for parameters passed in, how many times it is called and various other nice things.

I added this snippet taken from the rails helpers to aid in finding any errant HTML that bleads through. (I'm not sure how to call it from the RSpec so I just copied the code into a private method).


 private

  def excerpt(text, phrase, radius = 100, excerpt_string = "...")
    if text.nil? || phrase.nil? then return end
    phrase = Regexp.escape(phrase)

    if found_pos = text.chars =~ /(#{phrase})/i
      start_pos = [ found_pos - radius, 0 ].max
      end_pos   = [ found_pos + phrase.chars.length + radius, text.chars.length ].min

      prefix  = start_pos > 0 ? excerpt_string : ""
      postfix = end_pos < text.chars.length ? excerpt_string : ""

      prefix + text.chars[start_pos..end_pos].strip + postfix
    else
      nil
    end
  end

and you can see the call that shows me where the errant <b> is...

puts excerpt(response.body, "<b>")

I also have some setup code that handles the login and log out mocking, but I'll leave that for the end user to sort out ;)

So I think this will make sure that now and in the future this particular view will not bleed user input HTML.

Once I did the complex one above the rest of the views were much easier and quicker to implement. Here is an example of a really simple one...

  it "should escape all user input" do
    @person= mock_model(Person, :name => 'person name <b>', :first_name => 'person first name <b>', :last_name => 'person last name <b>', :alias => 'person alias <b>', :show_gender => 'Male', :about_me => 'about <b>', :updated_at => DateTime.now, :created_at => DateTime.now, :pets => [])    

    assigns[:person] = @person

    render "/people/show"   

    response.should_not have_text(/<b>/)
  end

Couldn't be much simpler, but I found one place where I was not escaping the HTML!

Posted in ,  | Tags , ,  | 4 comments | no trackbacks