Sitemap generation

Interesting article on having rails generate a sitemap. In the past I have used the google site map tool

One of the comments also noted a plugin

Upgrading Rails to 1.2.x

Trying to upgrade to 1.2.1 but being very suspicious.

Lock your rails app to a particular version.

In config/environment.rb:

RAILS_GEM_VERSION = '1.1.6' unless defined? RAILS_GEM_VERSION

Thanks to Steven Bristol for this tip.

Tried the tip on one of my machines. render :partial_success :| script/server seems to run but rake test still uses the latest rails and therefore breaks horribly. I guess I’m not alone:

Mr. van doorn has trouble too.

update

My project was created in rails v1.0. It turns out you need to rake rails:update:configs to update the config/boot.rb file. After I did that then test methods respected the RAILS_GEM_VERSION = '1.1.6' in environment.rb

Thanks to my co-conspirator, Stephen Ryan with the hint.

Tagging

First of all I’m not so sure tagging is useful. This is more of an experiment than anything. When I use them, say in flickr, I tend to just browse around. Maybe that’s all they’re good for…

Can google search them?
How do I implement them into my sitemap?

I suppose I should add the ability to comment…

I’m using acts_as_taggable plugin. One item that makes your form much easier to code is a simple model alias alias tag_list= tag_with Now you can just use this in your form.

<label for="blog_tag_list">Tags</label>
<%= text_field 'blog', 'tag_list' %>

This also saves work in the controller since you just use your normal scaffold code to create and update

Fixture Scenarios update

FixtureScenarios has moved. I finally got it installed and I’m impressed; out of the box, it fixes fixtures so that they clean up after themselves.

finding relationships

I haven’t found a better way to discover all relationships between models other than loading in all models from the app/models directory and trying to instantiate them and then looking to see if they are a kind of ActiveRecord:Base. If they are then we can run reflect_on_all_associations.

In doing this I modified some code I found on the web to draw a .svg image of the relationships. Code looks like:

#!/usr/bin/env ruby
require "config/environment"
Dir.glob("app/models/*rb") { |f|
    require f
}

puts %{digraph x \{ 
  has_many1 [shape=point]
  has_many2 [shape=point]
  has_many1 -> has_many2 [label="Has many", color=red]
  belongs_to1 [shape=point]
  belongs_to2 [shape=point]
  belongs_to1 -> belongs_to2 [label="Belongs to", color=blue]
  has_and_belongs_to_many1 [shape=point]
  has_and_belongs_to_many2 [shape=point]
  has_and_belongs_to_many1 -> has_and_belongs_to_many2 [label="HaBtM", color=green]
  has_one1  [shape=point]
  has_one2  [shape=point]
  has_one1 -> has_one2 [label="Has one", color=gray]
  node [shape=box, style=filled, fillcolor=lightgray, width=2.5] 
}
Dir.glob("app/models/*rb") { |f|
    f.match(/\/([a-z_]+).rb/)
    classname = $1.camelize
    classname = $1 + 'QR' if classname =~ /(\w+)Qr$/ # total hack for question response models
    klass = Kernel.const_get classname
    obj = klass.new rescue next
    if obj.kind_of? ActiveRecord::Base
        puts classname
        klass.reflect_on_all_associations.each { |a|
          att = case a.macro
                  when :has_many : 'color=red, '# label="#{a.macro.to_s.humanize}"
                  when :belongs_to : 'color=blue, '
                  when :has_and_belongs_to_many : 'color=green, '
                  when :has_one : 'color=gray, '
                  else 'color=black, '
                  end
          puts %{ #{classname} -> #{a.name.to_s.camelize.singularize} [ #{att} fontsize="8"]}
        }
    end
}
puts "}"

1. Install Graphviz
2. Save that code in your lib or bin directory of your rails root. I called it rails_visual.rb then try:

PNG output
ruby bin/rails_visual.rb > surveil.dot; dot -Tpng -osurveil.png surveil.dot

SVG output
@ruby bin/rails_visual.rb > surveil.dot; dot -Tsvg -osurveil.svg surveil.dot @

Follow the image link to flickr to see the original (7959 × 1364) image

less is more

Use less to view your logs.

 less log/development.log

What? it looks ugly? with weird escape characters and such…?

Try this:

 less -r log/development.log

Why? Well if you have a large log file then it could take a while to load it into an editor and ‘less’ is smarter than that… Search using normal vi commands: / and ?

Move around in the file using SPACE, j,k,l,; or the arrow keys. g gets you to the top of the file and G gets you to the bottom.

h is help and as always RTFM, man less

Google-Geocode Gem Woe's

While using the very cool google-geocode gem for ruby, I ran into a small (read: big) problem.

Do a search for “Trinidad” by itself and you get something like:

Communication error: #<REXML::ParseException: Missing end tag for 'AdministrativeAreaName' (got "AdministrativeArea")

The problem lies not in the gem but in ruby’s REXML and how it deals with the xml google sends back. This only happens when international characters are involved.

After doing a little googling I saw a patch for rexml which I changed into a monkey patch for google-geocodes helper library rc-rest.

This monkey patch will solve all your accent mark woe’s.

class RCRest
  
  def get(method, params = {})
    url = make_url method, params

    url.open do |xml|
      body = xml.read

      res = REXML::Document.new Iconv.conv("UTF-8//Ignore", 'UTF-8', body)
  
      check_error res

      return parse_response(res)
    end
  rescue IOError, SystemCallError, SocketError, Timeout::Error,
         REXML::ParseException => e
    raise CommunicationError.new(e)
  rescue OpenURI::HTTPError => e
    begin
      xml = REXML::Document.new e.io.read
      check_error xml
    rescue REXML::ParseException => e
    end
    new_e = CommunicationError.new e
    new_e.message << "\n\nunhandled error:\n#{xml.to_s}"
    raise new_e
  end
  
end

The magical change is the inclusion of Iconv to make REXML happy.

Syntax Highlighting for Everyone!

I recently integrated a javascript based syntax highlighter into this blog. Its very easy to do and quiet useful. Here is a quick rundown. I also go over some alternative methods afterwards.

The software I ended up using was SyntaxHighlighter

Instructions can be found here just include some files, run the javascript and your gold.

Once thats in place all you have todo is invoke:

<pre name="code" class="yourlanguage">
   awesome code snippit here

Where ‘yourlanguage’ is one of ruby,perl,etc

There is a very useful option which allows you to match line numbers to the file you may be refering to (for example your code may begin on line 10).

<pre name="code" class="yourlanguage:firstline[10]">
   awesome code snippit here

Here is an example from line 35 from a rails controller (Note the line numbers on the left)

def show
  @owner = User.find(params[:user]) || User.find(1)
  @blogs = Blog.paginate :conditions => ["(user_id = ?) AND NOT disabled", @owner.id], 
    :order => 'updated_at DESC', :per_page => 5, :page => params[:page]
end

SyntaxHighlighter supports out of the box:

• Csharp
• C++
• CSS
• Delphi
• JavaScript
• Java
• Php
• Python
• Ruby
• SQL
• VisualBasic
• XML (Which works well for xhtml files)

In addition you can grab shBrushPerl.js which adds perl support.

And thats all there is to it! Syntax Highlighting with Client Side Javascript.

I would also like to point out some other ways to convert code into markup.

• Coderay integrates well with ruby on rails.
• GeSHi is a PHP based generator.
• Highlight is a command line (and gtk gui) based app.

At the very least Highlight’s console output can be grabbed and fed into your web application no matter what language. It also has a cool 256 Color Xterm output which is great for piping code into from grep or less.

Highlight also comes with a slew of existing color schemes in CSS which is nice.

There are many more highlighters out there, google is your friend.

Enjoy the shiny colors!

Easy Whole Site Searching

This is a method of searching all or some of your tables using Ruby on Rails and MySQL’s ability to do Full Text Searches. This method is all ruby without using ferret or sphinx ,both of which are very good tools.

This whole idea can probably be put into a plugin but I don’t have the time to mess with that yet.

First the model:

class Search < ActiveRecord::Base
  belongs_to :account
  belongs_to :searchable, :polymorphic => true 

  # this blindly creates an index for a particular klass / account combination.
  # there must be an account_id field in the klass being searched
  #
  # Note: This searches all columns of all records for this account which
  # is probably Too Much Information, TMI, Baby. You may want to define a
  # method called column_names_to_search which will restrict what is searched.
  #
  # Note: You may want to overide this method by defining your own
  # 'create_searchable_index' in your class.
  def self.create_searchable_index(account, klass)
    raise unless account.kind_of?(Account)
    return klass.create_searchable_index(account) if klass.respond_to?('create_searchable_index')
    Search.delete_all ["account_id = ? and searchable_type = ?", account[:id], klass.to_s]
    klass.find(:all, :conditions => "account_id = #{account[:id]}").each do |thing|
      if klass.respond_to?('column_names_to_search')
        text = klass.column_names_to_search.map{|col| thing.send(col).to_s}
      else
        text = klass.column_names.map{|col| thing.send(col).to_s}
      end
      Search.create! :account_id => account[:id], :searchable_id => thing.id, :searchable_type => thing.class.to_s, :searchable_text => text.join(' ')
    end
    Search.count :conditions => "account_id = #{account[:id]} and searchable_type = '#{klass}'"
  end

  def self.match(account, query, just_this_object = nil)
    matches = []
    sql = %{SELECT searchable_id, searchable_type FROM searches WHERE account_id = #{account.id} AND MATCH searchable_text AGAINST ('#{query}')}
    if just_this_object
      sql += %{ AND searchable_type = '#{just_this_object}'}
    end

    Search.find_by_sql(sql).each do |found|
      matches << eval("#{found.searchable_type}").find(found.searchable_id)
    end
    matches
  end
end

As the comments suggest you will probably be better off writing a method in the models you want to search. This allows you to expand certain associations or delete sensitive data. Here’s an example where I’m removing the id, updated_at, system, and account_id from the stuff I search, self.column_name_to_search. self.create_searchable_index does the heavy lifting of:

1. Deleting the old index
2. Finding all the groups for this account and looping on them to
   1. get just the columns I want
   2. add the user’s name to the searchable text so when we search for “Chad” we get the user and all the groups that use is in.
   3. create the entry in the Searches table.
3. Return the count of how many entries were created.

class Group < ActiveRecord::Base
  belongs_to :account
  has_many   :group_user_memberships, :order => 'position', :dependent=> :destroy
  has_many   :users, :through => :group_user_memberships, :order => "group_user_memberships.position, last_name, first_name, username"

# ... 

  private

  def self.column_names_to_search
      self.column_names - ["id", "updated_at", "system", "account_id"]
  end

  def self.create_searchable_index(account)
    Search.delete_all "account_id = #{account[:id]} and searchable_type = 'Group'"
    Group.find(:all, :conditions => ["account_id = ?",account[:id]], :order => 'name').each do |group|
      text = Group.column_names_to_search.map{|col| group.send(col).to_s}
      text << group.users.map(&:name)
      Search.create! :account_id => account[:id], :searchable_id => group[:id], :searchable_type => Group.to_s, :searchable_text => text.flatten.join(' ')
    end
    Search.count :conditions => "account_id = #{account[:id]} and searchable_type = 'Group'"
  end

end

The controller is very simple. It takes the form input which is displayed on everypage ala the layout. We use haml for markup.

-form_tag(:controller => 'search', :action => 'index' ) do
  %input.search_button{ :name => 'search_text', :type => 'text', :size=> '20', :value => (session[:search_text] || 'Search'), :onclick => 'this.focus();this.select()', :onfocus => 'this.select()' }/
  = submit_tag 'Search'

And finally the controller. There are 2 methods. One builds the indexes and the other does the match.

require 'benchmark'

class SearchController < ApplicationController
  layout 'home'
  before_filter :only_admin, :except => [:index]

  def index
    session[:search_text] = params[:search_text]
    @bench = Benchmark.measure do
      @matches = Search.match @user.account, session[:search_text] 
    end
  end

  def create_index
    count = 0
    SEARCH_CLASSES.each do |klass|
      count += Search.create_searchable_index(@account,klass)
    end
    flash[:success] = "Indexed #{count} records"
    redirect_to :controller => 'home', :action => 'index'
  end

  private

  def only_admin
    permit SUPERUSER_GROUP_NAME
  end
end

The view for the search results is pretty simple. Rather than create a unified view for all models that can be searched I create a partial called _search.haml in the views for their corresponding controller. You don’t have to do this but probably you will want to differentiate your models display results.

#search
  %br/
  %h3
    Found 
    = @matches.nitems
    matches for
    %span.italic
      = '"' + session[:search_text] + '"'
    in
    = sprintf "%2.3f", @bench.real
    seconds
  %br/
  
  %ul
    - @matches.each do |thing|
      %li.no_list_style
        %strong
          = thing.class.to_s
        %br/
        = render(:partial => %{#{thing.class.to_s.tableize}/search}, :locals => {:thing => thing}) #rescue thing.inspect
      %br/

acts_as_conference

We are all signed up! See you in Orlando.

A Window Into Functional Tests

So like any ruby blooded human, I create functional tests for my rails applications. However, things don’t always go as planned; a redirect instead of a success, but where to?, the assigns is right, but the flash was wrong, etc.

Sometimes you just need a way to peek at what you get back. If you enjoy gouging your eyes out you can do a puts @response or @response.body

A slightly better alternative is to spit the body to a file and preview it in firefox.

    tmpfile = File.new(tmpname = 'tmp/test_page.html', "w")
    tmpfile.puts @response.body
    tmpfile.close
    `firefox #{tmpname}`

Put this after any get, post, etc, and you will get a decent html output of your view (sans stylesheets and valid links) Although nothing is stopping you from outputting to public, running script/server, and viewing it from there.

• If your dealing with redirects, don’t forget about follow_redirect!
• If your crossing controllers, use integration tests =)

It’s not the be all end all of solutions, but it helps for a quick glimpse while fixing tests.
Hope it helps!

ActiveRecord.update_attributes has_many :security_holes, :through => :unkown_features

Among you railites who have successfully learned rails, watched tutorials, and generally feel comfortable about your abilities: Probably didn’t know there is a little security hole in your app.

It has to deal with update_attributes, has_many relationships, and a method made available on the parent in the relation.

Example:

class User < ActiveRecord::Base
  has_many :groups
end

In your view you have your pretty form with user’s name and other demographics they can enter in their profile. and a ‘save’ button that leads to a call to ‘update_attributes’.

The problem lies in the fact that has_many creates a method off your object called

 user.group_ids= 

Which allows you to pass in an array of ids and create associations en-mass.
the problem is that I can come in with firebug and add my own fields.

<!-- im in your html source adding my inputs -->
<input type="text" name="user[group_ids][]"/>
<input type="text" name="user[group_ids][]"/>
<input type="text" name="user[group_ids][]"/>

After filling those fields and submiting, if you inspect the params hash you will notice:
- “parent” => {…. “association_ids” => [“1”,“2”,“4”]}

And if you check your script/console and check the associations, they will be there assuming you have groups with id’s of 1,2, and 4.

The implications? If you use these groups for any kind of role based access, a user could assume a group with root/super/power user access!

The lesson?

Protect your attributes!

 attr_protected :group_ids 

But! a better idea would be to use:

 attr_accessible :name, :bio, :etc 

I hope this has helped you as much as it did me!

-TheHacker

GeoKit stubbing for faster tests

We recently added geocoding to after_save on an address model to keep track of peoples lat/lng, and found that it added substantial time to our tests. The solution? Easy.

Don’t really need to test that geolocating works, I mean.. GeoKit has its own unit tests. So Stub it! Stub it good!

Place this in your test_helper.rb (and inside Test::Unit::TestCase) for tolerable test times.

“This was written for mocha, but its easy to adapt to rspec or flexmock”

  setup :stub_geocoder
  def stub_geocoder
    geocode_payload = GeoKit::GeoLoc.new(:lat => 123.456, :lng => 123.456)
    geocode_payload.success = true
    GeoKit::Geocoders::MultiGeocoder.stubs(:geocode).returns(geocode_payload)
  end

“Just remember to turn it off if you plan to test any features that rely on actual geocoding!”

Awesome Nested Set Revealed!

Ok, there’s nothing new with nested sets in rails but I crawled through the Awsome Nested Set Code from CollectiveIdea on GitHub code and made a quick and dirty cheat sheet. Here you go.

Awesome Nested Set Cheat Sheet:

Basic Usage

Create a root node:

Put a new thing inside this root node:

Put a “new” new thing inside this “new_thing” node:

Now you should have something that resembles this:

Gratuitous picture of my dog Ella.

Accessing the data:

Class methods:

• Category.root – returns the first root node
• Category.roots – returns all root nodes

Instance methods:

• my_cat.root? – true if this is a root node
• my_cat.leaf? – true if this is a leaf node. It has no children.
• my_cat.leaves – returns array of all nodes without children.
• my_cat.child? – true if this is a child node. It has a parent.
• my_cat.root – root for this node.
• my_cat.parent – returns the immediate parent
• my_cat.self_and_ancestors – returns array of all parents and self
• my_cat.ancestors – returns array of all parents. Not including self.
• my_cat.self_and_siblings – returns array of brothers and sisters and self. All at that level.
• my_cat.siblings – returns array of brothers and sisters. Not including self.
• my_cat.level – returns the level of this object in the tree. root = 0
• my_cat.self_and_descendants – returns array of all children and self
• my_cat.descendants – returns array of all children. Not including self.
• my_cat.children – returns array of immediate children. Just those in the next level.
• my_cat.is_descendent_of?(obj) – returns true if self is nested under obj
• my_cat.is_or_is_descendent_of?(obj) – returns true if self is nested under obj or self is obj
• my_cat.is_ancestor_of?(obj) – returns true if nested by any obj
• my_cat.is_or_is_ancestor_of?(obj) – returns true if nested by any obj or self is obj

Flexible date input and manipulation in javascript with date.js

date.js is a great little JavaScript library that can make your life a lot easier.

If your used to ruby’s date functions then date.js will make you feel right at home.

date.js can:

• Parse strings into dates.
• Add and Subtract time in days,hours,months or years.
• Easily return “x” “day of week” of “month”
• Boolean assertions for day, week, month, year.
• Turn you into a JavaScript ninja.

Note: Some of the syntactic goodies require ‘sugar.js’

Going into the future is no problem for date.js

Interested in this Friday? April of this year?

What about the first friday of april? No Sweat!

Assert any date properties you want.

It can parse just about anything you throw at it.

There are also some fun number functions.

For more reading, checkout: date.js examples

Protect Staging Server with Password

When developing a new application and releasing often you may want to protect the site from prying eyes. A simple way to do this when using the nginx web server is to use basic http authentication.

I prefer to do this in the nginx config file so it doesn’t impact local development.

You’ll need to create a password file using htpasswd command line utility. This file should be relative to the install directory of nginx. The default install would place this file in /usr/local/nginx

Next, you need to tell nginx to use basic auth for any access to this (virtual) host. I place the 2 relevant lines in the server portion of the config file

1. …
   auth_basic “Login to see this alpha site.”;
   auth_basic_user_file this_hosts_password_file;
2. …
   }

A login will now be presented for any entry into the site.

Pretty Local Rdoc

From the RubyInside site a couple of gems to make your local rdocs look and behave nicer.

To reap the benefits: