Who is talking?

ActiveRecord with large result sets - part 1: select_all vs. find

about 5 hours ago | Pat Shaughnessy: Pat Shaughnessy - Home

It’s well documented that using ActiveRecord::Base.connection.select_all can speed up ActiveRecord database queries when you are expecting a large result set. This is because select_all just returns an array of hashes containing the results, and ActiveRecord doesn’t have to do the work of instantiating a new model object for each row in the result.

Here’s an example:

ActiveRecord::Base.connection.select_all('SELECT * FROM users')

This returns an array of hashes, instead of user model objects:

=> [{"name"=>"pat",
     "created_at"=>"2010-09-03 16:59:24.097209",
     "updated_at"=>"2010-09-03 16:59:24.097209",
     "id"=>1,
     "email"=>"pat@patshaughnessy.net"}, etc...]

You can also use “select_values” if you need just one value from a single database column:

ActiveRecord::Base.connection.select_values('SELECT name FROM users')

=> ["pat", etc... ]

If select_all is faster than using a normal call to ActiveRecord::Base.find, why not use select_all for everything? The reason is that the extra speed select_all offers comes with a heavy price:

You have to write the SQL statement yourself. Normally ActiveRecord generates the SQL statement for you automatically based on your associations, named scopes, etc.
You have to use a hash to access the result, instead of a model object. In this example, you would have to write user[“name”] instead of user.name. Aside from being harder to read and type, with a hash you also lose the opportunity to call model methods to process the value somehow.

So using select_all is a tradeoff: more speed vs. less coding convenience. Is it worth it?

Well, it might be if:

You expect a large number of results from a database query - this will make the speed benefit more noticeable.
It’s easy to write the SQL select statement, because you don’t have many different SQL statements to write based on different combinations of associations and scopes.
Or if you need to use an unusual, hand crafted SQL statement that ActiveRecord wouldn’t normally generate you might have to use select_all regardless of the speed benefit.

This week I ran some Rails performance tests to measure the actual speed difference for a very simple find(:all) query with no associations. What I found was:

For Rails 3, select_all was almost 2x times faster: it took ActiveRecord about 40% less time to return an array of hashes with select_all than it did to return an array of model objects.
For Rails 2.3.8, there was a less noticeable difference: only a 22% improvement.
Using Ruby 1.8.7 vs. Ruby 1.8.6 was much faster overall: about 2x faster for both select_all and find.

This rest of this article will show you how I setup and ran the performance tests, and what my detailed results were. Most of what I’m going to do below is taken form an excellent Rails guide article by Pratik Naik: Performance Testing Rails Applications.

Setting up for a Rails performance test

I’ll get started by creating a new Rails 3 app - using the 3.0.0 version which was released just last week!

$ rails new perf_test
      create
      create  README
      create  Rakefile
      create  config.ru
      create  .gitignore
      create  Gemfile
etc....

And now I’ll create a user model from the example above:

$ cd perf_test
$ rails generate model user name:string email:string
      invoke  active_record
      create    db/migrate/20100903165238_create_users.rb
      create    app/models/user.rb
      invoke    test_unit
      create      test/unit/user_test.rb
      create      test/fixtures/users.yml

$ rake db:migrate
(in /Users/pat/rails-apps/perf_test)
==  CreateUsers: migrating ==============================
-- create_table(:users)
   -> 0.0013s
==  CreateUsers: migrated (0.0014s) =====================

Next I’ll write a simple rake task in a file called lib/tasks/users.rake to create a specified number of users, so we can time queries based on different numbers of records. Also, having a separate rake task for this avoids the possibility of including the record create time in the performance tests.

task :populate_users => :environment do
  count = ENV['count'].to_i
  User.delete_all
  count.times do
    User.create :name  => 'pat',
                :email => 'pat@patshaughnessy.net'
  end
  puts "User count: #{User.count}"
end

Now I’ll create a new performance test using the Rails generator, and delete the test that came with the new app:

$ rails generate performance_test load_users
      invoke  test_unit
      create    test/performance/load_users_test.rb
$ rm test/performance/browsing_test.rb

Editing the load_users_test.rb file, I’ll add a couple simple tests that load all of the user records:

require 'test_helper'
require 'rails/performance_test_help'

class LoadUsersTest < ActionDispatch::PerformanceTest
  def test_find
    user_models = User.find :all
    puts "Loaded #{user_models.size} users"
  end

  def test_select_all
    user_hashes =
      ActiveRecord::Base.connection.select_all('SELECT users.* FROM users')
    puts "Loaded #{user_hashes.size} users"
  end
end

Finally, I’ll install “ruby-prof,” a helpful profiling tool that will allow us to use the rake test:profile command:

$ gem install ruby-prof
Building native extensions.  This could take a while...
Successfully installed ruby-prof-0.9.2
1 gem installed
Installing ri documentation for ruby-prof-0.9.2...
Installing RDoc documentation for ruby-prof-0.9.2...

And I need to add this gem in my Gemfile:

source 'http://rubygems.org'

gem 'rails', '3.0.0'

gem 'ruby-prof'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

Ok - great! Now I’ll create 10,000 users and then run the new performance tests:

$ rake populate_users count=10000
(in /Users/pat/rails-apps/perf_test)
User count: 10000

$ rake test:profile
(in /Users/pat/rails-apps/perf_test)
Loaded suite /Users/pat/.rvm/gems/...
Started
Loaded 2 users
LoadUsersTest#test_find (10 ms warmup)
Loaded 2 users
        process_time: 4 ms
              memory: unsupported
             objects: unsupported
.Loaded 2 users
LoadUsersTest#test_select_all (0 ms warmup)
Loaded 2 users
        process_time: 2 ms
              memory: unsupported
             objects: unsupported
.
Finished in 0.319634 seconds.

6 tests, 0 assertions, 0 failures, 0 errors

Huh? What happened? Why were there only two users loaded? Well it turns out that the 2 users came from the test/fixtures/users.yml fixtures file create by the model generator I ran earlier:

one:
  name: MyString
  email: MyString

two:
  name: MyString
  email: MyString

And the ten thousand users I created with my rake task were put into my development database... not my test database.

Retaining the contents of the test database between test suite runs

I can easily run my rake task using RAILS_ENV=test to fill the test database with users instead of the development database, but they will still be cleared out when the test database is purged and reloaded each time I run my profile test suite. What I really want to do is retain the contents of my test database each time I run the tests. Using code from a helpful stack overflow discussion on how to do this, I put this function into my Rakefile:

require File.expand_path('../config/application', __FILE__)
require 'rake'

Rake::TaskManager.class_eval do
  def remove_task(task_name)
    @tasks.delete(task_name.to_s)
  end
end

PerfTest::Application.load_tasks

...and I created a NOP task in my users.rake file, after calling remove_task:

Rake.application.remove_task 'db:test:prepare'

namespace :db do
  namespace :test do 
    task :prepare do |t|
    end
  end
end

Finally I need to be sure to delete the users.yml fixture file, or Rails will still delete and reload all of the users between each individual test:

$ rm test/fixtures/users.yml

Just how much faster is select_all vs. find?

Ok - now I’m all set to run some tests; let’s start with 1000 users records:

$ rake populate_users count=1000 RAILS_ENV=test
(in /Users/pat/rails-apps/perf_test)
User count: 1000

$ rake test:profile
(in /Users/pat/rails-apps/perf_test)
Loaded suite /Users/pat/.rvm/gems/ruby-1.8.7...
Started
Loaded 1000 users
LoadUsersTest#test_find (54 ms warmup)
Loaded 1000 users
        process_time: 398 ms
              memory: unsupported
             objects: unsupported
.Loaded 1000 users
LoadUsersTest#test_select_all (33 ms warmup)
Loaded 1000 users
        process_time: 241 ms
              memory: unsupported
             objects: unsupported
.
Finished in 0.99688 seconds.

6 tests, 0 assertions, 0 failures, 0 errors

Each performance profile test gives us three results: process time, memory usage and the number of Ruby objects created. However, since I’m not using the “GC Patched” (garbage collection patch) version of Ruby I only get the process time value. In a future blog post I’ll show how to update your Ruby interpreter with the patch that counts the number of objects created, and measures the amount of memory used. “Process time” refers to the amount of time used by the Ruby process, not the actual time elapsed (the “wall time”).

But for now, we can see that the first test, which uses ActiveRecord::Base.find, took 398ms to load 1000 rows from the SQLite database, and to return an array of 1000 user model objects. The second test, using ActiveRecord::Base.connection.select_all, took 241ms to load the same data but return it in the form of an array of hashes.

Let’s increase the number of users to 10,000 and repeat the test:

$ rake populate_users count=10000 RAILS_ENV=test
(in /Users/pat/rails-apps/perf_test)
User count: 10000

$ rake test:profile
(in /Users/pat/rails-apps/perf_test)
Loaded suite /Users/pat/.rvm/gems/ruby-1.8.7...
Started
Loaded 10000 users
LoadUsersTest#test_find (476 ms warmup)
Loaded 10000 users
        process_time: 3.99 sec
              memory: unsupported
             objects: unsupported
.Loaded 10000 users
LoadUsersTest#test_select_all (378 ms warmup)
Loaded 10000 users
        process_time: 2.42 sec
              memory: unsupported
             objects: unsupported
.
Finished in 7.730976 seconds.

6 tests, 0 assertions, 0 failures, 0 errors

It looks like the time taken for 10,000 users is simply 10x the amount taken for 1,000. In other words, this is a fairly linear process: it takes my laptop a certain number of milliseconds to process each user row: about 0.4 ms per row for ActiveRecord.find and 0.24ms for select_all.

Results

Here are my timings - I’m using a MacBook Pro with a 2.6 GHz Intel Core 2 Duo processor. My database server is SQLite 3.7.0, via the sqlite3-ruby-1.3.1 gem. FYI the SQLite gem version seems to be important; using an older version of this gem slowed down the results dramatically.

Rails 3.0.0/Ruby 1.8.7:

records	select_all	find	delta
1000	241ms	398ms	39%
10000	2,420ms	3,990ms	39%
100000	25,480ms	42,580ms	40%

Rails 2.3.8/Ruby 1.8.7:

records	select_all	find	delta
1000	262ms	336ms	22%
10000	2,660ms	3,390ms	22%
100000	27,770ms	35,490ms	22%

Rails 2.3.5/Ruby 1.8.6:

records	select_all	find	delta
1000	530ms	651ms	19%
10000	5,350ms	6,540ms	18%
100000	53,820ms	67,030ms	20%

Code record/playback using Rails 3 generators

about 1 month ago | Pat Shaughnessy: Pat Shaughnessy - Home

For a while I’ve been thinking that writing a Rails generator is a fairly difficult thing to do. First you need to learn about Thor and the Rails generator system: what sort of Ruby class you need to write, how to handle arguments, how to run commands like “copy_file”, etc. Then you need to write ERB files to produce the code that you’d like to generate, which is always a chore.

So last night I wrote a gem called generate_from_diff that let’s you create Rails 3 generators automatically using a code record/playback model. Here’s how it works:

You make a code change in some Rails project, and commit it to your Git repo.
You run a command from my gem to extract this code change from the Git repo into a new Rails 3 generator.
You later run this new generator in any other Rails 3 app, and your recorded code changes are “played back,” or applied to your new project, using the Unix patch utility written by Larry Wall.

You’ve created a Rails 3 generator without ever writing a single line of generator code!

Disclaimer: I just got this working last night, so it’s still very rough; but if the idea seems worthwhile I’ll clean it up and try to make it more robust and useable. Another disclaimer: none of this will work on Windows since it relies on the Unix patch utility.

Example: recording code into a new generator

Let’s look at an example to try to make this a bit clearer. Support I create a new Rails 3 application:

$ rails new first_app
      create  
      create  README
      create  Rakefile
      create  config.ru
      create  .gitignore
      create  Gemfile
etc...

And let’s run bundle install to be sure I have all the required gems. This is usually not necessary for a new, empty Rails app, but I want to have my Gemfile.lock file created... more on that in a moment.

$ cd first_app
$ bundle install
Fetching source index for http://rubygems.org/
Using rake (0.8.7) 
Using abstract (1.0.0) 
Using activesupport (3.0.0.beta4) 
Using builder (2.1.2) 
etc...

And let’s create a new Git repo here and check the empty application into it:

$ git init
Initialized empty Git repository in /Users/pat/.../first_app/.git/
$ git add .
$ git commit -m"New sample app"

This first Git revision will serve as the baseline for recording my new generator, which I’ll do in a minute. The reason I ran bundle install was to insure that the Gemfile.lock file would be included in the baseline... and so not included in the recorded code change.

Now let’s write some code that I can record into a new generator. Suppose at my company I want to create a controller that returns the build number, diagnostics and some other information about each of my Rails apps. I might do this by creating a new controller as follows:

$ rails generate controller build_info
      create  app/controllers/build_info_controller.rb
      invoke  erb
      create    app/views/build_info
      invoke  test_unit
      create    test/functional/build_info_controller_test.rb
      invoke  helper
etc...

And in this new controller I’ll add a single index action:

class BuildInfoController < ApplicationController
  def index
    render :text => 'Some interesting build info about this app...'
  end
end

Finally, I’ll add a route to send “build_info” requests to this action:

FirstApp::Application.routes.draw do |map|

  match 'build_info' => 'build_info#index'

...etc...

This is somewhat silly, but it’s simple enough to use as an example here. Now if I run my app I’ll get this fascinating page:

Next let’s “record” this sample code by using generate_from_diff to create a new Rails generator for it. First, we need to install generate_from_diff:

$ gem install generate_from_diff
Successfully installed generate_from_diff-0.0.1
1 gem installed
Installing ri documentation for generate_from_diff-0.0.1...
Installing RDoc documentation for generate_from_diff-0.0.1...

Next, let’s commit my new controller and routes.rb code changes:

$ git add .

$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#        new file:   app/controllers/build_info_controller.rb
#        new file:   app/helpers/build_info_helper.rb
#        modified:   config/routes.rb
#        new file:   test/functional/build_info_controller_test.rb
#        new file:   test/unit/helpers/build_info_helper_test.rb
#

$ git commit -m"Build info"
Created commit 037ca3b: Build info
 5 files changed, 22 insertions(+), 0 deletions(-)
 create mode 100644 app/controllers/build_info_controller.rb
 create mode 100644 app/helpers/build_info_helper.rb
 create mode 100644 test/functional/build_info_controller_test.rb
 create mode 100644 test/unit/helpers/build_info_helper_test.rb

One last detail: we need to edit the Gemfile to load generate_from_diff into this application:

source 'http://rubygems.org'

gem 'generate_from_diff'
gem 'rails', '3.0.0.beta4'
etc...

Finally we create our new generator by just running this command:

$ rails generate generator_from_diff build_info HEAD~1 HEAD
    create  lib/generators/build_info
    create  lib/generators/build_info/build_info_generator.rb
    create  lib/generators/build_info/USAGE
       run  git diff --no-prefix HEAD~1 HEAD from "."

Ok - what happened here was that I ran a generator called “generator_from_diff” that is located inside the generate_from_diff gem. I provided it with the name of the new generator I want to create: “build_info” in this example. This is similar to how the Rails 3 “generator generator” works: it generates a generator. But next I provide two Git revisions, in this example “HEAD~1” and “HEAD,” the first and second revisions in my Git repo. The first value is the baseline revision: what to compare to. In this example, this is my new, empty Rails application. The second revision is what code to record and save into the new generator, in this example this revision contains all of my controller and routes.rb changes.

Example: playing back code using a generator

Now let’s see if we can use this new Rails generator to copy the build_info controller and route into a different Rails app. First, let’s create a second, new Rails application:

$ cd ..
$ rails new second_app
    create  
    create  README
    create  Rakefile
    create  config.ru
...etc...
$ cd second_app

And next, let’s copy the new generator we just created in the first app, over to this new app:

$ mkdir lib/generators
$ cp -r ../first_app/lib/generators/build_info lib/generators

And now we can just run our new generator to playback the code changes that I recorded above:

$ rails generate build_info
      gsub  lib/generators/build_info/build_info.patch
       run  patch -p0 < /Users/pat/.../second_app/lib/generators/build_info/build_info.patch from "."
  patching file app/controllers/build_info_controller.rb
  patching file app/helpers/build_info_helper.rb
  patching file config/routes.rb
  patching file test/functional/build_info_controller_test.rb
  patching file test/unit/helpers/build_info_helper_test.rb

That’s it! Now I can run the second app and see the same build status page that we had before:

How does this actually work?

Here’s what is going on under the hood. First, when you record your code changes into the new generator like this:

$ rails generate generator_from_diff build_info HEAD~1 HEAD

... the “generator_from_diff” code actually runs the “git diff” command like this:

$ git diff HEAD~1 HEAD
diff --git a/app/controllers/build_info_controller.rb
           b/app/controllers/build_info_controller.rb
new file mode 100644
index 0000000..c44d83e
--- /dev/null
+++ b/app/controllers/build_info_controller.rb
@@ -0,0 +1,5 @@
+class BuildInfoController < ApplicationController
+  def index

...etc...

This produces a list of all the text changes that were made from one revision (HEAD~1) to another (HEAD). These are then saved into a file called “build_info.patch,” saved inside the new generator.

Later, the text differences, the “patch,” are applied to whatever new or existing files are found relative to the current directory when you run the generator. This copies the new controller file as well as the new route inside of routes.rb into the other application. The patch file is applied using this command:

patch -p0 < lib/generators/build_info/build_info.patch

I use patch instead of git apply to avoid the need to match revision id’s; these will be different from one repo to another.

Ok sounds interesting - so where are you going with this next?

I think it’s cool to be able to “record” Rails generators without writing any code. If this seems like a useful idea, then I’ll spend some more time to clean it up and make it more robust. For example, I’m thinking of adding some code to warn you before the patch is run if there are unexpected files present, or if some expected files are missing.

Next, I’m considering enhancing the gem to perform search/replace using arguments or options that you specify when recording the generator. For example, suppose you recorded a series of code changes that had to do with a model called “Person.” But imagine that you want to be able to playback those code changes in a target application that might have a different model name, “User” instead of “Person” for example. Then the gem could search/replace on the patch file, both when it’s recorded and again when it’s played back, to cause the generated code to use User instead of Person.

OSCON

about 1 month ago | Pat Shaughnessy: Pat Shaughnessy - Home

Update: OSCON was a lot of fun... if anyone is interested in seeing the slides Alex and I wrote they’re up on slide share.

I just arrived in Portland for OSCON; my friend Alex Rothenberg and I are going to talk here about on Thursday about using Rails in the Enterprise… about some of the things he and I do at our day job while working with Rails and legacy technologies. We'll post the slides online later this week.

I won’t be blogging here, but if you’re interested in what’s going on at OSCON follow me on Twitter @patshaughnessy2, or the official tag is #OSCON. If you are in Portland definitely drop by and say hello!

Using LEFT OUTER JOIN with SearchLogic

about 1 month ago | Pat Shaughnessy: Pat Shaughnessy - Home

LEFT OUTER JOIN queries are a great way to handle associations that may contain missing or null values. For example, suppose I have an application with a has_one/belongs_to association between books and authors.

class Book < ActiveRecord::Base
  has_one :author
end
class Author < ActiveRecord::Base
  belongs_to :book
end

If I want to identify the books that don’t have any authors, I can use this SQL statement:

SELECT * FROM books LEFT OUTER JOIN authors ON authors.book_id = books.id
WHERE authors.book_id IS NULL

I need to use LEFT OUTER JOIN here since a normal INNER JOIN query would only return books that have authors. Or if I want to sort the books by their author, I could use:

SELECT * FROM books LEFT OUTER JOIN  authors ON authors.book_id = books.id
ORDER BY authors.name

This sort would work even if some of the books didn’t have an author assigned to them. If we had used a normal INNER JOIN query the books with no author would have been dropped from the sorted list.

Today I’m going to discuss how to use LEFT OUTER JOIN with ActiveRecord and SearchLogic, allowing you to handle associations that might have missing records easily and cleanly.

ActiveRecord :joins

Before we go any farther, let’s setup the book/author example so we can explore how ActiveRecord handles joins:

$ rails outer_join_example
$ cd outer_join_example
$ script/generate model book title:string
$ script/generate model author name:string book_id:integer
$ rake db:migrate

And don’t forget to add the has_one/belongs_to lines to the two new models:

class Book < ActiveRecord::Base
  has_one :author
end
class Author < ActiveRecord::Base
  belongs_to :book
end

And now let’s create some books and authors we can use to test with:

$ script/console 
Loading development environment (Rails 2.3.5)
>> [ 'One', 'Two', 'Three' ].each do |title|
?> book = Book.create :title => title
>> book.author = Author.create :name => "Author of Book #{title}"
>> book.save
>> end

Let’s get started by looking at how you would sort the books by their author’s name, using a normal INNER JOIN query:

>> ActiveRecord::Base.logger = Logger.new(STDOUT)
>> Book.find(:all, :joins => :author, :order => 'authors.name')
       .collect { |book| book.title }
   Book Load (1.4ms)   SELECT "books".* FROM "books" INNER JOIN "authors"
                       ON authors.book_id = books.id ORDER BY name
=> ["One", "Three", "Two"]

Great – here using find with the :joins option we’ve told ActiveRecord to load all the books, and to join with the authors table so we can sort on the authors.name column. But watch what happens if I add a couple of new books that have no author yet, and then repeat the same INNER JOIN sort query:

>> Book.create :title => 'Four'
>> Book.create :title => 'Five'
>> Book.find(:all, :joins => :author, :order => 'name')
       .collect { |book| book.title }
   Book Load (1.8ms)   SELECT "books".* FROM "books" INNER JOIN "authors"
                       ON authors.book_id = books.id ORDER BY name
=> ["One", "Three", "Two"]

Books “Four” and “Five” are dropped entirely. This is because INNER JOIN only includes records in the result set that have values from both the books and authors tables. Since there is no author record for these books, they don’t appear at all.

ActiveRecord :include

The simplest way to get around this problem is to use ActiveRecord’s :include option, instead of the :joins option. Let’s rewrite that call to Book.find and use :include instead:

>> Book.find(:all, :include => :author, :order => 'authors.name')
       .collect { |book| book.title }
   Book Load Including Associations (2.8ms)   SELECT "books"."id" AS t0_r0,
"books"."title" AS t0_r1, "books"."created_at" AS t0_r2,
"books"."updated_at" AS t0_r3, "authors"."id" AS t1_r0,
"authors"."name" AS t1_r1, "authors"."book_id" AS t1_r2,
"authors"."created_at" AS t1_r3, "authors"."updated_at" AS t1_r4 FROM "books"
LEFT OUTER JOIN "authors" ON authors.book_id = books.id ORDER BY authors.name
=> ["Four", "Five", "One", "Three", "Two"]

Now we get books “Four” and “Five” in the sorted list; this is because ActiveRecord uses a LEFT OUTER JOIN query when you specify the :include option. Note they appear first since their author name values are both null, which is sorted before any of the other authors. If you read the ActiveRecord log output, you’ll see it generated a very complex SQL statement that explicitly mentions each column of both the books and authors tables. It used a column naming pattern, “t0_r1” etc., to identify each column. The reason for all of this is that ActiveRecord is executing the “Load Including Associations” operation, which is loading each and every attribute for all of the associated objects. This guarantees that we get every possible value for all the books and their authors.

So this is perfect! Now I can write a named_scope to sort books by their author name, including null authors, like this:

class Book < ActiveRecord::Base
  has_one :author
  named_scope :sorted_by_author_with_nulls,
    { :include => :author, :order => 'authors.name' }
end

And trying it in the console:

>> Book.sorted_by_author_with_nulls.collect { |book| book.title }
  Book Load Including Associations (3.9ms)   SELECT "books"."id" AS t0_r0,
"books"."title" AS t0_r1, "books"."created_at" AS t0_r2,
"books"."updated_at" AS t0_r3, "authors"."id" AS t1_r0,
"authors"."name" AS t1_r1, "authors"."book_id" AS t1_r2,
"authors"."created_at" AS t1_r3, "authors"."updated_at" AS t1_r4 FROM "books"
LEFT OUTER JOIN "authors" ON authors.book_id = books.id ORDER BY authors.name
=> ["Four", "Five", "One", "Three", "Two"]

I can now also write this named_scope called “missing_author,” which returns just the books that have a missing author:

class Book < ActiveRecord::Base
  has_one :author
  named_scope :sorted_by_author_with_nulls,
    { :include => :author, :order => 'authors.name' }
  named_scope :missing_author,
    { :include => :author, :conditions => 'authors.name IS NULL' }
end

And in the console:

>> Book.missing_author.collect { |book| book.title }
  Book Load Including Associations (1.9ms)   SELECT "books"."id" AS t0_r0,
"books"."title" AS t0_r1, "books"."created_at" AS t0_r2,
"books"."updated_at" AS t0_r3, "authors"."id" AS t1_r0,
"authors"."name" AS t1_r1, "authors"."book_id" AS t1_r2,
"authors"."created_at" AS t1_r3, "authors"."updated_at" AS t1_r4 FROM "books"
LEFT OUTER JOIN "authors" ON authors.book_id = books.id
WHERE (authors.name IS NULL)
=> ["Four", "Five"]

What’s wrong with :include?

It sounds like I’m done and that the :include option is the perfect way to handle associations that might contain null or missing values. But not so fast… it turns out that using :include is often a bad idea. A great resource on :joins and :include options is Ryan Bates’s screen cast from 2009. If you think you need to use one of these find options, invest a few minutes and listen to Ryan’s explanation. Here I’ll just mention a couple of potential problems with using :include with named scopes:

1. It‘s potentially slow and wasteful. :include causes ActiveRecord to load every attribute for every included associated object, which is often much more information than you really need. If your named scope only requires one or two columns from the associated table, then using :include might be overkill.

2. As Ryan mentions, you lose control over the “SELECT” portion of the query. This means that if you write a named scope that uses :include, like the missing_author example above, then you can’t chain it together with other named scopes that contain a select option. For example, you couldn’t write code like this:

Book.missing_author.scoped(:select => 'DISTINCT title')

This might appear to work, but if you look at the query ActiveRecord generates you’ll notice that it doesn’t contain the DISTINCT keyword. This is because the Load Including Associations query ignores the select scope.

A better way to write a named_scope that uses LEFT OUTER JOIN is actually to go back to the :joins option, like this:

class Book < ActiveRecord::Base
  has_one :author
  named_scope :sorted_by_author_with_nulls, {
    :joins => 'LEFT OUTER JOIN authors ON authors.book_id = books.id',
    :order => 'authors.name'
  }
  named_scope :missing_author, {
    :joins => 'LEFT OUTER JOIN authors ON authors.book_id = books.id',
    :conditions => 'authors.id IS NULL'
  }
end

Here I’ve written the join clause of the query for each scope, typing in the LEFT OUTER JOIN syntax explicitly. Here’s the example from above using DISTINCT:

>> Book.missing_author.scoped(:select => 'DISTINCT title')
   Book Load (1.1ms)   SELECT DISTINCT title FROM "books"
                       LEFT OUTER JOIN authors ON authors.book_id = books.id
                       WHERE (authors.id IS NULL) 
=> [#<Book title: "Four">, #<Book title: "Five">]

It works now since the :joins option simply adds LEFT OUTER JOIN to the query without rewriting the entire SELECT statement. Then ActiveRecord combines the join with the select scope, inserting the DISTINCT keyword into the query.

Customizing SearchLogic to use LEFT OUTER JOIN

Ok – now using the same techniques from my last post, let’s see if we can customize SearchLogic to support this sort of named scope. First let’s install SearchLogic into our sample app:

$ script/plugin install http://github.com/binarylogic/searchlogic.git

And next, let’s implement a new named scoped called “without_author” that will work the same way as the “missing_author” scope from above. We’ll use method_missing the same way that SearchLogic does; see my last article for a more complete explanation. Here we go:

class Book < ActiveRecord::Base
  has_one :author
  class << self
    def method_missing(name, *args, &block)
      if name == :without_author
        named_scope :without_author,
          {
            :joins => 'LEFT OUTER JOIN authors
                       ON authors.book_id = books.id',
            :conditions => 'authors.id IS NULL'
          }
          without_author
      else
        super
      end
    end
  end
end

And let’s try it in the console:

>> Book.without_author.collect { |book| book.title }
   Book Load (1.3ms)   SELECT "books".* FROM "books" LEFT OUTER JOIN authors
   ON authors.book_id = books.id WHERE (authors.id IS NULL) 
=> ["Four", "Five"]

Works perfectly! All we need to do now is generalize this for any model and association. Following the pattern from the SearchLogic source code, to get a list of associated models I can call “reflect_on_all_associations.” This will return a list of AssociationReflection objects, each of which represents an association between this model (Book) and some other model (Author). See the ActiveRecord source code for more details.

Here’s what the code looks like with the call to reflect_on_all_associations:

class Book < ActiveRecord::Base
  has_one :author
  class << self
    def method_missing(name, *args, &block)
      association_names =
        reflect_on_all_associations.collect { |assoc| assoc.name }
      if name.to_s =~ /^without_(#{association_names.join("|")})$/
        named_scope :without_author,
          {
            :joins => 'LEFT OUTER JOIN authors
                       ON authors.book_id = books.id',
            :conditions => 'authors.id IS NULL'
          }
          without_author
      else
        super
      end
    end
  end
end

You can see that we construct a regex pattern from a list of associated model names, joined with the | character. For example, if there were 3 associated models, then we would use /without_(assoc1|assoc2|assoc3)/… in this example since Book has only one associated model, we have a simple regex pattern: /without_(author)/.

Let’s make sure the code still works in the console:

>> Book.without_author.collect { |book| book.title }
   Book Load (1.0ms)   SELECT "books".* FROM "books" LEFT OUTER JOIN authors
   ON authors.book_id = books.id WHERE (authors.id IS NULL) 
=> ["Four", "Five"]

So far so good. The next thing we need to do is replace the hard coded values in the call to named_scope. To do this, we’ll need the AssociationReflection object that corresponds to the matching association name. Here’s some code that does that:

def matching_association(match)
  @matching_association ||= reflect_on_all_associations.detect do |assoc|
    assoc.name.to_s == match
  end
end

If we pass in the matching name from the regex pattern above, e.g. matching_association($1), then we’ll get the corresponding association object. Once we have that, we can get the name of that association’s database table and primary key:

associated_table = matching_association($1).table_name
associated_key = matching_association($1).primary_key_name

One subtle point to note here: ActiveRecord's AssociationReflection.primary_key_name method actually returns the foreign key column for this association, the book_id column in the authors table, and not the primary key of the authors table, which would have just been id. It works properly, but possibly should have been named foreign_key_name. Anyway, now we’ll need these values to construct our LEFT OUTER JOIN and condition strings. These methods do that:

def join_clause(associated_table, associated_key)
  outer_join = "LEFT OUTER JOIN #{associated_table}"
  outer_join += " ON #{associated_table}.#{associated_key}"
  outer_join += " = #{quoted_table_name}.#{primary_key}"
end

def where_clause(associated_table, associated_key)
  "#{associated_table}.#{associated_key} IS NULL"
end

Finally, we can put it all together like this:

class Book < ActiveRecord::Base
  has_one :author

  class << self
    def method_missing(name, *args, &block)
      association_names =
        reflect_on_all_associations.collect { |assoc| assoc.name }
      if name.to_s =~ /^without_(#{association_names.join("|")})$/
        associated_table = matching_association($1).table_name
        associated_key = matching_association($1).primary_key_name
        named_scope name,
          {
            :joins => join_clause(associated_table, associated_key),
            :conditions => where_clause(associated_table, associated_key)
          }
        send(name)
      else
        super
      end
    end

    def matching_association(match)
      @matching_association ||= reflect_on_all_associations.detect do |assoc|
        assoc.name.to_s == match
      end
    end

    def join_clause(associated_table, associated_key)
      outer_join = "LEFT OUTER JOIN #{associated_table}"
      outer_join += " ON #{associated_table}.#{associated_key}"
      outer_join += " = #{quoted_table_name}.#{primary_key}"
    end

    def where_clause(associated_table, associated_key)
      "#{associated_table}.#{associated_key} IS NULL"
    end
  end
end

Note that I used “send(name)” to call the named scope we just created with named_scope. Let’s make sure it still all works properly in the console:

>> Book.without_author.collect { |book| book.title }
   Book Load (1.0ms)   SELECT "books".* FROM "books"
   LEFT OUTER JOIN authors ON authors.book_id = "books".id
   WHERE (authors.book_id IS NULL) 
=> ["Four", "Five"]

Phew – it does! Ideally I would have some RSpec examples setup to test this, instead of using the console.

Just like in my last article, the last thing I’ll do today is move these class methods out of the Book model and into a new module called SearchLogicExtensions, which in my application I saved into a file called config/initializers/search_logic_extensions.rb. This causes the method missing code to be loaded when my app starts up. At the bottom I extend ActiveRecord::Base with the new module, so the named scope can be used by every model in my application:

module SearchLogicExtensions
  def method_missing(name, *args, &block)
    association_names =
      reflect_on_all_associations.collect { |assoc| assoc.name }
    if name.to_s =~ /^without_(#{association_names.join("|")})$/
      associated_table = matching_association($1).table_name
      associated_key = matching_association($1).primary_key_name
      named_scope name,
        {
          :joins => join_clause(associated_table, associated_key),
          :conditions => where_clause(associated_table, associated_key)
        }
      send(name)
    else
      super
    end
  end

  def matching_association(match)
    @matching_association ||= reflect_on_all_associations.detect do |assoc|
        assoc.name.to_s == match
    end
  end

  def join_clause(associated_table, associated_key)
    outer_join = "LEFT OUTER JOIN #{associated_table}"
    outer_join += " ON #{associated_table}.#{associated_key}"
    outer_join += " = #{quoted_table_name}.#{primary_key}"
  end

  def where_clause(associated_table, associated_key)
    "#{associated_table}.#{associated_key} IS NULL"
  end
end

ActiveRecord::Base.extend(SearchLogicExtensions)

Using method_missing to customize SearchLogic

2 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

The Ruby interpreter calls method_missing on a Ruby object whenever it receives a message (method call) that it cannot handle. One of the best examples of using method_missing that I’ve come across is in the SearchLogic plugin, which allows you to dynamically create named scopes. Today I’m going to take some time to explain how method_missing works, show how it’s used by SearchLogic, and finally show how you can use method_missing yourself to customize SearchLogic’s behavior.

Simple sorting with SearchLogic

Suppose I have an ActiveRecord model called “book” with a “title” attribute:

$ rails books
$ cd books
$ script/generate model book title:string
$ rake db:migrate
$ script/console
>> ["one", "two", "three"].each { |title| Book.create :title => title }

The best way in Rails to display the books sorted by title would be to use a named scope like this in my model:

class Book < ActiveRecord::Base
  named_scope :sorted_by_title, { :order => 'title' }
end

If I use a trick my colleague Niranjan Sarade showed me, we can see the SQL produced by ActiveRecord for the named scope in the console, like this:

$ script/console
>> ActiveRecord::Base.logger = Logger.new(STDOUT)
>> Book.sorted_by_title.collect { |book| book.title }
  Book Load (1.6ms)   SELECT * FROM "books" ORDER BY title
=> ["one", "three", "two"]

This is a good example of why I’m a Rails developer: with just a single line of code in my model I can sort the values in a database column! But it gets even easier if I install SearchLogic:

$ script/plugin install git://github.com/binarylogic/searchlogic.git

Now I get a whole series of named scopes created for me automatically! For example, I can now just call the “ascend_by_title” and “descend_by_title” named scopes as if I had written them myself:

$ script/console
>> Book.ascend_by_title.collect { |book| book.title }
  Book Load (1.3ms)   SELECT * FROM "books" ORDER BY books.title ASC
=> ["one", "three", "two"]
>> Book.descend_by_title.collect { |book| book.title }
  Book Load (2.0ms)   SELECT * FROM "books" ORDER BY books.title DESC
=> ["two", "three", "one"]

Brilliant! Using SearchLogic I can filter/sort on any attribute of any model in my application without writing a single line of code. I can even sort and filter on attributes of associated models, e.g. if I had “Book has_many :authors,” I could sort books by their author’s names, or sort the authors for each book, etc., all without writing any SQL or even Ruby code.

Sorting with NULL values last

Recently at my day job I came across a business requirement to sort a list of values, always displaying the NULL or empty values at the end of the list. In our example, this would mean that there might be some books with missing titles:

$ script/console
>> 2.times { Book.create :title => nil }

Here’s the behavior I get from the ascend_by_title and descend_by_title named scopes with NULL values:

>> Book.ascend_by_title.collect { |book| book.title }
  Book Load (2.5ms)   SELECT * FROM "books" ORDER BY books.title ASC
=> [nil, nil, "one", "three", "two"]
>> Book.descend_by_title.collect { |book| book.title }
  Book Load (2.7ms)   SELECT * FROM "books" ORDER BY books.title DESC
=> ["two", "three", "one", nil, nil]

In other words, the NULL values are considered to be less than the other values by the database server, and are sorted accordingly. To get the behavior I want, I need to use a slightly more complex sorting pattern in a named scope, like this:

class Book < ActiveRecord::Base
  named_scope :sorted_by_title_nulls_last,
              { :order => 'title IS NULL, title' }
  named_scope :sorted_by_title_nulls_last_desc,
              { :order => 'title IS NULL, title DESC' } 
end

Trying it out in the console:

$ script/console
>> Book.sorted_by_title_nulls_last.collect { |book| book.title }
  Book Load (2.6ms)   SELECT * FROM "books" ORDER BY title IS NULL, title
=> ["one", "three", "two", nil, nil]
>> Book.sorted_by_title_nulls_last_desc.collect { |book| book.title }
  Book Load (2.5ms)   SELECT * FROM "books" ORDER BY title IS NULL, title DESC
=> ["two", "three", "one", nil, nil]

These named scopes first sort on “title IS NULL” and then on the actual title value, causing the NULL values to appear at the end. This code is fairly clean and would work fine – the problem I had at my day job, however, was that I needed this sorting behavior for about six different columns in various database tables. To make this work, I would need to repeat these two named scopes in each model for each attribute that I wanted to sort on. If only SearchLogic had supported this sorting behavior, I wouldn’t need to copy and paste all of the named scopes.

Using method_missing

Specifically, here’s the method that I wished SearchLogic had implemented for me:

>> Book.ascend_by_title_nulls_last
undefined method `ascend_by_title_nulls_last' for #<Class:0x2234978>

As you can see it doesn’t. But I mentioned above that SearchLogic works by using method_missing; let’s see if I can use method_missing myself and implement the NULLs last behavior… in other words, let’s see if I can use metaprogramming to implement the “nulls last” named scopes on all of my model classes all at once!

I’ll start by using the simplest possible implementation of method_missing:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      puts "This method is missing: #{name}"
    end
  end
end

Here “class << self” indicates that method_missing will be a class method on my Book class; Ruby calls method_missing on the class that is missing the method. The code here simply writes out a message when an unknown method is called:

>> Book.ascend_by_title_nulls_last
This method is missing: ascend_by_title_nulls_last
=> nil

Now I’m ready to think about how to implement the nulls last sorting. But not so fast: it turns out that I have just broken my model class! Aside from SearchLogic, ActiveRecord itself also uses method_missing extensively. The simplest examples of this are the “find_by_...” methods. For example, calling find_by_title should return the book record with the given title:

>> Book.find_by_title 'one'
This method is missing: find_by_title

But now instead of Book “one” I just get the debug message from method_missing. The correct solution here is to pass along the method_missing call to the super class, like this:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name == :ascend_by_title_nulls_last
        puts "This method is missing: #{name}"
      else
        super
      end
    end
  end
end

Let’s try find_by_title again in the console:

>> Book.find_by_title 'one'
  Book Load (0.5ms)
  SELECT * FROM "books" WHERE ("books"."title" = 'one') LIMIT 1
=> #<Book id: 1, title: "one", created_at: "2010-06-11 18:39:26", etc...
>> Book.ascend_by_title_nulls_last
This method is missing: ascend_by_title_nulls_last
=> nil

Sigh of relief – it works again! Looking at the if statement above, you can see that I check if the missing method is called “ascend_by_title_nulls_last,” in which case I write the debug message; if any other missing method is called I pass the call along to the super class. In this case, the super class is actually the SearchLogic module; it uses method_missing with super in exactly the same way that I do here. If the missing method is not recognized by SearchLogic, super is called again and finally ActiveRecord receives the method_missing call, which eventually evaluates find_by_title.

How does SearchLogic work?

SearchLogic uses method_missing as follows, the first time a missing method is called on an ActiveRecord model:

Check if the unknown method matches a certain pattern, e.g. “ascend_by_XYZ.”
Call “named_scope” if so, to create the corresponding named scope.
Run the new named scope, to return the desired query results.

If the same missing method is called again, it will no longer be missing since the corresponding named scope will now exist. ActiveRecord caches a list of scopes that are created by calls to named_scope for each model class.

Ok – let’s try this idea on my Book model:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name == :ascend_by_title_nulls_last
        named_scope :ascend_by_title_nulls_last,
                    { :order => 'title IS NULL, title' }
        ascend_by_title_nulls_last
      else
        super
      end
    end
  end
end

In the console again:

>> Book.ascend_by_title_nulls_last.collect { |book| book.title }
  Book Load (1.9ms)   SELECT * FROM "books" ORDER BY title IS NULL, title
=> ["one", "three", "two", nil, nil]

It works! The code above implements SearchLogic’s algorithm: if someone tries to use a named scope called “ascend_by_title_nulls_last” then actually create the scope at that moment with the proper sorting behavior.

Adding custom sorting to SearchLogic

Now I’m ready to generalize this for any model and attribute. First, I’ll look for any missing method name that matches a certain regex pattern (“ascend_by_XYZ_nulls_last”):

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope :ascend_by_title_nulls_last,
                    { :order => 'title IS NULL, title' }
        ascend_by_title_nulls_last
      else
        super
      end
    end
  end
end

The line highlighted above first converts the method name from a symbol to a string, and then matches it against the “nulls_last” syntax I’m looking for. Next, I’m still hard coding “title” in the named_scope call; let’s replace that with the proper value, and also use the name passed into method_missing instead of the hard coded symbol for the scope name:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
        ascend_by_title_nulls_last
      else
        super
      end
    end
  end
end

“$1” returns the string that matched the first expression contained in parentheses in the regex pattern above, “(\w+)” in this case. This will be the name of the attribute between ascend_by… and …nulls_last, taken from the missing method’s name. Now the proper named scope is created using this attribute name in the SQL fragment. So for example, if I call “Book.ascend_by_author_name_nulls_last” a named scope called “ascend_by_author_name_nulls_last” will be created, using :order => “author_name IS NULL, author_name.”

One last hard coded value to remove: the call to “ascend_by_title_nulls_last” still refers to title directly. To fix this, I just need to use “send(name)” – this calls the method whose name is in the “name” string, which is the named scope we just created. Here’s how that looks:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
        send(name)
      else
        super
      end
    end
  end
end

Now I can add in the case for the descending sort as well:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
        send(name)
      elsif name.to_s =~ /^descend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1} DESC" }
        send(name)
      else
        super
      end
    end
  end
end

The last thing I’ll do today is generalize this for any model in my application by moving the method_missing code into a module that I’ll call “SearchLogicExtensions,” and then extending ActiveRecord::Base with that:

module SearchLogicExtensions
  def method_missing(name, *args, &block)
    if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
      named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
      send(name)
    elsif name.to_s =~ /^descend_by_(\w+)_nulls_last$/
      named_scope name, { :order => "#{$1} IS NULL, #{$1} DESC" }
      send(name)
    else
      super
    end
  end
end
ActiveRecord::Base.extend(SearchLogicExtensions)

Note that I need to use ActiveRecord::Base.extend and not ActiveRecord::Base.include here, since my method_missing code calls “super” if the missing method does not match the pattern. “Extend” means that the methods of ActiveRecord::Base, including method_missing, will be overridden by the methods of the SearchLogicExtensions module, but will still be present and available via a call to “super.” Another important detail here is that I removed the “class << self” syntax. Since this is a module and not a class like Book was, I just define method_missing directly. My method_missing will be added as a class method to Book and all of my other models by the last line, when we extend ActiveRecord::Base. In my application I put this code into a file called “config/initializers/search_logic_extensions.rb,” which caused it to be loaded during the Rails initialization process. I could have also packaged the code up as a separate plugin.

That’s it for today; next time I’ll continue this discussion of metaprogramming with SearchLogic by showing how to sort with NULL values in an associated database table, using a LEFT OUTER JOIN query.

How to install Paperclip in a Rails 3 app

3 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

<u>Update June 2010:</u> I just heard from Jon Yurek in the comments below that he has, in fact, finished up the Rails 3 changes for Paperclip. This means that you can now just install Paperclip as usual in a Rails 3 app as a plugin:

$ rails plugin install git://github.com/thoughtbot/paperclip.git

... or as a gem by adding it to your Gemfile if you’ve already installed it with “gem install paperclip:”

source 'http://rubygems.org'

gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'paperclip'

I’ll leave my original article here as a reference – it was a fun learning experience trying out Paperclip with Rails 3, and the same ideas around Bundler, generators, etc., might still be helpful while using other gems or plugins with Rails 3.

To get Paperclip to work in a Rails 3 application, use this in your Gemfile:

gem 'paperclip', :git => 'git://github.com/thoughtbot/paperclip.git',
                 :branch => 'rails3'

… and this in application.rb:

module YourPaperclipApp
  class Application < Rails::Application

    Paperclip::Railtie.insert
etc...

  end
end

Right now it looks like Thoughtbot is finishing Rails 3 related changes in a “rails3” branch in their Paperclip github repository. The best thing to do if you have a Paperclip app you want to migrate to Rails 3 is simply to wait a bit longer for them to finish that work, test it and merge it back into the master branch.

The rest of this article is really not about Paperclip at all, but about Rails 3. Here’s what I learned about Rails 3 while troubleshooting Paperclip:

Let’s go ahead and troubleshoot Paperclip together in a new Rails 3 app. First I’ll begin by verifying the versions of Ruby and Rails I’m using now:

$ ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [i686-darwin9.8.0]
$ rails -v
Rails 3.0.0.beta3

And next I’ll create a sample app to use with Paperclip:

$ rails paperclip-sample-app
      create  
      create  README
      create  .gitignore
      create  Rakefile
      create  config.ru
      create  Gemfile
      create  app
      create  app/controllers/application_controller.rb
etc…

Now we’re ready to install Paperclip into my new app. But what should I do exactly? Should I use Paperclip as a plugin or a gem? I wasn’t sure what to do, so I simply tried both.

Fact 1: the command line has changed

First let’s install it as a plugin, since that’s the most straightforward. In Rails 3 the plugin install command has changed a bit vs. Rails 2.x:

$ cd paperclip-sample-app
$ rails plugin install git://github.com/thoughtbot/paperclip.git
Initialized empty Git repository in .../vendor/plugins/paperclip/.git/
remote: Counting objects: 77, done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 77 (delta 12), reused 20 (delta 0)
Unpacking objects: 100% (77/77), done.
From git://github.com/thoughtbot/paperclip
 * branch            HEAD       -> FETCH_HEAD

Next let’s use scaffolding to create a “User” model with a couple of attributes:

$ rails generate scaffold user name:string email:string
      invoke  active_record
      create    db/migrate/20100521034815_create_users.rb
      create    app/models/user.rb
      invoke    test_unit
      create      test/unit/user_test.rb
      create      test/fixtures/users.yml
       route  resources :users
      invoke  scaffold_controller
      create    app/controllers/users_controller.rb
      invoke    erb
      create      app/views/users
      create      app/views/users/index.html.erb
etc…
$ rake db:migrate

Fact 2: plugin generators have moved

The next step is to create a second migration for the additional database columns required by Paperclip. To make this easy, Paperclip provides a “paperclip” generator; let’s try that and specify that we want an “avatar” file attachment saved on the user model:

$ rails generate paperclip user avatar
Could not find generator paperclip.

Uhh… not what I expected. It looks like something has changed about Rails 3 generators that has broken the Paperclip generator. For now, let’s take a look at the Paperclip code to see if we can find the generator:

$ find vendor/plugins/paperclip -name *generator*
vendor/plugins/paperclip/generators
vendor/plugins/paperclip/generators/paperclip/paperclip_generator.rb

There it is… After some research, I found out that for Rails 3, plugin/gem generators need to be located inside a folder called “BASE_DIR/lib/generators” – we can see here that the Paperclip generator needs to be moved in order to comply with this new standard.

Fact 3: Rails 2.x generators don’t work at all

So let’s try just moving it and see what happens:

mv vendor/plugins/paperclip/generators vendor/plugins/paperclip/lib/.
$ rails generate paperclip user avatar
[WARNING] Could not load generator "generators/paperclip/paperclip_generator"
because it's a Rails 2.x generator, which is not supported anymore.
Error: uninitialized constant Rails::Generator.

Things are looking worse and worse! It turns out that the generators architecture for Rails 3 has been completely rewritten, and that generators written for Rails 2.x will simply not work at all in Rails 3. What to do now? Of course, I could simply hand code the migration for adding the avatar columns to the users table, and continue to work on my sample application. Instead, I decided to give up on the plugin entirely and to try using Paperclip as a gem.

Fact 4: You use Bundler and a “Gemfile” to declare gems

Let’s take a look at how gems are installed for a Rails 3 app. Rails 3 uses a new file called the “Gemfile,” which specifies which gems should be included in your application. This file is read and used by Bundler, which manages gems and their dependencies. We can specify that our application uses the Paperclip gem by adding a single line to the Gemfile like this:

source 'http://rubygems.org'
gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'paperclip'
etc…

This simply tells Bundler to install Paperclip from your default gem source: probably rubygems.org. Now I’ll delete the plugin I installed earlier and install the gem, using the “bundle install” command to install all of the gems in my Gemfile:

$ rm -rf vendor/plugins/paperclip/
$ bundle install
Fetching source index from http://rubygems.org/
Using rake (0.8.7) from system gems 
Using abstract (1.0.0) from bundler gems 

etc…

Installing paperclip (2.3.1.1) from rubygems repository at http://rubygems.org/ 

etc…

Bundler indicated that it found the official version of Paperclip on rubygems.org, downloaded and installed it. It also told us we have version 2.3.1.1. Let’s use the bundle show command and take a look at where Paperclip was installed to:

$ bundle show paperclip
  /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1

Bundler simply installed the gem in the standard location where all my other gems are located for my RVM version of Ruby 1.8.7, just as if I had run a gem install command manually. Now let’s try that generate command again and see if it works any better:

$ rails generate paperclip user avatar
DEPRECATION WARNING: RAILS_ROOT is deprecated! Use Rails.root instead. (called from expand_path at /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39)
/Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39:in `expand_path': can't convert #<class:0x16a87d0> into String (TypeError)
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:46:in `require'
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:46:in `require'
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:41:in `each'

Still broken! It looks like I’m just not running code that was intended to be used with Rails 3.

Fact 5: You can install a gem from a specific git repository branch

After more investigation, I noticed that there had been a lot of recent changes to the Paperclip github repository. At the time I wrote this, Thoughtbot was actively developing on a branch called “rails3.” I decided the best thing to do would be to try the code from the rails3 branch, hoping it might work better for me. Bundler makes this easy, since you can just specify git as a source for downloading a gem using a “git” option, as well as optionally a specific branch using a “branch” option, like this:

source 'http://rubygems.org'

gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'paperclip', :git => 'git://github.com/thoughtbot/paperclip.git',
                 :branch => 'rails3'
etc …

After saving this change to Gemfile, let’s re-run the bundle install command:

$ bundle install
Updating git://github.com/thoughtbot/paperclip.git
Fetching source index from http://rubygems.org/
Updating git://github.com/thoughtbot/paperclip.git
Using rake (0.8.7) from system gems 
Using abstract (1.0.0) from bundler gems 

…etc…

Using paperclip (2.3.2.beta1)
  from git://github.com/thoughtbot/paperclip.git (at rails3)

Hmm… interesting. Bundler is showing that it’s downloaded Paperclip from the github repository, and that it got the code at the head of the rails3 branch. Another interesting detail here is that I apparently now have version “2.3.2 beta1” of Paperclip. This is a good sign, since I have a more recent version than 2.3.1.1 (the rubygems.org version) and also it seems that Thoughtbot is actively working on it since it’s labeled “beta1.”

If we run bundle show again, we can see that Bundler has saved a special copy of Paperclip downloaded from github, along with the git commit id and branch of the version I have:

$ bundle show paperclip
      /Users/pat/.rvm/gems/ruby-1.8.7-p249/bundler/gems/paperclip-61f74de14812cabc026967a2b2c3ca8cbd2eed69-rails3

Now let’s try that generator once more:

$ rails generate paperclip user avatar
            create  db/migrate/20100521003113_add_attachment_avatar_to_user.rb

Yes! It’s working now!

Fact 6: Rails 3 frameworks are now based on Rails::Railtie

Let’s continue to put together my sample application by running the migration:

$ rake db:migrate

… and by editing my User model to call “has_attached_file:”

class User < ActiveRecord::Base
  has_attached_file :avatar
end

Before I start editing my views and adding the code to upload and display the avatar attachment, let’s start the server and see if Paperclip is working. Opening the users index page I get…

… more trouble! I’m definitely having a bad day… what now? Well it seems that Paperclip is just not being loaded at all, or is being initialized improperly for some reason. At this point I started to poke around the Paperclip source code a bit, and found that the code that includes the Paperclip module into ActiveRecord::Base was moved and is no longer being called. Since Paperclip is not included in my User/ActiveRecord class I get the error has_attached_file not defined, since that’s defined by Paperclip.

I found the include code in a file called “lib/paperclip/railtie.rb:”

require 'paperclip'

module Paperclip
  if defined? Rails::Railtie
    require 'rails'
    class Railtie < Rails::Railtie
      config.after_initialize do
        Paperclip::Railtie.insert
      end
    end
  end

  class Railtie
    def self.insert
      ActiveRecord::Base.send(:include, Paperclip)
      File.send(:include, Paperclip::Upfile)
    end
  end
end

I’m not quite sure what Thoughtbot’s plans are for Paperclip, but if you take some time to read through Yehuda Katz’s write up Rails and Merb Merge: Rails Core (Part 4 of 6), you’ll learn about how Rails frameworks like ActiveRecord and ActiveController have been recast as instances of this “Rails::Railtie” class. Possibly Paperclip will become one of these. Rails 3 has a new API for declaring how Railties are loaded and initialized, but it looks like this version of Paperclip and this version of Rails aren’t quite working correctly now.

Fact 7: Bundler does not call rails/init.rb in each gem

For now, the problem I’m having in my sample application is that the Paperclip::Railtie.insert method is not being called – the two lines I highlighted above need to be executed in order to enable “has_attached_file” to be present as a class method for ActiveRecord models. To make things more interesting, Thoughtbot did include a call to insert inside rails/init.rb, like this:

require 'paperclip/railtie'
Paperclip::Railtie.insert

… but for Rails 3, it turns out that Bundler no longer calls rails/init.rb.

Moving this line instead to config/application.rb will solve the problem:

module PaperclipSampleApp
  class Application < Rails::Application

    Paperclip::Railtie.insert

    etc…
    
  end
end

Alternatively, you could just create a file called “config/initializers/paperclip.rb” and put the call to insert there.

Now reloading the users index page we finally get Paperclip to work:

Instead of proceeding with my sample app now, I’m going to wait a few weeks while Thoughtbot finishes off the Rails 3 changes for Paperclip.

I don’t think troubleshooting these problems was a waste of time at all; in fact, it was a good excuse to get my hands dirty with Rails 3 and Bundler. Once Thoughtbot has finished their changes in the rails3 branch and merged them into the master I’ll update my tutorial from last year, and also update my Paperclip fork to support database BLOB storage for Rails 3.

How to install Paperclip in a Rails 3 app

3 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

$ rails plugin install git://github.com/thoughtbot/paperclip.git

... or as a gem by adding it to your Gemfile if you’ve already installed it with “gem install paperclip:”

source 'http://rubygems.org'

gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'paperclip'

To get Paperclip to work in a Rails 3 application, use this in your Gemfile:

gem 'paperclip', :git => 'git://github.com/thoughtbot/paperclip.git',
                 :branch => 'rails3'

… and this in application.rb:

module YourPaperclipApp
  class Application < Rails::Application

    Paperclip::Railtie.insert
etc...

  end
end

The rest of this article is really not about Paperclip at all, but about Rails 3. Here’s what I learned about Rails 3 while troubleshooting Paperclip:

Let’s go ahead and troubleshoot Paperclip together in a new Rails 3 app. First I’ll begin by verifying the versions of Ruby and Rails I’m using now:

$ ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [i686-darwin9.8.0]
$ rails -v
Rails 3.0.0.beta3

And next I’ll create a sample app to use with Paperclip:

$ rails paperclip-sample-app
      create  
      create  README
      create  .gitignore
      create  Rakefile
      create  config.ru
      create  Gemfile
      create  app
      create  app/controllers/application_controller.rb
etc…

Now we’re ready to install Paperclip into my new app. But what should I do exactly? Should I use Paperclip as a plugin or a gem? I wasn’t sure what to do, so I simply tried both.

Fact 1: the command line has changed

First let’s install it as a plugin, since that’s the most straightforward. In Rails 3 the plugin install command has changed a bit vs. Rails 2.x:

$ cd paperclip-sample-app
$ rails plugin install git://github.com/thoughtbot/paperclip.git
Initialized empty Git repository in .../vendor/plugins/paperclip/.git/
remote: Counting objects: 77, done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 77 (delta 12), reused 20 (delta 0)
Unpacking objects: 100% (77/77), done.
From git://github.com/thoughtbot/paperclip
 * branch            HEAD       -> FETCH_HEAD

Next let’s use scaffolding to create a “User” model with a couple of attributes:

$ rails generate scaffold user name:string email:string
      invoke  active_record
      create    db/migrate/20100521034815_create_users.rb
      create    app/models/user.rb
      invoke    test_unit
      create      test/unit/user_test.rb
      create      test/fixtures/users.yml
       route  resources :users
      invoke  scaffold_controller
      create    app/controllers/users_controller.rb
      invoke    erb
      create      app/views/users
      create      app/views/users/index.html.erb
etc…
$ rake db:migrate

Fact 2: plugin generators have moved

$ rails generate paperclip user avatar
Could not find generator paperclip.

$ find vendor/plugins/paperclip -name *generator*
vendor/plugins/paperclip/generators
vendor/plugins/paperclip/generators/paperclip/paperclip_generator.rb

Fact 3: Rails 2.x generators don’t work at all

So let’s try just moving it and see what happens:

mv vendor/plugins/paperclip/generators vendor/plugins/paperclip/lib/.
$ rails generate paperclip user avatar
[WARNING] Could not load generator "generators/paperclip/paperclip_generator"
because it's a Rails 2.x generator, which is not supported anymore.
Error: uninitialized constant Rails::Generator.

Fact 4: You use Bundler and a “Gemfile” to declare gems

source 'http://rubygems.org'
gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'paperclip'
etc…

$ rm -rf vendor/plugins/paperclip/
$ bundle install
Fetching source index from http://rubygems.org/
Using rake (0.8.7) from system gems 
Using abstract (1.0.0) from bundler gems 

etc…

Installing paperclip (2.3.1.1) from rubygems repository at http://rubygems.org/ 

etc…

$ bundle show paperclip
  /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1

$ rails generate paperclip user avatar
DEPRECATION WARNING: RAILS_ROOT is deprecated! Use Rails.root instead. (called from expand_path at /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39)
/Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39:in `expand_path': can't convert #<class:0x16a87d0> into String (TypeError)
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:46:in `require'
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:46:in `require'
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:41:in `each'

Still broken! It looks like I’m just not running code that was intended to be used with Rails 3.

Fact 5: You can install a gem from a specific git repository branch

source 'http://rubygems.org'

gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'paperclip', :git => 'git://github.com/thoughtbot/paperclip.git',
                 :branch => 'rails3'
etc …

After saving this change to Gemfile, let’s re-run the bundle install command:

$ bundle install
Updating git://github.com/thoughtbot/paperclip.git
Fetching source index from http://rubygems.org/
Updating git://github.com/thoughtbot/paperclip.git
Using rake (0.8.7) from system gems 
Using abstract (1.0.0) from bundler gems 

…etc…

Using paperclip (2.3.2.beta1)
  from git://github.com/thoughtbot/paperclip.git (at rails3)

If we run bundle show again, we can see that Bundler has saved a special copy of Paperclip downloaded from github, along with the git commit id and branch of the version I have:

$ bundle show paperclip
      /Users/pat/.rvm/gems/ruby-1.8.7-p249/bundler/gems/paperclip-61f74de14812cabc026967a2b2c3ca8cbd2eed69-rails3

Now let’s try that generator once more:

$ rails generate paperclip user avatar
            create  db/migrate/20100521003113_add_attachment_avatar_to_user.rb

Yes! It’s working now!

Fact 6: Rails 3 frameworks are now based on Rails::Railtie

Let’s continue to put together my sample application by running the migration:

$ rake db:migrate

… and by editing my User model to call “has_attached_file:”

class User < ActiveRecord::Base
  has_attached_file :avatar
end

Before I start editing my views and adding the code to upload and display the avatar attachment, let’s start the server and see if Paperclip is working. Opening the users index page I get…

I found the include code in a file called “lib/paperclip/railtie.rb:”

require 'paperclip'

module Paperclip
  if defined? Rails::Railtie
    require 'rails'
    class Railtie < Rails::Railtie
      config.after_initialize do
        Paperclip::Railtie.insert
      end
    end
  end

  class Railtie
    def self.insert
      ActiveRecord::Base.send(:include, Paperclip)
      File.send(:include, Paperclip::Upfile)
    end
  end
end

Fact 7: Bundler does not call rails/init.rb in each gem

require 'paperclip/railtie'
Paperclip::Railtie.insert

… but for Rails 3, it turns out that Bundler no longer calls rails/init.rb.

Moving this line instead to config/application.rb will solve the problem:

module PaperclipSampleApp
  class Application < Rails::Application

    Paperclip::Railtie.insert

    etc…
    
  end
end

Alternatively, you could just create a file called “config/initializers/paperclip.rb” and put the call to insert there.

Now reloading the users index page we finally get Paperclip to work:

Instead of proceeding with my sample app now, I’m going to wait a few weeks while Thoughtbot finishes off the Rails 3 changes for Paperclip.

How to create a slideshow using JQuery and Rails in 10 minutes or less

4 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

Today I want to demonstrate a JQuery slideshow tool that my friend Daniel Higginbotham wrote: Electric Slide. It’s tremendously simple to use, while still providing ways to customize and adapt its behavior as you need to.

Suppose you have a long series of slides containing text and/or images that you want to display as a slideshow on a web page; let’s use these Mickey images as an example:

You might represent these slides using a Rails ActiveRecord model called “Slide.” If you had one image per slide you could attach it to the slide model with Paperclip:

class Slide < ActiveRecord::Base
  has_attached_file :image
end

The simplest way to display all of these would be to draw a single vertical column on a web page and let the user scroll down to view all of the slides:

<% @slides.each do |slide| %>
  <%= image_tag slide.image.url %>
  <br/>
<% end %>

However, scrolling can be annoying especially if there are many images or a lot of text. Also, this isn’t a slideshow. I don’t see each image in the same location as the previous one. It’s harder to notice changes between the slides and also harder for me to surprise the user with something funny or unexpected in the following slide since they are all immediately visible.

Another simple solution would be to display each slide on a separate page and provide next/previous links, using code similar to this:

<%= image_tag @slide.image.url %>
<br>
<%= link_to 'Prev', @slide.higher_item unless @slide.first? %> &nbsp;
<%= link_to 'Next', @slide.lower_item unless @slide.last? %>

But this is also annoying since each click takes the user to a separate page, changing the URL and also adding to the browser’s history list. Since all of these images are part of a single presentation I’d like to display them on one page. Additionally I have to write all of the navigation code: I might use acts_as_list in my model class like in the ERB snippet above, and then write the HTML to display the next/prev links in the proper location, to hide them when necessary, to give them the proper styling, etc. In other words, it’s a fair amount of work.

Electric Slide

Electric Slide to the rescue! If you install Daniel’s Electric Slide JQuery code file in your app, all you have to do is use a <div class=“slide”> tag around each image or whatever slide content you have, call a JQuery function when your page is loaded and Electric Slide will take care of the rest:

<script type="text/javascript" charset="utf-8">
  $(function(){
    $("#slides").electricSlide();
  })
</script>
<div id="slides">
<% @slides.each do |slide| %>
  <div class="slide">
    <%= image_tag slide.image.url %>
  </div>
<% end %>
</div>

With just a little bit of CSS love, the slideshow can look like this (click through the screen shot to see a working example):

body {
  background:#EEEEDD;
}
.slide {
  display:none;
}
#slides .slide-header {
  display: none
}
#slides .slide-footer {
  margin:0;
  width: 200px;
}
.slide-footer .previous {
  width:75px;
  float:left;
}
.slide-footer .next {
  width:75px;
  float:right;
  text-align:right;
}

Now all the slides are displayed in a working slideshow inside my single web page! When I click on the previous or next links, I can watch Mickey move around without leaving the page. Let’s take a look at how my ERB and CSS code works and what Electric Slide is doing:

First I’ve identified each of my slide’s content using <div class=“slide”>. I just included that in my iteration over the slide array. For me, each of these <div> tags contains a Mickey image, but they could contain any HTML content you would like.
Note I set “display: none” for the “.slide” style so the slides are initially hidden. This is required by Electric Slide, or else all of the slides will appear initially visible.
Next I added an on-load handler using JQuery syntax: $(function(){ …. }). JQuery will call this code when the page finishes loading.
In the on-load handler, I call the electricSlide function on my <div id=“slides”> parent object. This initializes Electric Slide. There are many options I could have passed in here, but today I’m just using all of the default values. Check out Daniel’s github repo for documentation and examples for more info on what to pass in here.
Electric Slide counts up the slides, and displays “previous” and “next” links for each one as necessary. I don’t need to worry about writing code to display the links or handle clicks on them: the navigation just works.
Electric Slide also creates <div> tags automatically to hold the previous/next links, these are called <div class=“slide-header”> and <div class=“slide-footer”>. I’ve styled these using CSS to hide the header and to align the links with the edge of the image in the footer.

Detailed step-by-step example

Let’s create a new Rails app from scratch, copy the mickey images into it, install Electric Slide and then display them all as a slideshow… all in 10 minutes or less!

$ rails mickey-slides
$ cd mickey-slides

Go ahead and download the Mickey images from my site; or feel free to use any images you have instead:

$ curl -O http://patshaughnessy.net/assets/2010/4/28/mickey-images.tar.gz
$ tar zxvf mickey-images.tar.gz 
images/
images/mickey1.jpg
images/mickey2.jpg
images/mickey3.jpg
images/mickey4.jpg
images/mickey5.jpg
images/mickey6.jpg

Now let’s use View Mapper to create my slide model with an “image” attachment:

$ sudo gem install view_mapper
$ ./script/generate scaffold_for_view slide --view paperclip:image
       error  The Paperclip plugin does not appear to be installed.

Oh yea… I forgot to install Paperclip; let’s do that now also and then repeat the View Mapper command:

$ ./script/plugin install git://github.com/thoughtbot/paperclip.git
$ ./script/generate scaffold_for_view slide --view paperclip:image
$ rake db:migrate

Next let’s load the images into our database using Paperclip in the Rails console:

$ ./script/console 
Loading development environment (Rails 2.3.5)
>> Dir.glob('images/*.jpg').each do |filename|
?> Slide.create :image => File.new(filename)
>> end
=> ["images/mickey1.jpg", "images/mickey2.jpg", "images/mickey3.jpg",
"images/mickey4.jpg", "images/mickey5.jpg", "images/mickey6.jpg"]

Now we just need to install Electric Slide – let’s just get Daniel’s entire github repo including the examples and documentation:

$ cd ..
$ git clone git://github.com/flyingmachine/electric-slide.git
$ cd mickey-slides

And now let’s replace our default Rails prototype javascript files with JQuery and the Electric Slide code from Daniel’s repository:

$ rm -rf public/javascripts
$ cp -r ../electric-slide/javascripts public/javascripts

Now edit app/views/layouts/slides.html.erb and include Electric Slide, JQuery and a new style.css file by replacing the existing stylesheet_link_tag ‘scaffold’ line with the lines highlighted below:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="content-type" content="text/html;charset=UTF-8" />
  <title>Slides: <%= controller.action_name %></title>
  <%= stylesheet_link_tag 'scaffold', 'styles' %>
  <%= javascript_include_tag 'jquery', 'jquery.sizes', 'jquery.electric-slide', 'jquery-ui' %>
</head>

<body>

<p style="color: green"><%= flash[:notice] %></p>

<%= yield %>

</body>
</html>

Now we can add the CSS code snippet from above into our app – copy this into a new file called public/stylesheets/styles.css, which we just included above in the layout:

body {
  background:#EEEEDD;
}
.slide {
  display:none;
}
#slides .slide-header {
  display: none
}
#slides .slide-footer {
  margin:0;
  width: 200px;
}
.slide-footer .previous {
  width:75px;
  float:left;
}
.slide-footer .next {
  width:75px;
  float:right;
  text-align:right;
}

And finally replace the scaffolding slide index view, app/views/slides/index.html.erb, with the code from above:

<script type="text/javascript" charset="utf-8">
  $(function(){
      $("#slides").electricSlide();
  })
</script>
<div id="slides">
<% @slides.each do |slide| %>
  <div class="slide">
    <%= image_tag slide.image.url %>
  </div>
<% end %>
</div>

Now start up your server at look for the Mickey slide show at http://localhost:3000/slides!

Creating associations to existing data part 3: has_many :through scaffolding

5 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

I just updated View Mapper to support scaffolding for models in a has_many, :through relationship. It generates a complex form that is a combination of the “belongs_to” scaffolding from part 1 of this series and the nested attributes scaffolding I wrote about in November:

Based on the programmer/assignment/project example from the ActiveRecord documentation page, this form will create a new programmer record and allow the user to add one or more assignments, each of which also has a name text field. For each new assignment the user can also select an existing project record. Here’s the Programmer model with the has_many :through association:

class Programmer < ActiveRecord::Base
  has_many :projects, :through => :assignments
  has_many :assignments
  accepts_nested_attributes_for :assignments,
                                :allow_destroy => true,
                                :reject_if => proc { |attrs|
                                  attrs['name'].blank? &&
                                  attrs['project_id'].blank?
                                }
end

This implements a many-many relationship between programmers and projects; the assignments model is used to map the projects with the programmers. I’ve also specified that the programmer model accepts_nested_attributes_for assignments… more on that below.

You can now use the “view_for” generator from View Mapper to generate the form above for your models using a new view called “has_many_existing:”

$ sudo gem install view_mapper
$ script/generate view_for programmer --view has_many_existing:projects

Assumptions and requirements

<u>Nested Attributes:</u> the form above works by using ActiveRecord’s nested attributes feature to save multiple assignments for a single programmer. Therefore, you need to be sure you call accepts_nested_attributes_for in your target model; if you forget to do this, you’ll get an error from View Mapper:

$ script/generate view_for programmer --view has_many_existing:projects
    warning  Model Programmer does not accept nested attributes
    for model Assignment.

To fix this problem you can use the code I showed above:

class Programmer < ActiveRecord::Base
   has_many :projects, :through => :assignments
   has_many :assignments
   accepts_nested_attributes_for :assignments,
                                 :allow_destroy => true,
                                 :reject_if => proc { |attrs|
                                   attrs['name'].blank? &&
                                   attrs['project_id'].blank?
                                 }
 end

The options I’ve specified here tell ActiveRecord it is allowed to delete assignment records (when the user clicks “remove” in the form) and to avoid creating empty assignment records if all of their attributes are blank (if the user clicked “Add Assignment” an extra time).

Or if you prefer you can generate the entire target model including the nested attributes call using the scaffold_for_view generator like this – specify the new model’s columns using the same syntax as the standard Rails scaffold generator:

$ script/generate scaffold_for_view programmer
                  first_name:string last_name:string
                  --view has_many_existing:projects

It’s easy to overlook one very elegant detail here about ActiveRecord’s nested attribtues feature: note that “project_id” is one of the nested attributes, generated by each of the project select list boxes. (They are implemented with collection_select; see part 1 of this series). Now when the new programmer form is submitted all of the associations for each assignment – and for the new programmer – are setup. In other words, after you save the new programmer record this way you can immediately access the associated projects through assignments: “programmer.projects” – very cool! And it's all seamless: I don't have to write any code in my controller to associate the projects or assignments with the new programmer.

<u>Correct associations among your models:</u> if you forget to put the proper associations in your three models the has_many :through behavior will not work. You need to have six associations setup among your three models like this:

class Programmer < ActiveRecord::Base
  has_many :projects, :through => :assignments
  has_many :assignments
end

class Project < ActiveRecord::Base
  has_many :programmers, :through => :assignments
  has_many :assignments
end

class Assignment < ActiveRecord::Base
  belongs_to :project
  belongs_to :programmer
end

View Mapper will help you out by displaying an error message if you’re missing one of these:

$ script/generate scaffold_for_view programmer name:string
                  --view has_many_existing:projects
   warning  Model Project does not contain a has_many association for Assignment.

…or if you’re missing one of the corresponding foreign key columns in the “through” model:

$ script/generate scaffold_for_view programmer name:string
                  --view has_many_existing:projects
    warning  Model Assignment does not contain a foreign key for Programmer.

<u>Has many existing model identified by name attribute:</u> In the form above, the Project records were identified in the select list boxes using their “name” attribute. Therefore, you need to insure that your existing model has a name column or method; if it does not View Mapper will display an error message like this:

$ script/generate scaffold_for_view programmer name:string
                  --view has_many_existing:projects
    warning  Model Project does not have a name attribute.

To fix this problem, add a “name” method to your existing model, or else you can specify that View Mapper use a different attribute (e.g. “code”) instead with this syntax:

$ script/generate scaffold_for_view programmer name:string
                  --view has_many_existing:projects[code]

<u>Associated model name method in through model:</u> The last requirement is that the through model, Assignment in this example, have a method (“project_name”) to display the name of its associated existing model. View Mapper requires this to avoid putting this code into the view:

 class Assignment < ActiveRecord::Base
   belongs_to :project
   belongs_to :programmer
   def project_name
     project.name if project
   end
 end

If you forget this method, View Mapper will remind you with this error message:

$ script/generate scaffold_for_view programmer name:string
                  --view has_many_existing:projects[code]
     warning  Model Assignment does not have a method project_code.

Detailed Example

Here’s a step by step example of how to create a Rails application from scratch that contains the has_many :through scaffolding:

$ rails hmt_example

Here’s our model to hold the existing data:

$ cd hmt_example
$ script/generate model project code:string

And the through model to associate projects with programmers; note I’ve included integer attributes as the foreign keys for both the existing model and the new model:

$ script/generate model assignment name:string
                        project_id:integer programmer_id:integer
$ rake db:migrate

Next edit the new models and enter the required associations along with the project_code method:

class Project < ActiveRecord::Base
  has_many :programmers, :through => :assignments
  has_many :assignments
end

class Assignment < ActiveRecord::Base
  belongs_to :programmer
  belongs_to :project
  def project_code
    project.code if project
  end
end

Now we’re ready to create the programmer has_many :through scaffolding; note I’ve specified “code” as the attribute to use to identify each project:

$ sudo gem install view_mapper
$ script/generate scaffold_for_view programmer
                  first_name:string last_name:string
                  --view has_many_existing:projects[code]
$ rake db:migrate

Note this won’t work yet for a has_and_belongs_to_many association; dealing with that is next on my View Mapper to do list.

Class Factory: Factory_girl-like syntax for dynamically creating Ruby classes

5 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

Class Factory will dynamically create classes using a factories model similar to factory_girl. But instead of passing a block with model attributes into the factory definition, you pass in a migration defining the attributes of the new model class you want to create:

ClassFactory.define :person do |p|
  p.string  :first_name
  p.string  :last_name
  p.integer :age
end

Now when you need a “Person” class in your tests you create one like this:

ClassFactory :person
=> Person(id: integer, first_name: string, last_name: string, age: integer)

This can be useful if you're writing tests for a gem or plugin and don't want to load the entire Rails environment, or have access to existing models in a target application. By default Class Factory creates ActiveRecord model classes, but using the :super option you can create any sort of Ruby class. Class Factory also makes it easy for each of your tests to use a different variation on a target class. For example, this will delete the Person model we created above, and create a new Person model that belongs to a group:

ClassFactory :person, :class_eval => 'belongs_to :group' do |p|
  p.string  :first_name
  p.string  :middle_name
  p.string  :last_name
  p.string  :group_id
end
=> Person(id: integer, first_name: string, middle_name: string, last_name: string,
   group_id: string)

Creating different variations of the same class can be useful if you're writing tests for a generator, plugin or some other code which has different behavior depending on what classes you run it against.

Options

Default: create a new ActiveRecord model, along with a corresponding table in your database:

ClassFactory :person

Execute a migration on the new table specified as a block, defining the attributes of the new model class:

ClassFactory :person do |p|
  p.string  :first_name
  p.string  :last_name
  p.integer :age
end

Create a class with a specified superclass (default is ActiveRecord::Base):

ClassFactory :person_array, :super => Array

If the super class is not a subclass of ActiveRecord::Base then Class Factory won't create a table or run a migration. You can use this to create plain Ruby object classes.

Create a class called “DifferentClass” instead of “Person:”

ClassFactory :person, :class => 'DifferentClass'

Run the given code inside the new class using class_eval:

ClassFactory :person, :class_eval => 'has_many :shoes'

Create a table with the given name, instead of a table called “people:”

ClassFactory :person, :class_eval => 'set_table_name :table_name',
             :table => 'table_name'

If you provide options when the factory is defined they will be applied to each class created with the factory. You can also provide options when you create a class, in which case they will override the factory options.

Install

gem install class_factory

Code

http://github.com/patshaughnessy/class_factory

Detailed Example

Start an irb session and require class_factory (this will also require active_record):

$ irb
> require 'rubygems'
=> true
> require 'class_factory'
=> true

Create an in-memory SQLite test database:

> ActiveRecord::Base.establish_connection({ :adapter => 'sqlite3',
  :database => ':memory:' })
=> #<ActiveRecord::ConnectionAdapters::ConnectionPool:0x19cfecc...

Define a person class factory, and create a Person class:

> ClassFactory.define :person do |p|
>   p.string  :first_name
>   p.string  :last_name
>   p.integer :age
> end
=> #<ClassFactory:0x19c6fac @definition={:name=>:person, :migration=>...
> ClassFactory :person
=> Person(id: integer, first_name: string, last_name: string, age: integer)

Now create an instance of a Person and count how many records we have in our test database:

> Person.create :first_name => 'Barack', :last_name => 'Obama', :age => 48
=> #<Person id: 1, first_name: "Barack", last_name: "Obama", age: 48>
> Person.count
=> 1

Next redefine the Person class and override the options set in the factory above; this time it will belong to a group. Note that the existing people table will be dropped and a new, empty people table created:

> ClassFactory :person, :class_eval => 'belongs_to :group' do |p|
>   p.string  :first_name
>   p.string  :middle_name
>   p.string  :last_name
>   p.string  :group_id
> end
=> Person(id: integer, first_name: string, middle_name: string, last_name: string,
   group_id: string)
> Person.count
=> 0

Create a group class that has many people:

> ClassFactory.define :group, :class_eval => 'has_many :people' do |g|
>   g.string :name
> end
=> #<ClassFactory:0x18a05d8 @definition={:class_eval=>"has_many :people", ...
> ClassFactory :group
=> Group(id: integer, name: string)

Finally, recreate the Barack person record and add him to the “presidents” group:

> g = Group.create :name => 'Presidents'
=> #<Group id: 1, name: "Presidents">
> p = Person.new :first_name => 'Barack', :last_name => 'Obama', :age => 48,
  :group => g
=> #<Person id: nil, first_name: "Barack", last_name: "Obama", group_id: 1,
            age: 48>
> p.save
=> true
> g.people
=> [#<Person id: 1, first_name: "Barack", last_name: "Obama", group_id: "1",
   age: 48>]

Getting started with Ruby metaprogramming

6 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

The Rails auto_complete plugin was my first exposure to Ruby metaprogramming. It’s code was simple enough for a Rails beginner like me to understand, but also just complex enough for me to learn something new. Specifically, I ran into metaprogramming when I took a close look at the “auto_complete_for” method and tried to figure out how it worked. I won’t spend any time here explaining what the auto_complete plugin does or what it’s used for, beyond to say that if you add this line to one of your controllers:

class CategoriesController < ApplicationController

  auto_complete_for :category, :name

  etc...

… a method called “auto_complete_for_category_name” will be automatically generated in that controller that will return a list of category records that have a name matching a given search query. This is very cool, and is a typical example of Ruby on Rails magic: you add one line to a class in your application and suddenly an entire feature or behavior is added, customized to the data and objects in your app!

This sort of thing is really what makes Ruby on Rails so amazing… but how does it work? Let’s take a look at the implementation of auto_complete_for method:

def auto_complete_for(object, method, options = {})
  define_method("auto_complete_for_#{object}_#{method}") do
    find_options = { 
      :conditions => [
                       "LOWER(#{method}) LIKE ?",
                       '%' + params[object][method].downcase + '%'
                     ],
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)

    @items = object.to_s.camelize.constantize.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, '#{method}' %>"

  end
end

You can find this code in the lib/auto_complete.rb file inside the rails/auto_complete repository on github. So what the heck does all of this mean? Let’s take a step-by-step look at this code, and see if we can figure it out.

To get started, let’s use the category/name example I used in my last article, and also that Ryan Bates used in his Auto-Complete Association screencast on the auto_complete plugin:

auto_complete_for :category, :name

Here we are passing “:category” into auto_complete_for as the value for “object,” and “:name” as the value for “method.” Now let’s repeat the auto_complete_for code, but substitute object with :category:

def auto_complete_for(:category, method, options = {})
  define_method("auto_complete_for_#{:category}_#{method}") do
    find_options = { 
      :conditions => [
                       "LOWER(#{method}) LIKE ?",
                       '%' + params[:category][method].downcase + '%'
                     ],
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)

    @items = :category.to_s.camelize.constantize.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, '#{method}' %>"

  end
end

In the code snippet above I’ve highlighted the places where the symbol :category appears. You can see that it’s used in a few different places, but the most important line is near the bottom: @items = :category.to_s.camelize.constantize… etc. Let’s evaluate each of the method calls on this one line, one at a time. The first call is “:category.to_s”. The “to_s” method name means “to string” and will convert the target object (the object you call to_s on) to a string. This means that the :category symbol will be converted to a string. So now let’s display the string ‘category’ in place and see what we are left with:

def auto_complete_for(:category, method, options = {})
  define_method("auto_complete_for_#{:category}_#{method}") do
    find_options = { 
      :conditions => [
                       "LOWER(#{method}) LIKE ?",
                       '%' + params[:category][method].downcase + '%'
                     ],
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)

    @items = 'category'.camelize.constantize.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, '#{method}' %>"

  end
end

You can see ‘category’ highlighted above where I’ve evaluated the to_s method. Now the next method call is “camelize” – what does this mean? The camelize method is one of a series of functions that Rails provides in the “ActiveSupport” gem, one of the components of the Rails framework. It converts the given string to camel case, for example “office_code” to “OfficeCode.” Since Ruby class names are written using camel case, this is often very useful for obtaining a class name from a string. In our example, the string “category” is converted into “Category” with an upper case “C” …

def auto_complete_for(:category, method, options = {})
  define_method("auto_complete_for_#{:category}_#{method}") do
    find_options = { 
      :conditions => [
                       "LOWER(#{method}) LIKE ?",
                       '%' + params[:category][method].downcase + '%'
                     ],
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)

    @items = 'Category'.constantize.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, '#{method}' %>"

  end
end

Now let’s take a look at the next method call on that same line of code: “constantize.” This converts a string into an actual Ruby constant, and returns an error if that constant doesn’t exist. In our example, the string “Category” is converted into the Ruby class Category:

def auto_complete_for(:category, method, options = {})
  define_method("auto_complete_for_#{:category}_#{method}") do
    find_options = { 
      :conditions => [
                       "LOWER(#{method}) LIKE ?",
                       '%' + params[:category][method].downcase + '%'
                     ],
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)

    @items = Category.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, '#{method}' %>"

  end
end

Now we can see that the complex line above evaluates to a simple ActiveRecord find call. The power of Ruby metaprogramming has enabled us to write a single helper function “auto_complete_for” that takes any symbol or string as an argument (e.g. :category), and performs a SQL query on the corresponding ActiveRecord class. The amazing part of this for me is how flexible and easy to use Ruby is: helper methods like camelize and constantize make it very easy to extract common bits of code you might write over and over again in your app, and generalize them into a single method that will apply to any target class. This would be possible in any programming language but it’s amazing just how easy it is to do with Ruby.

Let’s continue to simplify the auto_complete_for code by substituting a value for the “method” parameter – in our example method will become the symbol “:name” :

def auto_complete_for(:category, :name, options = {})
  define_method("auto_complete_for_#{:category}_#{:name}") do
    find_options = { 
      :conditions => [
                       "LOWER(#{:name}) LIKE ?",
                       '%' + params[:category][:name].downcase + '%'
                     ],
      :order => "#{:name} ASC",
      :limit => 10 }.merge!(options)

    @items = Category.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, '#{:name}' %>"

  end
end

Again you can see that the :name symbol is used in a few different places. Most commonly here the symbol is inserted in a string using this syntax: “#{:name}”. This is the standard Ruby #{} string interpolation operator, which is also implicitly converting the symbol into a string before inserting it into the surrounding string value, just as if we called to_s as above. So let’s replace “#{:name}” and “#{:category}” with the strings “name” and “category,” and then also insert them into the surrounding string values:

def auto_complete_for(:category, :name, options = {})
  define_method("auto_complete_for_category_name") do
    find_options = { 
      :conditions => [
                       "LOWER(name) LIKE ?",
                       '%' + params[:category][:name].downcase + '%'
                     ],
      :order => "name ASC",
      :limit => 10 }.merge!(options)

    @items = Category.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, 'name' %>"

  end
end

And now let’s replace the last parameter to auto_complete_for “options” with it’s default value since we aren’t providing that in our auto_complete_for :category, :name call:

def auto_complete_for(:category, :name, {})
  define_me thod("auto_complete_for_category_name") do
    find_options = { 
      :conditions => [
                       "LOWER(name) LIKE ?",
                       '%' + params[:category][:name].downcase + '%'
                     ],
      :order => "name ASC",
      :limit => 10 }.merge!({})

    @items = Category.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, 'name' %>"

  end
end

You can see that options was used in only one place: …merge!(options). Using merge with hashes is another extremely common technique in Ruby coding; it adds the key/value pairs from the hash you provide as a parameter to the hash you call merge on. Merge! is a variation on this which directly modifies the target object, as opposed to only returning the merged hash. Merge is very useful for metaprogramming because, as in this example, it makes it easy to allow the user/client code of a method to customize a hash of options that is passed into some other function. Here merge was used to allow the caller of auto_complete_for to pass in additional find options to the Category.find call. In our case since we didn’t pass in a value for options, it is set to {} and then has no effect on the find_options hash. So let’s just remove the call to merge!({}), which is a NOP anyway:

def auto_complete_for(:category, :name, {})
  define_method("auto_complete_for_category_name") do
    find_options = { 
      :conditions => [
                       "LOWER(name) LIKE ?",
                       '%' + params[:category][:name].downcase + '%'
                     ],
      :order => "name ASC",
      :limit => 10 }

    @items = Category.find(:all, find_options)

    render :inline => "<%= auto_complete_result @items, 'name' %>"

  end
end

Now the code is looking simpler and simpler. Let’s finish this example by evaluating the “define_method” call at the top. As you might guess, define_method is how you can dynamically create a new method in a class in Ruby. In auto_complete_for it’s used to add a new method to the controller in which you included the call to auto_complete_for, called “auto_complete_for_category_name.” In other words, adding the auto_complete_for :category, :name line to the CategoriesController was equivalent to adding this method definition to the class:

def auto_complete_for_category_name
  find_options = { 
    :conditions => [
                     "LOWER(name) LIKE ?",
                     '%' + params[:category][:name].downcase + '%'
                   ],
    :order => "name ASC",
    :limit => 10 }

  @items = Category.find(:all, find_options)

  render :inline => "<%= auto_complete_result @items, 'name' %>"

end

Now we’ve seen the real value of metaprogramming: the author of the auto_complete plugin was able to provide a simple helper method, auto_complete_for, which dynamically added this fairly complex method to your controller. The beauty and power of Ruby and Rails is just how easy this was to do: the generated function is tailored to use the Category model just as if you had written it yourself. When I first came across this code I was impressed not only by the power that this sort of metaprogramming provided, but also by how easy it was to do this using Ruby on Rails. Helper modules and methods, such as constantize in ActiveSupport, make it very, very easy to convert from symbols to strings to constants and back again… which is exactly what you need to do to write a general method that can be dynamically generated in this way.

Last week I asserted that calling auto_complete_for in your controller like this was equivalent to this code from Ryan Bates’ Auto-Complete Association screencast:

class CategoriesController < ApplicationController
  
    def index
      @categories =
        Category.find(:all, :conditions => ['name LIKE ?', "%#{params[:search]}%"])
    end
  
  etc...

Now you can see how this is true: the code Ryan wrote was a simple call to Category.find, with conditions that search for a name like the name provided in the “search” parameter. Looking at the simplified auto_complete_for_category_name method above, you can see that it calls Category.find in the same way. However, the auto_complete_for version is a bit different in that it:

Looks for a parameter called “category[name]” instead of “search,” and
Converts it to lower case, and
Passes a couple of other SQL options into Category.find: order by name ascending, and limit 10, and
Makes a call to render :inline to return the result to the browser without the need for a normal view code file.

Auto_complete_for does essentially the same thing that Ryan explained in his screen cast, but the elegance of Ruby metaprogramming allows the users of the auto_complete plugin to implement this search feature without writing any code at all.

Creating associations to existing data part 2: belongs_to with auto_complete

6 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

In my last post I started a series on how to write Rails forms that associate a new record with existing data. This sort of requirement comes up for me over and over again at my day job, and so I decided to support scaffolding for these forms in View Mapper.

Today I’ll continue by showing how to use the auto_complete plugin to select an existing record – exactly what Ryan Bates discussed in his screen cast Auto-Complete Association. Using the same Category/Product example, this form would allow the user to create a new product record, and associate it with an existing category tag:

To create scaffolding like this in your app with View Mapper, just run this command:

script/generate scaffold_for_view product name:string bar_code:integer
                --view belongs_to_auto_complete:category

View Mapper will validate you have a line “has_many :products” in your category model, that you have a category model to begin with, and also that the auto_complete plugin is installed before proceeding to generate this form. Also, View Mapper assumes the parent model, “Category” in this example, has a “name” column and will use that value to identify each category in the auto complete list. You can indicate to use a different parent model field instead with this syntax:

script/generate scaffold_for_view product name:string bar_code:integer
                --view belongs_to_auto_complete:category[display_name]

For more details on View Mapper, see the example below where I create a sample app from scratch.

Code review: model

Since Ryan explains auto complete association so well in his screen cast, I won’t repeat all of that information here. Instead, let’s take a look at the code View Mapper generates and compare it to what Ryan showed. First, in the product model Ryan has a “category_name” virtual attribute:

def category_name
  category.name if category
end
def category_name=(name)
  self.category = Category.find_or_create_by_name(name) unless name.blank?
end

This allows the view to display the category for each product easily, and also supports creating new categories on the fly when you submit a new product. The View Mapper scaffolding is a bit simpler and uses “find_by_name” instead of “find_or_create_by_name” since it assumes the category records already exist. Also, Ryan’s code uses “unless name.blank?” to avoid creating empty categories, while the View Mapper scaffolding assumes a blank category name indicates a product without a category, and allows the user to clear the category when editing an existing product. Either approach can make sense depending on the business requirements of your application. Here’s the View Mapper model code:

class Product < ActiveRecord::Base
  belongs_to :category
  def category_name
    category.name if category
  end
  def category_name=(name)
    self.category = Category.find_by_name(name)
  end
end

Code review: controller

In the controller code, the View Mapper scaffolding differs from Ryan’s solution more dramatically. To return the list of matching categories to the auto_complete plugin, Ryan adds this code to the categories controller to query the category records that have a name field that match a given search parameter:

def index
  @categories =
    Category.find(:all, :conditions => ['name LIKE ?', "%#{params[:search]}%"])
end

This makes a lot of sense: the categories controller should be used to generate a list of categories. However, for View Mapper I chose to use the products controller instead since the scaffolding generator already generates that code file, and to avoid the need for creating or modifying the categories controller also. View Mapper just adds this one line to the products controller to return the list of category names:

auto_complete_for :category, :name

This simple call actually achieves exactly the same thing as the Category.find call above. In my next post, I’ll take a look at the Ruby metaprogramming used by auto_complete_for and show how it automatically generates a method that executes the same SQL query.

Code review: view

Finally in the view we see a call to “text_field_with_auto_complete” to use the Prototype javascript library’s auto_complete feature. Here’s Ryan’s view code from the screen cast:

<%= text_field_with_auto_complete :product,
                                  :category_name,
                                  { :size => 15 },
                                  { :url => formatted_categories_path(:js),
                                    :method => :get,
                                    :param_name => 'search' }
%>

“:url => formatted_categories_path(:js)” calls the categories controller code above when the user starts to type in the text field, and the “:param_name =>‘search’” option passes the user’s text in as the search parameter. Ryan’s solution also uses a view file called “index.js.erb” to return the list of completion options in the proper format – this is called by the index action when the categories controller receives the “/categories.js” request:

<%= auto_complete_result @categories, :name %>

By contrast, View Mapper’s call to text_field_with_auto_complete looks like this:

<%= text_field_with_auto_complete :product,
                                  :category_name,
                                  {},
                                  { :method => :get,
                                    :url => '/auto_complete_for_category_name',
                                    :param_name => 'category[name]' }
%>

This is very similar, but uses “category[name]” as the search parameter and sets the AJAX url to “auto_complete_for_office_code”, since this is what “auto_complete_for” expects.

Ryan’s approach is more elegant since it follows the REST model for the Ajax URL and controller code: the categories controller is used to handle category related requests, and its index action is used to return the list of category values. The scaffolding code View Mapper generates uses the auto_complete plugin the way it was originally intended with the “auto_complete_for” function, but is a bit ugly in that the products controller returns the category values, and uses a custom action method name instead of the normal “index” action. There’s no need for the index.js.erb file since auto_complete_for renders the response inline.

The advantage of the View Mapper approach is that there’s no need for the categories controller at all, and also you don’t need to code the “index” action or the index.js.erb view file yourself. If you plan to have a categories controller in your application anyway, you might want to change the text_field_with_auto_complete call to use that controller instead.

Detailed example

To make sure you can get a working example on your computer, let’s run through a step by step example:

$ rails sample_app
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
etc...

First, let’s create a model to represent our existing data called “Office” that will have two attributes: “display_name” and “code:”

$ cd sample_app/
$ ./script/generate model office code:string display_name:string
      exists  app/models/
      exists  test/unit/
      exists  test/fixtures/
      create  app/models/office.rb
      create  test/unit/office_test.rb
      create  test/fixtures/offices.yml
      create  db/migrate
      create  db/migrate/20100212193446_create_offices.rb
$ rake db:migrate
(in /Users/pat/rails-apps/sample_app)
==  CreateOffices: migrating ==================================================
-- create_table(:offices)
   -> 0.0031s
==  CreateOffices: migrated (0.0034s) =========================================

And now let’s create a few sample office records:

$ ./script/console 
Loading development environment (Rails 2.3.5)
>> Office.create :display_name => 'Boston', :code => 'BOS'
>> Office.create :display_name => 'Boise', :code => 'BOI'
>> Office.create :display_name => 'Barcelona', :code => 'BAR'
>> exit

Now you can install View Mapper – you’ll need version 0.3.3 for the “belongs_to_auto_complete” view:

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources
$ sudo gem install view_mapper
Successfully installed view_mapper-0.3.3
1 gem installed
Installing ri documentation for view_mapper-0.3.3...
Installing RDoc documentation for view_mapper-0.3.3...

And now we can just run View Mapper’s “scaffold_for_view” generator to create the scaffolding code. Let’s try creating a new model called “Employee” that will belong to one of the existing offices:

$ ./script/generate scaffold_for_view employee name:string
                    --view belongs_to_auto_complete:office
       error  The auto_complete plugin does not appear to be installed.
$ ./script/plugin install git://github.com/rails/auto_complete.git
Initialized empty Git repository in /Users/pat/rails-apps/sample_app/vendor/plugins/auto_complete/.git/
remote: Counting objects: 13, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 13 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (13/13), done.
From git://github.com/rails/auto_complete
 * branch            HEAD       -> FETCH_HEAD

Trying again:

$ ./script/generate scaffold_for_view employee name:string
                    --view belongs_to_auto_complete:office
     warning  Model Office does not contain a has_many association for Employee.

Editing app/models/office.rb:

1 class Office < ActiveRecord::Base
2   has_many :employees
3 end

Trying a third time:

$ ./script/generate scaffold_for_view employee name:string
                    --view belongs_to_auto_complete:office
     warning  Model Office does not have a name attribute.

This time we get a warning that our existing model doesn’t have a “name” attribute (we chose “display_name” instead). To make this a bit more interesting, let’s use the “code” attribute for the auto_complete options. You can specify this to View Mapper with this syntax:

$ ./script/generate scaffold_for_view employee name:string
                    --view belongs_to_auto_complete:office[code]
      exists  app/models/
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/employees
      exists  app/views/layouts/
      exists  test/functional
etc...

Now we just run rake db:migrate again, start our server and we’re done!

$ rake db:migrate
(in /Users/pat/rails-apps/sample_app)
==  CreateEmployees: migrating ================================================
-- create_table(:employees)
   -> 0.0034s
==  CreateEmployees: migrated (0.0036s) =======================================

Creating associations to existing data part 1: belongs_to scaffolding

7 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

I decided it would be fun to look into various different types of Rails forms that allow you to create a new object that is associated with existing data. In my next few posts, I’ll explore different ways to select existing records, and also how to work with has_and_belongs_to_many and has_many, through relationships in a Rails form. It seems to me that these use cases are more common than the nested models form in the complex-form-examples sample app that creates new records but doesn't associate with existing ones.

To start with today, here’s the simplest such form I could think of – I call this a “belongs_to” form:

Here we can see a form for a new “shirt” record; along with the color and size the user can also select the person who owns the shirt. In this example, the shirt and person models have a has_many/belongs_to relationship. This form uses simple HTML <select> and <option> tags to display the list of people, and is generated by Rails ERB code that uses “collection_select” like this:

 1 <p>
 2   Person:<br />
 3   <%= f.collection_select(:person_id,
 4                           Person.all,
 5                           :id,
 6                           :name,
 7                           { :prompt => true })
 8   %>
 9 </p>

The parameters passed to collection_select here indicate that we want to display all of the people that exist, and set the value and label for each <option> tag to the id and name of each person respectively. When the form is submitted, the “person_id” field for this shirt is set to the “id” value of the selected person.

Belongs_to scaffolding with View Mapper

If you need a form like this in your app, you can use my View Mapper gem to generate it for your models as follows… first install it from gemcutter; you’ll need version 0.3.2 at least for this form:

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources 
$ sudo gem install view_mapper
Successfully installed view_mapper-0.3.2

Then you can generate the belongs_to view scaffolding you see above like this:

$ ./script/generate view_for shirt --view belongs_to

View Mapper will open the specified model (“Shirt”), detect the associated model(s) that Shirt belongs to, and then generate the form using collection_select along with all of the other standard scaffolding files. You can also have View Mapper generate the new Shirt model and the view scaffolding code at the same time like this:

./script/generate scaffold_for_view shirt color:string size:integer
                                    --view belongs_to:person

Here you’ve specified the desired attributes for the new shirt model, along with the fact that you want it to belong to a person.

To make the generated view code simple and concise, View Mapper makes a couple of assumptions about your models:

It assumes the parent model (“Person” in this example) has an attribute or method called “name.” This is used to display the list of people.
It also assumes the child model (“Shirt”) has a method to display the name of the parent model it belongs to (“person_name” in this example).

I decided not to make the View Mapper command line more complex than it already is by providing a way to pass the “name” attribute as another parameter. Instead you can just edit the scaffolding code it produces as desired.

Detailed example and code review

Let’s create a sample app now together using View Mapper, and then review how this “belongs_to view” works. First, create a new Rails app:

$ rails belongs_to
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
      create  config/environments
      etc...

And now let’s create the Person model with first and last name attributes; I’ll also use the console to create a few Person records so we have some existing data to work with:

$ cd belongs_to
$ ./script/generate model person first_name:string last_name:string
      exists  app/models/
      exists  test/unit/
      exists  test/fixtures/
      create  app/models/person.rb
      create  test/unit/person_test.rb
      create  test/fixtures/people.yml
      create  db/migrate
      create  db/migrate/20100124120247_create_people.rb
$ rake db:migrate
(in /Users/pat/rails-apps/belongs_to)
==  CreatePeople: migrating ===================================================
-- create_table(:people)
   -> 0.0030s
==  CreatePeople: migrated (0.0032s) ==========================================
 $ ./script/console 
Loading development environment (Rails 2.3.5)
>> Person.create :first_name => 'Barack', :last_name => 'Obama'
>> Person.create :first_name => 'George', :last_name => 'Bush'
>> Person.create :first_name => 'Bill',   :last_name => 'Clinton'

Now let’s go ahead and run View Mapper to create the form…

$ ./script/generate scaffold_for_view shirt color:string size:integer
                                      --view belongs_to:person
     warning  Model Person does not contain a has_many association for Shirt.

Here View Mapper is warning us that we haven’t called “has_many :shirts” in the Person model yet; let’s edit person.rb and enter that code:

1 class Person < ActiveRecord::Base
2   has_many :shirts
3 end

Now we can try again:

$ ./script/generate scaffold_for_view shirt color:string size:integer
                                      --view belongs_to:person
     warning  Model Person does not have a name attribute.

Above I mentioned that View Mapper assumes the parent model has a “name” attribute or method. This is required to know how to display each person record in the <select> drop down box on the form. Remember in the call to collection_select we passed in the symbol “:name” – this tells Rails to call the name method for the label of each <option> tag. So let’s create a name method in the Person model that displays the first and last name attributes together:

1 class Person &lt; ActiveRecord::Base
2   has_many :shirts
3   def name
4     "#{first_name} #{last_name}"
5   end
6 end

The highlighted code returns the first and last names concatenated together as a single name. Now View Mapper won’t complain and we can go ahead and create our new form:

$ ./script/generate scaffold_for_view shirt color:string size:integer
                                      --view belongs_to:person
      exists  app/models/
      exists  app/controllers/
… etc …

     create  app/views/shirts/_form.html.erb
… etc …
      exists  test/fixtures/
      create  app/models/shirt.rb
      create  db/migrate/20100124121032_create_shirts.rb

Note the output here looks just like what you get from the standard Rails scaffold generator, except for the one additional line I highlighted above. If you run your app now, you’ll be able to see scaffolding for the Shirts model at http://localhost:3000/shirts, and you’ll see the person select box on the new and edit forms.

Let me take a few more minutes to point out a couple of interesting details about the belongs_to scaffolding… first I’ve moved the form fields into a partial shared among the new and edit views; this is the new _form.html.erb file highlighted above in the generator output. If you look at new.html.erb, you’ll see a call to render :partial:

 1 <h1>New shirt</h1>
 2 
 3 <% form_for(@shirt) do |f| %>
 4   <%= render :partial => 'form', :locals => { :f => f } %>
 5   <p>
 6     <%= f.submit 'Create' %>
 7   </p>
 8 <% end %>
 9 
10 <%= link_to 'Back', shirts_path %>

The edit.html.erb file looks similar. The actual call to collection_select is in _form.html.erb; that way if you need to display the list of people differently, for example to use a method other than “name” for each person or possibly to use a filtered list of people instead of Person.all, then you just need to make your changes in one place.

And one more interesting detail: if you open up the new Shirts model that View Mapper generated you’ll see this:

1 class Shirt < ActiveRecord::Base
2   belongs_to :person
3   def person_name
4     person.name if person
5   end
6 end

The person_name method I highlighted above returns the person each shirt belongs to; it also checks if person is nil for that shirt. This simplifies the code in index.html.erb and show.html.erb; take a look at show.html.erb for example:

 1 <p>
 2   <b>Color:</b>
 3   <%=h @shirt.color %>
 4 </p>
 5 
 6 <p>
 7   <b>Size:</b>
 8   <%=h @shirt.size %>
 9 </p>
10 
11 <p>
12   <b>Person:</b>
13   <%=h @shirt.person_name %>
14 </p>
15 
16 <%= link_to 'Edit', edit_shirt_path(@shirt) %> |
17 <%= link_to 'Back', shirts_path %>

Here the Person field looks just like any other field from the Shirt model. There’s no need to repeat the check for person == nil in the view, and if you ever need to use a Person attribute other than name or if you needed to find the associated person in some more complex way, you’ll only need to make the change to the model and not in each view file.

Again, if you run View Mapper on two has_many/belongs_to models that you’ve already written in your app, you will first need to provide:

A “name” method in the parent model, if it’s not already an attribute, and
A “[parent_model]_name” method in the child model

If these two methods don’t exist, View Mapper will display warning messages and not proceed; this avoids the confusion you would run into when the scaffolding view code didn’t work.

Next time, I’ll repeat this example but use type ahead/auto_complete to select the person instead...

How does the complex-form-examples sample app work?

8 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

The complex-form-examples sample application written by Ryan Bates and then updated by Eloy Duran and many others is the standard example of how to implement a complex form in Rails. It shows how you can create and update more than one model using the same form. Last month, I wrote about how to create scaffolding for a complex form; using my View Mapper gem you can create a simplified version of the sample app right inside your application for your models.

Today I’d like to take some time to explain how my simplified version of the complex form actually works – the key to using scaffolding in your Rails application is understanding how it works so you can eventually modify and adapt it for you needs, and discard the code you don’t need. However, since this sample application is fairly complex I decided it would be more interesting and fun to follow a single code path through the app in a series of small steps, seeing how each small piece works in detail. To do this, I wrote a series of short blog pages or slides; to see it, click the “Follow Code Path” link… once you’re on the first page you’ll see links to move forward and backward through the slides.

Follow Code Path

Let me know here if you have any feedback on either the content or the style of the code path pages since they are just hard coded HTML for now and not part of my blog. Thanks!

How to convert a Rails plugin into a gem

8 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

Recently I decided to convert my fork of the auto_complete plugin into a gem; I called it “repeated_auto_complete.” In the end it was very easy to convert a plugin into a gem; all I had to do was:

Make sure there was a code file in the lib folder with the same name as the gem, and
Move or copy the init.rb into a subfolder called “rails.”

This is simple enough, but why do I need to do this? These changes seem rather odd, and also it took me about 3-4 hours of debugging to figure out what I needed to do. The answer has to do with the way the Rails framework loads gems… this is more confusing and complicated than you might think! The rest of this article will show exactly how this works in detail, comparing how gems and plugins are loaded.

The load path works the same way for plugins and gems

Rails treats the load path in the same way for gems as it does for plugins. This is a relief, and also not a surprise since gems and plugins are very similar to each other. The best way to get a sense of how the load path works with plugins and gems is just to inspect it directly in the console. To do this, let’s start by creating a new sample app:

$ rails sample
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
etc…

And now let’s install the auto_complete plugin:

$ cd sample
$ ./script/plugin install git://github.com/rails/auto_complete.git
Initialized empty Git repository in .git/
warning: no common commits
remote: Counting objects: 13, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 13remote:  (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (13/13), done.

If we start a Rails console we can use the command in bold to just look at the load path:

$ ./script/console 
Loading development environment (Rails 2.3.5)
>> $LOAD_PATH.each { |path| puts path }; nil
.../gems/activesupport-2.3.5/lib/active_support/vendor/i18n-0.1.3/lib
.../gems/activesupport-2.3.5/lib/active_support/vendor/tzinfo-0.3.12
.../gems/activesupport-2.3.5/lib/active_support/vendor/memcache-client-1.7.4
/Users/pat/rails-apps/sample/app/controllers/
/Users/pat/rails-apps/sample/app
/Users/pat/rails-apps/sample/app/models
/Users/pat/rails-apps/sample/app/controllers
/Users/pat/rails-apps/sample/app/helpers
/Users/pat/rails-apps/sample/lib
/Users/pat/rails-apps/sample/vendor/plugins/auto_complete/lib
/Users/pat/rails-apps/sample/vendor
.../gems/rails-2.3.5/lib/../builtin/rails_info/
.../gems/rails-2.3.5/lib
etc…

Here we can see the various application paths for my new sample app, as well as the paths of a few of the gems found on my laptop. For clarity, I've shortened the path to my gems folder, and there are many more gems that I’m not showing here. The line in bold indicates that the lib folder for the auto_complete plugin is included in the load paths array, allowing Rails to look inside the auto_complete plugin in order to find missing constants.

Now if I remove the auto_complete plugin…

$ rm -rf vendor/plugins/auto_complete

… and install the repeated_auto_complete gem (from gemcutter):

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources
$ sudo gem install repeated_auto_complete
Password:
Successfully installed repeated_auto_complete-0.1.0
1 gem installed
Installing ri documentation for repeated_auto_complete-0.1.0...
Installing RDoc documentation for repeated_auto_complete-0.1.0...

… and add a call to config.gem in config/environment.rb:

Rails::Initializer.run do |config|
…
  config.gem "repeated_auto_complete"
…
end

… and view the load path again:

$ ./script/console 
Loading development environment (Rails 2.3.5)
>> $LOAD_PATH.each { |path| puts path }; nil
.../gems/activesupport-2.3.5/lib/active_support/vendor/i18n-0.1.3/lib
.../gems/activesupport-2.3.5/lib/active_support/vendor/tzinfo-0.3.12
.../gems/activesupport-2.3.5/lib/active_support/vendor/memcache-client-1.7.4
/Users/pat/rails-apps/sample/app/controllers/
/Users/pat/rails-apps/sample/app
/Users/pat/rails-apps/sample/app/models
/Users/pat/rails-apps/sample/app/controllers
/Users/pat/rails-apps/sample/app/helpers
/Users/pat/rails-apps/sample/lib
.../gems/repeated_auto_complete-0.1.0/lib
/Users/pat/rails-apps/sample/vendor
.../gems/rails-2.3.5/lib/../builtin/rails_info/
.../gems/rails-2.3.5/lib
etc…

In bold I can see the gem’s lib folder appear just as the plugin’s lib folder did earlier. In fact, it even appears at the same position in the array so classes should be loaded in exactly the same way for a gem as they were for a plugin.

For a gem, you need to have the expected code file in your lib folder

This next issue caused me some serious headaches… hopefully this explanation will save you some time. To explore how gems are loaded by Rails, let’s unpack my “repeated_auto_complete” gem that I just installed above:

$ rake gems:unpack
(in /Users/pat/rails-apps/sample)
Unpacked gem: '/Users/pat/rails-apps/sample/vendor/gems/repeated_auto_complete-0.1.0'

Now I have a local copy of the gem’s code in my vendor/gems directory. Next, let’s see what happens when I delete the “repeated_auto_complete.rb” file from the lib folder – in other words, the code file with the same name as the gem:

$ rm vendor/gems/repeated_auto_complete-0.1.0/lib/repeated_auto_complete.rb 
$ ./script/console 
Loading development environment (Rails 2.3.5)
no such file to load -- repeated_auto_complete
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
.../gems/activesupport-2.3.5/lib/active_support/dependencies.rb:156:in `require'
.../gems/activesupport-2.3.5/lib/active_support/dependencies.rb:521:in `new_constants_in'
.../gems/activesupport-2.3.5/lib/active_support/dependencies.rb:156:in `require'
.../gems/rails-2.3.5/lib/rails/gem_dependency.rb:208:in `load'
.../gems/rails-2.3.5/lib/initializer.rb:307:in `load_gems'
.../gems/rails-2.3.5/lib/initializer.rb:307:in `each'
.../gems/rails-2.3.5/lib/initializer.rb:307:in `load_gems'
.../gems/rails-2.3.5/lib/initializer.rb:164:in `process'
.../gems/rails-2.3.5/lib/initializer.rb:113:in `send'
.../gems/rails-2.3.5/lib/initializer.rb:113:in `run'
/Users/pat/rails-apps/sample/config/environment.rb:9
etc…

Now we get an error trying to load the Rails environment! The reason why is simple: when Rails loads each gem specified in the environment.rb file with a call to config.gem, it tries to load a code file with exactly the same name. Let’s take a look at line 307 of initializer.rb which appears in the stack trace above:

def load_gems
  unless $gems_rake_task
    @configuration.gems.each { |gem| gem.load }
  end
end

What’s going on here is:

load_gems is a method of the Rails::Initializer object. This is the class that we refer to in environment.rb.
@configuration is the configuration object that was yielded to the initializer block in environment.rb… in order words the value of the “config” variable that we used in our call to “config.gem”
@configuration.gems is an array of GemDependency objects; each one created by a config.gem call. If you’re interested, you can see these are created at line 811 in initializer.rb.
For each GemDependency object Rails calls “load.”

Let’s take a look at the GemDependency.load method: line 208 in gem_dependency.rb, also in the stack trace above:

def load
  return if @loaded || @load_paths_added == false
  require(@lib || name) unless @lib == false
  @loaded = true
rescue LoadError
  puts $!.to_s
  $!.backtrace.each { |b| puts b }
end

When the GemDependency object was created earlier, two of its attributes were loaded with values as follows:

name: this is set to the name of the gem – “repeated_auto_complete” in my example
@lib: this is set to the value of the “:lib” option provided to the config.gem call.

So if you read the code above, you’ll see that Rails allows for three possible cases when loading a gem:

config.gem ‘repeated_auto_complete’ – in this case Rails will call require “repeated_auto_complete” and fail if a code file with that name is not present in the load path. This is what just happened to us above.
config.gem ‘repeated_auto_complete’, :lib => ‘something_else’ – in this case Rails will call require “something_else” and fail if a code file with that name is not present in the load path.
config.gem ‘repeated_auto_complete’, :lib => false – in this case Rails will not call require at all for this gem.

Note that for plugins none of this is an issue: Rails simply adds the plugin's lib folder to the load path array and that's it. But when you convert a plugin into a gem, you need to decide which variation of config.gem your users will have to put in environment.rb.

Init.rb has to move

The next thing Rails does after loading each plugin or gem is to execute a file called init.rb. If you’re the author of a gem or plugin this gives you a chance to initialize your code… for example to add certain modules you’ve written to classes in the application, etc. But as I mentioned at the beginning, if you’re writing a gem or converting a plugin into a gem, you need to be sure the init.rb file is located inside a folder called “rails.” Let’s see if we can find out how Rails does this; first let’s restore the original gem’s code:

$ rm -rf vendor/gems/repeated_auto_complete-0.1.0
$ rake gems:unpack
(in /Users/pat/rails-apps/sample)
Unpacked gem: '/Users/pat/rails-apps/sample/vendor/gems/repeated_auto_complete-0.1.0'

And now let’s edit the init.rb file, located at vendor/gems/repeated_auto_complete-0.1.0/rails/init.rb:

puts caller
ActionController::Base.send :include, AutoComplete
ActionController::Base.helper AutoCompleteMacrosHelper
ActionView::Helpers::FormBuilder.send :include, AutoCompleteFormBuilderHelper

I added the first line in bold: “puts caller.” This will display a stack trace leading to this file when we startup the sample application:

$ ./script/console 
Loading development environment (Rails 2.3.5)
.../gems/rails-2.3.5/lib/rails/plugin.rb:158:in `evaluate_init_rb'
.../gems/activesupport-2.3.5/lib/active_support/core_ext/kernel/reporting.rb:11:in `silence_warnings'
.../gems/rails-2.3.5/lib/rails/plugin.rb:154:in `evaluate_init_rb'
.../gems/rails-2.3.5/lib/rails/plugin.rb:48:in `load'
.../gems/rails-2.3.5/lib/rails/plugin/loader.rb:38:in `load_plugins'
.../gems/rails-2.3.5/lib/rails/plugin/loader.rb:37:in `each'
.../gems/rails-2.3.5/lib/rails/plugin/loader.rb:37:in `load_plugins'
.../gems/rails-2.3.5/lib/initializer.rb:369:in `load_plugins'
.../gems/rails-2.3.5/lib/initializer.rb:165:in `process'
.../gems/rails-2.3.5/lib/initializer.rb:113:in `send'
.../gems/rails-2.3.5/lib/initializer.rb:113:in `run'
/Users/pat/rails-apps/sample/config/environment.rb:9

This time I’ve bolded the “plugin.rb” file; if you look at line 152 in plugin.rb you’ll see this:

def evaluate_init_rb(initializer)
  if has_init_file?
    silence_warnings do
      # Allow plugins to reference the current configuration object
      config = initializer.configuration
      eval(IO.read(init_path), binding, init_path)
    end
  end
end

So this just calls “eval()” on the init.rb file, assuming that “init_path” indicates the path of this file, executing the plugin’s or gem’s initialization code. If you poke around a bit inside of plugin.rb, you’ll see this code for the Rails:Plugin class, which represents each plugin that Rails finds in your application:

def classic_init_path
  File.join(directory, 'init.rb')
end

def gem_init_path
  File.join(directory, 'rails', 'init.rb')
end

def init_path
  File.file?(gem_init_path) ? gem_init_path : classic_init_path
end

If we read the definition of init_path, we see that it uses either rails/init.rb or init.rb, whichever it finds first. This seems to indicate that for a Rails plugin, you can place init.rb either in the “rails” subfolder, or in the main plugin folder, and that it will find and use the copy in the “rails” folder if you happen to have both.

However, for a gem things don’t work this way. You can see why if you look down towards the bottom of the plugin.rb file:

class GemPlugin < Plugin
  # Initialize this plugin from a Gem::Specification.
  def initialize(spec, gem)
    directory = spec.full_gem_path
    super(directory)
    @name = spec.name
  end
  def init_path
    File.join(directory, 'rails', 'init.rb')
  end
end

It turns out that Rails uses a different class to represent gems, called “GemPlugin” (what a confusing name!). In this case we can see that init_path is defined to be the path rails/init.rb and nothing else. This means that gems intended to be used in a Rails application must put their init.rb file in the rails folder.

To summarize this logic:

Rails plugins can place init.rb either in the root plugin folder, or in a subfolder called “rails.” If they have both, the rails folder copy will be used.
Rails gems must place their init.rb file in a “rails” subfolder.

The actual reason why Rails was implemented this way was that possibly a gem might be used by more than one Ruby framework (e.g. Merb, Sinatra, etc.) and might have different init.rb code for each framework. But a Rails plugin can only be used in a Rails application. Finally, Rails has allowed for plugins to work in the original manner with init.rb in the root folder, or for a plugin to be a gem at the same time, with init.rb in the rails folder.

Scaffolding for auto complete on a complex, nested form

9 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

I just updated View Mapper to work with my fork of the Rails auto_complete plugin that allows for repeated text fields on the same complex form. This means that View Mapper can now generate scaffolding code that uses both nested attributes and the auto_complete plugin at the same time, to display a form like this:

To generate this sort of complex form for two of your models you’ll first need to install my “repeated_auto_complete” gem from gemcutter.org:

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources
$ sudo gem install repeated_auto_complete
Successfully installed repeated_auto_complete-0.1.0
1 gem installed
Installing ri documentation for repeated_auto_complete-0.1.0...
Installing RDoc documentation for repeated_auto_complete-0.1.0...

To learn more about repeated_auto_complete and what it does, see: http://patshaughnessy.net/repeated_auto_complete. Now you can generate a complex form like the one shown above for two of your models in a has_many/belongs_to, has_and_belongs_to_many or has_many, :through association by installing View Mapper (version 0.3.1 or later):

$ sudo gem install view_mapper
Successfully installed view_mapper-0.3.1
1 gem installed
Installing ri documentation for view_mapper-0.3.1...
Installing RDoc documentation for view_mapper-0.3.1...

… and then running the “view_for” generator with a view option called “has_many_auto_complete,” like this:

./script/generate view_for group --view has_many_auto_complete:people

Detailed Example

To see how easy it is to create a complex form using View Mapper, let’s create one from scratch in a brand new Rails app. You should be able to follow along using the commands below on your machine. First, let’s create a new Rails application:

$ rails complex_auto_complete
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
      create  config/environments
      create  config/initializers
      create  config/locales
    … etc..
      create  log/server.log
      create  log/production.log
      create  log/development.log
      create  log/test.log

The first thing I’ll do is install the auto_complete plugin. However, since I’m planning to use auto_complete on a complex form, I’ll need to get my fork of auto_complete which I’ve deployed as a gem on gemcutter.org:

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources
$ sudo gem install repeated_auto_complete
Successfully installed repeated_auto_complete-0.1.0
1 gem installed
Installing ri documentation for repeated_auto_complete-0.1.0...
Installing RDoc documentation for repeated_auto_complete-0.1.0...

And let’s update my new app to use the repeated_auto_complete gem by editing the config/environment.rb file:

Rails::Initializer.run do |config|
…etc…
  config.gem "repeated_auto_complete"
…etc…

If you prefer, you can also install this the old fashioned way, using “script/plugin install git://github.com/patshaughnessy/auto_complete.git”. Next, let’s generate a new model called “person” with a couple of fields for name and age, like the ones shown above in the screen shot:

$ cd complex_auto_complete/
$ ./script/generate model person name:string age:integer group_id:integer
      exists  app/models/
      exists  test/unit/
      exists  test/fixtures/
      create  app/models/person.rb
      create  test/unit/person_test.rb
      create  test/fixtures/people.yml
      create  db/migrate
      create  db/migrate/20091125195040_create_people.rb

Note that I’ve also included an integer field for the group id, since in a minute I’ll be adding a belongs_to association for people to groups.

Now I’m ready to use View Mapper… if you haven’t installed that yet, get it from gemcutter.org like this:

$ sudo gem install view_mapper
Successfully installed view_mapper-0.3.1
1 gem installed
Installing ri documentation for view_mapper-0.3.1...
Installing RDoc documentation for view_mapper-0.3.1...

You’ll need at least version 0.3.1 to use auto_complete on a complex form. Now I can use View Mapper to create scaffolding for a new “group” model that has many people with auto_complete like this:

$ ./script/generate scaffold_for_view group name:string
                    --view has_many_auto_complete:people
       error  Table for model 'person' does not exist
              - run rake db:migrate first.

Yes… I forgot to create the people table in my database; if we do that:

$ rake db:migrate
(in /Users/pat/rails-apps/complex_auto_complete)
==  CreatePeople: migrating ===================================================
-- create_table(:people)
   -> 0.0014s
==  CreatePeople: migrated (0.0015s) ==========================================

… and then re-run View Mapper:

$ ./script/generate scaffold_for_view group name:string
                    --view has_many_auto_complete:people
     warning  Model Person does not contain a belongs_to
              association for Group.

… we get a second error message! This time View Mapper is reminding me that I still need to add “belongs_to :group” to the person model in order to get the complex form to work. Let’s do that now:

class Person < ActiveRecord::Base
  belongs_to :group
end

And now I can run View Mapper once more:

$ ./script/generate scaffold_for_view group name:string
                    --view has_many_auto_complete:people
      exists  app/models/
…etc…
      create  app/models/group.rb
      create  test/unit/group_test.rb
      create  test/fixtures/groups.yml
      exists  db/migrate
      create  db/migrate/20091125195715_create_groups.rb
      create  app/views/groups/show.html.erb
      create  app/views/groups/_form.html.erb
      create  app/views/groups/_person.html.erb
      create  public/javascripts/nested_attributes.js
       route  map.connect 'auto_complete_for_group_name',
                          :controller => 'groups',
                          :action => 'auto_complete_for_group_name'
       route  map.connect 'auto_complete_for_person_name',
                          :controller => 'groups',
                          :action => 'auto_complete_for_person_name'
       route  map.connect 'auto_complete_for_person_age',
                          :controller => 'groups',
                          :action => 'auto_complete_for_person_age'

Now you can see the new scaffolding files View Mapper created, including some new scaffolding files peculiar to complex forms, like “nested_attributes.js,” “_form.html.erb,” and “_person.html.erb.” You may also have noticed View Mapper added three new routes related to the auto_complete plugin; these will handle the AJAX requests used to return the auto_complete options to the form.

Now to get it all to work, I just need to create the group table:

$ rake db:migrate
(in /Users/pat/rails-apps/complex_auto_complete)
==  CreateGroups: migrating ===================================================
-- create_table(:groups)
   -> 0.0013s
==  CreateGroups: migrated (0.0014s) ==========================================

Now running my server and creating a new group I see:

If you click “Add a Person” you’ll see nested fields for new Person records appear. This all works exactly the same way as the standard nested attributes scaffolding that I described in my last post. The only difference is that in this form, each of the text fields present in both the parent (“Group”) and child (“Person”) models are displayed using the “text_field_with_auto_complete” method.

I’ll try to write up a detailed walk through of how this scaffolding actually works as soon as I can… there are a lot of interesting details in the code that will be fun to look at. In the meantime, hopefully this scaffolding will make it easier for you to learn how to use auto_complete and nested attributes together in your app.

Scaffolding for complex forms using nested attributes

9 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

While the new nested attributes feature in Rails 2.3 greatly simplifies writing forms that operate on two or more models at the same time, writing a complex form is still a confusing and daunting task even for experienced Rails developers. To make this easier, I just added nested attribute support to my View Mapper gem. This means you can generate complex form scaffolding for two or more models in a has_many/belongs_to, has_and_belongs_to_many or has_many, through relationship.

Example:

If I have a group model that has many people and accepts nested attributes for them like this:

class Group < ActiveRecord::Base
  has_many :people
  accepts_nested_attributes_for :people, :allow_destroy => true
end

… and a person model that belongs to a group:

class Person < ActiveRecord::Base
  belongs_to :group
end

… then View Mapper will allow you to generate scaffolding that displays groups of people all at once, like this:

$ ./script/generate view_for group --view has_many:people
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/groups
…etc…
      create  app/views/groups/_form.html.erb
      create  app/views/groups/_person.html.erb
      create  public/javascripts/nested_attributes.js

Now if I open my Rails app and create a new group, I will see:

This looks just like the standard Rails scaffolding, but with one additional “Add a Person” link. If you click on it, you’ll see the attributes of the person model appear along with a “remove” link, indented to the right:

If I enter some values and submit, ActiveRecord will insert a new record into both the groups table and the people table, and set the group_id value in the new person record correctly:

View Mapper has:

inspected your group and person models to find their attributes (columns).
validated that they are in a has_many / belongs_to relationship, or in a has_and_belongs_to_many or a has_many, through relationship.
checked that you have a foreign key column (“group_id” by default for this example) in the people table if necessary. (The foreign key isn’t in the people table for has_and_belongs_to_many or has_many, through.)
generated scaffolding using your attribute and model names, and that uses Javascript to support the “Add a person” and “remove” links.

To get the add/remove links to work, I used a simplified version of the “complex-form-examples” sample application from Ryan Bates and Eloy Duran. Ryan has a few screen casts on this topic as well. In my next post I’ll explain how that works in detail, since understanding the details about how scaffolding works is the first step towards using it successfully in your app.

But for now, you can try this on your machine using the precise commands below…

Creating a new complex form from scratch

Let’s get started by creating a new Rails application; you will need to have Rails 2.3 or later in order to make this work:

$ rails complex-form
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
… etc …
      create  log/production.log
      create  log/development.log
      create  log/test.log

Using the same group has many people example from above, let’s generate a new person model:

$ cd complex-form
$ ./script/generate model person first_name:string last_name:string
      exists  app/models/
      exists  test/unit/
      exists  test/fixtures/
      create  app/models/person.rb
      create  test/unit/person_test.rb
      create  test/fixtures/people.yml
      create  db/migrate
      create  db/migrate/20091109204744_create_people.rb

And let’s run that migration to create the people table:

$ rake db:migrate
(in /Users/pat/rails-apps/complex-form)
==  CreatePeople: migrating ===================================================
-- create_table(:people)
   -> 0.0013s
==  CreatePeople: migrated (0.0014s) ==========================================

Now we’re ready to run View Mapper. View Mapper contains two generators; one is for creating scaffolding for an existing model, called “view_for,” which is what I used above. There’s also another generator called “scaffold_for_view” which will create a new model along with the scaffolding, using the same syntax as the standard Rails scaffold generator. Let’s use that here, since we have a new app and haven’t created the group model yet:

$ ./script/generate scaffold_for_view group name:string --view has_many:people
     warning  Model Person does not contain a belongs_to association for Group.

Here View Mapper is reminding me that I didn’t specify “belongs_to” in the person model. This saves me the trouble later of figuring out what’s wrong when my complex form doesn’t work. Let’s add that line to app/models/person.rb and try again:

class Person < ActiveRecord::Base
  belongs_to :group
end

$ ./script/generate scaffold_for_view group name:string --view has_many:people
     warning  Model Person does not contain a foreign key for Group.

Duh… I also forgot to include the “group_id” column when I generated the person model. I could have done that by including “group_id:integer” on the script/generate model command line above. Since I already have the person model now, let’s just continue by creating a new migration for the missing column:

$ ./script/generate migration add_group_id_column_to_people
      exists  db/migrate
      create  db/migrate/20091109205711_add_group_id_column_to_people.rb

Editing the migration file:

class AddGroupIdColumnToPeople < ActiveRecord::Migration
  def self.up
    add_column :people, :group_id, :integer
  end
etc…

And running the migration:

$ rake db:migrate
(in /Users/pat/rails-apps/complex-form)
==  AddGroupIdColumnToPeople: migrating =======================================
-- add_column(:people, :group_id, :integer)
   -> 0.0010s
==  AddGroupIdColumnToPeople: migrated (0.0012s) ==============================

Now let’s run View Mapper once more to see whether we have any other problems, or whether we’re ready to generate the complex form scaffolding:

$ ./script/generate scaffold_for_view group name:string --view has_many:people
      exists  app/models/
      exists  app/controllers/
…etc…
      create  app/models/group.rb
      create  test/unit/group_test.rb
      create  test/fixtures/groups.yml
      exists  db/migrate
      create  db/migrate/20091109210312_create_groups.rb
      create  app/views/groups/show.html.erb
      create  app/views/groups/_form.html.erb
      create  app/views/groups/_person.html.erb
      create  public/javascripts/nested_attributes.js

It worked! Just looking at the list of files that View Mapper created, you can get a sense of how it has customized the standard Rails scaffolding to implement the complex form: _form.html.erb, _person.html.erb, nested_attributes.js. More on these details in my next article.

One detail I will point out now is that in order to get you started in the right direction and to allow the complex form to work immediately, the scaffold_for_view generator included the has_many and accepts_nested_attributes_for calls in the new model:

class Group < ActiveRecord::Base
  has_many :people
  accepts_nested_attributes_for :people,
                                :allow_destroy => true,
                                :reject_if => proc { |attrs|
                                  attrs['first_name'].blank? &&
                                  attrs['last_name'].blank?
                                }
end

You don’t need to type in all of this code yourself and know the precise syntax of the accepts_nested_attributes_for method… it’s all generated for you. Later when you start to customize the scaffolding to work for your specific requirements, you’ll have a working example to look at right inside your app.

Finally, we’re need to run the migrations once more since the scaffold_for_view generator created a new group model and corresponding migration for the groups table:

$ rake db:migrate
(in /Users/pat/rails-apps/complex-form)
==  CreateGroups: migrating ===================================================
-- create_table(:groups)
   -> 0.0013s
==  CreateGroups: migrated (0.0014s) ==========================================

Now if you start up Rails and hit http://localhost:3000/groups/new, you’ll see the complex form!

Paperclip scaffolding

10 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

I just updated the View Mapper gem to support Paperclip. You can use it to generate scaffolding code that supports uploading and downloading Paperclip file attachments.

Creating a view for an existing model

If you have a model like this:

class Song < ActiveRecord::Base
  has_attached_file :mp3
end

… you can generate a “Paperclip view” for this model like this:

script/generate view_for song --view paperclip

This will generate a controller, view and other code files that support uploading and downloading files. If you run your app you’ll see the typical scaffolding user interface but with a file field for the “mp3” Paperclip attachment:

View Mapper has:

inspected your model (“Song” in this example) and found its Paperclip attachments and other standard ActiveRecord attributes
called the Rails scaffold generator and passed in the ActiveRecord columns it found
added additional view code to support Paperclip (e.g. set the form to use multipart/form-data encoding)
created a file field in the form for each Paperclip attachment (“mp3” in this example), as well as a link to each attachment in the show view code file.

If you’re not very familiar with Paperclip and how to use it or if you just want to get a Rails upload form working very quickly, then View Mapper can help you.

Creating an entirely new Paperclip model and view

View Mapper also provides a generator called “scaffold_for_view” that is identical to the standard Rails scaffold generator, except it will create the specified view. As an example, let’s create a new Rails app from scratch that uses Paperclip; you should be able to type in these precise commands on your machine and get this example to work.

First, let’s install View Mapper and create a new Rails app to display my MP3 library online (ignoring copyright issues for now):

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources
$ sudo gem install view_mapper
Successfully installed view_mapper-0.2.0
1 gem installed
Installing ri documentation for view_mapper-0.2.0...
Installing RDoc documentation for view_mapper-0.2.0...
$ rails music
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
etc…

And now we can generate a new “Song” model that has a Paperclip attachment called “MP3” using View Mapper like this:

$ cd music
$ ./script/generate scaffold_for_view song name:string artist:string
                    album:string play_count:integer --view paperclip:mp3
       error  The Paperclip plugin does not appear to be installed.

Wait… I forgot to install Paperclip; let’s do that and then try again:

$ ./script/plugin install git://github.com/thoughtbot/paperclip.git
Initialized empty Git repository in /Users/pat/rails-apps/music/vendor/plugins/paperclip/.git/
remote: Counting objects: 71, done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 71 (delta 7), reused 29 (delta 3)
Unpacking objects: 100% (71/71), done.
From git://github.com/thoughtbot/paperclip
 * branch            HEAD       -> FETCH_HEAD
$ ./script/generate scaffold_for_view song name:string artist:string
                    album:string play_count:integer --view paperclip:mp3
      exists  app/models/
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/songs
      exists  app/views/layouts/
      exists  test/functional/
etc…

Finally, we just need to run db:migrate – one minor detail here is that the scaffold_for_view generator included the Paperclip columns (“mp3_file_name,” “mp3_content_type,” etc…) in the migration file to create the songs table:

$ rake db:migrate
(in /Users/pat/rails-apps/music)
==  CreateSongs: migrating ====================================================
-- create_table(:songs)
   -> 0.0022s
==  CreateSongs: migrated (0.0024s) ===========================================

Now you can run your app and see the scaffolding UI I showed above, and will be able to upload and download MP3 files using Paperclip.

Let’s take a quick look at exactly what is different about the scaffolding code View Mapper generated vs. the standard Rails scaffolding code:

<h1>New song</h1>

<% form_for(@song, :html => { :multipart => true }) do |f| %>
  <%= f.error_messages %>
  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>

… etc …

  <p>
    <%= f.label :mp3 %><br />
    <%= f.file_field :mp3 %>
  </p>
  <p>
    <%= f.submit 'Create' %>
  </p>
<% end %>

<%= link_to 'Back', songs_path %>

The code in bold was generated by View Mapper specifically to support Paperclip since we used the “--view paperclip” command line option. You can see that “:html => { :multipart => :true }” was added to form_for to allow for file uploads, and also a file_field was added for the mp3 Paperclip attachment.

If you take a look at the show view, you’ll see:

<p>
  <b>Name:</b>
  <%=h @song.name %>
</p>

… etc …

<p>
  <b>Mp3:</b>
  <%= link_to @song.mp3_file_name, @song.mp3.url %><br>
</p>

<%= link_to 'Edit', edit_song_path(@song) %> |
<%= link_to 'Back', songs_path %>

Here a link to the file attachment was added, using Paperclip to provide the name and URL of the attachment.

Next I’ll be adding support for nested attributes and complex forms to View Mapper.

View Mapper: Scaffolding for your models and plugins

11 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

View Mapper will generate scaffolding illustrating how to write view code using a specified plugin or feature with your existing models. It can also generate new models.

A couple simple examples:

script/generate view_for office --view auto_complete:address

… will generate Rails scaffolding code that displays a form for an existing “office” model, with auto complete on the “address” field.

script/generate scaffold_for_view office address:string code:string
                                  --view auto_complete:address

… will generate the same form, but also create the “office” model class file, a migration file containing the “address” and “code” columns, and other standard scaffolding files as well.

The idea behind View Mapper is that it’s easy to write simple, concise model classes representing your domain objects using ActiveRecord, but very hard to implement the corresponding views using a combination of HTML, Javascript Rails helper functions, routes, controllers, etc. If you’re not very familiar with a certain plugin you want to use in your app, View Mapper can help you get started in the right direction by generating a working example with scaffolding code.

If you’re developing a Rails plugin or gem it’s easy to write your own View Mapper module for your plugin’s users to call with View Mapper.

Code: http://github.com/patshaughnessy/view_mapper

Install:

sudo gem install view_mapper

Usage:

Two generators are provided, called view_for and scaffold_for_view:

script/generate view_for model [ --view view_name:param ]

This will generate the specified view code for an existing model. The view_for generator will look for your model, inspect all of its columns and then generate standard Rails scaffolding containing a form field for each existing column.

If you also specify a view, then a custom view will be created using the specified Rails feature or plugin, using the specified parameter.

script/generate scaffold_for_view model attributes [ --view view_name:param ]

If you don’t specify a view, then this command is identical to the standard Rails scaffold generator.

If you do specify a view, then the entire working set of a model, views and controller will be generated to implement the specified Rails feature or plugin, using the specified parameter.

Views:

Right now, I’ve implemented eight views:

auto_complete: Uses the standard Rails auto_complete plugin to implement type ahead behavior for the specified field.
paperclip: Uses the Paperclip plugin to upload and download file attachments.
has_many: Displays a complex form to edit two or more associated models.
has_many_auto_complete: This is the same as has_many but also uses the auto_complete plugin to implement type ahead behavior for each text field. This view requires you to install my fork of the Rails auto_complete plugin.
belongs_to: Generates scaffolding that allows you to select an existing, associated model.
belongs_to_auto_complete: Generates scaffolding that allows you to select an existing, associated model using auto_complete.
has_many_existing: Generates scaffolding for a complex form to edit two models that have a has_many, :through association with a third model. Use this if you have a many-many relationship with existing data.
(Default) If no view is specified, then standard Rails scaffold code will be generated.

I’ll be implementing more views in the coming weeks and months. There is also an API for implementing your own View Mapper module, for example to generate code illustrating how to use a plugin or gem you are working on. In the future I’ll document this as well.

Auto_complete scaffolding

11 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

I’ve written a lot here about the Rails auto_complete plugin; I’ve also refactored the auto_complete plugin to support repeated fields and named scopes. Today I’d like to show how you can automatically generate Rails view and controller code with auto_complete behavior for one of your models using a new gem I’ve written called View Mapper. If you’ve never used the auto_complete plugin before this is a great way to learn quickly how to use it in your app; even if you are familiar with the plugin using scaffolding like this can help to get a working auto_complete form up and running quickly and let you concentrate on more important parts of your app.

Let’s say you have an existing model in your app called “Person:”

Class Person < ActiveRecord::Base
end

And suppose the Person model has two string attributes for the person’s name and the name of the office they work in:

class CreatePeople < ActiveRecord::Migration
  def self.up
    create_table :people do |t|
      t.string :name
      t.string :office
      etc…

Now let’s install View Mapper so we can generate an auto_complete view for our person model. Since I’ve only deployed view_mapper on gemcutter.org for now, you’ll also need to add gemcutter as a gem source if you haven’t already.

$ gem sources -a http://gemcutter.org
http://gemcutter.org added to sources
$ sudo gem install view_mapper
Successfully installed view_mapper-0.1.0
1 gem installed
Installing ri documentation for view_mapper-0.1.0...
Installing RDoc documentation for view_mapper-0.1.0...

Now with view_mapper you can run a single command to generate scaffolding that displays the your existing person model’s fields in a form with auto_complete type ahead behavior for the office field:

$ ./script/generate view_for person --view auto_complete:office
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/people
      exists  app/views/layouts/
      exists  test/functional/
      exists  test/unit/
      create  test/unit/helpers/
      exists  public/stylesheets/
      create  app/views/people/index.html.erb
      create  app/views/people/show.html.erb
      create  app/views/people/new.html.erb
      create  app/views/people/edit.html.erb
      create  app/views/layouts/people.html.erb
      create  public/stylesheets/scaffold.css
      create  app/controllers/people_controller.rb
      create  test/functional/people_controller_test.rb
      create  app/helpers/people_helper.rb
      create  test/unit/helpers/people_helper_test.rb
       route  map.resources :people
       route  map.connect 'auto_complete_for_person_office',
                          :controller => 'people',
                          :action => 'auto_complete_for_person_office'

This works just like the Rails scaffold generator, except that the view_for generator has also:

inspected your person model and found the name and office columns.
added a route for “auto_complete_for_person_office” to routes.rb.
added a call to “auto_complete_for :person, :office” to PersonController.
used text_field_for_auto_complete on the office field in your new and edit forms.
inserted “javascript_include_tag :defaults” into views/layouts/people.html.erb to load the prototype javascript library.

If you start up your application and create a few person records with names and addresses, then you will see the auto_complete plugin working!

With this working example right inside your application, you can easily review exactly how the view, route and controller files use auto_complete. After that you can adapt the view to fit into your application’s design and delete the scaffolding you don’t really need or want.

As another example, let’s create an entirely new Rails application completely from scratch, and use View Mapper to setup auto_complete inside it:

$ rails auto_complete_example
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
      etc…
$ cd auto_complete_example

First, let’s install the auto_complete plugin. (In a future post I’ll show how to use View Mapper with my fork of auto_complete in a complex form.)

$ ./script/plugin install git://github.com/rails/auto_complete.git
Initialized empty Git repository in /Users/pat/rails-apps/auto_complete_example/vendor/plugins/auto_complete/.git/
remote: Counting objects: 13, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 13 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (13/13), done.
From git://github.com/rails/auto_complete
 * branch            HEAD       -> FETCH_HEAD

Now that we have the plugin installed, let’s create our scaffolding. Along with the “view_for” generator I used above, View Mapper also provides a generator called “scaffold_for_view.” This works the same way, except it just creates a new model the same way the Rails scaffold generator does, instead of inspecting an existing model.

Let’s create the same person model we used above, and an auto_complete view:

$ ./script/generate scaffold_for_view person name:string office:string
                                      --view auto_complete:office
      exists  app/models/
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/people
      exists  app/views/layouts/
      exists  test/functional/
      exists  test/unit/
      create  test/unit/helpers/
      exists  public/stylesheets/
      create  app/views/people/index.html.erb
      create  app/views/people/show.html.erb
      create  app/views/people/new.html.erb
      create  app/views/people/edit.html.erb
      create  app/views/layouts/people.html.erb
      create  public/stylesheets/scaffold.css
      create  app/controllers/people_controller.rb
      create  test/functional/people_controller_test.rb
      create  app/helpers/people_helper.rb
      create  test/unit/helpers/people_helper_test.rb
       route  map.resources :people
  dependency  model
      exists    app/models/
      exists    test/unit/
      exists    test/fixtures/
      create    app/models/person.rb
      create    test/unit/person_test.rb
      create    test/fixtures/people.yml
      create    db/migrate
      create    db/migrate/20091001161349_create_people.rb
       route  map.connect 'auto_complete_for_person_office',
                          :controller => 'people',
                          :action =>'auto_complete_for_person_office'

Note the syntax is the same as the standard Rails scaffold generator, except I’ve added the “view” parameter to specify we want the auto_complete plugin to be used in the view.

Now we just need to migrate the database schema and run our app:

$ rake db:migrate
(in /Users/pat/rails-apps/auto_complete_example)
==  CreatePeople: migrating ===================================================
-- create_table(:people)
   -> 0.0012s
==  CreatePeople: migrated (0.0015s) ==========================================

And if you add a few records, you’ll see auto_complete working!

I’ll be adding more views to View Mapper soon, and in future posts here I’ll write about how to generate scaffolding for Paperclip file attachments, my version of auto_complete used on a complex form, and also how to write your own views for View Mapper.

Rails generator tutorial part 2: writing a custom manifest action

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

In part one of this tutorial I wrote a simple rails generator that creates a controller code file in app/controllers. This time I want to finish up my simple “VersionGenerator” example by enabling it to add a line to the routes.rb file. In other words, I want to be able to use a manifest action called “route” like this:

def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
      m.route :name => 'version',
              :controller => 'version',
              :action => 'display_version'
    end
  end

The problem for today is how to implement the “route” action, which is not supported by Rails by default. The Rails generator base class code does provide a few other actions you can use in your manifest, like “file” or “directory,” which create a file or directory. Rails even provides a method called “route_resources” to add a “map.resources” line to routes.rb, but not a simple named route line like what we need for this example.

This specific action might be useful for you, but my real goal is to explore how Rails generators work in general and discuss some of the issues you might run into if you need a custom manifest action like this for any purpose. Let’s get started by just adding the new action method we need right inside our generator class…

def route(route_options)
  puts "This will create a new route!"
end

…and see what happens when we run script/generate with manifest method above:

$ ./script/generate version ../build-info/version.txt 
   identical  app/controllers/version_controller.rb
This will create a new route!

Very promising! All we need to do is add the necessary code here and we’ll be all set. But first let’s take a closer look at what’s happening by adding a call to “caller” inside this new method and see how it’s being called by the Rails generator system:

def route(route_options)
  puts "This will create a new route!"
  puts caller
end

Here’s what is displayed when we run this (paths shortened for readability):

$ ./script/generate version ../build-info/version.txt
   identical  app/controllers/version_controller.rb
This will create a new route!
/usr/local/lib/ruby/1.8/delegate.rb:270:in `__send__'
/usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:47:in `send'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:47:in `send_actions'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:46:in `each'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:46:in `send_actions'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:31:in `replay'
/path/to/rails-2.3.2/lib/rails_generator/commands.rb:42:in `invoke!'
/path/to/rails-2.3.2/lib/rails_generator/scripts/../scripts.rb:31:in `run'
/path/to/rails-2.3.2/lib/commands/generate.rb:6
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
./script/generate:3

There’s a lot going on here; in fact, when I started to learn more about Rails generators and how they work I was shocked at how complicated their object model and design are. I won’t try to explain all of it here, but let’s take a closer look at the one line I bolded above: line 31 of lib/rails_generator/scripts.rb

Rails::Generator::Base.instance(options[:generator], args, options)
                      .command(options[:command]).invoke!

This is called when you run script/generate, by the “Rails::Generator::Scripts::Generate” class. Translating this line of Ruby into English, we get: “Create an instance of the specified generator, passing in the arguments and options provided on the command line; then create an instance of the specified command that uses that generator; then invoke the command.”

What this means is that actually Rails generator classes do not do the real work of generating code – Rails generator “commands” do! The lib/rails_generator/commands.rb file defines a series of command objects inside the Rails::Generator::Commands module. The two most important command classes are Create and Destroy. The “Create” command is called when you execute script/generate, like what happened in the stack trace above. This class contains the implementation of the “template” method we used before, plus various other methods like “file” and “directory” that create files and directories, etc.

The “Destroy” command is called when you execute script/destroy from the command line (I removed “puts caller” first before running this):

$ ./script/destroy version ../build-info/version.txt 
This will create a new route!
          rm  app/controllers/version_controller.rb

This class has all of the same methods that the Create class does, except that they perform the opposite action: instead of creating a file or directory, they delete them. The Destroy class also executes the actions in the opposite order, by looping backwards through the manifest that we recorded in our generator. Here the destroy command has deleted our version_controller.rb file we generated earlier. Note that it still displays “This will create a new route!”

If you’re interesting in diving into the real details about how Rails generators work, then read the code in commands.rb very carefully. In particular, look at how the “self.included” and “self.instance” methods at the top work; these methods are used in the line I showed above to create the specified command, supplying the specified generator as an argument to the command constructor. The “invoke!” method on commands actually plays back the recorded manifest. Also all of the actions that are available to your generator's manifest method are defined in this file. One other interesting detail I don’t have space to explain carefully here is that the command objects contain the corresponding generator class as a delegate object; in other words they contain an instance of the generator as a member variable:

# module Rails::Generator::Commands...
class Base < DelegateClass(Rails::Generator::Base)

This explains the top two lines in the stack trace above:

/usr/local/lib/ruby/1.8/delegate.rb:270:in `__send__'
/usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'

“DelegateClass” is defined by delegate.rb, which is a Ruby library that implements the delegate pattern. This is why we were able to add the new “route” method right in our generator; when Ruby found the “route” method missing in the Create command, it delegated the call to our generator object.

Obviously we have a problem here: our new “route” method needs to be able to remove a route from routes.rb if the Destroy command is called, as well as create a new one for the Create command. One way to implement our new “route” method would be check the “options[:command]” value that we saw above on line 31 of scripts.rb, like this:

def route(route_options)
  if options[:command] == :create
    puts "This will add a new route to routes.rb."
  elsif options[:command] == :destroy
    puts "This will remove the new route from routes.rb."
  end
end

Now if we run script/generate again, our new method will create a route:

$ ./script/generate version ../build-info/version.txt
      create  app/controllers/version_controller.rb
This will add a new route to routes.rb.

And if we run the destroy command, we’ll remove the route:

$ ./script/destroy version ../build-info/version.txt 
This will remove the new route from routes.rb.
          rm  app/controllers/version_controller.rb

It turns out a cleaner way to implement the “route” method is to directly add the method to both the Create and Destroy command classes. This allows me to call a utility method called “gsub_file” in the Rails::Generator::Commands::Base class which I wouldn’t have direct access to from my generator class. It also avoids the somewhat ugly if statement on the options[:command] value, and finally it might make it easier for me someday to refactor the new route methods into a separate module that I could use with various different generators that might need to add and remove routes.

Anyway, here’s the finished code for the entire generator:

class VersionGenerator < Rails::Generator::NamedBase
  attr_reader :version_path
  def initialize(runtime_args, runtime_options = {})
    super
    @version_path = File.join(RAILS_ROOT, name)
  end
  def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
      m.route :name => 'version',
              :controller => 'version',
              :action => 'display_version'
    end
  end
end

module Rails
  module Generator
    module Commands

      class Base
        def route_code(route_options)
          "map.#{route_options[:name]} '#{route_options[:name]}', :controller => '#{route_options[:controller]}', :action => '#{route_options[:action]}'"
        end
      end

# Here's a readable version of the long string used above in route_code;
# but it should be kept on one line to avoid inserting extra whitespace
# into routes.rb when the generator is run:
# "map.#{route_options[:name]} '#{route_options[:name]}',
#     :controller => '#{route_options[:controller]}',
#     :action => '#{route_options[:action]}'"

      class Create
        def route(route_options)
          sentinel = 'ActionController::Routing::Routes.draw do |map|'
          logger.route route_code(route_options)
          gsub_file 'config/routes.rb', /(#{Regexp.escape(sentinel)})/mi do |m|
              "#{m}\n  #{route_code(route_options)}\n"
          end
        end
      end

      class Destroy
        def route(route_options)
          logger.remove_route route_code(route_options)
          to_remove = "\n  #{route_code(route_options)}"
          gsub_file 'config/routes.rb', /(#{to_remove})/mi, ''
        end
      end

    end
  end
end

More or less copied from the existing route_resources action found in commands.rb, this works as follows: If we call script/generate then the route action method in the Create command class is called. This uses gsub_file to look for the “sentinel” or target area of the file and replaces it with the sentinel + the new route code. I also use the “logger” method to display log messages using the Rails::Generator::SimpleLogger class. Here’s what it looks like on the command line:

$ ./script/generate version ../build-info/version.txt
      create  app/controllers/version_controller.rb
       route  map.version 'version',
                 :controller => 'version',
                 :action => 'display_version'

If script/destroy is called, then the second route implementation in the Destroy class is called. This uses gsub_file to remove the route code. Finally, the “route_code” method returns the route code that we want to generate or remove from routes.rb. This time on the command line we get:

$ ./script/destroy version ../build-info/version.txt 
remove_route  map.version 'version',
                 :controller => 'version',
                 :action => 'display_version'
          rm  app/controllers/version_controller.rb

Tutorial: How to write a Rails generator

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

I’ve been working on a generator called ViewMapper recently that allows you to create view scaffolding from an existing model. I found writing a Rails generator to be somewhat confusing and hard to do: Where do I need to put my generator class? What does it need to be called? How does it work? This article will show step by step how to write your own Rails generator from scratch – hopefully it will save you some time if you ever need to write your own.

First let's think of some sample Rails code that I might want to generate as an example. This is admittedly contrived, but it’s short and simple enough to show here while still interesting enough to illustrate how a generator would work. Let’s suppose my Capistrano deployment scripts wrote the last commit information from Git into a file called “version.txt” in a folder RAILS_ROOT/../build-info:

commit 217d9bd1d1d508a0f7f1e7afa2b489b130196e33
Author: Pat Shaughnessy <pat@patshaughnessy.net>
Date:   Wed Aug 5 07:45:01 2009 -0400

Now with a controller like this I could display the info, helping me keep track of what version is running on a given development development, QA or production server:

class VersionController < ApplicationController
  VERSION_FILE = '../build-info/version.txt'
  def display_version
    path = File.join(RAILS_ROOT, VERSION_FILE)
    render path
  end
end

If I add a route to this action like this in config/routes.rb:

map.version 'version', :controller => 'version', :action => 'display_version'

…then I’ll get the Git last commit info by just opening http://servername/version in my browser. Obviously it would be simpler to just put the version.txt file under the public folder… but let’s continue with this contrived example for now. Now let’s say I have 5 or 10 different Rails apps, and that I’d like to add the same controller and route to each one. Let’s write a simple Rails generator that would make this easy to do, called the “version” generator. Here’s how to do it…

<u>Step 1: A skeleton Rails generator</u>

All Rails generators are derived from a class called “Rails::Generator::Base.” You can find the source code for this file in vendor/rails/railties/lib/rails_generator/base.rb (link is to the Rails 2.3.3 version). Definately read the source code; there is helpful documentation in the comments and understanding the base class code is indispensible if you plan to use it for your generator.

Here’s an empty, skeleton Rails generator that we can use as a starting point:

class VersionGenerator < Rails::Generator::Base
  def manifest
    record do |m|
      # Do something
    end
  end
end

The name of the class is important: since we want a “version” generator we need to create a class called “VersionGenerator.” The other important detail here is where this class is located: we need to put it in lib/generators/version/version_generator.rb. The reason for all of this is that the code called by script/generate takes the generator name and looks for the corresponding generator class in this location. We can check that our class is being found by just running script/generate with no parameters, like this:

$ ./script/generate 
Usage: ./script/generate generator [options] [args]
…etc…
Installed Generators
  Lib: version
  Rubygems: cucumber, feature, install_rubigen_scripts, integration_spec, rspec, rspec_controller, rspec_model, rspec_scaffold, session
  Builtin: controller, helper, integration_test, mailer, metal, migration, model, observer, performance_test, plugin, resource, scaffold, session_migration

Note that our new “version” generator is listed as “Lib: version,” in bold above. The other generators found on my system were either part of Rails or else part of a gem or plugin. If you don’t see your generator in this list, or if it has the wrong name then double check your folder name: it should be lib/generators/XYZ or lib/generators/version in this example. After you finish developing your generator you'll want to move it into a plugin or gem so it can be reused in different apps on your machine; then "version" would appear in the "Rubygems" list for example.

If we try running our empty generator, we’ll get no output or changes; not a surprise:

$ ./script/generate version

If you see an error like this:

$ ./script/generate version
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:
in `gem_original_require': no such file to load --
/path/to/myapp/lib/generators/version/version_generator.rb (MissingSourceFile)

Then you have the wrong file name… it needs to be XYZ_generator.rb, or version_generator.rb in this example. Or if you see:

Missing VersionGenerator class in
  /path/to/myapp/lib/generators/version/version_generator.rb

… then you have the wrong class name inside this file. It needs to be XYZGenerator, or VersionGenerator in this example.

<u>Step 2: Creating code using an ERB template</u>

The way Rails generators work is by creating code using ERB, in just the same way that view code generates HTML for a Rails web site. To see how this works and to take the next step towards writing our version generator, let’s create a new template file for our version controller. Create a new text file called “controller.rb” and put it into this folder: lib/generators/version/templates/controller.rb; in other words, into a new subfolder under version called “templates.” Next, paste in the controller code from above:

class VersionController < ApplicationController
  VERSION_FILE = '../build-info/version.txt'
  def display_version
    path = File.join(RAILS_ROOT, VERSION_FILE)
    render path
  end
end

Now if we add this line to our manifest method back in the VersionGenerator class:

def manifest
  record do |m|
    m.template('controller.rb', 'app/controllers/version_controller.rb')
  end
end

… a new code file called version_controller.rb will be generated inside of the app/controllers folder in our app when we run the generator:

$ ./script/generate version
    create  app/controllers/version_controller.rb

If you take a look at this new version_controller.rb file, you’ll just see the code we pasted into the controller.rb template file. Here’s what happened when we ran script/generate version:

The Rails code called by script/generate looked for the “version” generator, by looking for a class called “VersionGenerator” in the lib/generators/version folder, among other places.
It called the “manifest” method in VersionGenerator. Using the “record” utility method, a series of actions are recorded that the Rails generator base classes will execute later. In our case, there’s only one action: “template.”
The template action indicates that the Rails generator code should run an ERB transformation to generate new code, using the “controller.rb” file. The generated code will then be copied to the app/controllers/version_controller.rb file.

To make our generator interesting, let’s provide a parameter to it that indicates the path of the file containing the version information, instead of hard coding “version.txt.” To do that we can take advantage of another Rails built in class called “Rails::Generator::NamedBase”. If we derive our VersionGenerator class from NamedBase instead of Base, then we’ll get the ability to take a name parameter from the script/generate command line for free. Let start by changing our base class in version_generator.rb:

class VersionGenerator < Rails::Generator::NamedBase

Now if you run the generator, you’ll get some helpful usage information:

$ ./script/generate version
Usage: ./script/generate version VersionName [options]
Etc…

Here the NamedBase class has written the usage info, explaining that an additional parameter is now expected. If we re-run the generator and add the version file name as a parameter, here’s what we’ll get:

$ ./script/generate version ../build-info/version.txt
   identical  app/controllers/version_controller.rb

This shows a helpful feature of Rails generators: they check if you’re about to overwrite some existing code with the new, generated code. In this case, since we already had a controller called “version_controller.rb” the Rails generator code displayed the “identical” message. Our new generated controller is identical to the previous one since we haven’t used the version name parameter yet. To take advantage of the version file name parameter and our new NamedBase base class, we need to add some new code to VersionGenerator, in bold:

class VersionGenerator < Rails::Generator::NamedBase
  attr_reader :version_path
  def initialize(runtime_args, runtime_options = {})
    super
    @version_path = File.join(RAILS_ROOT, name)
  end
  def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
    end
  end
end

The way this works is:

The NamedBase base class parses the script/generate command line, expecting an additional parameter which indicates the name of some object.
This name is provided in "name," an instance variable/attribute of the NamedBase class.
The VersionGenerator class assumes "name" indicates the relative path of a version file, determines the full path of the file and then saves the full path in the @version_path attribute.

I mentioned above that Rails generators work by running an ERB transformation; to try using ERB in this example, we just need to refer to the new “version_path” attribute inside the controller.rb template file, like this:

class VersionController < ApplicationController
  VERSION_PATH = '<%= version_path %>'
  def display_version
    render VERSION_PATH
   end
end

The “version_path” variable in this template refers to the version_path attribute of the VersionGenerator class above. I’ve also removed the File.join line from here since we now handle that in the generator itself. If we run the generate command and provide the name of the version file, we’ll get this result:

$ ./script/generate version ../build-info/version.txt
overwrite app/controllers/version_controller.rb? (enter "h" for help) [Ynaqdh] Y
       force  app/controllers/version_controller.rb

As I mentioned above, Rails checks whether you’re about to overwrite some existing code file. Since in this case our controller code file is now different that what we had before (VERSION_PATH and not VERSION_FILE; no File.join, etc.) we get an overwrite warning. I just entered “Y” to overwrite the old file.

If we look at our new controller, app/controllers/version_controller.rb, we now have:

class VersionController < ApplicationController
  VERSION_PATH = '/path/to/myapp/../build-info/version.txt'
  def display_version
    render VERSION_PATH
   end
end

We’ve successfully generated a new VersionController class, based on a parameter passed to our “version” generator. While this is a good start, to be able to use this action in our app we still need to add a route to config/routes.rb. It turns out this is a lot harder to do with a generator, since we will need to insert a new line of code in an existing file, and not just generate a new routes.rb file. We will also need to worry about how to remove the route line when script/destroy is run. It’s possible to do all of this with a Rails generator, but will require a more thorough understanding of how Rails generators actually work. I’ll explain all of that in my next post…

Generating view scaffolding code for existing models

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

(Updated October 2009)

I just rewrote and cleaned up the code I describe below here and published it as a new gem called View Mapper on gemcutter.org. See my usage article for more details. I also rethought and redesigned the commands I describe below… the part about nested attributes will not work for now. I’m planning to reimplement that soon.

I’ve been thinking for a while that a generator that creates view scaffolding for an existing model or models would be really useful. For example, I want to be able to write this by hand:

class Group < ActiveRecord::Base
  has_many :people
end
class Person < ActiveRecord::Base
  belongs_to :group
end

… and then run a generator and get a working view to edit and display groups of people. Recently I started developing a generator called “ViewMapper” that does this; see: http://github.com/patshaughnessy/view_mapper.

The idea is that it creates a view that maps to your existing models. Writing model classes is often very easy; using ActiveRecord allows you to write concise, short code files. However, writing a view that displays a form for these models is often very hard, requiring knowledge of a long list of Rails functions, macros, classes, as well as the usual HTML and Javascript. Right now ViewMapper it just a plugin; so you can install it into your app like this:

$ ./script/plugin install git://github.com/patshaughnessy/view_mapper.git

Probably it should be packaged as a gem instead… this is still work-in-progress. At the moment it supports:

Creating simple form based on the columns of an existing model
Creating a complex form based on the columns of two associated models in a many-one relationship, using nested attributes

If you’re interested and have time, try the examples below or better yet try the generator on some of your own models and let me know what you think.

<u>Example 1:</u> Create a view based on the columns of an existing model

Let’s say I have an existing model class that manages a series of person records:

class Person < ActiveRecord::Base
end

Now to create the corresponding view with ViewMapper, just run this command:

$ ./script/generate view_for person
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/people
      exists  app/views/layouts/
      exists  test/functional/
      create  test/unit/helpers/
      exists  public/stylesheets/
      create  app/views/people/index.html.erb
      create  app/views/people/show.html.erb
      create  app/views/people/new.html.erb
      create  app/views/people/edit.html.erb
      create  app/views/layouts/people.html.erb
      create  public/stylesheets/scaffold.css
      create  app/controllers/people_controller.rb
      create  test/functional/people_controller_test.rb
      create  app/helpers/people_helper.rb
      create  test/unit/helpers/people_helper_test.rb
       route  map.resources :people

If you run your Rails app you will see the usual scaffolding UI:

This looks just like the scaffolding page you would have gotten from the Rails scaffolding generator. In fact, I’ve based the “view_for” generator class (ViewForGenerator) on the existing Rails ScaffoldGenerator class, so it generates the same code.

The difference here is that the scaffolding code was based on the properties of the existing Person model. Here’s what the view_for generator did:

Find the specified model class
Inspect it and get a list of column names and data types
Create the scaffolding code as usual

If you also want to create your model class at the same time, then you don’t need ViewMapper; you would just run the standard Rails scaffold generator like this:

$ ./script/generate scaffold person name:string age:integer

ViewMapper provides you with the flexibility to create the view scaffolding after the fact for a model you or someone else has already created in your app.

<u>Example 2:</u> Create a view using a complex form based on two associated models in a many-one relationship, using Rails 2.3 nested attributes

Let’s suppose you have two related models that describe groups of people:

class Group < ActiveRecord::Base
  has_many :people
end
class Person < ActiveRecord::Base
  belongs_to :group
end

Again, writing model classes is really easy. With just a few lines of code I have told ActiveRecord to manage two separate tables and their relationship.

Now you can use ViewMapper to create a complex form for creating and editing groups and people at the same time like this:

$ ./script/generate nested_attributes_view_for group containing:people
       error  Model 'Group' does not accept nested attributes
              for child models 'people'

This error indicates that my Group model is missing the “accepts_nested_attributes_for” directive. Since the generator creates view code that relies on the nested attributes feature, it won’t let you create the view for a model that is missing this line. This sort of helpful error message is possible since ViewMapper is inspecting an existing model class, and not just creating new scaffold code.

Now if we add the missing line (suddenly my model is less concise... nested attributes aren't that easy to use yet!):

class Group < ActiveRecord::Base
  has_many :people
  accepts_nested_attributes_for :people,
    :allow_destroy => true,
    :reject_if => proc { |attrs| attrs['name'].blank? && attrs['age'].blank? }
end

…we can re-run the generator:

$ ./script/generate nested_attributes_view_for group containing:people
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/groups
      exists  app/views/layouts/
      exists  test/functional/
      create  test/unit/helpers/
      exists  public/stylesheets/
      create  app/views/groups/index.html.erb
      create  app/views/groups/show.html.erb
      create  app/views/groups/new.html.erb
      create  app/views/groups/edit.html.erb
      create  app/views/layouts/groups.html.erb
      create  public/stylesheets/scaffold.css
      create  app/controllers/groups_controller.rb
      create  test/functional/groups_controller_test.rb
      create  app/helpers/groups_helper.rb
      create  test/unit/helpers/groups_helper_test.rb
       route  map.resources :groups

And now if we run our app, we get view scaffolding illustrating how to create and edit people and groups all at the same time:

If I save the group…

And re-editing the same group:

More to come… as I said this is work-in-progress. As a next step I’m planning to improve the scaffolding code for nested attributes by adding javascript to dynamically add/delete the child objects. But after that I'm thinking about:

Adding a “paperclip_view_for” generator that creates a file upload form for a model that contains a “has_attached_file” directive.
Adding a “auto_complete_view_for” generator that adds “auto_complete_for” to the controller and uses “text_field_for_auto_complete” in the form.
Adding the ability for you to extend this with your own scaffolding code.

Taming the beast: Using JRuby and RSpec to test a J2EE application

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Working with J2EE applications is something like wandering in a jungle: you never quite know what wild animal you’ll find around the next corner… it might be an ORM tool like Hibernate; it could be an application framework like Spring or Struts with lots of confusing XML files, or it might just be a long list of obscure JAR files that you need to find and download… whatever it is, you’re guaranteed to spend countless hours wasting time learning things you really didn’t want to know. This article will show how you can get your J2EE application under control by using JRuby and RSpec… but first, let’s take a quick look at what RSpec is and how it’s normally used with Ruby.

Using RSpec with Ruby

If you are a Ruby developer, you would probably write code similar to this to add up an array of numbers:

class Calculator
  def self.sum numbers
    numbers.inject do |sum, x|
      sum+x
    end
  end
end

This is simple enough to run:

$ irb
irb(main):001:0> require 'calculator.rb'
=> true
irb(main):002:0> Calculator.sum [2, 3]
=> 5

One of the best things about Ruby are all of the different testing tools available to you – for example, you could write a test for this method using RSpec like this:

require 'calculator.rb'
describe "Calculator" do
  it "should add numbers correctly" do
    Calculator.sum([1, 2]).should == 3
    Calculator.sum([2, 2]).should == 4
    Calculator.sum([2, 3, 4]).should == 9
  end
end

And then run the spec as follows:

$ spec calculator_spec.rb 
.
Finished in 0.001703 seconds
1 example, 0 failures

RSpec allows you to write test code that is readable, and also behavior-oriented; that is, the tests reflect the way an end user might actually behave. In fact, RSpec is really just the first step towards behavior-driven-development. The Ruby community also benefits from other tools such as WebRat, Cucumber, etc., that can make testing very easy and effective.

A J2EE sample app

Let’s rewrite the “sum” Ruby method above using Java. To make this feel more like an actual J2EE application, we’ll use a service class that will perform the actual sum operation for us, and set it up with the Spring framework. We can start by writing an XML file called “ApplicationContext.xml” for Spring to use, and declare a bean called “calculatorService:”

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN"
  "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>
  <bean id="calculatorService"
    class="calculator.CalculatorServiceImpl">
  </bean>
</beans>

In a real J2EE app, we would probably have an interface called “CalculatorService” like this:

package calculator;
public interface CalculatorService {
  public int sum(int[] array);
}

And then we’d implement it using a concrete class, like this:

package calculator;
public class CalculatorServiceImpl implements CalculatorService {
  public int sum(int[] array)
  {
    int sum = 0;
    for (int number: array)
    {
      sum += number;
    }
    return sum;
  }
}

And finally, let’s write a simple Java command line client for this so we can test running it from the command line:

package calculator;
import org.springframework.context.support.ClassPathXmlApplicationContext;
public class CalculatorApp {
  public static void main(String[] args) throws Exception {
    int[] array = { 2, 3 };
    ClassPathXmlApplicationContext application_context =
      new ClassPathXmlApplicationContext("ApplicationContext.xml");
    CalculatorService calculator =
      (CalculatorService)application_context.getBean("calculatorService");
    System.out.println("2 + 3 is: " + Integer.toString(calculator.sum(array)));
  }
}

This will also help us figure out how to write the ruby spec later. If you run this from Eclipse or your favorite Java IDE, you’ll get:

Jun 25, 2009 12:21:58 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@3479e304: display name [org.springframework.context.support.ClassPathXmlApplicationContext@3479e304]; startup date [Thu Jun 25 12:21:58 EDT 2009]; root of context hierarchy
Jun 25, 2009 12:21:58 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [ApplicationContext.xml]
Jun 25, 2009 12:21:58 PM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@3479e304]: org.springframework.beans.factory.support.DefaultListableBeanFactory@604788d5
Jun 25, 2009 12:21:58 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@604788d5: defining beans [calculatorService]; root of factory hierarchy
2 + 3 is: 5

This is starting to feel like an actual “Enterprise” J2EE application now that we have an XML config file and lots of confusing information being logged!

Running RSpec with JRuby

Now that we have a J2EE application to test, let’s get started with JRuby. First, you will need to download and install JRuby. This is really just a matter of downloading the TAR file and then placing the JRuby bin folder on your path. Test that you have it setup properly by running this command:

$ jruby --version
jruby 1.1.5 (ruby 1.8.6 patchlevel 114) (2008-11-03 rev 7996) [x86_64-java]

Now let’s update our Ruby spec from above and get it to work with Java. Here’s what we had for Ruby:

require 'calculator.rb'
describe "Calculator" do
  it "should add numbers correctly" do
    Calculator.add([1, 2]).should == 3
    Calculator.add([2, 2]).should == 4
    Calculator.add([2, 3, 4]).should == 9
  end
end

The key to using JRuby to test Java is to add this line at the top of the spec:

require 'java'

This tells JRuby that we want to allow the Ruby code to call Java directly. But what code should we try to call? The nice thing about the Spring framework is that is makes a series of “beans,” i.e. Java objects, available to us. The reason I went to the trouble of adding the Spring framework to this sample app is that for a real J2EE application using Spring to create and load a Java object will be the simplest path towards testing your target application. What I did while testing a real J2EE application was to:

Identify what business logic I wanted to test
Look for the Java object that was the top-level, simplest interface to that business logic under the user interface layer
Find a bean in the ApplicationContext.xml file that corresponded to that Java object. In my case, there actually was no such bean, and I had to slightly modify the application I was testing by adding a new <bean> tag to one of the Spring XML files.

If your application is not using Spring, you might simply be able to create a Java object directly from your Ruby code, or in the worst case scenario you might have to make Java code changes to the target J2EE app to break dependencies that the object you’d like to test has on other objects, interfaces, services, etc., allowing you to create it in isolation. Michael Feathers has written an entire book on dependency breaking techniques.

Back to this sample app: here in our Ruby spec we can follow the same pattern that I used above in the command line Java client: calling “ClassPathXmlApplicationContext” to get the application context, and then creating a bean with getBean. The other thing we need to add are “include_class” directives that indicate to JRuby which Java classes should be loaded from the classpath and made available to the Ruby script. Here’s the spec code with the new JRuby code changes in bold:

require 'java'
include_class 'org.springframework.context.ApplicationContext'
include_class
  'org.springframework.context.support.ClassPathXmlApplicationContext'
describe "Calculator" do
  it "should add numbers correctly" do
    application_context =
      ClassPathXmlApplicationContext.new "ApplicationContext.xml"
    calculator =  application_context.getBean "calculatorService"
    calculator.sum([1, 2]).should == 3
    calculator.sum([2, 2]).should == 4
    calculator.sum([2, 3, 4]).should == 9
  end
end

Now let’s try to run it using JRuby. Another trick with JRuby is knowing how to use the “-S” option – this allows you to run a Ruby command like “gem” or “spec,” but inside a JRuby session. So here’s how to run our new spec using JRuby:

$ jruby -S spec calculator_spec.rb 
(eval):1:in `include_class': cannot load Java class
  org.springframework.context.ApplicationContext (NameError)
  from /Users/pat/src/jruby-1.1.5/lib/ruby/site_ruby/1.8/builtin/javasupport/core_ext/object.rb:38:in `eval'
  from /Users/pat/src/jruby-1.1.5/lib/ruby/site_ruby/1.8/builtin/javasupport/core_ext/object.rb:67:in `include_class'
  from /Users/pat/src/jruby-1.1.5/lib/ruby/site_ruby/1.8/builtin/javasupport/core_ext/object.rb:38:in `each'
  from /Users/pat/src/jruby-1.1.5/lib/ruby/site_ruby/1.8/builtin/javasupport/core_ext/object.rb:38:in `include_class'
  from calculator_spec.rb:2

So what is the error message all about? This just means that JRuby wasn’t able to find the ApplicationContext class from Spring on the classpath. But what is the classpath anyway? It would be nice to be able to simply specify the classpath using a command line option, the way you do with a java command line using “-cp” for example. But for JRuby you need to specify the classpath as an environment setting. To make this easier, I wrote a simple shell script for running a JRuby spec:

BASE=`pwd`
CLASSPATH=$BASE/lib/spring-2.5.1.jar
export CLASSPATH
jruby -S spec $1

This just gets the current working directory and constructs the classpath setting, indicating where to find the Spring JAR file. Finally we export the classpath value and call JRuby. This will work on the Mac and Linux; for Windows you would need something slightly different. Anyway, now I can run my specs like this:

$ ./jruby-spec.sh calculator_spec.rb 
F
1)
NativeException in 'Calculator should add numbers correctly'
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
org/springframework/util/ClassUtils.java:73:in `<clinit>'
org/springframework/core/io/DefaultResourceLoader.java:52:in `<init>'
org/springframework/context/support/AbstractApplicationContext.java:198:in `<init>'
org/springframework/context/support/AbstractRefreshableApplicationContext.java:80:in `<init>'
org/springframework/context/support/AbstractXmlApplicationContext.java:58:in `<init>'
org/springframework/context/support/ClassPathXmlApplicationContext.java:119:in `<init>'
org/springframework/context/support/ClassPathXmlApplicationContext.java:66:in `<init>'
sun/reflect/NativeConstructorAccessorImpl.java:-2:in `newInstance0'
sun/reflect/NativeConstructorAccessorImpl.java:39:in `newInstance'
sun/reflect/DelegatingConstructorAccessorImpl.java:27:in `newInstance'
java/lang/reflect/Constructor.java:513:in `newInstance'
org/jruby/javasupport/JavaConstructor.java:226:in `new_instance'
org/jruby/java/invokers/ConstructorInvoker.java:100:in `call'
org/jruby/java/invokers/ConstructorInvoker.java:180:in `call'
etc...

Oops – it turns out that Spring actually requires the Apache Commons logging JAR file to be on the classpath also. This sample app is definitely reminding me of a complex J2EE app running on WebSphere or WebLogic! Let’s add commons-logging-1.0.4.jar to the classpath in my shell script and try again:

$ ./jruby-spec.sh calculator_spec.rb 
Jun 25, 2009 1:58:15 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@4c6c3e: display name [org.springframework.context.support.ClassPathXmlApplicationContext@4c6c3e]; startup date [Thu Jun 25 13:58:15 EDT 2009]; root of context hierarchy
Jun 25, 2009 1:58:15 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [ApplicationContext.xml]
F
1)
NativeException in 'Calculator should add numbers correctly'
org.springframework.beans.factory.BeanDefinitionStoreException:
  IOException parsing XML document from class path resource
  [ApplicationContext.xml]; nested exception is java.io.FileNotFoundException:
  class path resource [ApplicationContext.xml] cannot be opened because
  it does not exist
org/springframework/beans/factory/xml/XmlBeanDefinitionReader.java:334:in `loadBeanDefinitions'
org/springframework/beans/factory/xml/XmlBeanDefinitionReader.java:295:in `loadBeanDefinitions'

Once again, I’ve forgotten something in my class path: this time it’s the ApplicationContext.xml file which I passed into ClassPathXmlApplicationContext in the spec. As the name implies, I need to put the XML file on the classpath in order for Spring to be able to find it. Let’s cut to the chase and put everything I need onto the classpath… here’s the final version of jruby-spec.sh (the classpath line split into two for readability):

BASE=`pwd`
CLASSPATH=$BASE/bin:$BASE/resources:$BASE/lib/spring-2.5.1.jar:
  $BASE/lib/commons-logging-1.0.4.jar
export CLASSPATH
jruby -S spec $1

Now JRuby will be able to find everything that it needs: Spring, Apache Common Logging, the ApplicationContext.xml file, and also the application’s class files saved under “bin” by Eclipse. Now if I run the spec once more it should all work:

$ ./jruby-spec.sh calculator_spec.rb 
Jun 25, 2009 2:02:22 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1717d968: display name [org.springframework.context.support.ClassPathXmlApplicationContext@1717d968]; startup date [Thu Jun 25 14:02:22 EDT 2009]; root of context hierarchy
Jun 25, 2009 2:02:22 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [ApplicationContext.xml]
Jun 25, 2009 2:02:22 PM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@1717d968]: org.springframework.beans.factory.support.DefaultListableBeanFactory@5a21fdc8
Jun 25, 2009 2:02:22 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@5a21fdc8: defining beans [calculatorService]; root of factory hierarchy
F
1)
TypeError in 'Calculator should add numbers correctly'
for method sum expected [[I]; got: [org.jruby.RubyArray];
  error: argument type mismatch calculator_spec.rb:8:
Finished in 0.70559 seconds
1 example, 1 failure

What’s this error all about? I do have the correct classpath now, but what does “RubyArray” mean? I thought I passed an array of integers into the calculator service:

calculator.sum([1, 2]).should == 3

Well it turns out that Ruby and JRuby aren’t quite the same thing. Since JRuby is actually a Java application itself, it implements each Ruby class with a corresponding Java class. For the Ruby Array class, JRuby has created a class called “org.jruby.RubyArray.” When you call Java code and pass in Ruby objects, JRuby actually provides the Java equivalent of these Ruby objects to the Java code. That’s why we get an error here; our calculator service doesn’t expect a RubyArray – it expects int[] instead.

To avoid this error, we need to convert the RubyArray into a normal Java array, using a JRuby method called “to_java()”, like this:

calculator.sum([1, 2].to_java(:int)).should == 3

To_java takes a symbol as a parameter, which indicates what type each element of the Ruby array should be converted into. Now our spec will pass!

$ ./jruby-spec.sh calculator_spec.rb 
Jun 25, 2009 2:15:37 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1717d968: display name [org.springframework.context.support.ClassPathXmlApplicationContext@1717d968]; startup date [Thu Jun 25 14:15:37 EDT 2009]; root of context hierarchy
Jun 25, 2009 2:15:37 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [ApplicationContext.xml]
Jun 25, 2009 2:15:37 PM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.springframework.context.support.ClassPathXmlApplicationContext@1717d968]: org.springframework.beans.factory.support.DefaultListableBeanFactory@5a21fdc8
Jun 25, 2009 2:15:37 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@5a21fdc8: defining beans [calculatorService]; root of factory hierarchy
.
Finished in 0.28237 seconds
1 example, 0 failures

Taking a step back, I think what we’ve done is quite interesting; we have:

Pulled a J2EE application out of Eclipse, IntelliJ or whatever IDE you project uses, and started to run it from the command line instead. For me, avoiding a confusing and complex IDE makes the application easier to work with… more like a Rails app.
Documented what the classpath needs to be and what JAR files the application requires in an understandable text file format, rather than having it hidden away in a confusing Eclipse dialog box.
Exposed the J2EE application’s business logic to testing below the user interface layer – nothing here involves a web browser or user interface testing tool like Selenium.
Started to apply behavior driven development to a J2EE application, using the best tools available, which happen to be written in Ruby.

In a future post, I’ll try to take the next step of using Cucumber features to test this same J2EE sample app. It will be interesting to find out if Cucumber works will with JRuby.

Auto complete for complex forms using nested attributes in Rails 2.3

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

I just updated my fork of the auto_complete plugin to support Rails 2.3 nested attributes. Thanks to Anthony Frustagli for the code and ideas that I used as the basis for this fix.

<u>Basic usage:</u>

To use auto_complete on a complex form with nested attributes, just call “text_field_for_auto_complete” right on the form builder object yielded by form_for or fields_for, like in this example:

<% parent_form.fields_for :children do |child_form| %>
  <%= child_form.text_field_with_auto_complete :name, {},
        { :method => :get, :skip_style => true } %>
<% end %>

If you have Rails 2.3, this code will iterate over each child object and display a text field with auto complete support. My plugin will generate HTML and Javascript that works even when repeated in a loop like this. Also note that I’ve left off the object name parameter from text_field_with_auto_complete. It’s not needed now, since the object is indicated by the surrounding call to fields_for. The other parameters are optional and are taken unchanged from the original auto_complete plugin:

“:method => :get” indicates GET requests should be used by the AJAX calls to load the pick list values, avoiding problems with CSRF protection.
And “:skip => :true” indicates that the inline CSS stylesheet used by the auto complete drop down Prototype code should be skipped. Since we’re iterating over child objects we don’t want the same CSS code repeated once for each; instead include it once in a parent object’s call to text_field_for_auto_complete or else just include it manually somewhere.

That’s it – it should just work. If you’re interested in learning more about how to use nested attributes and what my plugin is actually doing, read on…

<u>Details:</u>

To learn more, let’s take a look at a simple nested attribute example, using the Projects/Tasks models from Ryan Bates' complex forms screen cast:

class Project < ActiveRecord::Base
  has_many :tasks
  accepts_nested_attributes_for :tasks, :allow_destroy => true
end
class Task < ActiveRecord::Base
  belongs_to :project
end

A project has many tasks, and each task belongs to a project. Here I’ve also declared that each project “accepts nested attributes for” tasks. This is a new method added to ActiveRecord in Rails 2.3… for lots of examples and explanation just take a look directly at the new nested_attributes.rb code file in Rails 2.3. In a nutshell, “accepts_nested_attributes_for” tells ActiveRecord that the project model should be able to save the attributes of the associated task model objects when a project is saved. This means that when I submit my project form, it can also contain a series of task fields as well. For example, my view code might look something like this:

<% form_for @project do |project_form| %>
  <p>
    <%= project_form.label :name, "Project:" %>
    <%= project_form.text_field :name %>
  </p>
  <% project_form.fields_for :tasks do |task_form| %>
    <%= task_form.label :name, "Task:" %>
    <%= task_form.text_field :name %>
  <% end %>
<% end %>

This displays a name text field for the project, and then calls “fields_for” again right on the form builder yielded by form_for. This is new for Rails 2.3. In earlier versions of Rails you had to explicitly iterate over the child objects and call fields_for for each one. Now in Rails 2.3, you can call fields_for as a method of the parent form and it will automatically iterate over all of the child objects and call fields_for. If we take a look at the HTML generated by this example form, we’ll find something like:

<input id="project_name" name="project[name]"
  size="30" type="text" value="Some project" />
<input id="project_tasks_attributes_0_id"
  name="project[tasks_attributes][0][id]" type="hidden" value="1" />
<input id="project_tasks_attributes_0_name"
  name="project[tasks_attributes][0][name]" type="text" value="Task one" />
<input id="project_tasks_attributes_1_id"
  name="project[tasks_attributes][1][id]" type="hidden" value="2" />
<input id="project_tasks_attributes_1_name"
  name="project[tasks_attributes][1][name]" type="text" value="Task two" />

I’ve simplified this to make it more readable. You can see the iteration by project_form.fields_for :tasks, and that for each task there’s an <input> tag for the “name” field, along with another hidden <input> tag containing the task’s “id” attribute. The most important detail here is the name given to each of these tags: “project[tasks_attributes][0][name]” for example. Since the tasks are nested attributes of the project, they are displayed using the PARENT_OBJECT[CHILD_OBJECTS_attributes][INDEX][FIELD] pattern, while for the project we get the simple OBJECT[FIELD] pattern. This is the key to making nested attributes work. In our project model, when we called “has_many :tasks”, Rails defined some new methods for us on the Project class to handle tasks: tasks, tasks=, task_ids, task_ids= and a couple of others as well. Now with Rails 2.3, when we call “accepts_nested_attributes_for :tasks” Rails defines another new method for Project called tasks_attributes= in order to process all of the new nested parameters for tasks when the complex project form is submitted. This is the reason for the “_attributes” in the naming pattern used in the form.

Now… how do we get auto complete to work for this form? The problem with auto complete on a complex form has always been that the Javascript and HMTL used by the Prototype library assumes that the <input> tag, <div> tag and related Javascript code would be unique on the HTML page. If you just call the text_field_with_auto_complete macro from the standard auto_complete plugin like this…

<% project_form.fields_for :tasks do |task_form| %>
  <%= text_field_with_auto_complete :task, :name, {},
        { :method => :get, :skip_style => true } %>
<% end %>

… it will not work. The first problem is that text_field_with_auto_complete does not know that fields_for is iterating over the child tasks, or which task is currently being processed in the iteration. But even if you were able to identify the current task object somehow, you would still get HTML like this:

<input id="task_name" name="task[name]" size="30" type="text" />
<div class="auto_complete" id="task_name_auto_complete"></div>
<script type="text/javascript">
//<![CDATA[
var task_name_auto_completer = new Ajax.Autocompleter('task_name',
  'task_name_auto_complete', '/projects/auto_complete_for_task_name',
  {method:'get'})
//]]>
</script>

…

<input id="task_name" name="task[name]" size="30" type="text" />
<div class="auto_complete" id="task_name_auto_complete"></div>
<script type="text/javascript">
//<![CDATA[
var task_name_auto_completer = new Ajax.Autocompleter('task_name',
  'task_name_auto_complete', '/projects/auto_complete_for_task_name',
  {method:'get'})
//]]>
</script>

Now the <input id=“task_name”> tag is repeated on the same page, and the Javascript call to Ajax.Autocompleter('task_name', … ) will not work since the browser will not be able to identify which <input> tag to use.

If you use my plugin instead of the original auto_complete plugin…

$ rm -rf vendor/plugins/auto_complete
$ ./script/plugin install git://github.com/patshaughnessy/auto_complete.git
Initialized empty Git repository in /Users/pat/rails-app/vendor/plugins/auto_complete/.git/
remote: Counting objects: 20, done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 20 (delta 5), reused 0 (delta 0)
Unpacking objects: 100% (20/20), done.
From git://github.com/patshaughnessy/auto_complete
 * branch            HEAD       -> FETCH_HEAD

… and restart your Rails app, then you can change your view to call text_field_with_auto_complete as a method of the form builder, like this:

<% project_form.fields_for :tasks do |task_form| %>
  <%= task_form.text_field_with_auto_complete :name, {},
  { :method => :get, :skip_style => true } %>
<% end %>

Note that I’ve also dropped :task as a parameter since that’s implicit in the call to fields_for. In fact, since text_field_with_auto_complete is now a method of the FormBuilder object (“task_form”), it has access to the task object currently being processed in the iteration. Now if you refresh the same form you’ll instead get this HTML instead:

<input id="project_tasks_attributes_0_name"
  name="project[tasks_attributes][0][name]"
  size="30" type="text" value="Task one" />
<div class="auto_complete" id="project_tasks_attributes_0_name_auto_complete">
</div><script type="text/javascript">
//<![CDATA[
var project_tasks_attributes_0_name_auto_completer =
  new Ajax.Autocompleter('project_tasks_attributes_0_name',
  'project_tasks_attributes_0_name_auto_complete',
  '/projects/auto_complete_for_task_name',
  {method:'get', paramName:'task[name]'})
//]]>
</script>

…

<input id="project_tasks_attributes_1_name"
  name="project[tasks_attributes][1][name]"
  size="30" type="text" value="Task two" />
<div class="auto_complete" id="project_tasks_attributes_1_name_auto_complete">
</div><script type="text/javascript">
//<![CDATA[
var project_tasks_attributes_1_name_auto_completer =
  new Ajax.Autocompleter('project_tasks_attributes_1_name',
  'project_tasks_attributes_1_name_auto_complete',
  '/projects/auto_complete_for_task_name',
  {method:'get', paramName:'task[name]'})
//]]>
</script>

This looks much better, and will actually work for the following reasons:

The <input> tags have the correct names, using the PARENT_OBJECT[CHILD_OBJECTS_attributes][INDEX][FIELD] pattern from fields_for. This means that the field values will be processed properly by ActiveRecord when the form is submitted.
My changes to the auto_complete plugin have picked up the child object index, 0 and 1 in this example, and included it in the <input> tag’s id, the <div> tag id and well as the associated Javascript code that calls Ajax.Autocompleter. Since all of the tag id’s are unique, the auto complete behavior works properly again for each text field.
The original “task” class name and “name” field name are passed unchanged into the Ajax calls to the server. This means that in your controller you can continue to use “auto_complete_for :task, :name” as usual, without worrying about the complex form and the fact that the task fields are repeated multiple times, etc.:
```
Ajax.Autocompleter('project_tasks_attributes_1_name',     
  'project_tasks_attributes_1_name_auto_complete',
  '/projects/auto_complete_for_task_name',
  {method:'get', paramName:'task[name]'})
```
Here the third parameter to Ajax.Autocompleter, "/projects/auto_complete_for_task_name", is the AJAX URL which you need to account for in routes.rb, and paramName:'task[name]' tells the auto_complete_for handler in your controller to get the task names as usual, and protects the server side code from all of the complexity around the tag id, names, child object index, etc.

Repeated auto complete plugin usage change

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

I’ve forked the auto_complete plugin to support repeated text fields in a complex form; see http://patshaughnessy.net/repeated_auto_complete for more details.

If you had downloaded my plugin in the past, I’ve just made a couple of changes that will require some simple code changes to your app:

You no longer need to or are able to use “auto_complete_form_for” or “auto_complete_fields_for.” I decided this was confusing and unnecessary. Now my plugin just mixes the text_field_with_auto_complete method right into the standard FormBuilder class. Just use form_for or fields_for as usual.
I also dropped the object name parameter from text_field_with_auto_complete. Since text_field_with_auto_complete is a method of the form builder, the target object is indicated by the surrounding call to fields_for or form_for and so doesn’t need to be repeated. Now using form.text_field_with_auto_complete is very similar to using form.text_field or the other form builder methods: you just need to specify the column/field name.

So if you are using my old plugin with a Rails 2.2 or earlier app like this:

<% for person in @group.people %>
  <% auto_complete_fields_for "group[person_attributes][]", person do |form| %>
    Person <%= person_form.label :name %><br />
    <%= form.text_field_with_auto_complete :person, :name, {},
                                           {:method => :get}  %>
  <% end %>
<% end %>

… you should drop “auto_complete_” and “:person” and just use code like this instead:

<% for person in @group.people %>
  <% fields_for "group[person_attributes][]", person do |form| %>
    Person <%= person_form.label :name %><br />
    <%= form.text_field_with_auto_complete :name, {},
                                           {:method => :get}  %>
  <% end %>
<% end %>

And if you have Rails 2.3 or later and are using nested attributes, this would become:

<% form_for @group do |group_form| -%>
  <% group_form.fields_for :people do |person_form| %>
    Person <%= person_form.label :name %><br />
    <%= person_form.text_field_with_auto_complete :name, {},
          { :method => :get, :skip_style => true } %>
  <% end %>
<% end %>

Paperclip sample app part 3: saving file attachments in a database BLOB column

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

In part 1 of this series, I showed how to create a simple Rails web site that uses the Paperclip plugin from Thoughtbot to upload and display image files. Then in part 2, I went on to change the sample app to download the image files through a Rails controller and not just through a direct call to Apache. This would be useful if you wanted to implement security for file attachments or for a variety of other reasons.

This time I’d like to show how to modify the same sample application to save the file attachments in a database BLOB column, instead of on your web server’s file system. To jump ahead and just get the working code, look at the “part3” folder in the github repo: http://github.com/patshaughnessy/paperclip-sample-app.

But before we actually work on the sample app, a disclaimer: Don’t try this at home! Serving file content directly from the file system via Apache or some other web server will always be faster and simpler than loading the file attachments from a database table… Apache and other web servers were designed to load and serve files quickly, and there’s normally no need to issue an expensive SQL query or to make another network connection to a database server just to send files to a web browser.

So why in the world would you ever want to pay the extra performance penalty and move the files into a database table? Here are a couple of reasons:

Your client or employer wants you to. Some companies insist on using a commercial, “enterprise” RDMS system to save file contents for one reason or another. My employer, for example, has many years of experience using Oracle and is comfortable managing large numbers of files that are stored in an Oracle table, while the thought of managing, replicating, backing up, etc., files that are simply saved on a Linux file system seems much more complex and unfamiliar.
Security. If you need to encrypt the contents of super-secret file attachments, storing them in a database might be an easier solution if you’re willing to spend money on a database server like Oracle. And database encryption aside, generally information in a database server can be more easily protected and audited than files on your web server's file system can be.

Anyway, let’s move on and actually change the sample app to save the files in a BLOB column. The first thing we will need to do is to use the version of Paperclip that I modified; the actual Paperclip plugin from Thoughtbot does not support storing files in a BLOB column. I added some code to Paperclip – a new “storage module” – to make this possible. See my code changes to learn more.

So let’s delete the original plugin and install my version:

$ cd /path/to/paperclip-sample-app
$ rm -rf vendor/plugins/paperclip
$ ./script/plugin install git://github.com/patshaughnessy/paperclip.git

Now that we have the modified plugin installed, let’s go ahead and create the BLOB columns that we will use to save the files. I tried to design the database storage module to be easy to use; one of the decisions I made was around what these BLOB columns should be called. I decided by default to use “[attachment]_file” as the name for the primary file attachment, and “[attachment]_[style]_file” for the other styles. If you want to use other column names, you just need to specify the names in the call to “has_attached_file” in the model. See my usage post for more info.

For this sample app I’ll go ahead and use the default column names: “avatar_file,” “avatar_small_file” and “avatar_thumb_file.” Here’s how to create those columns for a MySQL database. First create a new migration as usual:

$ ./script/generate migration add_attachments_blob_avatar_to_user
exists  db/migrate
create  db/migrate/20090528173400_add_attachments_blob_avatar_to_user.rb

… and then edit the new migration file and add the bolded code to it:

class AddAttachmentsBlobAvatarToUser < ActiveRecord::Migration
  def self.up
    execute 'ALTER TABLE users ADD COLUMN avatar_file LONGBLOB'
    execute 'ALTER TABLE users ADD COLUMN avatar_small_file LONGBLOB'
    execute 'ALTER TABLE users ADD COLUMN avatar_thumb_file LONGBLOB'
  end

  def self.down
    remove_column :users, :avatar_file
    remove_column :users, :avatar_small_file
    remove_column :users, :avatar_thumb_file
  end
end

Normally to create a BLOB column you would use “add_column :users, :avatar_file, :binary.” This would work fine for Oracle and other database servers. MySQL, however, supports four different types of BLOBs: TINYBLOB (256 bytes), BLOB (64k bytes), MEDIUMBLOB (16MB) and LONGBLOB (4GB). (See http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html for more information.) If we used the Rails migrations :binary column type, then we would get normal BLOBs and have an upper limit for the file attachment size of 64k, which is not normally enough. Unfortunately, there’s no way to specify LONGBLOB for MySQL using Rails migrations, so you have to use the “execute” migration and write an actual SQL statement to add the column.

Next go ahead and run your migrations and create the new columns:

$ rake db:migrate
(in /path/to/paperclip-sample-app)
==  AddAttachmentsBlobAvatarToUser: migrating =================================
-- execute("ALTER TABLE users ADD COLUMN avatar_file LONGBLOB")
   -> 0.0585s
-- execute("ALTER TABLE users ADD COLUMN avatar_small_file LONGBLOB")
   -> 0.0266s
-- execute("ALTER TABLE users ADD COLUMN avatar_thumb_file LONGBLOB")
   -> 0.0116s
==  AddAttachmentsBlobAvatarToUser: migrated (0.0973s) ========================

Now we just need to tell Paperclip to use database storage instead of the default file system storage and we’re ready to try our app and see if it saves the files into the new BLOB columns. Add the bolded parameter to has_attached_file in the User model:

class User < ActiveRecord::Base
  has_attached_file :avatar,
                    :storage => :database,
                    :styles => { :thumb => "75x75>", :small => "150x150>" },
                    :url => '/:class/:id/:attachment?style=:style'
end

Note that I also removed the “:path” parameter; this value would be ignored by the database storage module anyway since the files will be stored in the DB. Let’s try it out! Start up the sample app, and re-edit a user record to upload a new image file:

Select a file, click “Update” to submit the form and the file will be processed by ImageMagick, and saved into our new BLOB columns by the database storage module in Paperclip…

What? Where’s the image? It turns out that we still need to make a code change to the UsersController to download the image file from the BLOB column instead of from the file system. I’ll get to this in a moment. But first, let’s look at the console and see if the new files were saved into the database properly:

$ ./script/console 
Loading development environment (Rails 2.3.2)
>> User.first
=> #<User id: 1, name: "Mickey Mouse", email: "mickey@disney.com",
created_at: "2009-05-28 17:27:00", updated_at: "2009-05-28 17:37:42",
avatar_file_name: "mickey-mouse.jpg", avatar_content_type: "image/jpeg",
avatar_file_size: 137233, avatar_updated_at: "2009-05-28 17:37:42",
avatar_file: "\377???JFIF\000\001\002\001\001h\001h\000\000\377?\021\b\002\210\001?\001\021\000\002\021\001\003\021\001\377?\204\000\001\001\001\001\001\001\001...",
avatar_small_file: "\377???JFIF\000\001\001\001\001h\001h\000\000\377?C\000\003\002\002\002\002\002\003\002\002\002\003\003\003\003\004\006\004\004\004\004\004\b\006\006\005\006...",
avatar_thumb_file: "\377???JFIF\000\001\001\001\001h\001h\000\000\377?C\000\003\002\002\002\002\002\003\002\002\002\003\003\003\003\004\006\004\004\004\004\004\b\006\006\005\006...">

Here you can see the three new BLOB columns, avatar_file, avatar_small_file and avatar_medium_file, and the first few bytes of each column’s value. In fact, if you’re a real geek you’ll notice that the first few bytes contain “JFIF,” which is probably the image file type specification (not sure)… so we know we are seeing the binary contents of the three versions of the Mickey image here. Great!

Actually, not so great: there’s a subtle problem here we need to worry about. I actually typed in a very simple ActiveRecord command, “User.first,” and it loaded all three of the image files’ contents into memory just so that I could inspect the value of the user record in the console. This was convenient now, since I was actually interested in knowing whether or not each of the files was saved properly in the DB. However, this is potentially a big performance problem in general. Imagine if the files were very large… it could take a long time for all 3 of the files to be fetched by a SQL query and returned to the Rails ActiveRecord object. Do I really want or need this to happen every time I access a User record? Usually when I load a User record it’s just because I need to check the value of one of the metadata columns, like the user’s name, email address or something else. And if I do need one of the files, why should I have to load all three files? Spending the time required to load the file contents for each image style is a big performance penalty that isn’t usually necessary.

The solution I came up for this problem was to enable Paperclip to add a method to your model class called “select_without_file_columns_for” that you can use as a named scope or, even better, a default scope. It returns a :select scope hash that will exclude the BLOB columns from the SQL query that ActiveRecord issues to load each record. If you’re using Rails 2.3 or higher, you can use select_without_file_columns_for as a default scope in your model like this:

class User < ActiveRecord::Base
  has_attached_file :avatar, :storage => :database,
                    :styles => { :thumb => "75x75>", :small => "150x150>" },
                    :url => '/:class/:id/:attachment?style=:style'
  default_scope select_without_file_columns_for(:avatar)
end

To learn more about default scopes and to see a couple of other examples, read this: http://ryandaigle.com/articles/2008/11/18/what-s-new-in-edge-rails-default-scoping … or http://m.onkey.org/2009/3/24/default-scopes-and-inheritance-to-the-rescue. Let’s see how this works in the console again after adding the default scope:

$ ./script/console 
Loading development environment (Rails 2.3.2)
>> User.first
=> #<User id: 1, name: "Mickey Mouse", email: "mickey@disney.com",
created_at: "2009-05-28 17:27:00", updated_at: "2009-05-28 17:37:42",
avatar_file_name: "mickey-mouse.jpg", avatar_content_type: "image/jpeg",
avatar_file_size: 137233, avatar_updated_at: "2009-05-28 17:37:42">

Cool. Now the BLOB columns are not displayed. “Default scope” refers to the fact that the SQL used by ActiveRecord by default to load records is automatically changed. I didn’t have to pay the price of loading each of the files, and I was still able to load all of the other User columns using ActiveRecord the way I normally would. To see what happened, execute the “select_without_file_columns_for” method directly in the console:

>> User.select_without_file_columns_for :avatar
=> {:select=>"id,name,email,created_at,updated_at,avatar_file_name,
  avatar_content_type,avatar_file_size,avatar_updated_at"}

It‘s a lot more common to use a default or named scope with :conditions (to modify the WHERE clause) or :order (to modify the ORDER BY clause) but in this case I’ve used :select to specify which columns should be loaded by ActiveRecord (the SELECT portion of the SQL). If you look at the hash, you’ll see all of the User columns listed, except for avatar_file, avatar_small_file and avatar_thumb_file.

If you’re using Rails 2.2 or earlier default scope is not available yet, and you will need to use a named scope, like this:

class User < ActiveRecord::Base
  has_attached_file :avatar,
                    :storage => :database,
                    :styles => { :thumb => "75x75>", :small => "150x150>" },
                    :url => '/:class/:id/:attachment?style=:style'
  named_scope :without_file_data, select_without_file_columns_for(:avatar)
end

And you will need to use the named scope explicitly to avoid loading the files, like this:

$ ./script/console 
Loading development environment (Rails 2.3.2)
>> User.without_file_data.first
=> #<User id: 1, name: "Mickey Mouse", email: "mickey@disney.com",
created_at: "2009-05-30 12:14:03", updated_at: "2009-05-30 12:14:03",
avatar_file_name: "mickey-mouse.jpg", avatar_content_type: "image/jpeg",
avatar_file_size: 137233, avatar_updated_at: "2009-05-30 12:14:03">

A good way to understand exactly what ActiveRecord is doing is to use a Ruby trick and open up the User class and add some debug code to it. Try entering this code into your console:

>> class User < ActiveRecord::Base
>>   def self.find_by_sql(sql)
>>     puts "DEBUG: #{sql}"
>>     super
>>   end
>> end
=> nil

What this code does is open up our User model class, and override the “find_by_sql” ActiveRecord method to display the SQL statement before calling the original base class method to execute it. Find_by_sql is the method that the various different ActiveRecord find methods all call once they have constructed a SQL statement. For example, find :first, find :all, first, last… all of these eventually call find_by_sql. Assuming we have the version of User with the named_scope, we can see what SQL statements are issued with or without the select_without_file_columns_for scope:

>> User.first
DEBUG: SELECT * FROM `users`  LIMIT 1
=> #<User id: 1, name: "Mickey Mouse", email: "mickey@disney.com",
created_at: "2009-05-28 17:27:00", updated_at: "2009-05-28 17:37:42",
avatar_file_name: "mickey-mouse.jpg", avatar_content_type: "image/jpeg",
avatar_file_size: 137233, avatar_updated_at: "2009-05-28 17:37:42",
avatar_file: "\377???JFIF\000\001\002\001\001h\001h\000\000\377?\021\b\002\210\001?\001\021\000\002\021\001\003\021\001\377?\204\000\001\001\001\001\001\001\001...",
avatar_small_file: "\377???JFIF\000\001\001\001\001h\001h\000\000\377?C\000\003\002\002\002\002\002\003\002\002\002\003\003\003\003\004\006\004\004\004\004\004\b\006\006\005\006...",
avatar_thumb_file: "\377???JFIF\000\001\001\001\001h\001h\000\000\377?C\000\003\002\002\002\002\002\003\002\002\002\003\003\003\003\004\006\004\004\004\004\004\b\006\006\005\006...">
>> User.without_file_data.first
DEBUG: SELECT id,name,email,created_at,updated_at,avatar_file_name,
  avatar_content_type, avatar_file_size,avatar_updated_at FROM `users` LIMIT 1
=> #<User id: 1, name: "Mickey Mouse", email: "mickey@disney.com",
created_at: "2009-05-28 17:27:00", updated_at: "2009-05-28 17:37:42",
avatar_file_name: "mickey-mouse.jpg", avatar_content_type: "image/jpeg",
avatar_file_size: 137233, avatar_updated_at: "2009-05-28 17:37:42">

Here we can see that ActiveRecord will load all of the User columns that are listed above in the hash we pass to named_scope… all of the columns except for the BLOB columns. This is different from what ActiveRecord does by default, which is a simple SELECT * FROM …. statement.

Enough about ActiveRecord internals… let’s get back to the sample app and finish it up:

So why didn’t the image appear here properly? The reason is that in UsersController I’m still using the code that accesses the file on the file system and streams it to the client using send_file (see my last post for more info):

def avatars
    user = User.find(params[:id])
    style = params[:style] ? params[:style] : 'original'
    send_file user.avatar.path(style),
              :type => user.avatar_content_type
  end

Obviously send_file is no longer going to work for us. Instead we need to use a similar function in ActionController::Streaming called send_data, which takes the binary data directly as a parameter instead of a file. And to access the file’s contents, I’ve added a method called “file_contents” to Paperclip that returns the actual file contents for the given style, or for the original style by default. Here’s how to put it all together; replace the “avatars” method in UsersController with this new version instead:

def avatars
    user = User.find(params[:id])
    style = params[:style] ? params[:style] : 'original'
    send_data user.avatar.file_contents(style),
              :type => user.avatar_content_type
  end

Only the line in bold has changed. We just call user.avatar.file_contents, pass in the specified style and then pass along the data to send_data. If you restart the app and refresh your browser, now you should see the image again:

Now we are seeing the binary image file data loaded from the BLOB column by ActiveRecord and streamed down to the browser by send_data.

One last detail about this: since usually everyone will use the same controller code to load the file contents for a given style from the BLOB, and then pass it along to send_data, I enabled Paperclip to add another utility method, this time to your controller, to make this even easier:

class UsersController < ApplicationController
  downloads_files_for :user, :avatar
etc…

If you call “downloads_files_for” from your controller like this and specify the model and file attachment, it will automatically generate the correct controller method for you, and call it “avatars” or whatever the plural version of your attachment name is. I chose the name to make it easy and natural to create a route to it in routes.rb. No need to even think about send_data or how to get the file contents from Paperclip at all! Nothing could be simpler. However, if you need to implement security or some other business rules around downloading files, then you might need to add that business logic to the “avatars” method above. Either way, it’s very simple.

Paperclip sample app part 2: downloading files through a controller

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Last time I wrote about how to quickly setup a Rails application using scaffolding that allows users to upload image files and then display them using the Paperclip plugin. Paperclip does the simplest thing possible by default: it saves the file attachments right on the file system of your web server, allowing you to download them to users easily using Apache or whatever web server you have installed.

Today I’d like to take that sample app one step further and show how to use a Rails controller to download the files, instead of directly through Apache. To get the finished code just go to http://github.com/patshaughnessy/paperclip-sample-app and look at the “part2” folder.

There are a variety of reasons why you might want to do this, including:

Security: you don’t want to expose all of the file attachments to all users of your web site. Instead, you want to implement some business rules and show some files to some users, but not to others.
Auditing/logging: you want to keep track of who is viewing which files, or how many times they are viewing them.
You want to hide the actual location of the files from users, and instead map the files to URLs in some other pattern or manner.
Or, you might want not want to store the files on your web server’s file system at all, but instead in a database table or somewhere else. In Part 3 of this series, I’ll show how to do this…

The common thread here is that you want to execute some Ruby code every time a users accesses a file, and the way to do that is by routing the download requests through a controller.

Let’s pick up where we left off last time:

If we take a look at some of the HTML source for this page:

<h1>Listing users</h1>
<table>
  <tr>
    <th>Photo</th>
    <th>Name</th>
    <th>Email</th>
  </tr>
<tr>
    <td><img alt="Mickey-mouse"
      src="/system/avatars/1/thumb/mickey-mouse.jpg?1242395876" /></td>
    <td>Mickey Mouse</td>
    <td>mickey@disney.com</td>
    <td><a href="/users/1">Show</a></td>
    <td><a href="/users/1/edit">Edit</a></td>

… we can see that Paperclip’s “url” function which we called in index.html.erb is returning a pointer to the actual location of the file on the web server’s hard drive, under the public/system folder:

$ find public/system
public/system
public/system/avatars
public/system/avatars/1
public/system/avatars/1/original
public/system/avatars/1/original/mickey-mouse.jpg
public/system/avatars/1/small
public/system/avatars/1/small/mickey-mouse.jpg
public/system/avatars/1/thumb
public/system/avatars/1/thumb/mickey-mouse.jpg

Now let’s say that we want to implement some simple security around these images…. reason #1 from my list above. The first thing we’ll need to do, then, is to remove the image files from the public folder and instead save them in some non-public place on our web server, for example:

$ mkdir non-public
$ mv public/system non-public/.

Now let’s double check that Apache can’t find the files in their new location:

Great! We see a missing image as expected. No users can see the files unless we run some bit of Ruby code to enable access. Now… how can we use a controller to download the files through Rails, instead of through Apache? The first thing we need to do is add a route to routes.rb for accessing the files.

If you open up routes.rb and look at what the scaffolding generator created for us, you’ll see this:

ActionController::Routing::Routes.draw do |map|
  map.resources :users
  ...etc...
  map.connect ':controller/:action/:id'
  map.connect ':controller/:action/:id.:format'
end

The map.resources line indicates that our application supports a series of routes that handle the four REST actions: GET, POST, PUT and DELETE. The best way to get a handle on how the routes work is by running “rake routes” to list out all the URL patterns that Rails will match on:

$ rake routes
(in /Users/pat/rails-apps/paperclip-sample-app)
    users GET    /users(.:format)          {:action=>"index", :controller=>"users"}
          POST   /users(.:format)          {:action=>"create", :controller=>"users"}
 new_user GET    /users/new(.:format)      {:action=>"new", :controller=>"users"}
edit_user GET    /users/:id/edit(.:format) {:action=>"edit", :controller=>"users"}
     user GET    /users/:id(.:format)      {:action=>"show", :controller=>"users"}
          PUT    /users/:id(.:format)      {:action=>"update", :controller=>"users"}
          DELETE /users/:id(.:format)      {:action=>"destroy", :controller=>"users"}
                 /:controller/:action/:id           
                 /:controller/:action/:id(.:format)

The last two lines are the “default routes,” which connect any URL matching the pattern controller/action/id to the corresponding controller. We could just go ahead and use the default routes, but to learn a bit more about how map.resources works, let’s create a new URL pattern for our users that we’ll use in a minute to download the avatar file attachments with. Edit routes.rb and add the :member parameter in bold:

ActionController::Routing::Routes.draw do |map|
  map.resources :users, :member => { :avatars => :get }
etc...

If we now re-run rake routes, we can see that a new URL pattern was created for us that we can use to download the avatar images via a GET request:

$ rake routes
(in /Users/pat/rails-apps/paperclip-sample-app)
      users GET    /users(.:format)          {:action=>"index", :controller=>"users"}
            POST   /users(.:format)          {:action=>"create", :controller=>"users"}
   new_user GET    /users/new(.:format)      {:action=>"new", :controller=>"users"}
  edit_user GET    /users/:id/edit(.:format) {:action=>"edit", :controller=>"users"}
avatars_user GET   /users/:id/avatars(.:format) {
                                              :action=>"avatars",
                                              :controller=>"users"
                                             }
       user GET    /users/:id(.:format)      {:action=>"show", :controller=>"users"}
            PUT    /users/:id(.:format)      {:action=>"update", :controller=>"users"}
            DELETE /users/:id(.:format)      {:action=>"destroy", :controller=>"users"}
                   /:controller/:action/:id           
                   /:controller/:action/:id(.:format)

Now to get the avatar for user 7, for example, we can issue a URL like this:

http://localhost:3000/users/7/avatars

… and the request will be routed to the “avatars” action in the “users” controller (plural since a user might have more than one style of avatar). So now let’s go right ahead and implement the avatars method and add some code to download a file to the client. The way to do that is to use ActionController::Streaming::send_file. It’s simple enough; we just need to pass the file’s path to send_file as well as the MIME content type which the client uses as a clue for deciding how to display the file, and that’s it! Let’s hard code these values for now and see if it all works (update the path here for your machine):

class UsersController < ApplicationController
  def avatars
    send_file '/path/to/non-public/system/avatars/1/original/mickey-mouse.jpg',
      :type => 'image/jpeg'
  end

Now if you type http://localhost:3000/users/1/avatars into your browser you should see the mickey image again. If not, then double check your code changes, where the files are actually located on the hard drive now and also try stopping and reloading the Rails app since changes to routes.rb are cached and are only loaded when Rails is initialized.

Instead of hard coding the path in the avatars method, we obviously need to be able to handle requests for any avatar file attachment for any user record. Before we enhance our code to do this, let’s take a few minutes to configure Paperclip and tell it where the files are now stored on the file system, and which URL we have configured our routes.rb file to use. This will make our work coding in the controller a lot easier, and also indicate to Paperclip where new file attachments should be uploaded to. To do this, we need to add a couple of parameters to our call to has_attached_file in our User model (user.rb), shown here in bold (again, update the path for your machine):

class User < ActiveRecord::Base
  has_attached_file :avatar,
    :styles => { :thumb => "75x75>", :small => "150x150>" },
    :path => '/path/to/non-public/system/avatars/1/original/mickey-mouse.jpg',
    :url => 'users/1/avatars'
end

Just to take one step at a time, I’ve hard coded the URL and path again here in the model. But now we can generalize our code in UserController to handle any user, like this:

def avatars
  user = User.find(params[:id])
  send_file user.avatar.path, :type => user.avatar_content_type
end

Now we can test http://localhost:3000/users/1/avatars again to be sure that we haven’t broken anything. If it’s all working, lets’ proceed to clean up the hard coding in user.rb. It turns out that Paperclip uses the same interpolations idea that we saw above in routes.rb. So I can use symbols like :rails_root, :id, :style, etc., and they will be evaluated to the values I expect and need. Here’s the finished code in my model:

has_attached_file :avatar,
  :styles => { :thumb => "75x75>", :small => "150x150>" },
  :path => 
    ':rails_root/non-public/system/:attachment/:id/:style/:basename.:extension',
  :url => '/:class/:id/:attachment'

If we open up the console and take a look at our user object there, we can see that Paperclip is substituting the correct values for each of the symbols I’ve provided:

$ ./script/console
Loading development environment (Rails 2.3.2)
>> User.first.avatar.url
=> "/users/1/avatars?1242395876"       (time stamp appended here automatically)
>> User.first.avatar.path
=> "/Users/pat/.../non-public/system/avatars/1/original/mickey-mouse.jpg"

Now let’s go back and test our original application again with our new code:

The new route in routes.rb
The new action in UserController
And the :url and :path parameters added to has_attached_file in the User model.

Oops… we see the large image again. It doesn’t work! What happened? Well, it turns our that we forgot one detail in our new avatars method in UserController. Our code always returns the default, or “original” style of the file attachment, but in the users index view we actually display the thumbnail image using image_tag user.avatar.url(:thumb). So our controller code needs to be able to handle requests for other styles as well. To do this, we need to pass the requested style somehow. The simplest thing to do is just to add the style as a URL parameter to the download request, like this:

  http://localhost:3000/users/1/avatar?style=thumb

(We could also add the style to the URL's path, but that would require another change to routes.rb.) To make this work, first I need to tell Paperclip about how I want to handle the style value in the URL:

has_attached_file :avatar,
  :styles => { :thumb => "75x75>", :small => "150x150>" },
  :path => 
    ':rails_root/non-public/system/:attachment/:id/:style/:basename.:extension',
  :url => '/:class/:id/avatar?style=:style'

And now I need to handle this new parameter in my controller:

def avatars
  user = User.find(params[:id])
  style = params[:style] ? params[:style] : 'original'
  send_file user.avatar.path(style),
            :type => user.avatar_content_type
end

I don’t need to change anything in index.html.erb since there we call image_tag user.avatar.url(:thumb) which picks up the new URL pattern from Paperclip.

And now, if I’ve got all of this correct, I should be able to finally see the thumbnail image again on the index page:

And finally we have files being downloaded by Ruby code present in the UserController class. If we actually wanted to implement security, logging or some other sort of logic we would just add code to the avatars method in UserController. For example, avatars could return a 401 (unauthorized) error if the user wasn’t logged in, or didn’t have access to view Mickey’s image for some reason.

That’s it for now; next time I’ll modify this sample app once more to demonstrate how we can store the image files in a database table, instead of in the “non-public” folder or anywhere on the file system.

Paperclip sample app

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

(Update October 2009)
I just updated a gem I wrote called View Mapper that will generate all of the code I describe below… you can use View Mapper to generate working scaffolding code that uploads/downloads files using Paperclip, or only view scaffolding code that works with an existing model in your app; for more details see: http://patshaughnessy.net/2009/10/16/paperclip-scaffolding

I love scaffolding. Many experienced Rails developers scoff at the idea of using scaffolding to generate Rails code: it’s ugly; it probably means you don’t understand how to write the code yourself; it generates a lot more code than you need, etc., etc. However, for a beginning Rails developer working on her/his own like me who isn’t surrounded by a team of Ruby experts, scaffolding is an essential tool and can help to get started in the right direction. Also, even for experienced Rubyists scaffolding can be a great way to quickly (minutes, not hours or days) get a simple app up and running to use for demos, UI wireframes, spiking some technical issue, etc.

This post will demonstrate how to use scaffolding to create a new Rails app from scratch that uses the Paperclip plugin to upload and display an image file. Feel free to copy/paste pieces of code from the narrative below and use them in your app, or you can just skip to the chase and get the finished version from github and run that on your machine.

There are a lot of other good tutorials out there about this; see:

http://burm.net/2008/10/07/the-ruby-on-rails-paperclip-plugin-tutorial-easy-image-attachments, or
http://jimneath.org/2008/04/17/paperclip-attaching-files-in-rails
and of course Ryan Bates has a screen cast on this topic also.

I’ll take on the risk of repeating material that’s already out there in order to show how easy it is to get a working Paperclip application up and running using scaffolding. The fact that just a few commands and lines of code are required illustrates just how simple and powerful Paperclip’s design is. In my next post, I’ll proceed to change this sample app to demonstrate how to save the uploaded files in a database column instead of on the web server’s file system, using my modified version of Paperclip.

FYI At the time I wrote this, Rails was at version 2.3.2:

$ rails --version
Rails 2.3.2

Let’s get started by creating a new Rails application:

$ rails paperclip-sample-app
      create  
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
      create  config/environments
      create  config/initializers
      create  config/locales
      create  db
      create  doc
      create  lib
      create  lib/tasks
      create  log
      etc...

Before we go any farther, let’s setup our database.yml file and create a new MySQL database to use with the sample app. Replace the contents of config/database.yml with this:

development:
    adapter: mysql
    database: paperclip_sample_app_development
    username: root
    password: 
    host: localhost

Enter the proper username and password for MySQL if they are not “root” and null. And then run this from the command line:

$ cd paperclip-sample-app
$ rake db:create
(in /Users/pat/rails-apps/paperclip-sample-app)

Ok, now we have a MySQL database to work with. Next, let’s go ahead and install the Paperclip plugin. The best thing to do is just to get the latest version from github; Thoughtbot frequently updates it with bug fixes, enhancements, etc.:

$ ./script/plugin install git://github.com/thoughtbot/paperclip.git
Initialized empty Git repository in /Users/pat/rails-apps/paperclip-sample-app/vendor/plugins/paperclip/.git/
remote: Counting objects: 62, done.
remote: Compressing objects: 100% (50/50), done.
remote: Total 62 (delta 6), reused 39 (delta 4)
Unpacking objects: 100% (62/62), done.
From git://github.com/thoughtbot/paperclip
 * branch            HEAD       -> FETCH_HEAD

Now that we have an empty, shell application created and the Paperclip plugin installed, we can use scaffolding to add some working code to it. Let’s use the same “user” and “avatar” example Thoughtbot does on the Paperclip project page. The idea is that the sample will contain a table of users, and each user will have an avatar image displayed in the web site. So to get started, I’ll just create a new “user” model with string columns for the name and email address:

$ ./script/generate scaffold user name:string email:string
      exists  app/models/
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/users
      exists  app/views/layouts/
      exists  test/functional/
      exists  test/unit/
      create  test/unit/helpers/
      exists  public/stylesheets/
      create  app/views/users/index.html.erb
      create  app/views/users/show.html.erb
      etc...

Now we need to generate the database columns necessary for Paperclip on our new model object using script/generate:

$ ./script/generate paperclip user avatar
      exists  db/migrate
      create  db/migrate/20090430084151_add_attachments_avatar_to_user.rb

And let’s go ahead and create the users table using db:migrate:

$ rake db:migrate
(in /Users/pat/rails-apps/paperclip-sample-app)
==  CreateUsers: migrating ====================================================
-- create_table(:users)
   -> 0.0031s
==  CreateUsers: migrated (0.0032s) ===========================================

==  AddAttachmentsAvatarToUser: migrating =====================================
-- add_column(:users, :avatar_file_name, :string)
   -> 0.0063s
-- add_column(:users, :avatar_content_type, :string)
   -> 0.0069s
-- add_column(:users, :avatar_file_size, :integer)
   -> 0.0085s
-- add_column(:users, :avatar_updated_at, :datetime)
   -> 0.0081s
==  AddAttachmentsAvatarToUser: migrated (0.0311s) ============================

You can see that the Paperclip generator created columns in the users table called “avatar_file_name,” “avatar_content_type,” “avatar_file_size” and “avatar_updated_at.” Now we have our database schema setup. The next step is to just modify the code that was generated for us by the scaffolding and make the changes necessary for Paperclip. The first thing to do is to add a line to the user model and indicate that it has a file attachment called “avatar.” To do this, open app/models/user.rb and just add this one line:

class User < ActiveRecord::Base
  has_attached_file :avatar
end

And then edit the new user form (app/views/users/new.html.erb) and add a file field to use to upload files. There are actually two code changes you need to make: first you need to set the HTML form to encode the uploaded file data (and other fields) using MIME multiple part syntax, and then second you need to actually add the file upload field. Here’s the finished new.html.erb file with these two changes in bold:

<h1>New user</h1>

<% form_for(@user, :html => { :multipart => true }) do |f| %>
  <%= f.error_messages %>

  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>
  <p>
    <%= f.label :email %><br />
    <%= f.text_field :email %>
  </p>
  <p>
    <%= f.label :avatar %><br />
    <%= f.file_field :avatar %>
  </p>
  <p>
    <%= f.submit 'Create' %>
  </p>
<% end %>

<%= link_to 'Back', users_path %>

Also make the same changes to the edit form that was generated by the scaffolding: app/views/users/edit.html.erb. The best thing to do would be to include the same ERB file (maybe called “_form.html.erb”) in both the new and edit form files. Ideally the scaffolding generator would have done this for us…

Now if we run our application we can upload an image file and attach it to a user:

If you submit this form, the image file will be uploaded to the server and saved on the file system. By default, Paperclip saves files inside a “system” folder it creates in your Rails app’s public folder. Let’s take a look at my public folder and see where the file went:

$ find public/system
public/system
public/system/avatars
public/system/avatars/1
public/system/avatars/1/original
public/system/avatars/1/original/mickey-mouse.jpg

This is one of the nice things about Paperclip: it just works. I don’t have to think about or worry about where the files are going to go; Thoughtbot has chosen simple default values that make sense. Here we can see that there are a series of folders created that correspond to the attachment name, model primary key and also the “style” of the attachment (more on that below).

If you want to or need to save the files in some other place on your server’s file system you can specify different options to has_attached_file in your model; see this write up for an example: http://travisonrails.com/2009/01/11/Changing-Paperclip-File-Storage-Location. Paperclip also supports saving the files in Amazon’s S3 storage service, and in my next post I’ll demonstrate how to save the file data inside the database itself, right in the users table in this example.

I’m almost done; now I just need to display the uploaded image somewhere; the simplest thing to do is just to add an image tag to the users show page. Again, my changes to the standard scaffolding code are in bold:

<p>
  <b>Name:</b>
  <%=h @user.name %>
</p>

<p>
  <b>Email:</b>
  <%=h @user.email %>
</p>

<p>
  <b>Avatar:</b>
  <%= image_tag @user.avatar.url %>
</p>

<%= link_to 'Edit', edit_user_path(@user) %> |
<%= link_to 'Back', users_path %>

Now we can see the image for our new user:

Since this image is bigger that what I would like, I can take advantage of Paperclip “styles” feature to generate a smaller version of it. To do that you will need to be sure you have ImageMagick installed on your server, which is what Paperclip uses behind the scenes to modify image files. Then all you need to do is just add two “styles” to your model, like this:

class User < ActiveRecord::Base
  has_attached_file :avatar,
                    :styles => {
                      :thumb => "75x75>",
                      :small => "150x150>"
                    }
end

The strings we pass in are actually options for ImageMagick's "convert" command; see it’s documentation for more details. And now in the show ERB we can just specify the “small” style in the image tag instead:

<%= image_tag @user.avatar.url(:small) %>

To see it, first I need to re-edit and re-upload the image (remember to add the file field code to edit.html.erb just like for new.html.erb):

Now when this form is submitted we will see the smaller image:

And let’s take another look at the public/system folder and see what Paperclip and ImageMagick have done for us:

$ find public/system
public/system
public/system/avatars
public/system/avatars/1
public/system/avatars/1/original
public/system/avatars/1/original/mickey-mouse.jpg
public/system/avatars/1/small
public/system/avatars/1/small/mickey-mouse.jpg
public/system/avatars/1/thumb
public/system/avatars/1/thumb/mickey-mouse.jpg

Again, this is very simple and just works! As a last step, let’s add the thumbnail image to the users index page so we can see Mickey without even clicking on that user record. This is as simple as editing app/views/users/index.html.erb and adding a new table column:

<table>
  <tr>
    <th>Photo</th>
    <th>Name</th>
    <th>Email</th>
  </tr>

<% @users.each do |user| %>
  <tr>
    <td><%= image_tag user.avatar.url(:thumb) %></td>
    <td><%=h user.name %></td>
    <td><%=h user.email %></td>
    <td><%= link_to 'Show', user %></td>
    <td><%= link_to 'Edit', edit_user_path(user) %></td>
    <td><%= link_to 'Destroy', user, :confirm => 'Are you sure?', :method => :delete %></td>
  </tr>
<% end %>
</table>

And now we just need to refresh the index page since the thumb image file was already generated:

And there you have it: a working file upload web site written in minutes. This was made possible by Rails scaffolding, and Paperclip's simple, elegant design.

Database storage for Paperclip: rewritten to use a single table

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Back in February I wrote an implementation of a new storage module for Paperclip that supports saving file attachments in a database table. My original implementation saved the file attachments in a separate database table, which was internally managed using a “has_many” relationship from the target model.

This month I decided to rewrite the code to use a simpler, single table approach. Instead of saving the files in a separate table, each file is saved in a BLOB column in the same table as the target model. This makes the database storage module easier and more intuitive to use, and moves it closer to the original intent of the Paperclip design.

Code: http://github.com/patshaughnessy/paperclip

New usage:

Install and use my version of the plugin:

script/plugin install git://github.com/patshaughnessy/paperclip.git

In your model specify the "database" storage option; for example:
```
has_attached_file :avatar, :storage => :database
```
The file will be stored in a column called [attachment name]_file (e.g. "avatar_file") by default. To specify a different BLOB column name, use :column, like this:
```
has_attached_file :avatar,
                  :storage => :database,
                  :column => 'avatar_data'
```
If you have defined different styles, these files will be stored in additional columns called [attachment name]_[style name]_file (e.g. "avatar_thumb_file") by default. To specify different column names for each style, use :column in the style definition, like this:

has_attached_file :avatar,
                  :storage => :database,
                  :styles => { 
                    :medium => {
                      :geometry => "300x300>",
                      :column => 'medium_file'
                    },
                    :thumb => {
                      :geometry => "100x100>",
                      :column => 'thumb_file'
                    }
                  }

You need to create these new columns in your migrations or you'll get an exception. Example:

add_column :users, :avatar_file, :binary
add_column :users, :avatar_medium_file, :binary
add_column :users, :avatar_thumb_file, :binary

Note the migration for the "binary" column type will not work for LONGBLOBs in MySQL. Here's an example migration for MySQL:

execute 'ALTER TABLE users ADD COLUMN avatar_file LONGBLOB'
execute 'ALTER TABLE users ADD COLUMN avatar_medium_file LONGBLOB'
execute 'ALTER TABLE users ADD COLUMN avatar_thumb_file LONGBLOB'

To avoid performance problems loading all of the BLOB columns every time you access your ActiveRecord object, a class method is provided on your model called “select_without_file_columns_for.” This is set to a hash that will instruct ActiveRecord::Base.find to load all of the columns <u>except</u> the BLOB/file data columns, for example:
```
{:select=>"id,name,avatar_file_name,avatar_content_type,..."}
```
If you’re using Rails 2.3, you can specify this as a default scope:
```
default_scope select_without_file_columns_for(:avatar)
```
Or if you’re using Rails 2.1 or 2.2 you can use it to create a named scope:
```
named_scope :without_file_data, select_without_file_columns_for(:avatar)
```
By default, URLs will be set to this pattern:
```
/:relative_root/:class/:attachment/:id?style=:style
```
Example:
```
/app-root-url/users/avatars/23?style=original
```
The idea here is that to retrieve a file from the database storage, you will need some controller's code to be executed. Once you pick a controller to use for downloading, you can add this line to generate the download action for the default URL/action (the plural attachment name), "avatars" in this example:
```
downloads_files_for :user, :avatar
```
Or you can write a download method manually if there are security, logging or other requirements. If you prefer a different URL for downloading files you can specify that in the model; e.g.:
```
has_attached_file :avatar,
                  :storage => :database,
                  :url =>'/users/show_avatar/:id/:style'
```
Add a route for the download to the controller which will handle downloads, if necessary. The default URL, /:relative_root/:class/:attachment/:id?style=:style, will be matched by the default route: :controller/:action/:id

For now, I’ve overwritten my code from February. I believe this implementation is cleaner and easier to use; if anyone did download and use my code from February let me know and I can help you migrate the file data from the separate table back to columns in primary table. I will be writing a script to do this for my own application.

When I have time in the next few days or weeks, I’ll post a sample application that illustrates all of these steps and shows exactly how to do all of this. If anyone has any questions or suggestions please let me know.

Database storage for Paperclip: rewritten to use a single table

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Code: http://github.com/patshaughnessy/paperclip

New usage:

Install and use my version of the plugin:

script/plugin install git://github.com/patshaughnessy/paperclip.git

In your model specify the "database" storage option; for example:
```
has_attached_file :avatar, :storage => :database
```
The file will be stored in a column called [attachment name]_file (e.g. "avatar_file") by default. To specify a different BLOB column name, use :column, like this:
```
has_attached_file :avatar,
                  :storage => :database,
                  :column => 'avatar_data'
```
If you have defined different styles, these files will be stored in additional columns called [attachment name]_[style name]_file (e.g. "avatar_thumb_file") by default. To specify different column names for each style, use :column in the style definition, like this:

has_attached_file :avatar,
                  :storage => :database,
                  :styles => { 
                    :medium => {
                      :geometry => "300x300>",
                      :column => 'medium_file'
                    },
                    :thumb => {
                      :geometry => "100x100>",
                      :column => 'thumb_file'
                    }
                  }

You need to create these new columns in your migrations or you'll get an exception. Example:

add_column :users, :avatar_file, :binary
add_column :users, :avatar_medium_file, :binary
add_column :users, :avatar_thumb_file, :binary

Note the migration for the "binary" column type will not work for LONGBLOBs in MySQL. Here's an example migration for MySQL:

execute 'ALTER TABLE users ADD COLUMN avatar_file LONGBLOB'
execute 'ALTER TABLE users ADD COLUMN avatar_medium_file LONGBLOB'
execute 'ALTER TABLE users ADD COLUMN avatar_thumb_file LONGBLOB'

To avoid performance problems loading all of the BLOB columns every time you access your ActiveRecord object, a class method is provided on your model called “select_without_file_columns_for.” This is set to a hash that will instruct ActiveRecord::Base.find to load all of the columns <u>except</u> the BLOB/file data columns, for example:
```
{:select=>"id,name,avatar_file_name,avatar_content_type,..."}
```
If you’re using Rails 2.3, you can specify this as a default scope:
```
default_scope select_without_file_columns_for(:avatar)
```
Or if you’re using Rails 2.1 or 2.2 you can use it to create a named scope:
```
named_scope :without_file_data, select_without_file_columns_for(:avatar)
```
By default, URLs will be set to this pattern:
```
/:relative_root/:class/:attachment/:id?style=:style
```
Example:
```
/app-root-url/users/avatars/23?style=original
```
The idea here is that to retrieve a file from the database storage, you will need some controller's code to be executed. Once you pick a controller to use for downloading, you can add this line to generate the download action for the default URL/action (the plural attachment name), "avatars" in this example:
```
downloads_files_for :user, :avatar
```
Or you can write a download method manually if there are security, logging or other requirements. If you prefer a different URL for downloading files you can specify that in the model; e.g.:
```
has_attached_file :avatar,
                  :storage => :database,
                  :url =>'/users/show_avatar/:id/:style'
```
Add a route for the download to the controller which will handle downloads, if necessary. The default URL, /:relative_root/:class/:attachment/:id?style=:style, will be matched by the default route: :controller/:action/:id

Filtering auto_complete pick lists – part 2: using named scopes

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

I just updated my customized version of the auto_complete plugin to allow you to provide a named scope to auto_complete_for, in order to filter the auto_complete pick list options differently than the plugin does by default. The updated code is now on github:

http://github.com/patshaughnessy/auto_complete

This is based on the ideas from my last post, Andrew Ng’s original post and my friend Alex’s suggestion to use named scopes instead of manually modifying the find options. Here’s an example of how to use it taken from the auto_complete sample app I posted in January:

Add a named scope to your target model: For example suppose tasks belong to projects and have a named scope “by_project” which joins on the projects table and returns the tasks belonging to the project with the given name:
```
class Task < ActiveRecord::Base
  belongs_to :project
  named_scope :by_project,
    lambda { |project_name| {
      :include => :project,
      :conditions => [ "projects.name = ?", project_name ]
    } }
end
```
In the controller, pass a block to auto_complete_for to specify that a named scope should be used to generate the competion options. Here the “by_project” named scope will be used to handle the task auto complete requests, using the “project” HTTP parameter:
```
auto_complete_for :task, :name do | items, params |
  items.by_project(params['project'])
end
```

In the view, optionally specify additional parameters you might want to pass into your named scope: in my sample app I have a field called “project_name” elsewhere on my form:

<% fields_for_task task do |f| -%>
  …
  <%= f.text_field_with_auto_complete :task,
        :name,
        {},
        {
          :method => :get,
          :with =>"value + '&project=' + $F('project_name')"
        } %>
  …
  <% end -%>

So how does this work? Let’s take a look at my new implementation of auto_complete_for:

def auto_complete_for(object, method, options = {})
  define_method("auto_complete_for_#{object}_#{method}") do
    model = object.to_s.camelize.constantize
    find_options = { 
      :conditions => [ "LOWER(#{model.quoted_table_name}.#{method}) LIKE ?",
        '%' + params[object][method].downcase + '%' ], 
      :order => "#{model.quoted_table_name}.#{method} ASC",
      :limit => 10 }.merge!(options)
    @items = model.scoped(find_options)
    @items = yield(@items, params) if block_given?
    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end
end

One minor change I made here was to call “quoted_table_name” on the given model to specify the table name in the SQL generated to retrieve the auto complete results later. This was needed in case, like in my sample application, the controller specifies a named scope that joins with another table containing columns with the same name as the target model. If this isn’t the case, adding the table name to the SQL is harmless.

However, the most important 2 lines here are in bold: first we call a function called “scoped” to create an anonymous named scope based on the default auto_complete options “find_options:”

@items = model.scoped(find_options)

The exciting thing about this line, which Alex explained in his blog post, is that the use of named scopes delays the corresponding SQL statement from being executed until later when we actually access the query results in auto_complete_result. What happens instead is that an ActiveRecord:: NamedScope::Scope object is created, containing a temporary cache of the find options.

A good way to understand how this works is to try it in the Rails console:

complex-form-examples pat$ ./script/console 
Loading development environment (Rails 2.1.0)
>> find_options = { 
?>   :conditions => [ "LOWER(`tasks`.name) LIKE ?", '%t%' ], 
?>   :order => "`tasks`.name ASC",
?>   :limit => 10 }
=> {:order=>"name ASC", :conditions=>["LOWER(name) LIKE ?", "%t%"], :limit=>10}
>> Task.scoped(find_options)
=> [#<Task id: 4, project_id: 2, name: "Task 2a", due_at: nil, created_at: "2009-04-02 16:21:54",
updated_at: "2009-04-02 16:21:54">, #<Task id: 5, project_id: 2, name: "Task 2b", due_at: nil,
created_at: "2009-04-02 16:21:54", updated_at: "2009-04-02 16:21:54">, #<Task id: 6, project_id: 2,
name: "Task 2c", due_at: nil, created_at: "2009-04-02 16:21:54", updated_at: "2009-04-02
16:21:54">, #<Task id: 1, project_id: 1, name: "Task One", due_at: nil, created_at: "2009-04-02
16:21:30", updated_at: "2009-04-02 16:21:30">, #<Task id: 3, project_id: 1, name: "Task
Three", due_at: nil, created_at: "2009-04-02 16:21:30", updated_at: "2009-04-02 16:21:30">,
#<Task id: 2, project_id: 1, name: "Task Two", due_at: nil, created_at: "2009-04-02 16:21:30",
updated_at: "2009-04-02 16:21:30">]

Wait a minute! I thought the actual SQL execution was delayed by named scopes until I needed to access the results? Here the console has already displayed the query results, so the SQL statement must have been executed already. How and why did this happen? In this case, when you enter an expression into the Rails console and press ENTER, the expression is evaluated and then the “inspect” method is called on it. The problem is that the named scopes implementation has delegated the “inspect” method to another method, which executes the SQL statement and loads the query results.

We can use a trick in the console to open the ActiveRecord::NamedScope::Scope class and override the inspect method so the SQL is not executed, and prove to ourselves that “scoped()” actually does return a named scope object without executing the SQL statement:

>> module ActiveRecord
>> module NamedScope
>> class Scope
>> def inspect
>>   super # Avoids calling ActiveRecord::Base.find and calls Object.inspect
>> end
>> end
>> end
>> end
=> nil
>> Task.scoped(find_options)
=> #<ActiveRecord::NamedScope::Scope:0x21e65d4
      @proxy_options={:conditions=>["LOWER(`tasks`.name) LIKE ?", "%t%"],
          :order=>"`tasks`.name ASC", :limit=>10},
      @proxy_scope=Task(id: integer, project_id: integer, name: string,
          due_at: datetime, created_at: datetime, updated_at: datetime)>

So here we can see that “scoped” returns an ActiveRecord::NamedScope::Scope object, and that it has two interesting instance variables: proxy_scope and proxy_options. The first of these, proxy_options, contains the find options that were passed into the scoped() function, or into the “named_scope” declaration in your model. The second value, proxy_scope, indicates the parent scope or context in which this named scope object’s SQL statement should be run. In this example, that is the Task model itself. The named scope object is essentially a cache of the query options that will be user later when the query is executed.

Let’s see how this works in the auto_complete plugin. Back again to the new implementation of auto_complete_for, we have:

@items = model.scoped(find_options)
@items = yield(@items, params) if block_given?

The first line generates a ActiveRecord::NamedScope::Scope object, which is then passed into the block provided by the controller code, if any. Let’s take a look at my sample app’s implementation of the controller:

auto_complete_for :task, :name do | items, params |
  items.by_project(params['project'])
end

This is a good example of the second very cool feature of named scopes: that they are composable… in other words, that two or more named scopes can be combined together to form a single SQL statement that is executed only once! Let’s return to the same Rails console session with our redefined “inspect” method and see if we can understand a bit more about this:

>> Task.scoped(find_options).by_project 'Project One'
=> #<ActiveRecord::NamedScope::Scope:0x21d1864
  @proxy_options={
    :conditions=>["projects.name = ?", "Project One"],
    :include=>:project},
  @proxy_scope=
    #<ActiveRecord::NamedScope::Scope:0x21d1a30
      @proxy_options={
        :conditions=>["LOWER(`tasks`.name) LIKE ?", "%t%"],
        :order=>"`tasks`.name ASC",
        :limit=>10},
      @proxy_scope=Task(id: integer, project_id: integer, name: string, due_at: datetime, created_at: datetime, updated_at: datetime)
    >
  >

Now we can see that calling scoped(find_options).by_project just returns a chain of two named scopes: the first scope object with @proxy_scope set to the second one, and the second one with @proxy_scope set to the base model class. Later when this SQL query is executed, the code in NamedScope and ActiveRecord::Base will simply walk this chain of objects, accumulate the options into a single hash, convert the hash to SQL and execute it.

In auto_complete_for after the controller’s block returns, the “@items” value in auto_complete_for above is set to the parent/child named scope chain of objects, and then passed into auto_complete_result:

render :inline => "<%= auto_complete_result @items, '#{method}' %>"

Inside of auto_complete_result the @items value is used as if it were an array… like this:

def auto_complete_result(entries, field, phrase = nil)
  return unless entries
  items = entries.map { |entry|
    content_tag("li",
                phrase ? highlight(entry[field], phrase) : h(entry[field]))
  }
  content_tag("ul", items.uniq)
end

… to generate the HTML needed for the Prototype library’s implementation of the auto complete drop down box. The interesting thing here is that the SQL statement is executed as soon as the call to “map” is executed, accessing the elements of the “@items” array. This works because the ActiveRecord::NamedScope::Scope class redirects or delegates the [] and other array methods to ActiveRecord::Base.find. Here's the single SQL that is executed with the combined, accumulated query options:

SELECT `tasks`.`id` AS t0_r0, `tasks`.`project_id` AS t0_r1,
  `tasks`.`name` AS t0_r2, `tasks`.`due_at` AS t0_r3,
  `tasks`.`created_at` AS t0_r4, `tasks`.`updated_at` AS t0_r5,
  `projects`.`id` AS t1_r0, `projects`.`name` AS t1_r1,
  `projects`.`created_at` AS t1_r2, `projects`.`updated_at` AS t1_r3
FROM `tasks`
LEFT OUTER JOIN `projects` ON `projects`.id = `tasks`.project_id
WHERE ((projects.name = 'Project One') AND
  (LOWER(`tasks`.name) LIKE '%t%'))
ORDER BY `tasks`.name ASC LIMIT 10

In fact, in the original version of the auto_complete plugin before my changes to it for named scopes, the value passed into auto_complete_result was a simple array. The fact that named scopes are used now is entirely hidden from this code!

One last note here about named scope: as described in a comment in named_scope.rb from the Rails source code, the “proxy_options” method provides a convenient way to test the behavior of named scopes without actually checking the results of an actual SQL query. Here’s one of the tests I wrote for my new version of auto_complete_for:

def test_default_auto_complete_for
  get :auto_complete_for_some_model_some_field,
      :some_model => { :some_field => "some_value" }
  default_auto_complete_find_options = @controller.items.proxy_options
  assert_equal "`some_models`.some_field ASC", 
               default_auto_complete_find_options[:order]
  assert_equal 10, default_auto_complete_find_options[:limit]
  assert_equal ["LOWER(`some_models`.some_field) LIKE ?", "%some_value%"],
               default_auto_complete_find_options[:conditions]
end

Since I didn’t want to go to the trouble of setting up an actual in-memory database using SQLite, or to introduce mocha or some other mocking framework to the auto_complete tests, all I had to do was just call @controller.items.proxy_options and check that the find options are as expected. (I also had to expose “items” in the mock controller using attr_reader.) I have another test that checks that the controller’s block is called and it’s named scope options are present as expected… this test uses the proxy_scope method to walk up the chain to the parent named scope and get it’s proxy_options. See auto_complete_test.rb for details.

Filtering auto_complete pick lists – part 2: using named scopes

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

http://github.com/patshaughnessy/auto_complete

Add a named scope to your target model: For example suppose tasks belong to projects and have a named scope “by_project” which joins on the projects table and returns the tasks belonging to the project with the given name:
```
class Task < ActiveRecord::Base
  belongs_to :project
  named_scope :by_project,
    lambda { |project_name| {
      :include => :project,
      :conditions => [ "projects.name = ?", project_name ]
    } }
end
```
In the controller, pass a block to auto_complete_for to specify that a named scope should be used to generate the competion options. Here the “by_project” named scope will be used to handle the task auto complete requests, using the “project” HTTP parameter:
```
auto_complete_for :task, :name do | items, params |
  items.by_project(params['project'])
end
```

In the view, optionally specify additional parameters you might want to pass into your named scope: in my sample app I have a field called “project_name” elsewhere on my form:

<% fields_for_task task do |f| -%>
  …
  <%= f.text_field_with_auto_complete :task,
        :name,
        {},
        {
          :method => :get,
          :with =>"value + '&project=' + $F('project_name')"
        } %>
  …
  <% end -%>

So how does this work? Let’s take a look at my new implementation of auto_complete_for:

def auto_complete_for(object, method, options = {})
  define_method("auto_complete_for_#{object}_#{method}") do
    model = object.to_s.camelize.constantize
    find_options = { 
      :conditions => [ "LOWER(#{model.quoted_table_name}.#{method}) LIKE ?",
        '%' + params[object][method].downcase + '%' ], 
      :order => "#{model.quoted_table_name}.#{method} ASC",
      :limit => 10 }.merge!(options)
    @items = model.scoped(find_options)
    @items = yield(@items, params) if block_given?
    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end
end

However, the most important 2 lines here are in bold: first we call a function called “scoped” to create an anonymous named scope based on the default auto_complete options “find_options:”

@items = model.scoped(find_options)

A good way to understand how this works is to try it in the Rails console:

complex-form-examples pat$ ./script/console 
Loading development environment (Rails 2.1.0)
>> find_options = { 
?>   :conditions => [ "LOWER(`tasks`.name) LIKE ?", '%t%' ], 
?>   :order => "`tasks`.name ASC",
?>   :limit => 10 }
=> {:order=>"name ASC", :conditions=>["LOWER(name) LIKE ?", "%t%"], :limit=>10}
>> Task.scoped(find_options)
=> [#<Task id: 4, project_id: 2, name: "Task 2a", due_at: nil, created_at: "2009-04-02 16:21:54",
updated_at: "2009-04-02 16:21:54">, #<Task id: 5, project_id: 2, name: "Task 2b", due_at: nil,
created_at: "2009-04-02 16:21:54", updated_at: "2009-04-02 16:21:54">, #<Task id: 6, project_id: 2,
name: "Task 2c", due_at: nil, created_at: "2009-04-02 16:21:54", updated_at: "2009-04-02
16:21:54">, #<Task id: 1, project_id: 1, name: "Task One", due_at: nil, created_at: "2009-04-02
16:21:30", updated_at: "2009-04-02 16:21:30">, #<Task id: 3, project_id: 1, name: "Task
Three", due_at: nil, created_at: "2009-04-02 16:21:30", updated_at: "2009-04-02 16:21:30">,
#<Task id: 2, project_id: 1, name: "Task Two", due_at: nil, created_at: "2009-04-02 16:21:30",
updated_at: "2009-04-02 16:21:30">]

>> module ActiveRecord
>> module NamedScope
>> class Scope
>> def inspect
>>   super # Avoids calling ActiveRecord::Base.find and calls Object.inspect
>> end
>> end
>> end
>> end
=> nil
>> Task.scoped(find_options)
=> #<ActiveRecord::NamedScope::Scope:0x21e65d4
      @proxy_options={:conditions=>["LOWER(`tasks`.name) LIKE ?", "%t%"],
          :order=>"`tasks`.name ASC", :limit=>10},
      @proxy_scope=Task(id: integer, project_id: integer, name: string,
          due_at: datetime, created_at: datetime, updated_at: datetime)>

Let’s see how this works in the auto_complete plugin. Back again to the new implementation of auto_complete_for, we have:

@items = model.scoped(find_options)
@items = yield(@items, params) if block_given?

auto_complete_for :task, :name do | items, params |
  items.by_project(params['project'])
end

>> Task.scoped(find_options).by_project 'Project One'
=> #<ActiveRecord::NamedScope::Scope:0x21d1864
  @proxy_options={
    :conditions=>["projects.name = ?", "Project One"],
    :include=>:project},
  @proxy_scope=
    #<ActiveRecord::NamedScope::Scope:0x21d1a30
      @proxy_options={
        :conditions=>["LOWER(`tasks`.name) LIKE ?", "%t%"],
        :order=>"`tasks`.name ASC",
        :limit=>10},
      @proxy_scope=Task(id: integer, project_id: integer, name: string, due_at: datetime, created_at: datetime, updated_at: datetime)
    >
  >

render :inline => "<%= auto_complete_result @items, '#{method}' %>"

Inside of auto_complete_result the @items value is used as if it were an array… like this:

def auto_complete_result(entries, field, phrase = nil)
  return unless entries
  items = entries.map { |entry|
    content_tag("li",
                phrase ? highlight(entry[field], phrase) : h(entry[field]))
  }
  content_tag("ul", items.uniq)
end

SELECT `tasks`.`id` AS t0_r0, `tasks`.`project_id` AS t0_r1,
  `tasks`.`name` AS t0_r2, `tasks`.`due_at` AS t0_r3,
  `tasks`.`created_at` AS t0_r4, `tasks`.`updated_at` AS t0_r5,
  `projects`.`id` AS t1_r0, `projects`.`name` AS t1_r1,
  `projects`.`created_at` AS t1_r2, `projects`.`updated_at` AS t1_r3
FROM `tasks`
LEFT OUTER JOIN `projects` ON `projects`.id = `tasks`.project_id
WHERE ((projects.name = 'Project One') AND
  (LOWER(`tasks`.name) LIKE '%t%'))
ORDER BY `tasks`.name ASC LIMIT 10

def test_default_auto_complete_for
  get :auto_complete_for_some_model_some_field,
      :some_model => { :some_field => "some_value" }
  default_auto_complete_find_options = @controller.items.proxy_options
  assert_equal "`some_models`.some_field ASC", 
               default_auto_complete_find_options[:order]
  assert_equal 10, default_auto_complete_find_options[:limit]
  assert_equal ["LOWER(`some_models`.some_field) LIKE ?", "%some_value%"],
               default_auto_complete_find_options[:conditions]
end

Filtering auto_complete pick lists

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

I’ve written a lot here during the past few months about the auto_complete plugin and how to get it to work with repeated text fields on a complex form. Back in January I modified Ryan Bates’s complex forms sample app to illustrate how to use my version of the auto complete plugin to handle repeated text fields. Here’s what the form looks like in that sample app:

Here as the user types into a “Task” text field, a list of all of the existing task names in the system that match the characters typed in by the user are displayed in a drop down list. But what if I didn’t want to display all of the matching task names? What if I wanted to display only the tasks for a given project? Or if I wanted to filter the task names in some other way based on other field values?

In this simple example, what if I only wanted to display Tasks 2a, 2b and 2c, since they belonged to Project Two?

Today I took a look at this problem and expected to see a number of simple solutions, but instead I was surprised to find that it is fairly difficult to do this. I got started by reading this nice solution from Andrew Ng (nice work Andrew!). Andrew explains how to get the Prototype javascript code to pass an additional HTTP parameter to the server when the user types into an autocomplete field. This additional parameter can then be used to filter the list of auto complete options differently. I’ll let you read the details, but basically Andrew found that you can use a Javascript callback function like this to load a value from another field on your form, and pass it to the server in the Ajax request as an additional query string parameter:

<script type="text/javascript">
  new Ajax.Autocompleter(
    'task_name', 
    'task_name_auto_complete', 
    '/projects/auto_complete_for_task_name', 
    { callback: function(e, qs) {
        return qs + '&project=' + $F('project_name');
      }
    }
  );
</script>

(I’ve renamed the variables to use my project/tasks example.) What I didn’t like about this was the need to manually code all of this javascript; there must be a way to get the auto_complete plugin to do this instead… and there is! If you look at the definition of text_field_with_auto_complete in auto_complete_macros_helper.rb, you’ll see that it takes both tag_options and completion_options as parameters, and eventually calls auto_complete_field with the completion_options. Here’s what auto_complete_field looks like in the auto_complete plugin:

def auto_complete_field(field_id, options = {})
  function =  "var #{field_id}_auto_completer = new Ajax.Autocompleter("
  function << "'#{field_id}', "

etc...

  js_options = {}
  js_options[:tokens] = etc...
  js_options[:callback]   =
    "function(element, value) { return #{options[:with]} }" if options[:with]
  js_options[:indicator]  = etc...
  js_options[:select]     = etc...
  js_options[:paramName]  = etc...

etc...

  function << (', ' + options_for_javascript(js_options) + ')')
  javascript_tag(function)
end

If you look closely at the line I bolded above, you’ll see that we can actually generate Andrew’s Javascript callback function automatically by simply passing in a value for the “with” completion option when we call text_field_with_auto_complete in our view, like this:

text_field_with_auto_complete :task, :name, {},
  {
    :method => :get,
    :with =>"value + '&project=' + $F('project_name')"
  }

Again, this line of Javascript code is called when the user types into the task name field, and appends “&project=XYZ” to the query string for the Ajax request. “XYZ” is the name of the project typed in by the user on the same form, loaded with prototype’s “$F” (Form element get value) function. The “:method => :get” completion option is used to avoid problems with CSRF protection; see http://www.ruby-forum.com/topic/128970. If you look at your server’s log file, you’ll see HTTP requests that look something like this now:

127.0.0.1 - - [13/Mar/2009:16:17:03 EDT] "
GET /projects/auto_complete_for_task_name?task%5Bname%5D=T&project=Project%20Two
HTTP/1.1" 200 57

Here we can see the “auto_complete_for_task_name” route is called and given two request parameters: “task[name]” and “project”. The task name is the standard parameter generated by the autocompleter javascript, and “project” is the additional parameter created by the callback function generated by the :with option.

Now… how do we handle the “project” parameter in our controller code? Without modifying the auto_complete plugin itself, you would have to write your own controller method and not use the “auto_complete_for” macro at all. Andrew shows how to do this on his blog. What I want to explore here now is whether there’s a way to change the auto_complete_for method to allow for customizations of the query used to load the auto complete options.

To understand the problem a bit better, let’s take a look at how “auto_complete_for” is implemented in the auto_complete plugin:

def auto_complete_for(object, method, options = {})
  define_method("auto_complete_for_#{object}_#{method}") do
    find_options = { 
      :conditions => [ "LOWER(#{method}) LIKE ?", '%' +
        params[object][method].downcase + '%' ], 
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)
    @items = object.to_s.camelize.constantize.find(:all, find_options)
    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end	
end

When this is called as your Rails application initializes, it adds a new method to your controller called something like “auto_complete_for_task_name” with your model and column names instead. What we want to do is filter the query results differently, by using a new HTTP parameter – so we need to modify the “conditions” hash passed into find :all. At first I tried to do this by passing in different values for the “options” parameter, since that’s merged with the default options and then passed into find :all. However, the problem with this approach is that whatever you pass in using “options” will not have access to the request parameters, since it’s passed in when the controller is initialized, and not when the HTTP request is received.

So the solution is to pass in a block that is evaluated when the request is received, and when the generated method is actually called. I wrote a variation on auto_complete_for called “filtered_auto_complete_for:”

def filtered_auto_complete_for(object, method)
  define_method("auto_complete_for_#{object}_#{method}") do
    find_options = { 
      :conditions => [ "LOWER(#{method}) LIKE ?", '%' +
        params[object][method].downcase + '%' ], 
      :order => "#{method} ASC",
      :limit => 10 }
    yield find_options, params
    @items = object.to_s.camelize.constantize.find(:all, find_options)
    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end
end

Filtered_auto_complete_for takes a block and evaluates it when the actual HTTP Ajax request is received from the auto complete Javascript. The block is provided with the find options hash and also the request parameters. This enables the controller’s block to modify the find options in any way it would like, possibly using the HTTP request parameters provided. I’ve also removed the options parameter since that’s not necessary any more.

As an example, here’s my sample app’s controller code:

class ProjectsController < ApplicationController

  # Handle auto complete for project names as usual:
  auto_complete_for :project, :name

  # For task name auto complete, only display tasks
  # that belong to the given project: 
  filtered_auto_complete_for :task, :name do | find_options, params|
    find_options.merge!(
      {
        :include => :project,
        :conditions => [ "LOWER(tasks.name) LIKE ? AND projects.name = ?",
                         '%' + params['task']['name'].downcase + '%',
                         params['project'] ],
        :order => "tasks.name ASC"
      }
    )
  end

  def index
    @projects = Project.find(:all)
  end

  etc...

The code in this sample block modifies the find_options by adding “:include => :project”. This causes ActiveRecord to use a JOIN to include columns from the project table in the query (in this sample app Project has_many Tasks, and each Task belongs_to a Project). Then it matches on the project name, in addition to the portion of the task name typed in by the user so far. This limits the auto complete values to just the tasks that belong to the given project:

When I have time during the next few days I’ll add “filtered_auto_complete_for” to my forked version of the auto_complete plugin… first I need to write some unit tests for it, and be sure it works as intended. After that, I’ll post this sample app back on github and you can try it yourself.

Filtering auto_complete pick lists

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

In this simple example, what if I only wanted to display Tasks 2a, 2b and 2c, since they belonged to Project Two?

<script type="text/javascript">
  new Ajax.Autocompleter(
    'task_name', 
    'task_name_auto_complete', 
    '/projects/auto_complete_for_task_name', 
    { callback: function(e, qs) {
        return qs + '&project=' + $F('project_name');
      }
    }
  );
</script>

def auto_complete_field(field_id, options = {})
  function =  "var #{field_id}_auto_completer = new Ajax.Autocompleter("
  function << "'#{field_id}', "

etc...

  js_options = {}
  js_options[:tokens] = etc...
  js_options[:callback]   =
    "function(element, value) { return #{options[:with]} }" if options[:with]
  js_options[:indicator]  = etc...
  js_options[:select]     = etc...
  js_options[:paramName]  = etc...

etc...

  function << (', ' + options_for_javascript(js_options) + ')')
  javascript_tag(function)
end

text_field_with_auto_complete :task, :name, {},
  {
    :method => :get,
    :with =>"value + '&project=' + $F('project_name')"
  }

127.0.0.1 - - [13/Mar/2009:16:17:03 EDT] "
GET /projects/auto_complete_for_task_name?task%5Bname%5D=T&project=Project%20Two
HTTP/1.1" 200 57

To understand the problem a bit better, let’s take a look at how “auto_complete_for” is implemented in the auto_complete plugin:

def auto_complete_for(object, method, options = {})
  define_method("auto_complete_for_#{object}_#{method}") do
    find_options = { 
      :conditions => [ "LOWER(#{method}) LIKE ?", '%' +
        params[object][method].downcase + '%' ], 
      :order => "#{method} ASC",
      :limit => 10 }.merge!(options)
    @items = object.to_s.camelize.constantize.find(:all, find_options)
    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end	
end

def filtered_auto_complete_for(object, method)
  define_method("auto_complete_for_#{object}_#{method}") do
    find_options = { 
      :conditions => [ "LOWER(#{method}) LIKE ?", '%' +
        params[object][method].downcase + '%' ], 
      :order => "#{method} ASC",
      :limit => 10 }
    yield find_options, params
    @items = object.to_s.camelize.constantize.find(:all, find_options)
    render :inline => "<%= auto_complete_result @items, '#{method}' %>"
  end
end

As an example, here’s my sample app’s controller code:

class ProjectsController < ApplicationController

  # Handle auto complete for project names as usual:
  auto_complete_for :project, :name

  # For task name auto complete, only display tasks
  # that belong to the given project: 
  filtered_auto_complete_for :task, :name do | find_options, params|
    find_options.merge!(
      {
        :include => :project,
        :conditions => [ "LOWER(tasks.name) LIKE ? AND projects.name = ?",
                         '%' + params['task']['name'].downcase + '%',
                         params['project'] ],
        :order => "tasks.name ASC"
      }
    )
  end

  def index
    @projects = Project.find(:all)
  end

  etc...

Database storage for Paperclip

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Update April 2009:
I just rewrote this to use the same database table as the target model to save the file data, and not a separate database table. The “create_table” migration I describe here below will no longer work; instead you should create one or more columns in the same table as the target model to hold the file data. Please read this update for more details.

Paperclip from Thoughtbot is a fantastic bit of code that allows you to easily upload files to your Rails app and later manage them as just another attribute on your model object. If you’re not familiar with Paperclip you should start by reading Thoughtbot’s Paperclip intro page; Ryan Bates also did a screen cast on Paperclip usage. By default it supports saving the file attachments on the server file system, and also there’s an option for saving the files in Amazon’s S3 service. One reason I decided to use Paperclip in a recent project was that the implementation and usage were both much simpler and cleaner than attachment_fu, the other popular Rails plugin for file upload and management.

One thing that Thoughtbot decided not to implement was the ability to store files in a database table, rather than on the file system. It doesn’t make a lot of sense to do this for most normal web application deployments, since serving files via Apache directly from the file system is obviously much faster and avoids the need to call your Rails stack at all for download requests. However, I work in an Enterprise IT environment that has a lot of experience with Oracle, and finds it easier to manage file attachments with a database table. I also have requirements around file encryption, security, etc.

Since Paperclip doesn’t include a database storage option, I decided to write one. Here’s what I came up with: http://github.com/patshaughnessy/paperclip

I added a new storage module called Paperclip::Storage::Database. See lib/paperclip/storage.rb for details; Paperclip::Storage::Database is at the bottom of the file. I’d love any feedback or suggestions about the usage/design of how the database storage option would work with your application, or on the implementation itself.

I’ll be blogging here in the coming weeks with a detailed explanation of how Paperclip database storage works, and a working sample application that illustrates how to use it.

For now, here’s the usage description from lib/paperclip/storage.rb for specifying database storage for your Paperclip app. You need to follow these steps in addition to the standard Paperclip setup steps from Thoughtbot.

In your model specify the "database" storage option; for example:
```
  has_attached_file :avatar, :storage => :database
```
The files will be stored in a new database table named with the plural attachment name by default, "avatars" in this example.
You need to create this new storage table with at least these columns:
- file_contents
- style
- the primary key for the parent model (e.g. user_id)

Note the "binary" migration will not work for the LONGBLOG type in MySQL for the file_contents column. You may need to craft a SQL statement for your migration, depending on which database server you are using. Here's an example migration for MySQL:
```
create_table :avatars do |t|
  t.string :style
  t.integer :user_id
  t.timestamps
end
execute 'ALTER TABLE avatars ADD COLUMN file_contents LONGBLOB'
```
You can optionally specify any storage table name you want as follows:
```
  has_attached_file :avatar, :storage => :database,
                    :database_table => 'avatar_files'
```
By default, URLs will be set to this pattern:
```
  /:relative_root/:class/:attachment/:id?style=:style
```
Example:
```
  /app-root-url/users/avatars/23?style=original
```
The idea here is that to retrieve a file from the database storage, you will need some controller's code to be executed. Once you pick a controller to use for downloading, you can add this line to generate the download action for the default URL/action (the plural attachment name), "avatars" in this example:
```
  downloads_files_for :user, :avatar
```
Or you can write a download method manually if there are security, logging or other requirements. If you prefer a different URL for downloading files you can specify that in the model; e.g.:
```
  has_attached_file :avatar, :storage => :database,
                    :url => '/users/show_avatar/:id/:style'
```
Add a route for the download to the controller which will handle downloads, if necessary. The default URL (/:relative_root/:class/:attachment/:id?style=:style) will be matched by the default route:
```
map.connect ':controller/:action/:id'
```

Database storage for Paperclip

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Since Paperclip doesn’t include a database storage option, I decided to write one. Here’s what I came up with: http://github.com/patshaughnessy/paperclip

I’ll be blogging here in the coming weeks with a detailed explanation of how Paperclip database storage works, and a working sample application that illustrates how to use it.

In your model specify the "database" storage option; for example:
```
  has_attached_file :avatar, :storage => :database
```
The files will be stored in a new database table named with the plural attachment name by default, "avatars" in this example.
You need to create this new storage table with at least these columns:
- file_contents
- style
- the primary key for the parent model (e.g. user_id)

Note the "binary" migration will not work for the LONGBLOG type in MySQL for the file_contents column. You may need to craft a SQL statement for your migration, depending on which database server you are using. Here's an example migration for MySQL:
```
create_table :avatars do |t|
  t.string :style
  t.integer :user_id
  t.timestamps
end
execute 'ALTER TABLE avatars ADD COLUMN file_contents LONGBLOB'
```
You can optionally specify any storage table name you want as follows:
```
  has_attached_file :avatar, :storage => :database,
                    :database_table => 'avatar_files'
```
By default, URLs will be set to this pattern:
```
  /:relative_root/:class/:attachment/:id?style=:style
```
Example:
```
  /app-root-url/users/avatars/23?style=original
```
The idea here is that to retrieve a file from the database storage, you will need some controller's code to be executed. Once you pick a controller to use for downloading, you can add this line to generate the download action for the default URL/action (the plural attachment name), "avatars" in this example:
```
  downloads_files_for :user, :avatar
```
Or you can write a download method manually if there are security, logging or other requirements. If you prefer a different URL for downloading files you can specify that in the model; e.g.:
```
  has_attached_file :avatar, :storage => :database,
                    :url => '/users/show_avatar/:id/:style'
```
Add a route for the download to the controller which will handle downloads, if necessary. The default URL (/:relative_root/:class/:attachment/:id?style=:style) will be matched by the default route:
```
map.connect ':controller/:action/:id'
```

The auto_complete plugin modified for repeated fields

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

This version of auto_complete will support text fields that are repeated more than once on a complex form. It allows you to call text_field_with_auto_complete inside of fields_for or form_for.

Code: http://github.com/patshaughnessy/auto_complete

Install:

script/plugins install git://github.com/patshaughnessy/auto_complete.git

More information:

Note: my changes were previously in a separate plugin called “repeated_auto_complete,” but now I’ve merged the changes into this forked repository of auto_complete.

Sample app for auto complete on a complex form

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

In his “Complex Forms” series (part 1, part 2 and part 3) Ryan Bates does a fantastic job explaining how to create a complex form containing a series of parent/child text fields while still using simple, clean code. Ryan also pushed the sample application from the screen cast onto github, here: http://github.com/ryanb/complex-form-examples

Here’s what Ryan’s sample complex form looks like:

One problem I ran into while using Ryan’s suggestions on a complex form I was writing was how to get auto complete behavior to work properly using the auto_complete plugin for fields that are repeated, like the “task” field here. As I explained in a previous blog post, this causes a lot of problems for the auto_complete plugin since the <input id=””> attributes are no longer unique, breaking the javascript used for auto complete. I was able to solve the problem by modifying the auto_complete plugin to generate unique <input id=””> attributes, among other things.

Here I want to take some time to show how to use my modified auto_complete plugin, using the same sample application from Ryan’s screencast. To get started, let’s clone the git repository for the sample app - this command refers to my fork of Ryan's complex-form-examples repository: http://github.com/patshaughnessy/complex-form-examples

$ git clone git://github.com/patshaughnessy/complex-form-examples.git
Initialized empty Git repository in /Users/pat/rails-apps/complex-form-examples/.git/
remote: Counting objects: 192, done.
remote: Compressing objects: 100% (122/122), done.
remote: Total 192 (delta 71), reused 159 (delta 58)
Receiving objects: 100% (192/192), 86.19 KiB | 68 KiB/s, done.
Resolving deltas: 100% (71/71), done.

Ryan had saved various versions of the sample app in different git branches, so to avoid confusion I’ve saved my auto complete related changes in a branch called “auto_complete.” So next you should switch to that branch:

$ git checkout origin/auto_complete
Note: moving to "origin/auto_complete" which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b <new_branch_name>
HEAD is now at 4f3e908... Sample app code changes for auto_complete

Now you will see my changes in Ryans’ code, except for one more detail: I saved my version of the auto_complete plugin in this git repository as a submodule. To get the plugin’s code for this sample app you need to run these commands:

$ git submodule init
Submodule 'vendor/plugins/auto_complete' (git://github.com/patshaughnessy/auto_complete.git) registered for path 'vendor/plugins/auto_complete'
$ git submodule update
Initialized empty Git repository in /Users/pat/rails-apps/complex-form-examples/vendor/plugins/auto_complete/.git/
remote: Counting objects: 22, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 22 (delta 5), reused 0 (delta 0)
Receiving objects: 100% (22/22), 7.65 KiB, done.
Resolving deltas: 100% (5/5), done.
Submodule path 'vendor/plugins/auto_complete': checked out '0814a25a754a235c5cf6f7a258fa405059a5ca6f'

(Note that normally to install the plugin in your app you would just run “script/plugin install git://github.com/patshaughnessy/auto_complete.git” – the submodule is only present in this sample app.) Now to setup and run the application you just need to:

Enter your MySQL details in config/database.yml
Run rake db:migrate
Run script/server to launch the app

If you enter a few records you should be able to see the auto complete drop down, even for the repeated field:

Let’s review the changes I’ve made to Ryan’s code aside from adding my modified version of auto_complete to vendor/plugins. First, I added the standard auto_complete handlers to projects_controller.rb for both the project and task fields:

class ProjectsController < ApplicationController
  auto_complete_for :project, :name
  auto_complete_for :task, :name
…

Next I modified the project text field to use auto complete (in views/projects/_form.html.erb):

<p>
  <%= f.label :name, "Project:" %>
  <%= text_field_with_auto_complete :project, :name, {}, {:method => :get } %>
</p>

These two changes enable auto complete for the single project text field, just the same way you would with any text field and the standard auto_complete plugin. However, to get auto complete to work with the repeated tasks field, we need to use changes I’ve made to auto_complete. First, in helpers/projects_helper.rb change the “fields_for_task” method to use my new auto_complete_fields_for method, like this:

def fields_for_task(task, &block)
  new_or_existing = task.new_record? ? 'new' : 'existing'
  prefix = "project[#{new_or_existing}_task_attributes][]"
  auto_complete_fields_for(prefix, task, &block)
end

This causes my code in auto_complete to provide a custom form builder object, which we can use in the view as follows (views/projects/_task.html.erb):

<% fields_for_task task do |f| -%>
  <%= error_messages_for :task, :object => task %>
  <%= f.label :name, "Task:" %>
  <%= f.text_field_with_auto_complete :task, :name, {}, {:method => :get } %>
  <%= link_to_function "remove", "$(this).up('.task').remove()" %>
<% end -%>

Here I’ve called “text_field_with_auto_complete” as a method on the “f” form builder object yielded by fields_for_task. This will cause the auto complete script and HTML to be generated with unique <input id=””> attributes, allowing the auto complete behavior to work properly.

One other change I made was also to helpers/projects_helper.rb:

def add_task_link(name)
  link_to_remote "Add a task", :url => {
                                 :controller => "projects",
                                 :action => "add_task_script"
                               }
end

Here I’ve changed Ryan’s “link_to_function” call to “link_to_remote.” As Ryan explains in part 2 of his complex forms screen cast, link_to_function avoids an AJAX call to the server to obtain the HTML for each new task <input> tag, avoiding unnecessary load on the server since all of the task fields are the same. However, with my changes to auto_complete the HTML generated for the task field contains random numbers which are different for each copy of the field… meaning that we do need a separate call to the server to obtain the task field HTML and script. To handle the call from link_to_remote, I’ve added a new file, views/projects/add_task_script.rjs:

page.insert_html :bottom, :tasks, :partial => 'task', :object => Task.new

… which works essentially the same way as described by Ryan, but is called each time the user clicks “Add a task.”

The last change I made to the sample app is in routes.rb; these changes are required to allow the controller to map the Ajax requests, and to insure that these requests use GET, and not POST HTTP requests:

map.connect 'projects/auto_complete_for_project_name',
            :controller => 'projects',
            :action => 'auto_complete_for_project_name'
map.connect 'projects/auto_complete_for_task_name',
            :controller => 'projects',
            :action => 'auto_complete_for_task_name'
map.connect 'projects/add_task_script',
            :controller => 'projects',
            :action => 'add_task_script'
map.resources :projects,
              :collection => {
                :auto_complete_for_project_name => :get,
                :auto_complete_for_task_name => :get
              }

This certainly seems very ugly, and probably could be simplified! But for now, we need this code to avoid problems with CRSF protection; see http://www.ruby-forum.com/topic/128970.

Sample app for auto complete on a complex form

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

Update June 2009: I just added support to my version of auto_complete to support Rails 2.3 nested attributes; for more details see: http://patshaughnessy.net/repeated_auto_complete. This sample app will still work since the git submodule commands below refer to the old version of my auto_complete plugin; I’ll try to write another sample app or tutorial in the next week or two illustrating how to use Rails 2.3 to build this same app… in a much simpler way! If you download my version of auto_complete now from github, please refer to my description of the usage changes.

Here’s what Ryan’s sample complex form looks like:

$ git clone git://github.com/patshaughnessy/complex-form-examples.git
Initialized empty Git repository in /Users/pat/rails-apps/complex-form-examples/.git/
remote: Counting objects: 192, done.
remote: Compressing objects: 100% (122/122), done.
remote: Total 192 (delta 71), reused 159 (delta 58)
Receiving objects: 100% (192/192), 86.19 KiB | 68 KiB/s, done.
Resolving deltas: 100% (71/71), done.

$ cd complex-form-examples
$ git checkout origin/auto_complete
Note: moving to "origin/auto_complete" which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b <new_branch_name>
HEAD is now at 4f3e908... Sample app code changes for auto_complete

$ git submodule init
Submodule 'vendor/plugins/auto_complete' (git://github.com/patshaughnessy/auto_complete.git) registered for path 'vendor/plugins/auto_complete'
$ git submodule update
Initialized empty Git repository in /Users/pat/rails-apps/complex-form-examples/vendor/plugins/auto_complete/.git/
remote: Counting objects: 22, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 22 (delta 5), reused 0 (delta 0)
Receiving objects: 100% (22/22), 7.65 KiB, done.
Resolving deltas: 100% (5/5), done.
Submodule path 'vendor/plugins/auto_complete': checked out '0814a25a754a235c5cf6f7a258fa405059a5ca6f'

Enter your MySQL details in config/database.yml
Run rake db:migrate
Run script/server to launch the app

If you enter a few records you should be able to see the auto complete drop down, even for the repeated field:

class ProjectsController < ApplicationController
  auto_complete_for :project, :name
  auto_complete_for :task, :name
…

Next I modified the project text field to use auto complete (in views/projects/_form.html.erb):

<p>
  <%= f.label :name, "Project:" %>
  <%= text_field_with_auto_complete :project, :name, {}, {:method => :get } %>
</p>

def fields_for_task(task, &block)
  new_or_existing = task.new_record? ? 'new' : 'existing'
  prefix = "project[#{new_or_existing}_task_attributes][]"
  auto_complete_fields_for(prefix, task, &block)
end

This causes my code in auto_complete to provide a custom form builder object, which we can use in the view as follows (views/projects/_task.html.erb):

<% fields_for_task task do |f| -%>
  <%= error_messages_for :task, :object => task %>
  <%= f.label :name, "Task:" %>
  <%= f.text_field_with_auto_complete :task, :name, {}, {:method => :get } %>
  <%= link_to_function "remove", "$(this).up('.task').remove()" %>
<% end -%>

One other change I made was also to helpers/projects_helper.rb:

def add_task_link(name)
  link_to_remote "Add a task", :url => {
                                 :controller => "projects",
                                 :action => "add_task_script"
                               }
end

page.insert_html :bottom, :tasks, :partial => 'task', :object => Task.new

… which works essentially the same way as described by Ryan, but is called each time the user clicks “Add a task.”

map.connect 'projects/auto_complete_for_project_name',
            :controller => 'projects',
            :action => 'auto_complete_for_project_name'
map.connect 'projects/auto_complete_for_task_name',
            :controller => 'projects',
            :action => 'auto_complete_for_task_name'
map.connect 'projects/add_task_script',
            :controller => 'projects',
            :action => 'add_task_script'
map.resources :projects,
              :collection => {
                :auto_complete_for_project_name => :get,
                :auto_complete_for_task_name => :get
              }

This certainly seems very ugly, and probably could be simplified! But for now, we need this code to avoid problems with CRSF protection; see http://www.ruby-forum.com/topic/128970.

Repeated_auto_complete changes merged into auto_complete

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

In October I described how the auto_complete plugin doesn’t work when text fields are repeated more than once on a complex form. I went on to write a plugin called “repeated_auto_complete” which modified the way the standard auto_complete plugin works and fixed this problem by adding random numbers to <input id=""> attributes among other changes.

Since it’s much cleaner to have a single auto_complete plugin rather than two separate plugins, I’ve merged my changes to auto_complete into the original version, and pushed them to github as a new fork: http://github.com/patshaughnessy/auto_complete

To install and use my modified version of auto_complete first remove the standard auto_complete plugin from your app if necessary, and install with:

script/plugin install git://github.com/patshaughnessy/auto_complete.git

To use auto complete in a complex form, you write “auto_complete_fields_for” or “auto_complete_form_for” in your view, and then call text_field_with_auto_complete on the form builder object, as follows:

<% for person in @group.people %>
  <% auto_complete_fields_for "group[person_attributes][]", person do |form| %>
    Person <%= person_form.label :name %><br />
    <%= form.text_field_with_auto_complete :person, :name, {},
                                           {:method => :get}  %>
  <% end %>
<% end %>

To understand my changes to the plugin, let’s first look at how the original auto_complete works. If you add this line to your view:

<%= text_field_with_auto_complete :project, :name, {}, {:method => :get } %>

…then you get HTML and script that looks like this (style sheet omitted):

<input id="project_name" name="project[name]" size="30" type="text" />
<div class="auto_complete" id="project_name_auto_complete"></div>
<script type="text/javascript">
//<![CDATA[
var project_name_auto_completer = new Ajax.Autocompleter('project_name',
'project_name_auto_complete', '/projects/auto_complete_for_project_name',
{method:'get'})
//]]>
</script>

The original text_field_with_auto_complete method looked like this:

def text_field_with_auto_complete(object, method, tag_options = {},
                                  completion_options = {})
    (completion_options[:skip_style] ? "" : auto_complete_stylesheet) +
    text_field(object, method, tag_options) +
    content_tag("div", "", :id => "#{object}_#{method}_auto_complete",
                :class => "auto_complete") +
    auto_complete_field(
        "#{object}_#{method}",
        { 
          :url => { :action => "auto_complete_for_#{object}_#{method}" }
        }.update(completion_options))
  end

You can see that it calls “text_field” in ActionView::Helpers::FormHelper to generate the actual <input> tag for the form, in addition to generating the HTML and script needed for the auto completion behavior.

What I wanted to achieve in the modified plugin was to allow the view to contain code like this:

<% auto_complete_fields_for task do |f| %>
  <%= f.label :name, "Task:" %>
  <%= f.text_field_with_auto_complete :task, :name, {}, {:method => :get } %>
<% end %>

To make this work, we need a new version of text_field_with_auto_complete that calls text_field from ActionView::Helpers::FormBuilder, and not ActionView::Helpers::FormHelper, generating an <input> tag similar to what this call would generate:

<% fields_for task do |f| %>
  <%= f.text_field :name %>
<% end %>

To do this, I first refactored the original text_field_with_auto_complete in auto_complete_macros_helper.rb:

def text_field_with_auto_complete(object, method, tag_options = {},
                                  completion_options = {})
  auto_complete_field_with_style_and_script(object, method, tag_options,
                                            completion_options) do
    text_field(object, method, tag_options)
  end
end

def auto_complete_field_with_style_and_script(object, method,
                                              tag_options = {},
                                              completion_options = {})
  (completion_options[:skip_style] ? "" : auto_complete_stylesheet) +
  yield +
  content_tag("div", "", :id => "#{object}_#{method}_auto_complete",
              :class => "auto_complete") +
  auto_complete_field(
    "#{object}_#{method}",
    {
      :url => { :action => "auto_complete_for_#{object}_#{method}" } 
    }.update(completion_options))
end

Here I’ve introduced a new utility function called “auto_complete_field_with_style_and_script” that generates the same Javascript and style sheet for the view as before, but instead calls a block to generate the actual text field. Then I changed text_field_with_auto_complete to call this, providing a block to make the call to “text_field” in ActionView::Helpers::FormHelper with the proper names and options.

Now my new form builder class in auto_complete_form_helper.rb contains a version of text_field_with_auto_complete that looks like this:

def text_field_with_auto_complete(object,
                                  method,
                                  tag_options = {},
                                  completion_options = {})
  unique_object_name = "#{object}_#{Object.new.object_id.abs}"
  completion_options_for_original_name =
    {
      :url => { :action => "auto_complete_for_#{object}_#{method}"},
      :param_name => "#{object}[#{method}]"
    }.update(completion_options)
  @template.auto_complete_field_with_style_and_script(
        unique_object_name,
        method,
        tag_options,
        completion_options_for_original_name
  ) do
    text_field(method,
               {
                 :id => "#{unique_object_name}_#{method}"
               }.update(tag_options))
  end
end

Here the call to auto_complete_field_with_style_and_script passes a block that calls the other text_field from ActionView::Helpers::FormBuilder (note the “object” parameter is not present as above).

To allow the text field to be repeated on a complex form, I insure the object’s name is unique by adding a random number to it (“unique_object_name”). This unique name is then passed into both auto_complete_field_with_style_and_script and text_field, insuring that the <input> and related Javascript all work without problems, even if the text field is repeated more than once on the same form.

The last important detail here is that the completion options passed into auto_complete_field_with_style_and_script are generated using the original, unchanged (non-unque) object name, so that the Ajax calls to the server are made using the original name. This means no changes are required on the server side, and the same single line of code in your controller still works as usual:

auto_complete_for :task, :name

Next time I’ll post a sample application that uses this new plugin, and explain what changes you will need to make to your own application for auto_complete in a complex form.

Repeated_auto_complete changes merged into auto_complete

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

To install and use my modified version of auto_complete first remove the standard auto_complete plugin from your app if necessary, and install with:

script/plugin install git://github.com/patshaughnessy/auto_complete.git

<% for person in @group.people %>
  <% auto_complete_fields_for "group[person_attributes][]", person do |form| %>
    Person <%= person_form.label :name %><br />
    <%= form.text_field_with_auto_complete :person, :name, {},
                                           {:method => :get}  %>
  <% end %>
<% end %>

To understand my changes to the plugin, let’s first look at how the original auto_complete works. If you add this line to your view:

<%= text_field_with_auto_complete :project, :name, {}, {:method => :get } %>

…then you get HTML and script that looks like this (style sheet omitted):

<input id="project_name" name="project[name]" size="30" type="text" />
<div class="auto_complete" id="project_name_auto_complete"></div>
<script type="text/javascript">
//<![CDATA[
var project_name_auto_completer = new Ajax.Autocompleter('project_name',
'project_name_auto_complete', '/projects/auto_complete_for_project_name',
{method:'get'})
//]]>
</script>

The original text_field_with_auto_complete method looked like this:

def text_field_with_auto_complete(object, method, tag_options = {},
                                  completion_options = {})
    (completion_options[:skip_style] ? "" : auto_complete_stylesheet) +
    text_field(object, method, tag_options) +
    content_tag("div", "", :id => "#{object}_#{method}_auto_complete",
                :class => "auto_complete") +
    auto_complete_field(
        "#{object}_#{method}",
        { 
          :url => { :action => "auto_complete_for_#{object}_#{method}" }
        }.update(completion_options))
  end

What I wanted to achieve in the modified plugin was to allow the view to contain code like this:

<% auto_complete_fields_for task do |f| %>
  <%= f.label :name, "Task:" %>
  <%= f.text_field_with_auto_complete :task, :name, {}, {:method => :get } %>
<% end %>

To make this work, we need a new version of text_field_with_auto_complete that calls text_field from ActionVi

Who is talking?

Archive

about 5 hours ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 month ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 month ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 month ago | Pat Shaughnessy: Pat Shaughnessy - Home

2 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

3 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

3 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

4 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

5 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

5 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

6 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

6 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

7 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

8 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

8 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

9 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

9 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

10 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

11 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

11 months ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home

about 1 year ago | Pat Shaughnessy: Pat Shaughnessy - Home