How to Log Which Line Initiated a Query in Rails

Create a file (something like) config/initializers/active_record_log_subscriber.rb

…and paste this into its contents:

module LogQuerySource
  def debug(*args, &block)
    return unless super

    backtrace = Rails.backtrace_cleaner.clean caller

    relevant_caller_line = backtrace.detect do |caller_line|
      !caller_line.include?('/initializers/')
    end

    if relevant_caller_line
      logger.debug("  ↳ #{ relevant_caller_line.sub("#{ Rails.root }/", '') }")
    end
  end
end
ActiveRecord::LogSubscriber.send :prepend, LogQuerySource

Now, whenever a line of code initiates a SQL query the line will be logged.  This greatly improves the ease of tracking down inefficiencies such as like N + 1 queries.

If you would like to learn more about exactly how this mix-in works and how the original author went about figuring out how to add it, you can read more about it here.

 

Commenting and Documentation in Ruby and Rails

It’s always been frustrating to me that Ruby and Rails don’t really have strong templated standards on how to best document your code. There are several options available and it can all be very confusing and unclear.

There is one template I’ve found works best and its super simple:

# Brief explanation of what the code does.
#
# * *Args* :
# - ++ ->
# * *Returns* :
# -
# * *Raises* :
# - ++ ->
def my_method

end

While there are many other perfectly good documentation libraries this has the benefits of being simple to remember, very easy to read in code, and compiles out super nicely using

rake doc:app

Example (strongly based off this post):

# This is an example method commented the way I 
# like. It sums the three arguments and returns 
# that value.
#
# Lorem ipsum dolor sit amet, consectetur adipising,
# do eiusmod tempor incididunt ut labore et dolore. 
# Ut enim ad minim veniam, quis nostrud exercit
# laboris nisi ut aliquip ex ea commodo consequat.
#
# * *Args* :
#   - +apples+ -> the number of apples
#   - +oranges+ -> the number of oranges
#   - +pears+ -> the number of pears
# * *Returns* :
#   - the total number of fruit as an integer
# * *Raises* :
#   - +ArgumentError+ -> if any value is nil or negative
def sum_fruit(apples, oranges, pears)
  ...
end

Will render into something like this:


Other than this, I tend to try and follow the general conventions as outlined by the Rails contribution guidelines on commenting.

NotImplementedError Must Have Battered Wife Syndrome

I’ve been doing something very wrong my whole life.  I think it’s time to confess and seek help.

You see, like so many developers I see the NotImplementedError class in various languages and think, to my self “I haven’t had a chance to fully implement this class, so in my methods that my colleagues call that I didn’t finish yet, I will raise that exception to let them know.”

I am ashamed of myself.  And it’s time to get help.

You see the actual purpose of the NotImplementedError (as defined by the Ruby docs) is:

Raised when a feature is not implemented on the current platform. For example, methods depending on the fsync or fork system calls may raise this exception if the underlying operating system or Ruby runtime does not support them.

ERMERGERD!  I am a terrible person. Abusing this error class just because  my sub classes cannot meet the contractual obligation of the super class.

And I know I am not alone.

So whats the best practice, then?

Well, in Ruby’s case it turns out the answer is super simple; don’t raise an error at all! Instead, document the expectation and if a subclass fails to meet it a NameError, or often its subclass NoMethodError, will be raised automatically.

 

How to manually mark ActiveRecord attribute as dirty

Recently I had quite a conundrum. I was working on a project’s model that was using the ActiveRecord save method update_columns to avoid executing the callbacks on the model, however had the unexpected side-effect further down my stack when a method in the chain checked if a particular attribute changed, wasn’t working because as well as skipping validations and callbacks; it also bypasses the ActiveModel::Dirty logic!

This caused me quite a problem, as I feared that I would be forced to use a method of saving the record that triggered my callbacks (very undesirable).

Fortunately, there is a solution.  Because the ActiveModel::Dirty class is still loaded, you can manually mark with attributes are dirty by simply calling [attr_name]_will_change! before the change to track the attribute.

attribute_name_will_change!

Normally this is all done for you and now later on in the stack when you do something like

if attribute_name_changed?
  was = attribute_name_was
end

It will work, despite using a method to save that doesn’t automatically track the dirty attributes for you.

Actually the ActiveModel::Dirty module has some pretty cool stuff in it. Much of it never gets properly explored in the average Rails project, but it provides some hidden magic, like being able to fully revert an object and undo a save, providing a kind of diff of changes and some other really neat stuff.

A better way to benchmark

Today I found some magic.

include ActiveSupport::Benchmarkable

An ActiveSupport module that provides an AMAZINGLY elegant alternative to the classic Ruby Benchmark class; which I had always considered to be a bit ‘clunky’.

ActiveSupport::Benchmarkable provides you a simple method ‘benchmark’ which you can pass a few options and block through and output the benchmark result to the log. Wrap this block around expensive operations or possible bottlenecks to get a time reading for the operation.

For example, let’s say you thought your data transformattion method was taking too long; you could wrap it in a benchmark block.

<% benchmark 'v' do %>
  <%= expensive_record_transforming %>
<% end %>

That would write something like “Processing some data (143.6ms)” to the log, which you can now use to monitor or compare speed over time (make sure its not getting worse as data size increases) or compared to alternate methods for optimisation.

The ActiveSupport::Benchmarkable docs also indicate that, you may give an optional logger level (:debug, :info, :warn, :error) as the :level option. The default logger level value is :info.

<% benchmark 'Low-level files', level: :debug do %>
  <%= lowlevel_files_operation %>
<% end %>

Finally, you can pass true as the third argument to silence all log activity (other than the timing information) from inside the block. This is great for boiling down a noisy block to just a single statement that produces one log line:

<% benchmark 'Process data files', level: :info, silence: true do %>
  <%= expensive_and_chatty_files_operation %>
<% end %>

while these samples are indicative of what they would look like in a .html.erb file (view) – it also works in any other place you use the mix in.

I have personally used these to great success in controllers and models to help diagnose specific controller/model operations suspected to be slow.

This ActiveSupport module is an immensely powerful tool to have in a developers arsenal. I highly recommend it’s use.

How to Install ‘therubyracer’ or ‘libv8’ gem(s) on OSX

Recently, I need to move some Rails projects I was working on to new computer and this needs me to install all the dependencies for these projects.  While using bundler to install the gems; I encountered the following error:

extconf failed, exit code 1
Gem files will remain installed in /Users/ash/.rvm/gems/ruby-2.2.1/gems/libv8-3.16.14.3 for inspection.
Results logged to /Users/ash/.rvm/gems/ruby-2.2.1/extensions/x86_64-darwin-14/2.2.0-static/libv8-3.16.14.3/gem_make.out

An error occurred while installing libv8 (3.16.14.3), and Bundler cannot continue.
Make sure that `gem install libv8 -v '3.16.14.3'` succeeds before bundling.

Fortunately, with homebrew fixing this (on OSX 10.11, El Capitan at least) worked perfectly. Simply execute these commands:

brew install v8
gem install therubyracer
gem install libv8 -v '3.16.14.3' -- --with-system-v8

Estabilishing ActiveRecord Database Connections in Ruby (But Without Rails)

Anyone who has even the smallest amount of experience developing with Ruby on Rails knows that Rails has some pretty sweet configuration conventions which make switching between environments very easy. Switching from development to production to testing is as easy as changing the RAILS_ENV variable. No doubt Rails does some dark magic behind the scenes to trivialise this. But what if you’re writing an app in Ruby, without the Rails to guide you?

ActiveRecord 101

Establishing a database connection in ActiveRecord without Rails is pretty basic:

require 'active_record'

ActiveRecord::Base.establish_connection({
  adapter:  'sqlite3',
  database: 'db/test.sqlite3'
})

It establishes a connection to the specified database with the specified connection  configuration. If you were using MySQL or PostgreSQL you would provide the relevant configuration such as username, password and host. With the connection established, we can now start consuming the connection however we like. For example, we may want to use funky Rails’ style models (so we can easily create and populate tables and reference them):

ActiveRecord::Schema.define do
  create_table :things do |t|
    t.integer :id, :null => false
    t.string  :name
  end
end

class Thing < ActiveRecord::Base
end

Thing.create({
  id:   0,
  name: 'Broomstick'
})

Care about the Environment

The other magic in Rails is the environment selection. The main benefit with this is that we can have isolated connection configurations and have our Ruby app automatically select the correct settings for us (using our environment variable ENV).

require 'active_record'

conf = case ENV['DB']
when 'conf1'
{
  adapter: 'sqlite3',
  database: 'db/mydb1.sqlite3'
}
when 'conf2'
{
  adapter: 'sqlite3',
  database: 'db/mydb2.sqlite3'
}
else
  raise 'export DB=conf[n]'
end

ActiveRecord::Base.establish_connection conf

Mind you, before this will work you first need to set the environment variable:

export DB=conf1 # or conf2

Now, depending on ENV[‘DB’], the code will open a connection to the corresponding database.

Best Practises

It’s never a good idea to store sensitive information (like database connection information and usernames and password of any kind) in your code base. Rails uses an external database configuration file called database.yml to solve this problem, so lets implement one ourselves.

database.yml:

conf1:
  adapter: sqlite3
  database: db/mydb1.sqlite3

conf2:
  adapter: sqlite3
  database: db/mydb2.sqlite3

Loading this YAML file is very simple:

require 'active_record'
conf = YAML.load_file('database.yml')
ActiveRecord::Base.establish_connection conf[ENV['DB']]

Now we can set our environment variable same as before, but our code will look in a YAML file for the proper database configuration automatically, without having to store database connection information directly inside our code.

Association Cardinality in Rails

From time to time I’ve noticed people who struggle with cardinality and associations in Ruby on Rails. So, I thought I would attempt to create a cheat sheet here to help developers understand relationship cardinality and how it maps to associations.

ActiveRecord can be used to describe relations with one-to-one, one-to-many and many-to-many cardinality; where each model defines its relation to another. Let’s cover each of the three types of associations.

One-to-one

Use `has_one` in the base and `belongs_to` in the association:

class Family < ActiveRecord::Base
  has_one :home
end
class Home < ActiveRecord::Base
  belongs_to :family
end

A common question about a one-to-one association is ‘how to know which direction the has_one and belongs_to go?’  The correct way to know, is that whichever model has the foreign key, gets the `belongs_to`.  In this case, Home has the foreign key `family_id`.

One-to-one relationships are a bit odd, and as a general rule, if you find yourself using a lot of them, there is probably a better solution.

One-to-many

Use `has_many` in the base and `belongs_to` in the association:

class Family < ActiveRecord::Base
  has_many :parents
end
class Parent < ActiveRecord::Base
  belongs_to :family
end

This will be your most common relationship. As with one-to-one’s, the table with the foreign key gets the `belongs_to` (although this is a lot more obvious with a one-to-many). In this case the foreign key is `family_id`.

Many-to-many

These can be a lot more complicated and there is actually a couple of different ways to do it.

The first way involves a specific joining model. This results in 2 stages of has_many associations. It is referred to as `has_many :through` and is primarily used if you need to fully control the joining model/table:

class Family < ActiveRecord::Base
  belongs_to :parent
  belongs_to :kid
end
class Parent < ActiveRecord::Base
  has_many :kids, through: :families
end
class Kid < ActiveRecord::Base
  has_many :parents, through: :families
end

The second (and my preferred way) is to use the `has_and_belongs_to_many` method:

class Parent < ActiveRecord::Base
  has_and_belongs_to_many :kids 
end
class Kid < ActiveRecord::Base
  has_and_belongs_to_many :parents
end

The main difference (or disadvantage) with the `has_and_belongs_to_many`method is that the intermediary joining table and foreign keys need to be exactly named to match what Rails expects. Which many-to-many method you use ultimately depends on whether you need to work with the relationship model as its own entity directly.

MySQL utf8mb4 Encoding Breaks ActiveRecord’s Schema Setup

I recently wrote about the virtues of true UTF8 (utf8mb4) character sets in MySQL and how to change your database to use it. Today we will discuss a possible problem you may encounter when you do when programming on Ruby on Rails. The error looks something like this:

$ rake db:setup

Mysql::Error: Specified key was too long; max key length is 767 bytes:
CREATE UNIQUE INDEX unique_schema_migrations ON schema_migrations (version)

The problem exists because the utf8mb4 character set uses the full 4 bytes per character rather than the 1-3 of UTF8 (the character set most people mistakenly use thinking they’ll have full Unicode compliance). Because of this extra size, the schema_migration may no longer fit.

This small patch will set default mysql string column length to 191 instead of 255 which is the new index limit on utf8mb4 (aka real utf8).

# config/initializers/mysqlpls.rb
require 'active_record/connection_adapters/abstract_mysql_adapter'

module ActiveRecord
  module ConnectionAdapters
    class AbstractMysqlAdapter
      NATIVE_DATABASE_TYPES[:string] = { :name => "varchar", :limit => 191 }
    end
  end
end

 

‘belongs_to’ and ‘has_one’ Differentiated

One of the more common confusions with ActiveRecord associations is the difference between the `has_one` and `belongs_to` relationships.

However, they are not interchangeable, and there are some serious differences. A `belongs_to` can only go in the class that holds the foreign key whereas `has_one` means that there is a foreign key in another table that references this class. So `has_one` can only go in a class that is referenced by a column in another table.

So this is wrong:

class Transaction < ActiveRecord::Base
  # The transactions table has a order_id  
  has_one :order                
end

class Order < ActiveRecord::Base
  # The orders table has a transaction_id
  has_one :transaction          
end

So is this:

class Transaction < ActiveRecord::Base
  # The transactions table has a order_id
  belongs_to :order             
end

class Order < ActiveRecord::Base
  # The orders table has a transaction_id
  belongs_to :transaction     
end

For a two-way association, you need one of each, and they have to go in the right class. Even for a one-way association, it matters which one you use, and which direction you use it:

class Transaction < ActiveRecord::Base
  # This table has no foreign keys
  has_one :order             
end

class Order < ActiveRecord::Base
  # The orders table has a transaction_id
  belongs_to :transaction     
end