Our Blog

Respecting API Call Limit with Radian6

By Marina Lohova
June 29, 2012

A common practice in the online API world is to enforce the call limit. Twitter allows 150 API calls per hour. Shopify has 500 API calls per 5 minutes limit. You may learn how to work with Shopify call limit from this great article.

One of our projects was built around interaction with Radian6 platform. Radian6 is a new and growing data mining service with the default limit of 100 calls per hour. I will describe our take on the call limit implementation.

Introducing the call counter

First, we need to know how many calls have been executed in the current hour. Every external call increments the counter field on a special model until the counter reaches the limit. The counter is reset back to zero at the beginning of every hour.

script/rails g model RadianCallCount

In the migration:

class CreateRadianCallCounts < ActiveRecord::Migration
  def change
    create_table :radian_call_counts do |t|
      t.integer :count
      t.timestamps
    end
  end
end

In db/seeds.rb file

puts "Initializing Radian6 counter"
RadianCallCount.delete_all
RadianCallCount.create(:count => 0)

Let’s roll the counter!

rake db:migrate
rake db:seed

Scheduling the counter reset

It is necessary …

Three Things: ImageMagick, RequestBin, Responsivness

By Steph Skardal
June 28, 2012

It’s been a while since I wrote a Three Things blog article, where I share tidbits that I’ve learned that don’t quite make it into a full blog post, but are still worth sharing!

Image Stacking via Command Line

I recently needed to stack several images to CSS spritify them. I did some quick searching and found the following ImageMagick command line functionality:

convert add.png add_ro.png -append test.png
convert add.png add_ro.png +append test.png

The -append argument will stack the images vertically, and +append will stack the images horizontally.

RequestBin

Recently, I needed to quickly examine the contents sent during a payment transaction. Richard pointed me to RequestBin a neat and quick tool for examining the contents of an external request. I was testing the Elavon payment gateway integrated in an application using Piggybak. In my code, I made the following change:

class ElavonGateway < ViaklixGateway
  self.test_url = 'https://demo.myvirtualmerchant.com/VirtualMerchantDemo/process.do'
  self.live_url = 'https://www.myvirtualmerchant.com/VirtualMerchant/process.do'
+ self.test_url = 'http://requestb.in/abc1def2'

I ran through a single checkout, …

Google I/O 2012, Day 1

By Zed Jensen
June 27, 2012

I’m here in San Francisco at Google I/O 2012 with coworkers Kiel, Matt, and Ben.

Today began with Sergey Brin giving a really great keynote speech introducing a ton of new Google products and features, including Android 4.1 Jelly Bean, Google’s new Nexus 7 tablet, the Nexus Q, and the addition of magazines and TV to the Google Play Store. Not to mention Google Glass, still under heavy development.

Back in the “sandbox” demo area, we’ve been showing attendees this cool prototype cockpit Liquid Galaxy. A cool new feature introduced with this Google Earth build is helicopter imagery, which is a 3D building layer created by taking photos of the landscape from different angles. I have some pictures for you.

Google I/O attendees looking at the Galaxy:

Andrew Leahy explaining an aspect of the Liquid Galaxy to an interested attendee:

Setting up beforehand:

As it turned out, this isn’t the Liquid Galaxy that was intended for Google I/O. The newest cockpit frame was supposed to arrive some days ago, and was only found this morning. Hopefully we’ll be able to set it up tonight to have a better Liquid Galaxy experience for the next few days!

Speeding Up Integration Tests with PostgreSQL - Follow Up

By Szymon Lipiński
June 27, 2012

Last week I wrote a blog article about speeding up integration tests using PostgreSQL. I proposed there changing a couple of PostgreSQL cluster settings. The main drawback of this method is that those settings need to be changed for the whole cluster. When you have some important data in other databases, you can have a trouble.

In one of the comments Greg proposed using the unlogged table. This feature appeared in PostgreSQL 9.1. The whole difference is that you should use CREATE UNLOGGED TABLE instead of CREATE TABLE for creating all your tables.

For the unlogged table, the data is not written to the write-ahead log. All inserts, updates and deletes are much faster, however the table will be truncated at the server crash or unclean shutdown. Such table is not replicated to standby servers, which is obvious as there are replicated write-ahead logs. What is more important, the indexes created on unlogged tables are unlogged as well.

All the things I describe here are for integrations tests. When database crashes, then all the tests should be restarted and should prepare the database before running, so I really don’t care what happens with the data when something crashes.

The bad …

KISS: Slurping up File Attachments

By Steph Skardal
June 26, 2012

I’ve been heavily involved in an ecommerce project running on Rails 3, using Piggybak, RailsAdmin, Paperclip for file attachment management, nginx and unicorn. One thing that we’ve struggled with is handling large file uploads both in the RailsAdmin import process as well as from the standard RailsAdmin edit page. Nginx is configured to limit request size and duration, which is a problem for some of the large files that are uploaded, which are large purchasable, downloadable files.

To allow these uploads, I brainstormed how to decouple the file upload from the import and update process. Phunk recently worked on integration of Resque, a popular Rails queueing tool which worked nicely. However, I ultimately decided that I wanted to go down a simpler route. The implementation is described below.

Upload Status

First, I created an UploadStatus model, to track the status of any file uploads. With RailsAdmin, there’s an automagic CRUD interface connected to this model. Here’s what the migration looked like:

class CreateUploadStatuses < ActiveRecord::Migration
  def change
    create_table :upload_statuses do |t|
      t.string :filename, :nil => false
      t.boolean :success, :nil …

Simple Example of Dependency Injection with Rails

By Brian Buchalter
June 25, 2012

Today I came across a great opportunity to illustrate dependency injection in a simple context. I had a Rails partial that was duplicated across two subclasses. The partial was responsible for displaying options to create a new record from the data of the current record. It also offered two types of copy, shallow and deep. The shallow copy used a button to POST data, while the deep copy offered a form with some additional options. The only difference between the partials was the path to post data to. Let’s see this in code.

#app/views/fun_event/_copy_options.html.erb
button_to(t("create_and_edit_shallow_copy"), fun_event_path(:from_event => @event.id, :return => true), :    id => "shallow_copy_btn")

form_tag(fun_event_path(:return => true)) do
  #form code
end

#app/views/boring_event/_copy_options.html.erb
button_to(t("create_and_edit_shallow_copy"), boring_event_path(:from_event => @event.id, :return => true), :    id => "shallow_copy_btn")

form_tag(boring_event_path(:return => true)) do
  #form code
end

The first, failed iteration

To remove the duplication, I passed in a path option into the partial, replacing specific …

.rbenv and Passenger: Working through an Upgrade

By Steph Skardal
June 22, 2012

Yesterday, I worked on upgrading the Piggybak demo application, which runs on Piggybak, an open source Ruby on Rails ecommerce plugin developed and maintained by End Point. The demo was running on Ruby 1.8.7 and Rails 3.1.3, but I wanted to update it to Ruby 1.9.* and Rails 3.2.6 to take advantage of improved performance in Ruby and the recent Rails security updates. I also wanted to update the Piggybak version, since there have been several recent bug fixes and commits.

One of the constraints with the upgrade was that I wanted to upgrade via .rbenv, because End Point has been happily using .rbenv recently. Below are the steps Richard and I went through for the upgrade, as well as a minor Passenger issue.

Step 1: .rbenv Installation

First, I followed the instructions here to install rbenv and Ruby 1.9.3 locally under the user that Piggybak runs under (let’s call it the steph user). I set the local Ruby version to my local install. I also installed bundler using the local Ruby version.

Step 2: bundle update

Next, I blew away the existing bundle config for my application, as well as the installed bundler gem files for the application. I followed the standard steps to install and …

Speeding Up Integration Tests with PostgreSQL

By Szymon Lipiński
June 22, 2012

Many people tend to say they don’t want to write tests. One of the reasons is usually that the tests are too slow. The tests can be slow because they are written in a bad way. They can also be slow because of slow components. One such component is usually a database.

The great thing about PostgreSQL is that all types of queries are transactional. It simply means that you can start a transaction, then run the test, which can add, delete and update all the data and database structure it wants. At the end of the integration test, there should be called rollback which just reverts all the changes. It means the next test will always have the same database structure and data before running, and you don’t need to manually clear anything.

For running the integration tests we need a test database. One of the most important things when running test is speed. Tests should be fast, programmers don’t like to wait ages just to know that there is something wrong.

We can also have a read only databases for the tests. Then you don’t need to worry about the transactions, however you always need to ensure the tests won’t change anything. Even if you assume your tests won’t make any changes, it is …