Our Blog

Paginating API call with Radian6

By Marina Lohova
August 24, 2012

I wrote about Radian6 in my earlier blog post. Today I will review one more aspect of Radian6 API - call pagination.

Most Radian6 requests return paginated data. This introduces extra complexity of making request several times in the loop in order to get all results. Here is one simple way to retrieve the paginated data from Radian6 using the powerful Ruby blocks.

I will use the following URL to fetch data:

/data/comparisondata/1338958800000/1341550800000/2777/8/9/6/

Let’s decypher this.

• 1338958800000 is start_date, 1341550800000 is end_date for document search. It’s June, 06, 2012 - July, 06, 2012 formatted with date.to_time.to_i * 1000.

• 2777 is topic_id, a Radian6 term, denoting a set of search data for every customer.

• 8 stands for Twitter media type. There are various media types in Radian6. They reflect where the data came from. media_types parameter can include a list of values for different media types separated by commas.

• 9 and 6 are page and page_size respectively.

First comes the method to fetch a single page.

In the Radian6 wrapper class:

def page(index, &block)
  data = block.call(index) 
  articles, count = data['article'], data[ …

Merging Two Google Accounts: My Experience

By Steph Skardal
August 21, 2012

Before I got married, I used a Gmail account associated with my maiden name (let’s call this account A). And after I got married, I switched to a new gmail address (let’s call this account B). This caused daily annoyances as my use of various Google services was split between the two accounts.

Luckily, there are some services in Google that allow you to easily toggle between two accounts, but there is no easy way to define which account to use as the default for which service, so I found myself toggling back and forth frequently. Unfortunately, Google doesn’t provide functionality to merge multiple Google accounts. You would think they might, especially given my particular situation, but I can see how it’s a bit tricky in logically determining how to merge data. So, instead, I set off on migrating all data to account B, described in this email.

Consider Your Google Services

First things first, I took at look at the Google Services I used. Here’s how things broke down for me:

• Gmail: Account A forwards to account B. I always use account B.
• Google+: Use through account A.
• Google Analytics: Various accounts divided between account A and account B. …

Using Different PostgreSQL Versions at The Same Time.

By Szymon Lipiński
August 20, 2012

When I work for multiple clients on multiple different projects, I usually need a bunch of different stuff on my machine. One of the things I need is having multiple PostgreSQL versions installed.

I use Ubuntu 12.04. Installing PostgreSQL there is quite easy. Currently there are available two versions out of the box: 8.4 and 9.1. To install them I used the following command:

~$ sudo apt-get install postgresql-9.1 postgresql-8.4 postgresql-client-common

Now I have the above two versions installed.

Starting the database is also very easy:

~$ sudo service postgresql restart
 * Restarting PostgreSQL 8.4 database server   [ OK ]
 * Restarting PostgreSQL 9.1 database server   [ OK ]

The problem I had for a very long time was using the proper psql version. Both database installed their own programs like pg_dump and psql. Normally you can use pg_dump from the higher version PostgreSQL, however using different psql versions can be dangerous because psql uses a lot of queries which dig deep into the PostgreSQL internal tables for getting information about the database. Those internals sometimes change from one database version to another, so the best solution is to use the psql from the …

Hidden inefficiencies in Interchange searching

By Jeff Boes
August 13, 2012

A very common, somewhat primitive approach to Interchange searching uses an approach like this:

The search profile contains something along the lines of

  mv_search_type=db
  mv_search_file=products
  mv_column_op=rm
  mv_numeric=0
  mv_search_field=category

[search-region]
  [item-list]
    [item-field description]
  [/item-list]
[/search-region]

In other words, we search the products table for rows whose column “category” matches an expression (with a single query), and we list all the matches (description only). However, this can be inefficient depending on your database implementation: the item-field tag issues a query every time it’s encountered, which you can see if you “tail” your database log. If your item-list contains many different columns from the search result, you’ll end up issuing many such queries:

[item-list]
    [item-field description], [item-field weight], [item-field color],
    [item-field size], [item field ...]
  ...

resulting in:

SELECT description FROM products WHERE sku='ABC123'
SELECT weight FROM products WHERE sku='ABC123'
SELECT color FROM products WHERE sku='ABC123'
SELECT size FROM products WHERE sku='ABC123'
... …

Rails 3 ActiveRecord caching bug ahoy!

By Brian Gadoury
August 2, 2012

Sometimes bugs in other people’s code makes me think I might be crazy. I’m not talking Walter Sobchak gun-in-the-air-and-a-Pomeranian-in-a-cat-carrier crazy, but “I must be doing something incredibly wrong here” crazy. I recently ran into a Rails 3 ActiveRecord caching bug that made me feel this kind of crazy. Check out this pretty simple caching setup and the bug I encountered and tell me; Am I wrong?

I have two models with a simple parent/child relationship defined with has_many and belongs_to ActiveRecord associations, respectively. Here are the pertinent bits of each:

class MimeTypeCategory < ActiveRecord::Base
  # parent class
  has_many :mime_types

  def self.all
    Rails.cache.fetch("mime_type_categories") do
    MimeTypeCategory.find(:all, :include => :mime_types)
  end
end

class MimeType < ActiveRecord::Base
  # child class
  belongs_to :mime_type_category
end

Notice how in MimeTypeCategory.all, we are eager loading each MimeTypeCategory’s children MimeTypes because our app tends to use those MimeTypes any time we need a MimeTypeCategory. Then, we cache that entire data structure because it’s a good candidate for caching and we like our app to be fast. …

Postgres Open 2012

By David Christensen
July 31, 2012

I’m excited to be speaking at the 2012 Postgres Open Conference; Chicago, September 17-19. (Conference details at http://www.postgresopen.org/.)

My talk will be “Choosing a Logical Replication System: Slony vs Bucardo”.

I look forward to seeing many of you there!

Ruby on Rails software developer needed

By Jon Jensen
July 31, 2012

This position has been filled. See our active job listings here.

We’re looking for another talented Ruby on Rails developer to consult with our clients and develop their Ruby on Rails web applications. If you like to focus on solving business problems and can take responsibility for getting a job done well without intensive oversight, please read on!

End Point is a 17-year-old web consulting company based in New York city, with 28 full-time employees working mostly remotely from home offices. Our team is made up of strong ecommerce, database, and system administration talent, working together using ssh, Screen and tmux, IRC, Google+ Hangouts, Skype, and good old phones.

We serve over 200 clients ranging from small family businesses to large corporations, using a variety of open source technologies including Ruby, Python, Perl, Git, PostgreSQL, MySQL, RHEL/​CentOS, Debian, and Ubuntu.

What is in it for you?

• Work from your home office
• Flexible full-time work hours
• Annual bonus opportunity
• Health insurance benefit
• 401(k) retirement savings plan
• Ability to move without being tied to your job location

What you will be doing:

• Consult with clients to determine their web application needs
• Build, test, release, and maintain web applications for our …

Are you sure you want a MacBook Pro?

By Brian Buchalter
July 30, 2012

The “UltraBooks” trend has been been made popular by Apple’s efforts at making the computer an appliance. There are clear benefits to this, but for enthusiasts, it means less control to upgrade to hardware of their choosing. Let’s contrast this experience with my own Dell Vostro 3500. It was purchased two years ago and yet through upgrades, it has many features the recently refreshed MacBook Pro with Retina display lacks, while offering the same processing power.

MacBook Pro Advantages

Let’s be clear, the MacBook Pro (MBP) has a number advantages over my Dell. The Retina display of course trounces my 1376×768 resolution on a 15" display. For designers, photographers, and video editors, Retina is a great asset. For someone who spends their day inside Vim and SSH, not so much. Similarly, the MBP’s seven hour battery life is impressive, although I’m sure with my processor intensive work load, it would be lower. But for me, I’m not terribly mobile. I need to be mobile from room to room perhaps, but I don’t find myself without power and needing to complete mission critical work less frequently than my replaceable 90 watt-hour battery can provide over 3 to 4 hours. Similarly, weight …