Our Blog

Debian Postgres readline psql problem and the solutions

By Greg Sabino Mullane
August 2, 2011

There was a bit of a controversy back in February as Debian decided to replace libreadline with libedit, which affected a number of apps, the most important of which for Postgres people is the psql utility. They did this because psql links to both OpenSSL and readline, and although psql is compatible with both, they are not compatible with each other!

By compatible, I mean that the licenses they use (OpenSSL and readline) are not, in one strict interpretation, allowed to be used together. Debian attempts to live by the letter and spirit of the law as close as possible, and thus determined that they could not bundle both together. Interestingly, Red Hat does still ship psql using OpenSSL and readline; apparently their lawyers reached a different conclusion. Or perhaps they, as a business, are being more pragmatic than strictly legal, as it’s very unlikely there would be any consequence for violating the licenses in this way.

While libreadline (the library for GNU readline) is a feature rich, standard, mature, and widely used library, libedit (sadly) is not as developed and has some important bugs and shortcomings (including no home page, apparently, and no Wikipedia page!). This …

Cuenca, Ecuador

By Benjamin Goldstein
July 28, 2011

Given that End Point uses a distributed office structure and that I do almost all of my work online, in theory I should be able to work just about anywhere there’s a good Internet connection. That sounds neat, but is it in fact true? Well, earlier this year I put the theory to a serious test. By nature I’m not all that adventurous a traveller, but my wife is, and she’s fluent in Spanish. Our teenage sons are going to be grown men all too soon, so if we were ever to take the plunge into living abroad as a family, I realized it was now or never.

In looking for a place to go we had some criteria:

• a safe place in Latin America without an excess of walls
• good or at least reasonable Internet connectivity
• soccer training for our 15-year-old
• high school in Spanish for our 16-year-old.

We are very lucky that my friend Tovias suggested his home town (more or less) and volunteered his family to look after us there!

So, in February I picked up and relocated with my family to Cuenca, Ecuador, for just over three months. I worked. My wife, Gina, homeschooled one of our sons, and generally kept us all going in this beautiful and historical city.

Here is a Gigapan I took of Cuenca from Turi …

Company Presentation: jQuery and Rails

By Steph Skardal
July 27, 2011

Yesterday, I gave a company presentation on jQuery and Rails. The talk covered details on how jQuery and Rails work together to build rich web applications, with a considerable amount of focus on AJAX methods. Check out the slides here:

One piece of knowledge I took away from the talk is how different the Rails 3 approach is for unobtrusive AJAX behavior using helpers like link_to_remote and remote_form_for. Mike Farmer made a recommendation to read the rails.js source here to see how onclick behavior is handled in Rails 3.

Rails Optimization: Advanced Techniques with Solr

By Steph Skardal
July 22, 2011

Recently, I’ve been involved in optimization on a Rails 2.3 application. The application had pre-existing fragment caches throughout the views with the use of Rails sweepers. Fragment caches are used throughout the site (rather than action or page caches) because the application has a fairly complex role management system that manages edit access at the instance, class, and site level. In addition to server-side optimization with more fragment caching and query clean-up, I did significant asset-related optimization including extensive use of CSS sprites, combining JavaScript and CSS requests where ever applicable, and optimizing images with tools like pngcrush and jpegtran. Unfortunately, even with the server-side and client-side optimization, my response times were still sluggish, and the server response was the most time consuming part of the request for a certain type of page that’s expected to be hit frequently:

A first stop in optimization was to investigate if memcached would speed up the site, described in this article Unfortunately, that did not improve the speed much.

Next, I re-examined the debug log to see what was taking so much time. The debug log looked like this …

Announcing pg_blockinfo!

By David Christensen
July 14, 2011

I’m pleased to announce the initial release of pg_blockinfo. It is a tool to examine your PostgreSQL heap data files, written in Perl.

Similar in purpose to pg_filedump, it is used to display (and soon validate) buffer-page-level information for PostgreSQL page/heap files.

pg_blockinfo aims to work in a portable and non-destructive way, using read-only “mmap”, sys-level IO functions, and “unpack” in order to minimize any Perl overhead.

What we buy for the compromise of writing this in Perl instead of C is two-fold:

1. portability/low impact — pg_blockinfo has no other dependencies than Perl and several core Perl modules and will work in environments where you can’t or won’t easily install other packages or compile based on specific headers.

2. expressibility — while not currently supported in full, one of pg_blockinfo’s future goals is to allow you to specify criteria for display of both page-level and tuple-level info. It will allow you to define arbitrary Perl expressions to filter the objects you’re looking at (i.e., pages, tuples, etc; think “grep” but on a tuple level). It will support a DSL for querying based off of the named fields as well as allow you to supply arbitrary Perl …

Raw Caching Performance in Ruby/Rails

By Steph Skardal
July 12, 2011

Last week, I set up memcached with a Rails application in hopes of further improving performance after getting a recommendation to pursue it. We’re already using many Rails low-level caches and fragment caches throughout the application because of its complex role management system. Those are stored in NFS on a NetApp filer, and I was hoping switching to memcached would speed things up. Unfortunately, my http request performance tests (using ab) did not back this up: using file caching on NFS with the NetApp was about 20% faster than memcached from my tests.

I brought this up to Jon, who suggested we run performance tests on the caching mechanism only rather than testing caching via full http requests, given how many layers of the stack are involved and influence the overall performance number. From the console, I ran the following:

$ script/console   # This app is on Rails 2.3
> require 'benchmark'
> Rails.cache.delete("test")
> Rails.cache.fetch("test") { [SomeKlass.first, SomeKlass.last] }
> # to emulate what would potentially be stored with low-level cache
> Benchmark.bm(15) { |x| x.report("times:") { 10000.times do; Rails …

Google+

By Steph Skardal
July 11, 2011

Over the weekend, I dug into Google+ a bit. I wanted to share a few notes about the experience with my coworkers and the world.

Speed

Google has done a great job on performance from what I can tell. They’ve followed their own recommendations on optimization by doing things like implementing CSS sprites, caching static assets, and gzipping content. I can’t do performance tests on my authenticated account at WebPageTest.org, but the perceived performance is great.

User Interface

Parts of the user interface look similar to Facebook. But Google deviated from their norm of utilitarian/pragmatic design according to this article, and I appreciate their focus on aesthetics here. This combined with the speed makes for a delightful user experience.

Circles

Google offers limited sharing functionality with what they call “Circles”, similar to Facebook groups. In my case, I have the following circles:

• Friends
• Family
• End Point
• Web People (professional contacts)
• Photography

Circles are integrated into every part of Google+, which makes it easy to limit posts, photos, videos or other content to individuals or circles. In addition to limiting sharing permissions, you can also limit viewing …

Home router problems with .0 IP address

By Jon Jensen
July 1, 2011

In our work the occasional mysterious problem surfaces which makes me appreciate how tractable and sane the majority of the challenges are. Here I’ll tell the story of one of the mysterious problems.

In Internet routing of IPv4 addresses, there’s nothing inherently special about an IP address that ends in .0, .255, or anything else. It all depends on the subnet. In the days before CIDR (Classless Inter-Domain Routing) brought us arbitrary subnet masks, there were classes of routing, most commonly A, B, and C. And the .0 and .255 addresses were special.

That was a long time ago, but it can still cause occasional trouble today. One of our hosting providers assigned us an IP address ending in .0, which we used for hosting a website. It worked fine, and was in service for many months before we heard any reports of trouble.

Then we heard a report from one of our clients that they could not access that website from their home, but they could from their office. We couldn’t ever figure out why.

Next one of our own employees found that he could not access the website from his home, but he could from other locations.

Finally we had enough evidence when a friend from the open source community …