Our Blog

Raising open file descriptor limits for Dovecot and nginx

By Jon Jensen
May 23, 2011

Recently we’ve needed to increase some limits in two excellent open-source servers: Dovecot, for IMAP and POP email service, and nginx, for HTTP/HTTPS web service. These are running on different servers, both using Red Hat Enterprise Linux 5.

First, let’s look at Dovecot. We have a somewhat busy mail server and as it grew busier, it occasionally hit connection limits when the server itself still has plenty of available capacity.

Raising the number of processes in Dovecot is easy. Edit /etc/dovecot.conf and change from the prior (now commented-out) limits to the new limits:

#login_max_processes_count = 128
login_max_processes_count = 512

and later in the file:

#max_mail_processes = 512
max_mail_processes = 2048

However, then Dovecot won’t start at all due to a shortage of available file descriptors. There are various ways to change that, including munging the init scripts, changing the system defaults, etc. The most standard and non-interventive way to do so with this RHEL 5 Dovecot RPM package is to edit /etc/sysconfig/dovecot and add:

ulimit -n 131072

That sets the shell’s maximum number of open file descriptors allowed in the init script /etc/init.d/dovecot before the Dovecot …

Coding Tips from RailsConf 2011

By Steph Skardal
May 20, 2011

A couple of the sessions I attended on Day 1 of RailsConf 2011 were along the lines of how to write good Rails code: Keeping Rails on the Tracks by Mikel Lindsaar and Confident Code by Avdi Grimm. Here’s a mishmash summary from these talks. Although the talks didn’t focus on Ruby and Rails techniques, both talks had plenty of examples and tips for writing maintainable code that apply to my world of consulting and development.

In the first talk, Mikel talked about what he meant by keeping Rails on the Tracks and the balance that we might often experience between just working code and sexy code. Maintainable code lands in the middle there somewhere. And he briefly touched on the fact that budget matters in writing maintainable code, trade-offs are part of life, and that you are selling a solution and not code, which means that clients don’t really care about the technology as long as it’s maintainable.

Mikel’s first pro tip for building a maintainable Rails app is to treat deployment as a first class citizen by having trivial deployment that takes advantage of existing tools like Chef and Puppet. Using Bundler will help you avoid additional pain, but be careful of avoiding locking …

JavaScript and APIs at RailsConf 2011

By Steph Skardal
May 19, 2011

A big trend of RailsConf 2011 has been the presence of JavaScript, whether it be talk of CoffeeScript, node.js, Sproutcore, backbone.js, etc. I attended Yehuda Katz’s talk on Building Rails Apps for the Rich Client on Day 4 of the conference.

Part 1: Rails versus Sinatra Smackdown

Yehuda gave a two part talk about developing API-heavy applications. The first bit addressed why to develop in Rails rather than Sinatra if your application is API-heavy and doesn’t appear to be utilizing valuable parts of Rails? This is fairly applicable to a couple of Sinatra projects that I’ve done at End Point — I like Sinatra a lot, but at some point you begin to replicate parts of Rails. Yehuda explained that because it’s easy to develop web applications efficiently using Rails conventions, developers can become forgetful/ignorant of the underlying functionality of Rails that doesn’t ship with something like Sinatra, much like how working with an ORM can breed developers who aren’t familiar with the underlying database interactions. The checklist for things that Rails manages we might forget about includes:

• ActionDispatch internals
• Session deserialization
• Browser standards mode
• Cookie abstraction …

Only Try This At Home

By Josh Williams
May 19, 2011

Taken by Josh 6 years to the day before the release of 9.1 beta 1

For the record, 9.1 is gearing up to be an awesome release. I was tinkering and testing PostgreSQL 9.1 Beta 1 (… You are beta testing, too, right?) … and some of the new PL/Python features caught my eye. These are minor among all the really cool high profile features, to be sure. But it made me think back to a little bit of experimental code written some time ago, and how these couple language additions could make a big difference.

For one reason or another I’d just hit the top level postgresql.org website, and suddenly realized just how many Postgres databases it took to put together what I was seeing on the screen. Not only does it power the content database that generated the page, of course, but even the lookup of the .org went through Afilias and their Postgres-backed domain service. It’s a pity the DBMS couldn’t act as the middle layer between those.

Or could it?

That’s a shortened form of it just for demonstration purposes (the original one had things like a table browser) … but it works. For example, on this test 9.1 install, hit http://localhost:8000/public/webtest and the following …

Sass at RailsConf 2011

By Steph Skardal
May 18, 2011

With the inclusion of the Scss gem in Rails 3.1, RailsConf is a nice time to get a refresher on Sass/Scss functionality. Sass defines itself as syntactically awesome stylesheets, or a CSS meta language built to provide more powerful functionality to manipulate website appearances with efficiency and elegance. Note that Sass has two syntaxes and the examples presented in this article use the newer Scss syntax. Around the time of RailsConf two years ago, Sass was a included in Spree, an open-source Ruby on Rails ecommerce framework that End Point supports. At the time, I was skeptical about Sass inclusion in Spree because it wasn’t being leveraged to it’s full potential and had hopes of taking advantage of Sass, but a few months later it was removed from the core. Since then, I haven’t worked with Sass on other projects but hope to do so moving forward after being reminded of it’s features and of the fact that it will be included in Rails 3.1 as a default. I attended Chris Eppstein’s talk on Sass and explain a few features related to real-life use cases of CSS manipulation.

Variables

While working on a new feature, your client says, “I want this to be the same red that we use all …

Rails 3 at RailsConf 2011

By Steph Skardal
May 17, 2011

A keynote by DHH kicked off Day 2 of RailsConf, where much of his talk was spent discussing new asset behavior in Rails 3.1. Here’s a run down of a few topics I found worth noting:

Asset Pipeline

DHH started by explaining how although a Rails application has lovely organization in terms of Ruby files, the assets (stylesheets, images, and JavaScripts) have become a junk drawer where junk continues to pile in. Once there’s more than a handful of files, the broken window theory applies and no one tries to maintain organization of those assets. This gets nasty, like a honey badger.

With Rails 3.1, the asset pipeline addresses the junk drawer. Assets directories (images, stylesheets, and JavaScripts) are now created in the app, lib, and vendor assets directories as they pertain to the main app, plugin dependencies, or introduce new library dependencies like a jquery calendar date select plugin. There’s also the introducton of appending to config.assets.paths, which allows you to add new directories that store these assets in arbitrary directories. The new asset pipeline allows you to store these assets in different organization, which encourages JavaScript files to be stored based on …

RailsConf 2011 — Day One

By Jason Dixon
May 16, 2011

Today was the first official day of RailsConf 2011. As with most technical conferences, this one spent the first day with tutorials and workshops. For those of us without paid access to the tutorial sessions, the BohConf was a nice way to spend our first day of the four-day event.

BohConf is described as the “unconference” of RailsConf. It’s a loosely organized collection of presentations, mini-hackathons and barcamp-style meetings. I spent the first half of Monday at the BohConf. Of particular interest to me was Chris Eppstein and Scott Davis’ introduction to Sass and Compass. I’ve dabbled with Sass in the past but only recently learned of Compass.

Sass is a great way to construct your CSS without the tedious duplication that’s typical of most modern spreadsheets. Introducing programming features like variables, inheritance and nested blocks, Sass makes it easy to keep your source material concise and logical. Once your source declarations are ready, compile your production spreadsheets with Sass or Compass.

Compass is effectively a framework for easy construction and deployment of spreadsheets using Sass. To hear Scott describe it, “Compass is to Sass as Rails is to Ruby”. …

DBD::Pg and the libpq COPY bug

By Greg Sabino Mullane
May 13, 2011

(image by kvanhorn)

Version 2.18.1 of DBD::Pg, the Perl driver for Postgres, was just released. This was to fix a serious bug in which we were not properly clearing things out after performing a COPY. The only time the bug manifested, however, is if an asynchronous query was done immediately after a COPY finished. I discovered this while working on the new version of Bucardo. The failing code section was this (simplified):

## Prepare the source
my $srccmd = "COPY (SELECT * FROM $S.$T WHERE $pkcols IN ($pkvals)) TO STDOUT";
$fromdbh->do($srccmd);

## Prepare each target
for my $t (@$todb) {
    my $tgtcmd = "COPY $S.$T FROM STDIN";
    $t->{dbh}->do($tgtcmd);
}

## Pull a row from the source, and push it to each target
while ($fromdbh->pg_getcopydata($buffer) >= 0) {
    for my $t (@$todb) {
        $t->{dbh}->pg_putcopydata($buffer);
    }
}

## Tell each target we are done with COPYing
for my $t (@$todb) {
    $t->{dbh}->pg_putcopyend();
}

## Later on, run an asynchronous command on the source database
$sth{track}{$dbname}{$g} = $fromdbh->prepare($SQL, {pg_async => PG_ASYNC});
$sth{track}{$dbname}{$g}->execute();

This gave the …