Our Blog

Debugging obscure Postgres problems with strace

By Greg Sabino Mullane
June 20, 2013

Image by Flickr user Alexandre Duret-Lutz

One of the nice things about being a Postgres consultant is the sheer variety of interesting problems you get to solve. Here’s one that recently popped up, and a walkthrough of how I solved it. One of our clients had this strange error pop up when they were trying to start Postgres:

FATAL:  too many private dirs demanded

This is a very rare and esoteric error. A peek at the source code showed that this error only appears in src/backend/storage/file/fd.c, like so:

DIR *
AllocateDir(const char *dirname)
{
    DIR        *dir;

    DO_DB(elog(LOG, "AllocateDir: Allocated %d (%s)",
               numAllocatedDescs, dirname));

    /*
     * The test against MAX_ALLOCATED_DESCS prevents us from overflowing
     * allocatedDescs[]; the test against max_safe_fds prevents AllocateDir
     * from hogging every one of the available FDs, which’d lead to infinite
     * looping.
     */
    if (numAllocatedDescs >= MAX_ALLOCATED_DESCS ||
        numAllocatedDescs >= max_safe_fds - 1)
        elog(ERROR, "too many private dirs demanded");

So it appeared as if we ran into some sort of safety valve that was meant to bail out before …

Spree’s New Release Policy

By Brian Buchalter
June 17, 2013

Spree has recently updated its documentation regarding contributions and has included (maybe for the first time?) an official Release Policy. This is an important step forward for the Spree community so that developers can communicate to clients the potential costs and benefits when upgrading Spree, understand how well Spree supports older releases, and gauge the overall speed of the “upgrade treadmill”.

Deprecation Warnings

Deprecation warnings are to be added in patch releases (i.e. 2.0.2) and the code being deprecated will only be removed in minor versions. For example, if a deprecation warning is added in 2.0.2, 2.0.3 will still contain the same deprecation warning but in 2.1.0 the deprecation warning and the code will be gone.

Deprecation warnings are very helpful for developers, but without a robust test suite exercising your application, it’s easy for deprecation warnings to go unnoticed. A strong test suite coupled with deprecation warnings helps you manage your client’s expectations about how upgrades can affect your Spree customizations and extensions.

Master Branch Receives All Patches

Master branch receives all patches, including new features and breaking API changes …

Making use of a Unix Pipe

By Mike Farmer
June 13, 2013

Developing in a Unix-based environment has many wonderful advantages. Thanks to Gary Bernhardt of DestroyAllSoftware Screencasts, I’ve recently discovered a new use for the Unix pipe. A pipe in Unix does exactly what you might think it would do by its name. Send something in one side and watch it come out the other. If you’ve done much in the shell, you’ve probably used pipes before where you’ve probably piped some output from one command to another. Here’s an example:

$ cat foo.txt | grep bar

This command simply says take the output of cat and sends it to the input of grep. Pipes used in this way can yield very powerful commands in the shell.

There is another pipe in Unix and this is a named pipe. A named pipe, or a FIFO (First In, First Out), works similarly to command line pipe. You put stuff in one end and it comes out the other. To create a named pipe, you use the mkfifo command.

$ mkfifo my_fifo
$ ls -l
...
prw-rw-r--  1 mikefarmer mikefarmer      0 Jun  5 21:22 my_fifo

Notice the “p” at the beginning of the file list. The “p” designates this file as a named pipe to the system. To try out our pipe, we will, in one terminal, listen for anything coming out of the pipe. Then in …

Railsbridge NYC

By Bianca Rodrigues
June 13, 2013

Last week I attended a Ruby on Rails workshop hosted by Railsbridge NYC. The organization promotes diversity in tech by introducing web development concepts to a community of technology professionals and enthusiasts. It’s geared towards women, but open to all.

Installfest

As the website describes, the workshop consists of an Installfest (about a 2 hour event), followed by a full day of learning the following day. The Installfest is a series of step-by-step instructions to install Ruby, Rails and other tools you would need for your particular OS. The detailed instructions helped make the entire process effortless, and most students were able to install all the tools without major issues.

An Introduction to Ruby and Rails

The next day was the actual workshop, beginning promptly at 9am. The skill level of the women in attendance varied; some were programmers proficient in other languages, while others, like myself, were new to programming. After a short welcome presentation, the organizers divided the group into two, with the “more advanced” students heading off to their own room for a slightly more quick-paced session.

I remained in the beginner group, which was significantly larger. …

Converting root filesystem from ext3 to ext4 on CentOS and RHEL 5.9

By Jon Jensen
June 12, 2013

Here’s a quick explanation of the procedure to convert the root / filesystem on RHEL and CentOS 5.9 from ext3 to ext4, because ext3 wasn’t available during install time.

Note that this is not a configuration Red Hat supports, but it works fine. (I believe you cannot convert the /boot filesystem to ext4 on standard RHEL/CentOS 5 because its GRUB can’t handle it, but you can convert all the other filesystems.)

Ideally do this only on a fairly freshly-installed system you don’t mind losing. Back everything up first unless this is a system you don’t mind destroying! This is a risky operation and (ahem) things can go wrong.

You’ll need direct console or KVM access to the server. You can do without that if you can remount -o ro / but that usually won’t work with sshd or other daemons that keep files open on the / filesystem.

You will of course need to adapt the current kernel version and root filesystem block device path in the examples below.

Now, to live dangerously:

• yum -y install e4fsprogs
• Edit /etc/fstab so that the / filesystem is mounted as ext4 (which works with the existing ext3 filesystem as well). If you’re using a battery-backed RAID controller you may want to add the …

Installing PostgreSQL without Root

By Szymon Lipiński
June 12, 2013

PostgreSQL can be installed using installers prepared for your operation system. However this way you just depend on the installation settings chosen by the packages mainainers. Installation requires root privileges, on some machines programmers are not allowed to do that. What’s more, this way you rather will not install the PostgreSQL beta version.

The only way to install Postgres without root privileges, in home directory, is to compile it from sources. That’s not very difficult.

Download Sources

First of all you need to download sources. I use GitHub for getting the latest sources. There is a Postgres GitHub mirror. I clone that, but you could just download zip file.

Unpack it somewhere, and you have the Postgres sources you need.

Install Needed Software

For compiling Postgres you will need some libraries and programs. The complete list can be found in Postgres documentation.

I’m using Ubuntu, the packages I use for compiling Postgres are:

• gcc — C compiler
• libreadline6, libreadline6-dev — readline support
• zlib1g, zlib1g-dev — compression library used internally by Postgres
• libpython2.7, libpython2.7-dev — for compiling with PL/Python support

If you are using different system, …

PostgreSQL Functional Indexes

By Szymon Lipiński
June 10, 2013

PostgreSQL has got the great feature named “functional indexes”. A normal index just stores sorted values of some field. It is great for searching, as the values are already sorted.

You can create an index with a simple query like:

CREATE INDEX i_test ON test (i);

It will store all values of column i from table test. This index can be used with a query like:

SELECT * FROM test WHERE i < 100 ORDER BY i;

Functional Indexes

There is also something I like most. Index can store all values you want, they don’t need to be values from the table. You can use values calculated from the table columns. They will be sorted, so searching with those indexes will be pretty fast.

Creating such index is simple:

CREATE INDEX i_test_lower_i ON test (lower(i));

The main rule is: this index can be used if you have the same function call in your query, something like:

SELECT * FROM test WHERE lower(i) = 'aaa';

Example

Let’s check something more complicated. My test table looks like:

CREATE TABLE test(t timestamp);

I filled this table with sample data. We need some bigger number of rows:

INSERT INTO test(t) SELECT generate_series(now() - '1 year'::interval, now(), '1 minute'); …

Creating custom button graphics in Android

By Zed Jensen
June 7, 2013

In the Android timesheet app I’m working on, I have a scrollable layout of RadioButtons for the user to pick how much time they’ve spent on a project (see my earlier blog post about it), and for that I use custom button graphics to make it look nice. So, I’m going to show you how to do that with 9-patch PNGs and selector XML.

First, what’s a 9-patch PNG? A 9-patch is a special PNG image where you specify regions that can be stretched to make room for text. Android will automatically resize a 9-patch to best fit whatever contents you give it. The tool you need to create a 9-patch image is included in the Android SDK Tools, so download that if you haven’t already.

More information about 9-patch images can be found here.

Okay! I’ve got custom button graphics (72x72 for HDPI screens), drawn in the Gimp and saved in my project’s res/drawable-hdpi/ folder as button_selected.png and button_unselected.png:

To convert it to a 9-patch, browse to the tools/ directory of the Android SDK and run draw9patch. This will open a window with a graphical editor on the left, and a button preview on the right. The editor window is for specifying which parts of the image will be stretched. The …