Our Blog

Tip: Find all non-UTF-8 files

By David Christensen
April 9, 2010

Here’s an easy way to find all non-UTF-8 files for later perusal:

find . -type f | xargs -I {} bash -c "iconv -f utf-8 -t utf-16 {} &>/dev/null || echo {}" > utf8_fail

I’ve needed this before for converting projects over into UTF-8; obviously certain files are going to be binary and will show up in this list, so manual vetting will need to be done before converting all your images over into UTF-8.

Modifying Models in Rails Migrations

By Sonny Cook
April 8, 2010

One problem that has haunted me in the past was making modifications to the model in migrations. Specifically, stuff like removing or changing associations. At the time the migration is expected to run, the file for the model class will have been updated already, so it is hard use that in the migration itself, even though it would be useful.

In this case I found myself with an even slightly trickier example. I have a model that contains some address info. Part of that is an association to an external table that lists the states. So part of the class definition was like so:

Class Contact {
 belongs_to :state
 ...
}

What I needed to do in the migration was to remove the association and introduce another field called “state” which would just be a varchar field representing the state part of the address. The two problems the migration would encounter are:

1. the state association would not exist at the time it ran
2. and even if it did, there would be a name conflict between it and the new column I wanted

to get around these restrictions I did this in my migration:

Contact.class_eval {
      belongs_to :orig_state,
                 :class_name => "State", …

Git Submodules: What is the Ideal Workflow?

By Steph Skardal
April 5, 2010

Last week, I asked some coworkers at End Point about the normal workflow for using git submodules. Brian responded and the discussion turned into an overview on git submodules. I reorganized the content to be presented in a FAQ format:

How do you get started with git submodules?

You should use git submodule add to add a new submodule. So for example you would issue the commands:

git submodule add git://github.com/stephskardal/extension1.git extension
git submodule init

Then you would git add extension (the path of the submodule installation), git commit.

What does the initial setup of a submodule look like?

The super project repo stores a .gitmodules file. A sample:

[submodule "extension1"]
        path = extension
        url = git://github.com/stephskardal/extension1.git
[submodule "extension2"]
        path = extension_two
        url = git://github.com/stephskardal/extension2.git

When you have submodules in a project, do you have to separately clone them from the master project, or does the initial checkout take care of that recursively for you?

Generally, you will issue the commands below when you clone a super project repository. These commands will …

Spree and Software Development: Git and Ruby techniques

By Steph Skardal
March 31, 2010

Having tackled a few interesting Spree projects lately, I thought I’d share some software evelopment tips I’ve picked up along the way.

Gem or Source?

The first decision you may need to make is whether to run Spree from a gem or source. Directions for both are included at the Spree Quickstart Guide, but the guide doesn’t touch on motivation from running from a gem versus source. The Spree documentation does address the question, but I wanted to comment based on recent experience. I’ve preferred to build an application running from the gem for most client projects. The only times I’ve decided to work against Spree source code was when the Spree edge code had a major change that wasn’t available in a released gem, or if I wanted to troubleshoot the internals of Spree, such as the extension loader or localization functionality.

If you follow good code organization practices and develop modular and abstracted functionality, it should be quite easy to switch back and forth between gem and source. However, switching back and forth between Spree gem and source may not be cleanly managed from a version control perspective.

git rebase

Git rebase is lovely. Ethan describes some examples of …

Xen MAC mismatch VNC mouse escape HOWTO

By Jon Jensen
March 30, 2010

This is a story that probably shouldn’t need to be told if everything is in documentation somewhere. I’m not using any fancy virtualization management tools and didn’t have an easy time piecing everything together, so I thought it’d be worth writing down the steps of the manual approach I took.

Dramatis personæ:

• Server: Red Hat Enterprise Linux 5.4 with Xen kernel
• Guest virtual server: CentOS 5.4 running paravirtualized under Xen
• Workstation: Ubuntu 9.10

The situation: I updated the CentOS 5 Xen virtual guest via yum and rebooted to load the new Linux kernel and other libraries such as glibc. According to Xen as reported by xm list, the guest had started back up fine, but the host wasn’t reachable over the network via ping, http, or ssh, including from the host network.

The guest wasn’t using much CPU (as shown by xm top), so I figured it wasn’t just a slow-running fsck during startup. And I was familiar with the iptables firewall rules on this guest, so I was fairly sure I wasn’t being blocked there. I needed to get to the console to see what was wrong.

The way I’ve done this before is using VNC to access the virtual console remotely. The Xen host was configured to accept VNC …

PostgreSQL Conference East 2010 review

By Greg Sabino Mullane
March 29, 2010

I just returned from the PostgreSQL Conference East 2010. This is one of the US “regional” Postgres conferences, which usually occur once a year on both the East and West coast. This is the second year the East conference has taken place in my home town of Philadelphia.

Overall, it was a great conference. In addition to the talks, of course, there are many other important benefits to such a conference, such as the “hallway tracks”, seeing old friends and clients, meeting new ones, and getting to argue about default postgresql.conf settings over lunch. I gave a 90 minute talk on “Postgres for non-Postgres people” and a lightning talk on the indispensable tail_n_mail.pl program.

This year saw the conference take place at a hotel for the first time, and this was a big improvement over the previous school campus-based conferences. Everything was in one building, there was plenty of space to hang out and chat between the talks, and everything just felt a little bit easier. The one drawback was that the rooms were not really designed to lecture to large numbers of people (e.g. no stadium seating), but this was not too much of an issue for most of the talks.

A few of the talks I attended …

LibrePlanet 2010: Eben Moglen and the future of Oracle in free software

By Selena Deckelmann
March 23, 2010

I just got back from Libre Planet 2010, a conference for free software activists put on by the Free Software Foundation. I imagine most readers of this blog are familiar with the language debate over free software vs. open source. Much of the business and software community has settled into using open source as the term of choice, but Libre Planet is certainly a place where saying “free software” is the norm.

I presented two talks—​one on how to give good talks by connecting with your audience, and a second about non-coding roles in free software communities. The first talk is built on my work with user groups and giving presentations at primarily free software conferences over the last five years. The second was built off of the great work of Josh Berkus, for a talk that he first gave at a mini-conference I arranged the day before OSCON 2007 for Postgres.

One talk I attended surprised me with an important discussion of the future of the open source database market.

Eben Moglen spoke about the future of the Free Software Foundation and the new challenges that software freedom faces in a world increasingly dominated by network services—​social networking, collaboration tools and …

Using psql \o to append to a file

By Josh Tolley
March 22, 2010

I had a slow query I was working on recently, and wanted to capture the output of EXPLAIN ANALYZE to a file. This is easy, with psql’s \o command:

5432 josh@josh# \o explain-results

Once EXPLAIN ANALYZE had finished running, I wanted the psql output back in my psql console window. This, too, is easy, using the \o command without a filename:

5432 josh@josh# \o

But later, after adding an index or two and changing some settings, I wanted to run a new EXPLAIN ANALYZE, and I wanted its output appended to the explain-analyze file I built earlier. At least on my system, \o will normally overwrite the target file, which would mean I’d lose my original results. I realize it’s simple to, say, pipe output to a new file (“explain-analyze-2”), but I wasn’t interested. Instead, because \o can also accept a pipe character and a shell command to pipe its output to, I did this:

5432 josh@josh# \o | cat - >> explain-results

Life is good.

Update: A helpful commenter pointed out I hadn’t actually used the same files in the original post. Oops. Fixed.