Our Blog

Red Hat acquires Qumranet

By Jon Jensen
September 17, 2008

I missed the news a week and a half ago that Red Hat has acquired Qumranet, makers of the Linux KVM virtualization software. They say they’ll be focusing on KVM for their virtualization offerings in future versions of Red Hat Enterprise Linux, though still supporting Xen for the lifespan of RHEL 5 at least. (KVM is already in Fedora.)

Given that Ubuntu also chose KVM as their primary virtualization technology a while back, this should mean even easier use of KVM all around, perhaps making it the default choice on Linux. (Ubuntu supports other virtualization as well.)

Also, something helpful to note for RHEL virtualization users: Red Hat Network entitlements for up to 4 Xen guests carry no extra charge if entitled the right way.

In even older Red Hat news, Dag Wieers wrote about Red Hat lengthening its support lifespan for RHEL by one year for RHEL 4 and 5.

That means RHEL 5 (and thus also CentOS 5) will have full support until March 2011, new media releases until March 2012, and security updates until March 2014. And RHEL 4, despite its aging software stack, will receive security updates until February 2012!

That’s very helpful in making it easier to choose the time of migration …

UTOSC 2008 wrap-up

By Jon Jensen
September 15, 2008

Using Vyatta to Replace Cisco Gear

At the 2008 Utah Open Source Conference I attended an interesting presentation by Tristan Rhodes about the Vyatta open source networking software. Vyatta’s software is designed to replace Cisco appliances of many sorts: WAN routers, firewalls, IDSes, VPNs, and load balancers. It runs on Debian GNU/Linux, on commodity hardware or virtualized.

A key selling point is the price/performance benefit vs. Cisco (prominently noted in Vyatta’s marketing materials), and the IOS-style command-line management interface for experienced Cisco network administrators. Regular Linux interfaces are available too, though Tristan wasn’t positive that writes would stick in all cases, as he’s mostly used the native Linux tools for monitoring and reading, not writing.

Pretty cool stuff, and Vyatta sells pre-built appliances and support too. The Vyatta reps were handing out live CDs, but I haven’t had a chance to try it out yet. Presentation details are here.

Google App Engine 101

Jonathan Ellis did a presentation and then hands-on workshop on Google App Engine, which I found especially useful because he’s a longtime Python and Postgres user. His talk on SQLAlchemy last …

Machine virtualization on the Linux desktop

By Jon Jensen
September 12, 2008

In the past I’ve used virtualization mostly in server environments: Xen as a sysadmin, and VMware and Virtuozzo as a user. They have worked well enough. When there’ve been problems they’ve mostly been traceable to network configuration trouble.

Lately I’ve been playing with virtualization on the desktop, specifically on Ubuntu desktops, using Xen, kvm, and VirtualBox. Here are a few notes.

Xen: Requires hardware virtualization support for full virtualization, and paravirtualization is of course only for certain types of guests. It feels a little heavier on resource usage, but I haven’t tried to move beyond lame anecdote to confirm that.

kvm: Rumored to have been not ready for prime time, but when used from libvirt with virt-manager, has been very nice for me. It requires hardware virtualization support. One major problem in kvm on Ubuntu 8.04 is with the CD/DVD driver when using RHEL/CentOS guests. To work around that, I used the net install and it worked fine.

VirtualBox: This was for me the simplest of all for desktop stuff. I’ve used both the OSE (Open Source Edition) in Ubuntu and Sun’s cost-free but proprietary package on Windows Vista. The current release of VirtualBox only …

Know your tools under the hood

By David Christensen
September 11, 2008

Git supports many workflows; one common model that we use here at End Point is having a shared central bare repository that all developers clone from. When changes are made, the developer pushes the commit to the central repository, and other developers see the relevant changes on subsequent pulls.

We ran into an issue today where after a commit/push cycle, suddenly pulls from the shared repository were broken for downstream developers. It turns out that one of the commits had been created by root and pushed to the shared repository. This worked fine to push, as root had read-write privileges to the filesystem, however it meant that the loose objects which the commit created were in turn owned by root as well; fs permissions on the loose objects and the updated refs/heads/branch prevented the read of the appropriate files, and hence broke the pull behavior downstream.

Trying to debug this purely on the reported messages from the tool itself would have resulted in more downtime at a critical time in the client’s release cycle.

There are a couple of morals here:

• Don’t do anything as root that doesn’t need root privileges. :-)
• Understanding how git works at a low level enabled a …

Fun with 72GB disks: Filesystem performance testing

By Selena Deckelmann
September 9, 2008

If you haven’t heard, the Linux Plumbers Conference is happening September 17-19, 2008 in Portland, OR. It’s a gathering designed to attract Linux developers—​kernel hackers, tool developers and problem solvers.

I knew a couple people from the Portland PostgreSQL User Group (PDXPUG) interested in pitching an idea for a talk on filesystem performance. We wanted to examine performance conventional wisdom and put it to the test on some sweet new hardware, recently donated for performance testing Postgres.

Our talk was accepted, so the three of us have been furiously gathering data, and drawing interesting conclusions, ever since. We’ll be sharing 6 different assumptions about filesystem performance, tested on five different filesystems, under five types of loads generated by fio, a benchmarking tool designed by kernel hacker Jens Axboe to test I/O.

Look forward to seeing you there!

Small changes can lead to significant improvements

By Steve McIntosh
September 5, 2008

Case in point: We’ve been investigating various system management tools for both internal use and possibly for some of our clients. One of these, Puppet from Reductive Labs has a lot of features that I like a lot and good references (Google uses it to maintain hundreds of Mac OS X laptop workstations).

I was asked to see if I could identify any performance bottlenecks and perhaps fix them. With the aid of dtrace (on my own Mac OS X workstation) and the Ruby dtrace library it was easy to spot that a lot of time was being eaten up in the “checksumming” routines.

As with all system management tools, security is really important and part of that security is making sure the files you are looking at and using are exactly the files you think they are. Thus as part of surveying a system for modified files, they are each checksummed using an MD5 hash.

To speed things up, at a small reduction in security, the Puppet checksumming routines have a “lite” option which only feeds the first 512 bytes of a file into the MD5 algorithm instead of the entire file, which can be quite large.

As with most security packages these days, the way you implement an MD5 hash is to get a “digest” object, …

Stepping into version control

By David Christensen
September 5, 2008

It’s no secret that we here at End Point love and encourage the use of version control systems to generally make life easier both on ourselves and our clients. While a full-fledged development environment is ideal for maintaining/​developing new client code, not everyone has the time to be able to implement these.

A situation we’ve sometimes found is clients editing/​updating production data directly. This can be through a variety of means: direct server access, scp/​sftp, or web-based editing tools which save directly to the file system.

I recently implemented a script for a client who uses a web-based tool for managing their content in order to provide transparent version control. While they are still making changes to their site directly, we now have the ability to roll back any changes on a file-by-file basis as they are created, modified, or deleted.

I wanted something that was: (1) fast, (2) useful, and (3) stayed out of the user’s way. I turned naturally to Git.

In the user’s account, I executed git init to create a new Git repository in their home directory. I then git added the relevant parts that we definitely wanted under version control. This included all of the …

Standardized image locations for external linkage

By Jeff Boes
September 3, 2008

Here’s an interesting thought: https://boingboing.net/2008/09/01/publishers-should-al.html

Nutshell summary: publishers should put cover images of books into a standard, predictable location (like http://www.acmebooks.com/covers/{ISBN}.jpg).

This could be extended for almost any e-commerce site where the product image might be useful for reviews, links, etc.

At very least, with Interchange action maps, a site could capture external references to such image requests for further study. (E.g., internally you might reference a product image as [image src=“images/products/current{SKU}”], but externally as “/products/{SKU}.jpg”; the actionmap wouldn’t be used for the site, but only for other sites linking to your images.)