Test::Database Postgres support
By Greg Sabino Mullane
February 8, 2009
At our recent company meeting, we organized a ‘hackathon’ at which the company was split into small groups to work on specific projects. My group was Postgres-focused and we chose to add Postgres support to the new Perl module Test::Database.
This turned out to be a decent sized task for the few hours we had to accomplish it. The team consisted of myself (Greg Sabino Mullane), Mark Johnson, Selena Deckelmann, and Josh Tolley. While I undertook the task of downloading the latest version and putting it into a local git repository, others were assigned to get an overview of how it worked, examine the API, and start writing some unit tests.
In a nutshell, the Test::Database module allows an easy interface to creating and destroying test databases. This can be a non-trivial task on some systems, so putting it all into a module make sense (as well as the benefits of preventing everyone from reinventing this particular wheel). Once we had a basic understanding of how it worked, we were off.
While all of our tasks overlapped to some degree, we managed to get the job done without too much trouble, and in a fairly efficient manner. We made a new file for Postgres, added in all the required …
perl postgres git
David Mamet on software development
By Jon Jensen
February 7, 2009
I recently read the book True and False: Heresy and Common Sense for the Actor (1997) by David Mamet. (A gift from Uncle Steve. Thanks!) Mamet’s film The Spanish Prisoner is one of my favorites, and I liked Oleanna, Things Change, and Glengarry Glen Ross a lot too. Wait, looking over his writing credits at IMDb, I see I’d forgotten The Winslow Boy, State and Main, and Wag the Dog, which were good too.
Enough of the fanboyism. This little book of his is packed with what sounds like good advice for actors. Not having been an actor in anything more than a school play or two, I can only imagine, but I was surprised to find myself nodding in agreement during many passages because they applied so well to my field, software development.
I’ve selected a few quotations that seemed especially apt:
I was once at a marriage ceremony where the parties swore “to try to be faithful, to try to be considerate …” That marriage was, of course, doomed. Any worthwhile goal is difficult to accomplish. To say of it “I’ll try” is to excuse oneself in advance. Those who respond to our requests with “I’ll try” intend to deny us, and call on us to join in the hypocrisy—as if there were some merit in …
books development
NY Puppet Users Group meeting
By Steven Jenkins
February 4, 2009
Several of us from End Point attended the first NY Puppet Users Group meeting last night, and we had a great time. Brian Gupta and Larry Ludwig did an excellent time coordinating, even though the weather and the venue didn’t cooperate. After a little shuffling, we found ourselves at The Perfect Pint, where we had a large table in a fairly quiet room where we could meet each other and talk about Puppet.
The group was split fairly evenly between those who have used Puppet and those who are considering it: everyone there understood the need for solid configuration management (CM), so much of the discussions (at least at my end of the table) centered around
-
comparisons of different CM systems
-
challenges in getting Puppet on legacy systems (e.g., that may not even have Ruby)
-
features people would like to see in Puppet
-
how to make the business case for Puppet
All in all it was a great time, and I really appreciated Brian and Larry’s leadership, as well as their flexibility. Not only did they move the venue at the last minute to better accommodate the group, but they were super hosts in coordinating the date and time: several of us were only in NY for a short time, but they …
automation conference
Slony1-2.0.0 + PostgreSQL 8.4devel
By Josh Tolley
February 2, 2009
Many people use Slony to replicate PostgreSQL databases in various interesting ways. Slony is a bit tough to get used to, but works very well, and can be found in important places at a number of high-load, high-profile sites. A few weeks back I set up Slony1-2.0.0 (the latest release) with a development version of PostgreSQL 8.4, and kept track of the play-by-play, as follows:
Starting Environment
On this machine, PostgreSQL is installed from the CVS tree. I updated the tree and reinstalled just to have a well-known starting platform (output of each command has been removed for brevity).
jtolley@uber:~/devel/pgsql$ make distclean
jtolley@uber:~/devel/pgsql$ cvs update -Ad
jtolley@uber:~/devel/pgsql$ ./configure --prefix=/home/jtolley/devel/pgdb
jtolley@uber:~/devel/pgsql$ make
jtolley@uber:~/devel/pgsql$ make install
jtolley@uber:~/devel/pgsql$ cd ../pgdb
jtolley@uber:~/devel/pgdb$ initdb data
jtolley@uber:~/devel/pgdb$ pg_ctl -l ~/logfile -D data start
The –prefix option in ./configure tells PostgreSQL where to install itself.
Slony uses a daemon called slon to do its work, and slon connects to a database over TCP, so I needed to configure PostgreSQL to allow TCP connections …
postgres
Git it in your head
By Ethan Rowe
February 1, 2009
Git is an interesting piece of software. For some, it comes pretty naturally. For others, it’s not so straightforward.
Comprehension and appreciation of Git are not functions of intellectual capacity. However, the lack of comprehension/appreciation may well indicate one of the following:
-
Mistakenly assuming that concepts/procedures from other VCSes (particularly non-distributed “traditional” ones like CVS or Subversion) are actually relevant when using Git
-
Not adequately appreciating the degree to which Git’s conception of content and history represent a logical layer, as opposed to implementation details
CVS and Subversion both invite the casual user to basically equate the version control repository and all operations around it to the file system itself. They ask you to understand how files and directories are treated and tracked within their respective models, but that model is basically oriented around files and directories, period. Yes, there are branches and tags. Branches in particular are entirely inadequate in both systems. They don’t really account for branching as a core possibility that should be structured into the logical model itself; consequently, both systems …
git
Using cron and psql to transfer data across databases
By Greg Sabino Mullane
February 1, 2009
I recently had to move information from one database to another in an automatic function. I centralized some auditing information such that specific information about each database in the cluster could be stored in a single table, inside a single database. While I still needed to copy the associated functions and views to each database, I was able to make use of the new “COPY TO query”feature to do it all on one step via cron.
At the top of the cron script, I added two lines defining the database I was pulling the information from (“alpha”), and the database I was sending the information to (“postgres”):
PSQL_ALPHA='/usr/bin/psql -X -q -t -d alpha'
PSQL_POSTGRES='/usr/bin/psql -X -q -t -d postgres'
From left to right, the options tell psql to not use any psqlrc file found (-X), to be quiet in the output (-q), to print tuples only and no header/footer information (-t), and the name of the database to connect to (-d).
The cron entry that did the work looked like this:
*/5 * * * * (echo "COPY audit_mydb_stats FROM STDIN;" && $PSQL_ALPHA -c "COPY (SELECT *, current_database(), now(), round(date_part('epoch'::text, now())) FROM …postgres
Take pleasure in small things
By Ethan Rowe
January 29, 2009
We’re in the midst of our 2009 company meeting, and are having our first-ever “hackathon”. The engineering team is divided into several working groups focusing on a variety of free software projects.
As a distributed organization, we don’t always get a lot of opportunity to write code and do our “real work” side-by-side like this. And it’s a pleasure to witness and to participate.
Just thought I’d share.
conference
Why not OpenAFS?
By Steven Jenkins
January 28, 2009
OpenAFS is not always the right answer for a filesystem. While it is a good network filesystem, there are usage patterns that don’t fit well with OpenAFS, and there are some issues with OpenAFS that should be considered before adopting or using it.
First, if you don’t really need a network filesystem, the overhead of OpenAFS may not be worthwhile. If you mostly write data, but seldom read it across a network, then the cache of OpenAFS may hinder performance rather than help. OpenAFS might not a good place to put web server logs, for example, that get written to very frequently, but seldom read.
OpenAFS is neither a parallel filesystem nor a high-performance filesystem. In high-performance computing (HPC) situations, a single system (or small set of systems) may write a large amount of data, and then a large number of systems may read from that. In general, OpenAFS does not scale well for multiple parallel reads of read-write data, but it scales very well for parallel reads of replicated read-only data. Because read-only replication is not instantaneous, depending on the latencies that can be tolerated, OpenAFS may or may not be a good choice. If you need to write and immediately …
openafs