Our Blog

DBD::Pg moves to Git!

By David Christensen
June 14, 2011

Just a note to everyone that development the official DBD::Pg DBI driver for PostgreSQL source code repository has moved from its old home in Subversion to a Git repository. All development has now moved to this repo.

We have imported the SVN revision history, so it’s just a matter of pointing your Git clients to:

$ git clone git://bucardo.org/dbdpg.git

For those who prefer, there is a GitHub mirror:

$ git clone git://github.com/bucardo/dbdpg.git

Git is available via many package managers or by following the download links at https://git-scm.com/download for your platform.

Enjoy!

MongoDB replication from Postgres using Bucardo

By Greg Sabino Mullane
June 12, 2011

One of the features of the upcoming version of Bucardo (a replication system for the PostgreSQL RDBMS) is the ability to replicate data to things other than PostgreSQL databases. One of those new targets is MongoDB, a non-relational ‘document-based’ database. (to be clear, we can only use MongoDB as a target, not as a source)

To see this in action, let’s setup a quick example, modified from the earlier blog post on running Bucardo 5. We will create a Bucardo instance that replicates from two Postgres master databases to a Postgres database target and a MongoDB instance target. We will start by setting up the prerequisites:

sudo aptitude install postgresql-server \
perl-DBIx-Safe \
perl-DBD-Pg \
postgresql-contrib

Getting Postgres up and running is left as an exercise to the reader. If you have problems, the friendly folks at #postgresql on irc.freenode.net will be able to help you out.

Now for the MongoDB parts. First, we need the server itself. Your distro may have it already available, in which case it’s as simple as:

aptitude install mongodb

For more installation information, follow the links from the MongoDB Quickstart page. For my test box, I ended up installing from source …

June 8, 2011: World IPv6 Day

By Josh Williams
June 7, 2011

I’m a little surprised they didn’t do it today. 06-06, what better day for IPv6? Oh well, at least Hurricane Electric was awesome enough to send a Sage certification shirt just in time!

June 8th, 2011 is the day! In just a couple days time World IPv6 Day begins. Several of the largest and most popular sites on the Internet, and many others, turn on IPv6 addresses for a 24-hour interval. Many of them already have it, but you have to seek it out on a separate address. Odds are if you’re seeking that out specifically you’re configured well enough to not have any problems. But with IPv6 configured on the primary addresses of some of the largest Internet sites, people that don’t specifically know they’re testing something become part of the test. That’s important to track down exactly what composes those 1-problem-in-2,000 configurations, and assess if that’s even an accurate number these days.

Not sure about your own connection? https://test-ipv6.com/ is an excellent location to run a number of tests and see how v6-enabled you are. Odds are you’ll end up at one end of the spectrum or the other. But if there’s a configuration glitch that could help you track it down.

At End Point we …

SeniorNet

By Jeff Boes
June 7, 2011

SeniorNet is an organization dedicated to bringing education and technology access to older adults (50+). I began volunteering with them recently, coaching an intermediate Windows class. My role is to shadow half the class, watching for anyone who gets “stuck” or goes astray, as the instructor leads them through basic operations such as formatting text, creating spreadsheets, and so on.

I started this as a way to give back to the community, and to explore things I might want to do later in my career (namely, teach), but I’m finding that this is giving me some unexpected insights in design (visual and functional) for less-experienced computer users. I’m sure there are lots and lots of formal studies on how over-50ers learn to use computers vs. under-20ers, but there’s nothing like seeing it first-hand. I’ve already started mulling over how this new insight might affect the way I structured an on-line application (e.g., an e-commerce checkout) that would cater to an older audience. And while I don’t have any answers there, I feel better (and humble) for having begun the process, even at this late date.*

*I’m among the very oldest, if not the actual oldest of the End Point crew, …

Using Set Operators with Ruby Arrays

By Mike Farmer
June 7, 2011

The Array class in Ruby has many methods that are extremely useful. I frequently find myself going to the RDoc just to review the different methods and keeping myself up-to-speed on what options are available for manipulating my data using the native methods. Often, I find that there is already a method that exists that can simplify a big chunk of code that I wrote that was confusing and complex.

In a recent project, I needed a way to handle a complex user interface problem that was caused by a many-to-many (has-and-belongs-to-many) database model. The solution that I came up with was an amazingly simple implementation for a problem that could have involved writing some very convoluted and complex algorithms that would have muddied my code and required me to write extensive tests. As it turns out, I had just read up on Array set operators (Ruby methods) and the solution became easier and monumentally more eloquent.

Introducing the Union, Difference, and Intersection

Since Arrays essentially act as a set[1], they can be manipulated using the set operations union, difference, and intersection. If you go do the Array rdoc, however, you’ll notice no methods with these names. So here is …

Bucardo multi-master for PostgreSQL

By Greg Sabino Mullane
June 6, 2011

The original Bucardo

The next version of Bucardo, a replication system for Postgres, is almost complete. The scope of the changes required a major version bump, so this Bucardo will start at version 5.0.0. Much of the innards was rewritten, with the following goals:

Multi-master support

Where “multi” means “as many as you want”! There are no more pushdelta (master to slaves) or swap (master to master) syncs: there is simply one sync where you tell it which databases to use, and what role they play. See examples below.

Ease of use

The bucardo program (previously known as ‘bucardo_ctl’) has been greatly improved, making all the administrative tasks such as adding tables, creating syncs, etc. much easier.

Performance

Much of the underlying architecture was improved, and sometimes rewritten, to make things go much faster. Most striking is the difference between the old multi-master “swap syncs” and the new method, which has been described as “orders of magnitudes” faster by early testers. We use async database calls whenever possible, and no longer have the bottleneck of a single large bucardo_delta table.

Improved logging

Not only are more details provided, there is now the ability …

Paperclip in Spree: Extending Product Image Sizes

By Steph Skardal
June 6, 2011

Spree uses the popular gem Paperclip for assigning images as attachments to products. The basic installation requires you to install the gem, create a migration to store the paperclip-specific fields in your model, add the has_attached_file information to the model with the attachment, add the ability to upload the file, and display the file in a view. In Spree, the Image model has an attached file with the following properties:

class Image < Asset
  ...
  has_attached_file :attachment,
                    :styles => { :mini => '48x48>',
                      :small => '100x100>',
                      :product => '240x240>',
                      :large => '600x600>'
                    },
                    :default_style => :product,
                    :url => "/assets/products/:id/:style/:basename.:extension",
                    :path => ":rails_root/public/assets/products/:id/:style/:basename.:extension"
  ...
end

As you can see, when an admin uploads an image, four image sizes are created: large, product, small, and mini.

Four images are created per product image uploaded in Spree …

Saving time with generate_series()

By Josh Tolley
May 31, 2011

I was giving a presentation once on various SQL constructs, and, borrowing an analogy I’d seen elsewhere, described PostgreSQL’s generate_series() function as something you might use in places where, in some other language, you’d use a FOR loop. One attendee asked, “So, why would you ever want a FOR loop in a SQL query?” A fair question, and one that I answered using examples later in the presentation. Another such example showed up recently on a client’s system where the ORM was trying to be helpful, and chose a really bad query to do it.

The application in question was trying to display a list of records, and allow the user to search through them, modify them, filter them, etc. Since the ORM knew users might filter on a date-based field, it wanted to present a list of years containing valid records. So it did this:

SELECT DISTINCT DATE_TRUNC('year', some_date_field) FROM some_table;

In fairness to the ORM, this query wouldn’t be so bad if some_table only had a few hundred or thousand rows. But in our case it has several tens of millions. This query results in a sequential scan of each of those records, in order to build a list of, as it turns out, about fifty total …