Our Blog

On End Point’s Development Environment

By Steph Skardal
March 11, 2015

A few recent conversations have sparked my interest in writing up a blog post that summarizes the familiar elements of our development environments. The majority of End Pointers work remotely, but many of the tools listed below are common to many developers.

• ssh/sftp: We do primarily remote development. Many of us are familiar with local development, but remote development with camps (see next point) is typically the most efficient arrangement in working with multiple development instances that are accessible to clients for testing and staging.
• camps: DevCamps are a tool specific to and created by End Point, which are development instances with an entire webserver, database, and app server stack, similar to containers like Docker. Check out the DevCamps website for more information.
• vim/emacs/nano: While most of our employees use vim or emacs for command-line editors, nano is an inefficient but easy to use editor that we can suggest to new developers. Not many of us use IDEs, if at all.
• screen/tmux: screen and tmux are our preferred terminal multitasking and sharing.
• command-line database interaction (specifically psql and mysql ad-hoc querying): Working with an SQL database …

Postgres searchable release notes—​one page with all versions

By Greg Sabino Mullane
March 11, 2015

The inability to easily search the Postgres release notes has been a long-standing annoyance of mine, and a recent thread on the pgsql-general mailing list showed that others share the same frustration. One common example when a new client comes to End Point with a mysterious Postgres problem. Since it is rare that a client is running the latest Postgres revision (sad but true), the first order of business is to walk through all the revisions to see if a simple Postgres update will cure the problem. Currently, the release notes are arranged on the postgresql.org web site as a series of individual HTML pages, one per version. Reading through them can be very painful—​especially if you are trying to search for a specific item. I whipped up a Perl script to gather all of the information, reformat it, clean it up, and summarize everything on one giant HTML page. This is the result: https://bucardo.org/postgres_all_versions.html

Please feel free to use this page however you like. It will be updated as new versions are released. You may notice there are some differences from the original separate pages:

• All 270 versions are now on a single page. Create a local greppable version with: …

SCaLE 13x

By Jacob Minshall
March 4, 2015

I recently went to the Southern California Linux Expo (SCaLE). It takes place in Los Angeles at the Hilton, and is four days of talks, classes, and more, all focusing around Linux. SCaLE is the largest volunteer run open source conference. The volunteers put a lot of work into the conference, from the nearly flawless wireless network to the AV team making it as easy as plugging in a computer to start a presentation.

One large focus of the conference was the growing DevOps community in the Linux world. The more DevOps related talks drew the biggest crowds, and there was even a DevOps focused room on Friday. There are a wide range of DevOps related topics but the two that seemed to draw the largest crowds were configuration management and containerization. I decided to attend a full day talk on Chef (a configuration management solution) and Docker (the new rage in containerization).

The Thursday Chef talk was so full that they decided to do an extra session on Sunday. The talk was more of an interactive tutorial than a lecture, so everyone was provided with an AWS instance to use as their Chef playground. The talk started with the basics of creating a file, installing a package, and …

Working with Annotator: Part 2

By Steph Skardal
March 4, 2015

A while back, I wrote about my work on Ruby on Rails based H2O with Annotator, an open source JavaScript library that provides annotation functionality. In that article, I discussed the history of my work with annotations, specifically touching on several iterations with native JavaScript functionality developed over several years to handle colored highlight overlapping on selected text.

Finally, last July we completed the transition to Annotator from custom JavaScript development, with the caveat that the application had quite a bit of customization hooked into Annotator’s easily extensible library. But, just a few months after that, I revisited the customization, described in this post.

Separation of UI Concerns

In our initial work with Annotator, we had extended it to offer a few additional features:

• Add multiple tags with colored highlights, where content can be tagged on the fly with a color chosen from a set of predefined colors assigned to the tag. Text would then be highlighted with opacity, and colors combined (using xColor) on overlapping highlights.
• Interactive functionality to hide and show un-annotated text, as well as hide and show annotated text with specific tags. …

Temporary tables in SQL query optimization

By Jeff Boes
February 25, 2015

SQL queries can get complex in a big hurry. If you are querying multiple tables, and in particular if your query involves operations like UNION and INTERSECT, then you can find yourself in a big, messy pile of SQL. Worse, that big pile will often run slowly, sometimes to the point where a web application times out!

I won’t inflict a real-world example on you; that would be cruel. So let’s look at a “toy” problem, keeping in mind that this won’t illustrate any time-savings, just the technique involved.

Here’s the original SQL:

SELECT p.* FROM products p
JOIN (SELECT * FROM inventory WHERE /* complex clause here */) i USING (sku)
UNION ALL
SELECT p.* FROM clearance_products p
JOIN (SELECT * FROM inventory WHERE /* complex clause here */) i USING (sku)

Bonus hint: using “UNION ALL“ instead of just “UNION” will allow the query processor to skip an unnecessary step here. “UNION ALL” says you know the rows on either side of the clause are unique. “UNION” means the results will be post-processed to remove duplicates. This might save you more than a smidgen of time, depending on how large the two sub-queries get.

Now, many times the query optimizer will just do the right thing here. But …

Postgres ON_ERROR_ROLLBACK explained

By Greg Sabino Mullane
February 24, 2015

(picture by Steve Wilson)

Way back in 2005 I added the ON_ERROR_ROLLBACK feature to psql, the Postgres command line client. When enabled, any errors cause an immediate rollback to just before the previous command. What this means is that you can stay inside your transaction, even if you make a typo (the main error-causing problem and the reason I wrote it!). Since I sometimes see people wanting to emulate this feature in their application or driver, I thought I would explain exactly how it works in psql.

First, it must be understood that this is not a Postgres feature, and there is no way you can instruct Postgres itself to ignore errors inside of a transaction. The work must be done by a client (such as psql) that can do some voodoo behind the scenes. The ON_ERROR_ROLLBACK feature is available since psql version 8.1.

Normally, any error you make will throw an exception and cause your current transaction to be marked as aborted. This is sane and expected behavior, but it can be very, very annoying if it happens when you are in the middle of a large transaction and mistype something! At that point, the only thing you can do is rollback the transaction and lose all of your work. For …

Testing your chef repo pull requests with chef-zero, Vagrant and Jenkins

By Wojtek Ziniewicz
February 18, 2015

All Liquid Galaxy setups deployed by End Point are managed by Chef. Typical deployment consists of approx 3 to 9 Linux boxes from which only 1 is managed and the rest is an ISO booted from this machine via network with copy-on-write root filesystem. Because of this, typical deployment involves more steps than just updating your code and restarting application. Deployment + rollback may be even 10 times longer compared with typical web application. Due to this fact, we need to test our infrastructure extensively.

What are we to do in order to make sure that our infrastructure is tested well before it hits production?

Scary? It’s not.

Workflow broken down by pieces

• lg_chef.git repo—​where we keep cookbooks, environments and node definitions
• GitHub pull request—​artifact of infrastructure source code tested by Jenkins
• Vagrant—​virtual environment in which chef is run in order to test the artifact. There’s always 1 master node and few Vagrant boxes that boot an ISO from master via tftp protocol
• chef-zero—​Chef flavor used to converge and test the infrastructure on the basis of GitHub pull request
• chef-server/chef-client—​Chef flavor used to converge and test production and …

Postgres pg_dump implicit cast problem patched!

By Greg Sabino Mullane
February 16, 2015

(image of brain coral by Lara Danielle)

One of the many reasons I love Postgres is the responsiveness of the developers. Last week I posted an article about the dangers of reinstating some implicit data type casts. Foremost among the dangers was the fact that pg_dump will not dump user-created casts in the pg_catalog schema. Tom Lane (eximious Postgres hacker) read this and fixed it up—​the very same day! So in git head (which will become Postgres version 9.5 someday) we no longer need to worry about custom casts disappearing with a pg_dump and reload. These same-day fixes are not an unusual thing for the Postgres project.

For due diligence, let’s make sure that the casts now survive a pg_dump and reload into a new database via psql:

psql -qc 'drop database if exists casting_test'
psql -qc 'create database casting_test'
psql casting_test -xtc 'select 123::text = 123::int'
ERROR:  operator does not exist: text = integer
LINE 1: select 123::text = 123::int
                                 ^
HINT:  No operator matches the given name and argument type(s). You might need to add explicit type casts.

psql casting_test -c 'create function pg_catalog.text(int) …