Our Blog

My Favorite Git Commands

By Steph Skardal
September 16, 2013

Git is a tool that all of us End Pointers use frequently. I was recently reviewing history on a server that I work on frequently, and I took note of the various git commands I use. I put together a list of the top git commands (and/or techniques) that I use with a brief explanation.

git commit -m “****”

This is a no-brainer as it commits a set of changes to the repository. I always use the -m to set the git commit message instead of using an editor to do so. Edit: Jon recommends that new users not use -m, and that more advanced users use this sparingly, for good reasons described in the comments!

git checkout -b branchname

This is the first step to setting up a local branch. I use this one often as I set up local branches to separate changes for the various tasks I work on. This command creates and moves you to the new branch. Of course, if your branch already exists, git checkout branchname will check out the changes for that local branch that already exists.

git push origin branchname

After I’ve done a bit of work on my branch, I push it to the origin to a) back it up in another location (if applicable) and b) provide the ability for others to reference the branch. …

PostgreSQL 9.3 Released

By Szymon Lipiński
September 10, 2013

Yesterday PostgreSQL 9.3 was released. It contains many great new features, below is a simple description of those I think are most important. There are many more than the short list, all of them can be find at PostgreSQL 9.3 Release Notes website.

One of the most important features of the new release is the long list of bug fixes and improvements making the 9.3 version faster. I think it is the main reason for upgrading. There are also many new features which your current application will not possibly use, but faster database is always better.

The new mechanism of background workers gives us quite new possibilities to run a custom process in the background. I’ve got a couple of ideas for implementing such background tasks like a custom message queue, or postgres log analyzer, or a tool for accessing PostgreSQL using HTTP (and JSON—​just to have API like the NoSQL databases have).

Another nice feature, which I haven’t checked yet, is data checksums. Something really useful for checking data consistency at data files level. It should make all the data updates slower, but I haven’t checked how much slower, there will be another blog post about that.

There is also parallel pg_dump …

Fixed Navigation Bar: HTML, CSS, and JavaScript Breakdown

By Steph Skardal
September 2, 2013

Something that I’ve seen frequently these days on content rich sites are fixed navigation bars, or small abbreviated header in the form of a horizontal bar at the top of the screen that shows after a user has scrolled below the header. Here’s an example:

A live example of an abbreviated fixed navigation bar at the top of the articles at ABCNews.com. The background has a grey opaque layer for demonstration purposes only.

I recently implemented this functionality for H2O, and I’ll go through the tools needed to do this.

HTML Markup

First thing’s first, you need the HTML markup for this. The only tricky thing here is that the horizontal bar must be outside of any wrapping dividers on the page that are confining the content to a set width. For example, if your content is limited to 900 pixels in width, the horizontal bar markup must be outside that constraint. Here’s what the HTML might look like:

<div id="fixed_bar">
  <div class="wrapper">
    Links & content here.
  </div>
</div>

Note that in the above HTML, the “wrapper” div may be constraining the content width to match the remaining content, such as in the example above. This HTML may …

YAPC Europe in Kyiv

By Jon Jensen
August 24, 2013

This year’s YAPC (Yet Another Perl Conference) Europe was held in Kyiv, Ukraine on August 12-14. There was a full schedule of three tracks of interesting talks at the conference, spread over three days.

I spoke just after lunch on the first day. My talk was Adventures in Perl Packaging, on our experience at End Point building a custom-compiled perl RPM for RHEL/CentOS, and many hundreds of CPAN modules into RPMs in a custom Yum repository. I also touched on similarities in Debian, alternative ways of getting custom perl & CPAN using perlbrew, plenv, and Carton (akin to Ruby’s rvm, rbenv, and bundler), and others’ efforts at packaging Perl modules in RPMs. I had several good follow-up conversations later about this and have some plans about how we may do things better in RHEL/CentOS 7.

Larry Wall, creator of Perl (and also patch!), was at this conference, and it was fun to talk with him and his wife Gloria again. Larry pretty definitively settled one question: In recent months an idea has been floated that the next version of Perl 5 might simply be renamed to Perl 7 to skip over the seemingly endlessly under construction Perl 6. That would solve an annoying marketing problem, …

Log Jam: Be careful with local syslog

By Josh Williams
August 22, 2013

Elephant by Shaun Tarlton

All they really wanted to do is log any query that takes over 10 seconds. Most of their queries are very simple and fast, but the application generates a few complicated queries for some actions. Recording anything that took longer than 10 seconds allowed them to concentrate on optimizing those. Thus, the following line was set in postgresql.conf:

log_min_duration_statement = 10

Log Everything

A little while back, Greg wrote about configuring Postgres to log everything, and the really good reasons to do so. That isn’t what they intended to to here, but is effectively what happened. The integer in log_min_duration_statement represents milliseconds, not seconds. With a 10ms threshold it wasn’t logging everything the database server was doing, but enough that this performance graph happened:

Reconstructed I/O Utilization

That is, admittedly, a fabricated performance plot. But it’s pretty close to what we were seeing at the time. The blue is the fast SAS array where all the Postgres data resides, showing lower than normal utilization before recovering after the configuration change. The maroon behind it is the SATA disk where the OS (and /var/log) resides, …

We’re hiring Rails developers!

By Jon Jensen
August 22, 2013

This position has been filled. See our active job listings here.

We’re looking for a few more talented Ruby on Rails developers to consult with our clients and develop their web applications. Do you like to focus on solving business problems? Do you take responsibility for getting a job done well without intensive oversight? Then please read on!

End Point is an 18-year-old web consulting company based in New York City, with 38 full-time employees working mostly remotely from home offices in the United States, Canada, and Europe. Our team is made up of strong ecommerce, database, and system administration talent, working together using ssh, tmux and screen, IRC, phone, Google+ hangouts, and Skype.

We serve over 200 clients ranging from small family businesses to large corporations, using a variety of open source technologies including Ruby, Python, Perl, Git, PostgreSQL, MySQL, CouchDB, Redis, Elasticsearch, jQuery, and many more, on Linux.

What you will be doing:

• Help clients determine their web application needs
• Build, test, release, and maintain web applications for our clients
• Work with open source tools and contribute back as opportunity arises
• Use your desktop platform of choice: Linux, Mac OS X, or Windows

What you will need: …

The Un-unaccentable Character

By Josh Williams
August 18, 2013

I typed “Unicode” into an online translator, and it responded saying it had no idea what the language was but it roughly translates to “Surprise!”

Recently a client sent over a problem getting some of their Postgres data through an ASCII-only ETL process. They only needed to worry about some occasional accent marks, and not any of the more uncommon or odd Unicode characters, thankfully. ☺ Or so we thought. The unaccent extension was a great starting point, but the problem they sent over boiled down to this:

postgres=# SELECT unaccent('e é ѐ');
 unaccent 
----------
 e e ѐ
(1 row)

unaccent() worked, except for that odd ѐ, which then failed the ETL task. That’s exactly what unnaccent is supposed to handle. The character è even appears in the unaccent.rules file. So what gives?

Well, if you’re in the habit of piping blog posts through hexdump (and who isn’t?) then you probably already know the answer. But even if not, you may already suspect that we’re dealing with a different character that just looks the same. And you’d be right. Specifically, the è in the rules file is from the more common Latin set, and the ѐ that doesn’t work is from the Cyrillic set. Pretty …

Give me my /var/log/rpmpkgs back!

By Emanuele “Lele” Calò
August 12, 2013

When switching from RHEL5 to RHEL6 everyone had fears and hopes about things which would have been lost and gained.

One of the lost ones is /var/log/rpmpkgs which is a nice tool which helps system administrator staying sane when a server rebuild or migration is needed by giving them the list of packages installed up to the day before.

How this feature works is that basically a daily cronjob dumps the installed packages in the log file /var/log/rpmpkgs, along with various information, for the sake of system maintainers.

What happened is that while this tool was included in the RPM package ’til RHEL5 (and CentOS 5.x), when releasing RHEL6 (and CentOS 6.x) they decided to split it and create a specific package called rpm-cron.

So if you’re among the ones who misses this useful feature, please fire up your SSH connection and type

yum install rpm-cron

And rejoice of the useful tool being back where it should be.