Our Blog

Postfix Address Verification

By Marco Matarazzo
May 28, 2015

We recently upgraded some mail servers, moving from Exim to Postfix in the process. These server works as a front line spam/RBL filter, rejecting invalid message and relaying valid ones to different SMTP based on the destination domain.

While looking for the best configuration layout to achieve this, we found that Postfix has a very useful and interesting feature: Address Verification. This technique allows the Postfix server to check that a sender or a recipient address is valid before accepting a message, preventing junk messages from entering the queue.

How does Address Verification work?

Upon receiving a message Postfix will probe the preferred MTA for the address. If that address is valid the message is accepted and processed, otherwise it is rejected.

Message Probes does not actually go through the whole delivery process; Postfix will just connect to the MTA, send a HELO + MAIL FROM + RCPT TO sequence and check its response. Probe checks results are cached on disk, minimizing network and resource impact. During this check the client is put “on hold”; if the probe takes too much a temporary reject is given; a legitimate mail server will have no problem retrying the delivery …

Postgres “unsupported frontend protocol” mystery

By Greg Sabino Mullane
May 28, 2015

"Koala Portrait" by James Niland

The wonderful tail_n_mail program continues to provide me with new mysteries from our Postgres clients. One of the main functions it provides is to send an immediate email to us when an unexpected FATAL (or ERROR or PANIC) message appears in the Postgres logs. While these are often simple application errors, or deeper problems such as running out of disk space, once in a blue moon you see something completely unexpected. Some time ago, I saw a bunch of these messages appear in an email from a tail_n_mail email:

[1] From files A to B Count: 2
First: [A] 2015-12-01T06:30:00 server1 postgres[1948]
Last:  [B] 2015-12-01T06:30:00 server2 postgres[29107]
FATAL: unsupported frontend protocol 65363.19778: server supports 1.0 to 3.0

I knew what caused this error in general, but decided to get to the bottom of the problem. Before we go into the specific error, let’s review what causes this particular message to appear. When a Postgres client (such as psql or DBD::Pg) connects to Postgres, the first thing it does is to issue a startup message. One of the things included in this request is the version of the Postgres protocol the client wishes to use. Since …

Connected to PgBouncer or Postgres?

By Greg Sabino Mullane
May 18, 2015

Image by David Wright

Determining if your current database connection is using PgBouncer, or going directly to Postgres itself, can be challenging, as PgBouncer is a very low-level, transparent interface. It is possible, and here are some detection methods you can use.

This was inspired by someone asking on the Perl DBD IRC channel if it was possible to easily tell if your current database handle (usually “$dbh”) is connected to PgBouncer or not. Since I’ve seen this question asked in other venues, I decided to take a crack at it.

There are actually two questions to be answered: (1) are we connected to PgBouncer, and if so, (2) what pool_mode is being run? The quickest and easiest way I found to answer the first question is to try and connect to a non-existent database. Normally, this is a FATAL message, as seen here:

$ psql testdb -p 5432
testdb=# \c ghostdb
FATAL:  database "ghostdb" does not exist
Previous connection kept
testdb=#

However, a slightly different ERROR message is returned if the same thing is attempted while connected to PgBouncer:

$ psql testdb -p 6432
testdb=# \c ghostdb
ERROR:  No such database: ghostdb
Previous connection kept
testdb=#

Thus, an ERROR …

Job opening: Web developer

By Jon Jensen
May 13, 2015

This position has been filled. See our active job listings here.

We are looking for another talented software developer to consult with our clients and develop their web applications in AngularJS, Node.js, Ruby on Rails, and other technologies. If you like to focus on solving business problems and can take responsibility for getting a job done well without intensive oversight, please read on!

What is in it for you?

• Flexible full-time work hours
• Health insurance benefit
• Paid holidays and vacation
• 401(k) retirement savings plan (U.S. employees)
• Annual bonus opportunity
• Ability to move without being tied to your job location

What you will be doing:

• Work from your home office, or from our offices in New York City and the Tennessee Tri-Cities area
• Consult with clients to determine their web application needs
• Build, test, release, and maintain web applications for our clients
• Work with open source tools and contribute back as opportunity arises
• Use your desktop platform of choice: Linux, Mac OS X, Windows
• Learn and put to use new technologies
• Direct much of your own work

What you will need:

• Professional experience building reliable server-side apps in Ruby on Rails, Node.js and Express, Django, CakePHP, etc.
• Good front-end web skills with …

Non-English Google Earth Layers on the Liquid Galaxy

By Josh Ausborne
May 12, 2015

The availability to activate layers within Google Earth is one of the things that makes Earth so powerful. In fact, there are many standard layers that are built into Earth, including weather, roads, place names, etc. There are also some additional layers that have some really interesting information, including one I noticed relatively recently called “Appalachian Mountaintop Removal” which is interesting to me now that I live in Tennessee.

As you can see, however, that while some of these available layers are interesting on a desktop, they’re not necessarily very visually appealing on a Liquid Galaxy. We have identified a standard set of layers to enable and disable within Earth so that things don’t appear too cluttered while running. Some things we’ve disabled by default are the weather and the roads, as well as many levels of place names and boundaries. For example, we have boundaries of countries and water bodies enabled, but don’t want lines drawn for states, provinces, counties, or other areas such as those.

To disable these layers, we modify the GECommonSettings.conf file on the machines that running Earth. This file has everything pretty well spelled out in a …

Raw Packet Manipulation with Scapy

By Kirk Harr
April 29, 2015

Installation

Scapy is a Python-based packet manipulation tool which has a number of useful features for those looking to perform raw TCP/IP requests and analysis. To get Scapy installed in your environment the best options are to either build from the distributed zip of the current version, or there are also some pre-built packages for Red Hat and Debian derived linux OS.

Using Scapy

When getting started with Scapy, it’s useful to start to understand how all the aspects of the connection get encapsulated into the Python syntax. Here is an example of creating a simple IP request:

Welcome to Scapy (2.2.0)
>>> a=IP(ttl=10)
>>> a
<IP  ttl=10 |>
>>> a.dst="10.1.0.1"
>>> a
<IP  ttl=10 dst=10.1.0.1 |>
>>> a.src
'10.1.0.2'
>>> a.ttl
10

In this case I created a single request which was point from one host on my network to the default gateway on the same network. Scapy will allow the capability to create any TCP/IP request in raw form. There are a huge number of possible options for Scapy that can be applied, as well as huge number of possible packet types defined. The documentation with these options and …

Protect Interchange Passwords with Bcrypt

By Mark Johnson
April 28, 2015

Interchange default configurations have not done a good job of keeping up with the best available password security for its user accounts. Typically, there are two account profiles associated with a standard Interchange installation: one for admin users (access table) where the password is stored using Perl’s crypt() command (bad); and one for customers (userdb) where the password isn’t encrypted at all (even worse). Other hashing algorithms have long been available (MD5, salted MD5, SHA1) but are not used by default and have for some time not been useful protection. Part of this is convenience (tools for retrieving passwords and ability to distribute links into user assets) and part is inertia. And no small part was the absence of a strong cryptographic option for password storage until the addition of Bcrypt to the account management module.

The challenge we face in protecting passwords is that hardware continues to advance at a rapid rate, and with more computational power and storage capacity, brute-force attacks become increasingly effective and widely available. Jeff Jarmoc’s Enough with the Salts provides some excellent discussion and background on the subject. To counter …

RailsConf 2015 for the non-Attendee

By Steph Skardal
April 28, 2015

This blog post is really for myself. Because I had the unique experience of bringing a baby to a conference, I made an extra effort to talk to other attendees about what sessions shouldn’t be missed. Here are the top takeaways from the conference that I recommend (in no particular order):

• Watch DHH’s Keynote
• … and Aaron Patterson Keynote
• Watch Kent Beck’s Keynote
• Amelia Bedelia Learns to Code, by Kylie Stradley, was a popular talk from day 1. Video link forthcoming.
• Watch Shipping Ruby Apps with Docker by Bryan Helmkamp. This video was strongly recommended before jumping into Docker, which was a trendy topic in dev-ops talks.
• Watch Justin Searls’ talk Sometimes a Controller is Just a Controller.
• Watch Sandi Metz’s talk Nothing is Something.
• Read this book by Don Norman if you care about UX. Joe Mastey recommended it in his talk, Bringing UX to Your Code.
• Yehuda Katz & Tom Dale’s talk, How Rust Helped us Write Better Ruby was standing room only. Video link forthcoming. Also check out their fireside chat on Ember.js.
• Riding Rails for 10 Years by John Duff gives a great history of the evolution of Rails in the Shopify codebase.

Right now, the videos are all unedited from …