Our Blog

Increasing maildrop’s hardcoded 5-minute timeout

By Jon Jensen
September 4, 2009

One of the ways I like to retrieve email is to use fetchmail as a POP and IMAP client with maildrop as the local delivery agent. I prefer maildrop to Postfix, Exim, or sendmail for this because it doesn’t add any headers to the messages.

The only annoyance I have had is that maildrop has a hardcoded hard timeout of 5 minutes for delivering a mail message. When downloading a very long message such as a Git commit notification of a few hundred megabytes, or a short message with an attached file of dozens of megabytes, especially over a slow network connections, this timeout prevents the complete message from being delivered.

Confusingly, a partial message will be delivered locally without warning—​with the attachment or other long message data truncated. When fetchmail receives the error status return from maildrop, it then tries again, and given similar circumstances it suffers a similar fate. In the worst case this leads to hours of clogged tubes and many partial copies of the same email message, and no other new mail.

This maildrop hard timeout is compiled in and there is no runtime option to override it. Thus it is helpful to compile a custom build from source, specifying a …

Tests are not Specs

By Steven Jenkins
September 4, 2009

We’re big fans of Test Driven Development (TDD). However, a co-worker and I encountered some obstacles because we focused too intently on writing tests and didn’t spend enough up-front time on good, old-fashioned specifications.

We initially discussed the new system (which is a publish/subscribe interface used to do event management for a reasonably large system, which totals around 70K lines of Ruby). My co-worker did most of the design and put a high-level one-pager together to outline how things should work, wrote unit tests and a skeleton set of classes and modules, then handed the project to me to implement.

So far, so good. All I had to do was make all of the tests pass, and we were finished.

We only had unit tests, no integration tests, so there was no guarantee that once I was done coding, that the integration work would actually solve the problem at hand. In Testing (i.e., the academic discipline that studies testing), this is referred to as a validation problem: we may have a repeatable, accurate measure, but it’s measuring the wrong thing.

We knew that was a weakness, but we pressed ahead anyway, expecting to tackle that later. As an example, we identified 3 different …

Rejecting SSLv2 politely or brusquely

By Jon Jensen
September 2, 2009

Once upon a time there were still people using browsers that only supported SSLv2. It’s been a long time since those browsers were current, but when running an ecommerce site you typically want to support as many users as you possibly can, so you support old stuff much longer than most people still need it.

At least 4 years ago, people began to discuss disabling SSLv2 entirely due to fundamental security flaws. See the Debian and GnuTLS discussions, and this blog post about PCI’s stance on SSLv2, for example.

To politely alert people using those older browsers, yet still refusing to transport confidential information over the insecure SSLv2 and with ciphers weaker than 128 bits, we used an Apache configuration such as this:

# Require SSLv3 or TLSv1 with at least 128-bit cipher
<Directory "/">
    SSLRequireSSL
    # Make an exception for the error document itself
    SSLRequire (%{SSL_PROTOCOL} != "SSLv2" and %{SSL_CIPHER_USEKEYSIZE} >= 128) or %{REQUEST_URI} =~ m:^/errors/:
    ErrorDocument 403 /errors/403-weak-ssl.html
</Directory>

That accepts their SSLv2 connection, but displays an error page explaining the problem and suggesting some links to …

JavaScript fun with IE 8

By Max Cohan
September 1, 2009

I ran into, and found solutions for, two major gotchas targeting IE 8 with a jQuery-based (and rather JavaScript-heavy) web application.

First is to specify the ‘IE 8 Standard’ rendering mode by adding the following meta tag:

The default rendering mode is rather glitchy and tends to produce all sorts of garbage from ‘clean’ HTML and JavaScript. The result renders slightly different sizes, reports incorrect values from common jQuery calls, etc.

The default rendering also caused various layout issues (CSS handling looked more like IE 6 than IE 7). Also, minor errors (an extra ’’ tag on one panel) caused the entire panel to not render.

Another issue is the browser is overly lazy about invalidating the cache for AJAX pulled content, especially (X)HTML. This means that though you think you’re pulling current data, in reality it keeps feeding you the same old data. This also means that if you use the same exact URL for HTML & JSON data, you must add a parameter to avoid running into cache collisions. IE 8 only seemed to honor ‘Cache-control: no-cache’ in the header to cause it to behave properly.

On the other side, I’ve got a big thumbs up for jQuery. I was able to …

File test comparison table for shell, Perl, Ruby, and Python

By Jon Jensen
August 31, 2009

A few days ago, my co-worker Richard asked how in Python you would do the -x Bourne shell and Perl file test that checks whether a file is executable. This is (for me, at least) a really commonly used function but one I hadn’t needed to do yet in Python, so I looked it up.

That wasn’t so hard to find, but then I wondered about the other shell and Perl file tests that I use all the time. Finding equivalents for those was harder than I expected. A web search didn’t turn much up aside from language holy wars and limited answers, but I didn’t find any exhaustive list.

So I made my own. Below is a table comparing file test operators in the original Bourne shell-compatibles bash, ksh, and zsh; Perl’s expanded set; Ruby’s which was derived first from Perl; and equivalent Python code.

There are still some blanks where I didn’t find a good equivalent. Of course I’m sure it’s possible with enough custom logic to achieve the same end, but I have tried to stick with relatively simple formulations using built-in functions for now. I’ll be happy to fill in the blanks if any readers make suggestions.

Performance notes on avoiding multiple stats of the same file:

• Starting with Perl 5.9.1, file …

Interchange news

By Jon Jensen
August 24, 2009

Tomorrow we’ll be having an Interchange community meeting on IRC. All Interchange users and any other interested parties are invited to participate.

Also, just recently, End Point’s own David Christensen joined the Interchange Development Group and became a core committer. Congratulations, David, and keep up the good work!

Perl’s Scalar::Util::dualvar

By Jon Jensen
August 24, 2009

I just came across this fun Perl function that I can’t think of a (good) use for, but have to share.

In the Scalar::Util module is the function dualvar:

dualvar NUM, STRING

Returns a scalar that has the value NUM in a numeric context and the value STRING in a string context.

    $foo = dualvar 10, "Hello";
    $num = $foo + 2;                    # 12
    $str = $foo . " world";             # Hello world

Using that in the right place could lead a future programmer down some fun debugging paths!

Defining variables for rpmbuild

By Jon Jensen
August 20, 2009

RPM spec files offer a way to define and test build variables with a directive like this:

%define <variable> <value>

Sometimes it’s useful to override such variables temporarily for a single build, without modifying the spec file, which would make the changed variable appear in the output source RPM. For some reason, how to do this has been hard for me to find in the docs and hard for me to remember, despite its simplicity.

Here’s how. For example, to override the standard _prefix variable with value /usr/local:

rpmbuild -ba SPECS/$package.spec --define '_prefix /usr/local'