Our Blog

Using nginx to transparently modify/debug third-party content

By David Christensen
February 6, 2011

In tracking down a recent front-end bug for one of our client sites, I found myself needing to use the browser’s JavaScript debugger for stepping through some JavaScript code that lived in a mix of domains; this included a third-party framework as well as locally-hosted code which interfaced with—​and potentially interfered with—​said third-party code. (We’ll call said code foo.min.js for the purposes of this article.) The third-party code was a feature that was integrated into the client site using a custom domain name and was hosted and controlled by the third-party service with no ability for us to change directly. The custom domain name was part of a chain of CNAMEs which eventually pointed to the underlying actual IP of the third-party service, so their infrastructure obviously relied on getting the Host header correctly in the request to select which among many clients was being served.

It appeared as if there was a conflict between code on our site and that imported by the third party service. As part of the debugging process, I was stepping through the JavaScript in order to determine what if any conflicts there were, as well as their nature (e.g., conflicting library …

Monitoring with Purpose

By Jason Dixon
February 4, 2011

If you work on Internet systems all day like we do, there’s a good chance you use some sort of monitoring software. Almost every business knows they need monitoring. If you’re a small company or organization, you probably started out with something free like Nagios. Or maybe you’re a really small company and prefer to outsource your alerts to a web service like Pingdom. Either way, you understand that it’s important to know when your websites and mailservers are down. But do you monitor with purpose?

All too often I encounter installations where the Systems Administrator has spent countless hours setting up their checks, making sure their thresholds and notifications work as designed, without really considering what their response might be in the face of disaster (or an inconvenient page at 3am). Operations folk have been trained to make sure their systems are pingable, their CPU temperature is running cool and the system load is at a reasonable level. But what do you do when that alert comes in because the website load is running at 10 for the last 15 minutes? Is that bad? How can you be certain?

The art of monitoring isn’t simply reactive in nature. A good SysAdmin will …

In Our Own Words

By Steph Skardal
February 2, 2011

What do our words say about us?

Recently, I came across Wordle, a Java-based Google App Engine application that generates word clouds from websites and raw text. I wrote a cute little rake task to grab text from our blog to plug into Wordle. The rake task grabs the blog contents, uses REXML for parsing, and then lowercases the results. The task also applies a bit of aliasing since we use postgres, postgreSQL and pg interchangeably in our blog.

task :wordle => :environment do
   data = open('http://blog.endpoint.com/feeds/posts/default?alt=rss&max-results=999', 'User-Agent' => 'Ruby-Wget').read
   doc = REXML::Document.new(data)
   text = ''
   doc.root.each_element('//item') do |item|
     text += item.elements['description'].text.gsub(/<\/?[^>]*>/, "") + ' '
     text += item.elements['title'].text.gsub(/<\/?[^>]*>/, "") + ' '
   end
   text = text.downcase \
     .gsub(/\./, ' ')   \
     .gsub(/^\n/, '')   \
     .gsub(/ postgres /, ' postgresql ') \
     .gsub(/ pg /, ' postgresql ')
   file = File.new(ENV['filename'], …

Visit at DistribuTECH

By Ron Phipps
February 2, 2011

I had the chance to attend DistribuTECH in San Diego, CA this past week. DistribuTECH is billed as the utility industry’s leader in smart grid conference and exposition. End Point was present at the conference on behalf of Silver Spring Networks. Silver Spring Networks contracted with us to provide a Liquid Galaxy installation for their exhibit.

The Liquid Galaxy did its job from what I could tell. The exhibit was consistently surrounded with conference goers both interested in listening and watching the tours that were being presented as well as wanting to see what the Liquid Galaxy was all about. This was the first time I had seen the Liquid Galaxy and was quite impressed with how well it worked. I saw many people moving their bodies in sync with what was being displayed on the screen, showing that they felt immersed while within the galaxy. One gentleman knelt down while attempting to look under a graph that was being presented on the screen. This same person had returned to the exhibit several times, bringing colleagues back each time to “show off” what he had found.

I spent some time on the conference floor, checking out what was being displayed and seeing how others were …

Browser popularity

By Jon Jensen
February 1, 2011

It’s no secret that Internet Explorer has been steadily losing market share, while Chrome and Safari have been gaining.

But in the last couple of years I’ve been surprised to see how strong IE has remained among visitors to our website—​it’s usually been #2 after Firefox.

Recently this has changed and IE has dropped to 4th place among our visitors, and Chrome now has more than double the users that Safari does, as reported by Google Analytics:

That’s heartening. :)

JSON pretty-printer

By Jon Jensen
February 1, 2011

The other day Sonny Cook and I were troubleshooting some YUI JavaScript code and looking at some fairly complex JSON. It would obviously be a lot easier to read if each nested data structure were indented, and spacing standardized.

I threw together a little Perl program based on the JSON man page:

#!/usr/bin/env perl

use JSON;

my $json = JSON->new;
undef $/;
while (<>) {
    print $json->pretty->encode($json->decode($_));
}

It took all of 2 or 3 minutes and I even left out strictures and warnings. Living on the edge!

It turns a mess like this (sample from json.org):

{"glossary":{"title":"example glossary","GlossDiv":{"title":"S","GlossList":
{"GlossEntry":{"ID":"SGML","SortAs":"SGML","GlossTerm":"Standard Generalized Markup Language",
"Acronym":"SGML","Abbrev":"ISO 8879:1986","GlossDef":{"para":
"A meta-markup language,used to create markup languages such as DocBook.",
"GlossSeeAlso":["GML","XML"]},"GlossSee":"markup"}}}}}

into this much more readable …

jQuery Tips and an Ecommerce Demo

By Steph Skardal
January 31, 2011

I’ve recently been jumping back and forth between YUI and jQuery on several different client projects. I prefer working with jQuery, but whenever I work with another framework, I realize what’s out there and how I should continue to improve my skills in my preferred framework. I read up on jQuery tips and put together a summary of common tips I need to follow more explained here in an ecommerce demo.

The Setup

Before we get started, some notes on the ecommerce demo and performance testing:

• The fake product images come from DryIcons
• A JSON array contains product information (price, image, and title).
• The code runs on a quick and dirty sinatra app.
• console.time('task') and console.timeEnd('task') are used to debug task runtime for performance measurement
• The performance numbers provided in the article were measured in Chrome on Ubuntu where the average of 10 tests is reported. In all the tests, an additional for loop was added as part of the test to see measurable performance differences. See the notes at the bottom of the article on performance differences between Chrome and Firefox.

A screenshot from the demo app.

1. The first tip I came across was a recommendation to use a …

DROID 2 review

By Jon Jensen
January 24, 2011

I got a Motorola DROID 2 phone a couple of months ago and have assembled here my notes about how it has worked out so far. First, some background.

This is my second Android phone. My first was the Google Ion, basically the same as the HTC Magic. That was running standard Android 1.5 (Cupcake), while the DROID 2 runs Android 2.2 (Froyo) tweaked somewhat by Motorola. I’ve used several other Android phones belonging to friends and relatives.

Overall I like the Android operating system fairly well. Like everything, it can be improved. It’s been advancing at a fairly quick pace. It’s mostly free software. Too many phones are locked down and have to be broken into to change the operating system, but Android’s still a freer computing environment than most phones have and I hope the situation will improve over time.

I take for granted much of Android’s feature set: The excellent system-wide notification bar that many apps hook into and which is always easy to get to. The solid multitasking. Automatic screen adjustment for using the phone in landscape vs. portrait mode. The ability to mount the normal filesystem on the SD card from another computer via USB or by removing the SD card. The …