Our Blog

Sanity, thy name is not MySQL

By Jeff Boes
April 10, 2014

Probably old news, but I hit this MySQL oddity today after a long day dealing with unrelated crazy stuff and it just made me go cross-eyed:

CREATE TABLE foo (id integer, val enum('','1'));
INSERT INTO foo VALUES (1, '');
INSERT INTO foo VALUES (2, '1');
SELECT * FROM foo WHERE val = 1;

What row do you get? I’ll wait while you second- and third-guess yourself.

It turns out that the “enum” datatype in MySQL just translates to a set of unique integer values. In our case, that means:

• ’’ == 1
• ‘1’ == 2

So you get the row with (1,’’). Now, if that doesn’t confuse readers of your code, I don’t know what will.

Filling Gaps in Cumulative Sum in Postgres

By Szymon Lipiński
April 8, 2014

I found an interesting problem. There was a table with some data, among which there was a date and an integer value. The problem was to get cumulative sum for all the dates, however including dates for which we don’t have entries. In case of such dates we should use the last calculated sum.

Example Data

I will use an example table:

# CREATE TABLE test (d DATE, v INTEGER);

with sample data:

# INSERT INTO test(d,v)
  VALUES('2014-02-01', 10),
        ('2014-02-02', 30),
        ('2014-02-05', 10),
        ('2014-02-10', 3);

Then the data in the table looks like:

# SELECT * FROM test;
     d      |  v
------------+----
 2014-02-01 | 10
 2014-02-02 | 30
 2014-02-05 | 10
 2014-02-10 |  3
(4 rows)

What I want is to have a cumulative sum for each day. Cumulative sum is a sum for all the earlier numbers, so for the above data I want to get:

     d      |  v
------------+----
 2014-02-01 | 10
 2014-02-02 | 40
 2014-02-05 | 50
 2014-02-10 | 53
(4 rows)

The simple query for getting the data set like shown above is:

SELECT DISTINCT d, SUM(v) OVER (ORDER BY d) v
FROM test
ORDER BY d ASC;

Filling The Gaps

The query calculates the cumulative sum for each row. …

Open Source: Challenges of Modularity and Extensibility

By Steph Skardal
April 7, 2014

While I was at the I Annotate 2014 conference last week, I spoke with a couple developers about the challenges of working in open source. Specifically, the Annotator JavaScript library that we are using for H2O is getting a bit of a push from the community to decouple (or make more modular) some components as well as improve the extensibility. Similarly, Spree, an open source Ruby on Rails platform that End Point has sponsored in the past and continued to work with, made a shift from a monolithic platform to a modular (via Ruby gems) approach, and Piggybak started out as a modular and extensible ecommerce solution. I like doodling, so here’s a diagram that represents the ecosystem of building out an open source tool with a supplemental explanation below:

Here are some questions I consider on these topics:

• What is the cost of extensibility?

  • code complexity
  • code maintenance (indirectly, as code complexity increases)
  • harder learning curve for new developers (indirectly, as code complexity increases)
  • performance implications (possibly, indirectly, as code complexity increases)
  • difficulty in testing code (possibly)

• What is the cost of modularity?

  • same as cost of extensibility …

I Annotate 2014 Day 2: If it were easy, we would have already solved it

By Steph Skardal
April 5, 2014

H2O & Annotator

Yesterday I gave my talk on Integrating Annotator with H2O, which covered the specific implementation details of integrating the open source JavaScript based tool Annotator into H2O, including a history of annotation and highlights of some of the challenges. I’ll update the link here to my slides when they are available on the I Annotate conference website.

Anyways…

Version Control

One of the interesting recurring topics of the conference was the concept of version control, version control of text and other multi-media content, and how to present a user interface for version control that makes sense to non-developers (or those not familiar with code based version control). A simple example of what I mean by the problem of version control on the web is described in the following Facebook scenario:

• User A updates status on Facebook
• User B comments on User A’s status
• User C comments on User A’s status, with reference or comment to User B’s status
• User B edits original comment
• User C’s comment no longer is applicable given the context, and doesn’t make sense to users who have not seen User B’s original comment

Facebook doesn’t do anything about this …

I Annotate 2014 Conference: Day 1

By Steph Skardal
April 4, 2014

I’m here in San Francisco for the second annual I Annotate conference. Today I’m presenting my work on the H2O project, but in this post I’ll share a couple of focus points for the conference thus far, described below.

What do we mean by Annotations?

Annotation work is off the path of End Point’s ecommerce focus, and annotations means different things for different users, so to give a bit of context: To me, an annotation is markup tied to single target content (image, text, video). There are other interpretations of annotations, such as highlighted text with no markup (ie flagging some target content), and cases where annotations are tied to multiple pieces of target contents.

Annotations in General

One of the focuses of yesterday’s talks was the topic of how to allow for the powerful concept of annotations to succeed on the web. Ivan Herman of the W3C touched on why the web has succeeded, and what we can learn from that success to help the idea of annotations. The web has been a great idea, interoperable, decentralized, and open source and we hope that those concepts can translate to web annotations to help them be successful. Another interesting topic Tom Lehman of RapGenius …

2014 Mountain West Ruby Conference Day 2

By Brian Gadoury
March 27, 2014

This past Friday concluded my second Mountain West Ruby Conference right here in my backyard of Salt Lake City, Utah. Just like the 2013 MWRC, this year’s conference was great. It was also nice to meet up with fellow remote co-worker Mike Farmer for both days. Here are a few of my personal favorites from day 2 of the conference, which I almost missed when I had the audacity to show up without a Macbook Air/Pro. (Kidding!)

Randy Coulman — Affordances in Programming Languages

Affordances (“A quality of an object or environment that allows someone to perform an action”) are all around us. Randy opened with some examples of poor affordances such as a crosswalk button with two enticing-looking widgets that requires arrows drawn on the box to point to the one that actually is a button. Then, a counter-example showing the “walking” or “standing” footprints painted on an escalator and how they instantly and intuitively communicate how to best use the escalator.

Randy followed with a few examples of affordances in software. One of them was a simple Ruby implementation of an affordance called Deterministic Destructors. It was a method that acquired a resource, yielded to a block argument, …

Puppet, Salt, and DevOps (a review of the MountainWest DevOps conference)

By Spencer Christensen
March 27, 2014

Last week I attended the MountainWest DevOps conference held in Salt Lake City, Utah. This was a one day conference with a good set of presenters and lightning talks. There were several interesting topics presented, but I’ll only review a few I wanted to highlight.

I Serve No Master!

Aaron Gibson of Adaptive Computing discussed a very common problem with Puppet (and other configuration management systems): they work well in the scenario they were designed for but what about when the situation isn’t typical? Aaron had a situation where developers and QA engineers could instantiate systems themselves via OpenStack, however the process for installing their company’s software stack on those VMs was inconsistent, mostly manual, and took many hours. One of the pain points he shared, which I related to, was dealing with registering a puppet node with a puppet master—​the sometimes painful back and forth of certificate issuing and signing.

His solution was to remove the puppet master completely from the process. Instead he created a bash wrapper script to execute a workflow around what was needed, still using puppet manifests on each system but run locally. This wrapper tool, called …

ZNC: An IRC Bouncer

By Greg Davidson
March 27, 2014

Kickin’ it Old Skool

At End Point, we use IRC extensively for group chat and messaging. Prior to starting here I had been an occasional IRC user—​asking questions about various open source projects on Freenode and helping others as well. When I began to use IRC daily, I ran into a few things that bugged me and thought I would write about what I have done to mitigate those. While it might not be as fancy as Campfire, HipChat or Slack I’m happy with my setup now.

What did I miss?

The first thing that annoyed me about IRC was the lack of persistence. If I wasn’t logged on to the server, I missed out on the action. This was exacerbated by the fact that I live in the Pacific Time zone and lots of discussion takes place before my work day begins. Some people on our team solve this issue by running a terminal-based IRC client (like irssi or WeeChat inside a tmux or GNU Screen session on a remote server. This approach works well until the server needs to be rebooted (e.g. for OS or kernel updates etc.). It also introduces the limitation of using a terminal-based client which isn’t for everyone.

Test Driving IRC Clients

The next challenge was finding a good IRC client. In my attempt to …