Our Blog

Piggybak: Upgrade to Rails 4.1.0

By Steph Skardal
April 16, 2014

Piggybak and gems available in the demo (piggybak_variants, piggybak_giftcerts, piggybak_coupons, piggybak_bundle_discounts, piggybak_taxonomy) have been updated to Rails 4.1.0, Ruby 2.1.1 via Piggybak version gem 0.7.1. Interested in the technical details of the upgrade? Here are some fine points:

• Dependencies were refactored so that the parent Rails app controls the Rails dependency only. There was a bit of redundancy in the various plugin gemspec dependencies. This has been cleaned up so the parent Rails app shall be the canonical reference to the Rails version used in the application.
• Modified use of assets which require “//= require piggybak/piggybak-application” to be added to the assets moving forward. There have been several observed issues with precompling and asset behavior, so I simplified this by requiring this require to be added to the main Rails application.js for now. The engine file is supposed to have a way around this, but it has not behaved as expected, specifically on unique deployment architectures (e.g. Heroku). Patches welcome to address this.
• Tables migrated to namespaced tables, e.g. “orders” migrated to “piggybak_orders”. This is how namespaced engine …

jQuery Content Replacement with AJAX

By Jeff Boes
April 15, 2014

This is not a huge breakthrough, but it colored in some gaps in my knowledge so I thought I would share.

Let’s say you have a product flypage for a widget that comes in several colors. Other than some of the descriptive text, and maybe a hidden field for use in ordering one color instead of another, all the pages look the same. So your page looks like this (extremely simplified):

... a lot of boilerplate ...
<form id="order_item">
<input name="sku" type="hidden" value="WDGT-001-RED"/>
<input type="hidden"/>
</form>
... a lot more boilerplate ...

Probably the page is generated into a template based on a parameter or path segment:

http://.../app/product/WDGT-001-RED

What we’re going to add is a quick-and-dirty way of having your page rewrite itself on the fly with just the bits that change when you select a different version (or variant) of the same product. E.g.,

<select name="sku">
<option value="WDGT-001-RED">Cinnamon Surprise</option>
<option value="WDGT-002-BLK">Midnight Delight</option>
<option value="WDGT-003-YLW">Banana Rama</option>
</ …

SPF, DKIM and DMARC brief explanation and best practices

By Emanuele “Lele” Calò
April 15, 2014

Spam mail messages have been a plague since the Internet became popular and they kept growing more and more as the number of devices and people connected grew. Despite the numerous attempts of creation of anti-spam tools, there’s still a fairly high number of unwanted messages sent every day.

Luckily it seems that lately something is changing with the adoption of three (relatively) new tools which are starting to be widely used: SPF, DKIM and DMARC. Let’s have a quick look at each of these tools and what they achieve.

What are SPF, DKIM and DMARC

SPF (Sender Policy Framework) is a DNS text entry which shows a list of servers that should be considered allowed to send mail for a specific domain. Incidentally the fact that SPF is a DNS entry can also considered a way to enforce the fact that the list is authoritative for the domain, since the owners/administrators are the only people allowed to add/change that main domain zone.

DKIM (DomainKeys Identified Mail) should be instead considered a method to verify that the messages’ content are trustworthy, meaning that they weren’t changed from the moment the message left the initial mail server. This additional layer of trustability is …

Speeding Up Saving Millions of ORM Objects in PostgreSQL

By Szymon Lipiński
April 11, 2014

The Problem

Sometimes you need to generate sample data, like random data for tests. Sometimes you need to generate it with huge amount of code you have in your ORM mappings, just because an architect decided that all the logic needs to be stored in the ORM, and the database should be just a dummy data container. The real reason is not important—​the problem is: let’s generate lots of, millions of rows, for a sample table from ORM mappings.

Sometimes the data is read from a file, but due to business logic kept in ORM, you need to load the data from file to ORM and then save the millions of ORM objects to database.

This can be done in many different ways, but here I will concentrate on making that as fast as possible.

I will use PostgreSQL and SQLAlchemy (with psycopg2) for ORM, so all the code will be implemented in Python. I will create a couple of functions, each implementing another solution for saving the data to the database, and I will test them using 10k and 100k of generated ORM objects.

Sample Table

The table I used is quite simple, just a simplified blog post:

CREATE TABLE posts (
  id SERIAL PRIMARY KEY,
  title TEXT NOT NULL,
  body TEXT NOT NULL,
  payload TEXT NOT NULL …

Sanity, thy name is not MySQL

By Jeff Boes
April 10, 2014

Probably old news, but I hit this MySQL oddity today after a long day dealing with unrelated crazy stuff and it just made me go cross-eyed:

CREATE TABLE foo (id integer, val enum('','1'));
INSERT INTO foo VALUES (1, '');
INSERT INTO foo VALUES (2, '1');
SELECT * FROM foo WHERE val = 1;

What row do you get? I’ll wait while you second- and third-guess yourself.

It turns out that the “enum” datatype in MySQL just translates to a set of unique integer values. In our case, that means:

• ’’ == 1
• ‘1’ == 2

So you get the row with (1,’’). Now, if that doesn’t confuse readers of your code, I don’t know what will.

Filling Gaps in Cumulative Sum in Postgres

By Szymon Lipiński
April 8, 2014

I found an interesting problem. There was a table with some data, among which there was a date and an integer value. The problem was to get cumulative sum for all the dates, however including dates for which we don’t have entries. In case of such dates we should use the last calculated sum.

Example Data

I will use an example table:

# CREATE TABLE test (d DATE, v INTEGER);

with sample data:

# INSERT INTO test(d,v)
  VALUES('2014-02-01', 10),
        ('2014-02-02', 30),
        ('2014-02-05', 10),
        ('2014-02-10', 3);

Then the data in the table looks like:

# SELECT * FROM test;
     d      |  v
------------+----
 2014-02-01 | 10
 2014-02-02 | 30
 2014-02-05 | 10
 2014-02-10 |  3
(4 rows)

What I want is to have a cumulative sum for each day. Cumulative sum is a sum for all the earlier numbers, so for the above data I want to get:

     d      |  v
------------+----
 2014-02-01 | 10
 2014-02-02 | 40
 2014-02-05 | 50
 2014-02-10 | 53
(4 rows)

The simple query for getting the data set like shown above is:

SELECT DISTINCT d, SUM(v) OVER (ORDER BY d) v
FROM test
ORDER BY d ASC;

Filling The Gaps

The query calculates the cumulative sum for each row. …

Open Source: Challenges of Modularity and Extensibility

By Steph Skardal
April 7, 2014

While I was at the I Annotate 2014 conference last week, I spoke with a couple developers about the challenges of working in open source. Specifically, the Annotator JavaScript library that we are using for H2O is getting a bit of a push from the community to decouple (or make more modular) some components as well as improve the extensibility. Similarly, Spree, an open source Ruby on Rails platform that End Point has sponsored in the past and continued to work with, made a shift from a monolithic platform to a modular (via Ruby gems) approach, and Piggybak started out as a modular and extensible ecommerce solution. I like doodling, so here’s a diagram that represents the ecosystem of building out an open source tool with a supplemental explanation below:

Here are some questions I consider on these topics:

• What is the cost of extensibility?

  • code complexity
  • code maintenance (indirectly, as code complexity increases)
  • harder learning curve for new developers (indirectly, as code complexity increases)
  • performance implications (possibly, indirectly, as code complexity increases)
  • difficulty in testing code (possibly)

• What is the cost of modularity?

  • same as cost of extensibility …

I Annotate 2014 Day 2: If it were easy, we would have already solved it

By Steph Skardal
April 5, 2014

H2O & Annotator

Yesterday I gave my talk on Integrating Annotator with H2O, which covered the specific implementation details of integrating the open source JavaScript based tool Annotator into H2O, including a history of annotation and highlights of some of the challenges. I’ll update the link here to my slides when they are available on the I Annotate conference website.

Anyways…

Version Control

One of the interesting recurring topics of the conference was the concept of version control, version control of text and other multi-media content, and how to present a user interface for version control that makes sense to non-developers (or those not familiar with code based version control). A simple example of what I mean by the problem of version control on the web is described in the following Facebook scenario:

• User A updates status on Facebook
• User B comments on User A’s status
• User C comments on User A’s status, with reference or comment to User B’s status
• User B edits original comment
• User C’s comment no longer is applicable given the context, and doesn’t make sense to users who have not seen User B’s original comment

Facebook doesn’t do anything about this …