Our Blog

2015 Perl Dancer Conference videos

By Sam Batschelet
December 30, 2015

The 2015 Perl Dancer Conference has recently released the presentation videos. This year the conference was hosted in beautiful Vienna, Austria. Josh Lavin and I were both honored to attend the conference as well as give talks. Earlier, Josh wrote summaries of the conference:

Conference Recap

• Training Days
• Conference Days

Conference Presentations

SpaceCamps “The Final Frontier”

I gave a talk exploring new technologies for End Point’s own DevCamps development tool. During the presentation I detailed my research into containers and what a cloud-based development environment might look like.

SpaceCamps Presentation Video

AngularJS & Dancer for Modern Web Development

Josh detailed his experience migrating legacy applications utilizing Dancer, AngularJS, and modern Perl techniques. Josh highlighted the challenges he faced during the process, as well as lessons he learned along the way.

AngularJS & Dancer for Modern Web Development Presentation Video

Lightning Talks

Josh and I both gave short “lightning talks.” Josh’s was on Writing Unit Tests for a Legacy App (Interchange 5), and mine was on Plack & Interchange 5.

To review the rest of the presentations please checkout the …

Git: pre-receive hook error on CentOS 7

By Marco Matarazzo
December 29, 2015

We recently had to move a git repository from an old CentOS 5 to a new CentOS 7 server.

On the old CentOS 5 we had a recent, custom compiled version of git while on the new server we are using the system default old 1.8 version, shipped by the official CentOS repositories. And, as usual when you tell yourself “What could possibly go wrong?”, something did: every push began to return the dreaded “fatal: The remote end hung up unexpectedly” error.

After some time spent trying to debug the problem, we managed to isolate the problem to the pre-receive hook, specifically active on that repository. The script was very simple:

 #!/bin/bash
 read_only_users="alice bob"
 for user in $read_only_users
 do
     if [ $USER == $user ]; then
         echo "User $USER has read-only access, push blocked."
         exit 1
     fi
 done

… which apparently had no visible mistakes. On top of the lack of errors, this very same script used to work perfectly for years on the old server. Unfortunately, and quite disappointingly, even changing it to a simple:

 #!/bin/bash
 echo "These are not the droids you are looking for. Move along."

…did not help and the error …

Event Listener Housekeeping in Angular Apps

By Greg Davidson
December 22, 2015

I was recently debugging an issue where a large number of errors suddenly cropped up in an angular application. The client reported that the majority of the errors were occurring on the product wall which was an area of application I was responsible for. After some sleuthing and debugging I determined the culprit was a scroll event listener in an unrelated angular controller. When customers viewed this section of the application, the scroll listener was added to manage the visibility of some navigation elements. However, when the customer moved on to other sections of the site the listener continued to fire in a context it was not expecting.

Scroll event listeners fire very often so this explained the sheer volume of errors. The product wall is a tall section of the site with lots of content so this explained why the bulk of the errors were happening there. The solution was to simply listen to the $destroy event in the controller and unbind the troublesome scroll listener:

$scope.$on('$destroy', function() {
  $window.unbind('scroll');
});

Single page apps do not have the benefit of getting a clean state with each page load. Because of this it’s important to keep …

ROS architecture of Liquid Galaxy

By Jacob Minshall
December 18, 2015

ROS has become the pivotal piece of software we have written our new Liquid Galaxy platform on. We have also recently open sourced all of our ROS nodes on GitHub. While the system itself is not a robot per se, it does have many characteristics of modern robots, making the ROS platform so useful. Our system is made up of multiple computers and peripheral devices, all working together to bring view synced content to multiple displays at the same time. To do this we made use of ROS’s messaging platform, and distributed the work done on our system to many small ROS nodes.

Overview

Our systems are made up of usually 3 or more machines:

• Head node: Small computer that runs roscore, more of a director in the system.
• display-a: Usually controls the center three screens and a touchscreen + spacenav joystick.
• display-b: Controls four screens, two on either side of the middle three.
• display-$N: Controls more and more screens as needed, usually about four a piece.

Display-a and display-b are mostly identical in build. They mainly have a powerful graphics card and a PXE booted Ubuntu image. ROS has become our means to communicate between these machines to synchronize content across the …

ROS Platform Upgrades for Liquid Galaxy

By Dave Jenkins
December 14, 2015

For the last few months, End Point has been rolling out a new application framework along with updated display applications and monitoring infrastructure for the Liquid Galaxy Display Platform. These upgrades center on the ROS framework and allow a great number of functionality extensions and enhancements for the end user, as well as improvements to the stability and security of the core systems. It is intended that the 50+ systems that we currently maintain and support on behalf of our enterprise clients will be upgraded to this new platform.

ROS Overview

ROS is short for “Robot Operating System”. Just as it sounds, it is a framework used for controlling robots, and handles various environmental ‘inputs’ and ‘outputs’ well. End Point chose this framework in conjunction with related ongoing development projects on behalf of our enterprise clients. This system allows complex interactions from a touchscreen, camera, SpaceNav, or other device to be interpreted conditionally and then invoke other outputs such as displaying Google Earth, Street View, or other content on a given screen, speaker, or other output device. For more details, see: http://www.ros.org

Liquid Galaxy …

Testing Django Applications

By Zdeněk Maxa
December 14, 2015

This post summarizes some observations and guidelines originating from introducing the pytest unit testing framework into our CMS (Content Management System) component of the Liquid Galaxy. Our Django-based CMS allows users to define scenes, presentations and assets (StreetView, Earth tours, panos, etc) to be displayed on the Liquid Galaxy.

The purpose of this blog post is to capture my Django and testing study points, summarize useful resource links as well as to itemize some guidelines for implementing tests for newcomers to the project. It also provides a comparison between Python’s standard unittest library and the aforementioned pytest. Its focus is on Django database interaction.

Versions of software packages used

This post describes some of our experiences at End Point in designing and working on comprehensive QA/CI facilities for a new system which is closely related to the Liquid Galaxy.

The experiments were done on Ubuntu Linux 14.04:

• python (2.7.6) and its corresponding version of unittest
• django 1.7 (current recent is 1.9 but our CMS uses still 1.7 version)
• pytest-django 2.8.0
• pytest 2.7.2 (with py 1.4.30)
• virtualenv 13.1.2
• factory_boy 2.6.0

Testing Django …

Image Processing In The Cloud With Blitline and Wordpress

By Marina Lohova
December 1, 2015

Working with ImageMagick can be difficult. First, you have to get it installed on your OS (do you have Dev libs in place?), then you have to enable it in the language of your choice, then get it working in your application. After all that, do it all over again on the staging server where debugging may be complicated, and you may not have Admin rights. Meet Image Processing in the Cloud. Meet Blitline.

I’m doing a lot of things with Wordpress now, so we’ll set it up with Wordpress and PHP.

Step 1

Get a free developer account with Blitline, and note your application id.

Step 2

Get the Blitline PHP wrapper library Blitline_php. It’s clean and awesome, but unfortunately at the time of writing it was missing a few things, like being able to run your own Image Magick script and set a postback URL for when the job is finished. Yes, those are all useful features of Blitline cloud image processing! I’m still waiting on my pull request to be incorporated into the official version, so you can use mine that has these two useful features for now in my Blitline_php

Step 3

Now it’s time to integrate it in our application. Since it’s Wordpress, I’m doing it in the …

Odd pg_basebackup Connectivity Failures Over SSL

By Josh Williams
November 13, 2015

A client recently came to me with an ongoing mystery: A remote Postgres replica needed replaced, but repeatedly failed to run pg_basebackup. It would stop part way through every time, reporting something along the lines of:

pg_basebackup: could not read COPY data: SSL error: decryption failed or bad record mac

The first hunch we had was to turn off SSL renegotiation, as that isn’t supported in some OpenSSL versions. By default it renegotiates keys after 512MB of traffic, and setting ssl_renegotiation_limit to 0 in postgresql.conf disables it. That helped pg_basebackup get much further along, but they were still seeing the process bail out before completion.

The client’s Chef has a strange habit of removing my ssh key from the database master, so while that was being fixed I connected in and took a look at the replica. Two pg_basebackup runs later, a pattern started to emerge:

$ du -s 9.2/data.test*
67097452        9.2/data.test
67097428        9.2/data.test2

While also being a nearly identical size, those numbers are also suspiciously close to 64GB. I like round numbers, when a problem happens close to one that’s often a pretty good tell of some boundary or limit. On a hunch that …