md+lvm expansion from RAID 1 to RAID 5
By Josh Williams
March 9, 2014
VHS is on the way out, or so they tell me. A little while back I unearthed the family’s collection of old tape recordings, and have been digitizing everything in an effort to preserve all the old youth sports games and embarassing birthday parties. There’s no way I’m going to let my brother forget those. It’s a lot of video, and that takes up quite a bit of space. Between that, HD videos recorded more recently, Postgres test databases and source datasets, server backups, and so on, the 3TB of space on my local file server was quickly running out.
I know, right? My first hard drive was 40MB, in two 20MB partitions. And we were glad for that space!
Back in the present, now it was time to add another hard drive. This (otherwise) tiny rooted plug server contained two 3TB USB hard drives in a RAID-1 configuration through the Linux md module. On top of that is lvm. lvm itself could have been used for the RAID-1, but has the disadvantage of not being able to optimize multiple reads, whereas md can allow both disks in the mirror serve different reads at the same time.
I bought a third disk of the same model, so it could be added in to a RAID-5 configuration. These being big USB drives, …
linux sysadmin
Amazon Payments — Caveat Developer
By Carl Bailey
March 6, 2014
A client of ours needed me to install Amazon Payments for them. Now there are several shopping carts for which Amazon Payments can be installed as an option, and I assume they work just fine. This client was not so lucky, and I had to roll my own.
Amazon starts with a JavaScript widget that asks you to log in or create an Amazon account. The widget returns an order reference ID:
<div id="payWithAmazonDiv" style="padding-top: 1.2em;">
<br />
<img src="https://payments.amazon.com/gp/widgets/button?sellerId=[Amazonseller id]&size=large&color=orange"
style="cursor: pointer;"/>
</div>
<script type="text/javascript">
var amazonOrderReferenceId;
new OffAmazonPayments.Widgets.Button ({
sellerId: '[Amazon seller id]',
useAmazonAddressBook: true,
onSignIn: function(orderReference) {
amazonOrderReferenceId = orderReference.getAmazonOrderReferenceId();
window.location = 'https://www.yoursite.com/nextpage.html?session=' +
amazonOrderReferenceId;
},
onError: function(error) {
// your error handling code
alert('Amazon …ecommerce payments
Ansiblizing SSH Keys
By Cas Rusnov
March 3, 2014
It is occasionally the case that several users share a particular account on a few boxes, such as in a scenario where a test server and a production server share a deployment account, and several developers work on them. In these situations the preference is to authenticate the users with their ssh keys through authorized_keys on the account they are sharing, which leads to the problem of keeping the keys synchronized when they are updated and changed. We add the additional parameter that perhaps any given box will have a few users of the account that aren’t shared by the others, but otherwise allow a core of developers to access them. Now extend this scenario across hundreds of machines, and the maintenance becomes difficult or impossible when updating any of the core accounts. Obviously this is a job for a remote management framework like Ansible.
Our Example Scenario
We have developers Alice, Bob and Carla which need access to every box. We have additional developers Dwayne and Edward that only need access to one box each. We have a collection of servers: dev1, dev2, staging and prod. All of the servers have an account called web_deployment.
The authorized_keys for …
ansible ssh sysadmin
JavaScript Namespacing with the Rails Asset Pipeline
By Steph Skardal
March 3, 2014
Since the release of Rails 3 [a while back], I’ve had a lot of use with the asset pipeline. There can be minor headaches associated with it, but ultimately, the process of combining, minifying, and serving a single gzipped JavaScript and CSS file is a great gain in terms of reducing requests to speed up your web application. It’s a behavior that I’ve wanted to emulate in other platforms that I’ve used (including Interchange).
One headache that might come up from the asset pipeline is that JavaScript functions from various parts of the application might have the same name, and may override existing functions of the same name. The last defined function will be the one that executes. I’ve come up with a common pattern to avoid this headache, described below:
Set the Body ID
First, I set the body tag in my application layout file to be related to the controller and action parameters. Here’s what it looks like:
<body id="<%= "#{params[:controller].gsub(/\//, '_')}_#{params[:action]}" %>">
...
</body>
If I was less lazy, I could create a helper method to spit out the id.
Shared JS
I create a shared.js file, which contains JavaScript shared across …
javascript ruby rails
Liquid Galaxy at the Economist World Ocean Summit
By Dave Jenkins
March 3, 2014
 |
| The Liquid Galaxy with image capture equipment |
End Point had the distinct privilege of providing a Liquid Galaxy to the Economist World Ocean Summit held in Half Moon Bay, California last week. Originally conceived as a Google Earth and Google Street View platform, the Liquid Galaxy is rapidly becoming a premiere display for ocean content as well. Google is partnering with NGOs and corporations that are actively working toward ocean sustainability. Together, these organizations capture and categorize a great deal of ocean content, with most of that content publicly available via Street View. Some of this equipment was on display next to the Liquid Galaxy: a Trekker camera for remote areas and an underwater camera provided by the Catlin Seaview Survey for capturing hi-definition panoramas underwater.
This deployment used a custom configuration with six large HDTVs arrayed in a curved 3x2 panorama. The Liquid Galaxy is able to show this content as a single large surface with geometrically-adjusted angles. In short, it’s like a bay window on a submarine. Viewers are able to navigate to different locations through a touchscreen interface, and then rotate and zoom in on detailed …
google-earth visionport event
DBD::Pg prepared statement change
By Greg Sabino Mullane
February 28, 2014
One of the changes in the recently released DBD::Pg version 3 (in addition to the big utf8 change), is the addition of a new attribute, pg_switch_prepared. This accompanies a behavior change in the use of prepare/execute. DBD::Pg will now postpone creating a server-side PREPARE statement until the second time a query is run via the execute() method.
Technically, DBD::Pg will use PQexecParams (part of the underlying libpq system that DBD::Pg uses) the first time a statement is executed, and switch to using PQexecPrepared the second time the statement is executed (by calling PQprepare first). When it actually switches is controlled by the pg_switch_prepared attribute, which defaults to 2 (the behavior above). You can set it to 0 or 1 to always use PQexecPrepared (as the older versions did), or you can set it to -1 to always use PQexecParams and avoid creating prepared statements entirely.
The typical flow of events in a DBI script is to create a statement handle via the prepare() method, then call the execute() time with varying arguments as many times as needed.
#!perl
use strict;
use warnings;
use DBI;
my $DSN = 'DBI:Pg:dbname=postgres';
my $dbh = DBI->connect($DSN, …dbdpg perl postgres
Monitoring Productivity with RescueTime
By Mike Farmer
February 26, 2014
I’m always on the lookout for tools that help me be more productive in my daily work. Time tracking is always one of those difficult tasks for those who bill time by the hour. I’ll admit that there are days here and there where I know I’ve worked all day but at the end of the day I can’t remember where I spent my time and how long I’ve spent on different tasks. Last week I discovered a new tool that I have loved! It’s called RescueTime and it has changed how I approach my days.
RescueTime is a web service that you can sign up for free. You download a little app that runs in the background and monitors your productivity throughout the day. You can then log into the RescueTime website and see many different breakdowns of how you spent your day.
So how does it track productivity? It monitors how much time you spend in different applications on your computer. Then it tries to tag that time on a scale from distracting to very productive. You might be thinking that if you spend your whole day in a browser that wouldn’t be very helpful. But RescueTime is pretty smart and it also monitors which web sites you visit and lets you categorize that time as well.
The nicest thing about …
tips tools
MySQL, ASCII Null, and Data Migration
By Mark Johnson
February 26, 2014
Data migrations always have a wide range of challenges. I recently took on a request to determine the difficulty of converting an ecommerce shop’s MySQL 5.0 database to PostgreSQL 9.3, with the first (presumably “easier”) step being just getting the schema converted and data imported before tackling the more challenging aspect of doing a full assessment of the site’s query base to re-write the large number of custom queries that leverage MySQL-specific language elements into their PostgreSQL counterparts.
During the course of this first part, which had contained a number of difficulties I had anticipated, I hit one that I definitely had not anticipated:
ERROR: value too long for type character varying(20)
Surely, the error message is absolutely clear, but how could this possibly be? The obvious answer—that the varchar definitions were different lengths between MySQL and PostgreSQL—was sadly quite wrong (which you knew, or I wouldn’t have written this).
After isolating out the row in question, the first clear distinction of the data in question was the presence of the ASCII null character in it:
'07989409006\007989409'
OK, sure, that’s weird (and I have a hunch why it’s …
database mysql postgres