Bucardo, and Coping with Unicode
By Josh Tolley
March 12, 2014
Given the recent DBD::Pg 3.0.0 release, with its improved Unicode support, it seemed like a good time to work on a Bucardo bug we’ve wanted fixed for a while. Although Bucardo will replicate Unicode data without a problem, it runs into difficulties when table or column in the database include non-ASCII characters. Teaching Bucardo to handle Unicode data has been an interesting exercise.
Without information about its encoding, string data at its heart is meaningless. Programs that exchange string information without paying attention to the encoding end up with problems exactly like that described in the bug, with nonsense characters all over. Further, it’s impossible even to compare two different strings reliably. So not only would Bucardo’s logs and program output contain junk data, Bucardo would simply fail to find database objects that clearly existed, because it would end up querying for the wrong object name, or the keys of the hashes it uses internally would be meaningless. Even communication between different Bucardo processes needs to be decoded correctly. The recent DBD::Pg 3.0.0 release takes care of decoding strings sent from PostgreSQL, but other inputs, such as …
bucardo perl postgres replication unicode
Provisioning a Development Environment with Packer, Part 1
By Mike Farmer
March 12, 2014
I recently needed to reconstruct an old development environment for a project I worked on over a year ago. The codebase had aged a little and I needed old versions of just about everything from the OS and database to Ruby and Rails. My preferred method for creating a development environment is to setup a small virtual machine (VM) that mimics the production environment as closely as possible.
Introducing Packer
I have been hearing a lot of buzz lately about Packer and wanted to give it a shot for setting up my environment. Packer is a small command line tool written in the increasingly popular Go programming language. It serves three primary purposes:
- Building a machine based on a set of configuration parameters
- Running a provisioner to setup the machine with a desired set of software and settings
- Performing any post processing instructions on that machine
Packer is really simple to install and I would refer you to their great documentation to get it set up. Once set up, you will have the packer command at your disposal. To build a new machine, all you need to is call:
packer build my_machine.json
The file my_machine.json can be the name of any json file and contains all the …
devops environment tools
Implementing Background Fetch in iOS 7
By Kamil Ciemniewski
March 11, 2014
With the iOS7 being out and gaining market share, great features it introduced are becoming available to more and more users.
One such new feature is a set of so-called “background modes”.
States the application can be in, in iOS
To explain this new set of modes, let me give you a really quick intro to what modes are.
In iOS, at a given point in time, an app can be in one of the following states:
Not running
There is no process for the app in the system.
Inactive
The app is running in the foreground but currently is not receiving any events. (It may be executing other code though.) An app usually stays in this state only briefly as it transitions to a different state.
Active
The application is running and is receiving user input. The main user interface is visible on the display.
Background
The application is running. It’s not receiving user input. Its code is being executed but it will be switched to the suspended state very soon by the system.
Suspended
The app remains in memory, but it’s not being executed. It remains dormant until a user chooses to activate it again or a system switches it back to a background state to allow it to process certain kinds of data.
Background modes …
ios
Interchange table hacking
By Jeff Boes
March 11, 2014
Interchange has a powerful but terribly obscure table administration tool called the Table Editor. You can create, update, and delete rows, and even upload whole spreadsheets of data, but the Table Editor isn’t the most flexible thing in the world, so sometimes it just flat-out refuses to do what you want.
So you trick it.
A client wanted to upload data to a table that had a single-column primary key (serial), but also had a unique three-column key that was only used in the upload process (because the uploaded data was intended to replace rows with identical three-column combinations). Example:
In the table:
code: 243
field1: AAA
field2: BBB
field3: CCC
data-fields: ...
In the spreadsheet:
field1 field2 field3 data-fields...
AAA BBB CCC ...
In the database definition for this table, I had to add a secondary key definition for Interchange’s use:
Database my_table COMPOSITE_KEY field1 field2 field3
in addition to the original key:
Database my_table KEY code
Here’s the problem this presents: when you add a COMPOSITE_KEY to a table, the table editor refuses to show per-row checkboxes that allow you to delete rows. I thought I might have to write a custom admin …
interchange
md+lvm expansion from RAID 1 to RAID 5
By Josh Williams
March 9, 2014
VHS is on the way out, or so they tell me. A little while back I unearthed the family’s collection of old tape recordings, and have been digitizing everything in an effort to preserve all the old youth sports games and embarassing birthday parties. There’s no way I’m going to let my brother forget those. It’s a lot of video, and that takes up quite a bit of space. Between that, HD videos recorded more recently, Postgres test databases and source datasets, server backups, and so on, the 3TB of space on my local file server was quickly running out.
I know, right? My first hard drive was 40MB, in two 20MB partitions. And we were glad for that space!
Back in the present, now it was time to add another hard drive. This (otherwise) tiny rooted plug server contained two 3TB USB hard drives in a RAID-1 configuration through the Linux md module. On top of that is lvm. lvm itself could have been used for the RAID-1, but has the disadvantage of not being able to optimize multiple reads, whereas md can allow both disks in the mirror serve different reads at the same time.
I bought a third disk of the same model, so it could be added in to a RAID-5 configuration. These being big USB drives, …
linux sysadmin
Amazon Payments — Caveat Developer
By Carl Bailey
March 6, 2014
A client of ours needed me to install Amazon Payments for them. Now there are several shopping carts for which Amazon Payments can be installed as an option, and I assume they work just fine. This client was not so lucky, and I had to roll my own.
Amazon starts with a JavaScript widget that asks you to log in or create an Amazon account. The widget returns an order reference ID:
<div id="payWithAmazonDiv" style="padding-top: 1.2em;">
<br />
<img src="https://payments.amazon.com/gp/widgets/button?sellerId=[Amazonseller id]&size=large&color=orange"
style="cursor: pointer;"/>
</div>
<script type="text/javascript">
var amazonOrderReferenceId;
new OffAmazonPayments.Widgets.Button ({
sellerId: '[Amazon seller id]',
useAmazonAddressBook: true,
onSignIn: function(orderReference) {
amazonOrderReferenceId = orderReference.getAmazonOrderReferenceId();
window.location = 'https://www.yoursite.com/nextpage.html?session=' +
amazonOrderReferenceId;
},
onError: function(error) {
// your error handling code
alert('Amazon …ecommerce payments
Ansiblizing SSH Keys
By Cas Rusnov
March 3, 2014
It is occasionally the case that several users share a particular account on a few boxes, such as in a scenario where a test server and a production server share a deployment account, and several developers work on them. In these situations the preference is to authenticate the users with their ssh keys through authorized_keys on the account they are sharing, which leads to the problem of keeping the keys synchronized when they are updated and changed. We add the additional parameter that perhaps any given box will have a few users of the account that aren’t shared by the others, but otherwise allow a core of developers to access them. Now extend this scenario across hundreds of machines, and the maintenance becomes difficult or impossible when updating any of the core accounts. Obviously this is a job for a remote management framework like Ansible.
Our Example Scenario
We have developers Alice, Bob and Carla which need access to every box. We have additional developers Dwayne and Edward that only need access to one box each. We have a collection of servers: dev1, dev2, staging and prod. All of the servers have an account called web_deployment.
The authorized_keys for …
ansible ssh sysadmin
JavaScript Namespacing with the Rails Asset Pipeline
By Steph Skardal
March 3, 2014
Since the release of Rails 3 [a while back], I’ve had a lot of use with the asset pipeline. There can be minor headaches associated with it, but ultimately, the process of combining, minifying, and serving a single gzipped JavaScript and CSS file is a great gain in terms of reducing requests to speed up your web application. It’s a behavior that I’ve wanted to emulate in other platforms that I’ve used (including Interchange).
One headache that might come up from the asset pipeline is that JavaScript functions from various parts of the application might have the same name, and may override existing functions of the same name. The last defined function will be the one that executes. I’ve come up with a common pattern to avoid this headache, described below:
Set the Body ID
First, I set the body tag in my application layout file to be related to the controller and action parameters. Here’s what it looks like:
<body id="<%= "#{params[:controller].gsub(/\//, '_')}_#{params[:action]}" %>">
...
</body>
If I was less lazy, I could create a helper method to spit out the id.
Shared JS
I create a shared.js file, which contains JavaScript shared across …
javascript ruby rails