Our Blog

MySQL to PostgreSQL Migration Tips

By David Christensen
November 26, 2014

I recently was involved in a project to migrate a client’s existing application from MySQL to PostgreSQL, and I wanted to record some of my experiences in doing so in the hopes they would be useful for others.

Note that these issues should not be considered exhaustive, but were taken from my notes of issues encountered and/or things that we had to take into consideration in this migration process.

Convert the schema

The first step is to convert the equivalent schema in your PostgreSQL system, generated from the original MySQL.

We used mysqldump --compatible=postgresql --no-data to get a dump which matched PostgreSQL’s quoting rules. This file still required some manual editing to cleanup some of the issues, such as removing MySQL’s “Engine” specification after a CREATE TABLE statement, but this resulted in a script in which we were able to create a skeleton PostgreSQL database with the correct database objects, names, types, etc.

Some of the considerations here include the database collations/charset. MySQL supports multiple collations/charset per database; in this case we ended up storing everything in UTF-8, which matched the encoding of the PostgreSQL database, so there were no …

Integrate Twilio in Django

By Kulbir Singh
November 24, 2014

Twilio

Twilio is a powerful HTTP API that allows you to build powerful voice and SMS apps. The goal of this blog post is to help make building the SMS applications as simple as possible in django.

There is a already Twilio Python help library available. The open source twilio-python library lets us to write Python code to make HTTP requests to the Twilio API.

Installation

The easiest way to install twilio-python library is using pip. Pip is a package manager for Python.

Simply run following command in terminal.

$ pip install twilio

Twilio API Credentails

To Integrate twilio API in django application, we need TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN variables. These variables can be found by logging into your Twilio account dashboard. These variables are used to communicate with the Twilio API.

You’ll need to add them to your settings.py file:

TWILIO_ACCOUNT_SID = 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
TWILIO_AUTH_TOKEN = 'YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY'

Create a New App

We are going to interact with people using SMS, so I prefer to create an app named communication. I am assuming you’ve already installed Django.

Run following command in terminal.

$ django-admin.py …

When Postgres will not start

By Greg Sabino Mullane
November 24, 2014

Photo by Vincent_AF

One of the more frightening things you can run across as a DBA (whether using Postgres or a lesser system) is a crash followed by a complete failure of the database to start back up. Here’s a quick rundown of the steps one could take when this occurs.

The first step is to look at why it is not starting up by examining the logs. Check your normal Postgres logs, but also check the filename passed to the –log argument for pg_ctl, as Postgres may not have even gotten far enough to start normal logging. Most of the time these errors are not serious, are fairly self-explanatory, and can be cured easily—​such as running out of disk space. When in doubt, search the web or ask in the #postgresql IRC channel and you will most likely find a solution.

Sometimes the error is more serious, or the solution is not so obvious. Consider this problem someone had in the #postgresql channel a while back:

LOG: database system was interrupted while in recovery at 2014-11-03 12:43:09 PST
HINT: This probably means that some data is corrupted and you will have to use the last backup for recovery.
LOG: database system was not properly shut down; automatic recovery in progress
LOG: …

MongoDB and OpenStack — OSI Days 2014, India

By Selvakumar Arumugam
November 19, 2014

The 11th edition of Open Source India, 2014 was held at Bengaluru, India. The two day conference was filled with three parallel tech talks and workshops which was spread across various Open Source technologies.

In-depth look at Architecting and Building solutions using MongoDB

Aveekshith Bushan & Ranga Sarvabhouman from MongoDB started off the session with a comparison of the hardware cost involved with storage systems in earlier and recent days. In earlier days, the cost of storage hardware was very expensive, so the approach was to filter the data to reduce the size before storing into the database. So we were able to generate results from filtered data and we didn’t have option to process the source data. After the storage became cheap, we can now store the raw data and then we do all our filter/processing and then distribute it.

Earlier,

        Filter -> Store -> Distribute

Present,

        Store -> Filter -> Distribute

Here we are storing huge amount of data, so we need a processing system to handle and analyse the data in efficient manner. In current world, the data is growing like anything and 3Vs are phenomenal of growing (Big)Data. We need to handle the …

Brazilian Portuguese Liquid Galaxy website launch!

By Marco Manchego
November 17, 2014

End Point Corporation is pleased to announce the official launch of its new Brazilian Portuguese Liquid Galaxy website! The site, found at liquidgalaxy.pt.endpoint.com (now at visionport.com) officially signals the arrival of End Point’s Liquid Galaxy to Brazil, and aims to provide service to all current and future customers in what is South America’s largest and most dynamic market.

With a population over 200 million, Brazil is also a quick adopter of new technologies with sizeable industry sectors that can benefit directly from the implementation of a Liquid Galaxy. This includes a massive commodities sector, booming real estate, tourism and a vibrant media market, all of which are strong candidates for the technology.

Brazil is also a logical entry-point into the larger South American market in general. We’re confident that as we increase market penetration in Brazil, other opportunities in the region will soon follow. Dave Jenkins, our VP of Sales and Marketing, offers the following: “We’re excited to see this expansion into Brazil. I always see great things coming out of São Paulo and Rio whenever I go there for tech conferences, which are always booked to overflowing levels.” …

Novo website do Liquid Galaxy em Português!

By Marco Manchego
November 17, 2014

End Point Corporation tem o prazer de anunciar o lançamento oficial do seu novo website em Português! O site, liquidgalaxy.pt.endpoint.com (visionport.com) oficialmente sinaliza a chegada do Liquid Galaxy da End Point ao Brasil e tem como objetivo fornecer serviço a todos os atuais e futuros clientes em um dos maiores e mais dinâmicos mercados da América do Sul.

Com uma população de mais 200 milhões, o Brasil também é um rápido adoptante de novas tecnologias com um numero considerável de líderes do setor que podem beneficiar diretamente a implementação do Liquid Galaxy. Isto inclui um setor de commodities solido, uma expansão imobiliária cresente, turismo e um mercado de mídia vibrante, todos fortes candidatos para a nova tecnologia.

Brasil também é o ponto de entrada para o mercado sul-americano em geral. Estamos confiantes de que podemos aumentar a penetração no mercado Brasileiro, outras oportunidades na região irão seguir. Dave Jenkins, nosso vice-presidente de vendas e Marketing, oferece o seguinte: “nós estamos excitados para ver essa expansão no Brasil. Eu sempre vejo grandes coisas saindo de São Paulo e Rio, sempre participo das conferências tecnologicas, que estão …

Create a sales functionality within Spree 2.3 using Spree fancy

By Bianca Rodrigues
November 14, 2014

Introduction

I recently started working with Spree and wanted to learn how to implement some basic features. I focused on one of the most common needs of any e-commerce business—​adding a sale functionality to products. To get a basic understanding of what was involved, I headed straight to the Spree Developer Guides. As I was going through the directions, I realized it was intended for the older Spree version 2.1. This led to me running into a few issues as I went through it using Spree’s latest version 2.3.4. I wanted to share with you what I learned, and some tips to avoid the same mistakes I made.

Set-up

I’ll assume you have the prerequisites it lists including Rails, Bundler, ImageMagick and the Spree gem. These are the versions I’m running on my Mac OS X:

• Ruby: 2.1.2p95
  • Rails: 4.1.4
  • Bundler: 1.5.3
  • ImageMagick: 6.8.9-1
  • Spree: 2.3.4

What is Bundler? Bundler provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed. You can read more about the benefits of using Bundler on their website. If you’re new to Ruby on Rails and/or Spree, you’ll quickly realize how useful Bundler is when updating your gems.

After …

Can we Server Name Indicate yet?

By Josh Williams
November 13, 2014

The encryption times, they are a-changin’.

Every once in a while I’ll take a look at the state of SNI, in the hopes that we’re finally ready for putting it to wide-scale use.

It started a few years back when IPv6 got a lot of attention, though in reality very few end user ISPs had IPv6 connectivity at that time. (And very few still do! But that’s another angry blog.) So, essentially, IPv4 was still the only option, and thus SNI was still important.

Then earlier this year when Microsoft dropped [public] support for Windows XP. Normally this is one of those things that would be pretty far off my radar, but Internet Explorer on XP is one of the few clients* that doesn’t support SNI. So at that time, with hope in my heart, I ran a search through the logs on a few of our more active servers, only to find that roughly 5% of the hits are MSIE on Windows XP. So much for that.

(* Android < 3.0 has the same problem, incidentally. But it in contrast constituted 0.2% of the hits. So I’m not as worried about the lack of support in that case.)

Now in fairly quick succession a couple other things have happened: SSLv3 is out, and SSL certificates with SHA-1 signatures are out. This has me …