Postgres concurrent indexes and the curse of IIT
By Greg Sabino Mullane
April 28, 2016
Postgres has a wonderful feature called concurrent indexes. It allows you to create indexes on a table without blocking reads OR writes, which is quite a handy trick. There are a number of circumstances in which one might want to use concurrent indexes, the most common one being not blocking writes to production tables. There are a few other use cases as well, including:
- Replacing a corrupted index
- Replacing a bloated index
- Replacing an existing index (e.g. better column list)
- Changing index parameters
- Restoring a production dump as quickly as possible
In this article, I will focus on that last use case, restoring a database as quickly as possible. We recently upgraded a client from a very old version of Postgres to the current version (9.5 as of this writing). The fact that use of pg_upgrade was not available should give you a clue as to just how old the “very old” version was!
Our strategy was to create a new 9.5 cluster, get it optimized for bulk loading, import the globals and schema, stop write connections to the old database, transfer the data from old to new, and bring the new one up for reading and writing.
The goal was to reduce the …
postgres
Cybergenetics Helps Free Innocent Man
By Elizabeth Garrett Christensen
April 27, 2016
We all love a good ending. I was happy to hear that one of End Point’s clients, Cybergenetics, was involved in a case this week to free a falsely imprisoned man, Darryl Pinkins.
Darryl was convicted of a crime in Indiana in 1991. In 1995 Pinkins sought the help of the Innocence Project. His attorney Frances Watson and her students turned to Cybergenetics and their DNA interpretation technology called TrueAllele® Casework. The TrueAllele DNA identification results exonerated Pinkins. The Indiana Court of Appeals dropped all charges against Pinkins earlier this week and he walked out of jail a free man after fighting for 24 years to clear his name.
TrueAllele can separate out the people who contributed their DNA to a mixed DNA evidence sample. It then compares the separated out DNA identification information to other reference or evidence samples to see if there is a DNA match.
End Point has worked with Cybergenetics since 2003 and consults with them on security, database infrastructure, and website hosting. We congratulate Cybergenetics on their success in being part of the happy ending for Darryl Pinkins and his family!
More of the story is available at Cybergenetics’ Newsroom or …
community clients
We are bigger than VR gear - Liquid Galaxy
By
Yaqi Chen
April 27, 2016
Nowadays, virtual reality is one of the hottest topics in tech, with VR enabling users to enter immersive environments built up by computer technology. I attended Mobile World Congress 2016 a few weeks ago, and it was interesting to see people sit next to one another and totally ignore one another while they were individually immersed in their own virtual reality worlds.
When everyone is so addicted to their little magic boxes, they tend to lose their connections with people around them. End Point has developed a new experience in which users can watch and share their virtually immersive world together. This experience is called the Liquid Galaxy.
When a user stands in front of Liquid Galaxy and is surrounded by a multitude of huge screens arranged in a semicircle, he puts not only his eyes but his whole body into an unprecedented 3D space. These screens are big enough to cover the audience’s entire peripheral vision and bring great visual stimulation from all directions. When using the Liquid Galaxy system, the users become fully immersed in the system and the imagery they view.
Movie Night at End Point
This digital chamber can be considered a sort of VR movie theater, where …
visionport virtualization
Liquid Galaxy for Real Estate
By Ben Witten
April 22, 2016
The Liquid Galaxy, an immersive and panoramic presentation tool, is the perfect fit for any time you want to grab the attention of your audience and leave a lasting impression. The system has applications in a variety of industries (which include museums and aquariums, hospitality and travel, research libraries at universities, events, and real estate, to name a few) but no industry’s demand rivals the popularity seen in real estate.
The Liquid Galaxy provides an excellent tool for real estate brokerages and land use agencies to showcase their properties with multiple large screens showing 3D building models and complete Google Earth data. End Point can configure the Liquid Galaxy to highlight specific buildings, areas on the map, or any set of correlated land use data, which can then be shown in a dazzling display that forms the centerpiece of a conference room or lobby. We can program the Liquid Galaxy to show floor plans, panoramic interior photos, and even Google Street View “walking tours” around a given property.
A Liquid Galaxy in your office will provide your firm with a sophisticated and cutting edge sales tool. You will depart from the traditional ways of viewing, …
visionport
Client web browser logging
By Piotr Hankiewicz
April 21, 2016
Introduction
The current state of development for web browsers is still problematic. We have multiple browsers, each browser has plenty of versions. There are multiple operating systems and devices that can be used. All of this makes it impossible to be sure that our code will work on every possible browser and system (unfortunately). With proper testing, we can make our product stable and good enough for production, but we can’t expect that everything will go smoothly, well, it won’t. He is always somewhere, a guy sitting in his small office and using outdated software, Internet Explorer 6 for example. Usually you want to try to support as many as possible users, here, I will explain how to help find them. Then you just need to decide if it is worth fixing an issue for them.
Browser errors logging
What can really help us and is really simple to do is browser error logging. Every time an error occurs on the client side (browser will generate an error that the user most likely won’t see), we can log this error on the server side, even with a stack trace. Let’s see an example:
window.onerror = function (errorMsg, url, lineNumber, column, errorObj) {
$.post( …angular javascript
How to Build a Skyscraper
By Phineas Jensen
April 19, 2016
Another talk from MountainWest RubyConf that I enjoyed was How to Build a Skyscraper by Ernie Miller. This talk was less technical and instead focused on teaching principles and ideas for software development by examining some of the history of skyscrapers.
Equitable Life Building
Constructed from 1868 to 1870 and considered by some to be the first skyscraper, the Equitable Life Building was, at 130 feet, the tallest building in the world at the time. An interesting problem arose when designing it: it was too tall for stairs. If a lawyer’s office was on the seventh floor of the building, he wouldn’t want his clients to walk up six flights of stairs to meet with him.
Elevators and hoisting systems existed at the time, but they had one fatal flaw: there were no safety systems if the rope broke or was cut. While working on converting a sawmill to a bed frame factory, a man named Elisha Otis had the idea for a system to stop an elevator if its rope is cut. He and his sons designed the system and implemented it at the factory. At the time, he didn’t think much of the design, and didn’t patent it or try to sell it.
Otis’ invention became popular when he showcased it at the 1854 New York …
conference
Tuples in C#
By Dylan Wooters
April 19, 2016
Dynamic and functional languages have greatly influenced C# as the language has matured. The introduction of the ‘var’ and ‘dynamic’ keywords allowed C# developers to write code that resembles a dynamic language like Python. The fluent method chaining of one of C# greatest features, LINQ, came from functional programming concepts.
The tuple class is a good example of a recent C# feature that combines aspects of dynamic and functional paradigms. The tuple is a common built-in feature in Python, but most C# developers are probably unaware of its benefits and usage. Tuples aren’t a huge deal, but they come in handy in several modern programming scenarios.
Returning Multiple Values
Say we need to call two APIs to create a music playlist and publish it to a social media site. We also want to wrap this functionality in a single method and return a boolean for whether or not the call was successful, as well as a specific, user-friendly error message. A tuple is helpful in this scenario.
public static Tuple<bool, string> PublishPlaylist(string[] songIds)
{
var musicService = new MusicService();
var socialMediaService = new SocialMediaService();
string playlistId; …development csharp
Learning from data basics II: simple Bayesian Networks
By Kamil Ciemniewski
April 12, 2016
In my last article I presented an approach that simplifies computations of very complex probability models. It makes these complex models viable by shrinking the amount of needed memory and improving the speed of computing probabilities. The approach we were exploring is called the Naive Bayes model.
The context was the e-commerce feature in which a user is presented with the promotion box. The box shows the product category the user is most likely to buy.
Though the results we got were quite good, I promised to present an approach that gives much better ones. While the Naive Bayes approach may not be acceptable in some scenarios due to the gap between approximated and real values, the approach presented in this article will make this distance much, much smaller.
Naive Bayes as a simple Bayesian Network
When exploring the Naive Bayes model, we said that there is a probabilistic assumption the model makes in order to simplify the computations. In the last article I wrote:
The Naive Bayes assumption says that the distribution factorizes the way we did it only if the features are conditionally independent given the category.
Expressing variable dependencies as a graph
Let’s imagine …
machine-learning optimization probability ruby