Data Migration Tips
When you’re in the business of selling software to people, you tend to get a few chances to migrate data from their legacy software to your shiny new system. Most recently for me that has involved public health data exported from legacy disease surveillance systems into PostgreSQL databases for use by the open source EpiTrax system and its companion EMSA.
We have collected a few tips that may help you learn from our successes,
as well as our
mistakesparticularly educational experiences.
Your job is to satisfy your customers, and your customers want to know how the migration is progressing. Give them an answer, even if it’s just a generalization. This may be a burndown chart, a calculated percentage, a nifty graphic, or whatever, but something your project managers can show to their managers, to know more or less how far along things are.
Your job is also to know your system; that’s not the customer’s job. They shouldn’t have to get their data into a specific format for you to make use of it. Be as flexible as possible in the data format and structure you’ll accept. In theory, so long as your customer can provide the legacy …!-->
postgres data-processing database migration epitrax postgres
Implementing Basic HTTP Authentication in Rails
Nowadays it’s rather unusual to deploy HTTP Basic Authentication in a production web application. However, the need came up recently from a client. In a nutshell, due to integration requirements with a third party system, we had to provide a web app which expected credentials supplied via Basic HTTP Auth and validated against an external web service.
Luckily for us, like a great many other things, this is very easy to implement with Ruby on Rails.
Setting up a new Rails project
If you want to work along with me, and you like Docker and VS Code, take a look at this blog post to learn about the easiest way to set up an environment for development with Ruby on Rails in a container.
If not, you can follow the official docs for installing Ruby.
Once you have your environment with Ruby ready, we can go ahead and create a new Rails project to demonstrate how to set up Basic HTTP Auth.
Creating the new project
First, install the rails gem:
$ gem install rails
Then, make sure you are located in the directory where you want to create the new project and do:
$ rails new . --minimal -O
--minimalis a new option to
rails newadded in version 6.1 that disables a lot of default features …
ruby rails authentication
Bypassing a CDN to browse a website directly on your origin host
Using a content distribution network (CDN) has many advantages over serving a website directly, and for any reasonably large website, you should use one. Those advantages include:
- Caching at each of the CDN’s PoPs (points of presence).
- Often thousands of PoPs around the world, so traffic will be quick for everyone regardless of how far away they are from your origin server.
- Blocking of some Bad Guys automatically at the edge, including DDoS (distributed denial of service attacks) mitigation help.
- Origin IP address insulation. Hiding the origin IP address is useful to protect against DDoSes, since CDNs are generally well-defended against DDoS and your origin server probably is not as much.
You should generally be cautious about revealing your websites' origin IP address. We serve other sites from our origin directly, so we don’t worry too much about sharing it here.
Straight to the source
Sometimes, though, you need to bypass your CDN and test your website directly on its origin server. For example, if you need to test that your website would still work if the CDN goes down, or to sidestep CDN caching or content modification when troubleshooting a problem.
It can be …!-->
Using Devise for Authentication in Rails Without Database Stored Accounts
We can pretty much say that thanks to the venerable Devise gem, the authentication problem has been solved in Ruby on Rails. There are some instances however, when the requirements veer a little further away from convention and some customization needs to happen.
Such was the case of a recent project where we had to implement authentication and session management on a small web application that would serve as an API gateway into other system components. The interesting part was that there was no database to store accounts, and credentials would have to be validated against an external web service.
Luckily for us, the Devise gem is customizable enough to be able to fulfill this requirement via custom authentication strategies. With custom authentication strategies, one can implement completely bespoke authentication logic while still enjoying a lot of the features that Devise offers out of the box.
In this article, we’re going to walk through doing just that. Let’s get started.
To learn more about this capability of Devise, and how it relates to the underlying Warden concepts, here are some interesting sources:
ruby rails authentication
Developing Rails Apps in a Dev Container with VS Code
One of the great gifts from the advent of Docker and containers is the ability to get a good development environment up and running very quickly. Regardless of programming language or tech stack, there is probably an image in DockerHub or elsewhere that you can use to set up a container for development, either verbatim or as a basis for more complex setups.
Moreover, even if your development environment is complex, once you have containerized it, it’s easy to replicate for new team members.
To demonstrate that, we’re going to walk through setting up such an environment for developing Ruby on Rails applications.
Setting up a Ruby Dev Container
As I alluded to before, all we need is Docker, VS Code, and the extension. Once you have those installed, we can easily create a new Docker container ready for Ruby on Rails development and have VS Code connect to it, resulting in a fully featured development environment.
Creating the configuration file
To get started, create a new directory and open …!-->
ruby rails docker vscode containers
Updating Ruby on Rails
Updating your app to the latest versions of the framework it was built on, and dependencies it uses, is an important part of the development process. It may seem like a waste to invest time and money into it, but it can bring as much value as a new feature.
One good thing about using a framework like Ruby on Rails is that security features are baked in. This saves development time as the developer doesn’t have to re-create the wheel for logins, permissions, authentication, etc. There are many users of the framework who work together to can catch and patch vulnerabilities. Unfortunately, this means if your app hasn’t been updated its weaknesses become more obvious. A black hat attacker has easy access to a list of past Rails vulnerabilities.
Have you ever been to a website that hasn’t been updated for a while and found that everything moves slower than you’re used to? As technology improves and functions are optimized application processing time can be reduced. Most releases come with a performance update that can help your application keep up with the best of them.
The gems your application uses also come out with updates to add new features and …!-->
rails ruby update
Kubernetes environment variables, ConfigMaps and Secrets
Photo by Jeffry Johar
There are 3 ways to set environment variables for the container in the Kubernetes Pod: hard-coding, ConfigMaps, and Secrets, each with its own use case.
For those who are taking the Certified Kubernetes Administrator exam, you need to know all of these by heart. These skills fall under the domain of workloads and scheduling, which is 15% of the exam.
Let’s go over how to create environment variables based on these methods.
This is the method that enables us to define the environment variables in the
containers section of the Pod manifest. When using this method the environment variables will be visible when we describe the Pod. The following is an example of defining the environment variables
PET02=dog in an nginx container.
As with most Kubernetes resources, there are 2 ways of creating Pods. You can use either one of them. The first way is the imperative way, using the kubectl CLI. This is the preferred method for the CKA exam because it is convenient, fast, and saves time. The second way is the declarative way which requires you to build the YAML file and apply it.
kubectl run mynginx --image=nginx …
How to write end-to-end & component tests with Cypress in Vue.js
Is writing tests painful for you? In this tutorial, I explain how to handle UI testing with Cypress and hope to convince you that writing tests is not always so tedious and expensive, but can be fun instead.
As an example we are going to use a to-do app built using Vue. We will learn:
- How to install and set up Cypress.
- How to create a simple to-do app with Vue 3.
- How to write end-to-end tests.
- How to write component tests.
How to install and set up Cypress
First let’s create a new Vue project using the Vue CLI.
Install Vue CLI if you don’t have it in your machine:
npm install -g @vue/cli
Create a project (pick the
vue create todo-app
todo-appproject and install Cypress:
npm install cypress --save-dev
No dependencies, extra downloads, or changes to your code are required!
package.json. In the
scriptssection, add a command, …