Changing Passenger Nginx Timeouts
It may frighten you to know that there are applications which take longer than Passenger’s default timeout of 10 minutes. Well, it’s true. And yes, those application owners know they have bigger fish to fry. But when a customer needs that report run today being able to lengthen a timeout is a welcomed stopgap.
Tracing the timeout
There are many different layers at which a timeout can occur, although these may not be immediately obvious to your users. Typically they receive a 504 and an ugly “Gateway Time-out” message from Nginx. Review the Nginx error logs both at the reverse proxy and application server, you might see a message like this:
upstream timed out (110: Connection timed out) while reading response header from upstream
If you’re seeing this message on the reverse proxy, the solution is fairly straight forward. Update the proxy_read_timeout setting in your nginx.conf and restart. However, it’s more likely you’ve already tried that and found it ineffective. If you expand your reading of the Nginx error you might notice another clue.
upstream timed out (110: Connection timed out) while reading response header from upstream,
upstream: …
hosting rails
RHEL 6 glibc IPv6 DNS resolution bug
We ran into an unpleasant bug in a Red Hat Enterprise Linux 6 glibc update a couple of weeks ago. It since has made its way into CentOS 6 as well.
The problem manifested itself in our particular case with this error from the Postfix mailer:
Jun 29 01:55:23 sl37 kernel: smtp[7093]: segfault at 1 ip 00007ffc0e455596 sp 00007fff99948f60 error 6 in libresolv-2.12.so[7ffc0e449000+16000]
But it affects all DNS resolution on the host, not just for mail.
If you have any IPv6 resolvers at all listed in /etc/resolv.conf, all your DNS resolution is likely to be broken with this version of glibc:
glibc-2.12-1.80.el6.x86_64
To work around the problem, you can either:
- Use only IPv4 DNS resolvers (comment out the IPv6 resolvers for now)
- or downgrade to the previous version of glibc using yum downgrade
Red Hat is aware of the bug and you can track progress toward a resolution in Bugzilla bug #835090.
If you’re using IPv6, watch out for this! If not, you’re fine.
ipv6 linux redhat sysadmin
Company Presentation: Ember, Backbone and Friends
We had a “virtual” company meeting today using Big Blue Button with Free Conference Call HD for audio. I gave an overview on the upcoming crop of JavaScript MVC Frameworks and how they attempt to mitigate some of the issues faced by JavaScript-heavy web applications. Later on, I spoke more specifically about Backbone.js and Ember and demonstrated an example Todo List application from Addy Osmani’s TodoMVC project. If you are interested in evaluating and learning more about client-side frameworks, TodoMVC is definitely worth a look.
Slides from the talk
I used Google Presentations (now Google Slides) for this as I have for several other talks lately and have found it really great to use and collaborate on presentations with others. Check out the slide deck from my talk below:
Q&A
During the Q&A time afterward we discussed LocalStorage a little bit as it’s used for the storage layer for each of the TodoMVC todo list applications. We covered some of the options which allow client-side MVC frameworks to sync data between the client and RESTful server-side web services.
We also covered how much the Chrome Developer Tools have improved recently and I encouraged my colleagues to …
javascript
Independence Day … in Belarus
This month of July I am working from Belarus, my home country.
On July 3 we celebrated Independence Day of Belarus here, quite in sync with the Independence Day in United States.
Belarus was occupied by German troops for 3 years since 1941 during World War II, and on July 3, 1944 the Soviet army managed to release the capital of Belarus—Minsk.
Independence Day celebration starts with the military parade including tanks, jets, helicopters, and other bulky equipment.
The rehearsal of the parade is usually carried out a day before the parade in my neighborhood. Sometimes it is pretty unnerving to hear all the noise tanks and jets make on the streets and in the air.
The celebration continues with outdoor concerts and traditional Belorussian folk songs performances in many beautiful parks of the city.
It finishes with the fireworks display near “Stella”—a monument built to memorialize the fortieth anniversary of the victory in the Great Patriotic War. Great Patriotic War is how we call a considerable part of World War II that happened on the territory of the former Soviet Union.
Though not as famous as Macy’s fireworks, the “Stella” fireworks display was very cool this year. I also …
travel remote-work
Code School: Journey into Mobile Review
Yesterday, I took the Journey into Mobile course over at Code School, an online training program with a handful of web technology courses. I just started a large mobile project for Paper Source, so this was good timing. Because a paid membership to Code School is required to participating in the training, I won’t share too many of the course details here. But I’ll share a few tidbits that will hopefully interest you in taking the course or learning more about mobile development.
The course was divided into 5 high level lessons (or levels):
- Relative Font Size
- Fluid Layouts
- Adaptive Design
- Responsiveness Adventures
- Responsive Media
The first two topics covered working with proportional font sizes and content regions, and how to convert your existing layout to proportions (percentage or ems) to create a fluid layout which included proportional font sizes. In the Adaptive Design lesson, the CSS3 supported @media query was introduced. I’ve used the media query on the responsive design for The Best Game Apps and will be using it for Paper Source. Some examples of @media queries include:
@media (min-width: 655px) and (max-width: 1006px) {
# styles specific to browser width 655-1006 …
browsers mobile
Postgres log_statement='all' should be your default
Setting the PostgreSQL log_statement parameter to ‘all’ is always your best choice; this article will explain why. PostgreSQL does not have many knobs to control logging. The main one is log_statement, which can be set to ’none’ (do not ever set it to this!), ‘ddl’ or ‘mod’ (decent but flawed values), or ‘all’, which is what you should be using. In addition, you probably want to set log_connections = on, log_disconnections = on, and log_duration = on. Of course, if you do set all of those, don’t forget to set log_min_duration_statement = -1 to turn it off completely, as it is no longer needed.
The common objections to setting log_statement to ‘all’ can be summed up as Disk Space, Performance, and Noise. Each will be explained and countered below. The very good reasons for having it set to ‘all’ will be covered as well: Troubleshooting, Analytics, and Forensics/Auditing.
Objection: Disk Space
The most common objection to logging all of your SQL statements is disk space. When log_statement is set to all, every action against the database …
database postgres
Respecting API Call Limit with Radian6
A common practice in the online API world is to enforce the call limit. Twitter allows 150 API calls per hour. Shopify has 500 API calls per 5 minutes limit. You may learn how to work with Shopify call limit from this great article.
One of our projects was built around interaction with Radian6 platform. Radian6 is a new and growing data mining service with the default limit of 100 calls per hour. I will describe our take on the call limit implementation.
Introducing the call counter
First, we need to know how many calls have been executed in the current hour. Every external call increments the counter field on a special model until the counter reaches the limit. The counter is reset back to zero at the beginning of every hour.
script/rails g model RadianCallCount
In the migration:
class CreateRadianCallCounts < ActiveRecord::Migration
def change
create_table :radian_call_counts do |t|
t.integer :count
t.timestamps
end
end
end
In db/seeds.rb file
puts "Initializing Radian6 counter"
RadianCallCount.delete_all
RadianCallCount.create(:count => 0)
Let’s roll the counter!
rake db:migrate
rake db:seed
Scheduling the counter reset
It is necessary to …
rails social-networks api
Three Things: ImageMagick, RequestBin, Responsivness
It’s been a while since I wrote a Three Things blog article, where I share tidbits that I’ve learned that don’t quite make it into a full blog post, but are still worth sharing!
Image Stacking via Command Line
I recently needed to stack several images to CSS spritify them. I did some quick searching and found the following ImageMagick command line functionality:
convert add.png add_ro.png -append test.png
convert add.png add_ro.png +append test.png
The -append argument will stack the images vertically, and +append will stack the images horizontally.
RequestBin
Recently, I needed to quickly examine the contents sent during a payment transaction. Richard pointed me to RequestBin a neat and quick tool for examining the contents of an external request. I was testing the Elavon payment gateway integrated in an application using Piggybak. In my code, I made the following change:
class ElavonGateway < ViaklixGateway
self.test_url = 'https://demo.myvirtualmerchant.com/VirtualMerchantDemo/process.do'
self.live_url = 'https://www.myvirtualmerchant.com/VirtualMerchant/process.do'
+ self.test_url = 'http://requestb.in/abc1def2'
I ran through a single checkout, …
browsers mobile