Perl Web Frameworks

2022-04-19T00:00:00+00:00

CGI

When I started programming, back in the day, CGI (the Common Gateway Interface) was still widely used. Usually the Apache webserver would just execute a script or a binary with some environment variables set and serve whatever the executable sent to the standard output, while keeping the standard error in the logs.

This simple and straightforward mechanism can still be used for small programs, but larger applications usually want to save the start-up time and live longer than just a single request.

At that time Perl was used far more often than now, and it had (and still has) the CGI.pm module to help the programmer to get the job done.

#!/usr/bin/env perl

use utf8;
use strict;
use warnings;
use CGI;

my $q = CGI->new;
print $q->header;
my $name = 'Marco';
print $q->p("Hello $name");
print "\n";

And it will output:

./cgi.pl
Content-Type: text/html; charset=ISO-8859-1

<p>Hello Marco</p>

Here the script mixes logic and formatting and the encoding it produces by default tells us that this comes from another age. But if you want something which is seldom used and gets executed on demand without persisting in the machine’s memory, this is still an option.

Please note that there are frameworks which can work in CGI mode, so there is no reason to use CGI.pm, beside having to maintain legacy programs.

Mojolicious

Fast-forward to 2022.

Nowadays Perl is just another language among dozens of them. But it still gets the job done and lets you write nice, maintainable code like any other modern language.

Mojolicious is currently the top choice if you want to do web development in Perl. It is an amazing framework, with a large and active community, and appears to have collected the best concepts that other web frameworks from other languages have to offer.

Let’s hack an app in a couple of minutes in a single file, like during the CGI days:

#!/usr/bin/env perl
use utf8;
use strict;
use warnings;

use Mojolicious::Lite;

get '/' => sub {
    my ($c) = @_;
    $c->stash(name => "Marco");
    $c->render(template => 'index');
};

app->start;

__DATA__
@@ index.html.ep
Hello <%= $name %>

Here the structure is a bit different.

First, there’s a Domain Specific Language (DSL) to give you some sugar. This is the “Lite” version, while in a well-structured Mojolicious app one prefers to write class methods. We declare that the root (/) URL path of the application is going to execute some code. It populates the “stash” with some variables, and finally renders a template which can access the stashed variables. If you execute the script, you get:

./mojo.pl cgi 2> /dev/null
Status: 200 OK
Content-Length: 12
Date: Fri, 08 Apr 2022 12:33:52 GMT
Content-Type: text/html;charset=UTF-8

Hello Marco

The logging to the standard error stream is:

[2022-04-08 14:33:52.92508] [163133] [debug] [82ae3iV2] GET "/"
[2022-04-08 14:33:52.92532] [163133] [debug] [82ae3iV2] Routing to a callback
[2022-04-08 14:33:52.92565] [163133] [debug] [82ae3iV2] Rendering template "index.html.ep" from DATA section
[2022-04-08 14:33:52.92610] [163133] [debug] [82ae3iV2] 200 OK (0.001021s, 979.432/s)

This is basically what a modern framework is supposed to do.

The nice thing in this example is that we created a single-file prototype and launched it as a CGI. But we can also launch it as daemon and visit the given address with a browser, which is how you should normally deploy it, usually behind a reverse proxy like nginx.

./mojo.pl daemon
[2022-04-08 14:48:42.01827] [163409] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000
[2022-04-08 14:48:48.53687] [163409] [debug] [CwM6zoUQ] GET "/"
[2022-04-08 14:48:48.53715] [163409] [debug] [CwM6zoUQ] Routing to a callback
[2022-04-08 14:48:48.53752] [163409] [debug] [CwM6zoUQ] Rendering template "index.html.ep" from DATA section
[2022-04-08 14:48:48.53808] [163409] [debug] [CwM6zoUQ] 200 OK (0.001209s, 827.130/s)

If you want you can even launch it with HTTPS as well (please note the syntax to pass the certificates).

./mojo.pl daemon -l 'https://[::]:8080?cert=./ssl/fullchain.pem&key=./ssl/privkey.pem' -m production

For a small application listening on a high port this is already enough and the whole deployment problem goes away.

Speaking about deployment, Mojolicious has basically no dependencies other than the core modules and comes with a lot of goodies, for example a non blocking user-agent.

Recently a legacy application needed to make some API calls. To speed up the process, we wanted to make the requests in parallel. And here’s the gist of the code:

package MyApp::Async;

# ... more modules here

use Mojo::UserAgent;
use Mojo::Promise;

# .... other methods here

sub example {
    my $email = 'test@example.com';
    my $ua = Mojo::UserAgent->new;
    foreach my $list ($self->get_lists) {
        my $promise = $ua->post_p($self->_url("/api/v2/endpoint/$list->{code}"),
                                  json => { email => $email })
          ->then(sub {
                     my $tx = shift;
                     my $res = $tx->result;
                     if ($res->code =~ m/^2/) {
                         $self->_update_db($data);
                     }
                     else {
                         die $tx->req->url . ' ' . $res->code;
                     }
                 });
        push @promises, $promise;
    }
    my $return = 0;
    Mojo::Promise->all(@promises)->then(sub { $return = 1 }, sub { $return = 0})->wait;
    return $return;
}

So a bunch of requests are run in parallel and then synced before returning. Does it remind you of JavaScript? Of course. A lot of common paradigms taken from other languages and frameworks were implemented here, and you can find the best of them in this nice package.

But the point here is that it doesn’t need dozens of new modules installed or upgraded. It’s just a single module in pure Perl that you can even install in your application tree. This is a huge advantage if you’re dealing with a legacy application which uses an old Perl tree and you want to play safe.

So, if you’re starting from scratch, go with Mojolicious. It lets you prototype fast and doesn’t let you down later.

However, starting from scratch is not always an option. Actually, it’s a rare opportunity. There’s a whole world of legacy applications and they generate real money every day. It’s simply not possible or even desirable to throw away something that works for something that would do the same thing but in a “cooler” way. In ten years, the way we’re coding will look old anyway.

Interchange

Wait. Isn’t Interchange an old e-commerce framework? Yes, it’s not exactly a generic web framework, on the contrary, it’s a specialized one, but it’s still a framework and you can still do things in a maintainable fashion. The key is using the so-called action maps:

ActionMap jump <<EOR
sub {
    # get the path parameters
    my ($action, @args) = split(/\//, shift);

    # get the query/body parameters
    my $param = $CGI->{param};

    # redirect to another page
    $Tag->deliver({ location => $final });

    # or serve JSON
    $Tag->deliver({ type => 'application/json', body => $json_string });

    # or serve a file
    $Tag->deliver({ type => 'text/plain', body => $bigfile });

    # or populate the "stash" and serve a template page
    $Tag->tmp(stash_variable => "Marco");
    $CGI->{mv_nextpage} = "test.html";
}
EOR

In pages/test.html you would put this template:

<p>Hello [scratch stash_variable]</p>

Now, I can’t show you a simple script which demonstrates this and you’ll have to take my word for it since we can’t go through the installation process here for a demo.

Interchange is old, and it shows its years, but it is actively maintained. It lacks many of Mojo’s goodies, but you can still do things in a reasonable way.

In the example the code will execute when a path starting with /jump/ is requested. The whole path is passed to the routine, so you can split at /, apply your logic, and finally either set $CGI->{mv_nextpage} to a file in the pages directory or output the response body directly with deliver. This way you can easily build, as a classical example, an API.

It’s a bit of a poor man’s MVC but it works. That’s basically the core of what a framework like Dancer does.

Dancer (1 & 2)

Dancer is basically Ruby’s Sinatra ported to Perl. As already mentioned, ideas developed in other languages and frameworks are often ported to Perl, and this is no exception.

Let’s see it in an action:

#!/usr/bin/env perl
use strict;
use warnings;
use Dancer2;

get '/' => sub {
    my $name = "Marco";
    return "Hello $name\n";
};

start;

Start the script:

Dancer2 v0.400000 server 22969 listening on http://0.0.0.0:3000

Try it with curl:

$ curl -D - http://0.0.0.0:3000
HTTP/1.0 200 OK
Date: Mon, 11 Apr 2022 07:22:18 GMT
Server: Perl Dancer2 0.400000
Server: Perl Dancer2 0.400000
Content-Length: 12
Content-Type: text/html; charset=UTF-8

Hello Marco

If in the script you say use Dancer; instead of use Dancer2, you get:

$ curl -D - http://0.0.0.0:3000
HTTP/1.0 200 OK
Server: Perl Dancer 1.3513
Content-Length: 12
Content-Type: text/html
X-Powered-By: Perl Dancer 1.3513

Hello Marco

Dancer’s core doesn’t do much more than routing. And you’ll also notice that the syntax is very similar to Mojolicious::Lite. So to get something done you need to start installing plugins which will provide the needed glue to interact with a database, work with your template system of choice, and more.

Today you would wonder why you should use Dancer and not Mojolicious, but when Dancer was at the peak of its popularity the games were still open. There were plenty of plugins being written and published on CPAN.

Around 2013 Dancer’s development team decided to rewrite it to make it better. The problem was that plugins and templates needed to be adapted as well. I’m under the impression that the energy got divided and the momentum was lost. Now there are two codebases and two plugin namespaces which do basically the same thing, because for the end user there is not much difference.

Catalyst

So what was attracting people to Dancer? When Dancer came out, Perl had a great MVC framework, which is still around, Catalyst. (And note that the main Mojolicious developer was on the Catalyst team.)

Now, the problem is that to get started with Catalyst, even if it has plenty of documentation, you need to be already acquainted with a lot of concepts and technologies. For example, the tutorial starts to talk about Template Toolkit and the DBIx::Class ORM very early.

These two modules are great and powerful and they deserve to be studied, but for someone new to modern web development, or even to Perl, it feels (and actually is) overwhelming.

So, why would you choose Catalyst today? Catalyst has the stability which Mojo, at least at the beginning, lacked, while backward compatibility is a priority for Catalyst. The other way to look at this is that Catalyst doesn’t see much current development, but someone could see this as a feature.

Even if Catalyst predates all the hyper-modern features that Mojo has, it’s still a modern framework, and a good one. I can’t show you a self contained script (you need a tree of files), but I’d like to show you what makes it very nice and powerful:

package MyApp::Controller::Root;

use Moose;
use namespace::autoclean;

BEGIN { extends 'Catalyst::Controller'; }

# start the chain with /foo/XX
sub foo :Chained('/') CaptureArgs(1) {
    my ($self, $c, $arg) = @_;
    $c->stash(name => "$arg");
}

# /foo/XX/bar/YY
sub bar :Chained('foo') Args(1) {
    my ($self, $c, $arg) = @_;
    $c->detach($c->view('JSON'));
}

# /foo/XX/another/YY
sub another :Chained('foo') Args(1) {
    my ($self, $c, $arg) = @_;
    $c->detach($c->view('HTML'));
}

So, if you hit /foo/marco/bar/test the second path fragment will be processed by the first method (CaptureArgs(1)) and saved in the stash. Then the second bar method will be chained to it and the name will be available in the stash. The last method will be hit with /foo/marco/another/test2. (Incidentally, please note that Mojolicious has nested routes as well.)

Now, I think it’s clear that in this way you can build deep hierarchies of paths with reusable components. This works really great with the DBIx::Class ORM, where you can chain queries as well. As you can imagine, this is far from a simple setup. On the contrary, this is an advanced setup for people who already know their way around web frameworks.

Conclusion

So, to sum up this excursion in the amazing land of Perl web frameworks: If you build something from scratch, go with Mojolicious. It’s your best bet. If nothing else, it’s super-easy to install, with basically no dependencies.

However, there’s no need to make a religion out of it. Rewriting code without a clear gain is a waste of time and money. A good developer should still be able to write maintainable code with the existing tools.

2015 Perl Dancer Conference videos

2015-12-30T00:00:00+00:00

The 2015 Perl Dancer Conference has recently released the presentation videos. This year the conference was hosted in beautiful Vienna, Austria. Josh Lavin and I were both honored to attend the conference as well as give talks. Earlier, Josh wrote summaries of the conference:

Conference Recap

Conference Presentations

SpaceCamps “The Final Frontier”

I gave a talk exploring new technologies for End Point’s own DevCamps development tool. During the presentation I detailed my research into containers and what a cloud-based development environment might look like.

SpaceCamps Presentation Video

AngularJS & Dancer for Modern Web Development

Josh detailed his experience migrating legacy applications utilizing Dancer, AngularJS, and modern Perl techniques. Josh highlighted the challenges he faced during the process, as well as lessons he learned along the way.

AngularJS & Dancer for Modern Web Development Presentation Video

Lightning Talks

Josh and I both gave short “lightning talks.” Josh’s was on Writing Unit Tests for a Legacy App (Interchange 5), and mine was on Plack & Interchange 5.

To review the rest of the presentations please checkout the Perl Dancer Conference YouTube channel.

Summary

The Perl Dancer community continues to flourish and the conference this year hosted a record 5 core Dancer developers. Dancer is about to release the finalized version of its long awaited plugin infrastructure for Dancer2. A lot of work on this was completed during the conference. Being an organizer of the conference, it brings me great joy to see this success. This news along with the release of Perl 6, I am certain 2016 will be a wonderful year for not only Dancer but the entire Perl community.

AngularJS & Dancer for Modern Web Development

2015-10-30T00:00:00+00:00

At the Perl Dancer Conference 2015, I gave a talk on AngularJS & Dancer for Modern Web Development. This is a write-up of the talk in blog post form.

Legacy Apps

It’s a fact of life as a software developer that a lot of us have to work with legacy software. There are many older platforms out there, still being actively used today, and still supporting valid businesses. Thus, legacy apps are unavoidable for many developers. Eventually, older apps are migrated to new platforms. Or they die a slow death. Or else the last developer maintaining the app dies.

Oh, to migrate

It would be wonderful if I could migrate every legacy app I work on to something like Perl Dancer. This isn’t always practical, but a developer can dream, right?

Of course, every circumstance is different. At the very least, it is helpful to consider ways that old apps can be migrated. Using new technologies can speed development, give you new features, and breathe new life into a project, often attracting new developers.

As I considered how to prepare my app for migration, here are a few things I came up with:

Break out of the Legacy App Paradigm
- Consider that there are better ways to do things than the way they’ve always been done
Use Modern Perl
- Use object-oriented Perl
- Use tests
- Use CPAN modules and don’t reinvent the wheel
Organize business logic
- Try to avoid placing logic in front-end code

You are in a legacy codebase

I explored how to start using testing, but I soon realized that this requires methods or subroutines. This was the sad realization that up till now, my life as a Perl programmer had been spent doing scripting. My code wasn’t testable, and looked like a relic with business logic strewn about.

Change

I set out to change my ways. I started exploring object-oriented Perl using Moo, since Dancer2 uses Moo. I started trying to write unit tests, and started to use classes and methods in my code.

Essentially, I began breaking down problems into smaller problems. This, after all, is how the best methods are written: short and simple, that do just one thing. I found that writing code this way was fun.

Crash

I quickly realized that I wasn’t able to run tests in my Legacy App, as it couldn’t be called from the command line (at least not out of the box, and not without weird hacks). Thus, if my modules depended on Legacy App code, I wouldn’t be able to call them from tests, because I couldn’t run these tests from the shell.

This led me to a further refinement: abstract away all Legacy App-specific code from my modules. Or, at least all the modules I could (I would still need a few modules to rely on the Legacy App, or else I wouldn’t be using it it all). This was a good idea, it turned out, as it follows the principle of Separation of Concerns, and the idea of Web App + App, which was mentioned frequently at the conference.

Now I was able to run tests on my modules!

Move already

This whole process of “getting ready to migrate” soon began to look like yak shaving. I realized that I should have moved to Dancer earlier, instead of trying to do weird hacks to get the Legacy App doing things as Dancer would do them.

However, it was a start, a step in the right direction. Lesson for me, tip for you.

And, the result was that my back-end code was all the more ready for working with Dancer. I would just need to change a few things, and presto! (More on this below.)

Front-End

With the back-end looking tidier, I now turned to focus on the front-end. There was a lot of business logic in my front-end code that needed to be cleaned up.

Here is an example of my Legacy App front-end code:

@_TOP_@
<h1>[scratch page_title]</h1>
[perl]
   my $has_course;
   for (grep {$_->{mv_ib} eq 'course'} @$Items) {
      $has_course++;
   }
   return $has_course ? '<p>You have a course!</p>' : '';
[/perl]
<button>Buy [if cgi items]more[else]now[/else][/if]</button>
@_BOTTOM_@

As you can see, the Legacy App allowed the embedding of all sorts of code into the HTML page. I had Legacy App tags (in the brackets), plus something called “embedded perl”, plus regular HTML. Add all this together and you get Tag Soup.

This kind of structure won’t look nice if you attempt to view it on your own machine in a web browser, absent from the Legacy App interpreting it. But let’s face it, this code doesn’t look nice anywhere.

Separation of Concerns

I thought about how to apply the principle of Separation of Concerns to my front-end code. One thing I landed on, which isn’t a new idea by any means, is the use of “HTML + placeholders”, whereby I would use some placeholders in my HTML, to be later replaced and filled in with data. Here is my first attempt at that:

@_TOP_@
[my-tag-attr-list
    page_title="[scratch page_title]"
    has_course="[perl] ... [/perl]"
    buy_phrase="Buy [if cgi items]more[else]now[/else][/if]"
]

    <h1>{PAGE_TITLE}</h1>
    {HAS_COURSE?}<p>You have a course!</p>{/HAS_COURSE?}
    <button>{BUY_PHRASE}</button>

[/my-tag-attr-list]
@_BOTTOM_@

What I have here uses the Legacy App’s built-in placeholder system. It attempts to set up all the code in the initial “my-tag-attr-list”, then the HTML uses placeholders (in braces) which get replaced upon the page being rendered. (The question-mark in the one placeholder is a conditional.)

This worked OK. However, the logic was still baked into the HTML page. I wondered how I could be more ready for Dancer? (Again, I should have just gone ahead and migrated.) I considered using Template::Toolkit, since it is used in Dancer, but it would be hard to add to my Legacy App.

Enter AngularJS (or your favorite JavaScript framework)

AngularJS is a JavaScript framework for front-end code. It displays data on your page, which it receives from your back-end via JSON feeds. This effectively allows you to separate your front-end from your back-end. It’s almost as if your front-end is consuming an API. (Novel idea!)

After implementing AngularJS, my Legacy App page looked like this (not showing JavaScript):

@_TOP_@
<h1 ng-bind="page.title"></h1>
<p ng-if="items.course">You have a course!</p>
<button ng-show="items">Buy more</button>
<button ng-hide="items">Buy now</button>
@_BOTTOM_@

Now all my Legacy App is doing for the front-end is basically “includes” to get the header/footer (the TOP and BOTTOM tags). The rest is HTML code with ng- attributes. These are what AngularJS uses to “do” things.

This is much cleaner than before. I am still using the Legacy App back-end, but all it has to do is “routing” to call the right module and deliver JSON (and do authentication).

Here’s a quick example of how the JavaScript might look:

<html ng-app="MyApp">
...
<script src="angular.min.js"></script>
<script>
  angular.module / factory / controller
  $scope.items = ...;
</script>
</html>

This is very simplified, but via its modules/factories/controllers, the AngularJS code handles how the JSON feeds are displayed in the page. It pulls in the JSON and can massage it for use by the ng- attributes, etc.

I don’t have to use AngularJS to do this—I could use a Template::Toolkit template delivered by Dancer, or any number of other templating systems. However, I like this method, because it doesn’t require a Perl developer to use. Rather, any competent JavaScript developer can take this and run with it.

Migration

Now the migration of my entire app to Dancer is much easier. I gave it a whirl with a handful of routes and modules, to test the waters. It went great.

For my modules that were the “App” (not the “Web App” and dependent on the Legacy App), very few changes were necessary. Here is an example of my original module:

package MyApp::Feedback;
use MyApp;
my $app = MyApp->new( ... );
sub list {
    my $self = shift;
    my $code = shift
        or return $app->die('Need code');
    my $rows = $app->dbh($feedback_table)->...;
    return $rows;
}

You’ll see that I am using a class called MyApp. I did this to get a custom die and a database handle. This isn’t really the proper way to do this (I’m learning), but it worked at the time.

Now, after converting that module for use with Dancer:

package MyApp::Feedback;
use Moo;
with MyApp::HasDatabase;
sub list {
    my $self = shift;
    my $code = shift
        or die 'Need code';
    my $rows = $self->dbh->...;
    return $rows;
}

My custom die has been replaced with a Perl die. Also, I am now using a Moo::Role for my database handle. And that’s all I changed!

Before

The biggest improvements were in things that I “stole” from Dancer. (Naturally, Dancer would do things better than I.) This is my Legacy App’s route for displaying and accepting feedback entries. It does not show any authentication checks. It handles feeding back an array of entries for an item (“list”), a single entry (GET), and saving an entry (POST):

sub _route_feedback {
    my $self = shift;
    my (undef, $sub_action, $code) = split '/', $self->route;
    $code ||= $sub_action;
    $self->_set_status('400 Bad Request');   # start with 400
    my $feedback = MyApp::Feedback->new;
    for ($sub_action) {
        when ("list") {
            my $feedbacks = $feedback->list($code);
            $self->_set_tmp( to_json($feedbacks) );
            $self->_set_path('special/json');
            $self->_set_content_type('application/json; charset=UTF-8');
            $self->_set_status('200 OK') if $feedbacks;
        }
        default {
            for ($self->method) {
                when ('GET') {
                    my $row = $feedback->get($code)
                        or return $self->_route_error;
                    $self->_set_tmp( to_json($row) );
                    $self->_set_path('special/json');
                    $self->_set_content_type('application/json; charset=UTF-8');
                    $self->_set_status('200 OK') if $row;
                }
                when ('POST') {
                    my $params = $self->body_parameters
                        or return $self->_route_error;
                    $params = from_json($params);
                    my $result = $feedback->save($params);
                    $self->_set_status('200 OK') if $result;
                    $self->_set_path('special/json');
                    $self->_set_content_type('application/json; charset=UTF-8');
                }
            }
        }
    }
}

After

Here are those same routes in Dancer:

prefix '/feedback' => sub {
    my $feedback = MyApp::Feedback->new;
    get '/list/:id' => sub {
        return $feedback->list( param 'id' );
    };
    get '/:code' => sub {
        return $feedback->get( param 'code' );
    };
    post '' => sub {
        return $feedback->save( scalar params );
    };
};

Dancer gives me a lot for free. It is a lot simpler. There’s still no authentication shown here, but everything else is done. (And I can use an authentication plugin to make even that easy.)

TMTOWTDI

For the front-end, we have options on how to use Dancer. We could have Dancer deliver the HTML files that contain AngularJS. Or, we could have the web server deliver them, as there is nothing special about them that says Dancer must deliver them. In fact, this is especially easy if our AngularJS code is a Single Page App, which is a single static HTML file with AngularJS “routes”. If we did this, and needed to handle authentication, we could look at using JSON Web Tokens.

Now starring Dancer

In hindsight, I probably should have moved to Dancer right away. The Legacy App was a pain to work with, as I built my own Routing module for it, and I also built my own Auth checking module. Dancer makes all this simpler.

In the process, though, I learned something…

Dancer is better?

I learned you can use tools improperly. You can do Dancer “wrong”. You can write tag soup in anything, even the best modern tools.

You can stuff all your business logic into Template::Toolkit tags. You can stuff logic into Dancer routes. You can do AngularJS “wrong” (I probably do).

Dancer is better:

Dancer is better when (thanks to Matt S Trout for these):

Routes contain code specific to the Web.
Routes call non-Dancer modules (where business logic lives; again, Web App + App).
The route returns the data in the appropriate format.

These make it easy to test. You are effectively talking to your back-end code as if it’s an API. Because it is.

The point is: start improving somewhere. Maybe you cannot write tests in everything, but you can try to write smart code.

Lessons learned

Separate concerns
Keep it testable
Just start somewhere

The end. Or maybe the beginning…

Perl Dancer Conference 2015 Report—Conference Days

2015-10-30T00:00:00+00:00

In my last post, I shared about the Training Days from the Perl Dancer 2015 conference, in Vienna, Austria. This post will cover the two days of the conference itself.

While there were several wonderful talks, Gert van der Spoel did a great job of writing recaps of all of them (Day 1, Day 2), so here I’ll cover the ones that stood out most to me.

Day One

Dancer Conference, by Alexis Sukrieh (used with permission)

Sawyer X spoke on the State of Dancer. One thing mentioned, which came up again later in the conference, was: Make the effort, move to Dancer 2! Dancer 1 is frozen. There have been some recent changes to Dancer:

Middlewares for static files, so these are handled outside of Dancer
New Hash::MultiValue parameter keywords (route_parameters, query_parameters, body_parameters; covered in my earlier post)
Delayed responses (asynchronous) with delayed keyword:
- Runs on the server after the request has finished.
- Streaming is also asynchronous, feeding the user chunks of data at a time.

Items coming soon to Dancer may include: Web Sockets (supported in Plack), per-route serialization (currently enabling a serializer such as JSON affects the entire app—later on, Russell released a module for this, which may make it back into the core), Dancer2::XS, and critic/linter policies.

Thomas Klausner shared about OAuth & Microservices. Microservices are a good tool to manage complexity, but you might want to aim for “monolith first”, according to Martin Fowler, and only later break up your app into microservices. In the old days, we had “fat” back-ends, which did everything and delivered the results to a browser. Now, we have “fat” front-ends, which take info from a back-end and massage it for display. One advantage of the microservice way of thinking is that mobile devices (or even third parties) can access the same APIs as your front-end website.

OAuth allows a user to login at your site, using their credentials from another site (such as Facebook or Google), so they don’t need a password for your site itself. This typically happens via JavaScript and cookies. However, to make your back-end “stateless”, you could use JSON Web Tokens (JWT). Thomas showed some examples of all this in action, using the OX Perl module.

One thing I found interesting that Thomas mentioned: Plack middleware is the correct place to implement most of the generic part of a web app. The framework is the wrong part. I think this mindset goes along with Sawyer’s comments about Web App + App in the Training Days.

Mickey Nasriachi shared his development on PONAPI, which implements the JSON API specification in Perl. The JSON API spec is a standard for creating APIs. It essentially absolves you from having to make decisions about how you should structure your API.

Panorama from the south tower of St. Stephen’s cathedral, by this author

Gert presented on Social Logins & eCommerce. This built on the earlier OAuth talk by Thomas. Here are some of the pros/cons to social login which Gert presented:

Pros—customer:
- Alleviates “password fatigue”
- Convenience
- Brand familiarity (with the social login provider)
Pros—eCommerce website:
- Expected customer retention
- Expected increase in sales
- Better target customers
- “Plug & Play” (if you pay)—some services exist to make it simple to integrate social logins, where you just integrate with them, and then you are effectively integrated with whatever social login providers they support. These include Janrain and LoginRadius
Cons—customer:
- Privacy concerns (sharing their social identity with your site)
- Security concerns (if their social account is hacked, so are all their accounts where they have used their social login)
- Confusion (especially on how to leave a site)
- Usefulness (no address details are provided by the social provider in the standard scope, so the customer still has to enter extra details on your site)
- Social account hostages (if you’ve used your social account to login elsewhere, you are reluctant to shut down your social account)
Cons—eCommerce website:
- Legal implications
- Implementation hurdles
- Usefulness
- Provider problem is your problem (e.g., if the social login provider goes down, all your customers who use it to login are unable to login to your site)
- Brand association (maybe you don’t want your site associated with certain social sites)
Cons—social provider:
- ???

Šimun Kodžoman spoke on Dancer + Meteor = mobile app. Meteor is a JavaScript framework for both server-side and client-side. It seems one of the most interesting aspects is you can use Meteor with the Android or iOS SDK to auto-generate a true mobile app, which has many more advantages than a simple HTML “app” created with PhoneGap. Šimun is using Dancer as a back-end for Meteor, because the server-side Meteor aspect is still new and unstable, and is also dependent on MongoDB, which cannot be used for everything.

End Point’s own Sam Batschelet shared his work on Space Camp, a new container-based setup for development environments. This pulls together several pieces, including CoreOS, systemd-nspawn, and etcd to provide a futuristic version of DevCamps.

Day Two

Conference goers, by Sam
(used with permission)

Andrew Baerg spoke on Taming the 1000-lb Gorilla that is Interchange 5. He shared how they have endeavored to manage their Interchange development in more modern ways, such as using unit tests and DBIC. One item I found especially interesting was the use of DBIx::Class::Fixtures to allow saving bits of information from a database to keep with a test. This is helpful when you have a bug from some database entry which you want to fix and ensure stays fixed, as databases can change over time, and without a “fixture” your test would not be able to run.

Russell Jenkins shared HowTo Contribute to Dancer 2. He went over the use of Git, including such helpful commands and tips as:

git status –short –branch
Write good commit messages: one line summary, less than 50 characters; longer description, wrapped to 72 characters; refer to and/or close issues
Work in a branch (you shall not commit to master)
“But I committed to master” –> branch and reset
git log –oneline –since=2.weeks
git add –fixup
The use of branches named with “feature/whatever” or “bugfix/whatever” can be helpful (this is Russell’s convention)

There are several Dancer 2 issues tagged “beginner suitable”, so it is easy for nearly anyone to contribute. The Dancer website is also on GitHub. You can even make simple edits directly in GitHub!

It was great to have the author of Dancer, Alexis Sukrieh, in attendance. He shared his original vision for Dancer, which filled a gap in the Perl ecosystem back in 2009. The goal for Dancer was to create a DSL (Domain-specific language) to provide a very simple way to develop web applications. The DSL provides “keywords” for use in the Dancer app, which are specific to Dancer (basically extra functionality for Perl). One of the core aspects of keeping it simple was to avoid the use of $self (a standby of object-oriented Perl, one of the things that you just “have to do”, typically).

Alexis mentioned that Dancer 1 is frozen—Dancer 2 full-speed ahead! He also shared some of his learnings along the way:

Fill a gap (define clearly the problem, present your solution)
Stick to your vision
Code is not enough (opensource needs attention; marketing matters)
Meet in person (collaboration is hard; online collaboration is very hard)
Kill the ego—you are not your code

While at the conference, Alexis even wrote a Dancer2 plugin, Dancer2::Plugin::ProbabilityRoute, which allows you to do A/B Testing in your Dancer app. (Another similar plugin is Dancer2::Plugin::Sixpack.)

Also check out Alexis’ recap.

Finally, I was privileged to speak as well, on AngularJS & Dancer for Modern Web Development. Since this post is already pretty long, I’ll save the details for another post.

Summary

In summary, the Perl Dancer conference was a great time of learning and building community. If I had to wrap it all up in one insight, it would be: Web App + App—that is, your application should be a compilation of: Plack middleware, Web App (Dancer), and App (Perl classes and methods).

Perl Dancer Conference 2015 Report — Training Days

2015-10-28T00:00:00+00:00

I just returned from the Perl Dancer Conference, held in Vienna, Austria. It was a jam-packed schedule of two days of training and two conference days, with five of the nine Dancer core developers in attendance.

Kohlmarkt street, Wien, by this author

If you aren’t familiar with Perl Dancer, it is a modern framework for Perl for building web applications. Dancer1 originated as a port of Ruby’s Sinatra project, but has officially been replaced with a rewrite called Dancer2, based on Moo, with Dancer1 being frozen and only receiving security fixes. The Interchange 5 e-commerce package is gradually being replaced by Dancer plugins.

Day 1 began with a training on Dancer2 by Sawyer X and Mickey Nasriachi, two Dancer core devs. During the training, the attendees worked on adding functionality to a sample Dancer app. Some of my takeaways from the training:

Think of your app as a Dancer Web App plus an App. These should ideally be two separate things, where the Dancer web app provides the URL routes for interaction with your App.
The lib directory contains all of your application. The recommendation for large productions is to separate your app into separate namespaces and classes. Some folks use a routes directory just for routing code, with lib reserved for the App itself.
It is recommended to add an empty .dancer file to your app’s directory, which indicates that this is a Dancer app (other Perl frameworks do similarly).
When running your Dancer app in development, you can use plackup -R lib bin/app.psgi which will restart the app automatically whenever something changes in lib.
Dancer handles all the standard HTTP verbs, except note that we must use del, not delete, as delete conflicts with the Perl keyword.
There are new keywords for retrieving parameters in your routes. Whereas before we only had param or params, it is now recommended to use:
- route_parameters,
- query_parameters, or
- body_parameters
- all of which can be used with ->get(‘foo’) which is always a single scalar, or ->get_all(‘foo’) which is always a list.
- These allow you to specify which area you want to retrieve parameters from, instead of being unsure which param you are getting, if identical names are used in multiple areas.

Day 2 was DBIx::Class training, led by Stefan Hornburg and Peter Mottram, with assistance from Peter Rabbitson, the DBIx::Class maintainer.

DBIx::Class (a.k.a. DBIC) is an Object Relational Mapper for Perl. It exists to provide a standard, object-oriented way to deal with SQL queries. I am new to DBIC, and it was a lot to take in, but at least one advantage I could see was helping a project be able to change database back-ends, without having to rewrite code (cue PostgreSQL vs MySQL arguments).

I took copious notes, but it seems that the true learning takes place only as one begins to implement and experiment. Without going into too much detail, some of my notes included:

Existing projects can use dbicdump to quickly get a DBIC schema from an existing database, which can be modified afterwards. For a new project, it is recommended to write the schema first.
DBIC allows you to place business logic in your application (not your web application), so it is easier to test (once again, the recurring theme of Web App + App).
The ResultSet is a representation of a query before it happens. On any ResultSet you can call ->as_query to find the actual SQL that is to be executed.
DBIx::Class::Schema::Config provides credential management for DBIC, and allows you to move your DSN/username/password out of your code, which is especially helpful if you use Git or a public GitHub.
DBIC is all about relationships (belongs_to, has_many, might_have, and has_one). many_to_many is not a relationship per se but a convenience.
DBIx::Class::Candy provides prettier, more modern metadata, but cannot currently be generated by dbicdump.
For deployment or migration, two helpful tools are Sqitch and DBIx::Class::DeploymentHandler. Sqitch is better for raw SQL, while DeploymentHandler is for DBIC-managed databases. These provide easy ways to migrate, deploy, upgrade, or downgrade a database.
Finally, RapidApp can read a database file or DBIC schema and provide a nice web interface for interacting with a database. As long as you define your columns properly, RapidApp can generate image fields, rich-text editors, date-pickers, etc.

The training days were truly like drinking from a firehose, with so much good information. I am looking forward to putting this into practice!

Stay tuned for my next blog post on the Conference Days.

Documenting web services with Perl POD and AJAX

2015-06-26T00:00:00+00:00

Perl POD is a handy, convenient, but low-tech approach to embedded documentation. Consider a web service in Dancer:

get time => sub {
  return scalar(localtime());
};

(Disclaimer: my actual use-case of this technique was even more legacy: I was documenting Interchange Actionmaps that returned images, JSON, etc.)

Your application might have several, or even dozens of these, with various parameters, returning data in JSON or TXT or CSV or who-knows-what. I chose to document these in Perl POD (Plain Old Documentation) format, e.g.,

=pod

=head1 time

Retrieves the current time

=over 3

=item Parameters

None.

=item Example

=begin html

<script src="/js/example-time.js" type="text/javascript"></script>

=end html

=back

=cut

This block gets inserted right in-line with the web service code, so it’s immediately obvious to anyone maintaining it (and thus has the best chance of being maintained if and when the code changes!). Now I can generate an HTML page directly from my Perl code:

$ pod2html MyPackage.pm

Your output looks something like this (excerpted for clarity):

time

Retrieves the current time Parameters

None.

Example

Where the magic comes in is the JavaScript code that allows an in-line example, live and accurate, within the documentation page. You’ll actually get something more like this:

time

Retrieves the current time Parameters

None.

Example

(results appear here)

Note that the code I have below is not factored by choice; I could move a lot of it out to a common routine, but for clarity I’m leaving it all in-line. I am breaking up the script into a few chunks for discussion, but you can and should construct it all into one file (in my example, “js/example-time.js”).

/* example-time.js */
$(document).ready(
  function(){
    $('script[src$="/example-time.js"]').after(
"<form action='\"/time\"'>" +
/* Note 1 */
"<input data\"="" type='\"submit\"' value='\"Get'/>" +
"<input name="hide" none\"="" result\"="" style='\"display:' type='\"button\"' value='\"Hide'/>" +
"</form>" +
"<div id='\"time-result\"'></div>"
    );

Note 1: This being a painfully simple example of a web service, there are no additional inputs. If you have some, you would add them to the HTML being assembled into the

tag, and then using jQuery, add them below to the url parameter, or into the data structure as required by your particular web service.

This step just inserts a simple into the document. I chose to embed the form into the JavaScript code, rather than the POD, because it reduces the clutter and separates the example from the web service.

    var $form = $('form[action="/time"]');
    $form.submit(function(){
      $.ajax(
        {
          'url': $form.attr('action') /* Note 1 also */,
          'data': {},
          'dataType': 'text',
          'async': false,
          'success':
             function(data){
                 $('#time-result').html($('<pre;//>').html(data))
                     .addClass('json');
             },

Here we have a submit handler that performs a very simple AJAX submit using the form’s information, and upon success, inserts the results into a result

as a pre-formatted block. I added a “json” class which just tweaks the font and other typographic presentation a bit; you can provide your own if you wish.

I’m aware that there are various jQuery plug-ins that will handle AJAX-ifying a form, but I couldn’t get the exact behavior I wanted on my first tries, so I bailed out and just constructed this approach.

          'error':
             function(){
                 $('#time-result').html('Error retrieving data!')
                     .removeClass('json');
             },
/* */

(That stray-looking comment above is just a work-around for the syntax highlighter.)

Error handling goes here. If you have something more comprehensive, such as examining the result for error codes or messages, this is where you’d put it.

          'complete':
             function(){
                 $form.find('input[name="hide"]').show();
             }
         }
      );
      return false;
    }).find('input[type="button"]').click(function(){
      $('#time-result').html('');
    });
  }
);

And just a bit of UI kindness: we have a “hide” button to make the example go away. Some of my actual examples ran to dozens of lines of JSON output, so I wanted a way to clean up after the example.

Filling in header elements with Dancer and Template::Flute

2015-02-05T00:00:00+00:00

Inserting content into Dancer output files involves using a templating system. One such is Template::Flute. In its simplest possible explanation, it takes a Perl hash, an XML file, and an HTML template, and produces a finished HTML page.

For a project involving Dancer and Template::Flute, I needed a way to prepare each web page with its own set of JavaScript and CSS files. One way is to construct separate layout files for each different combination of .js and .css, but I figured there had to be a better way.

Here’s what I came up with: I use one layout for all my typical pages, and within the header, I have:

 <link class="additional_style" href="/css/additional.css" rel="stylesheet" type="text/css"/>
 <script class="additional_script" src="/javascripts/additional.js" type="text/javascript">
 </script>

The trick here is, there’s no such files “additional.css” and “additional.js”. Instead, those are placeholders for the actual CSS and JS files I want to link into each HTML file.

My Perl object has these fields (in addition to the other content):

$context->{additional_styles}    = [
	{ url => '/css/checkout.css' },
	{ url => '/css/colorbox.css' },
];
$context->{additional_scripts}   = [
	{ url => '/javascripts/sprintf.js' },
	{ url => '/javascripts/toCurrency.js' },
	{ url => '/javascripts/jquery.colorbox-min.js' },
	{ url => '/javascripts/checkout.js' },
];

while my XML file looks like this:

<specification>
<list iterator="additional_styles" name="additional_style">
  <param field="url" name="additional_style" target="href"/>
</list>
<list iterator="additional_scripts" name="additional_script">
  <param field="url" name="additional_script" target="src"/>
</list>
</specification>

So we have all the elements, but unless you have used all this before, you may not realize how we get the output. (Skip to the punchline if that’s not true.)

The XML file is a connector that tells Template::Flute how to mix the Perl hash into the HTML template. Usually you connect things via class names, so in the case of:

<link class="additional_style" href="/css/additional.css" rel="stylesheet" type="text/css"/>

the class name in the HTML and the name field in the XML connect, while the iterator field in the XML and the hash key in the Perl hashref do as well. The case of a

<list>

means that the hash value must be an arrayref of hashrefs, i.e.,

{
	"additional_style" => [
		{ url => "..." },
		...,
	],
}

Important note: if the hash value is undefined, you’ll get a run-time error when you try to expand the HTML template, and if you have an empty arrayref, the result of the expansion is an empty string (which is just what you want).

And so, through the magic of Template::Flute, what the browser sees is:

<link class="additional_style" href="/css/checkout.css" rel="stylesheet" type="text/css"/>
<link class="additional_style" href="/css/colorbox.css" rel="stylesheet" type="text/css"/>
<link class="additional_style" href="/css/admin/admin.css" rel="stylesheet" type="text/css"/>...
...
<script class="additional_script" src="/javascripts/sprintf.js" type="text/javascript">
</script>
<script class="additional_script" src="/javascripts/toCurrency.js" type="text/javascript">
</script>
<script class="additional_script" src="/javascripts/jquery.colorbox-min.js" type="text/javascript">
</script>
<script class="additional_script" src="/javascripts/checkout.js" type="text/javascript">
</script>

Cleaner redirection in Perl Dancer

2015-01-21T00:00:00+00:00

Recently I worked on a project using the Perl web application framework Dancer that had multiple paths to order a product:

 /product => /cart => /checkout => /receipt

That’s the standard approach. Then there was a “phone order” approach:

 /create_order => /checkout => /receipt

A “phone order” is one taken down (usually by phone), where the user who is logged in is not the same as the user who “owns” the order. Thus, one user is ordering on behalf of another: the order must be recorded as part of the second user’s order history, the various shipping and billing information must come from that user’s stored information, and even the product pricing has to be calculated as though that customer were doing the ordering rather than the logged-in user.

As a consequence, the phone order page flow actually ended up as:

 get /create_order => post /create_order => /checkout

The submission of the /create_order page was processed in an environment that knew about this “proxy” ordering arrangement, thus could do some particularly special-case processing, and then the idea was to pass off to the /checkout page, which would finalize the order including payment information.

All well and good, but when it came time to implement this, I was faced with a minor inconvenience and a bad choice:

Since /checkout was itself a POSTed page, I needed to reach that page with a set of form parameters in hand. So my original plan was:

 post '/create_order' => sub {
   ... # do my special-case processing, and then:
   forward '/checkout', { param1 => $value1, ... };
 };

While this works, the problem is that “forward” as a Dancer directive doesn’t interact with the browser: it just interrupts your path handling of “/create_order” and resumes at “/checkout”. So the browser, innocent of these shenanigans, remains on “/create_order”. It would be so much cleaner (darn my OCD!) if the browser ended up at “/checkout”.

That means you need to redirect the request, though. I.e.,

 post '/create_order' => sub {
   ... # do my special-case processing, and then:
   redirect '/checkout';  # hmm, something's missing here
 };

Hmm, redirect doesn’t support a parameter hash. Oh, well, no problem:

   redirect url_for('/checkout', { param1 => $value1, ... });

That gets the job done, but at a price: now instead of a nice, clean URL at my final destination, I get:

   .../checkout?param1=value1&param2=...

So, still not right. Some research and mailing-list inquiries led me to:

Why doesn’t HTTP have POST redirect?

Short version: you can’t get there from here. Redirection is supposed to be “idempotent”, meaning you can repeat them without harm. That’s why when you refresh the page after a form submission, browsers will ask for permission to re-submit the form rather than just silently refreshing the page.

So what’s the option? Well, I can think of two approaches here:

One: instead of redirecting with parameters, store the parameters in the session:

 post '/create_order' => sub {
   ... # do my special-case processing
   session 'create_order_for_checkout' => { param1 => $value1, ... };
   redirect '/checkout';
 };
 post '/checkout' => sub {
   my $params = (session 'create_order_for_checkout')
     || params();
   ...
 };

Two: do away with the post handler for ‘/create_order’ altogether, and move the processing inside the post handler for ‘/checkout’. The merits of that depend on how complex the /create_order handler is.

I’m leaning toward the first approach, definitely.

Ecommerce Innovation 2013

2013-10-16T00:00:00+00:00

Mark Johnson and I went to the Ecommerce Innovation 2013 conference in beautiful Hancock, NY. The event was hosted by Sam Batschelet of West Branch Resort. The conference was spread out over three days and was very well planned. We had plenty of time in between talks to mingle with the other people. All of the talks were very insightful and informative. I found the mixture of technology and marketing talks beneficial. I have already discussed some things with my clients that I learned.

A brief overview of the talks

Jure Kodzoman of Informa had two different subjects.
- His first talk was about Template::Flute which is a Perl based template system which is the default template for Interchange 6. It utilizes the use of html classes to figure out where to parse in the data returned from your Perl code. Overall it seems pretty straight forward to use.
- His second talk was about the current state of the new Interchange 6 demo store.
Ana Kozole of Informa had a talk named “Remarketing with Banners” that was really informative.The base of this is to have the ability to show specific banners to visitors on different websites. She discussed different remarketing techniques including creating specific lists based on different criteria like all visitors, people who got to the checkout page but didn’t checkout or people who used a coupon code etc. You can also use remarketing lists for search ads.
Luka Klemenc of Informa gave two talks.
- He discussed some a CRM system that they had developed in house and some of the pros and cons.
- Luka gave us short talk about the ways to know whether or not your newsletter is effective.
Josh Lavin of Perusion talked about a new template for Interchange 5 called Strap which is based on Bootstrap version 2.3.2. With this new template they have created a bunch of page and url changes to make the stock Interchange much more SEO friendly. This also makes a few underlying changes like assuming the username to login would be your email address and creating a multiple page checkout.
Mike Heins of Perusion gave us a brief history of Interchange and compared it to modern day frameworks like Dancer. He also gave us a brief overview of PCI compliance and how Interchange holds up. He introduced us to the features of the Perusion Payment Server which is a remote credit card processing system that helps with PCI compliance.
Stefan Hornburg of LinuXia Systems discussed two different topics with us.
- gave us an overview of where the Interchange 6 project currently is and where it’s going. He gave us some code samples of the way we can do simple things like add an item to the cart, fetch the subtotal of the cart and talk to the database.
- Stefan walked us through an integration he did for OpenERP with Interchange 5 using RPC::XML.
Mark Johnson gave us a review of the modifications to Interchange 5 to allow web servers like nginx or Apache to cache entire pages. He discussed how we modified Interchange 5 for a customer to help with a DDoS attack. He laid out all of the new usertags and directives you will need to set to get pages to be cachable including some “gotchas” like not sending cookie information if you want this page to be cached. We hope that this feature will be included in the Interchange 5.8.1 release.
Sam Batschelet gave a talk about DevCamps and the reasons why it is so great. He discussed things like using Perlbrew and Carton in camps to help get around the fact that most Linux operating systems ship with a pretty old version of Perl. He also expanded on a few features that we hope to get released soon.

The last big item on the schedule was a 2 hour round table discussion about the database schema for Interchange 6. It was a very good discussion with many different opinions for adjustments. Most of us based our suggestions on past experience with clients. I do not think we are finished making adjustments to it but we are on the right path to a very flexible setup.

Overall I thought the conference was a great success. It was great to meet in person some people I had only seen on a mailing list before and pass around ideas for the future. I cannot wait to see what cool new things we will have to discuss next year!

First Dance: a Gentle Introduction to Dancer.pm for Web Services

2013-10-07T00:00:00+00:00

I’ve been dabbling in Dancer (Version One, not so much Version Two). Our first opportunity to create a production-worthy Dancer application just rolled out with encouraging results, so I thought I would outline some things we tried, and things we learned. Don’t worry if you don’t know a thing about Dancer; I’ll educate you a little as we go along.

First, some background. This application was a re-host and rewrite of an existing application written in JavaScript and plain old CGI Perl scripts, returning either HTML or fairly simple JSON objects. Even the database it connected to was ancient DBM files. Those files were created by extracting data from the main website’s database, so that part was easy to replace—we just had to use the extraction queries (mostly) to connect directly.

We chose Dancer as the platform for this effort because of its purely Perl footprint, and its relatively easy deployment.

Since I didn’t really want to rewrite the front-end JavaScript code, I left that mostly as-is and concentrated on replacing the CGI scripts. The first step was to move all their code into a non-Dancer Perl module, which communicated with the outside world entirely in the form of hashrefs:

sub do_something_great {
    my $opt = shift;
    my $output = {};
    # do something great!
    return $output;
}

Next, as a stop-gap measure, I replaced the innards of all the CGI scripts to turn CGI arguments into a hashref and call my new processing stuff. Note that so far, no actual Dancer code has been involved. This was intentional; since my Dancer expertise was still a bit thin, I wanted to have as few app-caused problems as possible, when I started Dancing.

But now we should stop for a moment and learn just enough about Dancer to be less dangerous.

Dancer (details at the first link in this article) is a web application framework: in essence, it’s a way to convert URLs into data. You feed in a URL, you get back a web page, a structure, maybe a side effect or three. In order to connect your application to a browser, you need a series of routes:

package MyApp::Routes;
use Dancer;

get '/index.html' => sub {
    print 'hello, world';
}

Dancer sets up some easy-to-grasp syntax that lets you associate a clump of Perl code (an anonymous sub, in the example above) with a URL and a method. Here, a GET of “/index.html” runs the code, and sends the output back to the browser.

...
post '/myapp/do_this.html' => sub {
    do_this(param->{foo});
    return q{It's done.};
};

Likewise, we can specify more complex URLs (such as “/myapp/do_this.html?foo=bar”), with POST methods, and access the CGI parameters via the param() function. It just returns a hashref, which we can address as shown.

Dancer provides quite a bit of support infrastructure: config files, database connections, plug-ins for templating HTML or other output, etc. Since this article is a Gentle Introduction, I refer you to the link cited above if you want to delve deeper.

Now it was pretty simple to make gradual progress: I’d eliminate one CGI script at a time, replacing it with a Dancer route (stored separately from my main processing module, so that the processing could continue to be ignorant of Dancer). Most of the routes were very simple, since the paths they had to recognize were simple and straightforward, with simple CGI parameter lists. E.g.,

get '/notes.html' => sub {
    my $context = MyApp::get_notes({
        dbh    => database(),
        order  => params->{order},
        raw    => 1,
    });
    template 'notes', $context, { layout => 'plain' };
};

This example is one of the routes that produces HTML. The “template” built-in is just a layer wrapped around your preferred templating system (such as Template::Toolkit); it looks for a template file called “notes”, using a set of values specified by “$context”, and wraps the result in a “layout” file called “plain”.

(At the risk of grossly oversimplifying: your “template” can be thought of as all of your output HTML document within the tag, while the “layout” is everything else, with a spot in the middle for your template’s output.)

By contrast, one simple route that is designed to return JSON to the JavaScript side:

get '/orders.html' => sub {
    my $orders = MyApp::get_orders({
        dbh    => database(),
        date   => params->{date},
    });
    return to_json($orders || [], { pretty => params->{pretty} // 0 });
};

We had to make some adjustments to our Apache configuration to make this all work.

RewriteRule    ^/D/myapp/(.*) http://$SOME_ADDR:5001/$1 [P]

As you can see here, we decided to set up the Dancer app as a service on a particular port, and we settled on a particular prefix for our webservice requests to distinguish them from other traffic. All the requests in the JavaScript were adjusted to this new style.

Our main Dancer application has the usual one-liner construction*:

dance;

*Okay, totally not true. For this application, there was a ton of file-parsing and environment-loading to do first, but it was all just a big work-around for some very specific things needed in the app, and not really anything to do with Dancer per se.

Now, we added one more layer to this: Starman. This provided us with a way to simply wrap our Dancer application with the necessary extra bits to turn it into a “service”. That way it starts and stops with a command line interface, logs to a particular path, doesn’t just quit if a user session is ended, etc.

I don’t have enough background to describe it more than that; hopefully, we’ll have a follow-on post here about Starman from one of my colleagues with more “chops”, soon. One thing we learned at the cost of some sanity was: * Don’t put an “exit” statement in your Dancer script if you plan to use Starman*. We never learned why this messed it up, but I assume it’s got something to do with Starman absorbing your code into itself and turning it into a Perl subroutine.

eCommerce Innovation Conference 2013

2013-09-20T00:00:00+00:00

The eCommerce Innovation Conference 2013 is a new conference being held in Hancock, New York, between October 8th and 11th. The conference aims to discuss everything ecommerce with a focus on Perl-based solutions including Dancer and Interchange. It isn’t geared directly to any one specific type of person unlike most conferences. The current speakers list include in-house ecommerce software developers, consultants, sales managers, project managers, and marketing experts. The talk topics range from customer relationship management to template engines for Perl.

Mark Johnson and I are both going to be speaking at the conference. Also there will be Mike Heins, creator of Interchange, and Stefan Hornburg, longtime Interchange development group “team captain”.

Mark is going to be discussing full page caching in Interchange 5. This is becoming a more frequent request from our larger customers. They want to be able to do full page caching to allow the web browser and a caching proxy server alone to handle most requests leaving Interchange and the database open to handle more shopping-based requests like add to cart or checkout. This is a commonly-used architecture in many application servers, and my colleague David Christensen has several new features already in use by customers to make full-page caching easier, which are expected to go into Interchange 5.8.1 soon.

I will be doing a talk on multi-site setup in Interchange 5. This is a request we have received frequently over the years. Companies may either already have some kind of wholesale website or just want to have multiple websites use the same database and programming but allow for different website designs. They normally need to control what website a product will show up on and possibly adjust the price accordingly. I’ll discuss the different methods we have used to accomplish this at End Point.

I see on the schedule that Sam Batschelet will be speaking about the camps system and some new capabilities he’s added for perlbrew and Carton, among other things. We are also using camps some places with perlbrew and plenv, so it will be interesting to compare notes. I hope we’ll see some discussion and/or contribution to the open source DevCamps project soon!

It promises to be a very nice conference with lots of diverse information!

Challenges in testing Dancer 2.0 apps

2013-07-09T00:00:00+00:00

I’ve been dabbling in Dancer, and I managed to put together a moderately complex re-hosting of a web application with Dancer 1.0, including a preliminary package of unit tests via Test::More.

Spoiler alert: I don’t yet have a solution, but I thought maybe this blog post would organize my thoughts to where someone else might peek in and spot my problem.

A bit of introduction follows for those who may not have been down this path before.

Testing a web application can take one of two general approaches:

Test the underlying code by calling it directly, one program to another, with no web server involved. This is pretty straightforward, although you may have to rig up some replacements for the environment (such as if your code expects CGI parameters, or reacts to things in the web server environment such as cookies, remote IP address, etc.). In any case, you have to recognize that you are now testing the code logic, not the actual interaction of your code as a subsystem in the web server.
Test the web application in its “native environment”, by issuing requests to its associated web server and examining the responses (as web pages, JSON, what-have-you). This is much preferred, as it will catch all sorts of subtle bugs, if you haven’t exactly reproduced the environment (for instance, “correctly” spelling HTTP_REFERRER in your rigged-up environment, whereas the misspelled HTTP_REFERER is actually in use).

The module Dancer2::Test provides a small set of routines for supporting this second approach. For instance, you can verify that your routes are set up correctly via “route_exists”:

route_exists [ GET => '/index.html' ], "We can get the home page";

This behaves like the various test-helper routines in Test::More. If the test succeeds (in this case, a route is defined that matches the path), great; if not, an error message is reported using the string parameter.

You can also verify that the correct page has been delivered without errors:

my $response = dancer_response GET => '/something';
is $response->{status}, 200, '/something returns OK';
like $response->{content}, qr{Something from the /something page};

dancer_response is the preferred way to go about this, because it captures status, headers, and content in one operation, saving you time but also avoiding the potential for disturbing the state of your application by subsequent requests.

Now, this article’s title starts with “Challenge”, so I should let the other shoe drop. The Dancer2::Test setup doesn’t seem to have a way to preserve “state” on the server between requests. So if your application supports, for instance, a “login” page and a “modify my account” page, you don’t really have a way to test the latter page, as you can’t remain logged in.

Custom helper subs in Dancer templates

2013-03-30T00:00:00+00:00

I recently was writing some code using the Dancer Perl web framework, and had a set of HTML links in the template:

<a href="/">Home</a> |
<a href="/contact">Contact</a> |
[etc.]

Since it’s possible this app could be relocated to a different path, say, /something/deeper instead of merely /, I wanted to use Dancer’s handy uri_for() routine to get the full URL, which would include any path relocation. (This concept will be familiar to Interchange 5 users from its [area] and [page] tags.)

The uri_for function isn’t available in templates. The easiest way to cope would be to just use it in my route sub where it works fine, and store the results in the template tokens as strings. But then for any new URL needed I would have to update the route sub and the template, and this feels like a quintessential template concern.

I found this blog post explaining how to add custom functions to be used in templates, and it worked great. Now my template can look like this:

<a href="<% uri_for('/') %>">Home</a> |
<a href="<% uri_for('contact') %>">Contact</a> |
[etc.]

And the URLs are output fully qualified:

http://localhost:3000/
http://localhost:3000/contact

Which is not always what I’d want, but in this case is.

The only final concern is that I am using Dancer version 1.3111 and I got this warning upon using the before_template setup mentioned in the blog:

Dancer::before_template has been deprecated since version 1.3080. use hooks!
But use hook 'before_template' => sub {} now instead.

So I updated my code, and the final result looks like this:

hook 'before_template' => sub {
    my $tokens = shift;
    $tokens->{uri_for} = \&uri_for;
};

And that made both Dancer and me happy.

Perl Web Frameworks

CGI

Mojolicious

Interchange

Dancer (1 & 2)

Catalyst

Conclusion

2015 Perl Dancer Conference videos

Conference Recap

Conference Presentations

SpaceCamps “The Final Frontier”

AngularJS & Dancer for Modern Web Development

Lightning Talks

Summary

AngularJS & Dancer for Modern Web Development

Legacy Apps

Oh, to migrate

You are in a legacy codebase

Change

Crash

Move already

Front-End

Separation of Concerns

Enter AngularJS (or your favorite JavaScript framework)

Migration

Before

After

TMTOWTDI

Now starring Dancer

Dancer is better?

Dancer is better:

Lessons learned

Perl Dancer Conference 2015 Report—​Conference Days

Day One

Day Two

Summary

Perl Dancer Conference 2015 Report — Training Days

Documenting web services with Perl POD and AJAX

time

time

Filling in header elements with Dancer and Template::Flute

Cleaner redirection in Perl Dancer

Ecommerce Innovation 2013

A brief overview of the talks

First Dance: a Gentle Introduction to Dancer.pm for Web Services

eCommerce Innovation Conference 2013

Challenges in testing Dancer 2.0 apps

Custom helper subs in Dancer templates

Perl Dancer Conference 2015 Report—Conference Days