Interchange Redis Sessions

2026-06-16T00:00:00+00:00

Photo by Josh Ausborne, 2022.

Interchange¹ now supports storing user sessions in Redis², an in-memory data store, as a core feature of Interchange 5.12. Sessions are where Interchange keeps client state that makes a catalog operate as a continuous experience–the shopping cart, form values, login status, and other data that persist from one request to the next. How and where that state is stored have a direct impact on a busy catalog’s performance, and Redis gives developers a fast, purpose-built option that sits alongside the storage backends Interchange has always offered.

Session storage in Interchange

By default, Interchange writes each session to a file on disk, and it has long been able to store sessions in a relational database instead by way of Vend::SessionDB, configured with the SessionType DBI and SessionDB directives. The database approach is the natural choice once a catalog is served by more than one Interchange server, since every server needs to read and write the same pool of sessions.

Relational databases certainly can function in this capacity, but a high-traffic session table is not where they shine. Every page view reads, writes, and locks a session row, and that constant churn competes with database performance for the application. Redis is built precisely for this kind of small, hot, frequently updated key/value data, and that makes it a compelling choice for session storage.

Using Redis for sessions

Enabling Redis sessions takes two directives in your catalog’s catalog.cfg. Set SessionType to Redis, and point SessionDB at the address and port of your Redis server:

Variable REDIS_SERVER 127.0.0.1:6379
SessionType Redis
SessionDB   __REDIS_SERVER__

That is the whole of the required configuration. Behind the scenes, the new Vend::SessionRedis module ties Interchange’s session storage to Redis through the standard Redis CPAN module³, so Redis and that module must be installed and reachable from your Interchange host.

Redis sessions also work optionally with the session compression recently introduced.⁴ If you set SessionDBCompression to a viable compression algorithm, session data is compressed on the way into Redis and decompressed on the way out, exactly as it is for database sessions:

SessionDBCompression Zstd

More lists and MoreDBTable

Some catalogs use Interchange’s MoreDB feature to cache search-result sets (“more” lists) into the session database rather than writing to files on disk. This feature is often leveraged for the same reason that SessionDB is used: to support a distributed, load-balanced server environment. MoreDB relies on a real database table, and Redis–having no support in Interchange’s database abstraction layer–cannot provide one. To bridge that gap, there is a new MoreDBTable directive that tells MoreDB which relational table to use for more lists, independent of where sessions themselves live:

MoreDB yes
ifdef REDIS_SERVER
   include dbconf/__SESSIONDB_ENGINE__/sessions.__SESSIONDB_ENGINE__.__MOREDB_EXT__
   MoreDBTable sessions
endif

When you run Redis sessions together with MoreDB, MoreDBTable is required. Without Redis it remains optional: MoreDB falls back to SessionDB just as it always has. The strap demo ships ready-made schema definitions for MySQL, PostgreSQL, and SQLite, so the supporting table is created with the right column types.

Locking and a friendlier welcome for bots

Concurrent requests for the same session have to be serialized so two clicks don’t trample each other’s data. For this release, the RDBMS session backend was reworked to lock session records with proper database transactions rather than the old approach of inserting and deleting lock rows. Redis gets its own locking scheme built on Redis’s atomic SETNX (“set if not exists”) operation, exposed through a new has_lock() method. The mechanism is general enough that any other storage system able to offer the same primitive–Memcached, for example–could adopt it.

Redis sessions add one more practical refinement. Automated crawlers generate an enormous number of “hit and run” sessions that are created once and never used again, and on a database-backed catalog they pile up until something prunes them. Vend::SessionRedis instead gives every brand-new session a fixed 30-minute expiration. A human user comfortably makes a second request within that window, at which point the session is promoted to the full lifetime set by your SessionExpire directive. A bot that never returns simply expires on its own quickly, limiting data bloat, and Redis reclaims the memory without any housekeeping on your part.

Wrapping up

Redis sessions give Interchange developers a fast, horizontally scalable place to keep session state, with optional compression, a robust locking model, and built-in resistance to bot-driven session bloat–all configured with a couple of lines in catalog.cfg. As with the rest of Interchange, the implementation lives in core and is open for the community to extend; the has_lock() interface in particular invites support for additional storage backends.

References

Interchange Compression for SessionDB

2025-04-30T00:00:00+00:00

New support for compression of sessions and more lists stored in a RDBMS has been added to core Interchange.¹^,²

A new module, Vend::Util::Compress, operates as a general interface for compressing and uncompressing scalar data in Interchange. The module currently offers hooks for the following compression algorithms:

Zstd (preferred)
Gzip
Brotli

Additional algorithms can be easily added as needed.

Each algorithm depends on the developer having installed the necessary CPAN modules for the given compression type:

IO::Compress::Zstd/IO::Uncompress::UnZstd³
IO::Compress::Gzip/IO::Uncompress::Gunzip⁴
IO::Compress::Brotli/IO::Uncompress::Brotli⁵

Vend::Util::Compress exports compress() and uncompress() on demand. In scalar context, they return the reference to the scalar holding the transformed data. In list context, they return additional measurements from the process.

List compress() returns an array of:

$ref
$before_size
$after_size
$elapsed_time
$alert

List uncompress() returns an array of:

$ref
$elapsed_time
$alert

Any errors encountered when called in scalar context are written to the catalog error log. Errors encountered when called in list context are returned in $alert.

Compression Performance Comparison

The following chart summarizes the results of running the same Interchange instance through several steps of session growth to measure performance and impact of the three supported compression algorithms. Tests ranged from a minimum session size of 68 kilobytes up to a maximum of 110 megabytes. Fields marked in green indicate the best performance of the particular parameter, across all algorithms, for each given session size. Any fields marked in yellow indicate performance measurements reaching a level of concern. Any fields marked in red indicate measurements reaching a level of unacceptable performance in any typical situation.

For size reduction, all 3 algorithms produced a substantial decrease across the board, generally exceeding 90% and increasing the reduction factor as the session grew. All 3 algorithms also uncompressed the session on retrieval at comparable rates, well within tolerance. Where we see significant separation is in the time to compress. Zstd is the fastest across the board, but really stands out when working with larger sessions, when the impact of compression is most needed for reducing transfer latency between the Interchange and database servers.

Configuring Your Catalog to Use Vend::Util::Compress

To begin using session compression, add the new catalog configuration parameter, SessionDBCompression, set to one of the aforementioned compression algorithms. Note the values for specifying compression type to use are case-sensitive. For example, to enable Zstd:

SessionDBCompression Zstd

An empty value for the parameter bypasses compression. An invalid value logs an error and passes through the data unmodified. If the catalog is configured for file-based sessions, the setting has no impact.

The same compression applies to both sessions and more lists when MoreDB is also set.

References

Interchange 3rd Party Tax Support

2023-03-21T00:00:00+00:00

New 3rd-party tax API support has been added to core Interchange¹.

In the wake of the Wayfair court decision², many businesses running Interchange catalogs lack the necessary tools for full compliance. A new translation layer has been created in Vend::Tax to connect the standard sales tax structures and routines that operate within Interchange, and the development of vendor-specific 3rd-party tax providers. The goal of the Vend::Tax framework is to create a space to allow for development of any number of vendor-specific tax services to support tax calculation in Interchange.

Vend::Tax defines 3 new tags to support its function:

[tax-lookup]: Returns calculated tax amount determined by specific 3rd-party provider. Tax may be estimated or live lookup, depending on settings. Data required to calculate tax will be provider dependent.
[load-tax-averages]: Requests and stores tax averages for running in estimate mode, for providers that support it. Stores estimates by default in the tax_averages table. Further, allows for local tracking of jurisdictions with nexus, which can be used by live lookups to determine if a particular lookup can be skipped entirely. See load_tax_averages Job and tax_averages table definition in the Strap demo.
[send-tax-transaction]: Reports to provider the resulting tax transaction for a given order, for providers that support it. By default, operates based on transactions.tax_sent field. 0 or empty indicates transaction not reported. 1 indicates transaction successfully reported. -1 indicates an error attempting to report transaction, requiring manual intervention. See send_tax_transactions Job in Strap demo.

Conceptually, Vend::Tax is patterned off of Vend::Payment for payment transactions. It provides the defined interface that Interchange will use, via the 3 tags shown above, but must delegate to vendor-specific implementations through tax gateway modules.

Additionally, connectors have already been constructed for TaxJar³ and Avalara⁴ in core Interchange, so that catalogs can convert to use either of these popular services for calculating their sales tax with an upgrade and a few adjustments to their catalog configuration. Both TaxJar and Avalara provide sales tax calculation, product categorization, and reporting tools that allow merchants to comply with Wayfair and keep their focus on their business.

While currently only TaxJar and Avalara are supported — see Vend::Tax::TaxJar and Vend::Tax::Avalara — Vend::Tax was designed to facilitate and encourage the development of any 3rd-party providers through the creation of new Vend::Tax::<ServiceName> modules. A review of either of the TaxJar or Avalara tax gateway modules should be instructive for the expected interface with the 3 usertags.

For merchants who have, or wish to establish, relationships with other tax providers, they are free to build, or contract with consultants to build, their own provider’s vendor-specific module and use that to back the work performed within Vend::Tax. Any such development projects are encouraged to be submitted for inclusion in Interchange core, giving the merchant and developer the benefit of community improvements and core inclusion on future upgrades.

Configuring Your Catalog to Use Vend::Tax

In order to leverage [tax-lookup] to back the [salestax] tag, use it with the SalesTax directive. For example:

SalesTax  [tax-lookup service=TaxJar]

The service parameter should correspond with the segment of the namespace of the tax gateway module after Vend::Tax. Here, the example enables TaxJar in module Vend::Tax::TaxJar.

If your business requires sufficient customization, the existing tax gateway modules can be subclassed, keeping all your customizations separate from core code and safe from collisions during upgrades. An example of such an expansion to Avalara would be to create module Vend::Tax::Avalara::MyBiz to subclass Vend::Tax::Avalara and change the config setup to:

SalesTax  [tax-lookup service=Avalara::MyBiz]

Additionally, using the Strap setup, you can easily enable TaxJar specifically by setting the following variables in variable.txt:

TAXSERVICE TaxJar
TAXTOKEN [your TaxJar token]
NEXUS_ADDRESS [business address]
NEXUS_CITY [business city]
NEXUS_STATE [business state postal code]
NEXUS_ZIP [business zip]
NEXUS_COUNTRY [business ISO two-letter country code]

For Avalara in Strap, you also would set the TAXSERVICE and NEXUS_ variables, but then set your account API user and password:

TAXSERVICE Avalara
AVALARA_USER [your API user]
AVALARA_PASSWORD [your API password]
NEXUS_ADDRESS [business address]
NEXUS_CITY [business city]
NEXUS_STATE [business state postal code]
NEXUS_ZIP [business zip]
NEXUS_COUNTRY [business iso two-letter country code]

Finally, like the payment modules, in order to make a particular tax service available, it must be required from interchange.cfg. For example:

Require module Vend::Tax::TaxJar

References

Interchange rust_link connector

2023-03-06T00:00:00+00:00

The Interchange ecommerce system recently reached 27 years old, measuring from the first public release of its predecessor MiniVend by its creator Mike Heins. It is still hard at work in quite a few ecommerce sites, serving pages, accepting and processing orders, managing warehouse logistics, and more. That is quite an accomplishment in the software world!

The Interchange server/daemon

Interchange is written in Perl and runs on Linux and other Unix-like servers as a daemon (persistent background process) that listens for requests. Why does it need to run as a daemon?

Like many applications, Interchange starts with a relatively slow initialization procedure that takes a couple of seconds to compile code, load modules, read configuration, connect to databases, and validate everything. We want it to do that only once when the daemon is started, and not for each user request, so it can make quick responses.

Web server connector

General-purpose web servers normally sit in front of an application server, optimized to make speedy encrypted TLS sessions for HTTPS, control access to resources, log requests, redirect old URLs, route traffic to various applications, and directly serve static files such as HTML, CSS, JavaScript, and images.

How does the web server talk to the Interchange application server? Several protocols to do that would later become common, including:

FastCGI (1996)
Apache JServ Protocol or AJP (1997) for the Java world
HTTP protocol reverse proxying for Ruby on Rails and eventually most other platforms

But in 1995 when Interchange was created, there was no widely-used standard, so it used its own custom protocol implemented in a small CGI program.

The data flows through these steps:

A user makes a request with a web browser to a web server, commonly Apache httpd.
The web server runs a new instance of the link program and passes request information to it.
The link program reformats the request into Interchange link protocol.
It then sends the request to the Interchange server.
It receives back a response from Interchange.
It sends the response to the web server.
It exits.
The web server sends the response back to the user’s browser.

Interchange link protocol

The Interchange link protocol uses plain text divided into lines to pass its information.

There are 2 sections:

arg for command-line arguments, now rarely used, but long ago where search text appeared before the advent of HTML forms
env for environment variables.

After each section label is the number of items in that section.

Then that many items are given on one line each, beginning with the number of bytes in the value, followed by a space and then the value.

Finally an end section concludes the preliminary values, and then the HTTP response body, if any, follows unmodified after the entity label and its length in bytes.

Here is an example request with an empty request body, as the CGI link program passes it to Interchange:

arg 0
env 29
52 CONTEXT_DOCUMENT_ROOT=/opt/homebrew/var/www/cgi-bin/
24 CONTEXT_PREFIX=/cgi-bin/
35 DOCUMENT_ROOT=/opt/homebrew/var/www
25 GATEWAY_INTERFACE=CGI/1.1
97 HTTP_ACCEPT=text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
34 HTTP_ACCEPT_ENCODING=gzip, deflate
35 HTTP_ACCEPT_LANGUAGE=en-US,en;q=0.5
26 HTTP_CONNECTION=keep-alive
10 HTTP_DNT=1
23 HTTP_HOST=ruka.lan:8080
32 HTTP_UPGRADE_INSECURE_REQUESTS=1
86 HTTP_USER_AGENT=Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0
50 MINIVEND_SOCKET=/Users/user/interchange/etc/socket
18 PATH=/bin:/usr/bin
13 QUERY_STRING=
25 REMOTE_ADDR=192.168.1.215
17 REMOTE_PORT=36534
18 REQUEST_METHOD=GET
19 REQUEST_SCHEME=http
27 REQUEST_URI=/cgi-bin/strap1
52 SCRIPT_FILENAME=/opt/homebrew/var/www/cgi-bin/strap1
27 SCRIPT_NAME=/cgi-bin/strap1
24 SERVER_ADDR=192.168.1.59
28 SERVER_ADMIN=you@example.com
20 SERVER_NAME=ruka.lan
16 SERVER_PORT=8080
24 SERVER_PROTOCOL=HTTP/1.1
17 SERVER_SIGNATURE=
36 SERVER_SOFTWARE=Apache/2.4.56 (Unix)
end

Note that the CGI specification has HTTP request headers passed in environment variables beginning with HTTP_ followed by the header name with - replaced by _ and all letters capitalized.

The other environment variables give information about the request and context from the web server.

The environment variables appear in no particular order, but I sorted them here for readability.

Interchange link implementations

Several implementations of this link functionality exist:

Two sibling programs named vlink (for UNIX sockets in the filesystem) and tlink (for TCP sockets on the IP network), written in C and compiled, giving fast startup time and run speed
Perl equivalents named vlink.pl and tlink.pl which are easy to customize and need no compilation, but are comparatively slow, adding ~100 ms or more to each request
A Perl module Interchange::Link that runs precompiled in the Apache httpd module mod_perl and supports both UNIX and TCP sockets
A compiled C module mod_interchange that also runs in the Apache httpd server and supports both UNIX and TCP sockets

You can even run Interchange entirely inside Apache mod_perl such that no connector link program is needed. But that has been only rarely used in production because it tightly couples Interchange with the web server and means it runs the application as the web server operating system user, which is not ideal for security.

After all these years, the original vlink compiled C program remains the default and popular way to connect a web server via CGI to Interchange. It is simple to use, fairly fast, and works with any web server that supports the CGI protocol.

Enter Rust

The C vlink program isn’t broken, so we don’t really need to fix it. But occasionally we want to customize some part of the vlink behavior. Doing so in C can take more work than in more modern languages, depending on the programmer and what changes are needed.

But more importantly, programming in C introduces risk of bugs that could affect every request. C code is fertile ground for unsafe memory handling bugs that often lead to security problems.

Tim Hutt reviewed all the documented bugs of the curl project and found that over half were memory errors, and he noted that Google found that 70% of Chrome’s high-severity security bugs are memory errors. That is despite some of the world’s extremely talented and careful programmers working on those projects!

These are the kinds of errors that the programming language Rust eliminates by disallowing such code at compilation time. Plenty of other new languages have cropped up in the past decade or so, vying to replace C and/or C++ and be safe from memory errors while retaining speed and low-level machine access by producing compiled native machine code: Go, Swift, Zig, Nim, and V are a few competitors, alongside Rust.

Rust is one of the more complex ones because you have to think about memory management instead of relying on a garbage collector. But that also can make it one of the highest performers using the least amount of memory. Rust gets plenty of press, much of it enthusiastic, so you can go read about it on your own if you’d like to learn more!

So two years ago I decided to port the vlink program to Rust so I could see how much work it took, how pleasant it was, and how fast and memory-efficient the resulting code was. Porting the link program to one of those other new languages would also be fun, but I haven’t done that yet!

rust_link

I called this program rust_link. Initially I made it as simple as possible, just a proof of concept: It consisted of one long function, had a hardcoded path to the Interchange UNIX socket file, and it died ungracefully on any error.

It came together fairly quickly and was fun to write, so I made it configurable via environment variables that can be set in the web server, added error handling, and broke it into smaller functions like the original vlink has.

Then I came across the Rust crate (Rust’s term for a module) called multisock, which wraps up UNIX and INET socket support in a single package. Using that made it straightforward to merge vlink and tlink into a single program.

Most of the other link programs retry connecting to the Interchange socket every 1 or 2 seconds for something like 30 seconds. This is to handle gracefully a brief outage while the Interchange server is being restarted. However, if instead the Interchange server was overloaded or entirely down, this could lead to a big backlog of CGI link processes. I implemented this retry functionality too, and decided to have it check twice as often, but only for a maximum of 10 seconds. That still gives plenty of time for the link program to retry during a restart of the Interchange daemon, but lets it fail faster when the daemon isn’t likely to come back.

There is one HTML document in the program, for an HTTP 503 error response body. Putting that HTML text directly in the code would mean using pesky \ string escapes for quotation marks, newlines, etc., and then most text editors are unable to highlight the HTML syntax for readability. Since Rust has the easy include_str! macro, I put the HTTP 503 error response HTML body into a separate 503.html file for simpler editing.

Rust impressions

I was still new to Rust as I wrote this program, so there were various learning bumps along the way. It was a fun project. The Rust Programming Language book is excellent, and it is free online and also available in print.

One developer I read describing the experience of writing Rust code said that it can involve a lot of wrestling with the compiler to make it happy with variable lifetimes (borrowing and ownership), types, traits, and other details, but that once a program compiled, it usually worked without flaw.

I found that to most often be the case for me with rust_link. I had to correctly separate different sorts of data with distinct types for:

byte arrays for the data sent from and to the web server over stdin and stdout streams
“OS strings” for environment variable names & values and file paths & names
UTF-8 strings in source code, messages and string-link type conversions

This required more thoughtful consideration and work up front, but eliminated the possibility of things later going wrong because of sloppiness about exactly what kind of data we have in each case.

The rustc compiler is very helpful in pointing the way with detailed error messages noted in relevant excerpts of the problem code, and gives references to further reading in the documentation.

Testing in Rust

The Rust ecosystem and documentation aim to make testing a natural part of development, and as easy as possible. So I wrote unit tests to exercise some of my helper functions for representing numbers as strings for the wire protocol, parsing environment variable values into sockets, and getting errors where expected.

I also wrote a multi-threaded integration test to exercise sending the simplest response over a socket and verifying what is received on the other side. This is still very basic and could be fleshed out with more arguments, a response body with and without a CGI-specified CONTENT_LENGTH, etc.

Being able to run some tests is nice when first compiling the program for another platform, as I did here:

❯ cargo test
   Compiling rust_link v1.0.0 (/Users/user/repos/interchange/dist/src/rust_link)
    Finished test [unoptimized + debuginfo] target(s) in 1.48s
     Running unittests src/main.rs (target/debug/deps/rust_link-d008c1110239f0a1)

running 16 tests
test tests::get_entity_content_length_bad - should panic ... ok
test tests::get_socket_addr_from_env_host_ipv4 ... ok
test tests::get_entity_content_length_zero ... ok
test tests::get_entity_content_length_empty ... ok
test tests::get_socket_addr_from_env_host_ipv4_bad_host - should panic ... ok
test tests::get_entity_content_length_missing ... ok
test tests::get_socket_addr_from_env_host_ipv4_bad_port - should panic ... ok
test tests::get_socket_addr_from_env_host_ipv4_alt_port ... ok
test tests::number_to_text_bytes_0 ... ok
test tests::number_to_text_bytes_1 ... ok
test tests::number_to_text_bytes_259 ... ok
test tests::get_socket_addr_from_env_host_ipv6 - should panic ... ok
test tests::get_socket_addr_from_env_host_name ... ok
test tests::get_socket_addr_from_env_missing_vars - should panic ... ok
test tests::get_socket_addr_from_env_unix ... ok
test tests::send_arguments_output ... ok

test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.58s

Performance and resources

I developed rust_link on Linux (x86_64) and also built and tested it on macOS (ARM 64-bit) and Linux on Raspberry Pi (ARM 32-bit), where it also worked fine, as expected.

The speed of this new rust_link compares well to the classic vlink. Startup time is the main concern as there is no significant processing done once the program is running. Measurements are essentially the same, within the margin of error.

Some of the link program implementations use a limited buffer size (such as 16 KiB for vlink and tlink) to spool data from Interchange back to the web server. It is common practice to send large response bodies as files directly from a web server rather than from dynamic Interchange responses to limit the time an Interchange worker process is occupied. Because of that, and since most servers have far more memory available now than in years past, I let rust_link read the entire response body into memory rather than using a fixed buffer size.

Because Rust crates are compiled into the rust_link executable, not linked as shared libraries as is typical with C programs, the executable program size is bigger for rust_link than vlink: 9× more on disk under ARM64 macOS! But that increase is from a small base of ~34 KiB to ~312 KiB. (This is for executables stripped of debugging symbols.) On modern operating systems with copy-on-write behavior, most of that memory is shared so it is a one-time cost, not multiplying per running CGI program instance.

Publication

Since it works as well as the other implementations, I figured it was worth finally adding some documentation and releasing it as open source software alongside the other Interchange link programs, available for anyone who wants to avoid the risks of C to deploy with or without customization.

The code is on GitHub ready to use and the rust_link/README.md explains how to build and install it.

Reference

Interchange ecommerce project
Common Gateway Interface (CGI)
- RFC 3875 section 4.4 which defines the ancient and now almost completely defunct convention for CGI script command line arguments to contain an “indexed” HTTP query
FastCGI
Apache JServ Protocol (AJP)
HTTP reverse proxy
Rust programming language
- Would Rust secure cURL? by Tim Hutt
- Rust crate multisock
- From Perl to Rust

Perl Web Frameworks

2022-04-19T00:00:00+00:00

CGI

When I started programming, back in the day, CGI (the Common Gateway Interface) was still widely used. Usually the Apache webserver would just execute a script or a binary with some environment variables set and serve whatever the executable sent to the standard output, while keeping the standard error in the logs.

This simple and straightforward mechanism can still be used for small programs, but larger applications usually want to save the start-up time and live longer than just a single request.

At that time Perl was used far more often than now, and it had (and still has) the CGI.pm module to help the programmer to get the job done.

#!/usr/bin/env perl

use utf8;
use strict;
use warnings;
use CGI;

my $q = CGI->new;
print $q->header;
my $name = 'Marco';
print $q->p("Hello $name");
print "\n";

And it will output:

./cgi.pl
Content-Type: text/html; charset=ISO-8859-1

<p>Hello Marco</p>

Here the script mixes logic and formatting and the encoding it produces by default tells us that this comes from another age. But if you want something which is seldom used and gets executed on demand without persisting in the machine’s memory, this is still an option.

Please note that there are frameworks which can work in CGI mode, so there is no reason to use CGI.pm, beside having to maintain legacy programs.

Mojolicious

Fast-forward to 2022.

Nowadays Perl is just another language among dozens of them. But it still gets the job done and lets you write nice, maintainable code like any other modern language.

Mojolicious is currently the top choice if you want to do web development in Perl. It is an amazing framework, with a large and active community, and appears to have collected the best concepts that other web frameworks from other languages have to offer.

Let’s hack an app in a couple of minutes in a single file, like during the CGI days:

#!/usr/bin/env perl
use utf8;
use strict;
use warnings;

use Mojolicious::Lite;

get '/' => sub {
    my ($c) = @_;
    $c->stash(name => "Marco");
    $c->render(template => 'index');
};

app->start;

__DATA__
@@ index.html.ep
Hello <%= $name %>

Here the structure is a bit different.

First, there’s a Domain Specific Language (DSL) to give you some sugar. This is the “Lite” version, while in a well-structured Mojolicious app one prefers to write class methods. We declare that the root (/) URL path of the application is going to execute some code. It populates the “stash” with some variables, and finally renders a template which can access the stashed variables. If you execute the script, you get:

./mojo.pl cgi 2> /dev/null
Status: 200 OK
Content-Length: 12
Date: Fri, 08 Apr 2022 12:33:52 GMT
Content-Type: text/html;charset=UTF-8

Hello Marco

The logging to the standard error stream is:

[2022-04-08 14:33:52.92508] [163133] [debug] [82ae3iV2] GET "/"
[2022-04-08 14:33:52.92532] [163133] [debug] [82ae3iV2] Routing to a callback
[2022-04-08 14:33:52.92565] [163133] [debug] [82ae3iV2] Rendering template "index.html.ep" from DATA section
[2022-04-08 14:33:52.92610] [163133] [debug] [82ae3iV2] 200 OK (0.001021s, 979.432/s)

This is basically what a modern framework is supposed to do.

The nice thing in this example is that we created a single-file prototype and launched it as a CGI. But we can also launch it as daemon and visit the given address with a browser, which is how you should normally deploy it, usually behind a reverse proxy like nginx.

./mojo.pl daemon
[2022-04-08 14:48:42.01827] [163409] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000
[2022-04-08 14:48:48.53687] [163409] [debug] [CwM6zoUQ] GET "/"
[2022-04-08 14:48:48.53715] [163409] [debug] [CwM6zoUQ] Routing to a callback
[2022-04-08 14:48:48.53752] [163409] [debug] [CwM6zoUQ] Rendering template "index.html.ep" from DATA section
[2022-04-08 14:48:48.53808] [163409] [debug] [CwM6zoUQ] 200 OK (0.001209s, 827.130/s)

If you want you can even launch it with HTTPS as well (please note the syntax to pass the certificates).

./mojo.pl daemon -l 'https://[::]:8080?cert=./ssl/fullchain.pem&key=./ssl/privkey.pem' -m production

For a small application listening on a high port this is already enough and the whole deployment problem goes away.

Speaking about deployment, Mojolicious has basically no dependencies other than the core modules and comes with a lot of goodies, for example a non blocking user-agent.

Recently a legacy application needed to make some API calls. To speed up the process, we wanted to make the requests in parallel. And here’s the gist of the code:

package MyApp::Async;

# ... more modules here

use Mojo::UserAgent;
use Mojo::Promise;

# .... other methods here

sub example {
    my $email = 'test@example.com';
    my $ua = Mojo::UserAgent->new;
    foreach my $list ($self->get_lists) {
        my $promise = $ua->post_p($self->_url("/api/v2/endpoint/$list->{code}"),
                                  json => { email => $email })
          ->then(sub {
                     my $tx = shift;
                     my $res = $tx->result;
                     if ($res->code =~ m/^2/) {
                         $self->_update_db($data);
                     }
                     else {
                         die $tx->req->url . ' ' . $res->code;
                     }
                 });
        push @promises, $promise;
    }
    my $return = 0;
    Mojo::Promise->all(@promises)->then(sub { $return = 1 }, sub { $return = 0})->wait;
    return $return;
}

So a bunch of requests are run in parallel and then synced before returning. Does it remind you of JavaScript? Of course. A lot of common paradigms taken from other languages and frameworks were implemented here, and you can find the best of them in this nice package.

But the point here is that it doesn’t need dozens of new modules installed or upgraded. It’s just a single module in pure Perl that you can even install in your application tree. This is a huge advantage if you’re dealing with a legacy application which uses an old Perl tree and you want to play safe.

So, if you’re starting from scratch, go with Mojolicious. It lets you prototype fast and doesn’t let you down later.

However, starting from scratch is not always an option. Actually, it’s a rare opportunity. There’s a whole world of legacy applications and they generate real money every day. It’s simply not possible or even desirable to throw away something that works for something that would do the same thing but in a “cooler” way. In ten years, the way we’re coding will look old anyway.

Interchange

Wait. Isn’t Interchange an old e-commerce framework? Yes, it’s not exactly a generic web framework, on the contrary, it’s a specialized one, but it’s still a framework and you can still do things in a maintainable fashion. The key is using the so-called action maps:

ActionMap jump <<EOR
sub {
    # get the path parameters
    my ($action, @args) = split(/\//, shift);

    # get the query/body parameters
    my $param = $CGI->{param};

    # redirect to another page
    $Tag->deliver({ location => $final });

    # or serve JSON
    $Tag->deliver({ type => 'application/json', body => $json_string });

    # or serve a file
    $Tag->deliver({ type => 'text/plain', body => $bigfile });

    # or populate the "stash" and serve a template page
    $Tag->tmp(stash_variable => "Marco");
    $CGI->{mv_nextpage} = "test.html";
}
EOR

In pages/test.html you would put this template:

<p>Hello [scratch stash_variable]</p>

Now, I can’t show you a simple script which demonstrates this and you’ll have to take my word for it since we can’t go through the installation process here for a demo.

Interchange is old, and it shows its years, but it is actively maintained. It lacks many of Mojo’s goodies, but you can still do things in a reasonable way.

In the example the code will execute when a path starting with /jump/ is requested. The whole path is passed to the routine, so you can split at /, apply your logic, and finally either set $CGI->{mv_nextpage} to a file in the pages directory or output the response body directly with deliver. This way you can easily build, as a classical example, an API.

It’s a bit of a poor man’s MVC but it works. That’s basically the core of what a framework like Dancer does.

Dancer (1 & 2)

Dancer is basically Ruby’s Sinatra ported to Perl. As already mentioned, ideas developed in other languages and frameworks are often ported to Perl, and this is no exception.

Let’s see it in an action:

#!/usr/bin/env perl
use strict;
use warnings;
use Dancer2;

get '/' => sub {
    my $name = "Marco";
    return "Hello $name\n";
};

start;

Start the script:

Dancer2 v0.400000 server 22969 listening on http://0.0.0.0:3000

Try it with curl:

$ curl -D - http://0.0.0.0:3000
HTTP/1.0 200 OK
Date: Mon, 11 Apr 2022 07:22:18 GMT
Server: Perl Dancer2 0.400000
Server: Perl Dancer2 0.400000
Content-Length: 12
Content-Type: text/html; charset=UTF-8

Hello Marco

If in the script you say use Dancer; instead of use Dancer2, you get:

$ curl -D - http://0.0.0.0:3000
HTTP/1.0 200 OK
Server: Perl Dancer 1.3513
Content-Length: 12
Content-Type: text/html
X-Powered-By: Perl Dancer 1.3513

Hello Marco

Dancer’s core doesn’t do much more than routing. And you’ll also notice that the syntax is very similar to Mojolicious::Lite. So to get something done you need to start installing plugins which will provide the needed glue to interact with a database, work with your template system of choice, and more.

Today you would wonder why you should use Dancer and not Mojolicious, but when Dancer was at the peak of its popularity the games were still open. There were plenty of plugins being written and published on CPAN.

Around 2013 Dancer’s development team decided to rewrite it to make it better. The problem was that plugins and templates needed to be adapted as well. I’m under the impression that the energy got divided and the momentum was lost. Now there are two codebases and two plugin namespaces which do basically the same thing, because for the end user there is not much difference.

Catalyst

So what was attracting people to Dancer? When Dancer came out, Perl had a great MVC framework, which is still around, Catalyst. (And note that the main Mojolicious developer was on the Catalyst team.)

Now, the problem is that to get started with Catalyst, even if it has plenty of documentation, you need to be already acquainted with a lot of concepts and technologies. For example, the tutorial starts to talk about Template Toolkit and the DBIx::Class ORM very early.

These two modules are great and powerful and they deserve to be studied, but for someone new to modern web development, or even to Perl, it feels (and actually is) overwhelming.

So, why would you choose Catalyst today? Catalyst has the stability which Mojo, at least at the beginning, lacked, while backward compatibility is a priority for Catalyst. The other way to look at this is that Catalyst doesn’t see much current development, but someone could see this as a feature.

Even if Catalyst predates all the hyper-modern features that Mojo has, it’s still a modern framework, and a good one. I can’t show you a self contained script (you need a tree of files), but I’d like to show you what makes it very nice and powerful:

package MyApp::Controller::Root;

use Moose;
use namespace::autoclean;

BEGIN { extends 'Catalyst::Controller'; }

# start the chain with /foo/XX
sub foo :Chained('/') CaptureArgs(1) {
    my ($self, $c, $arg) = @_;
    $c->stash(name => "$arg");
}

# /foo/XX/bar/YY
sub bar :Chained('foo') Args(1) {
    my ($self, $c, $arg) = @_;
    $c->detach($c->view('JSON'));
}

# /foo/XX/another/YY
sub another :Chained('foo') Args(1) {
    my ($self, $c, $arg) = @_;
    $c->detach($c->view('HTML'));
}

So, if you hit /foo/marco/bar/test the second path fragment will be processed by the first method (CaptureArgs(1)) and saved in the stash. Then the second bar method will be chained to it and the name will be available in the stash. The last method will be hit with /foo/marco/another/test2. (Incidentally, please note that Mojolicious has nested routes as well.)

Now, I think it’s clear that in this way you can build deep hierarchies of paths with reusable components. This works really great with the DBIx::Class ORM, where you can chain queries as well. As you can imagine, this is far from a simple setup. On the contrary, this is an advanced setup for people who already know their way around web frameworks.

Conclusion

So, to sum up this excursion in the amazing land of Perl web frameworks: If you build something from scratch, go with Mojolicious. It’s your best bet. If nothing else, it’s super-easy to install, with basically no dependencies.

However, there’s no need to make a religion out of it. Rewriting code without a clear gain is a waste of time and money. A good developer should still be able to write maintainable code with the existing tools.

Our Vue Storefront “Proof of Concept” Experience

2020-08-10T00:00:00+00:00

Recently we experimented internally with integrating Vue Storefront and Magento 2.3. Vue Storefront is an open source Progressive Web App (PWA) that aims to work with many ecommerce platforms.

What initially piqued our interest was the possibility of integrating Vue Storefront with the venerable ecommerce back-end platform Interchange, which many of our clients use. Vue Storefront’s promise of ease of integration with any ecommerce backend made us curious to see whether it would make a good modern front-end for Interchange.

Since Vue Storefront seems to be most commonly used with Magento, we decided to start our experiment with a standard Vue Storefront/Magento 2.3 proof-of-concept integration.

PoC of Vue Storefront/Magento 2.3

OK, to be honest, at the beginning we blindly expected that Vue Storefront would be a copy/paste front-end template solution that would fairly easily be made to work with its standard integration to a Magento backend. Sadly, this was not the case for us.

Before beginning our journey here, to summarize the Vue Storefront integration with Magento let’s have a look at this diagram to see what components are included:

Figure 1

At first, we wanted to see how all these components can be installed and run on a single server with modest resources.

I walked through the Vue Storefront documentation and a few blog posts to figure out Vue Storefront and Magento integration.

Preparing the Environment

I downloaded and installed the following:

OS: CentOS Linux 7 (64-bit)
PHP 7.2.26
Magento 2.3 with sample data
Elasticsearch 5.6
Redis
Docker
Vue Storefront API
Vue Storefront
mage2vuestorefront bridge

Installation of those components is fairly easy. We started our virtual server with 4 GB of memory, which we thought should be plenty for a toy setup.

Indexing Elasticsearch with mage2vuestorefront

PHP and Magento 2.3 had a series of memory usage issues. While running the mage2vuestorefront indexer Magento used most of the memory and caused Elasticsearch to go down each time I tried to index Elasticsearch. Then I configured PHP and Apache httpd server to use a very modest amount memory and made Magento 2.3 work on this server without making it crash due to unavailable memory but the performance became a nightmare on the PHP & Magento 2.3 side. mage2vuestorefront ran without issue, but very slowly.

I am not very familiar with the Magento 2 API, but while indexing Elasticsearch with mage2vuestorefront it makes several individual API calls. Especially if you have several products in Magento 2, mage2vuestorefront calls the Magento API for almost all products. Due to PHP and Magento’s high memory usage available memory might become a problem during indexing.

The Vue Storefront developers explain a big reason why they chose Elasticsearch as one of the essential components of this integration:

Vue Storefront defines itself backend-agnostic PWA e-commerce solution where Vue Storefront is a storefront as the name dictates, and Elasticsearch works as a datastore for catalog and its sibling data such as taxrule, products and so on.

I am unsure if the backend-agnostic approach even really applies to the suggested implementation of Vue Storefront with Magento 2, or if does apply with significant headaches. I’ll detail my concerns about this later.

Viewing the Vue Storefront Application

The responsive and offline supportive design of Vue Storefront is elegant. It uses Elasticsearch via vue-storefront-api as its indexing search engine to provide a seamless user experience. Although the overall integration and the produced data structure looks complex there are only a few catalog data stores that need to be indexed on Elasticsearch. Also, since this is an ecommerce integration, you can expect that only products and product-related data are going to be available in such an integration.

In the Vue Storefront and Vue Storefront API app home directories (vue-storefront/ and vue-storefront-api/) I ran npm start to start the Node.js apps, then navigated to the Vue Storefront homepage to view its default template:

Figure 2

There is a hamburger menu in the top-left which opens the categories menu of Vue Storefront:

Figure 3

As I mentioned, the data being viewed is relatively simple according to the count of the catalog types (essentially Products, Categories, Attributes, and Tax Rules) which are already categorized and these are a bunch of specific sets of data.

By searching through the categories Women > Tops > Jackets, the search output list is shown by Vue Storefront:

Figure 4

Designing a New Approach for Interchange

When exploring the features and components of Vue Storefront we discussed how we can adapt Interchange to use Vue Storefront. We ended up with two main options:

Keep the same approach of Vue Storefront’s current integration with Magento and other ecommerce backends using Elasticsearch and vue-storefront-api as the backend gateways
Remove the Elasticsearch and vue-storefront-api dependencies and create a new API in Interchange producing similar API outputs to the ones vue-storefront-api & Elasticsearch are creating. I also found a similar approach where they integrated SAP Hybris with Vue Storefront. I thought that this way would fit our intention.

Inspired By A Similar Approach of Vue Storefront + SAP Commerce

Vue Storefront + SAP Commerce: Open-source PWA Storefront Integration (+DEMO)

This article discusses several approaches, including Vue Storefront developers’ boilerplate integration and their custom solutions. In the following table they show different options and comments on their resolutions.

Figure 5

As you can see in Figure 5, they decided to go with the 4th option as their most suitable and least problematic solution. Here is their statement of why:

After some analysis, we have come to believe that using custom middleware only for a bunch of simple requests is an additional burden and thick layer of complexity. We decided to get rid of it as well.

So, option #4 turned out to be the best choice. For the option #4, we need to parse Magento and Elasticsearch APIs and generate the compatible responses.

Here is how their solution looks according to option #4:

Figure 6

Using this approach, they decided to remove the Elasticsearch and Vue Storefront API components from the Vue Storefront boilerplate integration and developed their SAP Commerce custom Vue Storefront API to produce responses similar to the responses taken from Elasticsearch through Vue Storefront API to pretend that Vue Storefront is taking its responses from Elasticsearch API + Magento API.

We decided to discard Elasticsearch when integrating Interchange with Vue Storefront for simplicity, and to cut down on the cost such additional search solutions create.

After our discussions, I was inspired by this SAP Commerce + Vue Storefront integration demo. We finally decided to move forward with creating an Interchange API to produce the similar responses to those from the Elasticsearch & Magento APIs, so we could make the Interchange API pretend to be the Elasticsearch API + Magento API.

We need to mention the Magento 2 API, or at least the Magento 2 data structure, because the Vue Storefront API and Vue Storefront boilerplate code is heavily designed to be taking ecommerce data (Products, Categories, Product Attributes, Tax Rules etc.) almost identical to the Magento API’s provided data.

So our Interchange + Vue Storefront integration looks like this:

Figure 7

Compare Figure 1 with Figure 7 to see the difference between this approach and Vue Storefront’s suggested solution.

Interchange API Prototype Design

In order to achieve the API prototyping for our Interchange/Vue Storefront integration, I cloned vue-storefront and vue-storefront-api, which I had previously integrated with Magento 2.3, into a new user, keeping all settings the same. This cloned Vue Storefront application started initially with the same configuration as vue-storefront-api and Elasticsearch were not excluded yet. So I decided to replace the vue-storefront-api endpoints that vue-storefront is calling with endpoints from the new Interchange API.

As the default SAP Commerce/Vue Storefront integration suggests, I walked through the outputs of search results in “product”, “category”, and “attributes”. Product data JSON output has some universal attributes like name or title, description, or price, but it also has some custom product attributes adapted from the Magento 2 API (specifically, the data structure of Magento is applied).

At this point I started questioning if Vue Storefront is not really platform agnostic, or if it is, but with some headaches. The next section will detail these headaches but to summarize, adapting the data structure of new ecommerce backends to Vue Storefront is a real pain. It forces you to use its data structure since that’s a part of core development, and they rely mainly on Elasticsearch, and I guess because of that the developers of Vue Storefront believe that Elasticsearch is an essential part of this “agnostic” approach. Refer to Getting data from Elasticsearch for custom implementations using the core Elasticsearch lib.

Anyway, I moved on and produced product and product categories outputs that Vue Storefront expects as the search results at the homepage and the search box output results.

Documentation is another pain. This is discussed further in the next section, but briefly, it’s hard to say that you could clearly pick the data structure that all kind of output results from the documentation. You need to dig into the TypeScript code, and the exact search output result is what is actually being expected.

One conflict was that the identifiers are required to be numeric, not strings. If your ecommerce data structure doesn’t provide its identifiers as numbers you’ll need to alter and add a new numeric unique identifier along with the existing one, which is what I did.

Product attributes also need to be adapted to Vue Storefront (really to Magento’s data structure). The attribute names, attribute lists, attributes’ assigned values, and categories attached to the products and configurable child products all come along with the product list on a search output. If your ecommerce data is not structured similarly to Magento’s data structure, you need to dig and restructure even for minimal changes like product attributes and configurable child products. For example, if your data structure doesn’t allow configurable child products then you can probably only provide each option (such as product color or size options) as a new product, which will be a real pain.

Most of Vue Storefront’s app endpoints are defined in config/default.json which is the default config on a new install. On the other hand, the ecommerce catalog data (which is indexed in Elasticsearch) is available through the Elasticsearch implementation which is by default defined in lib/search/adapter/api/searchAdapter.ts. In my case I needed to alter this implementation, which is generating the Elasticsearch (actually the Vue Storefront API) endpoints dynamically by looking up the related catalog index (product, category, attributes, taxes, etc.), and change the Elasticsearch definition in the Vue Storefront config file to the new Interchange API, adapted to emulate the Vue Storefront API and Elasticsearch:

...
  "elasticsearch": {
    "httpAuth": "",
    "host": "/api/catalog",
    "index": "vue_storefront_catalog",
...

I needed to set the Interchange API URL to the host property and not necessarily set an API-defining URL sub-path name to the index property above. This way our API would pretend to be the Elasticsearch endpoint as long as it can generate the JSON output that Vue Storefront expects.

These configuration changes generally satisfied Vue Storefront’s expectations.

Common and Exceptional Issues

Many concerns and challenges are discussed in this article on Vue Storefront + SAP Commerce integration which I mentioned earlier.

A review of some issues I detailed earlier:

Required numeric unique IDs for all catalogs (products, categories, etc.)
Poor documentation
Catalog data structure is managed on the Vue Storefront side, heavily relying on its own data types and models. Your backend may need to be adapted to meet Vue Storefront’s expectations, even if you are using Elasticsearch.
A structure which is constantly being reorganized, creating a big maintenance concern.

Conclusion

Despite my concerns, I thought we could still implement our Interchange API with a front-end framework, but I hadn’t asked the question: Is it reusable enough to mantain? Each of our clients would have different UI and UX expectations. One UI template model could never be enough, which would mean continuous front-end template design and enough maintenance to match the operational cost of existing Interchange front-end solutions. Adapting existing data to Vue Storefront’s new data model would bring more costs along with it.

While Vue Storefront’s offline working model is well designed, for the purpose of integrating with a non-Magento backend, this is overshadowed by its many issues. You would be better off looking elsewhere for a more modular, truly platform-agnostic, easier-maintained, component- and data model-independent front-end.

Ecommerce sales tax primer

2020-01-13T00:00:00+00:00

Co-authored by Greg Hanson

Source image

Tax collection is one of the topics du jour for those of us in the ecommerce industry. Since state and local authorities are now able to levy taxes on ecommerce goods, taxation for online stores has become quite complicated. The purpose of this post is to give you some next steps and ideas on implementation if you’re new to the topic and need to know how to get started on tax collection for your ecommerce business.

Current ecommerce sales tax policy stems from the 2018 U.S. Supreme Court decision South Dakota v. Wayfair, Inc. Since that decision, favoring South Dakota, 30 states have enacted legislation to require ecommerce stores to pay sales tax if they fit the definition of having an ‘economic nexus’, that is, they do enough business in the state to be worth taxing.

Talk to your Tax Accountant

So the first and most important note is to get your own legal counsel in regards to your taxes. There are many rules and things are changing every month with local and state authorities, so you’ll need reliable counsel on the topic.

If you’re looking for someone to help, make sure this person has:

Knowledge about product variants. For example, clothing may not be taxed in some areas.
Familiarity with tax policy in the entire country, and not just one local retail area.
The ability to help you determine in which states you have a tax ‘nexus’ and need to pay sales tax. For many small businesses, you might only do enough business to pay sales tax in your home state and a few large ones like California or New York.

Research Software

Luckily for anyone starting to collect sales tax, there are some very good SaaS (software as a service) systems out there to make tax collection, reporting, and compliance easy. These software systems typically integrate with your ecommerce store by providing the store with the correct tax amount for the sale and collecting information for you on your reports and filing information to get the taxes paid to the correct authorities. Some systems might even file things for you.

After you’ve worked with your tax accountant on what you need, start looking at the available software. The two main companies End Point has worked with are TaxJar and Avalara, though there are a number of other vendors in the market.

Why use an automated tax solution?

They automatically update tax rates as they change with local and state regulation.
They can integrate into your checkout processes via API or plugin to automatically calculate the correct tax for the right location and product.
They can have product-specific taxes, for things that might not be taxed like food and clothing in certain states.
They give you end-of-year reports and help with your payments.
You can customize settings, in case there are some states where you have a ‘nexus’ (are subject to sales tax) and other places where sales tax payments are not required.

What to look for in automated tax software:

Choose a reputable company with large brand presence. You don’t want to do anything experimental and unreliable here.
Pick something with documentation you can understand. You don’t want working with your tax software to be a pain, and you might need to refer to their documentation when configuring it or changing things in the future. Make sure that company speaks your language and you can easily get answers to your questions.
This company should be able to guide you through all aspects of tax collections, filing and payment. You will legally be liable for taxes on all sales in states where you have nexus, whether you collect the taxes or not! So make sure the company you select can work with you not only to set up the system, but on an ongoing basis providing support in filing reports and making payments.
Integration with your platform is a key component of what you choose. Both Avalara and TaxJar have existing plugins for sites running on WordPress, Shopify, BigCommerce, Magento, and others. Keep in mind though that the integration might be different depending on your platform. If you’re on a custom platform, talk to your development team about integrations; they can read the docs and give you an estimate and recommendation for ease of implementation (that’s where we come in for many clients). For Interchange stores, we have integration code for both TaxJar and Avilara to leverage.
Consider how your inventory or ERP system might be affected. Many of our ecommerce clients sell in-house or over the phone. Consider how your other systems might need to tie into this new tax system.

Implementation

So you’ve done the hard part, right? You sorted out what states you need to be compliant with and picked a software solution. Now all you need to do is get it working. This is really where your software consultant, such as us at End Point, would come in and get you to the finish line. The steps to implementation are:

Set up your account and pay for your tax solution software.
Work with the tax solution provider to set up any required bank accounts or payment channels.
Configure your settings.
Share the API key and information with your developer.
Test the implementation. I recommend you do this with several orders in different scenarios: products that do and don’t have sales tax, locations that do and don’t have tax, locations that tax shipping, etc. Test all the variants you know about.
Go live with your tax solution on your site.
Make sure to check back for your reports and filings for later in the year.

Need help picking a system or looking at implementation? Call us today and we can help.

Other resources

National Braille Press Releases Redesigned Website

2019-08-28T00:00:00+00:00

End Point congratulates National Braille Press on the recent release of their redesigned website, launched earlier this week! National Braille Press (NBP) supports the blind and low-vision community with events, initiatives, resources, children’s programs, and a bookstore. NBP has been working with End Point since 2002 and hosts and maintains their website, which is based on the Interchange open-source ecommerce platform.

Project Phases

End Point started working with NBP on this redesign project in late 2017. We began with a user experience (UX) review & content strategy project which resulted in design directives for our in-house team. Design and plans were completed in mid-2018 and implementation began then. End Point built a new, more powerful search feature and developed a number of ‘cornerstone’ pages while NBP staff worked tirelessly in-house to convert the rest of the pages on their website to the new designs.

Leveraging NBP In-House Resources

NBP’s wide reach in their community and long history means their website has hundreds of content pages. This can be costly to convert, and for some non-profits, can make a redesign project cost-prohibitive. End Point engineers Greg Davidson and Marco Pessotto worked alongside NBP staff helping them learn the Bootstrap web development toolkit and the steps they needed to take to integrate each page into the new designs. Using these in-house resources dramatically reduced the overall project cost while allowing the content to be converted carefully by hand with accuracy and care.

WCAG 2.0, AA Standard

NBP’s user base is heavily centered in the low-vision community, so compliance with user accessibility standards was a main focus of the redesign effort. End Point worked with NBP to make sure all of the engineers on the project were trained in the Web Content Accessibility Guidelines (WCAG 2.0) AA standards and guidelines. Testing for compliance was done with AXE and WAVE Chrome accessibility plugins. NBP also completed direct user testing with screen readers and devices.

Congrats again to NBP on completing this project with us!

Client Case Study: Vervante—Publishing, Production and Fulfillment Services

2017-08-28T00:00:00+00:00

A real-life scenario

The following is a real-life example of services we have provided for one of our clients.

Vervante Corporation provides a print on demand and order fulfillment service for thousands of customers, in their case, “Authors”. Vervante needed a way for these authors to keep track of their products. Essentially they needed an Inventory management system. So we designed a complete system from the ground up that allows Vervante’s authors many custom functions that simply are not offered in a pre-built package anywhere.

This is also a good time to mention that you should always view your web presence, in fact your business itself, as a process, not a one time “setup”. Your products will change, your customers will change, the web will change, everything will change. If you want your business to be successful, you will change.

Some Specifics

While it is beyond the scope of this case study to describe all of the programs that were developed for Vervante, it will be valuable for the reader to sample just a few of the areas to understand how diverse a single business can be. Here are a few of the functions we have built from scratch, over several years to continue to provide Vervante, their authors, and even their vendors with efficient processes to achieve their daily business needs.

Requirements

Author Requirement—First, in some cases, the best approach to a problem is to use someone else’s solution! Vervante’s authors have large data files that are converted to a product, and then shipped on demand as the orders come in. So we initially provided a custom file transfer process so that customers could directly upload their files to a server we set up for Vervante. Soon Vervante’s rapid growth outpaced the efficacy of this system, so we investigated and determined the most efficient and cost-effective approach was to incorporate a 3rd party service. So we recommended a well known file transfer service and wrote a program to communicate with the file transfer service API. Now a client can easily describe and upload large files to Vervante.

View File Save Process

Storage Requirement—The remote storage of these large files caused Vervante a dramatic inefficiency as relates to access times, as they worked daily on these files to format, organize, and create product masters. So we needed to provide Vervante with a local server to act as a file server that was on their local network (LAN), where it could be rapidly accessed and manipulated. This was a challenge, as Vervante did not have IT personnel on site. So we purchased an appropriate server, set up everything in our offices, and shipped the complete server to them! They plugged the server into their local network, and with a long phone call, we had the server up and running and remotely managed.
Author Requirement—On the website, the authors first wanted to see what they had in inventory. Some customers provided Vervante with some product components that needed to be included with a complete product, while others relied on Vervante to build all components of their products. They also requested a way to set minimum inventory stock requirements.

So we built an interface that would allow authors to:

(a) See their current stock levels for all products,

(b) View outstanding orders for these items,

For example a finished product may consist of a book, a CD, and a DVD. A customer may supply the CD and require Vervante to produce the book and the DVD “on demand” for the product. We created a system that tracked all items at a “base” item level, and then allowed Vervante to “build” products with as many of these “base” items as necessary, to create the final product. The base items could be combined to create an item, and two or more items could be combined to produce yet another item. It is a recursive item inventory system, built from scratch specifically for Vervante.

Vervante Vendor (fulfillment warehouse) Requirement—Additionally, the fulfillment warehouse that receives, stores, builds and ships end user products, needed access to this system. They had several needs including:
- Retrieving pending orders for shipment
- Creating packing / pick slips for the orders
- Create shipping labels for orders
- Manage returns
- Input customer supplied inventory
- Input fulfillment created inventory

In our administrative user interface for the fulfillment house, we developed a series of customer specific processes to address the above needs. Here is a high level example of how a few of the items on the list above are achieved:

- The fulfillment house logs into the user admin first thing in the morning, and prints the outstanding orders.
- The “orders” are formatted similar to a packing slip, and each slip has all line items of the order, and a bar code imprinted on the slip.
- This document is used as a “pick” slip, and is placed in a “pick” basket. The user then goes through the warehouse, gathers the appropriate items, and when complete the order is placed on a feed belt to the shipper location.
- When the basket lands in front of the shipper, that person checks the contents of the basket against the slip, and then uses a bar code scanner to scan the order. That scan triggers a query into our system that returns all applicable shipping data into an Endicia or UPS system. 
- A shipping label is created, and the shipping cost and tracking information is returned to the our system.
- Additionally the inventory is decremented accordingly when the order receives a shipping label and tracking number.

Requirements: administrative / accounting—Vervante also needed an administrative / accounting arm, designed to control all of the accounting functions such as:
- Recording customers’ fulfillment charges
- Recording customers’ sales (Vervante sells product for the customers as well as fulfilling outside orders)
- Determining fulfillment vendor fees and payments
- Tracking shipping costs
- Monthly billing of all customers
- Monthly payments for all customers.
- Interface with in-house accounting systems and keeping systems in sync
- Tracking and posting outside order transactions

The above described processes are just a few of the processes that we developed from scratch, and matched to Vervante’s needs. It is also a tiny portion of their system.

Last, but not least

Oh, and one other interesting fact: When Vervante first came to us several years ago, they had fewer than 20 customers. Today, they provide order fulfillment and print on demand services for nearly 4000 customers. So when we say to plan ahead for growth, we have experience in that area.

Zero Pricing in Interchange using CommonAdjust

2017-06-26T00:00:00+00:00

Product pricing can be quite complex. A typical Interchange catalog will have at least one table in the ProductFiles directive (often products plus either options or variants) and those tables will often have one or more pricing fields (usually price and sales_price). But usually a single, static price isn’t sufficient for more complex needs, such as accessory adjustments, quantity pricing, product grouping–not to mention promotions, sales, or other conditional features that may change a product’s price for a given situation, dependent on the user’s account or session.

Typically to handle these variety of pricing possibilities, a catalog developer will implement a CommonAdjust algorithm. CommonAdjust can accommodate all the above pricing adjustments and more, and is a powerful tool (yet can become quite arcane when reaching deeper complexity). CommonAdjust is enabled by setting the PriceField directive to a non-existent field value in the tables specified in ProductFiles.

To give an adequate introduction and treatise on CommonAdjust would be at a minimum its own post, and likely a series. There are many elements that make up a CommonAdjust string, and subtle operator nuances that instruct it to operate in differing patterns. It is even possible for elements themselves to return new CommonAdjust atoms (a feature we will be leveraging in this discussion). So I will assume for this writing that the reader is familiar generally with CommonAdjust and we will implement a very simple example to demonstrate henceforth.

To start, let’s create a CommonAdjust string that simply replaces the typical PriceField setting, and we’ll allow it to accommodate a static sales price:

ProductFiles products
PriceField 0
CommonAdjust :sale_price ;:price

The above, in words, indicates that our products live in the products table, and we want CommonAdjust to handle our pricing by setting PriceField to a non-existent field (0 is a safe bet not to be a valid field in the products table). Our CommonAdjust string is comprised of two atoms, both of which are settors of type database lookup. In the products table, we have 2 fields: sale_price and price. If sale_price is “set” (meaning a non-zero numeric value or another CommonAdjust atom) it will be used as it comes first in the list. The semicolon indicates to Interchange “if a previous atom set a price by this point, we’re done with this iteration” and, thus, the price field will be ignored. Otherwise, the next atom is checked (the price field), and as long as the price field is set, it will be used instead.

A few comments here:

The bare colon indicates that the field is not restricted to a particular table. Typically, to specify the field, you would have a value like “products:price” or “variants:price”. But part of the power of ProductFiles holding products in different tables is you can pick up a sku from any of them. And at that point, you don’t know whether you’re looking at a sku from products, variants, or as many additional tables as you’d like to grab products from. But if all of them have a price and sales_price field, then you can pick up the pricing from any of them by leaving the table off. You can think of “:price” as “*:price” where asterisk is “table this sku came from”.
The only indicator that CommonAdjust recognizes as a terminal value is a non-zero numeric value. The proposed price is coerced to numeric, added on to the accumulated price effects of the rest of the CommonAdjust string (if applicable), and the final value is tested for truth. If it is false (empty, undef, or 0) then the process repeats.
What happens if none of the atoms produce a non-zero numeric value? If Interchange reaches the end of the original CommonAdjust string without hitting a set atom, it will relent and return a zero cost.

At this point, we finally introduce our situation, and one that is not at all uncommon. What if I want a zero price? Let’s say I have a promotion for buy one product, get this other product for free. Typically, a developer would be able to expect to override the prices from the database optionally by leveraging the “mv_price” parameter in the cart. So, let’s adjust our CommonAdjust to accommodate that:

CommonAdjust $ ;:sale_price ;:price

The $ settor in the first atom means “look in the line-item hash for the mv_price parameter and use that, if it’s set”. But as we’ve discussed above, we “set” an atom by making it a non-zero numeric value or another CommonAdjust atom. So if we set mv_price to 0, we’ve gained nothing. CommonAdjust will move on to the next atom (sale_price database settor) and pick up that product’s pricing from the database. And even if we set that product’s sale_price and price fields to 0, it means everyone purchasing that item would get it for free (not just our promotion that allows the item to be free with the specific purchase of another item).

In the specific case of using the $ settor in CommonAdjust, we can set mv_price to the keyword “free”, and that will allow us to price the item for 0. But this restricts us to only be able to use $ and mv_price to have a free item. What if the price comes from a complex calculation, out of a usertag settor? Or out of a calc block settor? The special “free” keyword doesn’t work there.

Fortunately, there is a rarely used CommonAdjust settor that will allow for a 0 price item in a general solution. As I mentioned above, CommonAdjust calculations can themselves return other CommonAdjust atoms, which will then be operated on in a subsequent iteration. This frees us from just the special handling that works on $ and mv_price as such an atom can be returned from any of the CommonAdjust atoms and work.

The settor of interest is >>, and according to what documentation there is on it, it was never even intended to be used as a pricing settor! Rather, it was to be a way of redirecting to additional modes for shipping or tax calculations, which can also leverage CommonAdjust for their particular purposes. However, the key to its usefulness here is thus: it does not perform any test on the value tied to it. It is set, untested, into the final result of this call to the chain_cost() routine and returned. And with no test, the fact that it’s Perly false as numeric 0 is irrelevant.

So building on our current CommonAdjust, let’s leverage >> to allow our companion product to have a zero cost (assuming it is the 2nd line item in the cart):

[calcn]
    $Items->[1]{mv_price} = '>>0';
    return;
[/calcn]

Now what happens is, $ in the first atom picks up the value out of mv_price and, because it’s a CommonAdjust atom, is processed in a second iteration. But this CommonAdjust atom is very simple: take the value tied to >> and return it, untested.

Perhaps our pricing is more complex than we can (or would like to) support with using $. So we want to write a usertag, where we have the full power of global Perl at our disposal, but we still have circumstances where that usertag may need to return zero-cost items. Using the built-in “free” solution, we’re stuck, short of setting mv_price in the item hash within the usertag, which we may not want to do for a variety of reasons. But using >>, we have no such restriction. So let’s change CommonAdjust:

CommonAdjust $ ;[my-special-pricing] ;:sale_price ;:price

Now instead of setting mv_price in the item, let’s construct [my-special-pricing] to do some heavy lifting:

UserTag my-special-pricing Routine <<EOR
sub {
    # A bunch of conditional, complicated code, but then ...
    elsif (buy_one_get_one_test($item)) {
        # This is where we know this normally priced item is supposed to be
        # free because of our promotion. Excellent!

        return '>>0';
    }
    # remaining code we don't care about for this discussion
}
EOR

Now we haven’t slapped a zero cost onto the line item in a sticky fashion, like we do by setting mv_price. So presumably, above, if the user gets sneaky and removes the “buy one” sku identified by our promotion, our equally clever buy_one_get_one_test() sniffs it out, and the 0 price is no longer in effect.

For more information on CommonAdjust, see the Custom Pricing section of ‘price’ glossary entry. And for more examples of leveraging CommonAdjust for quantity and attribute pricing adjustments, see the Examples section of the CommonAdjust document entry.

Reach customers and drive sales with MailChimp

2016-08-22T00:00:00+00:00

It’s a good idea for ecommerce stores to regularly contact their customers. This not only reminds customers that your business exists, but also allows the sharing of new products and resources that can enrich the lives of your customers and clients. One of the easiest ways to stay in touch is by using an email newsletter service, such as MailChimp.

<img alt="" border="0" height="200" src="/blog/2016/08/reach-customers-and-drive-sales-with/image-0.png" title="Freddie, the MailChimp mascot" width="191"/>

MailChimp offers the regular suite of email newsletter services: lists, campaigns, and reports—but in addition, they allow an ecommerce store to integrate sales data back into MailChimp. When you have detailed shopping statistics for each subscriber, it opens new possibilities for customized marketing campaigns.

Endless possibilities

For example, imagine you have an email mailing list with 1,000 recipients. Instead of mailing the same generic newsletter to each subscriber, what if you could segment the list to identify your 100 best customers, and email them a special campaign?

Additional ideas could include:

Reach out to inactive subscribers, offering a coupon
Invite your best customers to a secret sale
Re-engage customers who placed items in their cart, but left without purchasing
Offer complementary products to purchasers of Product X

Automatic marketing

Once your store has sales data for subscribers, and you’ve decided on the campaigns you want to run with this data, the next step is to automate the process. This is where MailChimp’s Automation feature comes in. Spend some time up-front to craft the automated campaigns, then sit back and let MailChimp run them for you; day in, day out.

Steps to implement

There are several off-the-shelf integrations for ecommerce stores, including Magento and BigCommerce.

Users of Perl and Interchange can use our newly-released toolsets of the Mail::Chimp3 CPAN module and the integration for Interchange5.

Most businesses already have an email newsletter. Hopefully, you are sending regular email campaigns with it. This is a great first step. Going beyond this to segment your email list and reach out to these segments with relevant information to each of them, is the next step. Not only can this increase your sales, but it also respects your clients’ and customers’ time and preferences. It’s a win-win for all.

Additional resource: MailChimp for Online Sellers

Improve SEO URLs for Interchange search pages

2016-01-27T00:00:00+00:00

This is an article aimed at beginner-to-intermediate Interchange developers.

A typical approach to a hierarchical Interchange site is:

Categories -> Category -> Product

I.e., you list all your categories as links, each of which opens up a search results page filtering the products by category, with links to the individual product pages via the flypage.

Recently I upgraded a site so the category URLs were a bit more SEO-friendly. The original category filtering search produced these lovely specimens:

/search.html?fi=products&st=db&co=1&sf=category&se=Shoes&op=rm
   &sf=inactive&se=yes&op=ne&tf=category&ml=100

but what I really wanted was:

/cat/Shoes.html

Such links are easier to communicate to users, more friendly to search engines, less prone to breakage (e.g., by getting word-wrapped in email clients), and avoid exposing details of your application (here, we’ve had to admit publicly that we have a table called “products” and that some items are “inactive”; a curious user might decide to see what happens if they change “sf=inactive&se=yes” to some other expression).

Here’s how I attacked this.

Creating a category listing page

First, I copied my “results.html” page to “catpage.html”. That way, my original search results page can continue to serve up ad hoc search results.

The search results were displayed via:

[search-region]
...
[/search-region]

I converted this to a database query:

[query sql="SELECT * FROM products WHERE NOT inactive AND category = [sql-quote][cgi category][/sql-quote]"
 type=list prefix=item]
...
[/query]

I chose to use a prefix other than the default since it would avoid having to change so many tags in the page, and now both the original search page and new catpage would look much the same internally (and thus, if desired, I could refactor them in the future).

Note that I’ve defined part of the API for this page: the category to be searched is set in a CGI variable called “category”.

In my specific case, there was additional tinkering with this tag, because I had nested [query] tags already in the page within the search-region.

Creating a “cat” actionmap

In order to translate a URL containing SEO-friendly “/cat/Shoes.html” into my search, I need an actionmap. Here’s mine; it’s very simple.

Actionmap cat <<"CODE"
sub {
  my $url = shift;
  my @url_parts = split '/' => $url;
  shift @url_parts if $url_parts[0] eq 'cat';

  $CGI->{mv_nextpage} = 'catpage.html';
  $CGI->{category} = shift @url_parts;
  return 1;
}
CODE

Actionmaps are called when Interchange detects that a URL begins with the actionmap’s name; here “cat”. They are passed a parameter containing the URL fragment (after removing all the site stuff). Here, that would be (e.g.) “/cat/Shoes”. We massage the URL to get our category code, and set up the page to be called along with the CGI parameter(s) it expects.

Cleaning up the links

At the start of this article I noted that I may have a page listing all my categories. In my original setup, this generated links using a construction like this:

<a href="[area href=search form=|fi=products
         st=db
         sf=category
         se=Shoes
         tf=category
         ml=100|]">
  Shoes
</a>

Now my links are the much simpler:

<a href="[area cat/Shoes]">Shoes</a>

In my specific case, these links were generated within a [query] loop, but the approach is the same.

Note: the Strap demo supports SEO-friendly URLs out-of-the-box, and that it is included with the latest Interchange 5.10 release.

2015 Perl Dancer Conference videos

2015-12-30T00:00:00+00:00

The 2015 Perl Dancer Conference has recently released the presentation videos. This year the conference was hosted in beautiful Vienna, Austria. Josh Lavin and I were both honored to attend the conference as well as give talks. Earlier, Josh wrote summaries of the conference:

Conference Recap

Conference Presentations

SpaceCamps “The Final Frontier”

I gave a talk exploring new technologies for End Point’s own DevCamps development tool. During the presentation I detailed my research into containers and what a cloud-based development environment might look like.

SpaceCamps Presentation Video

AngularJS & Dancer for Modern Web Development

Josh detailed his experience migrating legacy applications utilizing Dancer, AngularJS, and modern Perl techniques. Josh highlighted the challenges he faced during the process, as well as lessons he learned along the way.

AngularJS & Dancer for Modern Web Development Presentation Video

Lightning Talks

Josh and I both gave short “lightning talks.” Josh’s was on Writing Unit Tests for a Legacy App (Interchange 5), and mine was on Plack & Interchange 5.

To review the rest of the presentations please checkout the Perl Dancer Conference YouTube channel.

Summary

The Perl Dancer community continues to flourish and the conference this year hosted a record 5 core Dancer developers. Dancer is about to release the finalized version of its long awaited plugin infrastructure for Dancer2. A lot of work on this was completed during the conference. Being an organizer of the conference, it brings me great joy to see this success. This news along with the release of Perl 6, I am certain 2016 will be a wonderful year for not only Dancer but the entire Perl community.

Perl Dancer Conference 2015 Report—Conference Days

2015-10-30T00:00:00+00:00

In my last post, I shared about the Training Days from the Perl Dancer 2015 conference, in Vienna, Austria. This post will cover the two days of the conference itself.

While there were several wonderful talks, Gert van der Spoel did a great job of writing recaps of all of them (Day 1, Day 2), so here I’ll cover the ones that stood out most to me.

Day One

Dancer Conference, by Alexis Sukrieh (used with permission)

Sawyer X spoke on the State of Dancer. One thing mentioned, which came up again later in the conference, was: Make the effort, move to Dancer 2! Dancer 1 is frozen. There have been some recent changes to Dancer:

Middlewares for static files, so these are handled outside of Dancer
New Hash::MultiValue parameter keywords (route_parameters, query_parameters, body_parameters; covered in my earlier post)
Delayed responses (asynchronous) with delayed keyword:
- Runs on the server after the request has finished.
- Streaming is also asynchronous, feeding the user chunks of data at a time.

Items coming soon to Dancer may include: Web Sockets (supported in Plack), per-route serialization (currently enabling a serializer such as JSON affects the entire app—later on, Russell released a module for this, which may make it back into the core), Dancer2::XS, and critic/linter policies.

Thomas Klausner shared about OAuth & Microservices. Microservices are a good tool to manage complexity, but you might want to aim for “monolith first”, according to Martin Fowler, and only later break up your app into microservices. In the old days, we had “fat” back-ends, which did everything and delivered the results to a browser. Now, we have “fat” front-ends, which take info from a back-end and massage it for display. One advantage of the microservice way of thinking is that mobile devices (or even third parties) can access the same APIs as your front-end website.

OAuth allows a user to login at your site, using their credentials from another site (such as Facebook or Google), so they don’t need a password for your site itself. This typically happens via JavaScript and cookies. However, to make your back-end “stateless”, you could use JSON Web Tokens (JWT). Thomas showed some examples of all this in action, using the OX Perl module.

One thing I found interesting that Thomas mentioned: Plack middleware is the correct place to implement most of the generic part of a web app. The framework is the wrong part. I think this mindset goes along with Sawyer’s comments about Web App + App in the Training Days.

Mickey Nasriachi shared his development on PONAPI, which implements the JSON API specification in Perl. The JSON API spec is a standard for creating APIs. It essentially absolves you from having to make decisions about how you should structure your API.

Panorama from the south tower of St. Stephen’s cathedral, by this author

Gert presented on Social Logins & eCommerce. This built on the earlier OAuth talk by Thomas. Here are some of the pros/cons to social login which Gert presented:

Pros—customer:
- Alleviates “password fatigue”
- Convenience
- Brand familiarity (with the social login provider)
Pros—eCommerce website:
- Expected customer retention
- Expected increase in sales
- Better target customers
- “Plug & Play” (if you pay)—some services exist to make it simple to integrate social logins, where you just integrate with them, and then you are effectively integrated with whatever social login providers they support. These include Janrain and LoginRadius
Cons—customer:
- Privacy concerns (sharing their social identity with your site)
- Security concerns (if their social account is hacked, so are all their accounts where they have used their social login)
- Confusion (especially on how to leave a site)
- Usefulness (no address details are provided by the social provider in the standard scope, so the customer still has to enter extra details on your site)
- Social account hostages (if you’ve used your social account to login elsewhere, you are reluctant to shut down your social account)
Cons—eCommerce website:
- Legal implications
- Implementation hurdles
- Usefulness
- Provider problem is your problem (e.g., if the social login provider goes down, all your customers who use it to login are unable to login to your site)
- Brand association (maybe you don’t want your site associated with certain social sites)
Cons—social provider:
- ???

Šimun Kodžoman spoke on Dancer + Meteor = mobile app. Meteor is a JavaScript framework for both server-side and client-side. It seems one of the most interesting aspects is you can use Meteor with the Android or iOS SDK to auto-generate a true mobile app, which has many more advantages than a simple HTML “app” created with PhoneGap. Šimun is using Dancer as a back-end for Meteor, because the server-side Meteor aspect is still new and unstable, and is also dependent on MongoDB, which cannot be used for everything.

End Point’s own Sam Batschelet shared his work on Space Camp, a new container-based setup for development environments. This pulls together several pieces, including CoreOS, systemd-nspawn, and etcd to provide a futuristic version of DevCamps.

Day Two

Conference goers, by Sam
(used with permission)

Andrew Baerg spoke on Taming the 1000-lb Gorilla that is Interchange 5. He shared how they have endeavored to manage their Interchange development in more modern ways, such as using unit tests and DBIC. One item I found especially interesting was the use of DBIx::Class::Fixtures to allow saving bits of information from a database to keep with a test. This is helpful when you have a bug from some database entry which you want to fix and ensure stays fixed, as databases can change over time, and without a “fixture” your test would not be able to run.

Russell Jenkins shared HowTo Contribute to Dancer 2. He went over the use of Git, including such helpful commands and tips as:

git status –short –branch
Write good commit messages: one line summary, less than 50 characters; longer description, wrapped to 72 characters; refer to and/or close issues
Work in a branch (you shall not commit to master)
“But I committed to master” –> branch and reset
git log –oneline –since=2.weeks
git add –fixup
The use of branches named with “feature/whatever” or “bugfix/whatever” can be helpful (this is Russell’s convention)

There are several Dancer 2 issues tagged “beginner suitable”, so it is easy for nearly anyone to contribute. The Dancer website is also on GitHub. You can even make simple edits directly in GitHub!

It was great to have the author of Dancer, Alexis Sukrieh, in attendance. He shared his original vision for Dancer, which filled a gap in the Perl ecosystem back in 2009. The goal for Dancer was to create a DSL (Domain-specific language) to provide a very simple way to develop web applications. The DSL provides “keywords” for use in the Dancer app, which are specific to Dancer (basically extra functionality for Perl). One of the core aspects of keeping it simple was to avoid the use of $self (a standby of object-oriented Perl, one of the things that you just “have to do”, typically).

Alexis mentioned that Dancer 1 is frozen—Dancer 2 full-speed ahead! He also shared some of his learnings along the way:

Fill a gap (define clearly the problem, present your solution)
Stick to your vision
Code is not enough (opensource needs attention; marketing matters)
Meet in person (collaboration is hard; online collaboration is very hard)
Kill the ego—you are not your code

While at the conference, Alexis even wrote a Dancer2 plugin, Dancer2::Plugin::ProbabilityRoute, which allows you to do A/B Testing in your Dancer app. (Another similar plugin is Dancer2::Plugin::Sixpack.)

Also check out Alexis’ recap.

Finally, I was privileged to speak as well, on AngularJS & Dancer for Modern Web Development. Since this post is already pretty long, I’ll save the details for another post.

Summary

In summary, the Perl Dancer conference was a great time of learning and building community. If I had to wrap it all up in one insight, it would be: Web App + App—that is, your application should be a compilation of: Plack middleware, Web App (Dancer), and App (Perl classes and methods).

Perl Dancer Conference 2015 Report — Training Days

2015-10-28T00:00:00+00:00

I just returned from the Perl Dancer Conference, held in Vienna, Austria. It was a jam-packed schedule of two days of training and two conference days, with five of the nine Dancer core developers in attendance.

Kohlmarkt street, Wien, by this author

If you aren’t familiar with Perl Dancer, it is a modern framework for Perl for building web applications. Dancer1 originated as a port of Ruby’s Sinatra project, but has officially been replaced with a rewrite called Dancer2, based on Moo, with Dancer1 being frozen and only receiving security fixes. The Interchange 5 e-commerce package is gradually being replaced by Dancer plugins.

Day 1 began with a training on Dancer2 by Sawyer X and Mickey Nasriachi, two Dancer core devs. During the training, the attendees worked on adding functionality to a sample Dancer app. Some of my takeaways from the training:

Think of your app as a Dancer Web App plus an App. These should ideally be two separate things, where the Dancer web app provides the URL routes for interaction with your App.
The lib directory contains all of your application. The recommendation for large productions is to separate your app into separate namespaces and classes. Some folks use a routes directory just for routing code, with lib reserved for the App itself.
It is recommended to add an empty .dancer file to your app’s directory, which indicates that this is a Dancer app (other Perl frameworks do similarly).
When running your Dancer app in development, you can use plackup -R lib bin/app.psgi which will restart the app automatically whenever something changes in lib.
Dancer handles all the standard HTTP verbs, except note that we must use del, not delete, as delete conflicts with the Perl keyword.
There are new keywords for retrieving parameters in your routes. Whereas before we only had param or params, it is now recommended to use:
- route_parameters,
- query_parameters, or
- body_parameters
- all of which can be used with ->get(‘foo’) which is always a single scalar, or ->get_all(‘foo’) which is always a list.
- These allow you to specify which area you want to retrieve parameters from, instead of being unsure which param you are getting, if identical names are used in multiple areas.

Day 2 was DBIx::Class training, led by Stefan Hornburg and Peter Mottram, with assistance from Peter Rabbitson, the DBIx::Class maintainer.

DBIx::Class (a.k.a. DBIC) is an Object Relational Mapper for Perl. It exists to provide a standard, object-oriented way to deal with SQL queries. I am new to DBIC, and it was a lot to take in, but at least one advantage I could see was helping a project be able to change database back-ends, without having to rewrite code (cue PostgreSQL vs MySQL arguments).

I took copious notes, but it seems that the true learning takes place only as one begins to implement and experiment. Without going into too much detail, some of my notes included:

Existing projects can use dbicdump to quickly get a DBIC schema from an existing database, which can be modified afterwards. For a new project, it is recommended to write the schema first.
DBIC allows you to place business logic in your application (not your web application), so it is easier to test (once again, the recurring theme of Web App + App).
The ResultSet is a representation of a query before it happens. On any ResultSet you can call ->as_query to find the actual SQL that is to be executed.
DBIx::Class::Schema::Config provides credential management for DBIC, and allows you to move your DSN/username/password out of your code, which is especially helpful if you use Git or a public GitHub.
DBIC is all about relationships (belongs_to, has_many, might_have, and has_one). many_to_many is not a relationship per se but a convenience.
DBIx::Class::Candy provides prettier, more modern metadata, but cannot currently be generated by dbicdump.
For deployment or migration, two helpful tools are Sqitch and DBIx::Class::DeploymentHandler. Sqitch is better for raw SQL, while DeploymentHandler is for DBIC-managed databases. These provide easy ways to migrate, deploy, upgrade, or downgrade a database.
Finally, RapidApp can read a database file or DBIC schema and provide a nice web interface for interacting with a database. As long as you define your columns properly, RapidApp can generate image fields, rich-text editors, date-pickers, etc.

The training days were truly like drinking from a firehose, with so much good information. I am looking forward to putting this into practice!

Stay tuned for my next blog post on the Conference Days.

Old Dog & New Tricks — Giving Interchange a New Look with Bootstrap

2015-09-02T00:00:00+00:00

Introduction

So your Interchange based website looks like it walked out of a disco… but you spent a fortune getting the back end working with all those custom applications…. what to do?

Interchange is currently being used in numerous very complex implementations. Trying to adapt another platform to replace Interchange is a formidable task, and in many cases the “problem” that users are trying to “solve” by replacing Interchange, can be remedied by a relatively simple face lift to the front end. One of the main attractions to Interchange in the past was its ability to scale from a small mom & pop eCommerce application, to a mid level support system for a larger company and its related back end systems. Once the connection to those back end systems has been created, and for as long as you use those related systems, Interchange will continue to be the most economic choice for the job. But that leaves the front end, the one that your customers see and use (with their phones and tablets) that becomes the most immediate target for “modernization”.

Granted, there are new and alternate ways of accessing and presenting data and views to users, but many of those alternatives are also accessible and inter-operable with Interchange. So while there are several topics that can be investigated at length, we will focus first on a popular front end framework that can help Interchange present a modern responsive theme to the end users. That framework is Bootstrap. Bootstrap is a good basic start for breathing new life into your Interchange application. Bootstrap uses a reasonably generic approach to HTML, CSS, and JavaScript frameworking. This blends nicely with the Interchange application approach: providing a basic, repeatable, broad based and well supported foundation that can then be crafted into whatever the developer and their client may need.

My intent in this post, is to give you links to the tools available to implement Bootstrap into your Interchange application, and in subsequent posts explain our development process which may help you in how you use the tools. Like any development, there are many ways to accomplish various tasks, and knowing why certain things were done, can help.

Alternatively, you may simply want to just download the “strap” catalog, and get started. At the time of this writing, Josh Lavin has been reviewing and refining the package, and the most recent version can be found at his Github account: Bootstrap template for Interchange. It will soon become the standard catalog template in Interchange. So, if you are not really interested in why and how this template was developed, feel free to go to the git repository in the link, clone a copy and get busy! The README is very informative, and if you are an Interchange developer you probably won’t have much trouble implementing this.

If you are interested in some of the major changes that we made to the old “standard” templating approach, and how you can apply those changes to an existing catalog without having to start from scratch, stay tuned in for my next post.

Perusion has joined End Point!

2015-07-31T00:00:00+00:00

We are excited to announce an expansion of End Point’s ecommerce clientele and our developer ranks! The ecommerce consulting company Perusion has joined End Point. Perusion was founded in 2002 by Mike Heins and Greg Hanson. It quickly became a small powerhouse in the open source ecommerce space, focusing on Interchange, Perl, and MySQL on Linux. We were pleased to welcome Perusion in a merger with End Point at the beginning of July.

Mike Heins is the original creator of MiniVend in 1996. In 2000, Mike’s consultancy and MiniVend were acquired by the ecommerce startup Akopia. With numerous improvements, including the addition of a new full-featured administrative back-office, the new open source ecommerce platform Interchange was created. Akopia was acquired by Red Hat in 2001, and in 2002 the Interchange project became independent, led by its creators and a group of other open source developers who maintain it to this day.

Greg Hanson is a serial entrepreneur and business manager and has worked extensively with data systems of many types. In the mid-1990s he started a computer products ecommerce company, Valuemedia, and oversaw every aspect of its evolution. Greg joined Mike to launch Perusion in 2002, and is now a client consultant and developer. He has shepherded several businesses from small mom & pop operations into strong companies providing goods and services around the world.

Josh Lavin began creating websites professionally in 1998 at his consultancy Kingdom Design. He grew his work into the ecommerce space, helping many companies sell their products online for the first time. Josh joined Perusion in 2007, bringing skills in marketing and user experience that are just as important as his development abilities. In recent years he has enjoyed moving client sites over to responsive front-end designs that work well on desktop, tablet, and mobile phone.

Perusion’s development and hosting clients have joined End Point as well. Some of the noteworthy sites that bear mentioning here are American Welding Society, Bluestone Perennials, Vervanté, Bulk Herb Store, Northern Sun, Penn Herb Company, Air Delights, and Solar Pathfinder.

At End Point we got our start in the industry in 1995 by creating dynamic database-backed websites for our clients. Our tools of choice in those early days were Linux, Apache, msql and MySQL, and Perl. In the late 1990s we began focusing more on ecommerce websites specifically, and we added MiniVend and Interchange to the mix.

Later we branched out into PostgreSQL, Ruby on Rails and Spree, Python and Django, Perl Dancer, NodeJS, and other platforms, while we continued to support and enhance Interchange and the sites running on it. Today we still host and develop many successful ecommerce sites running Interchange, taking many thousands of orders worth millions of dollars every day. So we are delighted to have Perusion join us as we to continue to grow the Interchange-based part of our business.

We have already seen constructive collaboration in this merger, with longtime End Point employees able to add more helping hands to Perusion projects and further breadth of capabilities and depth of support, Perusion developers bringing their expertise to bear, and Perusion contacts leading to new projects and new business.

Perusion has always believed in going above and beyond the call of duty and has made their clients’ success their own goal. Likewise, we at End Point feel this is more than a business, and we value the personal relationships we have developed with each other and our clients over the years. We look forward to the new possibilities that are now available through this change, and are glad to welcome Perusion on board at End Point!

Protect Interchange Passwords with Bcrypt

2015-04-28T00:00:00+00:00

Interchange default configurations have not done a good job of keeping up with the best available password security for its user accounts. Typically, there are two account profiles associated with a standard Interchange installation: one for admin users (access table) where the password is stored using Perl’s crypt() command (bad); and one for customers (userdb) where the password isn’t encrypted at all (even worse). Other hashing algorithms have long been available (MD5, salted MD5, SHA1) but are not used by default and have for some time not been useful protection. Part of this is convenience (tools for retrieving passwords and ability to distribute links into user assets) and part is inertia. And no small part was the absence of a strong cryptographic option for password storage until the addition of Bcrypt to the account management module.

The challenge we face in protecting passwords is that hardware continues to advance at a rapid rate, and with more computational power and storage capacity, brute-force attacks become increasingly effective and widely available. Jeff Jarmoc’s Enough with the Salts provides some excellent discussion and background on the subject. To counter the changing landscape, the main line of defense moves toward ensuring that the work required to create and test a given stored password is too expensive, too time-consuming, for brute-force attacks to be profitable.

One of the best options for handling encrypted password storage with a configurable “hardware cost” is Bcrypt. We chose to integrate Bcrypt into Interchange over other options primarily because of its long history of operation with no known exploits, and its cost parameter that allows an administrator to advance the work required to process a password slowly over time as hardware continues to increase in efficiency. The cost feature introduces an exponential increase in calculation iterations (i.e., required processing power and time) as powers of 2, from 1 (2 iterations, essentially no cost) to 31 (2^31, or 2,147,483,648 iterations). Ideally an administrator would want to identify a cost that causes no perceptible penalty to end users, but would be such a burden to any brute-force attack as to have no worthwhile return on investment to crack.

Converting an existing user base from any of the existing encryption schemes to Bcrypt is trivial in Interchange. The existing UserDB profile is changed to the “bcrypt” option and the “promote” boolean set to true. Promote allows your users to continue to validate against their existing stored password, but after the next access will upgrade their storage to the Bcrypt password. In the mean time, a backend process could be developed using the construct_bcrypt() routine in Vend::UserDB to update all outstanding accounts prior to being updated organically.

If the switch on the front end involves going from no encryption to any encrypted storage, including Bcrypt, and your front end uses the default tools for retrieving lost passwords, you’ll also need to construct some new code for resetting passwords instead. There is no such facility for the admin, and since the admin accounts are typically far more valuable than the front end accounts, making the change for the admin should be the first priority and have the least effort involved.

Switching accounts to Bcrypt password storage is a simple, effective means for increasing protection on your users’ and business’ information. Every bit as importantly, it also helps protect your business’ reputation, that can be severely damaged by a data breech. Lastly, in particular for your admin accounts, Bcrypt password storage is useful in meeting PCI DSS requirements for strong password hashing.

Advanced Product Filtering in Ecommerce

2015-03-18T00:00:00+00:00

One of my recent projects for Paper Source has been to introduce advanced product filtering (or faceted filtering). Paper Source runs on Interchange, a perl-based open source ecommerce platform that End Point has been involved with (as core developers & maintainers) for many years.

In the case of Paper Source, personalized products such as wedding invitations and save the dates have advanced filtering to filter by print method, number of photos, style, etc. Advanced product filtering is a very common feature in ecommerce systems with a large number of products that allows a user to narrow down a set of products to meet their needs. Advanced product filtering is not unlike faceted filtering offered by many search engines, which similarly allows a user to narrow down products based on specific tags or facets (e.g. see many Amazon filters on the left column). In the case of Paper Source, I wrote the filtering code layered on top of the current navigation. Below I’ll go through some of the details with small code examples.

Data Model

The best place to start is the data model. A simplified existing data model that represents product taxonomy might look like the following:

Basic data model linking categories to products.

The existing data model links products to categories via a many-to-many relationship. This is fairly common in the ecommerce space – while looking at a specific category often identified by URL slug or id, the products tied to that category will be displayed.

And here’s where we go with the filtering:

Data model with filtering layered on top of existing category to product relationship.

Some notes on the above filtering data model:

filters contains a list of all the files. Examples of entries in this table include “Style”, “Color”, “Size”
filters_categories links filters to categories, to allow finite control over which filters show on which category pages, in what order. For example, this table would link category “Shirts” to filters “Style”, “Color”, “Size” and the preferred sort order of those filters.
filter_options includes all the options for a specific filter. Examples here for various options include “Large”, “Medium”, and “Small”, all linked to the “Size” filter.
filter_options_products links filter options to a specific product id with a many to many relationship.

One thing to consider before coding are the business rules pertaining to filter option exclusivity. If a product is assigned to one filter option, can it also have another filter option for that same filter type? IE, if a product is marked as blue, can it also be marked as red? When a user filters by color, can they filter to select products that are both blue and red? Or, if a product is is blue, can it not have any other filter options for that filter? In the case of Paper Source product filtering, we went with the former, where filter options are not exclusive to each other.

A real-life example of filter non-exclusivity is how Paper Source filters wedding invitations. Products are filtered by print method and style. Because some products have multiple print methods and styles, non-exclusivity allows a user to narrow down to a specific combination of filter options, e.g. a wedding invitation that is both tagged as “foil & embossed” and “vintage”.

URL Structure

Another thing to determine before coding is the URL structure. The URL must communicate the current category of products and current filter options (or what I refer to as active/activated filters).

I designed the code to recognize one component of the URL path as the category slug, and the remaining paths to map to the various filter option url slugs. For example, a URL for the category of shirts is “/shirts”, a URL for large shirts “/shirts/large”, and the URL for large blue shirts “/shirts/blue/large”. The code not only has to accept this format, but it also must create consistently ordered URLs, meaning, we don’t want both “/shirts/blue/large” and “/shirts/large/blue” (representing the same content) to be generated by the code. Here’s what simplified pseudocode might look like to retrieve the category and set the activated filters:

my @url_paths = split('/', $request_url); #url paths is e.g. /shirts/blue/large
my $category_slug = shift(@url_paths)
# find Category where slug = $category_slug
# redirect if not found
# @url_paths is active filters

Applying the Filtering

Next, we need a couple things to happen:

If there is an activated filter for any filter option, apply it.
Generate URLs to toggle filter options.

First, all products are retrieved in this category with a query like this:

SELECT products.*,
  COALESCE((SELECT GROUP_CONCAT(fo.url_slug) FROM filters_options_item foi
    JOIN filters_options fo ON fo.id = foi.filter_option_id
    WHERE foi.product_id = products.id), '') AS filters
FROM products
JOIN categories_products cp ON cp.product_id = products.id
JOIN categories c ON c.id = cp.category_id
WHERE c.url_slug = ?

Next is where the code gets pretty hairy, so instead I’ll try to explain with pseudocode:

#@filters = all applicable filters for current category
# loop through @filters
  # loop through filter options for this filter
  # filter product results to include any selected filter options for this filter
  # if there are no filter options selected for this filter, include all products
  # build the url for each filter option, to toggle the filter option (on or off)
# loop through @filters (yes, a second time)
  # loop through filter options for this filter
  # count remaining products for each filter option, if none, set filter option to inactive
  # build the final url for each filter option, based on all filters turned on and off

My pseudocode shows that I iterate through the filters twice, first to apply the filter and determine the base URL to toggle each filter option, and second to count the remaining filtered products and build the URL to toggle each filter. The output of this code is a) a set of filtered products and b) a set of ordered filters and filter options with corresponding counts and links to toggle on or off.

Here’s a more specific example:

Let’s say we have a set of shirts, with the following filters & options: Style (Long Sleeve, Short Sleeve), Color (Red, Blue), Size (Large, Medium, Small).
A URL request comes in for /shirts/blue/large
The code recognizes this is the shirts category and retrieves all shirts.
First, we look at the style filter. No style filter is active in this request, so to toggle these filters on, the activation URLs must include “longsleeve” and “shortsleeve”. No products are filtered out here.
Next, we look at the color filter. The blue filter option is active because it is present in the URL. In this first loop, products not tagged as blue are removed from the set of products. To toggle the red option on, the activation URL must include “red”, and to toggle the blue filter off, the URL must not include “blue”, which is set here.
Next, we look at the size filter. Products not tagged as large are removed from the set of products. Again, the large filter has to be toggled off in the URL because it is active, and the medium and small filter need to be toggled on.
In the second pass through filters, the remaining items applicable to each filter option are counted, for long sleeve, short sleeve, red, medium, and small options. And the URLs are built to turn on and off all filter options (e.g. applying the longsleeve filter will yield the URL “/shirts/longsleeve/blue/large”, applying the red filter will yield the URL “/shirts/blue/red/large”, turning off the blue filter will yield the URL “/shirts/large”).

The important thing to note here is the double pass through filters is required to build non-duplicate URLs and to determine the product count after all filter options have been applied. This isn’t simple logic, and of course changing the business rules like exclusivity will change the loop behavior and URL logic.

Alternative Approaches

Finally, a few notes regarding alternative approaches here:

Rather than going with the blacklist approach described here, one could go with a whitelist approach where a set of products is built up based on the filter options set.
Filtering could be done entirely via AJAX, in which case URL structure may not be a concern.
If the data is simple enough, products could potentially be filtered in the database query itself. In our case, this wasn’t feasible since we generate product filter option details from a number of product attributes, not just what is shown in the simplified product filter data model above.

Interchange Loop Optimization

2015-02-09T00:00:00+00:00

It’s important to understand both how loops work in Interchange and the (very) fundamental differences between interpolating Interchange tag language (ITL) and the special loop tags (typically referred to as [PREFIX-*] in the literature). Absent this sometimes arcane knowledge, it is very easy to get stuck with inefficient loops even with relatively small loop sets. I’ll discuss both the function of loops and interpolation differences between the tag types while working through a [query] example. While all loop tags–[item-list], [loop], [search-list], and [query]–process similarly, it is to [query] where most complex loops will gravitate over time (to optimize the initiation phase of entering the loop) and where we have the most flexibility for coming up with alternative strategies to mitigate sluggish loop-processing.

Loop Processing

All loop tags are container tags in Interchange, meaning they have an open and close tag, and in between is the body. Only inside this body is it valid to define [PREFIX-] tags (notable exception of [PREFIX-quote] for the sql arg of [query]). This is because the [PREFIX-] tags are not true ITL. They are tightly coupled with the structure of the underlying rows of data and they are processed by distinct, optimized regular expressions serially. Outside the context of the row data from a result set, they are meaningless.

Moreover, the body of a loop tag is slurped into a scalar variable (as all bodies of container tags are handled via the ITL parser) and for each row in the record set of the loop, the contents are acted upon according to the [PREFIX-] tags defined within the body. The first important distinction to recognize here is, the per-row action on this scalar is limited to only the [PREFIX-] tags. The action occurring at loop time ignores any embedded ITL.

At the end of each row’s processing, the copy of the body tied to that one row is then concatenated to the results of all previous rows thus processed. For a loop with N rows (assuming no suppression by [if-PREFIX-*] conditionals) that means every instance of ITL originally placed into the loop body is now present N times in the fully assembled body string. Simple example:

[loop list='1 2 3']
[tmp junk][loop-code][/tmp]
[if scratch junk == 2]
I declare [loop-code] to be special!
[else]
Meh. [loop-code] is ordinary.
[/else]
[/if]
[/loop]

Once this result set with N=3 is processed, but before Interchange returns the results, the assembled return looks like the following string:

[tmp junk]1[/tmp]
[if scratch junk == 2]
I declare 1 to be special!
[else]
Meh. 1 is oridinary
[/else]
[/if]

[tmp junk]2[/tmp]
[if scratch junk == 2]
I declare 2 to be special!
[else]
Meh. 2 is oridinary
[/else]
[/if]

[tmp junk]3[/tmp]
[if scratch junk == 2]
I declare 3 to be special!
[else]
Meh. 3 is oridinary
[/else]
[/if]

Some important observations:

It doesn’t take much ITL to turn a loop body into a monster interpolation process. One must consider the complexity of the ITL in the body by a factor of the number of rows (total, or the “ml” matchlimit value).
ITL does nothing to short-circuit action of the [PREFIX-*] tags. Having [PREFIX-param], [PREFIX-calc], etc. inside an ITL [if] means all those loop tags parse regardless of the truth of the if condition.

ITL vs. Loop Tags

ITL maps to routines, both core and user-defined, determined at compile time. They are processed in order of discovery within the string handed to the ::interpolate_html() routine and have varied and complex attributes that must be resolved for each individual tag. Further, for many (if not most) tags, the return value is itself passed through a new call to ::interpolate_html(), acting on all embedded tags, in an action referred to as reparse. There is, relatively speaking, a good deal of overhead in processing through ::interpolate_html(), particularly with reparse potentially spawning off a great many more ::interpolate_html() calls.

Loop tags, by contrast, map to a pre-compiled set of regular expressions. In contrast to crawling the string and acting upon the tags in the order of discovery, each regex in turn is applied globally to the string. The size of the string is limited to the exact size of the single loop body, and there is no analogue to ITL’s reparse. Further, given this processing pattern, a careful observer might have noted that the order of operations can impact the structure. Specifically, tags processed earlier cannot depend on tags processed later. E.g., [PREFIX-param] processes ahead of [PREFIX-pos], and so:

[if-PREFIX-pos 2 eq [PREFIX-param bar]]

will work, but:

[if-PREFIX-param bar eq [PREFIX-pos 2]]

will not. While the above is a somewhat contrived exampled, the impacts of loop tag processing can be more easily seen in an example using [PREFIX-next]:

[PREFIX-next][PREFIX-param baz][/PREFIX-next]
Code I only want to run when baz is false, like this [PREFIX-exec foo][/PREFIX-exec] call

Because [PREFIX-next] is the absolute last loop tag to run, every other loop tag in the block is run before the next condition is checked. All [PREFIX-next] does is suppress the resulting body from the return, unlike Perl’s next, which short-circuits the remaining code in the loop block.

An Optimization Example

As long as you’re familiar with the idiosyncracies of [PREFIX-*] tags, you should make every effort to use them instead of ITL because they are substantially lighter weight and faster to process. A classic case that can yield remarkable performance gains is to directly swap an embedded [perl] or [calc] block with an equivalent [PREFIX-calc] block.

Let’s take a typical query with little consideration given to whether we use loop tags or ITL, not unlike many I’ve seen where the resource has just become unusably slow. This code originally was developed processing 50 records per page view, but the team using it has requested over time to increase that count.

[query
    list=1
    ml=500 [comment]Ouch! That's a big N[/comment]
    sql="
        SELECT *
        FROM transactions
        WHERE status = 'pending'
        ORDER BY order_date DESC
    "
]
Order [sql-param order_number]
[if global DEVELOPMENT]
    Show [sql-exec stats_crunch][sql-param stats][/sql-exec], only of interest to developers
[/if]
Date: [convert-date format="%b %d, %Y at %T"][sql-param order_date][/convert-date]
[if cgi show_inventory]
Inv:
[if cgi show_inventory eq all]
* Shipped: [either][sql-param is_shipped][or]pending[/either]
* Count: [inventory type=shipped sku=[sql-param sku]]
[/if]
* On Hand: [inventory type=onhand sku=[sql-param sku]]
* Sold: [inventory type=sold sku=[sql-param sku]]
* Shipping: [inventory type=shipping sku=[sql-param sku]]
[/if]
Order details:
    <a href="[area
                href=order_view
                form="
                    order=[sql-param order_number]
                    show_status=[either][cgi show_status][or]active[/either]
                    show_inventory=[cgi show_inventory]
                "
            ]">View [sql-param order_number]</a>
[/query]

Considering this block out of context, it doesn’t seem all that unreasonable. However, let’s look at some of the pieces individually and see what can be done.

We use [if] in 3 different circumstances in the block. However, those values they test are static. They don’t change on any iteration. (We are excluding the potential of any other ITL present in the block from changing their values behind the scenes.)
[convert-date] may be convenient, but it is only one of a number of ways to address date formatting. Our database itself almost certainly has date-formatting routines, but one of the benefits of [convert-date] is you could have a mixed format underlying the data and it can make sense out of the date to some degree. So perhaps that’s why the developer has used [convert-date] here.
Good chance that stats_crunch() is pretty complicated and that’s why the developer wrote a catalog or global subroutine to handle it. Since we only want to see it in the development environment, it’d be nice if it only ran when it was needed. Right now, because of ITL happening on reparse, stats_crunch() fires for every row even if we have no intention of using its output.
We need that link to view our order, but on reparse it means ::interpolate_html() has to parse 500 [area] tags along with [either] and [cgi] x 500. All of these tags are lightweight, but parsing numbers are really going to catch up to us here.

Our goal here is to replace any ITL we can with an equivalent use of a loop tag or, absent the ability to remove ITL logically, to wrap that ITL into a subroutine that can itself be called in loop context with [PREFIX-exec]. The first thing I want to address are those [if] and [either] tags, the lowest hanging fruit:

[query
    list=1
    ml=500
    sql="
        <b>SELECT *,
            '[if global DEVELOPMENT]1[/if]' AS is_development,
            [sql-quote][cgi show_inventory][/sql-quote] AS show_inventory,
            COALESCE(is_shipped,'pending') AS show_inventory_shipped</b>
        FROM transactions
        WHERE status = 'pending'
        ORDER BY order_date DESC
    "
]
Order [sql-param order_number]
<b>[if-sql-param is_development]</b>
    Show [sql-exec stats_crunch][sql-param stats][/sql-exec], only of interest to developers
<b>[/if-sql-param]</b>
Date: [convert-date format="%b %d, %Y at %T"][sql-param order_date][/convert-date]
<b>[if-sql-param show_inventory]</b>
Inv:
<b>[if-sql-param show_inventory eq all]</b>
* Shipped: <b>[sql-param show_inventory_shipped]</b>
* Count: [inventory type=shipped sku=[sql-param sku]]
<b>[/if-sql-param]</b>
* On Hand: [inventory type=onhand sku=[sql-param sku]]
* Sold: [inventory type=sold sku=[sql-param sku]]
* Shipping: [inventory type=shipping sku=[sql-param sku]]
<b>[/if-sql-param]</b>
Order details:
    <a href="[area
                href=order_view
                form="
                    order=[sql-param order_number]
                    show_status=[either][cgi show_status][or]active[/either]
                    show_inventory=[cgi show_inventory]
                "
            ]">View [sql-param order_number]</a>
[/query]

By moving those evaluations into the SELECT list of the query, we’ve reduced the number of interpolations to arrive at those static values to 1 or, in the case of the [either] tag, 0 as we’ve offloaded the calculation entirely to the database. If is_shipped could be something perly false but not null, we would have to adjust our field accordingly, but in either case could still be easily managed as a database calculation. Moreover, by swapping in [if-sql-param is_development] for [if global DEVELOPMENT], we have kept stats_crunch() from running at all when in the production environment.

Next, we’ll consider [convert-date]:

Date: [convert-date format="%b %d, %Y at %T"][sql-param order_date][/convert-date]

My first attempt would be to address this similarly to the [if] and [either] conditions, and try to render the formatted date from a database function as an aliased field. However, let’s assume the underlying structure of the data varies and that’s not easily accomplished, and we still want [convert-date]. Luckily, Interchange supports that same tag as a filter, and [PREFIX-filter] is a loop tag:

Date: <b>[sql-filter convert_date."%b %d, %Y at %T"]</b>[sql-param order_date]<b>[/sql-filter]</b>

[PREFIX-filter] is very handy to keep in mind as many transformation tags have a filter wrapper for them. E.g., [currency] -> [PREFIX-filter currency]. And if the one you’re looking at doesn’t, you can build your own, easily.

Now to look at that [inventory] tag. The most direct approach assumes that the code inside [inventory] can be run in Safe, which often it can even if [inventory] is global. However, if [inventory] does run-time un-Safe things (such as creating an object) then it may not be possible. In such a case, we would want to create a global sub, like our hypothetical stats_crunch(), and invoke it via [PREFIX-exec]. However, let us assume we can safely (as it were) invoke it via the $Tag object to demonstrate another potent loop option: [PREFIX-sub].

[if-sql-param show_inventory]
<b>[sql-sub show_inventory]
    my $arg = shift;
    return $Tag->inventory({ type => $arg, sku => $Row->{sku} });
[/sql-sub]</b>
Inv:
[if-sql-param show_inventory eq all]
* Shipped: [sql-param show_inventory_shipped]
* Count: <b>[sql-exec show_inventory]shipped[/sql-exec]</b>
[/if-sql-param]
* On Hand: <b>[sql-exec show_inventory]on_hand[/sql-exec]</b>
* Sold: <b>[sql-exec show_inventory]sold[/sql-exec]</b>
* Shipping: <b>[sql-exec show_inventory]shipping[/sql-exec]</b>
[/if-sql-param]

Let’s go over what this gives us:

[PREFIX-sub] creates an in-line catalog sub that is compiled at the start of processing, before looping actually begins. As such, the [PREFIX-sub] definitions can occur anywhere within the loop body and are then removed from the body to be parsed.
The body of the [PREFIX-exec] is passed to the sub as the first argument. We use that here for our static values to the “type” arg. If we also wanted to access [sql-param sku] from the call, we would have to include that in the body and set up a parser to extract it out of the one (and only) arg we can pass in. Instead, we can reference the $Row hash within the sub body just as we can do when using a [PREFIX-calc], with one minor adjustment to our [query] tag–we have to indicate to [query] we are operating on a row-hash basis instead of the default row-array basis. We do that by adding the hashref arg to the list:

[query
    list=1
    ml=500
    <b>hashref=1</b>

We still have access to the full functionality of [inventory] but we’ve removed the impact of having to parse that tag 2000 times (in the worst-case scenario) if left as ITL in the query body. If we run into Safe issues, that same sub body can either be created as a pre-compiled global sub or, if available, we can set our catalog AllowGlobal in which case catalog subs will no longer run under Safe.

Finally, all we have left to address is [area] and its args which themselves have ITL. I will leverage [PREFIX-sub] again as an easy way to manage the issue:

<b>[sql-sub area_order_view]
    my $show_status = $CGI->{show_status} || 'active';
    return $Tag->area({
        href => 'order_view',
        form => "order=$Row->{order_number}\n"
              . "show_status=$show_status\n"
              . "show_inventory=$CGI->{show_inventory}",
    });
[/sql-sub]</b>
Order details:
    <a href="<b>[sql-exec area_order_view][/sql-exec]</b>">View [sql-param order_number]</a>

By packaging all of [area]’s requirements into the sub body, I can address all of the ITL at once.

So now, let’s put together the entire [query] rewrite to see the final product:

[query
    list=1
    ml=500
    hashref=1
    sql="
        SELECT *,
            '[if global DEVELOPMENT]1[/if]' AS is_development,
            [sql-quote][cgi show_inventory][/sql-quote] AS show_inventory,
            COALESCE(is_shipped,'pending') AS show_inventory_shipped
        FROM transactions
        WHERE status = 'pending'
        ORDER BY order_date DESC
    "
]
Order [sql-param order_number]
[if-sql-param is_development]
    Show [sql-exec stats_crunch][sql-param stats][/sql-exec], only of interest to developers
[/if-sql-param]
Date: [sql-filter convert_date."%b %d, %Y at %T"][sql-param order_date][/sql-filter]
[if-sql-param show_inventory]
[sql-sub show_inventory]
    my $arg = shift;
    return $Tag->inventory({ type => $arg, sku => $Row->{sku} });
[/sql-sub]
Inv:
[if-sql-param show_inventory eq all]
* Shipped: [sql-param show_inventory_shipped]
* Count: [sql-exec show_inventory]shipped[/sql-exec]
[/if-sql-param]
* On Hand: [sql-exec show_inventory]on_hand[/sql-exec]
* Sold: [sql-exec show_inventory]sold[/sql-exec]
* Shipping: [sql-exec show_inventory]shipping[/sql-exec]
[/if-sql-param]
[sql-sub area_order_view]
    my $show_status = $CGI->{show_status} || 'active';
    return $Tag->area({
        href => 'order_view',
        form => "order=$Row->{order_number}\n"
              . "show_status=$show_status\n"
              . "show_inventory=$CGI->{show_inventory}",
    });
[/sql-sub]
Order details:
    <a href="[sql-exec area_order_view][/sql-exec]">View [sql-param order_number]</a>
[/query]

Voila! Our new query body is functionally identical to the original body, though admittedly a little more complicated to set up. However, the trade-off in efficiency is likely to be substantial.

I recently worked on a refactor for a client that was overall very similar to the above example, with a desired N value of 250. The code prior to refactoring took ~70s to complete. Once we had completed the refactor using the same tools as I’ve identified here, we brought down processing time to just under 3s, losing no functionality.

Time taken optimizing Interchange loops will almost always pay dividends.

Interchange 5.8.2 Release with Bcrypt Encryption Support

2014-07-09T00:00:00+00:00

The most recent release of Interchange contains support for adding encryption to user passwords with bcrypt. This provides for a significant improvement over the previously supported encryption options. Whether you are starting with a new user table, or have a long-established user table with any of the other supported encryption types, you can take advantage of this enhancement to Vend::UserDB.

In order to utilize bcrypt, you will need to have successfully installed the Digest::Bcrypt and Crypt::Random modules. Once installed, it is recommended (though not required, ironically) you add these as Required modules to interchange.cfg to ensure Interchange can find them and will report back immediately if not:

Require module Digest::Bcrypt
Require module Crypt::Random

To use bcrypt, configure your UserDB directive with the following options:

bcrypt—set to 1
cost—desired cost between 1 and 31 (optional; current default is 13)
bcrypt_pepper—random string used to apply a unique padding pattern (also optional, but strongly recommended)

To encourage adoption of bcrypt, there are two mechanisms available that allow a user table already using another encryption type to convert to using bcrypt. The first option is to add the “promote” directive—set to 1—to your UserDB configuration. This will cause passwords stored by any other encryption type to update to bcrypt upon the next successful access of that user account. This causes no disruption to site operations or your users, but it does mean you have to allow full adoption of bcrypt to advance organically as your users return to your site over time.

The second option builds on “promote” by allowing the developer to construct a background process to slurp in the existing passwords stored by another encryption type and generate a bcrypt storage value of the encrypted password, by utilizing the construct_bcrypt() subroutine in Vend::UserDB. construct_bcrypt() takes a single hashref argument with a required key “password” and optional keys “type” and “profile”. In this use case, password holds the encrypted password extracted from the existing user table and type contains one of “sha1”, “md5”, “md5_salted”, or “crypt”, corresponding to the encryption algorithm used to produce the password originally. The profile key maps to the profile name assigned to control this user table in the UserDB directive. Normally, this will be either “default”, which controls the userdb table, or “ui”, which controls the access table. Note that the profile used must already be configured to use bcrypt, or construct_bcrypt() will die. If profile is omitted, defaults to “default”. It is anticipated the developer would construct an Interchange job to query all the users and encrypted passwords still stored by the previous algorithm, pass each into construct_bcrypt(), and update the source record’s password field with the return value from construct_bcrypt().

For any configuration of UserDB utilizing bcrypt, it is recommended to enable “promote”. Not only is it a painless way to begin the transition of your encrypted user table to use bcrypt, it also ensures continuing access for your users even if you advance with the second option. For a large user table, it may take weeks for your background process to complete the transition. In the mean time, any users who access the site before their passwords are updated will still have access to the site using promote. Lastly, over time you will likely want to increase the cost parameter of bcrypt to keep pace with hardware advances. By using promote, users returning to your site will have their passwords updated to use the current cost in addition to new users being stored at the current cost. In short, if you bcrypt, use promote.

Example configuration changes for UserDB using bcrypt for the access table. Replace:

UserDB  ui  crypt 1

with:

UserDB  ui  bcrypt         1
UserDB  ui  promote        1
UserDB  ui  cost           13
UserDB  ui  bcrypt_pepper  mrPIQ4$.qgJrbvD(5CStB.p8b8ABbpl+:0.`I`*J`)j{=7KY*b(M@QXXg+9B*-b

Of course, your bcrypt_pepper will be some suitable random string you generate and keep private. The above is just an example.

Also be aware that the password storage will be larger than any previous stored passwords, up to 63 characters in length. So be sure your user table’s password field is sized (or resized, if necessary) to hold at least 63 characters.

Peppering your Padding

To provide additional support to protect weaker passwords against brute-force attacks, the process pads out password length to at least 72 characters. This padding is considerably more effective by adding a secret pepper that ensures the pattern cannot be reproduced without knowing the pepper value. This is, however, no substitute for enforcing good minimum standards on your passwords to ensure they are not weak in the first place, and does absolutely nothing to protect against a hacker guessing passwords through the front end.

Once a password hash is created with a particular pepper (which includes the possibility of no pepper) it is permanently dependent on that value. That is to say, the pepper should be settled on prior to introducing the use of bcrypt. If the pepper is changed, any hashes created and stored with the previous pepper (or no pepper) will become inaccessible. Even “promote” won’t help here because in order for promote to function, both encryption types (the original and the target) must be able to work against the supplied password. Once a new value of “bcrypt_pepper” is introduced, Vend::UserDB loses access to the old pepper and, thus, cannot use it to test the validity of the password proffered by the user against the existing stored data structure.

If it becomes unavoidable to change a pepper, a developer would have to create a new UserDB profile with the old pepper, write custom code to use the new profile to validate the password obtained from the user and, upon success, update the password field in the user table in the custom code with the new structure built with the profile holding the new pepper. This could be done using construct_bcrypt() but is in no way natively supported as a promotable option. Moreover, there is no mechanism to allow such an update to occur as a background process, as is available to migrate the other encryption types to bcrypt.

Interchange form pitfalls

2014-05-13T00:00:00+00:00

I’ll warn you going in: this is obscure Interchange internal form-wrangling voodoo, and even if you are familiar with that, it’s not going to be an easy trip to get from here to the end of the story. Fair warning!

Interchange form-handling has many, many features. Some of them are pretty obscure. One such is the concept of a “click map”: a snippet of named code, stored on the server. Specifically, it is stored within the user’s session, which is a chunk of storage associated with a browser cookie, and can contain all sorts of stuff—but which is primarily used for things like the shopping cart, the user’s identity, and so on.

But the tricky business here is when we put something into the session that says, in effect, “when we do this, do that, too”.

(Note: you don’t have to store your click map code in the session. In fact, it’s probably better not to, for a number of good reasons. You can put it in a sort of global configuration file, in which each individual snippet needs a unique name – which is precisely the requirement that would have prevented this bug, had I or one of my predecessors chosen to embrace that restriction.)

An Interchange page is both code and presentation (that is, variable assignments, ITL and/or Perl, plus HTML). No, this isn’t a great design feature—but it was considered so in an earlier, more innocent time.

I had one application that continued to present bugs that resisted solution, in fact they were virtually impossible to reproduce. Until one day, when I was testing a theory (which was totally wrong, but led me into the right spot): I landed on a page, then refreshed the page—and saw a totally different page. Here’s what happened, as simply as I can represent it.

Page1.html:
<form ...>
[set Continue]
mv_nextpage=Page2.html
[/set]
<input type="submit" name="mv_click" value="Continue"
</form>

Page2.html:
<form ...>
[set Continue]
mv_nextpage=Page3.html
[/set]
<input type="submit" name="mv_click" value="Continue"
</form>

When you request Page1.html, before the page is delivered to the browser, the session variable “Continue” is initialized with the code:

mv_nextpage=Page2.html

In Interchange terms, this tells Interchange where to go when form processing is complete. When we click on the “Continue” button, form processing happens on the server, and the “mv_click” button’s code is referenced * by name * as “Continue”. That takes us to “Page2.html”. When that page is prepared, Interchange overwrites the “Continue” variable with its new data, which is also a next-page setup.

Now, if we click the button on Page1.html, but then refresh the page on the resulting Page2, form processing is run again—but this time, our call-by-name to “Continue” executes the code that Page2 set up for its Continue button—not the Page1 setup. In other words: clicking the button on Page1.html sends us to page 2, but refreshing Page2.html * sends us to Page3.html*.

I can’t tell you what a relief this was to finally reproduce this bug. The simple fix was to just change the names involved, to “Continue1” and “Continue2”. There’s certainly a lot of power in Interchange form processing, but hoo boy—with great power comes great potential to shoot yourself in the foot.

Significant Whitespace in an Interchange UserTag

2014-03-18T00:00:00+00:00

Here’s a quick issue I ran into with an Interchange UserTag definition; I’d made some changes to a custom UserTag, but upon restarting the Interchange daemon, I ended up with a message like the following:

UserTag 'foo_user_tag' code is not a subroutine reference

This was odd, as I’d verified via perl -cw that the code in the UserTag itself was valid. After hunting through the changes, I noticed that the UserTag definition file was itself in DOS line-ending mode, but the changes I’d made were in normal Unix line-ending mode. This was apparently sufficient reason to confuse the UserTag parser.

Sure enough, changing all line endings in the file to match resulted in the successful use of the UserTag. For what it’s worth, it did not matter whether the line endings in the file were Unix or DOS, so long as it was consistent within the file itself.

Interchange table hacking

2014-03-11T00:00:00+00:00

Interchange has a powerful but terribly obscure table administration tool called the Table Editor. You can create, update, and delete rows, and even upload whole spreadsheets of data, but the Table Editor isn’t the most flexible thing in the world, so sometimes it just flat-out refuses to do what you want.

So you trick it.

A client wanted to upload data to a table that had a single-column primary key (serial), but also had a unique three-column key that was only used in the upload process (because the uploaded data was intended to replace rows with identical three-column combinations). Example:

In the table:

code: 243
field1: AAA
field2: BBB
field3: CCC
data-fields: ...

In the spreadsheet:

field1  field2  field3  data-fields...
 AAA     BBB     CCC     ...

In the database definition for this table, I had to add a secondary key definition for Interchange’s use:

Database  my_table  COMPOSITE_KEY  field1 field2 field3

in addition to the original key:

Database  my_table  KEY  code

Here’s the problem this presents: when you add a COMPOSITE_KEY to a table, the table editor refuses to show per-row checkboxes that allow you to delete rows. I thought I might have to write a custom admin page to carry this off, but then I had an inspiration—the REAL_NAME attribute of the Database directive:

Database  my_table_edit my_table_edit.txt __SQLDSN__
Database  my_table_edit REAL_NAME my_table
Database  my_table_edit KEY code
Database  my_table_edit  PREFER_NULL code
Database  my_table_edit  AUTO_SEQUENCE

This chunk of config-code tells Interchange that a second table can be accessed in the database, which is in fact the same table as the first (not a view, not a copy, but the actual table), but it doesn’t have the COMPOSITE_KEY. When Interchange is restarted with this new definition in place, the Table Editor will show this second table with the familiar per-row checkboxes, even as it refuses to show them for the original table.

Phew. I dodged a bullet with that, as I didn’t want to have to write a special page just to mimic the Table Editor.

Accepting Bitcoin with BitPay

2014-02-25T00:00:00+00:00

End Point recently began discussions with long-time Interchange client FrozenCPU about their desire to accept bitcoin payments. Because of the newness of bitcoin, as well as its non-traditional nature as a cryptocurrency, traditional payment gateways do not support it. While ideally suited to function as an exchange medium for ecommerce, we had to seek solutions that allowed for its integration into the checkout process.

BitPay Payment Gateway

After investigating their options, FrozenCPU elected to partner with BitPay to handle bitcoin payments. BitPay provides merchants with a solution similar in function to PayPal, where users are either redirected to the BitPay servers to complete payment, or the interaction with BitPay can be embedded directly into the merchant site via an iframe. In either case, all communication between customer and payment processor is direct.

Customers with a digital wallet then approve a transfer for the converted amount of bitcoin. Merchants need not display or maintain conversion rates; they can continue to use their native currency and BitPay will manage the exchange rate directly.

The payment negotiation is managed by the creation of an invoice which tracks the state of the payment through a sequence of status updates. Each status update can be thought of as analogous to a credit card transaction, but instead of a number of independent transactions related through references, the same invoice and identifier are used throughout.

The BitPay Process

At the point to finalize an order, a number of steps must be taken to complete the transaction using bitcoin. Again, the general process is most similar to PayPal, although no component of address management is included.

At the point where an authorization is normally issued for a credit card payment, instead the merchant generates a new invoice from BitPay, which starts in the status “new”.
Upon successful invoice generation, the merchant can either redirect the customer to BitPay’s server to present the payment screen, or embed the same through an iframe to keep the customer on site, if preferred.
Customer authorizes payment to transfer bitcoin from their digital wallet at the exchange rate provided from BitPay.
Once transferred, the invoice status moves to “paid” and the customer can then be presented with the order receipt, or in the case of redirect can click back on the merchant-provided backlink to the site to produce the receipt.
BitPay then negotiates a validation process over the bitcoin network, where the payment is “complete” after 6 block confirmations. Merchants can choose how quickly they want to accept the payment as valid, where a higher speed increases the odds of failing validation later in the process:
- High speed: immediately consider the payment “confirmed” once customer authorizes payment.
- Medium speed: consider payment “confirmed” once BitPay receives 1 block confirmation. Typically 5-10 minutes.
- Low speed: wait for all 6 block confirmations before considering the payment “confirmed”. Typically 45-60 minutes.
All payments must ultimately receive 6 block confirmations within 1 hour to complete. If this threshold cannot be met, the invoice status becomes “invalid” and the merchant must decide how to proceed.- Once the invoice status is “complete”, the transaction cannot be reversed. No chargebacks. Funds can be received by the merchant in the local currency or in bitcoin.
Invoices can be created with flags to either send notifications to an API the merchant creates, a designated email address, or both. This allows the merchant to keep on top of the transaction lifecycle in real time and minimize delays or disruptions on fulfillment.

Bumps in the Road

This was our first venture into an emerging field with a young technology. We hit a few difficulties on our path to implementation:

Not every status change on the invoice generates a notice, even when the fullNotifications flag is set to true. Customers have 15 minutes to pay an invoice, and if that time elapses the invoice status moves to “expired”. That left us with a dangling order because the IPN does not fire on that status update. We had to compensate by developing a monitoring script to look for those dangling orders older than 15 minutes, make an API call to BitPay to confirm the invoice status of “expired”, and cancel the order.
We initially received no update messages on status changes to the invoice that sourced back to problems in BitPay’s proxy software. However, BitPay was diligent in their efforts to troubleshoot the issue with us and they were able to upgrade the packages causing the problem. Their responsiveness in this matter certainly separates BitPay from other payment processors who are either entirely unresponsive to system bugs or are content to allow problems to persist through their regular release schedule.
Development and testing was complicated by the lack of a development environment at BitPay.

We are looking forward to the anticipated success of accepting bitcoin for payment on FrozenCPU and the opportunity to explore this offering with our many other ecommerce clients.

Interchange Redis Sessions

Session storage in Interchange

Using Redis for sessions

More lists and MoreDBTable

Locking and a friendlier welcome for bots

Wrapping up

References

Interchange Compression for SessionDB

Compression Performance Comparison

Configuring Your Catalog to Use Vend::Util::Compress

References

Interchange 3rd Party Tax Support

Configuring Your Catalog to Use Vend::Tax

References

Interchange rust_link connector

The Interchange server/​daemon

Web server connector

Interchange link protocol

Interchange link implementations

Enter Rust

rust_link

Rust impressions

Testing in Rust

Performance and resources

Publication

Reference

Perl Web Frameworks

CGI

Mojolicious

Interchange

Dancer (1 & 2)

Catalyst

Conclusion

Our Vue Storefront “Proof of Concept” Experience

PoC of Vue Storefront/​Magento 2.3

Preparing the Environment

Indexing Elasticsearch with mage2vuestorefront

Viewing the Vue Storefront Application

Designing a New Approach for Interchange

Inspired By A Similar Approach of Vue Storefront + SAP Commerce

Interchange API Prototype Design

Common and Exceptional Issues

Conclusion

Ecommerce sales tax primer

Talk to your Tax Accountant

Research Software

Why use an automated tax solution?

What to look for in automated tax software:

Implementation

Other resources

National Braille Press Releases Redesigned Website

Project Phases

Leveraging NBP In-House Resources

WCAG 2.0, AA Standard

Client Case Study: Vervante—​Publishing, Production and Fulfillment Services

A real-life scenario

Some Specifics

Requirements

Last, but not least

Zero Pricing in Interchange using CommonAdjust

Reach customers and drive sales with MailChimp

Endless possibilities

Automatic marketing

Steps to implement

Go beyond the simple newsletter

Improve SEO URLs for Interchange search pages

Creating a category listing page

Creating a “cat” actionmap

Cleaning up the links

2015 Perl Dancer Conference videos

Conference Recap

Conference Presentations

SpaceCamps “The Final Frontier”

AngularJS & Dancer for Modern Web Development

Lightning Talks

Summary

Perl Dancer Conference 2015 Report—​Conference Days

Day One

Day Two

Summary

The Interchange server/daemon

PoC of Vue Storefront/Magento 2.3

Client Case Study: Vervante—Publishing, Production and Fulfillment Services

Perl Dancer Conference 2015 Report—Conference Days