Containerizing Magento with Docker Compose: Elasticsearch, MySQL and Magento

2020-08-27T00:00:00+00:00

Magento is a complex piece of software, and as such, we need all the help we can get when it comes to developing customizations for it. A fully featured local development environment can do just that, but these can often times be very complex as well. It’d be nice to have some way to completely capture all the setup for such an environment and be able to get it all up and running quickly, repeatably… even with a single command. Well, Docker containers can help with that. And they can be easily provisioned with the Docker Compose tool.

In this post, we’re going to go in depth into how to fully containerize a Magento 2.4 installation for development, complete with its other dependencies Elasticsearch and MySQL. By the end of it, we’ll have a single command that sets up all the infrastructure needed to install and run Magento, and develop for it. Let’s get started.

Magento 2.4 application components

The first thing that we need to know is what the actual components of a Magento application are. Starting with 2.4, Magento requires access to an Elasticsearch service to power catalog searches. Other than that, we have the usual suspects for typical PHP applications. Here’s what we need:

MySQL
Elasticsearch
A web server running the Magento application

In terms of infrastructure, this is pretty straightforward. It would cleanly translate into three separate machines talking to each other via the network, but in the Docker world, each of these machines become containers. Since we need multiple containers for our infrastructure, things like Docker Compose can come in handy to orchestrate the creation of all that. So let’s get to it.

Creating a shared network

Since we want to create three separate containers that can talk to each other, we need to ask the Docker engine to create a network for them. This can be done with this self-explanatory command:

docker network create magento-demo-network

magento-demo-network is the name I’ve chosen for my network but you can choose whatever is most appropriate.

You can run the following command to check your newly created network:

docker network ls

Output usually looks like this:

$ docker network ls
NETWORK ID    NAME                  DRIVER  SCOPE
bd562b9cf5a4  bridge                bridge  local
adb9ec2365c5  host                  host    local
2dba8d97410e  magento-demo-network  bridge  local
c3473c60ed52  none                  null    local

There’s our magento-demo-network network among other networks that Docker creates by default.

Containerizing MySQL

Getting a MySQL instance up and running is super easy these days thanks to Docker. There’s already an official image for MySQL in Docker Hub so we will use that. We can set it up with this command:

docker run -d \
  --name magento-demo-mysql \
  --network magento-demo-network \
  --network-alias mysql \
  -p 3306:3306 \
  -v magento-demo-mysql-data:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=password \
  -e MYSQL_USER=kevin \
  -e MYSQL_PASSWORD=password \
  -e MYSQL_DATABASE=magento_demo \
  mysql:5.7

And just like that, we have a running MySQL instance. Running docker ps can get you a list of currently running containers. The one we just created should show up there.

$ docker ps
CONTAINER ID  IMAGE      COMMAND                 CREATED         STATUS         PORTS                              NAMES
b73739ad5d66  mysql:5.7  "docker-entrypoint.s…"  22 seconds ago  Up 21 seconds  0.0.0.0:3306->3306/tcp, 33060/tcp  magento-demo-mysql

Let’s go through each one of the options from that command now to understand it better.

docker run -d: Runs the container in detached mode. This means that it’s run in the background as a daemon. Control is returned to the console immediately.
--name magento-demo-mysql: This is the name of our container. Normally, Docker will generate random names for containers. In this case, we want to give it a name to refer to it with other Docker commands.
--network magento-demo-network: Tells Docker to run the container as part of the magento-demo-network network that we created earlier. This is the network that we will use for all of our containers.
--network-alias mysql: This is the name of this container within the network. This is how other containers in the network will be able to reference it. We’ll see that come to life a bit later.
-p 3306:3306: Sets up our new MySQL container to allow connections over port 3306. This is MySQL’s default port, which Magento will use to connect to it. This basically says “requests coming over the network to port 3306 of this container are going to be handled by the service installed in this container that listens to port 3306”. That service happens to be MySQL.
-v magento-demo-mysql-data:/var/lib/mysql: Creates a Docker volume. Specifically, we’re setting this one up to store the data files from MySQL. We need to do this so that the data stored in our MySQL container is persisted across shutdowns. magento-demo-mysql-data is the name of the volume and /var/lib/mysql is the directory within the MySQL container where that volume is mounted. In other words, any files stored in that directory are going to be stored within the volume instead. The volume is stored by Docker in the host machine, outside the container. /var/lib/mysql is the default directory where MySQL stores databases.
-e MYSQL_ROOT_PASSWORD=password: Is the password for the root user for MySQL. This is passed into the containerized MySQL via environment variables. Hence the -e option.
-e MYSQL_USER=kevin: Creates a new login in MySQL with kevin as its username.
-e MYSQL_PASSWORD=password: Sets the word password as the password for that kevin user.
-e MYSQL_DATABASE=magento_demo: Creates a database named magento_demo.
mysql:5.7: This is the image that we’re using for our container. 5.7 specifies the version that we want to run. The mysql image in Docker Hub contains a few more versions. Or “tags”, in Docker words.

Connecting to this container

docker ps showed us that our container was running. We can also interact with it. Here are a couple of ways of doing it:

Connecting from within the container

The easiest way of connecting to the MySQL instance is by running mysql CLI client from within the container itself. You can do that with:

docker exec -it magento-demo-mysql mysql -u kevin -p

Here’s how that command works:

docker exec -it is used to run commands inside a container in interactive mode. Just what we need here in this case because we’re running mysql, which is an interactive CLI.
magento-demo-mysql is the name we gave our container in the docker run command from before via the --name magento-demo-mysql option. This is why it’s useful to give names to containers: so we can use them in commands like this.
mysql -u kevin -p is the command that’s run within the container. This is just the usual way of connecting to a MySQL server instance using the mysql CLI client. We use kevin because that’s what we set MYSQL_USER to when we created our container before.

After running the previous command, the console will ask you for your password. We set that to password via MYSQL_PASSWORD so that’s what we need to type in. This will eventually result in the mysql prompt showing up. Run show databases to confirm that the magento_demo database that we specified via MYSQL_DATABASE got created.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| magento_demo       |
+--------------------+
2 rows in set (0.00 sec)

You can Ctrl + D your way out of that when you’re done exploring the containerized MySQL instance.

Connecting directly from the host machine

We can also connect to the MySQL instance running in the container, directly from our host machine. We can use:

mysql -h localhost -P 3306 --protocol=tcp -u kevin -p

Note that it is required that the mysql CLI client is installed in the host machine for this to work.

Same as before, mysql will ask you for the password and, once typed in, it will give you its prompt.

Containerizing Elasticsearch

Like MySQL, there’s an official Elasticsearch Docker image up in Docker Hub. As a result, getting a working Elasticsearch installation is a piece of cake. It’s done with a command like this:

docker run -d \
  --name magento-demo-elasticsearch \
  --network magento-demo-network \
  --network-alias elasticsearch \
  -p 9200:9200 \
  -p 9300:9300 \
  -e "discovery.type=single-node" \
  elasticsearch:7.8.1

You can validate that the Elasticsearch is running with curl localhost:9200/_cat/health. That should return something like this:

$ curl localhost:9200/_cat/health
1597622135 23:55:35 docker-cluster green 1 1 0 0 0 0 0 0 - 100.0%

Alright! That was easy enough. Again, thanks to Docker, we have an application that’s somewhat complex to set up, up and running in a matter of seconds.

Like before, let’s dissect that command that we used. Very similar to the MySQL one, only with some Elasticsearch specific settings:

docker run -d: Same as with the MySQL container, runs it in detached mode.
--name magento-demo-elasticsearch: Gives the container a friendly name.
--network magento-demo-network: Puts the container in the same network as the rest of our infrastructure.
--network-alias elasticsearch: Is the name by which other containers in the network can refer to this contianer.
-p 9200:9200: Opens port 9200 so that other containers within the network can talk to this one.
-p 9300:9300: Same thing but for a different port.
-e "discovery.type=single-node": Sets up the discovery.type environment variable that the image uses to configure Elasticsearch with.
elasticsearch:7.8.1: Specifies that our container will be running version 7.8.1 of Elasticsearch.

Containerizing Magento

Now this is the step where things get a little bit more involved. Nothing crazy however, so let’s get into it.

The Dockerfile

There’s no image of Magento 2 that would be able to get us up and running as quickly as with MySQL or Elasticsearch, at least not that I could find, so we’re going to have to create our own. We can create our own images with the help of Dockerfiles. A Dockerfile is a file that contains all the specifications needed for a container. The Docker engine uses it to create images which can then be used as basis for running containers.

Here’s a Dockerfile for Magento 2.4 that I came up with:

# /path/to/project/Dockerfile

# Our image is based on Ubuntu.
FROM ubuntu

# Here we define a few arguments to the Dockerfile. Specifically, the
# user, user id and group id for a new account that we will use to work
# as within our container.
ARG USER=docker
ARG UID=1000
ARG GID=1000

# Install PHP, composer and all extensions needed for Magento.
RUN apt-get update && apt-get install -y software-properties-common curl

RUN add-apt-repository ppa:ondrej/php
RUN apt-get update && apt-get install -y php
RUN apt-get update && apt-get install -y \
    php-mysql php-xml php-intl php-curl \
    php-bcmath php-gd php-mbstring php-soap php-zip \
    composer

# Install Xdebug for a better developer experience.
RUN apt-get update && apt-get install -y php-xdebug
RUN echo "xdebug.remote_enable=on" >> /etc/php/7.4/mods-available/xdebug.ini
RUN echo "xdebug.remote_autostart=on" >> /etc/php/7.4/mods-available/xdebug.ini

# Install the mysql CLI client.
RUN apt-get update && apt-get install -y mysql-client

# Set up a non-root user with sudo access.
RUN groupadd --gid $GID $USER \
    && useradd -s /bin/bash --uid $UID --gid $GID -m $USER \
    && apt-get install -y sudo \
    && echo "$USER ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USER \
    && chmod 0440 /etc/sudoers.d/$USER

# Use the non-root user to log in as into the container.
USER ${UID}:${GID}

# Set this as the default directory when we connect to the container.
WORKDIR /workspaces/magento-demo

# This is a quick hack to make sure the container has something to run
# when it starts, preventing it from closing itself automatically when
# created. You could also remove this and run the container with `docker
# run -t -d` to get the same effect. More on `docker run` further below.
CMD ["sleep", "infinity"]

Feel free to go through the comments in the file above for more details, but essentially, this Dockerfile describes what a machine ready to run Magento should look like. It’s got PHP and all the necessary extensions, Xdebug, and Composer. It also includes the mysql CLI client.

Importantly, it allows for creating a user account with sudo access. Later, we’ll use this capability to create a user account, inside the container that mimics the one we’re using in our host machine, effectively using the same user both inside and outside the container. The purpose of this is to make it possible to work on the Magento source code files from inside the container without having to deal with Linux permission issues when we try to do the same from outside the container (that is, directly via the host machine).

The image

Alright, now that we have our image defined in the form of our Dockerfile, let’s create it. To do that, we go into our project directory, create a new file named Dockerfile:

cd /path/to/project
touch Dockerfile

Then use a text editor to save the contents from above into it, and finally run this command:

docker build \
  --build-arg USER=kevin \
  --build-arg UID=$(id -u) \
  --build-arg GID=$(id -g) \
  -t magento-demo-web .

Here’s what this all means:

docker build: Is the command to build images from Dockerfiles.
--build-arg USER=kevin: Specifies the username for the account with sudo access that we will log into our container as. I’ve chosen kevin here but you should use the one you’re logged in as on your machine.
--build-arg UID=$(id -u): Uses the id -u to pass in the Id of the currently logged in user.
--build-arg GID=$(id -g): Uses the id -g to pass in the Group Id of the currently logged in user.
-t magento-demo-web .: Specifies the name of the resulting image to be magento-demo-web. The . is a reference to the current working directory from where we’re running the command, which is where our Dockerfile is located.

Run docker image ls and you should see our new home grown magento-demo-web image along with the other ones that we’ve downloaded from Docker Hub:

REPOSITORY        TAG     IMAGE ID      CREATED         SIZE
magento-demo-web  latest  90d311df434f  22 minutes ago  452MB
mysql             5.7     718a6da099d8  12 days ago     448MB
ubuntu            latest  1e4467b07108  3 weeks ago     73.9MB
elasticsearch     7.8.1   a529963ec236  3 weeks ago     811MB

The container

Ok, now that we have an image that’s capable of running Magento, let’s put it to work by creating a container based on it. We do that with:

docker run -d \
  --name magento-demo-web \
  --network magento-demo-network \
  --network-alias web \
  -p 5000:5000 \
  -v ${PWD}:/workspaces/magento-demo \
  magento-demo-web

Line by line, this is telling the Docker engine to:

docker run -d: Run the container in detached mode. You could also add the -t argument which makes sure the container stays up and running even if there’s no program or service running within it. We don’t need that in this case though, because we defined our Dockerfile with that nifty sleep infinity command.
--name magento-demo-web: Set the name of our container to magento-demo-web.
--network magento-demo-network: Make our container part of the same network as the MySQL and Elasticsearch ones.
--network-alias web: Set our container’s name within the network.
-p 5000:5000: Open port 5000 to access our soon-to-be-running Magento app.
-v ${PWD}:/workspaces/magento-demo: Create a new volume that makes our current working directory the same as the /workspaces/magento-demo directory within the container. This is where we’ll store all the Magento files. Binding these directories makes it possible to access and modify the Magento files both from the container and from the host machine. This just makes things easier and more convenient for development purposes.
magento-demo-web: Use this image.

Running docker container ls will show a list of all running containers, including the one we just created:

docker container ls
CONTAINER ID  IMAGE                COMMAND                 CREATED        STATUS        PORTS                                           NAMES
4af35c42e0bb  magento-demo-web     "/bin/bash"             5 minutes ago  Up 5 minutes  0.0.0.0:5000->5000/tcp                          magento-demo-web
6c5ea65a7bd6  elasticsearch:7.8.1  "/tini -- /usr/local…"  2 hours ago    Up 2 hours    0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp  magento-demo-elasticsearch
b73739ad5d66  mysql:5.7            "docker-entrypoint.s…"  3 hours ago    Up 3 hours    0.0.0.0:3306->3306/tcp, 33060/tcp               magento-demo-mysql

Connecting to the container

With the container up and running, we can connect to it with:

docker exec -it magento-demo-web bash

You may remember this as the same command we used before to connect to the MySQL container. This time, however, we’re using it to connect to our magento-demo-web container, referenced by the name we gave it, and running bash on it in order to open a shell.

After that, a prompt like this should show up:

kevin@4af35c42e0bb:/workspaces/magento-demo$

We’re now inside our container. Notice how we’re automatically taken to /workspaces/magento-demo. This is just like we specified in our Dockerfile with the WORKDIR command. Feel free to run php -v or composer -V to validate that the setup from our Dockerfile got all the way into our container:

kevin@4af35c42e0bb:/workspaces/magento-demo$ php -v
PHP 7.4.9 (cli) (built: Aug  7 2020 14:30:01) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.9, Copyright (c), by Zend Technologies
    with Xdebug v2.9.6, Copyright (c) 2002-2020, by Derick Rethans
kevin@4af35c42e0bb:/workspaces/magento-demo$ composer -V
Composer 1.10.1 2020-03-13 20:34:27

Talking to other containers in the network

We also need to validate that our containers are actually able to talk to each other via the network that we set up. If all went according to plan, still from within our magento-demo-web container, this command should open a mysql session:

mysql -h mysql -u kevin -p

Notice how this time we don’t use localhost or 127.0.0.1 to connect to our MySQL instance. This time, we use mysql. This is the network alias we gave out MySQL container, so this is how our magento-demo-web sees it. To magento-demo-web, the MySQL container is just another machine in the same network.

Same deal for the Elasticsearch container. We can do something like this to talk to it:

curl elasticsearch:9200/_cat/health

Again, from the perspective of magento-demo-web, this is just another machine in the network which it can reach by using the elasticsearch network alias that we gave it when creating it.

Installing Magento in our container

Now that we have our environment ready for Magento, let’s install it. First order of business is to create the Composer project:

composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition ./install

If you’re familiar with Composer, then this should look very familiar to you. This command will download all the Magento files as specified by the magento/project-community-edition project from the https://repo.magento.com/ repository. There are a few gotchas though:

First, Magento is not openly available to download just like that. As such, Composer will ask for authentication in order to do so. Follow this guide to obtain the authentication keys from the Magento Marketplace. When Composer asks for a username, type in the public key; when it asks for password, type in the private key.
Second, you’ll notice that I specified ./install at the end of that command. This is where all the files will be downloaded. I’ve chosen this (an install directory inside our current one) because composer create-project will refuse to download the files in a directory that’s not empty. Ours isn’t, because we’ve got our Dockerfile in it. But that’s nothing to worry about, once Composer finishes downloading everything, we’ll just copy the files over to their rightful location at /workspaces/magento-demo. You can do so with some Linux sorcery like this:

(shopt -s dotglob; mv -v ./install/* .)

This Composer operation will take a good while, but when it’s done, make sure to move all the contents of ./install into /workspaces/magento-demo. We now need to actually install Magento:

bin/magento setup:install \
  --base-url=http://localhost:5000 \
  --db-host=mysql \
  --db-name=magento_demo \
  --db-user=kevin \
  --db-password=password \
  --admin-firstname=admin \
  --admin-lastname=admin \
  --admin-email=admin@admin.com \
  --admin-user=admin \
  --admin-password=admin123 \
  --language=en_US \
  --currency=USD \
  --timezone=America/New_York \
  --use-rewrites=1 \
  --elasticsearch-host=elasticsearch \
  --elasticsearch-port=9200

Even if you have never installed Magento before, the command above should be pretty straightforward. An interesting thing to note is how we’ve set up our database and Elasticsearch settings here:

  --db-host=mysql \
  --db-name=magento_demo \
  --db-user=kevin \
  --db-password=password \

and

  --elasticsearch-host=elasticsearch \
  --elasticsearch-port=9200

--db-host is the hostname of the machine where the MySQL server is running. We use our container’s network alias here. --db-name is the name of the database we created when initializing our container via the MYSQL_DATABASE environment variable. --db-user and --db-password are the credentials for the login that we created in the same manner. --elasticsearch-host is the network alias of our Elasticsearch container, and finally --elasticsearch-port is the port that we configured it to listen to.

As you can see, these are the same settings that we used to configure our MySQL and Elasticsearch containers. So make sure to do the same if you’ve been following along and decided to go with different values.

Once that command is done, we’re ready. We have a working Magento. Try it out by running this:

php -S 0.0.0.0:5000 -t ./pub/ ./phpserver/router.php

And navigating to localhost:5000 in your browser. You should see your empty Magento homepage.

Optional: Installing the sample data

If you’re planning some custom extension, or to just play with Magento to get to know it better, you may want to add some sample data. Luckily, the Magento devs have graciously provided such a thing in the form of a Composer package. If you want, you can install it with this recipe:

bin/magento sampledata:deploy
bin/magento setup:upgrade
bin/magento indexer:reindex
bin/magento cache:flush

bin/magento sampledata:deploy will also ask you for your Magento Makerplace keys so have them ready.

So turn off the built-in PHP server, run these, wait a good while, and fire up the built in server once more. Your Magento app should now have a catalog and all sorts of other data loaded in.

Composing it all together

Now that was a lot. It was much easier than having to set everything up from scratch without Docker, but still, I promised a minimal setup overhead. A single command. With Docker Compose we can do just that.

For containers, the usual workflow is a three step process:

Create the Dockerfile (sometimes omitted if we have a readily available image like it was the case with MySQL and Elasticsearch).
Create or download an image.
Run the container.

Docker Compose can help us by capturing all the settings needed to create containers in a single YAML file; which then can be taken by a CLI tool (i.e. docker-compose) and it can set up the complete infrastructure. This single file is named docker-compose.yml and this is what it may look like for our current setup:

version: "3.8"

# Listing our three containers. Or "services", as known by Docker Compose.
services:
    # Defining our MySQL container.
    # "mysql" will be the network alias for this container.
    mysql:
        image: mysql:5.7
        container_name: magento-demo-mysql
        networks:
            - magento-demo-network
        ports:
            - "3306:3306"
        volumes:
            - magento-demo-mysql-data:/var/lib/mysql
        environment:
            MYSQL_ROOT_PASSWORD: password
            MYSQL_USER: kevin
            MYSQL_PASSWORD: password
            MYSQL_DATABASE: magento_demo

    # Defining our Elasticsearch container
    # "elasticsearch" will be the network alias for this container.
    elasticsearch:
        image: elasticsearch:7.8.1
        container_name: magento-demo-elasticsearch
        networks:
            - magento-demo-network
        ports:
            - "9200:9200"
            - "9300:9300"
        environment:
            discovery.type: single-node

    # Defining our custom Magento 2 container.
    # "web" will be the network alias for this container.
    web:
        # The build section tells Docker Compose how to build the image.
        # This essentially runs a "docker build" command.
        build:
            context: .
            dockerfile: Dockerfile
            args:
                USER: kevin
                UID: 1000
                GID: 1000
        container_name: magento-demo-web
        networks:
            - magento-demo-network
        ports:
            - "5000:5000"
        volumes:
            - .:/workspaces/magento-demo

# The volume that is used by the MySQL container
volumes:
    magento-demo-mysql-data:

# The network where all the containers will live
networks:
    magento-demo-network:

As you can see, most of docker-compose.yml is more or less rewriting the docker run commands in a YAML format. With the exception of the web container/service which includes a build section that reflects the docker build command that was used to take the Dockerfile and turn it into an image.

If you want to try it out, make sure to remove all the infrastructure we’ve created, to avoid any conflicts. You can do so from your host machine with these commands:

docker container rm -f magento-demo-web magento-demo-elasticsearch magento-demo-mysql
docker image rm magento-demo-web
docker network rm magento-demo-network
docker volume rm magento-demo-mysql-data

Make sure you’re in the directory where the Dockerfile lives in the host machine. Then create a new docker-compose.yml file and put all the content above into it. Finally, run:

docker-compose up -d

This will take a little while, but by the end of it, you’ll have a complete infrastructure with the three containers that we’ve created step by step throughout this article. With the docker-compose.yml file, docker-compose up essentially takes care of running all of our docker build and docker run commands.

The -d option means that the the command will run in the background and give you back control of your console. You can also run it without it if you want the console to show the logs from the containers.

You can still see the logs even in detached mode with:

docker-compose logs

You can also inspect the running containers. For that, you can use:

docker-compose ps

Output will look something like this:

$ docker-compose ps
           Name                         Command               State                       Ports                     
--------------------------------------------------------------------------------------------------------------------
magento-demo-elasticsearch   /tini -- /usr/local/bin/do ...   Up      0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp
magento-demo-mysql           docker-entrypoint.sh mysqld      Up      0.0.0.0:3306->3306/tcp, 33060/tcp             
magento-demo-web             sleep infinity                   Up      0.0.0.0:5000->5000/tcp

Notice how docker-compose ps gives us our container names just as we specified them in the docker-compose.yml file.

docker-compose has many other utilities, check them out with docker-compose --help.

Now, same as before, we still need to open a terminal into our Magento container to run some installation commands on it. To do so, we can run the following command:

docker-compose exec web bash

Notice how with docker-compose we refer to the container via its service name. That is, the name we gave the container under the services section of docker-compose.yml.

Of course, we can still use the same command that we used before, when we created our container directly with docker:

docker exec -it magento-demo-web bash

Now, once inside our container we need to install Magento again. Remember that we wiped out all the infrastructure we created manually, so these are fresh new containers; akin to new machines.

If you were running this from scratch you would just go ahead and do…

composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition ./install

and

(shopt -s dotglob; mv -v ./install/* .)

In this case, however, we already have all the Magento files in our directory, So we can save time and skip this step. We can reuse these files and just run bin/magento setup:install.

But since this is a new Magento installation, we do need to remove the config file before setup:install’ing. So go ahead and…

rm app/etc/env.php

…then:

bin/magento setup:install \
  --base-url=http://localhost:5000 \
  --db-host=mysql \
  --db-name=magento_demo \
  --db-user=kevin \
  --db-password=password \
  --admin-firstname=admin \
  --admin-lastname=admin \
  --admin-email=admin@admin.com \
  --admin-user=admin \
  --admin-password=admin123 \
  --language=en_US \
  --currency=USD \
  --timezone=America/New_York \
  --use-rewrites=1 \
  --elasticsearch-host=elasticsearch \
  --elasticsearch-port=9200

After a while, Magento will be fully installed in our new infrastructure created by Docker Compose and ready to be fired up via the PHP built in server:

php -S 0.0.0.0:5000 -t ./pub/ ./phpserver/router.php

Bonus: Interactive debugging with Visual Studio Code

So this is a fully functioning Magento installation with files that we can edit to our heart’s content. In terms of a “fully featured” development environment, however, we need to spruce it up a bit.

So install VS Code from https://code.visualstudio.com/ and install the Remote Development plugin.

Open a new VS Code window and open the command palette with Ctrl + Shift + P. In there, type in Remote-Containers: Attach to Running Container... and press Enter. In the menu that shows up, select our magento-demo-web container.

That will result in a new VS Code instance that is connected to the container. Open an integrated terminal in VS Code and you’ll see:

Now, install the PHP Debug extension so that we can take advantage of that Xdebug that we installed in our container via our Dockerfile.

Create a new launch configuration for interactive debugging with PHP by clicking on the “Run” button in the action bar to the left (Ctrl + Shift + D also works). Click the “create a launch.json file” link in the pane that appears. Then, in the resulting menu at the top of the window, select the “PHP” option. Here’s a screen capture for guidance:

That will result in a new .vscode/launch.json file created that contains the launch configuration for the PHP debugger.

Now let’s put a breakpoint anywhere, like in line 13 of the pub/index.php file; press the “Start debugging” button in the “Run” pane, near the top left of the screen (making sure that the “Listen to XDebug” option is selected), and start up the PHP built in server from VS Code’s integrated terminal with php -S 0.0.0.0:5000 -t ./pub/ ./phpserver/router.php. Now navigate to localhost:5000 in your browser and enjoy VS Code’s interactive debugging experience:

Summary

Whew! That was quite a bit. In this blog post, we’ve done a deep dive into how to set up all the pieces of a Magento application using Docker containers: MySQL, Elasticsearch, and Magento itself. Then, we captured all that knowledge into a single docker-compose.yml file which can be run with a single docker-compose up command to provision all the infrastructure in our local machine. As a cherry on top, we set up interactive debugging of our brand new Magento application with VS Code. Thanks to the safety net provided by these tools, I feel like I’m ready to really dig into Magento and start developing customizations, or debugging existing websites. If you’ve been following along this far, dear reader, I hope you do too.

Our Vue Storefront “Proof of Concept” Experience

2020-08-10T00:00:00+00:00

Recently we experimented internally with integrating Vue Storefront and Magento 2.3. Vue Storefront is an open source Progressive Web App (PWA) that aims to work with many ecommerce platforms.

What initially piqued our interest was the possibility of integrating Vue Storefront with the venerable ecommerce back-end platform Interchange, which many of our clients use. Vue Storefront’s promise of ease of integration with any ecommerce backend made us curious to see whether it would make a good modern front-end for Interchange.

Since Vue Storefront seems to be most commonly used with Magento, we decided to start our experiment with a standard Vue Storefront/Magento 2.3 proof-of-concept integration.

PoC of Vue Storefront/Magento 2.3

OK, to be honest, at the beginning we blindly expected that Vue Storefront would be a copy/paste front-end template solution that would fairly easily be made to work with its standard integration to a Magento backend. Sadly, this was not the case for us.

Before beginning our journey here, to summarize the Vue Storefront integration with Magento let’s have a look at this diagram to see what components are included:

Figure 1

At first, we wanted to see how all these components can be installed and run on a single server with modest resources.

I walked through the Vue Storefront documentation and a few blog posts to figure out Vue Storefront and Magento integration.

Preparing the Environment

I downloaded and installed the following:

OS: CentOS Linux 7 (64-bit)
PHP 7.2.26
Magento 2.3 with sample data
Elasticsearch 5.6
Redis
Docker
Vue Storefront API
Vue Storefront
mage2vuestorefront bridge

Installation of those components is fairly easy. We started our virtual server with 4 GB of memory, which we thought should be plenty for a toy setup.

Indexing Elasticsearch with mage2vuestorefront

PHP and Magento 2.3 had a series of memory usage issues. While running the mage2vuestorefront indexer Magento used most of the memory and caused Elasticsearch to go down each time I tried to index Elasticsearch. Then I configured PHP and Apache httpd server to use a very modest amount memory and made Magento 2.3 work on this server without making it crash due to unavailable memory but the performance became a nightmare on the PHP & Magento 2.3 side. mage2vuestorefront ran without issue, but very slowly.

I am not very familiar with the Magento 2 API, but while indexing Elasticsearch with mage2vuestorefront it makes several individual API calls. Especially if you have several products in Magento 2, mage2vuestorefront calls the Magento API for almost all products. Due to PHP and Magento’s high memory usage available memory might become a problem during indexing.

The Vue Storefront developers explain a big reason why they chose Elasticsearch as one of the essential components of this integration:

Vue Storefront defines itself backend-agnostic PWA e-commerce solution where Vue Storefront is a storefront as the name dictates, and Elasticsearch works as a datastore for catalog and its sibling data such as taxrule, products and so on.

I am unsure if the backend-agnostic approach even really applies to the suggested implementation of Vue Storefront with Magento 2, or if does apply with significant headaches. I’ll detail my concerns about this later.

Viewing the Vue Storefront Application

The responsive and offline supportive design of Vue Storefront is elegant. It uses Elasticsearch via vue-storefront-api as its indexing search engine to provide a seamless user experience. Although the overall integration and the produced data structure looks complex there are only a few catalog data stores that need to be indexed on Elasticsearch. Also, since this is an ecommerce integration, you can expect that only products and product-related data are going to be available in such an integration.

In the Vue Storefront and Vue Storefront API app home directories (vue-storefront/ and vue-storefront-api/) I ran npm start to start the Node.js apps, then navigated to the Vue Storefront homepage to view its default template:

Figure 2

There is a hamburger menu in the top-left which opens the categories menu of Vue Storefront:

Figure 3

As I mentioned, the data being viewed is relatively simple according to the count of the catalog types (essentially Products, Categories, Attributes, and Tax Rules) which are already categorized and these are a bunch of specific sets of data.

By searching through the categories Women > Tops > Jackets, the search output list is shown by Vue Storefront:

Figure 4

Designing a New Approach for Interchange

When exploring the features and components of Vue Storefront we discussed how we can adapt Interchange to use Vue Storefront. We ended up with two main options:

Keep the same approach of Vue Storefront’s current integration with Magento and other ecommerce backends using Elasticsearch and vue-storefront-api as the backend gateways
Remove the Elasticsearch and vue-storefront-api dependencies and create a new API in Interchange producing similar API outputs to the ones vue-storefront-api & Elasticsearch are creating. I also found a similar approach where they integrated SAP Hybris with Vue Storefront. I thought that this way would fit our intention.

Inspired By A Similar Approach of Vue Storefront + SAP Commerce

Vue Storefront + SAP Commerce: Open-source PWA Storefront Integration (+DEMO)

This article discusses several approaches, including Vue Storefront developers’ boilerplate integration and their custom solutions. In the following table they show different options and comments on their resolutions.

Figure 5

As you can see in Figure 5, they decided to go with the 4th option as their most suitable and least problematic solution. Here is their statement of why:

After some analysis, we have come to believe that using custom middleware only for a bunch of simple requests is an additional burden and thick layer of complexity. We decided to get rid of it as well.

So, option #4 turned out to be the best choice. For the option #4, we need to parse Magento and Elasticsearch APIs and generate the compatible responses.

Here is how their solution looks according to option #4:

Figure 6

Using this approach, they decided to remove the Elasticsearch and Vue Storefront API components from the Vue Storefront boilerplate integration and developed their SAP Commerce custom Vue Storefront API to produce responses similar to the responses taken from Elasticsearch through Vue Storefront API to pretend that Vue Storefront is taking its responses from Elasticsearch API + Magento API.

We decided to discard Elasticsearch when integrating Interchange with Vue Storefront for simplicity, and to cut down on the cost such additional search solutions create.

After our discussions, I was inspired by this SAP Commerce + Vue Storefront integration demo. We finally decided to move forward with creating an Interchange API to produce the similar responses to those from the Elasticsearch & Magento APIs, so we could make the Interchange API pretend to be the Elasticsearch API + Magento API.

We need to mention the Magento 2 API, or at least the Magento 2 data structure, because the Vue Storefront API and Vue Storefront boilerplate code is heavily designed to be taking ecommerce data (Products, Categories, Product Attributes, Tax Rules etc.) almost identical to the Magento API’s provided data.

So our Interchange + Vue Storefront integration looks like this:

Figure 7

Compare Figure 1 with Figure 7 to see the difference between this approach and Vue Storefront’s suggested solution.

Interchange API Prototype Design

In order to achieve the API prototyping for our Interchange/Vue Storefront integration, I cloned vue-storefront and vue-storefront-api, which I had previously integrated with Magento 2.3, into a new user, keeping all settings the same. This cloned Vue Storefront application started initially with the same configuration as vue-storefront-api and Elasticsearch were not excluded yet. So I decided to replace the vue-storefront-api endpoints that vue-storefront is calling with endpoints from the new Interchange API.

As the default SAP Commerce/Vue Storefront integration suggests, I walked through the outputs of search results in “product”, “category”, and “attributes”. Product data JSON output has some universal attributes like name or title, description, or price, but it also has some custom product attributes adapted from the Magento 2 API (specifically, the data structure of Magento is applied).

At this point I started questioning if Vue Storefront is not really platform agnostic, or if it is, but with some headaches. The next section will detail these headaches but to summarize, adapting the data structure of new ecommerce backends to Vue Storefront is a real pain. It forces you to use its data structure since that’s a part of core development, and they rely mainly on Elasticsearch, and I guess because of that the developers of Vue Storefront believe that Elasticsearch is an essential part of this “agnostic” approach. Refer to Getting data from Elasticsearch for custom implementations using the core Elasticsearch lib.

Anyway, I moved on and produced product and product categories outputs that Vue Storefront expects as the search results at the homepage and the search box output results.

Documentation is another pain. This is discussed further in the next section, but briefly, it’s hard to say that you could clearly pick the data structure that all kind of output results from the documentation. You need to dig into the TypeScript code, and the exact search output result is what is actually being expected.

One conflict was that the identifiers are required to be numeric, not strings. If your ecommerce data structure doesn’t provide its identifiers as numbers you’ll need to alter and add a new numeric unique identifier along with the existing one, which is what I did.

Product attributes also need to be adapted to Vue Storefront (really to Magento’s data structure). The attribute names, attribute lists, attributes’ assigned values, and categories attached to the products and configurable child products all come along with the product list on a search output. If your ecommerce data is not structured similarly to Magento’s data structure, you need to dig and restructure even for minimal changes like product attributes and configurable child products. For example, if your data structure doesn’t allow configurable child products then you can probably only provide each option (such as product color or size options) as a new product, which will be a real pain.

Most of Vue Storefront’s app endpoints are defined in config/default.json which is the default config on a new install. On the other hand, the ecommerce catalog data (which is indexed in Elasticsearch) is available through the Elasticsearch implementation which is by default defined in lib/search/adapter/api/searchAdapter.ts. In my case I needed to alter this implementation, which is generating the Elasticsearch (actually the Vue Storefront API) endpoints dynamically by looking up the related catalog index (product, category, attributes, taxes, etc.), and change the Elasticsearch definition in the Vue Storefront config file to the new Interchange API, adapted to emulate the Vue Storefront API and Elasticsearch:

...
  "elasticsearch": {
    "httpAuth": "",
    "host": "/api/catalog",
    "index": "vue_storefront_catalog",
...

I needed to set the Interchange API URL to the host property and not necessarily set an API-defining URL sub-path name to the index property above. This way our API would pretend to be the Elasticsearch endpoint as long as it can generate the JSON output that Vue Storefront expects.

These configuration changes generally satisfied Vue Storefront’s expectations.

Common and Exceptional Issues

Many concerns and challenges are discussed in this article on Vue Storefront + SAP Commerce integration which I mentioned earlier.

A review of some issues I detailed earlier:

Required numeric unique IDs for all catalogs (products, categories, etc.)
Poor documentation
Catalog data structure is managed on the Vue Storefront side, heavily relying on its own data types and models. Your backend may need to be adapted to meet Vue Storefront’s expectations, even if you are using Elasticsearch.
A structure which is constantly being reorganized, creating a big maintenance concern.

Conclusion

Despite my concerns, I thought we could still implement our Interchange API with a front-end framework, but I hadn’t asked the question: Is it reusable enough to mantain? Each of our clients would have different UI and UX expectations. One UI template model could never be enough, which would mean continuous front-end template design and enough maintenance to match the operational cost of existing Interchange front-end solutions. Adapting existing data to Vue Storefront’s new data model would bring more costs along with it.

While Vue Storefront’s offline working model is well designed, for the purpose of integrating with a non-Magento backend, this is overshadowed by its many issues. You would be better off looking elsewhere for a more modular, truly platform-agnostic, easier-maintained, component- and data model-independent front-end.

Magento 2: Creating a custom theme

2020-06-24T00:00:00+00:00

Photo by Maria Eklind, CC BY-SA 2.0

In my previous post, we went through the steps needed to create a custom module in Magento 2. While modules consist of a set of classes to add new features to Magento, a theme controls how these features, and the entire website in general, will be displayed to the user. As stated in the Magento guide, a theme uses a combination of custom templates, layouts, styles, and images to provide a consistent look and feel across a Magento store.

Creating a new Magento 2 theme

We can create a theme based on a default “parent” theme or create a standalone theme from scratch. In most cases, I would recommend the first option. For this example, we will use Luma as our parent theme. The other option would be inheriting from the default “blank” theme.

Here’s an initial task list to get our new theme ready:

Create a new directory for the theme
Create the registration.php script
Create the theme.xml information file
Activate the new theme

Creating a new directory for the theme

While all our backend code should go in app/code, the frontend content is expected to go in app/design. And as our theme will only apply design changes to the frontend content, we should create the new directory for it under the path app/design/frontend. If we want to create a theme for the admin area instead, we need to create the directory inside app/design/adminhtml.

Let’s create a directory named EndPoint (our vendor name, continuing with the example from our previous article) and a subdirectory inside it, MyTheme:

cd {website_root}
mkdir -p app/design/frontend/EndPoint/MyTheme

Creating registration.php

Similar to the file we created for our module, registration.php tells Magento to register the new theme with the name and location we specify. Our file will be located at app/design/frontend/EndPoint/MyTheme/registration.php and should have the following content:

<?php
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::THEME,
    'frontend/EndPoint/MyTheme',
    __DIR__
);

This way, Magento will know what path our theme will have.

Creating theme.xml

The next step is to create our theme information file, where we will specify the theme name and parent theme. So our app/design/frontend/EndPoint/MyTheme/theme.xml file should have the following content:

<?xml version="1.0"?>
<theme xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Config/etc/theme.xsd">
    <title>MyTheme</title>
    <parent>Magento/luma</parent>
</theme>

Optional: If we want our theme to be easily distributed as a package, we can also create a composer.json file in the theme’s root directory. The content for the file should be as follows, specifying the theme’s description, dependencies, version, and license types:

{
    "name": "endpoint/mytheme",
    "description": "My Theme by End Point",
    "require": {
        "magento/theme-frontend-luma": "100.0.*",
        "magento/framework": "100.0.*"
    },
    "type": "magento2-theme",
    "version": "100.0.1",
    "license": [
        "OSL-3.0",
        "AFL-3.0"
    ],
    "autoload": {
        "files": [
            "registration.php"
        ]
    }
}

Activating our new theme

That was easy! We have everything we need to activate our new theme. To do that we log in to our admin area and enable our theme. Once in the dashboard, we need to go to Content > Design > Configuration, edit our store view, and select our new theme from the dropdown list:

Magento will search for new themes every time we log in to the admin area, so our new theme will appear on the list automatically.

Adding custom content

We have the basic structure for our theme, but when enabled, it will look the same as its parent theme (Luma, in this case), since we didn’t add any design rules or static files yet. Let’s do some more things with our theme to change how it’s displayed in the frontend:

Create a custom etc/view.xml file
Add a custom logo
Add static files (JavaScript, CSS, images, fonts)
Add a custom layout

Creating a custom view file

etc/view.xml controls many frontend configurations like the product thumbnail width, how the product image gallery is displayed, and the image magnifier tool, among other things. To add our custom view file to our theme, we need to copy the existing file from our parent theme. For Luma, it will be located at vendor/magento/theme-frontend-luma/etc/view.xml. To copy the file, we need to run the following in our website’s root:

mkdir -p app/design/frontend/EndPoint/MyTheme/etc
cp vendor/magento/theme-frontend-blank/etc/view.xml app/design/frontend/EndPoint/MyTheme/etc/view.xml

And then we can use our preferred text editor to change the values we want, like setting a custom size for the images in the category page grid:

<image id="category_page_grid" type="small_image">
    <width>300</width>
    <height>300</height>
</image>

Adding a custom logo

Adding a logo to our theme is really simple. We just need to save our picture in SVG format as web/images/logo.svg. If we want to use a different filename or format for our logo, we will have to create a default layout file for our theme in the path /Magento_Theme/layout/default.xml inside our theme root with content similar to this:

<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
  <body>
    <referenceBlock name="logo">
      <arguments>
        <argument name="logo_file" xsi:type="string">images/custom_logo.png</argument>
        <argument name="logo_width" xsi:type="number">300</argument>
        <argument name="logo_height" xsi:type="number">200</argument>
        <argument name="logo_alt" xsi:type="string">Custom logo name</argument>
      </arguments>
    </referenceBlock>
  </body>
</page>

We can use different image formats such as SVG, PNG, or JPG. We can also use a custom width and height for the logo, and set a custom alternate text.

Adding static files (JavaScript/CSS/images/fonts)

All the static files should be located inside the web directory. Common static files include JavaScript files, stylesheets, images, and fonts. The JavaScript files should be located at web/js, stylesheets at web/css, images at web/images, and our custom fonts should be located at web/fonts.

All the static files will be published as direct links, without any processing from Magento, at the pub/static/frontend/EndPoint/MyTheme/en_US path. The default locale/language is en_US; we can change it for our theme if needed.

Adding a custom layout

Finally, if we want to use the new assets we added and have custom content on different sections of our website, we need to extend or override the existing layout files from our parent theme.

For example, if we want to add a reference to new stylesheet and JavaScript files we added, we need to extend the existing header layout from our parent theme. To do this, we will create a new layout file located at Magento_Theme/layout/default_head_blocks.xml:

<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
  <head>
    <!-- Custom stylesheet -->
    <css src="css/mytheme.css"/>

    <!-- Custom JavaScript -->
    <script src="js/mytheme.js"/>
  </head>
</page>

This way we will be adding a reference to a new stylesheet named mytheme.css that we have inside the web/css directory of our theme, and a new script named mytheme.js inside the web/js directory.

After we make all the desired changes to our theme, we will need to tell Magento to update the frontend. We need to deploy the new changes and clear the cache. We can achieve that by running this from our website root:

php bin/magento setup:static-content:deploy
php bin/magento cache:clean

This process can take some minutes to complete. After it’s done, we can go to our website to see if the frontend changes are applied:

Looks awesome! Of course, there’s a lot more we can do from there, from extending or overriding layouts from modules (Magento or third-party) to bundling scripts or using Less files for our custom styles. But, that is material for later posts, so that’s all for now! Please add any questions you might have below.

Magento 2: Creating a custom module

2020-04-01T00:00:00+00:00

Photo by Babatunde Olajide, cropped from original

A Magento module is a set of classes and routines that will depend on and interact with other Magento classes in order to add a specific feature to a Magento application. While a theme is orientated towards the front-end and user experience, a module is orientated towards backend logic and application flow.

We will need to create a custom module if we want to add or change the existing logic at a level where Magento doesn’t provide a setting or option for it. For example, if our business has a specific feature or set of features or requirements that are not common to the market, a module can fill that gap for us.

Creating a basic Magento 2 module

Creating a simple module in Magento 2 is not that hard. We will need to accomplish the following tasks:

Create a new directory for the module
Create a registration.php script
Create a etc/module.xml information file
Install the new module

Creating a new directory for the module

Where should the new directory for our module be placed? We have two options to choose from:

app/code/{vendor}/
vendor/{vendor}/

If your module is intended for a specific website you’re working on, you can use the first option. If you’re creating a module with the intention of it being used on several websites, it’s best to choose the second option. We’ll use the first for this example.

Let’s create a directory named EndPoint (our vendor name) with a subdirectory inside it, MyModule:

cd {website_root}
mkdir -p app/code/EndPoint/MyModule

Creating the registration.php script

The registration.php file tells Magento to register the new module under a specific name and location. Let’s create a file named app/code/EndPoint/MyModule/registration.php with the folllowing content:

<?php
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'EndPoint_MyModule',
    __DIR__
);

We’re telling Magento that our module will be named EndPoint_MyModule.

Creating the etc/module.xml information file

Now, let’s create our module information file, where we’ll specify the module version number. First, we need to create the etc directory inside app/code/EndPoint/MyModule:

mkdir app/code/EndPoint/MyModule/etc

then create module.xml with the following content:

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="EndPoint_MyModule" setup_version="1.0.0">
    </module>
</config>

Installing the new module

That’s it! We have everything we need to install our new module. Now we need to tell Magento we want to install and enable our new module. So from our website root we need to run:

php bin/magento setup:upgrade

Magento will output a list of module names and configuration updates, and our new module EndPoint_MyModule should be listed in that output.

Adding a custom route to our module

Now we have a working, enabled module, but it’s not doing anything yet! What’s a simple way to check that our module is enabled? Let’s set up a custom route, so if we hit a URL like https://{our_website}/mymodule/test/helloworld we can return a custom response from a controller.

Creating a custom route will need some steps on its own:

Create a new directory for the controller
Create a etc/routes.xml file
Create the controller
Upgrade the new module

Creating a new directory for the controller

First we need to create a new directory where the new PHP controller for our custom route will live. The new directory path should be:

app/code/EndPoint/MyModule/Controller

We can create as many directory levels we want, depending on our desired path. For example, if we create a class named Index in app/code/EndPoint/MyModule/Controller, the URL that will be routed to this controller will be https://{our_website}/mymodule/index (the “Controller” directory is ignored). If we create a class named HelloWorld in app/code/EndPoint/MyModule/Controller/Test, the resulting URL will be https://{our_website}/mymodule/test/helloworld.

Creating the etc/routes.xml file

routes.xml will tell Magento what base URL will be used for our module. First, we need to create the “frontend” directory where the routes.xml file needs to be placed:

mkdir app/code/EndPoint/MyModule/etc/frontend

In this example, we want the base URL to be MyModule, so we need to create an XML file inside the new directory that will route all requests made to the given URL to our module controllers:

<?xml version="1.0" ?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:App/etc/routes.xsd">
    <router id="standard">
        <route frontName="mymodule" id="mymodule">
            <module name="EndPoint_MyModule"/>
        </route>
    </router>
</config>

Creating the controller

If we want to respond to requests for https://{our_website}/mymodule/test/helloworld we first need to create the base Controller directory and the Test subdirectory:

mkdir -p app/code/EndPoint/MyModule/Controller/Test

Under this directory, we’ll create our custom Magento controller. All route controllers should extend \Magento\Framework\App\Action\Action. We also need to have a public construct() method to pass the context to our ancestor and an execute() function that will be called when the URL is hit:

<?php

namespace EndPoint\MyModule\Controller\Test;

class HelloWorld extends \Magento\Framework\App\Action\Action
{

    public function __construct(
        \Magento\Framework\App\Action\Context $context
    ) {
        parent::__construct(
            $context
        );
    }

    public function execute()
    {
        echo "Hello world!";
    }

}

Upgrading the new module

We have everything in place to tell Magento we have new changes to be deployed. How we do that? First, we need to upgrade our Magento setup. But since we added a new controller that gets parameters from the dependency injector in the construct, we also need to compile the dependency injection engine (including factories, proxies, and interceptors). And finally, we need to clear the cache so new content will be served from our custom URL:

php bin/magento setup:upgrade
php bin/magento setup:di:compile
php bin/magento cache:flush

This process can take a few minutes to complete, but after it’s done we can try to reach our new custom URL. If we get a response like the one below:

That means our module is working!

That’s all for now. In upcoming posts, we’ll start complicating things a bit by overriding Magento classes with our custom ones and creating custom controllers that will return information from the Magento core classes. We will also explore how to customize the front-end by creating a theme. Don’t forget to add any questions, suggestions, or issues in the comments below!

Ecommerce sales tax primer

2020-01-13T00:00:00+00:00

Co-authored by Greg Hanson

Source image

Tax collection is one of the topics du jour for those of us in the ecommerce industry. Since state and local authorities are now able to levy taxes on ecommerce goods, taxation for online stores has become quite complicated. The purpose of this post is to give you some next steps and ideas on implementation if you’re new to the topic and need to know how to get started on tax collection for your ecommerce business.

Current ecommerce sales tax policy stems from the 2018 U.S. Supreme Court decision South Dakota v. Wayfair, Inc. Since that decision, favoring South Dakota, 30 states have enacted legislation to require ecommerce stores to pay sales tax if they fit the definition of having an ‘economic nexus’, that is, they do enough business in the state to be worth taxing.

Talk to your Tax Accountant

So the first and most important note is to get your own legal counsel in regards to your taxes. There are many rules and things are changing every month with local and state authorities, so you’ll need reliable counsel on the topic.

If you’re looking for someone to help, make sure this person has:

Knowledge about product variants. For example, clothing may not be taxed in some areas.
Familiarity with tax policy in the entire country, and not just one local retail area.
The ability to help you determine in which states you have a tax ‘nexus’ and need to pay sales tax. For many small businesses, you might only do enough business to pay sales tax in your home state and a few large ones like California or New York.

Research Software

Luckily for anyone starting to collect sales tax, there are some very good SaaS (software as a service) systems out there to make tax collection, reporting, and compliance easy. These software systems typically integrate with your ecommerce store by providing the store with the correct tax amount for the sale and collecting information for you on your reports and filing information to get the taxes paid to the correct authorities. Some systems might even file things for you.

After you’ve worked with your tax accountant on what you need, start looking at the available software. The two main companies End Point has worked with are TaxJar and Avalara, though there are a number of other vendors in the market.

Why use an automated tax solution?

They automatically update tax rates as they change with local and state regulation.
They can integrate into your checkout processes via API or plugin to automatically calculate the correct tax for the right location and product.
They can have product-specific taxes, for things that might not be taxed like food and clothing in certain states.
They give you end-of-year reports and help with your payments.
You can customize settings, in case there are some states where you have a ‘nexus’ (are subject to sales tax) and other places where sales tax payments are not required.

What to look for in automated tax software:

Choose a reputable company with large brand presence. You don’t want to do anything experimental and unreliable here.
Pick something with documentation you can understand. You don’t want working with your tax software to be a pain, and you might need to refer to their documentation when configuring it or changing things in the future. Make sure that company speaks your language and you can easily get answers to your questions.
This company should be able to guide you through all aspects of tax collections, filing and payment. You will legally be liable for taxes on all sales in states where you have nexus, whether you collect the taxes or not! So make sure the company you select can work with you not only to set up the system, but on an ongoing basis providing support in filing reports and making payments.
Integration with your platform is a key component of what you choose. Both Avalara and TaxJar have existing plugins for sites running on WordPress, Shopify, BigCommerce, Magento, and others. Keep in mind though that the integration might be different depending on your platform. If you’re on a custom platform, talk to your development team about integrations; they can read the docs and give you an estimate and recommendation for ease of implementation (that’s where we come in for many clients). For Interchange stores, we have integration code for both TaxJar and Avilara to leverage.
Consider how your inventory or ERP system might be affected. Many of our ecommerce clients sell in-house or over the phone. Consider how your other systems might need to tie into this new tax system.

Implementation

So you’ve done the hard part, right? You sorted out what states you need to be compliant with and picked a software solution. Now all you need to do is get it working. This is really where your software consultant, such as us at End Point, would come in and get you to the finish line. The steps to implementation are:

Set up your account and pay for your tax solution software.
Work with the tax solution provider to set up any required bank accounts or payment channels.
Configure your settings.
Share the API key and information with your developer.
Test the implementation. I recommend you do this with several orders in different scenarios: products that do and don’t have sales tax, locations that do and don’t have tax, locations that tax shipping, etc. Test all the variants you know about.
Go live with your tax solution on your site.
Make sure to check back for your reports and filings for later in the year.

Need help picking a system or looking at implementation? Call us today and we can help.

Other resources

Running Magento 2 in Windows with XAMPP

2019-03-22T00:00:00+00:00

Photo by Nicole De Khors · Burst, Some Rights Reserved

Magento is an open source ecommerce platform, written in PHP and relying on MySQL/MariaDB for persistence. According to BuiltWith, Magento is the third most used platform in ecommerce websites. It began its life in 2008 with its first general release, and a major update (Magento 2) was released in 2015.

And now, more than three years after, Magento 1 is slowly dying: There won’t be any more quality fixes or security updates from June 2020, and there won’t be extended support for fixes or new payment methods. So the obvious choice will be Magento 2 from now on.

But is it fully tested yet? Is it stable enough? If we already have a website running with Magento 1, what should we do? Migrating to Magento 2 is not just hitting an “Update” button: Themes are incompatible, most extensions won’t work, and of course, there’s a big set of changes to get familiar with.

So a good approach might be to get a clean Magento 2 version deployed locally, to look what we need to do to get our website updated and running, test the backend, find where the configuration sections are located, and so on. And many business users, and even some developers like myself, have Microsoft Windows installed on our computers.

Environment setup

The environment I used for this testing installation was Windows 10 Professional. As a first step, we’ll need to make sure that localhost is published in our local hosts file:

Navigate to the folder %SystemRoot%\system32\drivers\etc
Backup the existing hosts file
Open a text editor with administrator rights
Open the hosts file
Make sure the first line after the commented (#) lines is 127.0.0.1 localhost and the second is ::1 localhost
Open a cmd window with administrator rights and run the command ipconfig /flushdns

Now we’re ready to install the environment needed to run Magento. I recommend using XAMPP, a free Apache distribution for Windows that includes MariaDB, PHP, and Perl in a single package. Magento 2 currently runs with PHP 7.2 and it will not work with newer versions.

Install XAMPP 7.2 with default settings: apachefriends.org/xampp-files/7.2.15/xampp-win32-7.2.15-0-VC15-installer.exe
Download Magento 2. You will need to register on the website first: magento.com/tech-resources/download
Create a new “magento” subfolder (or whatever name you prefer) inside the htdocs folder in the XAMPP installation (usually C:\xampp) and uncompress the Magento 2 archive there.
Start the XAMPP Control Panel from the Windows start menu. In the “Apache” section, click the “Config” button and, on the menu that appears, select “PHP (php.ini)”. Remove the semicolon before the extension=intl, extension=soap, and extension=xsl texts to enable the intl, soap and xsl extensions.

Screenshot of the XAMPP Control Panel with the PHP config menu displayed.

MySQL and Magento setup

We have all the files in place and the environment ready to start configuring the database and install Magento 2.

Start the Apache and MySQL services from the XAMPP Control Panel. Wait for the green status texts to appear.
Create a “magento” (or whatever name you prefer) database in MySQL from phpMyAdmin, installed already on XAMPP: localhost/phpmyadmin
Run the Magento 2 setup from localhost/magento (replace the “magento” part of the URL with whatever name you have chosen to host Magento). If the setup program requires to do any additional configuration change, do it as instructed. Do a screenshot or save the final page contents for later use.

Example of the page that will be displayed when the Magento 2 installation finishes.

Fixing known issues

When we finish the installation process, we will have a Magento 2 instance running on our host. But we’re not ready yet! There are a couple known bugs with Magento 2 and XAMPP at the moment I’m writing this post:

The Magento admin page may not work (it shows a blank page): Fix this issue by updating the “isPathInDirectories” function inside the Validator.php file as instructed in this article.
We might not be able to upload images (like a custom logo or product pictures): Fix this issue by updating the design_config_form.xml file as shown here.

After we finish making these changes, we will need to restart the Apache service from the XAMPP control panel. And that’s it! We should be ready to open the Magento front page, and login into the backend.

An empty home page with the default theme enabled in Magento 2.

This is how the Magento 2 back-end dashboard looks like once we login.

Setting up the store: A roadmap

The purpose of this article is to get Magento 2 up and running on Windows computer, and that’s what I hope we’ve achieved so far. But just to point in a direction on what to do next, below is my usual roadmap to set up the store, add a few products and publish the catalog on the home page.

Change the default store name and logo from the section Content > Design > Configuration and choose “Edit” in the “Default store view”.
Set up the default store configuration from the section Stores > Configuration.
Create the product categories from the section Catalog > Categories. Set up a category name and its dependency from another category, or from the default master category.
Create/edit the product attributes from the section Stores > Attributes (check the option “visible in storefront” to show the products in the product page). For example, custom attributes for book products would be Author, Publisher…
Add the attributes to the default attribute set from the section Stores > Attribute. This is required for the attributes to show up when adding a new product by default.
Create a set of products from the section Catalog > Products. Set the custom variations for each product from the “Customizable options” section. Set up a short and long description, a custom SKU for each customizable option, and a tax mode.
Add a widget to the homepage to show the product catalog from the section Content > Homepage > Edit > Show/Hide Editor > Add widget > Products list. Here we can also add any content we like, including HTML tags, images, and third-party content.

Conclusion

It’s possible—and pretty straightforward—to run a Magento 2 instance in Windows with XAMPP. But this is just a kick-off, a simple overview: there’s a lot more to do if we want to get a full ecommerce website that is ready to scale up and support enterprise-level traffic, like caching optimization, load balancing, and advanced server-side monitoring.

At End Point, we have professionals with a strong background in deploying powerful, reliable, fully responsive, fast, and SEO-optimized ecommerce websites. We have experience in migrating Magento to newer versions, and even creating full Magento 2 websites from scratch. Drop us a line if you want to hear more.

Ecommerce Shakeups: Magento Acquisition and Etsy Rate Increases

2018-06-19T00:00:00+00:00

If you’ve been paying any attention to much in the ecommerce world, there have been a couple shakeups and transitions that could affect how you look at your ecommerce options these days.

Adobe to Acquire Magento

A few weeks ago, it was announced Adobe would acquire Magento in a large acquisition. We’ve seen Magento clients come and go. It used to be the case that the Magento Community version was free and open source, but lacking in features, and the Magento Enterprise version was not free and came with many more features but was closed source.

But, times change, and admittedly I haven’t looked into the current Magento offerings until writing this post. The two current options for Magento are Magento Commerce Starter and Magento Commerce Pro, more details here. These plans are not for small potatoes, starting at $2k/mo. I can see how the cost of this is worth it in lieu of paying a full time developer, but this is not a good fit for small businesses just getting started.

There at not many public details on the acquisition, other than bringing Magento to Adobe’s range of “physical and digital goods across a range of industries, including consumer packaged goods, retail, wholesale, manufacturing, and the public sector”. Only time will tell.

Etsy Hikes Rates

I am personally connected to the craft industry by way of my own hobby, so I’ve heard rumblings of the changes going on within Etsy with a new CEO throughout the last year. They will be shutting down Etsy wholesale as of July 31st, 2018, closing Etsy Studio & Manufacturing later this year, and last week, they announced increasing transaction fees from 3.5% to 5% which will now also apply to shipping charges. With that money, they will be offering improved tools and marketing efforts. You can read the official announcement here and more Q&A from Etsy here.

There are so many overwhelming options when it comes to determining what the best ecommerce solution is for any size of ecommerce business, and whether or not now is the right time to move away from Etsy. There is no one size fits all approach, and the best solution is very dependent on the company size and sales, in-house personnel and their skillset, structural organization, and so much more.

There are many competing SaaS options like Shopify, Big Commerce, Big Cartel, Squarespace. We’ve helped set up these software as a solution environments for many clients and they can be a great fit for business owners interested in offloading support and maintenance. In addition to SaaS options, WooCommerce in WordPress is the most popular open-source [free] solution, although many plugins and themes for it are not free. There are plenty of other open source ecommerce options out there, running on a web framework/language (Ruby on Rails, Django, MVC in PHP, etc.) of your choosing.

Conclusion

Like I mentioned above, finding the right solution is very circumstantial. I tend to suggest starting with a low-investment starting point for ecommerce, in whatever shape or form that takes for you, and then incrementally moving up if that’s what you need.

While Etsy might offer a nice low-investment starting point, more exposure and marketing tools, you ultimately aren’t building your own brand on your own domain. I have seen a handful of people in the craft industry keep their Etsy shop open for a specific audience but grow things on their own domain running on another SaaS platform.

On the flip side, many of End Point clients have businesses that are further along and the more enterprisey versions or custom software development solutions make great sense for them, with or without in-house development staff.

If you’re interested in finding out more about setting up a SaaS shop, moving away from Etsy, or understanding your options with Magento, contact us to find out how we can support your business goals.

Reach customers and drive sales with MailChimp

2016-08-22T00:00:00+00:00

It’s a good idea for ecommerce stores to regularly contact their customers. This not only reminds customers that your business exists, but also allows the sharing of new products and resources that can enrich the lives of your customers and clients. One of the easiest ways to stay in touch is by using an email newsletter service, such as MailChimp.

<img alt="" border="0" height="200" src="/blog/2016/08/reach-customers-and-drive-sales-with/image-0.png" title="Freddie, the MailChimp mascot" width="191"/>

MailChimp offers the regular suite of email newsletter services: lists, campaigns, and reports—but in addition, they allow an ecommerce store to integrate sales data back into MailChimp. When you have detailed shopping statistics for each subscriber, it opens new possibilities for customized marketing campaigns.

Endless possibilities

For example, imagine you have an email mailing list with 1,000 recipients. Instead of mailing the same generic newsletter to each subscriber, what if you could segment the list to identify your 100 best customers, and email them a special campaign?

Additional ideas could include:

Reach out to inactive subscribers, offering a coupon
Invite your best customers to a secret sale
Re-engage customers who placed items in their cart, but left without purchasing
Offer complementary products to purchasers of Product X

Automatic marketing

Once your store has sales data for subscribers, and you’ve decided on the campaigns you want to run with this data, the next step is to automate the process. This is where MailChimp’s Automation feature comes in. Spend some time up-front to craft the automated campaigns, then sit back and let MailChimp run them for you; day in, day out.

Steps to implement

There are several off-the-shelf integrations for ecommerce stores, including Magento and BigCommerce.

Users of Perl and Interchange can use our newly-released toolsets of the Mail::Chimp3 CPAN module and the integration for Interchange5.

Most businesses already have an email newsletter. Hopefully, you are sending regular email campaigns with it. This is a great first step. Going beyond this to segment your email list and reach out to these segments with relevant information to each of them, is the next step. Not only can this increase your sales, but it also respects your clients’ and customers’ time and preferences. It’s a win-win for all.

Additional resource: MailChimp for Online Sellers

Sort product attribute options by the position property in Magento

2016-01-11T00:00:00+00:00

Introduction

Recently I was working with Magento 1.9.1, trying to get a list of dropdown attribute options sorted by the position property. However there is a known bug in Magento 1.9.1, where the position property is not respected.

I looked for a patch to fix this issue, however there was no official patch, and none of the available community fixes were good enough. So again, I needed to fix it by myself.

Tip! If you know how to apply a patch file it is here. If not, please continue.

Part 1

We need to overwrite some Magento core code unfortunately. The good thing is that there is a cool way of doing this in Magento so we don’t need to overwrite the files directly, we need to create a local copy.

Copy app/code/Core/Mage/Catalog/Model/Resource/Product/Type/Configurable/Attribute/Collection.php file to app/code/local/Mage/Catalog/Model/Resource/Product/Type/Configurable/Attribute/Collection.php. You need to create the whole directory structure. If you use Unix system it is simple as: (running from Magento root)

mkdir -p app/code/local/Mage/Catalog/Model/Resource/Product/Type/Configurable/Attribute/
cp app/code/Core/Mage/Catalog/Model/Resource/Product/Type/Configurable/Attribute/Collection.php app/code/local/Mage/Catalog/Model/Resource/Product/Type/Configurable/Attribute/Collection.php

Part 2

Fill a local file with a content from this source: https://gist.github.com/peter-hank/c917394ea9f1171ddeb8

Notes

After these changes it will work as expected and attribute options will be sorted by a position set.

This fix should work for any Magento 1.9.* version.

Competing with the big players in e-commerce

2011-06-20T00:00:00+00:00

While attending the Internet Retailer Conference and Exhibition in San Diego last week I had a few moments to speak with Nathan Barling, the CTO for Shoebacca.com. During our conversation he mentioned he would be speaking at the conference in the track titled: “Small Retailers: Winning Strategies in a tougher market”. I attended his talk and was impressed by the things that Shoebacca is doing to appear larger then they are, which helps them compete with the big players in their industry such as Zappos. The tactics that Nathan discussed can be applied to many industries in e-commerce and for all sizes of businesses, even those on limited budgets.

One of the first things Nathan discussed was to make your policies clear and to highlight them on the site so that people are aware of the rules. Nathan recommends this, especially in the case of Shoebacca, where many of their policies encourage people to shop on their site, by reducing risk to the customer. Some of their policies include:

Free ground shipping
Free return shipping
365 day return policy
110% price match for 14 days

Nathan mentioned many 3rd party tools that they leverage at their company, along with open source tools they are using to keep the costs down while providing a solid platform for their site. Some of the open source tools in use include Magento (e-commerce), Solr (search), and Gearman (jobs system).

One area where Shoebacca is doing cool work is in their “shoe art”. These are clips and ads through the site which present the product in a unique and appealing way. They give the illusion that Shoebacca is a large company with an extensive art department and studio for shooting the product. They also leverage the art provided by their vendors such as ads and logos so that their customers associate the site with the quality of those vendors such as Nike and Puma. Another area where they are getting assistance from their vendors is with giveaway merchandise. This is merchandise the vendor provides them to giveaway with purchases of other items on the site. This could be a sticker, water bottle, or jacket, all promo items that Shoebacca does not have to pay for, but is of some value to the end customer and may push the customer to complete the order.

One area that Nathan saw an immediate increase in conversions is with alternate payment methods. They chose to launch Amazon payments, and before they could even put a test order through someone had already checked out. Using alternate payment methods such as Amazon, Google or PayPal allows your customers to have the peace of mind that their credit card details are safe with a trusted company, while they are getting to know your company. Nathan said that many people on subsequent visits ended up entering their credit card directly into the site, after having a successful transaction through an alternate payment method.

Shoebacca is also going to be launching a subscription program in which people pay a nominal yearly fee and receive benefits such as a coupon, access to exclusive sales, free swag, upgrade to free 2 day shipping. The subscription program is a neat idea to help get your customers to commit to using your site on an ongoing basis.

I enjoyed my time away from the exhibit hall for Nathan Barling’s talk and took away quite a few things that I can suggest to my clients who are looking to improve their sales.

Spree Performance Benchmarking

2011-05-25T00:00:00+00:00

I see a lot of questions regarding Spree performance in the spree-user group, but they are rarely answered with metrics. I put together a quick script using the generic benchmark tool ab to review some data. Obviously, the answer to how well a site performs and scales is highly dependent on the host and the consumption of the web application, so the data here needs to be taken with a grain of salt. Another thing to note is that only two of the following use cases are running on Rails 3.0 — many of our current Spree clients are on Spree 0.11.2 or older. I also included one non-Spree Rails ecommerce application, in addition to a few non-Rails applications for comparison. All of the tests were run from my home network, so in theory there shouldn’t be bias on performance tests for sites running on End Point servers.

	ab -n 100
	-c 2 homepage	-c 20 homepage	-c 2 product page	-c 20 product page
Client #1 Spree: 0.11.2 Hosting: 4 cores, 512 GB RAM DB: MySQL # Products: <100	7.49	24.75	6.49	19.87	Requests per second
266.889	808.041	307.997	1006.552	Time per request (ms)
Client #2 Spree 0.11.2 Hosting: Engineyard, medium instance DB: MySQL # Products: 100s	5.32	20.28	5.36	18.03	Requests per second
375.713	986.309	373.289	1109.524	Time per request (ms)
Client #3 Spree: 0.9.0 Hosting: 4 cores, 1 GB RAM DB: PostgreSQL # Products: <100	4.91	25.39	1.98	6.54	Requests per second
407.135	787.782	1011.875	3060.062	Time per request (ms)
(Former) Client #4 Spree: 0.11.2 Hosting: Unknown DB: PostgreSQL # Products: >5000	20.69	8.84	10.15	19.28	Requests per second
96.673	2262.105	196.996	1037.146	Time per request (ms)
Client #5 Spree: 0.11.2 Hosting: EngineYard, small instance DB: MySQL # Products: 1	12.28	16.23	N/A	N/A	Requests per second
162.909	1231.945	N/A	N/A	Time per request (ms)
Client #6 Spree: 0.40 Hosting: 4 cores, 1 GB RAM DB: MySQL # Products: 50-100	3.61	8.93	2.96	3.06	Requests per second
553.569	2240.657	675.306	6539.433	Time per request (ms)
SpreeDemo Spree: Edge Hosting: Heroku, 2 dynos DB: Unknown # Products: 100s	8.17	12.79	4.7	5.48	Requests per second
244.831	1563.642	425.27	3652.927	Time per request (ms)
Client #7 *custom Rails ecommerce app Hosting: 1.0 GB RAM DB: MySQL # Products: 1000s	5.43	29.8	4.45	23.14	Requests per second
368.409	671.082	448.962	864.24	Time per request (ms)
Interchange Demo Hosting: 4 cores, 2 GB RAM DB: MySQL # Products: >500	7.41	55.27	7.5	13.93	Requests per second
269.942	361.875	266.492	1435.51	Time per request (ms)
Client #8 *PHP site, serves fully cached pages with nginx with no app server or db hits Hosting: 4 cores, 4 GB RAM	10.81	30.54	6.05	9.87	Requests per second
184.994	654.858	330.727	2027.092	Time per request (ms)
Magento Demo Hosting: Unknown DB: Unknown # Products: 100s	4.26	44.85	2.68	36.29	Requests per second
469.831	445.931	745.472	551.11	Time per request (ms)

Here’s the same data in graphical form:

Requests per Second

Time Per Request (ms)

We expect to see high performance on some of the sites with significant performance optimization. On smaller VPS, we expect to see the the server choke with higher concurrency.

Ecommerce Solutions: What are the Options?

2011-02-28T00:00:00+00:00

Lately, I’ve been evaluating ecommerce options for use on a side hobby/business. I’m obviously a developer, so in theory I could use one of End Point’s supported ecommerce frameworks or just write my own framework. But, my bottom line is that I don’t need the feature set offered by some of the ecommerce options out there and I don’t necessarily have the resources to develop a custom solution.

In addition to personal interest, End Pointers constantly encounter potential clients who aim to get a better understanding of the cost of using open source, our preferred ecommerce solution. I put together two infographics on ecommerce options, ongoing cost, feature sets, and the ability to customize. Before anyone flips out about the infographics, note that they represent my broad generalizations regarding the ongoing cost, feature sets and ability to customize. I’m intimately familiar with some of these options and less familiar with a couple of them.

Feature Set versus Ongoing Cost of Ecommerce Solutions

Ability to Customize versus Ongoing Cost of Ecommerce Solutions

Some notes on on the ecommerce solutions shown in the infographics:

Online payment service (Paypal): An online payment collection service like PayPal offers a minimal “ecommerce” feature set and might be suitable for someone looking to simply collect money. It also provides almost no ability to customize. The ongoing cost of PayPal is lower than many of the other options, where a percentage of each sale goes to PayPal.
Online catalog service (Etsy, eBay): An online catalog service such as Etsy or eBay offers very basic ecommerce listing features, little to no ability to customize, but has relatively low ongoing cost. For example, Etsy charges $0.20 to list a single item for four months and takes 3.5% of the sales fee.
Hosted ecommerce (shopify, Big Cartel, Big Commerce, Yahoo Merchant): A hosted ecommerce solution offers the ability to customize the appearance, typically in the form of a custom template language and has a basic ecommerce feature set. Ongoing costs would include the cost of the service, domain registration cost, and payment gateway (e.g. Authorize.NET) fees. Additional cost may apply if development services are required for building a custom template. Shopify’s basic solution costs $29/mo. with a max of 100 skus. Shopify offers several other higher-priced options which include more features and have less limitations. Most other hosted ecommerce solutions are similarly priced.
Open source ecommerce (Interchange, Spree, Magento, Zen Cart, osCommerce, prestaShop): An open-source ecommerce solution tends to have generic ecommerce features, provides the opportunity for a large amount of customization, but is typically more expensive than hosted ecommerce solutions and online catalog services. The software itself is free, but ongoing costs of hosting (server, domain registration, SSL certificate), development (any piece of customization that does not fit into the generic mold), and payment gateway fees apply. One positive about open source ecommerce is that additional plugins or add-ons are produced by members of the community. If the business needs are satisfied by the community-available extensions, cost from additional development or customization may be eliminated or reduced. I’d also probably group in existing open source plugins or modules for open source CMS solutions like WordPress and Drupal into this category: the generic solution is free, but additional resources may be required for customization.
Enterprise ecommerce (ATG, Magento Enterprise): From my experience, I’ve observed that enterprise ecommerce solutions tend to offer a large feature set and a similar ability to customize as open source frameworks. The ongoing cost is high: for example, Magento Enterprise starts at $12,990/year. In addition to the licensing cost, hosting, development, and payment gateway fees may apply.
Custom ecommerce (homegrown): The cost of writing your own ecommerce framework depends on the functionality requirements. The ability to customize is unlimited since the entire solution is custom. The feature set is likely to be proportional to resources spent on the project. Ongoing costs here include hosting, development and payment gateway fees.

Which one do you choose?

It has been my experience that most of End Point’s ecommerce clients need some type of customization: a custom appearance, discount functionality, shipping integration, payment gateway integration, social media integration, and other third-party integration. Choosing an option that allows easy customization tends to benefit our customers in the long run. We’re pretty biased at End Point towards open source ecommerce solutions, but my opinion recently is that with the advancement of web frameworks and web framework tools (e.g. gems in ruby), development of custom solutions can be done efficiently and may be a better option for a site that is outside the realm of standard ecommerce. For businesses not able to pay development or consulting costs, hosted ecommerce solutions are affordable and provide the essentials needed as the business grows.

Utah Open Source Conference 2010 part 1

2010-11-12T00:00:00+00:00

It’s been about a little over a month since the 2010 Utah Open Source Conference, and I decided to take a few minutes to review talks I enjoyed and link to my own talk slides.

Magento: Mac Newbold of Code Greene spoke on the Magento ecommerce framework for PHP. I’ve somewhat familiar with Magento, but a few things stood out:

He finds the Magento Enterprise edition kind of problematic because Varien won’t support you if you have any unsupported extensions. Some of his customers had problems with Varien support and went back to the community edition.
Magento is now up to around 30 MB of PHP files!
As I’ve heard elsewhere, serious customization has a steep learning curve.
The Magento data model is an EAV (Entity-Attribute-Value) model. To get 50 columns of output requires 50+ joins between 8 tables (one EAV table for each value datatype).
There are 120 tables total in default install—many core features don’t use the EAV tables for performance reasons.
Another observation I’ve heard in pretty much every conversation about Magento: It is very resource intensive. Shared hosting is not recommended. Virtual servers should have a minimum of 1/2 to 1 GB RAM. Fast disk & database help most. APC cache recommended with at least 128 MB.
A lot of front-end things are highly adjustable from simple back-end admin options.
Saved credit cards are stored in the database, and the key is on the server. I didn’t get a chance to ask for more details about this. I hope it’s only the public part of a public/secret keypair!

It was a good overview for someone wanting to go beyond marketing feature lists.

Node.js: Shane Hansen of Backcountry.com spoke on Node, comparing it to Tornado and Twisted in Python. He calls JavaScript “Lisp in C’s clothing”, and says its culture of asynchronous, callback-driven code patterns makes Node a natural fit.

Performance and parallel processing are clearly strong incentives to look into Node. The echo server does 20K requests/sec. There are 2000+ Node projects on GitHub and 500+ packages in npm (Node Package Manager), including database drivers, web frameworks, parsers, testing frameworks, payment gateway integrations, and web analytics.

A few packages worth looking into further:

express — web microframework like Sinatra
Socket-IO — Web Sockets now; falls back to other things if no Web Sockets available
hummingbird — web analytics, used by Gilt.com
bespin — “cloud JavaScript editor”
yui3 — build HTML via DOM, eventbus, etc.
connect — like Ruby’s Rack

I haven’t played with Node at all yet, and this got me much more interested.

Metasploit: Jason Wood spoke on Metasploit, a penetration testing (or just plain penetrating!) tool. It was originally in Perl, and now is in Ruby. It comes with 590 exploits and has a text-based interactive control console.

Metasploit uses several external apps: nmap, Maltego (proprietary reconnaissance tool), Nessus (no longer open source, but GPL version and OpenVAS fork still available), Nexpose, Ratproxy, Karma.

The reconnaissance modules include DNS enumeration, and an email address collector that uses the big search engines.

It can backdoor PuTTY, PDFs, audio, and more.

This is clearly something you’ve got to experiment with to appreciate. Jason posted his Metasploit talk slides which have more detail.

So Many Choices: Web App Deployment with Perl, Python, and Ruby: This was my talk, and it was a lot of fun to prepare for, as I got to take time to see some new happenings I’d missed in these three languages communities’ web server and framework space over the past several years.

The slides give pointers to a lot of interesting projects and topics to check out.

My summary was this. We have an embarrassment of riches in the open source web application world. Perl, Python, and Ruby all have very nice modern frameworks for developing web applications. They also have several equivalent solid options for deploying web applications. If you haven’t tried the following, check them out:

Perl: Dancer, Starman
Python: Flask
Ruby: Sinatra, Unicorn

That’s about half of my notes on talks, but all I have time for now. I’ll cover more in a later post.

Spree vs. Magento: Feature List Revisited

2010-07-13T00:00:00+00:00

A little over a month ago, I wrote an article on Spree vs. Magento Features. Recently, a client asked me to describe the features mentioned in that article. I thought this was another great opportunity to expand on my response to the client. So, here I am, revisiting ecommerce features in Spree and Magento. The original article can be referenced to compare availability of these features in Spree and Magento.

Features on a Single Product or Group of Product

Product reviews and/or ratings: functionality to allow customers to review and rate products. See a Backcountry.com product page for an example.
Product QnA: functionality allow customers to ask and answer questions on products. See a Backcountry.com product page for an example.
Product SEO (URL, title, meta data control): functionality to allow site administrators to manage product URLs, product page titles, and product meta data.
Advanced/flexible taxonomy: functionality to build a custom taxonomy / navigation structure for product browsing. For example, build multiple categories and subcategories with complex hierarchy. The taxonomy at Spree’s demo includes two categories of brand and category and subcategories in each.
SEO for taxonomy pages: functionality to allow site administrators to manage taxonomy URLs, taxonomy page titles, and taxonomy meta data.
Configurable product search: functionality to allow the developers and site administrators to adjust parameters used in search, such as products to show per page, and to show products with no on hand stock.
Bundled products for discount: functionality to allow site administrators to create bundles or group products together and then apply a discount to the entire bundle. For example, product X, Y and Z purchased together will yield a $10.00 discount.
Recently viewed products: functionality to show customers products they recently visited. This can be displayed on other product pages or the navigation pages to aid in navigation back to those products if the user would like to revisit the products. See the image below for an example of this functionality in action at Paper Source.

Soft product support/downloads: functionality to sell soft products such as mp3 or pdf files.
Product comparison: functionality to allow customers to compare multiple products, such as a comparison of price or technical features. See Backcountry.com for an example.
Upsell: functionality to encourage the customer to purchase a similar product from a higher price point, or to purchase an add on, may or may not include the functionality to allow site administrators to manage the upsell products.
Cross sell: functionality to encourage the customer to purchase related items, may or may not include the functionality to allow site administrators to manage the cross sell products. See the image below for an example of this functionality at Backcountry.com after adding an item to the cart.

Related items: functionality to display related items on product pages, may or may not include the functionality to allow site administrators to manage the related products.
RSS feed of products: functionality to produce a RSS feed with product detail releases. steepandcheap.com offers a RSS feed, however, they are unique in that they offer a one deal at a time business model, so the RSS feed contains a stream of products for sale. This may or may not include the functionality to allow site administrators to manage the RSS feed contents.
Multiple images per product: functionality to allow site administrators to upload multiple products per image. See a Spree demo product for an example.
Product option selection (variants): functionality to allow site administrators to create and manage variants for products to offer multiple variants per product, such as variants by size and color. See a Spree demo product for an example.
Wish list: functionality to allow customers to create product wish lists. See Amazon.com for a description of their wish list functionality.
Send product email to friend: functionality to allow customers to send emails to their friend to visit a specific product.
Product tagging / search by tagging: functionality to allow site administrators to assign tags to products for navigation or searching. See CityPass’s blog for an example of tag use in a content management system; in ecommerce context, the tags would navigate to a set of products instead of a set of blog articles.
Breadcrumbs: functionality that renders the product navigation hierarchy on navigation and product pages to allow customers to navigate to previous pages visited. See a Spree demo navigation page for an example.

CMS Features

Blogging functionality: functionality to allow site administrators to create, manage and display blog articles.
Static page management: functionality to allow site administrators to create, manage, and display static pages such as “About Us”, “Information”.
Media management: functionality to allow site administrators to create, manage, and display media such as images, video, audio. See the image below for an example of WordPress’s dashboard for content management.

Contact us form: functionality to allow customers to submit a request for contact. See End Point’s contact page for an example.
Polls: functionality to allow site administrators to create and manage basic polls, functionality to allow customers to submit answers to basic polls.

Checkout Support

One page checkout: functionality to allow customers to complete checkout on one page, rather than move forward through checkout through multiple address, payment pages. See Paper Source or Backcountry.com’s checkout processes for examples.
Guest checkout: functionality to checkout without creating a user account. Checkout at the Spree demo without being logged in for an example.
SSL support: functionality to configure use of SSL during checkout. In Spree’s case, the site administrator may turn SSL off during development and on during production.
Discounts: functionality to allow customers to apply discount coupons to orders for a percentage or dollar amount reduction.
Gift certificates: functionality to allow customers to purchase and use gift certificates as credit for purchases.
Saved shopping cart: the functionality to save the products in a customers shopping cart so their shopping cart will be pre-populated on their next visit.
Saved addresses: functionality to allow customers to create and manage addresses to be selected during checkout for billing or shipping rather than requiring the customer to re-enter their address. See the image below for an example of using saved addresses during Backcountry.com’s checkout.

Shipping Support

Real time rate lookup (UPS, USPS, FedEx): the functionality to request rates from UPS, USPS, or FedEx during checkout for more accurate rate pricing rather than using a flat shipping rate.
Order tracking: functionality to allow the site administrators to enter order tracking information and allow the user to review that tracking information, may or may not include sharing this information in a “Your order has been shipped” email.
Multiple shipments per order: functionality to allow site administrators to split orders into multiple packages if specific products can not be shipped at the same time or can not be shipped together. See the image below for an example of Spree’s backend shipping interface.

Complex rate lookup: functionality to calculate ship rates based on weight or price.
Free shipping: functionality to offer free shipping.

Payment Support

Multiple payment gateways: integration of multiple payment gateways such as Authorize.NET, Beanstream, Paypal, SagePay, etc.
Authorize.Net: integration of the Authorize.Net payment gateway; may or may include the use of profiles (Authorize.Net CIM).
Authorize and capture versus authorize only: functionality to allow site administrators to configure whether or not credit cards should be authorized only during checkout completion or authorized and captured. If the credit card is authorized only, site administrators may finalize an order by capturing on the backend interface.
Google Checkout: integration of Google Checkout.
Paypal Express: integration of Paypal Express.

Admin Features

Sales reporting: the functionality to display sales statistics, such as profits on sales or year-over-year sales.
Sales management tools: functionality to allow site administrators to create and manage product sales. For example, the site administrator might create a 50% off sale to cover 25% of the products to begin in a week and end in two weeks.
Inventory management: functionality to allow the site administrator to manage individual inventory units and their current state (on_hand, shipped, backordered) and order assignment.
Purchase order management: functionality to allow the site administrator to create and manage purchase orders. See the image below for an example of a potential backend interface for purchase orders in Spree.

Multi-tier pricing for quantity discounts: functionality to allow customers to buy large quantities of products at a discount, the functionality to allow the site administrator to manage the large quantity product discounts.
Landing page tool: functionality to create custom landing pages that may include targeted content or products, typically used for advertising or marketing.
Batch import and export of products: functionality to allow the site administrator to import and export products via admin interface or script rather than entering each product individually.
Multiple sales reports: See “Sales reporting” above.
Order fulfillment: functionality to allow site administrators to manage fulfillment (inventory selection, shipping) of orders.
Tax Rate Management: functionality to manage tax rates per zone, where zones are defined by states and/or countries. Note that in Spree, zones can only be defined by a combination of states and countries and tax rates can be tied to one or more zones.

User Account Features

User addresses: See “Saved addresses” above.
Feature rich user preferences: integration of various user account tools, such as address management, profile management, order review, etc.
Order tracking history: functionality to allow a customer to lookup their order history, may or may not include order tracking information.

Site Wide Features

Extensibility: functionality to extend the ecommerce core with modular components.
Appearance Theming: functionality to change the appearance of the site.
Ability to customize appearance at category or browsing level: functionality to create and manage custom and varied appearances for product browsing pages. For example, the categories “Jackets” and “Pants” may have different appearances, motivated by marketing or advertising.
Localization: the functionality to translate the ecommerce site to a different language. See the image below for a small example of localization in action in Spree.

Multi-store, single admin support: functionality to manage multiple stores from a single administrative location. An example of this might include http://store1.endpoint.com/, and http://store2.endpoint.com/, where both can be managed at http://admin.endpoint.com/.
Support for multiple currencies: functionality to translate product prices between currencies.
Web service API: functionality to retrieve data from the ecommerce application for third party use. See Spree’s documentation on the Spree API.
System wide SEO: general site-wide SEO functionality including features such as sitemap, googlebase integration, URL management, page title management.
Google Analytics: functionality to allow site administrators to create and manage Google Analytics Ids, functionality to track traffic and conversion on the frontend.
Active community: an active developer community with frequent core and extension contributions.

Most of the features described above are well known to ecommerce developers, but this list might also serve as a good checklist to review with a potential client during the estimate process to make sure expectations of an ecommerce platform are managed, especially with a young platform such as Spree where some features are not yet included in the core.

Spree vs Magento: A Feature List Comparison

2010-06-07T00:00:00+00:00

Note: This article was written in June of 2010. Since then, there have been several updates to Spree. Check out the current Official Spree Extensions or review a list of all the Spree Extensions.

This week, a client asked me for a list of Spree features both in the core and in available extensions. I decided that this might be a good time to look through Spree and provide a comprehensive look at features included in Spree core and extensions and use Magento as a basis for comparison. I’ve divided these features into meaningful broader groups that will hopefully ease the pain of comprehending an extremely long list :) Note that the Magento feature list is based on their documentation. Also note that the Spree features listed here are based on recent 0.10.* releases of Spree.

Features on a Single Product or Group of Product

Feature	Spree	Magento
Product reviews and/or ratings	Y, extension	Y
Product qna	N	N
Product seo (url, title, meta data control)	N	Y
Advanced/flexible taxonomy	Y, core	Y
Seo for taxonomy pages	N	Y
Configurable product search	Y, core	Y
Bundled products for discount	Y, extension	Y
Recently viewed products	Y, extension	Y
Soft product support/downloads	Y, extension	Y, I think so
Product comparison	Y, extension	Y
Upsell	N	Y
Cross sell	N	Y
Related items	Y, extension	Y
RSS feed of products	N	Y
Multiple images per product	Y, core	Y
Product option selection (variants)	Y, core	Y
Wishlist	Y, extension	Y
Send product email to friend	Y, extension	Y
Product tagging / search by tagging	N	Y
Breadcrumbs	Y, core	Y

CMS Features

Features	Spree	Magento
Blogging functionality	Y, extension	Y *extension
Static page management	Y, extension	Y
Media management	N	Y
Contact us form	Y, extension	Y
Polls	Y, extension	Y

Checkout Support

Feature	Spree	Magento
One page checkout	N	Y
Guest checkout	Y, core	Y
SSL Support	Y, core	Y
Discounts	Y, core	Y
Gift Certificates	N	Y
Saved Shopping Cart	N	Y
Saved Addresses	Y, extension	Y

Shipping Support

Feature	Spree	Magento
Real time rate lookup (UPS, USPS, Fedex)	Y, extension	Y
Order tracking	N	Y
Multiple shipments per order	Y, core	Y
Complex rate lookup	Y, extension	Y
Free shipping	Y, extension	Y

Payment Support

Feature	Spree	Magento
Multiple Payment Gateways	Y, core	Y
Authorize.net	Y, core	Y
Authorize and capture versus authorize only	Y, core	Y
Google Checkout	Y, extension	Y
Paypal Express	Y, extension	Y

Admin Features

Feature	Spree	Magento
Sales reporting	Y, core	Y
Sales Management Tools	N	Y
Inventory management	Y, core	Y
Purchase order management	N	Y
Multi-tier pricing for quantity discounts	N	Y
Landing page tool	Y, extension	Y
Batch import and export of products	Y, extension	Y
Multiple Sales reports	Y, core	Y
Order fulfillment	Y, core	Y
Tax Rate Management	Y, core	Y

User Account Features

Feature	Spree	Magento
User addresses	Y, extension	Y
Feature rich user preferences	N	Y
Order tracking history	Y, core	Y

System Wide Features

Feature	Spree	Magento
Extensibility	Y, core	Y
Appearance Theming	Y, core	Y
Ability to customize appearance at category or browsing level	N	Y
Localization	Y, core	Y
Multi-store, single admin support	Y, extension	Y
Support for multiple currencies	N	Y
Web Service API	Y, core	Y
SEO System wide: sitemap, google base, etc	Y, extension	Y
Google Analytics	Y, core	Y
Active community	Y, N/A	Y

The configurability and complexity of each feature listed above varies. Just because a feature is provided within a platform does not guarantee that it will meet the desired business needs. Magento serves as a more comprehensive ecommerce platform out of the box, but the disadvantage may be that adding custom functionality may require more resources (read: more expensive). Spree serves as a simpler base that may encourage quicker (read: cheaper) customization development simply because it’s in Rails and because the dynamic nature of Ruby allows for elegant extensibility in Spree, but a disadvantage to Spree could be that a site with a large amount of customization may not be able to take advantage of community-available extensions because they may not all play nice together.

Rather than focus on the platform features, the success of the development depends on the developer and his/her skillset. Most developers will say that any of the features listed above are doable in Magento, Spree, or Interchange (a Perl-based ecommerce platform that End Point supports) with an unlimited budget, but a developer needs to have an understanding of the platform to design a solution that is easily understood and well organized (to encourage readability and understandability by other developers), develop with standard principles like DRY and MVC-style separation of concerns, and elegantly abstract from the ecommerce core to encourage maintainability. And of course, be able to understand the business needs and priorities to guide a project to success within the given budget. Inevitably, another developer will come along and need to understand the code and inevitably, the business will often use an ecommerce platform longer than planned so maintainability is important.

Please feel free to comment on any errors in the feature list. I’ll be happy to correct any mistakes. Now, off to rest before RailsConf!

Spree at RailsConf

2009-05-11T00:00:00+00:00

Last week at RailsConf 2009, the Spree folks from End Point conducted a Birds of a Feather session to discuss Spree, an End Point sponsored open source rails ecommerce platform. Below is some of the dialog from the discussion (paraphrased).

Crowd: “How difficult is it to get Spree up and running from start to finish?”

Spree Crew: “This depends on the level of customization. If a customer simple needs to reskin the site, this shouldn’t take more than a week (hopefully much less than a full week). If the customer needs specific functionality that is not included in core functionality or extensions, you may need to spend some time developing an extension.”

Crowd: “How difficult is it to develop extensions in Spree?”

Spree Crew: “Spree extension work is based on the work of the Radiant community. Extensions are mini-applications: they allow you to drop a pre-built application into spree to override or insert new functionality. Documentation for extensions is available at the spree github wiki. We also plan to release more extensive Spree Guides documentation based on Rails Guides soon.”

Spree Crew: “How did you hear about Spree?”

Crowd: “My client and I found it via search engines. My client thought that Spree looked like a good choice.”

Spree Crew: “What other platforms did you consider before you found spree?”

Crowd: “Magento”, “Substruct”, “My client considered Magento, but I know several people that have developed with Magento and have found it difficult to override core functionality.”

Spree Crew: “What types of functionality were missing from Spree that you’d like to see developed in the future?”

Crowd: “My client wanted checkout split into multiple steps instead of the new single page checkout. I was able to implement this by overriding the Spree checkout library and checkout views.”, “My client needed complex inventory management.”, “My client needed split shipping functionality.”

Crowd: “What is the plan for Spree with regards to CMS development?”

Spree Crew: “There has been some discussion on the integration of a CMS into Spree. No one in the Spree community appears to be currently working on this. Contributions in this area are welcome. Also, Yehuda Katz is giving a talk on mountable apps—the Spree community would like to investigate the implications this has for Spree.”

Crowd: “What are the next steps for localization, especially multilingual product descriptions?”

Spree Crew: “This is on the radar for future Spree development. It is not currently in development, and again, contributions in this area are welcome.”

From the discussion, I took away that some of the desired features for Spree are inventory management, split shipping functionality, cms integration, and improved localization. I hope that the application of Spree continues to contribute to it’s progress. The Spree Crew also hopes to showcase some of the sites referenced above at the spree site.

SEO Ecommerce

2009-04-20T00:00:00+00:00

I recently read an article that discusses Magento SEO problems and solutions. This got me to think about common search engine optimization issues that I’ve seen in e-commerce. Below are some highlighted e-commerce search engine optimization issues. The Spree Demo, Interchange Demo, and Magento Demo are used as references.

Duplicate Home Pages (www, non-www, index.html)

Duplicate home pages can come in the form of a homepage with www and without www, a homepage in the form of http://www.domain.com/ and a homepage with some variation of “index” appended to the url, or a combination of the two. In the Interchange demo, http://demo.icdevgroup.org/i/demo1 and http://demo.icdevgroup.org/i/demo1/index.html are duplicate, http://demo.spreecommerce.com/ and http://demo.spreecommerce.com/products/ in the Spree demo, and finally http://demo.magentocommerce.com/ and http://demo.magentocommerce.com/index.php in the Magento demo.

External links positively influence search engine performance more if they are pointing to one index page rather than being divided between two or three home pages. Since the homepage most likely receives the most external links, this issue can be more problematic than other generated duplicate content. I’ve also seen this happen in several content management systems.

This article provides directions on mod_rewrite use to apply a 301 redirect from the www.domain.com/index.php homepage to www.domain.com. This solution or other redirect solutions can be applied to Spree, Interchange, and other ecommerce platforms.

Irrelevant Product URLs

A search engine optimization best practice is to provide relevant and indicative text in the product urls. In the Interchange demo, the default catalog uses the product sku in the product url (http://demo.icdevgroup.org/i/demo1/os28073.html). In Magento and Spree, product permalinks with relevant text are used in the product url. In wordpress, the author has the ability to set permalinks for articles. I am unsure if Magento gives you the ability to customize product urls. Spree does not currently give you the ability to manage custom product permalinks. However, for all of these ecommerce platforms, these fixes may all be in the works since it is important for ecommerce platforms to implement search engine optimization best practices.

Duplicate product content

I’ve observed several situations where products divided into multiple taxonomies results in duplicate content creation via different user navigation paths. For example, in the Spree demo, the “Ruby Baseball Jersey” can be reached through the Ruby brand page, the Clothing page, or the homepage. The three generated duplicate content urls are http://demo.spreecommerce.com/products/ruby-on-rails-ringer-t-shirt, http://demo.spreecommerce.com/t/brands/ruby/p/ruby-baseball-jersey, and http://demo.spreecommerce.com/t/categories/clothing/shirts/p/ruby-baseball-jersey.

Another example of this can be found in the Interchange demo. The left navigation taxonomy tree provides links to any product url with “?open=X,Y,Z” appended to the url. The “open” query string indicates how the DHTML tree should be displayed. For example, the “Digger Hand Trencher” has a base url of http://demo.icdevgroup.org/i/demo1/os28076.html. Depending on which tree nodes are exploded, the product can be reached at http://demo.icdevgroup.org/i/demo1/os28076.html?open=0,11,13,19, http://demo.icdevgroup.org/i/demo1/os28076.html?open=0,11,13, etc. This standard demo functionality yields a lot of duplicate content.

In Magento, products are the in the form of www.domain.com/product-name, although the article I mentioned above mentions that www.domain.com/category/product.html product urls were generated. Perhaps this was a recent fix, or perhaps the demo is configured to avoid generating this type of duplicate content.

Duplicate product page content is often used to indicate which breadcrumb should display or to track user click-through behavior (for example, did a user click on a “featured product”? a “best seller”? a specific “product advertisement”?). In Interchange, session ids are appended to urls which is another source of duplicate content. Instead of using the url to track user navigation or behavior, several other solutions such as using cookies, using a ‘#’ (hash), or using session data can be used to avoid duplicate content generation.

Performance

Performance should not be overlooked in ecommerce for search engine optimization. In March of 2008, Google wrote about how landing page load time will be incorporated into the Quality Score for Google Adwords—which is also believed to apply to regular search results. And github recently released some data on how performance improvements influenced http://www.github.com/ Googlebot visits.

Keeping a high content to text ratio, consolidation, minification, and gzipping CSS and JavaScript, and minimizing the use of JavaScript-based suckerfish can all improve search engine performance.

The Interchange default catalog has a simple template with minimal css and javascript includes, so the developer is responsible for sticking to best performance practices. The Magento demo appears to have decent content to text ratio, but still requires 5 css files that should be consolidated and minified if they are included on every page. Finally, Spree has undergone some changes in the last month and is moving in the direction of including one consolidated javascript file plus any javascript required for extensions on every page, and the upcoming release of Spree 0.8.0 will have considerable frontend view improvements.

Ecommerce platforms should have decent performance—yslow or this book on high performance website essentials are good resources.

Lacking basic CMS management

Basic CMS management such as the ability to manage and update page titles and page meta data is something that has been overlooked by ecommerce platforms in the past, but appears to have been given more attention recently. An ecommerce solution should also have functionality to create and manage static pages.

The Interchange demo does not have meta description and keyword functionality, however, page titles are equal to product names which is an acceptable default. It’s also very simple to add a static content page (as a developer) and would require just a bit more effort to have this content managed by a database in Interchange. The Spree core is missing some basic CMS management such as page title and meta data management, but this functionality is currently in development. One Spree contributer developed a Spree extension that provides management of simple static pages using a WYSIWYG editor. At the moment, Magento appears to have the most traditional content management system functionality out of the box.

Another area to improve CMS within Ecommerce is to determine a solution to integrate a blog. A quick Google search of “magento add blog” revealed how to set up a wordpress blog in Magento with an extension. One of End Point’s clients, CCI Beauty, also has wordpress integrated into their Interchange setup. Finally, there has been discussion about the development of “Spradiant”, or mixing spree and radiant.

Another missed opportunity in ecommerce platforms is finding a solution to elegantly blend content and product listings to target specific keywords. A “landing page” can have a page title, meta data, and content targeted towards a specific terms. http://www.backcountry.com/store/gear/arcteryx-vests.html and http://www.backcountry.com/store/gear/cargo-pant.html are examples of targeted terms with corresponding products. Going one step farther, search pages themselves can have managed content to attract keywords, such as a page title, and meta data for specific high traffic keywords with the related products. For example, http://www.domain.com/s/ruby_shirt could be a search page for “Ruby Shirt” which contains meaningful content and relevant products.

Mishandled Product Pagination

Finding a search engine optimization solution for pagination can be a difficult problem in ecommerce. When there are less than 100 products for a site, this shouldn’t be an issue because a simple taxonomy can appropriately group the products with low crawl depth. A website with 10,000 products must balance between keeping a low taxonomy depth to minimize crawl depth and ensure that all products are listed and indexable.

For example, products may be divided and fit into three levels of navigation: category, subcategory, and group. If there are 10,000 products, divided into 10 categories, 10 subcategories per category, and 10 product groups per category, 10 products can be shown on each group per page with no pagination. However, product taxonomy is not always so ideal. In some groups there may be 2 products and in others there may be 30. Pagination, or pages with an offset of product listings are generated to accommodate these product listings (for example, http://www.backcountry.com/store/group/61/Sun-Hats-Rain-Hats-Safari-Hats.html, http://www.backcountry.com/store/group/61/Sun-Hats-Rain-Hats-Safari-Hats-p1.html).

A few problems can arise from the pagination solution. First, by web 2.0 standards, the content should be generated via ajax. An SEO friendly ajax solution must be implemented—where the onclick event refreshes the content, but the links are still crawlable via search engine bots. Second, page 1 with no product offset will have 1 level less of crawl depth, therefore it will receive the most link juice from it’s parent page (subcategory). As a result, there must be thoughtful analysis of which products to present on that page: should high traffic pages get the traffic? should popular items be listed on the first page? should low traffic products be listed to try to bump the traffic on those pages? should products with the most “user interaction” (reviews, qna, ratings) be shown on that page? Another problem that comes up is that the page meta data and title will most likely be very similar since the content is a list of similar products. These two pages can essentially be competing for traffic and may be counted as duplicate content if the page titles and meta data are equal.

Interchange uses the more list to handle pagination, but this functionality is not search engine friendly as it generates urls such as http://demo.icdevgroup.org/i/demo1/scan/MM=3ffffa066192cba677e1428d7461ddc9:10:19:10.html?mv_more_ip=1&mv_nextpage=results&mv_arg=, http://demo.icdevgroup.org/i/demo1/scan/MM=3ffffa066192cba677e1428d7461ddc9:20:27:10.html?mv_more_ip=1&mv_nextpage=results&mv_arg=, etc. The Spree demo had some pagination implementation, but upon recent frontend changes, it is no longer included in the demo. The Magento demo was carefully arranged so that product group pages have no more than 9 products to avoid showing any pagination functionality. However, when modifying the number of products displayed per group or using the “Sort By” mechanism, ?limit=Y and &order=X&dir=asc is appended to the url—which can produce a large volume of duplicate content (try filters on this page).

It is difficult to determine which of the above problems is the most problematic. From personal experience, I have been involved in tackling all duplicate content issues, and then moving on to “optimization” opportunities such as enhancing the content management system. At the very least, developers and users of any ecommerce platform should be aware of common search engine optimization issues.

Containerizing Magento with Docker Compose: Elasticsearch, MySQL and Magento

Magento 2.4 application components

Creating a shared network

Containerizing MySQL

Connecting to this container

Connecting from within the container

Connecting directly from the host machine

Containerizing Elasticsearch

Containerizing Magento

The Dockerfile

The image

The container

Connecting to the container

Talking to other containers in the network

Installing Magento in our container

Optional: Installing the sample data

Composing it all together

Bonus: Interactive debugging with Visual Studio Code

Summary

Our Vue Storefront “Proof of Concept” Experience

PoC of Vue Storefront/​Magento 2.3

Preparing the Environment

Indexing Elasticsearch with mage2vuestorefront

Viewing the Vue Storefront Application

Designing a New Approach for Interchange

Inspired By A Similar Approach of Vue Storefront + SAP Commerce

Interchange API Prototype Design

Common and Exceptional Issues

Conclusion

Magento 2: Creating a custom theme

Creating a new Magento 2 theme

Creating a new directory for the theme

Creating registration.php

Creating theme.xml

Activating our new theme

Adding custom content

Creating a custom view file

Adding a custom logo

Adding static files (JavaScript/​CSS/​images/​fonts)

Adding a custom layout

Magento 2: Creating a custom module

Creating a basic Magento 2 module

Creating a new directory for the module

Creating the registration.php script

Creating the etc/module.xml information file

Installing the new module

Adding a custom route to our module

Creating a new directory for the controller

Creating the etc/routes.xml file

Creating the controller

Upgrading the new module

Ecommerce sales tax primer

Talk to your Tax Accountant

Research Software

Why use an automated tax solution?

What to look for in automated tax software:

Implementation

Other resources

Running Magento 2 in Windows with XAMPP

Environment setup

MySQL and Magento setup

Fixing known issues

Setting up the store: A roadmap

Conclusion

Ecommerce Shakeups: Magento Acquisition and Etsy Rate Increases

Adobe to Acquire Magento

Etsy Hikes Rates

Conclusion

Reach customers and drive sales with MailChimp

Endless possibilities

Automatic marketing

Steps to implement

Go beyond the simple newsletter

Sort product attribute options by the position property in Magento

Introduction

Part 1

Part 2

Notes

Competing with the big players in e-commerce

Spree Performance Benchmarking

PoC of Vue Storefront/Magento 2.3

Adding static files (JavaScript/CSS/images/fonts)