Using esbuild and pnpm to Set Up Frontend Asset Bundling in an ASP.NET Core Web App

2025-08-25T00:00:00+00:00

Recently we had to build an admin dashboard on ASP.NET Core. We decided to hit the ground running by looking for a template online. Luckily, we found a very cool one from Start Bootstrap that met our requirements. It was built using Bootstrap, had nice styling, and included examples of graphs, tables, and basic common pages like login and registration.

However, integrating the template into an ASP.NET Core web app was a bit involved. The template uses tools like Pug and Sass/SCSS, which are not supported out of the box by ASP.NET. So some work had to be done to properly integrate it.

We ended up with a nice approach for handling frontend asset bundling using pnpm and esbuild, with support for JavaScript and Sass.

You can find a codebase that uses the setup we’ll be discussing here on Github. It’s an empty ASP.NET Core Razor Pages project that implements Start Bootrap’s admin template. Feel free to review it alongside this article and/or use it for your own projects.

We’ve also uploaded to NuGet a template for an empty ASP.NET Core Razor Pages web app that has implemented the esbuild-based frontend asset bundling.

Installing pnpm and the necessary packages

First of all we need to have the .NET framework installed. We also need to install Node.js and the pnpm package manager.

$ dotnet --version
9.0.302
$ node --version
v22.18.0
$ pnpm --version
10.14.0

Once we have those, we need a Razor Pages project to apply the changes to. For our purposes, I’m going to assume we’re starting off with a fresh project, created using something like dotnet new webapp.

With that out of the way, we can create a package.json file in the root of our project:

$ pnpm init
Wrote to /path/to/code/package.json

{
  "name": "demo",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "packageManager": "pnpm@10.14.0"
}

We can adjust the resulting package.json to better reflect our project. Maybe remove the main and scripts.test settings, which we don’t need. Maybe also set a name and description.

In any case, now that we have this, we can install the Node.js packages that we need. I mentioned that we were going to use esbuild and SCSS so let’s install those as dev dependencies:

$ pnpm add esbuild esbuild-sass-plugin sass@1.78.0 --save-dev
Packages: +41
+++++++++++++++++++++++++++++++++++++++++
Progress: resolved 103, reused 50, downloaded 0, added 41, done

devDependencies:
+ esbuild 0.25.9
+ esbuild-sass-plugin 3.3.1
+ sass 1.78.0 (1.90.0 is available)

╭ Warning ───────────────────────────────────────────────────────────────────────────────────╮
│                                                                                            │
│   Ignored build scripts: esbuild.                                                          │
│   Run "pnpm approve-builds" to pick which dependencies should be allowed to run scripts.   │
│                                                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────╯

Done in 2.5s using pnpm v10.14.0

I’ve pinned the version of the sass package to 1.78.0 in order to work around some compatibility warnings with Bootstrap’s stylesheets. You may or may not need to do this in your own projects.

We have to address the warning message and allow esbuild to run scripts. Running pnpm approve-builds and selecting esbuild (by pressing either the “Space” or “a” keys) takes care of that:

$ pnpm approve-builds
✔ Choose which packages to build (Press <space> to select, <a> to toggle all, <i> to invert selection) · esbuild
✔ The next packages will now be built: esbuild.
Do you approve? (y/N) · true
node_modules/.pnpm/esbuild@0.25.9/node_modules/esbuild: Running postinstall script, done in 40ms

With that, the package.json file should be looking like this:

// ./package.json

{
  "name": "demo",
  "version": "1.0.0",
  "description": "",
  "scripts": {},
  "keywords": [],
  "author": "",
  "license": "ISC",
  "packageManager": "pnpm@10.14.0",
  "devDependencies": {
    "esbuild": "^0.25.9",
    "esbuild-sass-plugin": "^3.3.1",
    "sass": "1.78.0"
  }
}

There should also be a new pnpm-workspace.yaml file that contains the setting for the approval we gave for esbuild:

# ./pnpm-workspace.yaml

onlyBuiltDependencies:
  - esbuild

In this particular example, we’re adapting Start Bootrap’s admin template, so we’re going to install the packages that it needs:

$ pnpm add @fortawesome/fontawesome-free bootstrap chart.js jquery simple-datatables
Packages: +9
+++++++++
Progress: resolved 112, reused 59, downloaded 0, added 9, done

dependencies:
+ @fortawesome/fontawesome-free 7.0.0
+ bootstrap 5.3.7
+ chart.js 4.5.0
+ jquery 3.7.1
+ simple-datatables 10.0.0

Done in 2.9s using pnpm v10.14.0

These are the ones we need for this project but of course, in your own projects you can install whatever you want.

Authoring JavaScript and SCSS source files

Now that we have the Node.js part of the project set up, we need a place where we can put our JavaScript and SCSS files. For that, we created two new directories: JavaScript and Stylesheets.

Since we’re adapting Start Bootrap’s admin template, we copied all its SCSS files into the Stylesheets directory and put its JavaScript into the JavaScript directory. It all ended up looking like this:

 .
+├── JavaScript
+│   └── site.js
 ├── Pages
+├── Stylesheets
+│   ├── layout
+│   ├── navigation
+│   ├── plugins
+│   ├── variables
+│   ├── _global.scss
+│   ├── site.scss
+│   └── _variables.scss
 ├── wwwroot
 │   ├── css
 │   ├── js
 │   └── favicon.ico
 ├── appsettings.Development.json
 ├── appsettings.json
+├── package.json
+├── pnpm-lock.yaml
+├── pnpm-workspace.yaml
 ├── Program.cs
 ├── RazorStartBootstrapAdmin.csproj
 └── README.md

Of course, the specific file contents will be different in your own projects, but the take home message is that these two new directories are meant for the source files of your frontend assets.

Bundling frontend assets with esbuild

With that, we have the NodeJS package management ready, the tools we need, and a project directory structure that supports writing JavaScript and SCSS.

Now we need to put together an esbuild.config.mjs file that processes these files and produces the bundles. Here it is:

// ./esbuild.config.mjs

import * as esbuild from 'esbuild';
import { sassPlugin } from 'esbuild-sass-plugin';

// SCSS
// This is a list of all the SCSS files that we want to bundle and their
// respective output files. In this project's case, we have only one.
// We put the resulting file under ./wwwroot/css. "wwwroot" is the default
// location used to expose files to the public in ASP.NET Core Razor Pages
// projects.
const scssFiles = [
  { entry: './Stylesheets/site.scss', outfile: './wwwroot/css/site.css' },
];

// Here we tell esbuild to take the entry SCSS files as input and produce CSS
// files ready to be interpreted by a browser. Since the source files are SCSS,
// we use the esbuild-sass-plugin plugin to translate them into CSS.
scssFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    // Some libraries, like Bootstrap, include icons as woff and woff2 font
    // files. To handle this, we specify this loader setting. Instructing
    // esbuild to use the "file" loader, when it encounters such files. The
    // "file" loader takes care of copying the files into the output
    // location.
    // More info: https://esbuild.github.io/content-types/#file
    loader: { '.woff': 'file', '.woff2': 'file' },
    outfile: file.outfile,
    plugins: [sassPlugin()]
  });
});

// JavaScript
// Similar to the stylesheet files, for JavaScript we also declare a list of the
// files that we want to bundle.
const jsFiles = [
  { entry: './JavaScript/site.js', outfile: './wwwroot/js/site.js' },
];

// Bundling here is simpler than SCSS because we don't need custom loaders or
// plugins. Just specify entry points and output files and esbuild knows what to
// do.
jsFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    outfile: file.outfile,
  })
});

One important thing to note here is that with esbuild, we can safely use JavaScript modules and import statements for both JavaScript and SCSS. This means that our frontend logic and styling rules can be broken up into any number of files, forming a tree of dependencies through import statements. We need only to specify the root files, or entry points, and esbuild takes care of building a complete and self-contained bundle. Not really a revolutionary idea in the grand scheme of things, but not always a given when working with JavaScript on the browser.

With this, we can trigger the bundling process: node esbuild.config.mjs. However, it’d be nice to have it better integrated with a typical .NET development flow. For that, we can add the following under scripts in package.json:

// ./package.json

{
  // ...
  "scripts": {
    "build": "node esbuild.config.mjs"
  },
  // ...
}

This allows us to run pnpm run build instead.

Finally, we can have this command run automatically as part of dotnet build if we add the following to our .csproj file:

<!-- ./RazorStartBootstrapAdmin.csproj -->

<Project Sdk="Microsoft.NET.Sdk.Web">

  <!-- ... -->

  <Target Name="BundleFrontendAssets" BeforeTargets="Build">
    <Exec Command="pnpm install" />
    <Exec Command="pnpm run build" />
  </Target>

</Project>

All this will result in our build process producing ./wwwroot/css/site.css, and ./wwwroot/js/site.js bundles. Which we can include in our .cshtml files like we would any other .js or .css file:

<!-- ./Pages/Shared/_Layout.cshtml -->

<!-- ... -->
<link rel="stylesheet" href="~/css/site.css" asp-append-version="true" />
<!-- ... -->
<script src="~/js/site.js" asp-append-version="true"></script>
<!-- ... -->

Importing libraries as modules

One interesting aspect to note is that, with esbuild, we are using JavaScript modules with import statements. This is particularly important for libraries, since some of them won’t necessarily support being included as JavaScript modules by default or in an obvious manner.

Our site.js demonstrates this. For example, it loads Bootstrap and the Font Awesome icons using the following statements:

// ./JavaScript/site.js

import 'bootstrap';
import '@fortawesome/fontawesome-free/js/all';

// ...

This makes it so any page that includes site.js will also have Bootstrap and the Font Awesome icons available.

To get the same for something like JQuery, for example, the strategy is a little different:

// ./JavaScript/site.js

import jQuery from 'jquery';
window.$ = window.jQuery = jQuery;

// ...

A similar scenario happens with the stylesheets. For SCSS, we have to use the @import statement. Our site.scss contains examples of this. Here’s how we load Bootstrap’s CSS, for instance:

// ./Stylesheets/site.scss

// ...
@import "bootstrap/scss/bootstrap.scss";
// ...

In your projects, you’ll encounter libraries with varying levels of support for JavaScript modules, and you will have to adapt accordingly, depending on the library itself and how you want to use it.

Importing libraries via HTML tags

There are also some libraries that only work via traditional <script> tags. For these, you might want to have esbuild copy files directly from the node_modules directory into wwwroot, without any preprocessing. You could use something like this in your esbuild.config.mjs file for that:

// ./esbuild.config.mjs

// Raw files
const rawFiles = [
  { entry: './node_modules/path/to/package.js', outfile: './wwwroot/lib/package.js' },
  { entry: './node_modules/path/to/package.css', outfile: './wwwroot/lib/package.css' },
];

rawFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: false,
    sourcemap: false,
    loader: { ".*": 'copy' },
    outfile: file.outfile
  });
});

Then you can include the files from wwwroot using the appropriate <script> or <link rel="stylesheet"> tags.

Summary

So, in the end, using esbuild and pnpm to set up frontend asset bundling in an ASP.NET Core web app is perfectly doable and can be summarized like this:

Install NodeJS and pnpm
Create a package.json file with pnpm init
Install esbuild as a dev dependency with pnpm add esbuild --save-dev

Also make sure to run pnpm approve-builds to allow esbuild to run scripts
Install any additional preprocessor that you might need like sass with esbuild-sass-plugin, etc.
Create JavaScript and Stylesheets directories and put your source files in them
Add an esbuild.config.mjs file that can bundle your assets
Include the asset bundling step as part of the dotnet build command

Your package.json should end up looking something like this:

{
  "name": "",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    // Register this so that esbuild can be called with "pnpm run build"
    "build": "node esbuild.config.mjs"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    // Add your dependencies here.
  },
  "devDependencies": {
    "esbuild": "0.25.9",
    // Add other preprocessors and plugins here, like:
    "esbuild-sass-plugin": "3.3.1",
    "sass": "1.78.0"
  }
}

Here’s an example of an esbuild.config.mjs that covers the common scenarios:

import * as esbuild from 'esbuild';
import { sassPlugin } from 'esbuild-sass-plugin';

// SCSS
const scssFiles = [
  { entry: './Stylesheets/site.scss', outfile: './wwwroot/css/site.css' },
  // Add other stylesheet files here.
];

scssFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    loader: { '.woff': 'file', '.woff2': 'file' },
    outfile: file.outfile,
    plugins: [sassPlugin()]
  });
});


// JavaScript
const jsFiles = [
  { entry: './JavaScript/site.js', outfile: './wwwroot/js/site.js' },
  // Add other JavaScript files here.
];

jsFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    outfile: file.outfile,
  })
});

// Raw files
const rawFiles = [
  { entry: './node_modules/path/to/package.js', outfile: './wwwroot/lib/package.js' },
  { entry: './node_modules/path/to/package.css', outfile: './wwwroot/lib/package.css' },
  // Add other library files here.
];

rawFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: false,
    sourcemap: false,
    loader: { ".*": 'copy' },
    outfile: file.outfile
  });
});

This is the addition that the .csproj file needs in order to bundle the frontend assets during dotnet build:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <!-- ... -->

  <Target Name="BundleFrontendAssets" BeforeTargets="Build">
    <Exec Command="pnpm install" />
    <Exec Command="pnpm run build" />
  </Target>

</Project>

With that, you’re free to add the bundles and raw files in your pages using traditional methods like:

<link rel="stylesheet" href="~/css/site.css" asp-append-version="true" />
<link rel="stylesheet" href="~/lib/package.css" asp-append-version="true" />

and…

<script src="~/js/site.js" asp-append-version="true"></script>
<script src="~/lib/package.js" asp-append-version="true"></script>

As you create more entrypoint JavaScript and SCSS files, you have to remember to add them to their respective arrays in esbuild.config.mjs.

You can add more packages using pnpm add <package_name>. Just remember that they have to be imported by your JavaScript files as modules. If you need to add raw files, especially from libraries that don’t support being imported as JavaScript modules, the esbuild.config.mjs file also supports that thanks to the “raw files” section. These raw files can be loaded normally via <link> and <script> HTML tags.

All right, that’s it for now. We’ve seen how to integrate esbuild with ASP.NET Core in order to setup a basic frontend asset bundling process. We did that through adapting Smart Bootstrap’s free admin template into an ASP.NET Core Razor Pages web app project — a project that’s up on GitHub. We saw how to organize source files in our repo, a basic esbuild config script that handles the most common scenarios, and how to handle a few different cases when it comes to including NodeJS packages into our apps.

Using Razor templates to render HTML emails in ASP.NET Core

2025-08-15T00:00:00+00:00

A while ago I blogged about using Razor templates to render HTML emails in .NET. The method that I discussed there worked, but it was very verbose. Since then, .NET 8 has released, and with it came a simpler way of doing this. In this post we’ll explore how to use these new features to render HTML emails.

You can find all the code in this post on GitHub.

The plan

Similar to the original article, the objective is simple: Sending emails from an ASP.NET Core app, and having the contents of those emails be rendered fromfrom Razor templates. To that end, we need four pieces:

A class for sending emails.
A class for rendering Razor templates into strings.
A Razor template.
A class that puts it all together. That is, takes in parameters, renders the email, and sends it.

Step 1: Sending emails with the MailKit NuGet package

With the help of the MailKit NuGet package, sending emails in .NET is easy. Let’s install it with:

dotnet add package MailKit --version 4.13.0

We also need some configuration in the appsettings.json file, to define the settings needed to establish a connection with an SMTP server:

// ./appsettings.json

{
    // ...
    "MailSettings": {
        "Server": "<YOUR_SMTP_SERVER>",
        "Port": 587, // 25 or 465 or 587 or 2525
        "SenderName": "RazorEmails",
        "SenderEmail": "test@razoremails.com",
        "UserName": "<YOUR_SMTP_SERVER_USER_NAME>",
        "Password": "<YOUR_SMTP_SERVER_USER_PASSWORD>"
    },
    // ...
}

And here’s a simple Mailer class that uses the MailKit library and the configuration above to send emails:

// ./Mailers/Mailer.cs

using MailKit.Net.Smtp;
using MimeKit;

namespace RazorEmails.Mailers;

public class MailData
{
    public required string To { get; set; }
    public required string ToName { get; set; }
    public required string Subject { get; set; }
    public required string Body { get; set; }
}

public class Mailer
{
    private readonly ILogger<Mailer> _logger;
    private readonly IConfiguration _config;

    public Mailer(IConfiguration config, ILogger<Mailer> logger)
    {
        _logger = logger;
        _config = config;
    }

    // Not much to see here, just a method for sending emails using MailKit.
    // Docs are here: https://github.com/jstedfast/MailKit
    public async Task<bool> SendMailAsync(MailData mailData)
    {
        try
        {
            var emailMessage = new MimeMessage();

            emailMessage.From.Add(new MailboxAddress(MailSenderName, MailSenderEmail));
            emailMessage.To.Add(new MailboxAddress(mailData.ToName, mailData.To));

            emailMessage.Subject = mailData.Subject;

            var emailBodyBuilder = new BodyBuilder
            {
                TextBody = mailData.Body,
                HtmlBody = mailData.Body
            };

            emailMessage.Body = emailBodyBuilder.ToMessageBody();

            using var smtpClient = new SmtpClient();

            await smtpClient.ConnectAsync(
                MailServer,
                MailPort,
                MailKit.Security.SecureSocketOptions.StartTls
            );

            await smtpClient.AuthenticateAsync(MailUserName, MailPassword);
            await smtpClient.SendAsync(emailMessage);
            await smtpClient.DisconnectAsync(true);

            return true;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to send email to {To}", mailData.To);
            return false;
        }
    }

    private string MailSenderName => _config["MailSettings:SenderName"]!;
    private string MailSenderEmail => _config["MailSettings:SenderEmail"]!;
    private string MailServer => _config["MailSettings:Server"]!;
    private int MailPort => int.Parse(_config["MailSettings:Port"]!);
    private string MailUserName => _config["MailSettings:UserName"]!;
    private string MailPassword => _config["MailSettings:Password"]!;
}

Step 2: Rendering Razor templates into strings

Of course, we also need a way of rendering Razor templates. As mentioned in the beginning, .NET 8 made this easy. There’s a page in the official documentation and community blog posts talking about it. For our purposes, here’s a class that does it:

// ./Rendering/RazorViewRenderer.cs

using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Web;

namespace RazorEmails.Rendering;

public class RazorViewRenderer
{
    private readonly IServiceProvider _serviceProvider;
    private readonly ILoggerFactory _loggerFactory;

    public RazorViewRenderer(IServiceProvider serviceProvider, ILoggerFactory loggerFactory)
    {
        _serviceProvider = serviceProvider;
        _loggerFactory = loggerFactory;
    }

    public async Task<string> Render<TView, TViewModel>(TViewModel model) where TView : IComponent
    {
        await using var htmlRenderer = new HtmlRenderer(_serviceProvider, _loggerFactory);

        var html = await htmlRenderer.Dispatcher.InvokeAsync(async () =>
        {
            var output = await htmlRenderer.RenderComponentAsync<TView>(
                ParameterView.FromDictionary(
                    new Dictionary<string, object?> { { "Model", model } }
                )
            );

            return output.ToHtmlString();
        });

        return html;
    }
}

The Render method is where the magic happens. We’ll see how to use it soon, but for now, it’s interesting to look at the generic type parameters.

TView represents the strongly-typed Razor template that will be rendered. Technically, the “template” is actually a Razor component, as we’ll see later. That’s why we use where TView : IComponent as a generic type constraint. With it, we’re specifying that the given TView must inherit from Microsoft.AspNetCore.Components.IComponent, which is the base class of Razor components.

TViewModel, on the other hand, represents the type of the data structure that will be passed to the template as a parameter. It will contain data that the template will use to render itself.

Step 3: Defining the email templates

Okay, now we need to define our templates, along with the vehicles to pass data to them. These will be simple DTOs.

When sending emails from web applications, it is often useful to define a layout that all emails use to keep their styling consistent. Using Razor components, such a layout could look like this:

<!-- ./Rendering/Views/MainLayout.razor -->

@inherits LayoutComponentBase

<!DOCTYPE html>
<html lang="en">
<head>
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
  @Body
</body>
</html>

This is just a basic shell for an HTML document. Notice two interesting aspects of it:

@inherits LayoutComponentBase declares that our layout inherits from the LayoutComponentBase. This is necessary for the layout to actually act as a layout.
@Body determines where the contents of the components that use this layout will be rendered. In this case, we put it inside the <body> HTML tag, which seems appropriate.

Now to define the template for a basic email that uses this layout:

<!-- ./Rendering/Views/MessageEmail.razor -->

@using RazorEmails.Rendering.ViewModels

<LayoutView Layout="@typeof(MainLayout)">

<p>
  Hello @Model.Greeting.
</p>

<p>
  We have a message for you: @Model.Message
</p>

<p>
  Message sent at @DateTime.Now.ToString().
</p>

</LayoutView>

@code {
  [Parameter]
  public MessageEmailViewModel Model { get; set; } = default!;
}

This is a very straightforward Razor component that takes in a MessageEmailViewModel as a parameter and renders a few lines of text using the data coming in the parameter. By virtue of its file name, this component can be referenced in code by the MessageEmail class name. It uses tried and true Razor syntax, so little of this should be surprising if you’ve worked with Razor before.

One thing to notice is how we’re explicitly declaring the LayoutView element, pointing to the MainLayout that we wrote. This is how we tell the renderer to use our layout when rendering this component. MainLayout exists because that’s what we named the file that contains the layout.

Like I mentioned before, MessageEmailViewModel is a simple DTO that serves to pass some data to the template. It’s not very exciting, but this is what it looks like:

// ./Rendering/ViewModels/MessageEmailViewModel.cs

namespace RazorEmails.Rendering.ViewModels;

public class MessageEmailViewModel
{
    public required string Greeting { get; set; }
    public required string Message { get; set; }
}

Step 4: Putting it all together: the class that sends the email

And finally, here we have a class that puts these separate elements to work to send an email:

// ./Mailers/MessageMailer.cs

using RazorEmails.Rendering;
using RazorEmails.Rendering.ViewModels;
using RazorEmails.Rendering.Views;

namespace RazorEmails.Mailers;

public class MessageMailer
{
    private readonly Mailer _mailer;
    private readonly RazorViewRenderer _razorViewRenderer;

    public MessageMailer(Mailer mailer, RazorViewRenderer razorViewRenderer)
    {
        _mailer = mailer;
        _razorViewRenderer = razorViewRenderer;
    }

    public async Task SendAsync(string to, string toName, string greeting, string message)
    {
        string body = await _razorViewRenderer.Render<MessageEmail, MessageEmailViewModel>(
            new MessageEmailViewModel() { Greeting = greeting, Message = message }
        );

        await _mailer.SendMailAsync(new()
        {
            To = to,
            ToName = toName,
            Subject = $"{greeting}, we have a message for you.",
            Body = body
        });
    }
}

The SendAsync method sends the email. It takes a series of parameters and uses them to construct a MessageEmailViewModel object which is passed to the RazorViewRenderer’s Render method. It also specifies MessageEmail as the Razor component to render. RazorViewRenderer returns a string which is then sent to Mailer’s SendMailAsync method as the Body parameter, along with other parameters. The Mailer then uses these parameters to construct an email and send it.

To send emails, this class can be used like so:

// Imagine we're running this in an ASP.NET Core app and getting an instance of MessageMailer via DI...
private readonly MessageMailer _messageMailer;

//... then, within some method, we can do:
await _messageMailer.SendAsync(
    "recipient@example.com",
    "Recipient Name",
    "Mr. Recipient",
    "Have a good day."
);

Simple and clean.

All right! That’s it for now. In this article we saw how to leverage new .NET 8 features for rendering Razor components into strings. We implemented an email sending capability based on that and the MailKit NuGet package. It was a nice way of revisiting an old topic, now made easier thanks to the latest updates from .NET.

Processing payments with Authorize.Net in a Blazor app

2025-08-11T00:00:00+00:00

Blazor, the SPA framework from ASP.NET Core, is excellent technology. However, it is very particular when it comes to interacting with JavaScript. This is because Blazor apps are written in C# with Razor templates. So, if a Blazor app needs to interact with a JavaScript library (for example, to process credit card payments through Authorize.Net), special care is needed. In this article, we’ll see an approach on how to make that happen.

We’ll develop a Blazor WebAssembly standalone app which will include a form to capture credit card information and send it to Authorize.Net using their Accept.js frontend integration library. Then, Authorize.Net will return a payment token that represents the captured credit card. Our frontend will then submit the resulting token to a backend API to actually effectuate the payment.

This backend API, which we’ll also develop, will be a simple ASP.NET Core Web API. It’ll include an endpoint for submitting payment transactions to Authorize.Net, given the token obtained via Accept.js.

You can find the all the source code discussed here on GitHub.

The backend

Let’s begin by creating a new ASP.NET Core Web API project with this:

dotnet new webapi --use-controllers -o BlazorAuthorizeNet.WebApi

We also need to install Authorize.Net’s NuGet package:

cd BlazorAuthorizeNet.WebApi
dotnet add package AuthorizeNet.Core --version 8.0.1

Interacting with Authorize.Net

Now we have to create a class to submit payment transactions to Authorize.Net. You can arrive at something like this by reading the Authorize.Net documentation, but here’s what we will go with:

First, a class to set up the connection and serve as entry point.

// BlazorAuthorizeNet.WebApi/Payments/AuthorizeNetPaymentGateway.cs

using AuthorizeNet.Api.Contracts.V1;
using AuthorizeNet.Api.Controllers.Bases;
using BlazorAuthorizeNet.WebApi.Models;

namespace BlazorAuthorizeNet.WebApi.Payments;

// Provives an interface for interacting with the Authorize.Net payment processor.
public class AuthorizeNetPaymentGateway : IPaymentGateway
{
    private readonly IConfiguration _configuration;

    public AuthorizeNetPaymentGateway(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    // Submits a payment transaction to Authorize.Net
    public PaymentTransactionResult CreatePaymentTransaction(Order order)
    {
        PrepareConnection();
        return AuthorizeNetCreateTransaction.Run(order);
    }

    // This method sets up the AuthorizeNet.Core client library's connection to Authorize.Net's Web API.
    // The values for RunEnvironment and MerchantAuthentication are constructed based on app settings.
    private void PrepareConnection()
    {
        ApiOperationBase<ANetApiRequest, ANetApiResponse>.RunEnvironment =
            _configuration["AuthNetEnvironment"] == "Production" ?
                AuthorizeNet.Environment.PRODUCTION : AuthorizeNet.Environment.SANDBOX;

        ApiOperationBase<ANetApiRequest, ANetApiResponse>.MerchantAuthentication = new()
        {
            name = _configuration["AuthNetLoginId"],
            ItemElementName = ItemChoiceType.transactionKey,
            Item = _configuration["AuthNetTransactionKey"],
        };
    }
}

Then a class that implements the actual call to Authorize.Net’s endpoint and the parsing of the response.

// BlazorAuthorizeNet.WebApi/Payments/AuthorizeNetCreateTransaction.cs

using AuthorizeNet.Api.Contracts.V1;
using AuthorizeNet.Api.Controllers;
using BlazorAuthorizeNet.WebApi.Models;

namespace BlazorAuthorizeNet.WebApi.Payments;

// Calls the Authorize.Net "Create an Accept Payment Transaction" API endpoint.
// https://developer.authorize.net/api/reference/index.html#accept-suite-create-an-accept-payment-transaction
internal static class AuthorizeNetCreateTransaction
{
    // This method accepts an Order object, which is nothing more than a DTO that contains all the information needed
    // to piece together a request payload for Authorize.Net's "Create an Accept Payment Transaction" API endpoint.
    // It constructs a new object that represents such payload (i.e. the createTransactionRequest), submits the request,
    // and handles the respose.
    //
    // It returns a PaymentTransactionResult which is, again, a DTO that captures important information from the
    // response from Authorize.Net so that the code and issued the request can act according to the response.
    internal static PaymentTransactionResult Run(Order order)
    {
        // 1. Construct the request payload.
        var request = new createTransactionRequest
        {
            transactionRequest = new transactionRequestType
            {
                transactionType = transactionTypeEnum.authCaptureTransaction.ToString(),

                amount = order.Total,
                payment = BuildPayment(order),
                billTo = BuildBillingAddress(order),
                lineItems = BuildLineItems(order)
            }
        };

        // 2. Send the request.
        var controller = new createTransactionController(request);
        controller.Execute();

        var response = controller.GetApiResponse();

        // 3. Process the response.
        return BuildResult(response);
    }

    // As we've mentioned before, the way we're going to process payments is using Authorize.Net's Accept.js client side
    // library to tokenize a credit card. This is the method that constructs the part of the request that instructs
    // Authorize.Net to process the payment in that fashion. It uses an "opaqueDataType" which contains the "Nonce
    // descriptor" and "Nonce value" fields. These fields represent the tokenized credit card sent to the backend by the
    // frontend.
    private static paymentType BuildPayment(Order order) =>
        new()
        {
            Item = new opaqueDataType
            {
                dataDescriptor = order.PaymentMethodNonceDescriptor,
                dataValue = order.PaymentMethodNonceValue
            }
        };

    private static customerAddressType BuildBillingAddress(Order order) =>
        new()
        {
            firstName = order.BillingAddress.FirstName,
            lastName = order.BillingAddress.LastName,
            address = order.BillingAddress.StreetOne,
            city = order.BillingAddress.City,
            zip = order.BillingAddress.ZipCode,
            state = order.BillingAddress.StateCode,
            country = order.BillingAddress.CountryCode
        };

    private static lineItemType[] BuildLineItems(Order order) =>
        order.Items.Select(item => new lineItemType
        {
            itemId = item.Id.ToString(),
            name = item.ProductName,
            quantity = item.Quantity,
            unitPrice = item.UnitPrice
        }).ToArray();

    // This method constructs a PaymentTransactionResult based on the response from Authorize.Net.
    // It inspects the response in various ways to determine whether it was successful or erroneus and extracts useful
    // information from it.
    private static PaymentTransactionResult BuildResult(createTransactionResponse? response)
    {
        if (response == null)
        {
            return PaymentTransactionResult.Failure(
                errorMessage: "No response from gateway"
            );
        }

        if (response.messages.resultCode != messageTypeEnum.Ok)
        {
            if (response.transactionResponse != null && response.transactionResponse.errors != null)
            {
                return PaymentTransactionResult.Failure(
                    response.transactionResponse.errors[0].errorCode,
                    response.transactionResponse.errors[0].errorText
                );
            }
            else
            {
                return PaymentTransactionResult.Failure(
                    response.messages.message[0].code,
                    response.messages.message[0].text
                );
            }
        }

        if (response.transactionResponse.messages == null)
        {
            if (response.transactionResponse.errors != null)
            {
                return PaymentTransactionResult.Failure(
                    response.transactionResponse.errors[0].errorCode,
                    response.transactionResponse.errors[0].errorText
                );
            }
            else
            {
                return PaymentTransactionResult.Failure(
                    errorMessage: "Unexpected format for error response from gateway"
                );
            }
        }

        return PaymentTransactionResult.Success(
            response.transactionResponse.transId,
            response.transactionResponse.responseCode,
            response.transactionResponse.messages[0].code,
            response.transactionResponse.messages[0].description,
            response.transactionResponse.authCode
        );
    }
}

Those classes have the core logic for interacting with Authorize.Net to submit payment transactions using credit cards tokenized with Accept.js. There are additional supporting DTOs used here like PaymentTransactionResult and Order. You can see them on GitHub.

The AuthorizeNetPaymentGateway class also employs various configuration settings necessary for interacting with Authorize.Net: AuthNetEnvironment, AuthNetLoginId, and AuthNetTransactionKey. So we need to make sure to define them in the WebApi project’s appsettings.json file:

{
  // ...
  "AuthNetEnvironment": "Sandbox",
  "AuthNetLoginId": "<YOUR_AUTH_NET_LOGIN_ID>",
  "AuthNetTransactionKey": "<YOUR_AUTH_NET_TRANSACTION_KEY>"
}

These values are unique to the particular Authorize.Net account being used.

Defining the API endpoint

We also need to define the endpoint for the frontend to submit the payment. This controller does this:

// BlazorAuthorizeNet.WebApi/Controllers/PaymentsController.cs

using Microsoft.AspNetCore.Mvc;
using BlazorAuthorizeNet.WebApi.Models;
using BlazorAuthorizeNet.WebApi.Payments;

namespace BlazorAuthorizeNet.WebApi.Controllers;

[ApiController]
[Route("[controller]")]
public class PaymentsController : ControllerBase
{
    private readonly IPaymentGateway _paymentGateway;

    public PaymentsController(IPaymentGateway paymentGateway)
    {
        _paymentGateway = paymentGateway;
    }

    // POST: Payments
    [HttpPost]
    public ActionResult PostPayment([FromBody] PaymentPayload payload)
    {
        // To keep things simple, this endpoint expects the values obtained from Authorize.Net's Accept.js that
        // represent the tokenized credit card, encapsulated in PaymentPayload.
        //
        // For the details of the actual "product" that was "purchased", we simulate an order by creating a hardcoded
        // object here. Of course, for real world applications, the generation of the "Order" object would be much more
        // complex.
        Order order = new(
            Total: 100.00m,
            Items:
            [
                new OrderItem(1, "Product A", 1, 50.00m),
                new OrderItem(2, "Product B", 1, 50.00m)
            ],
            BillingAddress: new Address(
                FirstName: "John",
                LastName: "Doe",
                StreetOne: "123 Main St",
                StreetTwo: "Suite A",
                City: "New York",
                ZipCode: "12345",
                StateCode: "CA",
                CountryCode: "US"
            ),
            // Importantly, we include the payment token values here.
            PaymentMethodNonceValue: payload.PaymentMethodNonceValue,
            PaymentMethodNonceDescriptor: payload.PaymentMethodNonceDescriptor
        );

        // Then we simply submit the payment via our payment gateway object and return a response based on the result
        // we get from Authorize.Net.
        var result = _paymentGateway.CreatePaymentTransaction(order);

        if (result.IsSuccess) return Ok(result);
        else return BadRequest(result);
    }
}

This endpoint expects a request payload that includes the values obtained from Authorize.Net’s Accept.js after tokenizing the credit card. PaymentPayload represents the data structure and it looks like this:

// BlazorAuthorizeNet.WebApi/Models/PaymentPayload.cs

namespace BlazorAuthorizeNet.WebApi.Models;

public record PaymentPayload(
    string PaymentMethodNonceValue,
    string PaymentMethodNonceDescriptor
);

This means that when calling this endpoint, clients would have to provide a JSON payload that looks like this:

// POST Payments

{
    "PaymentMethodNonceValue": "<PAYMENT_METHOD_NONCE_VALUE>",
    "PaymentMethodNonceDescriptor": "<PAYMENT_METHOD_NONCE_DESCRIPTOR>"
}

Finally, we need to update the WebApi project’s Program.cs file in a couple of ways. We need to configure the Dependency Injection so that the AuthorizeNetPaymentGateway is available to the controller, and we also need to enable CORS. We do need CORS of course, because the project is a Web API meant to be called from a browser app, which may or may not be running in the same domain as the API. Here’s what it ends up looking like:

using BlazorAuthorizeNet.WebApi.Payments;

var builder = WebApplication.CreateBuilder(args);

+// Add services to the container.
+builder.Services.AddScoped<IPaymentGateway, AuthorizeNetPaymentGateway>();
+
+// CORS configuration to allow requests from other origins.
+builder.Services.AddCors(options =>
+{
+    options.AddDefaultPolicy(policy =>
+    {
+        var origins = builder.Configuration["AllowedOrigins"]?.Split(",");
+        if (origins == null || origins.Length == 0) return;
+
+        policy.WithOrigins(origins)
+            .AllowAnyHeader()
+            .AllowAnyMethod()
+            .AllowCredentials();
+    });
+});

builder.Services.AddControllers();
// Learn more about configuring OpenAPI at https://aka.ms/aspnet/openapi
builder.Services.AddOpenApi();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.MapOpenApi();
}

app.UseHttpsRedirection();

+// Enable CORS.
+app.UseCors();

app.UseAuthorization();

app.MapControllers();

app.Run();

Here we’ve included usage of a new configuration setting: AllowedOrigins. This is meant to be the URL where the frontend will be running. Whatever it is, we need to make sure to define it in the WebApi project’s appsettings.json file:

{
  // ...
  "AllowedOrigins": "<YOUR_FRONTEND_URL>",
}

The frontend

Now for the frontend, we need to create a project for a Blazor WebAssembly standalone app. This command does it:

dotnet new blazorwasm -o BlazorAuthorizeNet.Frontend

Next we create a new “Payment” page. This is the page where the interaction between Blazor and JavaScript will happen. This page will also submit the credit card information to Authorize.Net, and pass the resulting token to our backend API, to effectuate the payment. Here’s an overview of how it will all work:

We will define a new Payment.razor component which will include the form to capture the credit card information. It will also include logic to load the JavaScript module that will handle interactions with Authorize.Net. This JavaScript module will be defined in a new Payment.razor.js file and will take care of loading Authorize.Net’s Accept.js library. It will also wire up the logic to initiate the call to Authorize.Net when the user submits the form.

Once everything is initialized and rendered, the user will fill out the form and hit the submit button. Then the logic in Payment.razor.js will kick in and use the Accept.js library to send the credit card information to Authorize.Net, and obtain a single-use payment token. Then, it will invoke a method in the Payment.razor Blazor component, giving it the token so that it can be sent to our backend API. Finally, it will submit the payment to Authorize.Net.

Payment page initialization

Let’s start implementing the initialization side of things by defining our Payment.razor component. Here’s the template part:

<!-- BlazorAuthorizeNet.Frontend/Pages/Payment.razor -->

@page "/Payment"

@implements IAsyncDisposable

@inject IConfiguration Config
@inject HttpClient HttpClient
@inject ILogger<Payment> Logger;
@inject IJSRuntime JS

<PageTitle>Home</PageTitle>

<h1>Submit Payment</h1>

<section class="py-5">
    <div class="container px-4 px-lg-5 my-5">
        <h1>Checkout</h1>

        <div class="card mb-4 p-3">
            <h4 class="mb-3">Payment Information</h4>

            <div class="mb-3">
                <div class="form-floating">
                    <input
                        type="text" name="cardNumber" id="cardNumber"
                        class="form-control" placeholder="4111111111111111"
                    />
                    <label for="cardNumber" class="form-label">
                        Card Number
                    </label>
                </div>
            </div>

            <div class="row">
                <div class="col">
                    <div class="form-floating">
                        <input
                            type="text" name="expMonth" id="expMonth"
                            class="form-control" placeholder="12"
                        />
                        <label for="expMonth" class="form-label">
                            Expiration Month
                        </label>
                    </div>
                </div>
                <div class="col">
                    <div class="form-floating">
                        <input
                            type="text" name="expYear" id="expYear"
                            class="form-control" placeholder="34"
                        />
                        <label for="expYear" class="form-label">
                            Expiration Year
                        </label>
                    </div>
                </div>
                <div class="col">
                    <div class="form-floating">
                        <input
                            type="text" name="cardCode" id="cardCode"
                            class="form-control" placeholder="123"
                        />
                        <label for="cardCode" class="form-label">CVV</label>
                    </div>
                </div>
            </div>
        </div>

        <div class="d-flex justify-content-end">
            <button
                id="SubmitOrderButton" type="button"
                class="btn btn-outline-dark"
            >
                Place Order
            </button>
        </div>

        <ul id="AuthNetErrors" class="validation-errors"></ul>

        @if (!string.IsNullOrEmpty(errorMessage))
        {
            <div class="alert alert-danger" role="alert">
                @errorMessage
            </div>
        }

        @if (!string.IsNullOrEmpty(successMessage))
        {
            <div class="alert alert-success" role="alert">
                @successMessage
            </div>
        }
    </div>
</section>

The HTML template itself is very unremarkable. It’s just a form for capturing credit card number, expiration date, and CVV, along with a button to submit the form. What’s interesting about this code are the @inject statements at the top. Especially the @inject IJSRuntime JS one. IJSRuntime is the object that allows Blazor components to invoke JavaScript logic in the browser. We will see how the component uses it next.

Here’s the logic of the component:

// BlazorAuthorizeNet.Frontend/Pages/Payment.razor

@code {
    // We use this to store a reference to the current Blazor component. When passed to JavaScript code, it allows
    // JavaScript to call methods defined in this component.
    private DotNetObjectReference<Payment>? objRef;

    // This represents a JavaScript module. We use it to store a reference to Payment.razor.js and invoke functions
    // defined in it.
    private IJSObjectReference? module;

    private string? errorMessage;
    private string? successMessage;

    // This method initializes the reference to this Blazor component, loads the Payment.razor.js file, and calls the
    // initializeAuthNetAcceptJs function defined in it.
    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        if (firstRender)
        {
            // Create a reference to this Blazor component which we will use below to pass to the
            // initializeAuthNetAcceptJs function defined in Payment.razor.js.
            objRef = DotNetObjectReference.Create(this);

            // Here we're essentially issing a command to the browser's JavaScript runtime. We're using JavaScript's
            // import statement to load the Payment.razor.js file as a module.
            // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
            module = await JS.InvokeAsync<IJSObjectReference>(
                "import", "./Pages/Payment.razor.js"
            );

            // Finally, we call the initializeAuthNetAcceptJs function, passing it the DotNetObjectReference as well as
            // a few app settings. This method initializes Accept.js on the page.
            await module.InvokeVoidAsync(
                "initializeAuthNetAcceptJs",
                objRef,
                Config["AuthNetEnvironment"] ?? "Sandbox",
                Config["AuthNetLoginId"],
                Config["AuthNetClientKey"]
            );
        }
    }

    // The DotNetObjectReference and IJSObjectReference objects need to be disposed manually.
    public async ValueTask DisposeAsync()
    {
        objRef?.Dispose();

        if (module is not null)
        {
            await module.DisposeAsync();
        }
    }
}

Here’s where things start to get interesting. The OnAfterRenderAsync method demonstrates how Blazor interacts with JavaScript. In this method, which gets called by the framework when the component has completed rendering, we:

Make sure to run the logic only once using if (firstRender).
Instantiate a DotNetObjectReference object, which Blazor will pass to JavaScript to allow the JavaScript code to call Blazor back once it’s done its work.
Instantiate the JavaScript module defined in Payment.razor.js.
Call the initializeAuthNetAcceptJs function in the JavaScript module which will set up Accept.js.

We also have a DisposeAsync method that’s necessary for disposing resources like the DotNetObjectReference and IJSObjectReference instances.

Notice also how we’re using a few configuration settings here: AuthNetEnvironment, AuthNetLoginId, AuthNetClientKey, and WebApiUrl. These are defined in BlazorAuthorizeNet.Frontend/wwwroot/appsettings.json. Something like this:

// BlazorAuthorizeNet.Frontend/wwwroot/appsettings.json

{
  "WebApiUrl": "<YOUR_WEB_API_URL>",
  "AuthNetEnvironment": "Sandbox",
  "AuthNetLoginId": "<YOUR_AUTH_NET_LOGIN_ID>",
  "AuthNetClientKey": "<YOUR_AUTH_NET_CLIENT_KEY>"
}

Now we can look at the JavaScript to be defined in Payment.razor.js.

// BlazorAuthorizeNet.Frontend/Pages/Payment.razor.js

// Sets up the page for interaction with Accept.js.
export function initializeAuthNetAcceptJs(dotNet, authNetEnvironment, authNetLoginId, authNetClientKey) {
    loadScript(authNetEnvironment);
    setUpSubmit(authNetLoginId, authNetClientKey, dotNet);
}

// Loads the Accept.js frontend library.
function loadScript(authNetEnvironment) {
    // Accept.js offers two different versions of the library, one for production and one for testing (i.e. sandbox)
    // So, we need to load one or the other depending on the given "authNetEnvironment".
    const acceptJsUrl = authNetEnvironment == "Production" ?
        "https://js.authorize.net/v1/Accept.js" :
        "https://jstest.authorize.net/v1/Accept.js";

    console.log("Loading Authorize.Net Accept.js from: ", acceptJsUrl);

    // Manually create a <script> element and attach it to the current DOM.
    var script = document.createElement('script');
    script.src = acceptJsUrl;
    document.body.appendChild(script);
}

function setUpSubmit(authNetLoginId, authNetClientKey, dotNet) {
    // We'll implement this soon
}

As we’ve mentioned before, this is a JavaScript module. This module exposes one single function: initializeAuthNetAcceptJs. We export this function because that’s the one that the Blazor component needs to call. What this function does is set up the page for interacting with Authorize.Net via its Accept.js library.

For now, all it does is load the Accept.js library directly from Authorize.Net. Depending on the AuthNetEnvironment config setting, either the production version or the sandbox version of the library is loaded.

We also have a setUpSubmit function here and we’ll talk about it next.

Payment submission

Now the stage is set to allow users to enter their credit card information and submit it for payment processing. Let’s add this method to the Payment.razor component:

// BlazorAuthorizeNet.Frontend/Pages/Payment.razor

record PaymentResponseSuccess(string MessageDescription);
record PaymentResponseError(string ErrorMessage);

// The JSInvokable attribute allows this method to be invoked from JavaScript via the passed DotNetObjectReference.
[JSInvokable]
public async void SubmitOrder(string paymentMethodNonceValue, string paymentMethodNonceDescriptor)
{
    successMessage = null;
    errorMessage = null;

    // Calls the backend Web API's endpoint and sends it the nonce value and descriptor. That is, the tokenized
    // representation of the credit card that we obtained from Accept.js.
    using var response = await HttpClient.PostAsJsonAsync(
        $"{Config["WebApiUrl"]}/Payments",
        new
        {
            PaymentMethodNonceValue = paymentMethodNonceValue,
            PaymentMethodNonceDescriptor = paymentMethodNonceDescriptor
        }
    );

    // Handle the response by partsing the contents and displaying messages in the page accordingly.
    var isSuccess = response.IsSuccessStatusCode;

    if (isSuccess)
    {
        var success = await response.Content.ReadFromJsonAsync<PaymentResponseSuccess>();
        successMessage = success?.MessageDescription;
    }
    else
    {
        var error = await response.Content.ReadFromJsonAsync<PaymentResponseError>();
        errorMessage = error?.ErrorMessage;
    }

    StateHasChanged();
}

The first thing to note about the SubmitOrder method is that it is annotated with the [JSInvokable] attribute. This attribute allows the JavaScript code to call this method, through the DotNetObjectReference instance that it is given. Other than that, the method is straightforward. It calls the POST Payments endpoint, passing it the values returned by Authorize.Net (i.e. paymentMethodNonceValue and paymentMethodNonceDescriptor), which are given to our Blazor component by our JavaScript module.

And, of course, we need to update our JavaScript module to do the rest of the logic of calling Authorize.Net and this new SubmitOrder method. Here’s how we implement the setUpSubmit function in Payment.razor.js:

// BlazorAuthorizeNet.Frontend/Pages/Payment.razor.js

// Wires up the onclick event handler that submits the credit card information to Authorize.Net via Accept.js, captures
// the returned tokenized card, and sends it to the Blazor component.
function setUpSubmit(authNetLoginId, authNetClientKey, dotNet) {
    const submitButton = getSubmitButton();
    submitButton.addEventListener("click", () => doSubmit(authNetLoginId, authNetClientKey, dotNet));
}

function doSubmit(authNetLoginId, authNetClientKey, dotNet) {
    disableSubmitButton();
    clearErrors();

    // Send the credit card details to Authorize.Net, and...
    sendPaymentDataToAuthNet(
        authNetLoginId,
        authNetClientKey,
        // ...when successful, use the DotNetObjectReference given by Blazor to call the SubmitOrder on the
        // Payment.razor component. Sending it the tokenized credit card.
        response => {
            dotNet.invokeMethod(
                "SubmitOrder",
                response.opaqueData.dataValue,
                response.opaqueData.dataDescriptor
            );
        }
    );
}

// This function constructs the payload that Accept.js expects containing the credit card information and sends it.
function sendPaymentDataToAuthNet(authNetLoginId, authNetClientKey, onDone) {
    var authData = {
        clientKey: authNetClientKey,
        apiLoginID: authNetLoginId
    };

    var cardData = {
        cardNumber: document.getElementById("cardNumber").value?.replace(/\s+/g, ''),
        month: document.getElementById("expMonth").value,
        year: document.getElementById("expYear").value,
        cardCode: document.getElementById("cardCode").value
    };

    // In addition to credit cards, Accept.js also supports charging a bank account. This commented out code
    // demonstrates how to do that.
    /* var bankData = {
        accountNumber: document.getElementById('accountNumber').value,
        routingNumber: document.getElementById('routingNumber').value,
        nameOnAccount: document.getElementById('nameOnAccount').value,
        accountType: document.getElementById('accountType').value
    }; */

    var data = {
        authData,
        cardData
        /* bankData: */
    };

    console.log("Request to Authorize.Net: ", data);
    Accept.dispatchData(data, response => handleAuthNetResponse(response, onDone));
}

function handleAuthNetResponse(response, onDone) {
    console.log("Response from Authorize.Net: ", response);
    if (response.messages.resultCode === "Error") {
        console.log("Error from Authorize.Net: ", response);

        displayErrors(response.messages.message);
    } else {
        onDone(response);
    }

    enableSubmitButton();
}

// This function populates the "AuthNetErrors" <ul> with items containing descriptions of the errors returned by
// Auhtorize.NET.
function displayErrors(errors) {
    const errorList = document.getElementById("AuthNetErrors");

    errors.forEach(error => {
        const li = document.createElement("li");
        li.textContent = getErrorMessage(error.code);
        li.className = "validation-message";
        errorList.appendChild(li);
    });
}

// These are some of the most common errors that we should expect from Authorize.Net.
// Learn more at https://developer.authorize.net/api/reference/features/acceptjs.html#Appendix_Error_Codes
function getErrorMessage(errorCode) {
    const map = {
        "E_WC_04": "Please provide card number, expiration month, year and CVV.",
        "E_WC_05": "Please provide valid card number.",
        "E_WC_06": "Please provide valid expiration month.",
        "E_WC_07": "Please provide valid expiration year.",
        "E_WC_08": "Please provide a future expiration date.",
        "E_WC_15": "Please provide valid CVV.",
        "E_WC_20": "Please provide valid card number."
    };

    return map[errorCode] || "We couldn't process your card at this time. Please try again.";
}

function getSubmitButton() {
    return document.getElementById("SubmitOrderButton");
}

function enableSubmitButton() {
    const submitButton = getSubmitButton();
    submitButton.disabled = false;
}

function disableSubmitButton() {
    const submitButton = getSubmitButton();
    submitButton.disabled = true;
}

function clearErrors() {
    const errorList = document.getElementById("AuthNetErrors");
    errorList.innerHTML = "";
}

This code wires up logic so that when the form submit button is clicked: the captured credit card information is sent to Authorize.Net; any errors from Authorize.Net are handled and displayed to the user; and, when successful, the returned payment token is sent back to the Blazor component, so that it can send it to the backend API. Most of this logic is very similar to what you would find in the official documentation from Authorize.Net, except for the part that interacts with our Blazor component.

And that’s all for now! In this article we’ve seen how to build a Blazor WebAssembly standalone application, which interfaces with Authorize.Net through their Accept.js frontend library, to tokenize a credit card and submit payment transactions by calling a backend API.

Reducing page load times: Cloudflare and caching with ASP.NET

2025-01-14T00:00:00+00:00

When designing the architecture for a new website, it’s important to keep caching in mind. Caching allows you to store a copy of the pages or resources used by your web application in your local browser. Content distribution networks such as Cloudflare also leverage cache directives to distribute content to users more efficiently.

If we are going to use response caching on our web application, we should not include anything that depends on the identity context for the user in the page content in the response. Instead, we should render the same content to every user that hits the same URL. If we include any identity-specific content, the CDN will reuse it for other requests, which is a security and privacy risk.

A simple example is an ecommerce site where the customer logs in to make a purchase: If the homepage shows the user name and icon at the top bar, that content should be rendered from a non-cached request. Otherwise, CDN caching will show the same user and icon to other visitors. And worse: depending on the validations applied to the website, the profile page could even display personal information to other users!

When dealing with these situations, I often rely on asynchronous requests to a set of non-cached API endpoints to retrieve the identity information, and process it on the frontend to render the user-dependent content.

Adding response caching

In .NET Core, we have a simple way of adding response caching to an ASP.NET application. Let’s see how easy it is to enable caching on a Razor Pages app. First, in our Program.cs file, we need to add a couple lines:

+ builder.Services.AddResponseCaching();

var app = builder.Build();

+ app.UseResponseCaching();

Second, we need to include the [ResponseCache] attribute on our action methods to specify the caching options for that request. In the example below, we are adding a response cache directive of 30 seconds that will vary depending on the User-Agent header sent with the request:

+ [ResponseCache(VaryByHeader = "User-Agent", Duration = 30)]
public async Task<IActionResult> OnGet()
{
    return Page();
}

To test the new logic in place, we can navigate to the page in Chrome and review the response headers in DevTools. We should set the cache-control header to public, with the max-age attribute set to the duration in seconds we specified.

Using Cloudflare to distribute cached content

Once we have our cache responses configured, we can leverage Cloudflare to store our cached responses in the CDN, and improve our page load response time for the user while decreasing the number of requests to our server. Win-win!

By default, dynamic content is not cached in Cloudflare. This is a default configuration to avoid security implications: Cloudflare will disable caching on certain content type and file extensions (such as HTML, or in our case, ASP.NET pages). That means that once we deploy our web application and test it with the CDN active, we will get a response header similar to this:

The cf-cache-status header is a custom header added to the response by Cloudflare that indicates whether the content was cached by the CDN. In this case, we got the value DYNAMIC, meaning that Cloudflare does not consider the content eligible to cache.

To enforce caching, we need to add some additional rules: On Cloudflare’s dashboard, let’s select our website and, on the left nav, go to Rules → Page Rules. We will open a section where we can add custom rules that will apply to specific URL patterns. Here, I usually create two rules:

A page rule that will cache everything on the website. This rule will tell Cloudflare to cache dynamic and static content on the CDN. This is reasonable since we are aware of the implications of caching our pages, and already made the changes neccesary to avoid caching identity-related data.
A second page rule that will skip caching completely on the location where we will be delivering identity-specific content. For example, any API endpoints that will return the user details to be displayed at the top bar, or the user profile page, should be bypassed by the CDN. In our example, those API endpoints are inside the /api/user/* location.

Note: It’s important to set this rule to the first position in the list, so it’s not overridden by previous rules that were deployed.

Testing

After saving and deploying both rules, it’s time to test the results. When we navigate to our website’s homepage with Chrome, we will see the updated response headers:

The “HIT” value means the request hit a cached version of the resource in the CDN.

And finally, we should test the response headers on our identity-related API endpoints.

Yes! We’re getting a BYPASS value for the cf-cache-status header, meaning Cloudflare is not caching any content from that URL.

All right! This is as far as it goes for this post. Now, we have our ASP.NET pages cached, and Cloudflare is distributing the cached version through the CDN, reducing the number of requests to our servers and decreasing the response times for the user. Of course, this is just the tip of the iceberg; we can also add caching for assets and external resources, specify different strategies and durations for each resource type, and more. All that is material for upcoming blog posts.

Using a Containerized Nginx Proxy to Serve a Multi-Application .NET System

2024-09-24T00:00:00+00:00

We recently blogged about how we deployed a system made of multiple .NET applications using Docker containers. In order to make them accessible over the internet, we created a reverse proxy using Nginx.

In that case, we installed and configured the Nginx instance directly in the server, as opposed to the rest of the applications, which ran within containers. That approach did and still does work well for us.

In this article, we’re going to explore an alternative strategy. One where we push the containerization aspect further and deploy and run the Nginx instance itself in a Docker container.

Reintroducing the demo project

Like I said, our system has multiple runtime components, each one of them running in their own container. We have two ASP.NET Core web applications: an Admin Portal and a Web API. They live in this Git repository. And we also have a Postgres database, which the apps interact with.

We also have another repository where the deployment-related files are stored. Among others, there are the expected compose.yaml and Dockerfiles that describe the entire infrastructure.

Throughout this post we will update those deployment configuration files to add an Nginx reverse proxy.

Let’s see what that would look like.

Adding the proxy service in `compose.yaml`

First we need to add the new container in the compose.yaml’s services section. It doesn’t need to be too complicated:

# compose.yaml

services:

# ...

  proxy:
    # The proxy container will be based on this Dockerfile, which we'll define
    # soon.
    build:
      context: .
      dockerfile: Dockerfile.Proxy

    # Here we expose the Nginx proxy via port 8888. This can be anything. In
    # fact, if you want to do multiple parallel deployments on the same machine,
    # that is, with many Nginx instances running at the same time, you can
    # adjust this setting appropriately to prevent port conflicts. Making sure
    # that each instance has its own port.
    ports:
      - 8888:80

    # We want the proxy to start up after the admin-portal and web-api services
    # are up and running. The depends_on setting helps with that.
    depends_on:
      admin-portal:
        condition: service_started
      web-api:
        condition: service_started

#...

Writing the proxy `Dockerfile`

As you saw, the proxy configuration in compose.yaml leverages an external Dockerfile to build the container image that will run our Nginx proxy. This file is also very straightforward. It uses the official Nginx image from Docker Hub and it looks like this:

# Dockerfile.Proxy

# We can pick the version and flavor that we like. In our case here, this is an
# image based on the latest release of Nginx, and the latest release of Debian.
FROM nginx:1.27.1-bookworm

# Unsurprisingly, we have a custom configuration that we want the proxy to use.
# This is how we make sure it does. We copy it into the default location inside
# the image.
COPY proxy/nginx.conf /etc/nginx/nginx.conf

Configuring the proxy

Now we have to configure the Nginx proxy to route requests to both our web applications. Here’s an nginx.conf that does just that:

# proxy/nginx.conf

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;

    keepalive_timeout  65;

    # We have to comment out or remove this line to make sure the default
    # configuration that comes in the official image is not applied.
    # include /etc/nginx/conf.d/*.conf;

    # Our customizations start here:
    server {
        # With this listen directive, we configure our proxy to expect
        # connections coming from the default HTTP port: 80.
        listen 80;

        # This location directive makes sure all requests coming to URLs that
        # look like .../admin are routed to the Admin Portal web app.
        location ~ ^/admin(/?)(.*) {
            proxy_pass http://admin-portal:8080;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header Connection keep-alive;
        }

        # This location directive makes sure all requests coming to URLs that
        # look like .../api are routed to the Web API.
        location ~ ^/api(/?)(.*) {
            proxy_pass http://web-api:8080;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header Connection keep-alive;
        }
    }
}

What I’ve done here is take the default Nginx configuration file that comes right out of the box, and replace the include /etc/nginx/conf.d/*.conf; line with my own server block directive.

As explained in the official image’s Docker Hub page, a quick way of obtaining a copy of this default file is using this command: docker run --rm --entrypoint=cat nginx /etc/nginx/nginx.conf > /host/path/nginx.conf.

The most interesting parts are the proxy_pass directives that take care of redirecting traffic to the apps. Notice how they refer to the Admin Portal using http://admin-portal:8080 and to the Web API by http://web-api:8080. These are the internal naming of these components within the containers’ virtual network, which gets created automatically by Docker Compose.

Remember that this Nginx instance is not running in the host machine directly. Instead it’s running in a container. In the same virtual network as the other containers described in compose.yaml. That’s why it has to use the hostnames assigned by compose.yaml (i.e. admin-portal and web-api) and the port through which the ASP.NET Core apps running within accept requests (i.e. 8080).

This is the idea:

It also uses proxy_set_header directives to set Host and Connection headers. This is typical practice when it comes to Nginx reverse proxies. You can read more about that on Nginx’s website.

Configuring the PathBase in ASP.NET Core apps

There’s an additional step that we have to take to make all this work. We have configured our Nginx proxy to serve both applications under the same “server”, and rely on different URL paths (i.e. /admin vs /api) to determine which app will receive which request. This means that we have to perform further configuration in the apps so that routing is done properly. Thankfully, all it takes is a one-liner in each of the apps’ Program.cs files. After the usual var app = builder.Build(); line, we do the following:

For the Admin Portal, we add this:

// source/VehicleQuotes.AdminPortal/Program.cs

// ...

app.UsePathBase("/admin");

// ...

And for the Web API:

// source/VehicleQuotes.WebApi/Program.cs

// ...

app.UsePathBase("/api");

// ...

With that, the apps are ready to handle requests coming from the Nginx proxy, which will include either the /admin or /api sections.

Deploying

Finally, we can deploy. Go into the directory where all the deployment configuration files live. That’s the one with the compose.yaml file. In other words, the root of the deploy repo. Once in there, run:

docker compose up --build

After a while, Docker compose will have built and deployed everything for us. Now you can navigate to the apps in any browser at http://localhost:8888/admin and http://localhost:8888/api/swagger/index.html.

As you click around, you can see the Nginx logs with:

docker compose logs proxy -f

To bring it all back down, you can run:

docker compose down

Cool! At the end of the day, Nginx is just another program that runs as a process in an operating system. And as such, it can be run in a container. One of the nice aspects about this setup is the convenience of having an entire system described in a set of files, and being able to bring everything up with a single command.

Using Docker Compose to Deploy a Multi-Application .NET System

2024-07-13T00:00:00+00:00

This post was co-authored by Juan Pablo Ventoso

We recently developed a system that involved several runtime components. It was an ecommerce site that included a database, a web API, an admin control panel web app, and a frontend SPA.

There are many ways to deploy such a system. For us, we wanted the infrastructure to be easily replicable for multiple environments with slightly different configurations. We wanted to be able to have, for example, a production and a staging version that could be deployed easily, with minimal configuration changes. We also wanted the infrastructure to be captured in files and version controlled, to further help replicability and maintainability.

With all that in mind, Docker Compose seemed like an ideal option. We could author a series of configuration files, parameterize environment-specific changes and, with a single command, we could spin up a whole environment to run the various applications within the system.

In this blog post, I’ll explain how we did that using a demo .NET code base that has a similar set of components. Let’s get started.

Getting familiar with the demo project

In .NET terms, our demo code base is organized as a solution with multiple projects. Two of those projects are ASP.NET web applications: a Razor Pages web app (the admin portal), and an MVC Web API. The rest are class libraries that define the core domain logic, tests, and other utilities. For deployment purposes, the web application projects are the interesting ones, as they produce executables that actually need to run as processes in the server. So, including the database, our demo system has three runtime components:

The database.
The admin portal.
The web API.

In the world of Docker, that would translate into three separate containers. Considering Docker Compose, that means three separate services.

Throughout this post, we will, step by step, build a compose.yaml, a set of Dockerfiles, and other configuration files which can be used with Docker Compose to deploy our system.

You can find the system’s source code on GitHub. The final version of the deployment files we’ll build in this article are also on GitHub.

Including the code base repository as a Git submodule

Considering the separation of our components, the reasons for the organization of our deployment configuration files and our code base starts to become apparent. The deployment files will live in their own repository. They do need access to the system’s source code, though, in order to build and run the apps. And that source code lives in its own repo. So, the deployment repository will include a source subdirectory, which will be a Git submodule that points to the repository where the system’s source code is stored.

Here’s the file structure that we’re aiming for:

.
├── compose.yaml
├── the various dockerfiles...
├── source
│   ├── vehicle-quotes.sln
│   └── the varous projects...
└── any other files and directories...

Including a Git repository inside another Git repository as a submodule is easy. In our case, we already have the parent Git repo, which is the one where the deployment config files live. To add the source code repo to it, we run a command like this:

git submodule add git@github.com:megakevin/end-point-blog-dotnet-8-demo.git source

Pretty straightforward. The command is calling git submodule add and passing it the location of the repo on GitHub and the directory in which to clone it. That’s it. As a result of that command, Git will have created a new source directory with the contents of the end-point-blog-dotnet-8-demo repository, and a .gitmodules file with contents like these:

[submodule "source"]
  path = source
  url = git@github.com:megakevin/end-point-blog-dotnet-8-demo.git

As you can see, this file contains the information Git needs to know that this repo has a submodule, where it is, and where it comes from.

Deploying the database

Let’s begin actually building out our system components with the database. Our apps use a PostgreSQL database to store information. Luckily for us, getting a PostgreSQL database up and running with Docker Compose is easy. All it takes is a compose.yaml file like this:

# ./compose.yaml

services:
  # This is the name of the service: "db". Other containers will use that as a
  # hostname when they need to interact with it.
  db:
    # The container will be based in the "16.3-bookworm" release of the public
    # PostgreSQL image that's available in Docker Hub
    image: postgres:16.3-bookworm
    # Always restart the container automatically if it ever closes for whatever
    # reason.
    restart: always
    # Makes the database accessible at port 5432.
    ports:
      - 5432:5432
    # Stores the PostgreSQL data files in a docker volume so that they are not
    # lost when the container stops or restarts.
    volumes:
      - db-postgres-data:/var/lib/postgresql/data
    # Sets some basic configuration for the initial database in the PostgreSQL
    # instance: its name, and a pair of credentials to access it.
    environment:
      - POSTGRES_DB=vehicle_quotes
      - POSTGRES_USER=vehicle_quotes
      - POSTGRES_PASSWORD=password

# Defines the Docker volume that the db service uses to store its data files.
volumes:
  db-postgres-data:

This is a very standard definition of a Docker Compose service. We called it db, made it available via the standard PostgreSQL port (5432), and set some basic configurations for it. Check out the PostgreSQL image’s page in Docker Hub to see more complicated use cases.

To bring it to life, we can run docker compose up -d in the directory where we’ve located the compose.yaml file. Running that command will prompt Docker to download the PostgreSQL image, build a container with it, and run it. It’ll also create the db-postgres-data volume we configured and a “network” that all the services we put in the compose.yaml file will be part of. Thanks to the -d option, it will run in daemon mode; that is, as a background process. So as soon as it’s done, it gives us control of our terminal back:

$ docker compose up -d

...

[+] Running 3/3
 ✔ Network end-point-blog-dotnet-docker-deploy_default            Created  0.0s
 ✔ Volume "end-point-blog-dotnet-docker-deploy_db-postgres-data"  Created  0.0s
 ✔ Container end-point-blog-dotnet-docker-deploy-db-1             Created  0.1s

Now that we have a database up and running, we have a couple of options for connecting to it. If we have the psql command line client installed in our machine, we can connect to it directly:

$ psql -h localhost -d vehicle_quotes -U vehicle_quotes -W
Password:
psql (16.3 (Ubuntu 16.3-0ubuntu0.24.04.1))
Type "help" for help.

vehicle_quotes=#

If not, we could first connect to the container, and then open psql from there:

$ docker compose exec db bash
root@edc038da3aa4:/# psql -h localhost -d vehicle_quotes -U vehicle_quotes -W
Password:
psql (16.3 (Debian 16.3-1.pgdg120+1))
Type "help" for help.

vehicle_quotes=#

Notice how we pass db and bash as parameters to the docker compose exec command. db is the name of our service, and bash is the command that we wish to execute within it. In this case we just want to open a shell, so we use bash.

Nice. That’s all it takes to set up the database. Now on to the applications.

Deploying the admin portal web app

Like I mentioned before, our code base contains an admin portal web application. In order to deploy it, we need first to define an image, which we’ll do with a Dockerfile, and then add the configuration to run a container based on that image using Docker Compose.

The image is a self-contained package that includes all the software that the application needs to run. It’s like an executable program. One that can be run by Docker, instead of directly by the operating system. In order to build images, we use Dockerfiles. For this ASP.NET app, the Dockerfile will perform two main tasks: Specify how to build the app and how to run it. Here’s what a Dockerfile for the admin portal project could look like:

# ./Dockerfile.AdminPortal

# Pull the .NET SDK image from Microsoft's repository. We will use it to build
# our app. We also give it a name of "build" so that we can reference it later.
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

# Now we copy all *.csproj files from the code base into the image and run
# dotnet restore. This will download and install all the NuGet packages that the
# various projects require.
WORKDIR /source

COPY source/vehicle-quotes.sln .
COPY source/VehicleQuotes.AdminPortal/VehicleQuotes.AdminPortal.csproj ./VehicleQuotes.AdminPortal/
COPY source/VehicleQuotes.Core/VehicleQuotes.Core.csproj ./VehicleQuotes.Core/
COPY source/VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj ./VehicleQuotes.CreateUser/
COPY source/VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj ./VehicleQuotes.IntegrationTests/
COPY source/VehicleQuotes.RazorTemplates/VehicleQuotes.RazorTemplates.csproj ./VehicleQuotes.RazorTemplates/
COPY source/VehicleQuotes.UnitTests/VehicleQuotes.UnitTests.csproj ./VehicleQuotes.UnitTests/
COPY source/VehicleQuotes.WebApi/VehicleQuotes.WebApi.csproj ./VehicleQuotes.WebApi/
RUN dotnet restore

# Next we copy all of the source code for all of the projects.
COPY source/VehicleQuotes.AdminPortal/. ./VehicleQuotes.AdminPortal/
COPY source/VehicleQuotes.Core/. ./VehicleQuotes.Core/
COPY source/VehicleQuotes.CreateUser/. ./VehicleQuotes.CreateUser/
COPY source/VehicleQuotes.IntegrationTests/. ./VehicleQuotes.IntegrationTests/
COPY source/VehicleQuotes.RazorTemplates/. ./VehicleQuotes.RazorTemplates/
COPY source/VehicleQuotes.UnitTests/. ./VehicleQuotes.UnitTests/
COPY source/VehicleQuotes.WebApi/. ./VehicleQuotes.WebApi/

# Now that we have everything in place, we use the dotnet publish command to
# build the VehicleQuotes.AdminPortal project.
WORKDIR /source/VehicleQuotes.AdminPortal
RUN dotnet publish -c release -o /app --no-restore

# In this last step, notice how we pull a different image here, this is the
# image that Microsoft recommends to use for running ASP.NET apps.
#
# Since by this point we already built the app, and have the executable binary
# for it, we don't need the full SDK anymore. dotnet/aspnet is a lighter weight
# image designed exclusively for running apps. In other words, it has the .NET
# runtime redistributable, not the full SDK.
#
# So here, all we do is copy the build assets from the "build" image into this
# new runtime-only one and put them in a /app directory. Then we call the .NET
# CLI to run the web app's DDL.
FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=build /app ./
ENTRYPOINT ["dotnet", "VehicleQuotes.AdminPortal.dll"]

With a Dockerfile like this, we could run the app as a stand-alone container. However, we don’t want to run it like that. Instead, we want it to be a service that’s deployable via Docker Compose, as part of a bigger ecosystem. In order to do that, we add the following to our compose.yaml file.

# ./compose.yaml

services:
  # The service is called "admin-portal" because that's the application that it
  # contains.
  admin-portal:
    # This is slightly more complex than the "db" service. Instead of using the
    # "image" option to download an off-the-shelf PostgreSQL image from
    # dockerhub, we use "build" and point to the Dockerfile we created in the
    # previous step. We also specify the current directory (i.e. the ".") as the
    # context in which the build will be performed.
    build:
      context: .
      dockerfile: Dockerfile.AdminPortal
    # We also configure this service to restart automatically if for whatever
    # reason it stops.
    restart: always
    # "8080" is the default port that ASP.NET Core 8 web applications listen to.
    # So here, we set it up so that all requests coming to the host machine to
    # port 8001 get sent to this container's port 8080. That way, reaching the
    # ASP.NET app that's running within.
    ports:
      - 8001:8080
    # This part is particular to our app. The admin portal offers the capability
    # of uploading files. Here, we're defining a Docker volume to ensure that
    # the uploaded files persist across container restarts. In this case, we are
    # linking the host machine's ./uploads directory to the container's
    # /app/wwwroot/uploads directory, which is where the uploaded files are
    # saved.
    volumes:
      - ./uploads:/app/wwwroot/uploads
    # We can use this section so set any environment variables that our app may
    # need. In the case of our admin portal here, we set the running environment
    # to Development, specify a database connection string, and the location to
    # save the uploaded files.
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ConnectionStrings__VehicleQuotesContext=Host=db;Database=vehicle_quotes;Username=vehicle_quotes;Password=password
      - QuoteImagesPath=/app/wwwroot/uploads
    # Finally, "entrypoint" specifies the command that should be used when
    # running the container. In this case, this is essentially the same as the
    # last line of the Dockerfile we saw above.
    entrypoint: ["sh", "-c", "dotnet VehicleQuotes.AdminPortal.dll"]

# ...

With this, we’re ready to see the application running. Run docker compose up -d again to update the infrastructure and include the new service:

$ docker compose up -d

...

[+] Running 2/2
 ✔ Container end-point-blog-dotnet-docker-deploy-db-1            Running   0.0s
 ✔ Container end-point-blog-dotnet-docker-deploy-admin-portal-1  Started   0.3s

After a while of downloading and building, navigating to http://localhost:8001 should show this:

Adding a maintenance container

At this point, we have a problem though. If we try clicking on the “Quotes” link in the top navigation bar, we see this:

We need to run the database migrations. In order to do that, we need an environment that has all of our source code and the full .NET SDK with the Entity Framework Core command line tool. That is, an environment that can build the app and run the .NET CLI. While we could install that in the host machine and run our migrations that way; we could also encapsulate it in a container. That way we get the benefits of portability, etc.

Of course, that maintenance container needs an image. And images are defined in Dockerfiles. Here’s a Dockerfile that would serve our purpose:

# ./Dockerfile.Maintenance

# Same as before, we pull the official image that contains the full .NET SDK.
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

# Here we install the psql command line client. A useful tool to have in a
# container meant for system maintenance.
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
    && apt-get -y install --no-install-recommends postgresql-client

# We also install the dotnet-ef tool which allows us access to commands for
# managing the database and migrations.
RUN dotnet tool install dotnet-ef --global
ENV PATH="$PATH:/root/.dotnet/tools"

# Finally, we copy all of our source code.
WORKDIR /source

COPY source/vehicle-quotes.sln .
COPY source/VehicleQuotes.AdminPortal/VehicleQuotes.AdminPortal.csproj ./VehicleQuotes.AdminPortal/
COPY source/VehicleQuotes.Core/VehicleQuotes.Core.csproj ./VehicleQuotes.Core/
COPY source/VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj ./VehicleQuotes.CreateUser/
COPY source/VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj ./VehicleQuotes.IntegrationTests/
COPY source/VehicleQuotes.RazorTemplates/VehicleQuotes.RazorTemplates.csproj ./VehicleQuotes.RazorTemplates/
COPY source/VehicleQuotes.UnitTests/VehicleQuotes.UnitTests.csproj ./VehicleQuotes.UnitTests/
COPY source/VehicleQuotes.WebApi/VehicleQuotes.WebApi.csproj ./VehicleQuotes.WebApi/
RUN dotnet restore

COPY source/VehicleQuotes.AdminPortal/. ./VehicleQuotes.AdminPortal/
COPY source/VehicleQuotes.Core/. ./VehicleQuotes.Core/
COPY source/VehicleQuotes.CreateUser/. ./VehicleQuotes.CreateUser/
COPY source/VehicleQuotes.IntegrationTests/. ./VehicleQuotes.IntegrationTests/
COPY source/VehicleQuotes.RazorTemplates/. ./VehicleQuotes.RazorTemplates/
COPY source/VehicleQuotes.UnitTests/. ./VehicleQuotes.UnitTests/
COPY source/VehicleQuotes.WebApi/. ./VehicleQuotes.WebApi/

# No ENTRYPOINT here because there's no specific command to run.

This Dockerfile is very similar to the first portion of the one for the admin portal. It essentially creates a .NET development environment. Notice how it doesn’t include an ENTRYPOINT command. This is expected, as this container’s purpose is to allow us to connect to it to run any number of maintenance tasks on demand. There isn’t really any process that it needs to run upon start.

Now, we want to deploy the maintenance container when we do docker compose up -d. To that end, we add the following to the services section in our compose.yaml file:

# ./compose.yaml

services:

  # ...

  maintenance:
    # Similar to the admin-portal, we specify the context and Dockerfile to
    # build the image.
    build:
      context: .
      dockerfile: Dockerfile.Maintenance
    # The maintenance tasks will need to know where to find the database. So we
    # set the connection string environment variable.
    environment:
      - ConnectionStrings__VehicleQuotesContext=Host=db;Database=vehicle_quotes;Username=vehicle_quotes;Password=password
    # "sleep infinity" here makes sure the container doesn't close immediately
    # after starting up, which is the expected behavior when a container has no
    # start up command or has a command that just runs and ends. That is, as
    # opposed to a long running service, like a web app.
    command: sleep infinity

# ...

With these additions, running docker compose up -d again will result in the new maintenance container being created:

$ docker compose up -d

...

[+] Running 3/3
 ✔ Container end-point-blog-dotnet-docker-deploy-admin-portal-1  Running   0.0s
 ✔ Container end-point-blog-dotnet-docker-deploy-db-1            Running   0.0s
 ✔ Container end-point-blog-dotnet-docker-deploy-maintenance-1   Started   0.4s

Now we can finally connect to the brand new maintenance container:

$ docker compose exec maintenance bash
root@14613bcf1756:/source#

Then inspect the status of the migrations:

root@14613bcf1756:/source# dotnet ef migrations list -s ./VehicleQuotes.AdminPortal -p ./VehicleQuotes.Core
Build started...
Build succeeded.

...

20210625212939_AddLookupTables (Pending)
20210625224443_AddUniqueIndexesToLookupTables (Pending)
20210625232816_AddVehicleModelTables (Pending)
20210625234824_AddUniqueIndexesForVehicleModelTables (Pending)
20210627204444_AddQuoteRulesAndOverridesTables (Pending)
20210627213029_AddQuotesTable (Pending)
20210627230039_AddSeedDataForSizesAndBodyTypes (Pending)
20220530192346_FixDatetimeColumn (Pending)
20220605003253_AddIdentityTables (Pending)
20220609233914_AddUserApiKeysTable (Pending)
20240504211307_VariousNullabilityChanges (Pending)
20240606222539_AddQuoteImages (Pending)
root@14613bcf1756:/source#

And run them:

root@14613bcf1756:/source# dotnet ef database update -s ./VehicleQuotes.AdminPortal -p ./VehicleQuotes.Core
Build started...
Build succeeded.

...

Done.
root@14613bcf1756:/source#

With that done, we can now go to the browser again and bring up the http://localhost:8001/Quotes page. Which now looks like this:

An empty—but working—page for listing database records!

Deploying the web API

Now let’s look at how to deploy another of our system’s runtime components: the web API. The process of setting this up is nearly identical to the admin portal’s. After all, both are ASP.NET Core web applications. Really, the only difference is that one serves HTML and the other serves JSON. To the runtime, they are the same type of thing. And sure enough, here’s the web API’s Dockerfile:

# ./Dockerfile.WebApi

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

WORKDIR /source

# copy sln and csproj files and restore
COPY source/vehicle-quotes.sln .
COPY source/VehicleQuotes.AdminPortal/VehicleQuotes.AdminPortal.csproj ./VehicleQuotes.AdminPortal/
COPY source/VehicleQuotes.Core/VehicleQuotes.Core.csproj ./VehicleQuotes.Core/
COPY source/VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj ./VehicleQuotes.CreateUser/
COPY source/VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj ./VehicleQuotes.IntegrationTests/
COPY source/VehicleQuotes.RazorTemplates/VehicleQuotes.RazorTemplates.csproj ./VehicleQuotes.RazorTemplates/
COPY source/VehicleQuotes.UnitTests/VehicleQuotes.UnitTests.csproj ./VehicleQuotes.UnitTests/
COPY source/VehicleQuotes.WebApi/VehicleQuotes.WebApi.csproj ./VehicleQuotes.WebApi/
RUN dotnet restore

# copy everything else
COPY source/VehicleQuotes.AdminPortal/. ./VehicleQuotes.AdminPortal/
COPY source/VehicleQuotes.Core/. ./VehicleQuotes.Core/
COPY source/VehicleQuotes.CreateUser/. ./VehicleQuotes.CreateUser/
COPY source/VehicleQuotes.IntegrationTests/. ./VehicleQuotes.IntegrationTests/
COPY source/VehicleQuotes.RazorTemplates/. ./VehicleQuotes.RazorTemplates/
COPY source/VehicleQuotes.UnitTests/. ./VehicleQuotes.UnitTests/
COPY source/VehicleQuotes.WebApi/. ./VehicleQuotes.WebApi/

# build app
WORKDIR /source/VehicleQuotes.WebApi
RUN dotnet publish -c release -o /app --no-restore

# final image
FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=build /app ./
ENTRYPOINT ["dotnet", "VehicleQuotes.WebApi.dll"]

Very similar to the admin portal’s Dockerfile. The only differences are the sections for building and running the app towards the end of the file, on account of the directories and resulting DDL name being different. Other than that, they are pretty much identical.

Same story on the Docker Compose side of things:

# ./compose.yaml

services:

  #...

  web-api:
    build:
      context: .
      dockerfile: Dockerfile.WebApi
    restart: always
    # We want to expose this service in a different port than the admin portal,
    # we chose 8002.
    ports:
      - 8002:8080
    # A different set of environment variables, but the structure is the same.
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      - ConnectionStrings__VehicleQuotesContext=Host=db;Database=vehicle_quotes;Username=vehicle_quotes;Password=password
      - MailSettings__Server=SMTP_SERVER_URL
      - MailSettings__Port=2525
      - MailSettings__SenderName=Vehicle Quotes
      - MailSettings__SenderEmail=info@vehiclequotes.com
      - MailSettings__UserName=SMTP_SERVER_USER_NAME
      - MailSettings__Password=SMTP_SERVER_PASSWORD
    # The entrypoint command is different because the DLL has a different file
    # name.
    entrypoint: ["sh", "-c", "dotnet VehicleQuotes.WebApi.dll"]

#...

It has the same overall setup as the admin portal. This time we’ve chosen a different port, configured a different set of environment variables, and run the web API’s DLL. Other than that, it’s the same. Now docker compose up -d can be run again, and after a while you should see:

$ docker compose up -d

...

[+] Running 4/4
 ✔ Container end-point-blog-dotnet-docker-deploy-web-api-1       Started   0.4s
 ✔ Container end-point-blog-dotnet-docker-deploy-maintenance-1   Running   0.0s
 ✔ Container end-point-blog-dotnet-docker-deploy-db-1            Running   0.0s
 ✔ Container end-point-blog-dotnet-docker-deploy-admin-portal-1  Running   0.0s

The web API has a Swagger UI that should now be accessible at http://localhost:8002/swagger:

Excellent. We now have a very basic implementation of our deployment strategy. We have managed to build and run all the apps we need. We also have a special container with a full development environment that we can use to perform a number of maintenance tasks. However, there are various issues that we still need to address. Let’s do that next.

Looking at the logs

Before that, though, let’s look at the logs being produced by the applications that have been deployed. You can see the logs for the entire system with:

docker compose logs -f

That command produces a single log stream with messages from all the services. To look at the logs of a particular service, all we need to do is specify the service name. For example docker compose logs db -f or docker compose logs web-api -f.

That’s all there is to it as far as logs go. However, if we look at the logs for our current deployment, they reveal the first issue that we need to attend to…

Persisting data protection keys

If we look at the logs of the admin portal or web API containers, we see messages like this:

web-api-1       | warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
web-api-1       |       Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning

As it turns out, the data protection subsystem in ASP.NET Core (which is used for cookies and the like) depends on the app being able to store and persist files. As the message says, this is problematic for containers. Because upon container destruction, which is a usual and expected part of a container’s lifecycle, the internal container’s file system gets wiped out. That is, unless we use Docker volumes to store the files that should persist beyond a particular container’s life. So, in order to get rid of these warnings, that’s exactly what we’re going to have to do.

First we create a data-protection-keys directory in our host machine. And inside it, we create one directory for each of the ASP.NET Core apps. It ends up looking like this:

data-protection-keys/
├── admin-portal
└── web-api

Next we have to make sure the apps in the containers have access to these directories. In order to do that, we need to configure the compose.yaml so that it creates new Docker volumes that are linked to those locations:

# ./compose.yaml

services:
  admin-portal:
    #...
    volumes:
      - ./data-protection-keys/admin-portal:/data-protection-keys/admin-portal
    #...

  web-api:
    #...
    volumes:
      - ./data-protection-keys/web-api:/data-protection-keys/web-api
    #...
#...

This means that, for example, whenever an app in the container references /data-protection-keys/admin-portal; it will actually be accessing the ./data-protection-keys/admin-portal directory in the host machine. Same deal for the web-api one.

We also need to configure the applications themselves so that they store their data protection keys in the locations that we’ve created. For that, we add environment variables for the containers to set the paths:

# ./compose.yaml

services:
  admin-portal:
    #...
    environment:
      - AdminPortalDataProtectionKeysPath=/data-protection-keys/admin-portal
    #...

  web-api:
    #...
    environment:
      - WebApiDataProtectionKeysPath=/data-protection-keys/web-api
    #...
#...

And add the following code to each of the apps’ Program.cs files:

// ./source/VehicleQuotes.AdminPortal/Program.cs

builder.Services.AddDataProtection().PersistKeysToFileSystem(
    new DirectoryInfo(
        builder.Configuration["AdminPortalDataProtectionKeysPath"] ??
            throw new InvalidOperationException("Config setting 'AdminPortalDataProtectionKeysPath' not found.")
    )
);

// ./source/VehicleQuotes.WebApi/Program.cs

builder.Services.AddDataProtection().PersistKeysToFileSystem(
    new DirectoryInfo(
        builder.Configuration["WebApiDataProtectionKeysPath"] ??
            throw new InvalidOperationException("Config setting 'WebApiDataProtectionKeysPath' not found.")
    )
);

There isn’t much to comment about the code really. It’s just some .NET boilerplate to configure that specific detail of the data protection services. It also makes sure that the values are always present. Errors are raised if not.

Now that everything is wired up like that, we can hit docker compose up -d --build and the warning message should be gone if we look at the logs.

Notice how we used the --build option this time. That tells Docker that it needs to rebuild the images from scratch. Image rebuild means running through the Dockerfiles again. And that means running dotnet build again. In short, we have to do this to make sure that the code changes that we made are included in the new builds.

Storing sensitive information with secrets

Another issue that we need to address is how to handle sensitive information like passwords in our config files. So far we’ve been putting them in plain text in compose.yaml. The problem with this is that this file is meant to be pushed to version control and we don’t want passwords in there. The spread of sensitive information like that should be more controlled. Ideally, we’d store them in files that never leave the server where the system is deployed.

Docker Compose has a feature that works just like that. Through Docker Compose secrets, we can create text files outside of the compose.yaml and put the database password and connection string in them. These files will live only in the server, never uploaded to version control.

Let’s create a new secrets directory and create these two files within it:

./secrets/vehicle-quotes-db-connection-string.txt:

Host=db;Database=vehicle_quotes;Username=vehicle_quotes;Password=password

./secrets/vehicle-quotes-db-password.txt:
```
password
```

Ending up looking like this:

secrets/
├── vehicle-quotes-db-connection-string.txt
└── vehicle-quotes-db-password.txt

Now, at the bottom of compose.yaml, we add a new secrets section:

# ./compose.yaml

#...

secrets:
  vehicle-quotes-db-password:
    file: ./secrets/vehicle-quotes-db-password.txt
  vehicle-quotes-db-connection-string:
    file: ./secrets/vehicle-quotes-db-connection-string.txt

Then we include the secrets in the services that need them:

# ./compose.yaml

services:
  admin-portal:
    #...
    secrets:
      - vehicle-quotes-db-connection-string
    #...

  web-api:
    #...
    secrets:
      - vehicle-quotes-db-connection-string
    #...

  db:
    #...
    secrets:
      - vehicle-quotes-db-password
    #...

  maintenance:
    #...
    secrets:
      - vehicle-quotes-db-connection-string
    #...
#...

With this, Docker Compose will add files in the resulting containers under the /run/secrets/ directory with the contents of their referenced secrets. So, for example, in the case of the admin-portal container, a /run/secrets/vehicle-quotes-db-connection-string file will be created in the container’s internal file system, with the same contents as the ./secrets/vehicle-quotes-db-connection-string.txt file. Similar thing for the others.

Now that the secrets are materialized as files within the containers, let’s see how we put them to use.

For the admin-portal container, we injected the vehicle-quotes-db-connection-string secret into it. This file contains the database connection string. And we need to pass that to the running app via an environment variable. In order to do so, we can change the entrypoint command to this:

# ./compose.yaml

services:
  admin-portal:
  #...
  entrypoint: [
    "sh", "-c",
    "export ConnectionStrings__VehicleQuotesContext=$(cat /run/secrets/vehicle-quotes-db-connection-string) &&
    dotnet VehicleQuotes.AdminPortal.dll"
  ]

#...

We’ve modified the command to set the ConnectionStrings__VehicleQuotesContext to the contents of the /run/secrets/vehicle-quotes-db-connection-string file. The export command defines the environment variable for this particular command; and the $(cat ...) part returns the contents of the file.

We should also remove the ConnectionStrings__VehicleQuotesContext variable from the admin-portal service’s environment section.

For the web-api service, we do the same thing. The only difference is that its entrypoint command calls for VehicleQuotes.WebApi.dll instead of VehicleQuotes.AdminPortal.dll.

For the db service, the setup is a little different. A little simpler in fact. The official PostgreSQL image has a shortcut for specifying the database user password via Docker Compose secrets. All we need to do is define this new POSTGRES_PASSWORD_FILE environment variable and remove the POSTGRES_PASSWORD one:

# ./compose.yaml

services:
  #...

  db:
  #...
  environment:
    - POSTGRES_PASSWORD_FILE=/run/secrets/vehicle-quotes-db-password
    # - POSTGRES_PASSWORD=password     <-- remove this one
#...

Try docker compose up -d again and test the apps. Everything should still work well.

In the maintenance container, we also have to remove the ConnectionStrings__VehicleQuotesContext environment variable. That unfortunately means that the connection string will no longer be automatically available for us to run database related tasks. Just like other containers, it will be in a /run/secrets/vehicle-quotes-db-connection-string file. So, whenever we want to interact with the database, like when running migrations, we need to manually export the variable. Something like this:

$ docker compose exec maintenance bash
root@2732a06871c0:/source# export ConnectionStrings__VehicleQuotesContext=$(cat /run/secrets/vehicle-quotes-db-connection-string)
root@2732a06871c0:/source# dotnet ef migrations list -s ./VehicleQuotes.AdminPortal -p ./VehicleQuotes.Core
Build started...
Build succeeded.

...

Parameterizing `compose.yaml` to support multiple deployment environments

A common requirement when deploying applications is to be able to do so in multiple environments. There’s generally a “live” or “production” environment where the system runs and end users access it. There can also be others: staging, test, development, etc. Ideally, we’d use the same set of Docker and Compose files, with slight changes, in order to deploy variants of the system depending on the environment.

Some settings like ports, passwords, or SMTP credentials are the types of things that usually vary per environment. Luckily for us, Docker Compose supports .env files that can be used to parameterize certain aspects of the compose.yaml file.

We can extract the values that vary from our compose.yaml file, and put them in a separate .env file that looks like this:

ADMIN_PORTAL_PORT=8001
WEB_API_PORT=8002
DB_PORT=5432
POSTGRES_DB=vehicle_quotes
POSTGRES_USER=vehicle_quotes
MailSettings__Server=SMTP_SERVER_URL
MailSettings__Port=2525
MailSettings__SenderName=Vehicle Quotes
MailSettings__SenderEmail=info@vehiclequotes.com
MailSettings__UserName=SMTP_SERVER_USER_NAME
MailSettings__Password=SMTP_SERVER_PASSWORD

Here we have the ports that we want to expose the various services on, the database name and user name, and some email delivery settings. This file wont be pushed to version control, and each deployment will have its own version of it.

The compose.yaml file can reference the values defined in the .env file using the following syntax: ${VAR_NAME}. Here’s how we change our compose.yaml to take advantage of the settings defined in the .env file:

# ./compose.yaml

services:
  admin-portal:
    #...
    ports:
      - ${ADMIN_PORTAL_PORT}:8080
    #...

  web-api:
    #...
    ports:
      - ${WEB_API_PORT}:8080
    #...
    environment:
      #...
      - MailSettings__Server=${MAIL_SETTINGS_SERVER}
      - MailSettings__Port=${MAIL_SETTINGS_PORT}
      - MailSettings__SenderName=${MAIL_SETTINGS_SENDER_NAME}
      - MailSettings__SenderEmail=${MAIL_SETTINGS_SENDER_EMAIL}
      - MailSettings__UserName=${MAIL_SETTINGS_USER_NAME}
      - MailSettings__Password=${MAIL_SETTINGS_PASSWORD}
    #...

  db:
    #...
    ports:
      - ${DB_PORT}:5432
    #...
    environment:
      - POSTGRES_DB=${POSTGRES_DB}
      - POSTGRES_USER=${POSTGRES_USER}
      #...
    #...

#...

Once again, we can run docker compose up -d and everything should be fine.

Waiting for the database to be ready

Sometimes some services need to wait for others to come online before they can start up. A common scenario is to wait for the database to be ready before running apps that depend on it. We can do that in Docker Compose thanks to the depends_on and healthcheck settings. We can update our compose.yaml so that the admin portal and web API services only start after the database is up and running and ready to receive requests. Here’s how:

# ./compose.yaml

services:
  admin-portal:
    #...
    # This specifies that the admin-portal service depends on the db to be
    # healthy in order to start up. The db service's healthcheck setting is how
    # it determines whether it is healthy or not.
    depends_on:
      db:
        condition: service_healthy

  web-api:
    #...
    # Exact same setting as the one in admin-portal.
    depends_on:
      db:
        condition: service_healthy

  db:
    #...
    # Here we configure this service to offer a healthcheck that other services
    # can use to determine if it's ready to be depended upon. In this case, we
    # leverage PostgreSQL's pg_isready tool. It is called by the specified
    # timeout, at the specified interval, and with as many retries as specified
    # in the settings below.
    healthcheck:
      test: "pg_isready -U ${POSTGRES_USER}"
      interval: 10s
      timeout: 5s
      retries: 5

#...

The most interesting part is the healthcheck setting in the db service which leverages a PostgreSQL-specific tool to check whether the database is ready. Other software will have other methods to do checks like this, but PostgreSQL’s is thankfully pretty straightforward.

Serving the apps with NGINX

Another common pattern for serving web applications is to use NGINX as a reverse proxy that funnels HTTP traffic coming from the internet into the application. As you may have noticed, we haven’t talked about HTTPS so far. This aspect is something that can be elegantly handled by NGINX as well. In this last section we see how we can set up NGINX to expose our apps to the world.

First, we need to create some custom items in the default NGINX configuration file. Its default location will depend on the OS version and flavor you installed NGINX on, but for Rocky Linux 9, it usually lives in /etc/nginx/nginx.conf. First, we need to declare our upstream servers that will point to the ports where the web API and admin portals are listening:

upstream admin_portal {
    server localhost:8001;
}

upstream web_api {
    server localhost:8002;
}

Then, we will add a server that listens in the standard port 80 and proxies the /admin and /api URLs to the admin_portal and web_api upstream servers respectively:

server {
    listen 80;
    server_name vehiclequotes.com;

    location /admin {
        proxy_pass http://admin_portal;
        proxy_http_version 1.1;
        proxy_set_header Connection keep-alive;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /api {
        proxy_pass http://web_api;
        proxy_http_version 1.1;
        proxy_set_header Connection keep-alive;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Those are the basic settings needed to serve our applications in a single domain through different locations. This allows having a live frontend application that will eventually consume our API endpoints that lives in the same server and domain, avoiding possible Cross-Origin Resource Sharing (CORS) issues when establishing the workflow between the different web applications.

If we also have an SSL certificate that we want to use to securely serve our apps, we can use a free service such as Let’s Encrypt to create it. Once we’re in possession of the certificate files and placed them on the server, we need to perform a few extra tweaks to our nginx.conf file.

First, let’s make our server listen on port 443 (HTTPS), and point to our .cer and .key files in the updated entry:

server {
    listen 443 ssl;
    server_name vehiclequotes.com;
    ssl_certificate /etc/certs/live/vehiclequotes.com/fullchain.cer;
    ssl_certificate_key /etc/certs/live/vehiclequotes.com/vehiclequotes.com.key;

    location /admin {
        proxy_pass http://admin_portal;
        proxy_http_version 1.1;
        proxy_set_header Connection keep-alive;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location /api {
        proxy_pass http://web_api;
        proxy_http_version 1.1;
        proxy_set_header Connection keep-alive;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Second, let’s add a new server for port 80 (non-HTTPS) that will redirect permanently to our new secure location:

server {
    listen 80;
    server_name vehiclequotes.com;
    return 301 https://$server_name$request_uri;
}

Finally, let’s restart the NGINX service to apply our changes. The command that restarts the service will vary depending on the OS we’re running on the server. For Rocky Linux 9, we can do that by running the command sudo systemctl restart nginx.

You can find the resulting nginx.conf file in the project’s repo in GitHub.

That’s all for now

And that’s it! In this article, we’ve seen how we can approach deploying a .NET system into production using Docker Compose.

We saw how to organize the code and configuration files using Git submodules. We addressed a few important edge cases and gotchas like properly configuring Data Protection keys, having a container for performing maintenance tasks, ensuring certain files persist across restarts, and bringing up services in a certain order via depends_on settings.

We even saw how to allow our web applications to be accessible to the outside world with NGINX through a set of reverse proxying rules, and as a bonus, to be securely served with SSL.

Uploading multiple files in a single request in an ASP.NET Core application

2024-07-04T00:00:00+00:00

We recently developed a web application for maintaining an ecommerce site’s product catalog. Unsurprisingly, one of the features involved the management of product images. Specifically, we wanted to create a page where all the images of a given product were displayed and new ones could be uploaded.

In addition to that, this wasn’t a single-page application. We weren’t using a JavaScript framework and instead were relying on regular server-rendered views with ASP.NET Razor Pages. Even so, we wanted to create a user experience with a good balance of usability and development complexity. So, we decided to create the capability of uploading several image files at once within a single request — that is, within a single HTML form submission.

In this article, I’m going to describe the solution that we came up with in order to make this happen.

To demonstrate the approach, I’ll use a sample ASP.NET Core solution that I’ve been building out throughout several blog posts. Its main feature is calculating the value of used cars and offering quotes for them. The system stores all generated quotes as database records. Given this context, we’ll add the functionality to upload image files for a given quote.

Let’s get started.

Each section is accompanied by a given commit that includes all the changes discussed within them. Feel free to check those as well to see how the finished product takes shape step by step.

Where to store the images

Commit: d65fd8.

The first decision that we have to make is where to store the image files. There are various options afforded to us by ASP.NET and our underlying PostgreSQL database, but here’s how we’re going to do it: we’ll store records in the database that represent the images associated with each quote. These records won’t actually contain image binary data, however, only references to them in the form of their file names. The files themselves will be stored in the project’s wwwroot directory. That way they can be easily served to browsers, as everything inside that directory is accessible to clients.

Starting off with a fresh ASP.NET Core Web App (with Razor Pages) project, here’s how that’s done.

First we have to make sure that the app has static file serving enabled. This is done with this function call in the Program.cs file:

// VehicleQuotes.AdminPortal/Program.cs

app.UseStaticFiles();

Next, we need to create the new database structure that will store the image records. In our app, we already have a Quote entity, backed by a corresponding quotes table in the database. The plan is to create a new QuoteImage entity and update Quote to “have many” of them.

Here’s what the new class looks like:

// VehicleQuotes.WebApi/Models/QuoteImage.cs

namespace VehicleQuotes.WebApi.Models;

public class QuoteImage
{
    public int ID { get; set; }
    public string FileName { get; set; } = default!;

    public int QuoteId { get; set; }
    public Quote Quote { get; set; } = default!;
}

We also have to add this navigation property to the Quote entity’s class:

// VehicleQuotes.WebApi/Models/Quote.cs

public class Quote
{
    // ...
    public ICollection<QuoteImage> QuoteImages { get; set; } = [];
}

And finally declare its corresponding DbSet in our VehicleQuotesContext class:

// VehicleQuotes.WebApi/Data/VehicleQuotesContext.cs

public class VehicleQuotesContext : IdentityUserContext<IdentityUser>
{
    // ...
    public DbSet<QuoteImage> QuoteImages { get; set; }
    // ...
}

Now all that’s left is to create and run the migration with these commands:

dotnet ef migrations add AddQuoteImages
dotnet ef database update

With that, our model and database are ready to store references to image files for the quotes on record.

vehicle_quotes=# \d quote_images;
                          Table "public.quote_images"
  Column   |  Type   | Collation | Nullable |             Default
-----------+---------+-----------+----------+----------------------------------
 id        | integer |           | not null | generated by default as identity
 file_name | text    |           | not null |
 quote_id  | integer |           | not null |
Indexes:
    "pk_quote_images" PRIMARY KEY, btree (id)
    "ix_quote_images_quote_id" btree (quote_id)
Foreign-key constraints:
    "fk_quote_images_quotes_quote_id" FOREIGN KEY (quote_id) REFERENCES quotes(id) ON DELETE CASCADE

Check out this commit to see the migration.

A Razor Page for uploading many files

Commit: feaa57.

Now we need to create a new Razor Page for uploading the image files. The new page will be reachable at this route: quotes/edit/{id}. {id} is the identifier of the quote record for which images will be uploaded. To that end, we’ll create the usual pair of files that make up a Razor Page: The PageModel at VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.cs and the View (i.e. template) at VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.

We’ll start with the PageModel. Let’s create the new file with these contents:

// VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.cs

using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.RazorPages;
using Microsoft.EntityFrameworkCore;
using VehicleQuotes.WebApi;
using VehicleQuotes.WebApi.Models;

namespace VehicleQuotes.AdminPortal.Pages.Quotes;

public class EditModel : PageModel
{
    // This page will interact with the database so we need an instance of the
    // DbContext.
    private readonly VehicleQuotesContext _context;
    private readonly string _imagesPath;

    public EditModel(VehicleQuotesContext context)
    {
        _context = context;
        // Like we discussed, we will store the image files in the app's wwwroot
        // directory. Specifically, ~/wwwroot/uploads. It'd be better to
        // substitute this hard coded string with a config value from
        // appsettings.json.
        _imagesPath = "/path/to/wwwroot/uploads";
    }

    // The BindProperty attribute in this property makes it so that when a form
    // in this page is submitted, the framework takes the request payload and
    // uses it to populate this property. In our case, the form will submit a
    // set of files. We'll see how a little bit later.
    [BindProperty]
    public IEnumerable<IFormFile>? ImageFiles { get; set; }

    // This GET handler method does little else than render the corresponding
    // template when a browser requests the quotes/edit/{id} URL.
    // That is, the one defined in
    // VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.
    public IActionResult OnGet(int id) => Page();

    // This is the method that handles POST requests on this page. It inspects
    // the incoming request payload and uses it to produce new quote image
    // records as well as save the uploaded files under ~/wwwroot/uploads.
    public async Task<IActionResult> OnPostSaveAsync(int id)
    {
        // First, try to find the quote record by the given id and...
        var quote = await FindQuote(id);

        // ...if it's not found, return a proper 404 response.
        if (quote == null)
        {
            return NotFound();
        }

        // If images were uploaded...
        if (ImageFiles is not null)
        {
            // ...iterate over each one of them...
            foreach (var imageFile in ImageFiles)
            {
                // ...store them in the file system...
                var imageFileName = await SaveImageFile(imageFile);
                // ...and add a new corresponding record to the quote's related
                // image records.
                quote.QuoteImages.Add(new() { FileName = imageFileName });
            }
        }

        // Finally, submit the database changes...
        await _context.SaveChangesAsync();

        // ...and effectively reload the page.
        return RedirectToPage("./Edit", new { Id = id });
    }

    // This method queries the database to find quote records by id.
    private async Task<Quote?> FindQuote(int id) =>
        await _context.Quotes
            .FirstOrDefaultAsync(m => m.ID == id);

    // This method has some mostly straightforward code to save the files from
    // the incoming HTTP request into the file system.
    private async Task<string> SaveImageFile(IFormFile fileToSave)
    {
        var extension = Path.GetExtension(fileToSave.FileName).ToLowerInvariant();
        // The only interesting thing that we have to keep in mind is that we
        // need to come up with a naming scheme that won't produce conflicts.
        // Here, we've taken the naive approach of just using
        // Path.GetRandomFileName to create randomized names.
        var fileName = $"{Path.GetRandomFileName()}{extension}";
        var filePath = Path.Combine(_imagesPath, fileName);

        using var stream = System.IO.File.Create(filePath);
        await fileToSave.CopyToAsync(stream);

        return fileName;
    }
}

Next we need to create the View. The job of this View is to define an HTML form that supports uploading many files. We have two options for this. The first option involves creating a form with a single file input element that uses the multiple attribute. That will allow the user to pick many files at the same time using their operating system’s file picker dialog box, looking something like this:

Option number two involves defining a form with multiple file input elements with each accepting one file. The number of <input> elements cannot be static however. We want to allow users to upload as many images as they want. So, we’d also need some JavaScript to dynamically add new <input> elements as the user picks more and more files.

Both options are valid, and deciding on one over the other will depend on the user experience you’re interested in creating and the amount of effort you can devote to it. For us, we went with option #2: A dynamic set of individual file input elements.

Before making it fully dynamic, though, let’s go through the exercise of building it with a static number of <input> elements as a first step. I think that’ll help clarify the process better.

So here’s the cshtml file with three <input type="file"> elements:

<!-- VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml -->

@page "{id}"
@model VehicleQuotes.AdminPortal.Pages.Quotes.EditModel

@{
    ViewData["Title"] = "Details";
}

<h1>Edit</h1>

<h4>Add Images</h4>
<hr />

<div class="row">
    <!--
    The form doesn't need too many details. We do have to make sure to set its
    enctype attribute to multipart/form-data to allow it to include files.
    -->
    <form method="post" enctype="multipart/form-data">
        <div id="quote-images-container" class="row">
            <div class="col-3 mb-3">
                <!--
                These are regular old HTML file input elements. They all have
                the same name, which matches that of the property in the
                PageModel that we want the framework to populate.
                That is, "ImageFiles".
                -->
                <input
                    type="file" accept="image/*"
                    id="ImageFiles_0" name="ImageFiles" class="form-control"
                >
            </div>

            <div class="col-3 mb-3">
                <input
                    type="file" accept="image/*"
                    id="ImageFiles_1" name="ImageFiles" class="form-control"
                >
            </div>

            <div class="col-3 mb-3">
                <input
                    type="file" accept="image/*"
                    id="ImageFiles_2" name="ImageFiles" class="form-control"
                >
            </div>
        </div>

        <div class="form-group mb-3">
            <!-- This button submits the form. -->
            <button
                type="submit"
                asp-page-handler="Save"
                class="btn btn-primary"
            >
                Save
            </button>
        </div>
    </form>
</div>

<div class="row">
    <a asp-page="./Index">Back to List</a>
</div>

This is how we design our form so that when submitted, ASP.NET knows that it has to populate ImageFiles (the collection of IFormFiles that we have defined in the PageModel) with the uploaded files. All we have to do is make sure that all the input elements share the same name attribute, and that said name is the same as the target property. ImageFiles in this case. The elements’ id attribute is actually superfluous.

If you’re following along and want to run this next part on your environment, you’ll need a quote record to be able to bring it up in the page we’ve been building and add images to it. You could do so by running the VehicleQuotes.WebApi project with dotnet run and POST to it using curl:
curl --location 'http://localhost:8001/api/Quotes' \
--header 'Content-Type: application/json' \
--data '{
 "year": "2020",
 "make": "Mazda",
 "model": "Mazda 3",
 "bodyType": "Sedan",
 "size": "Compact",
 "itMoves": true,
 "hasAllWheels": true,
 "hasAlloyWheels": true,
 "hasAllTires": true,
 "hasKey": true,
 "hasTitle": true,
 "requiresPickup": true,
 "hasEngine": true,
 "hasTransmission": true,
 "hasCompleteInterior": true
}'
Then connect to the database however you like to and run the following query to learn its id:
select * from quotes;
Use that id in the URL to the page.

At this point, the feature should work. If you run the app with dotnet run, and navigate to the page we’ve been building at http://localhost:{YOUR_PORT}/Quotes/Edit/{QUOTE_ID}, you should see something like this:

Pick three image files, hit the blue “Save” button and the files should be uploaded into the wwwroot/uploads diretory…

$ ls wwwroot/uploads
sqyjb1ui.qnv.png  x1c4s4fj.rqb.png  zjaym20f.wcx.png

…and three new records should be created on the quote_images table:

vehicle_quotes=# select * from quote_images;
 id |    file_name     | quote_id
----+------------------+----------
  1 | sqyjb1ui.qnv.png |        1
  2 | zjaym20f.wcx.png |        1
  3 | x1c4s4fj.rqb.png |        1
(3 rows)

Uploading any number of files

Commit: 5aaff6.

Now that we’ve seen the general approach and key details of working with the framework to upload multiple files, it’s more clear what we need to do to support any number of files. Like I said, what we need is “some JavaScript to dynamically add new <input> elements as the user picks more and more files”.

Let’s start by removing the three file input elements (that is, the whole <div id="quote-images-container"> element) and replacing them with this:

<!-- VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml -->

<!-- ... -->

<div id="quote-images-container" class="row">
    <div class="col-3 mb-3">
        <input type="file" accept="image/*" id="new-quote-image-file" class="form-control" />
    </div>
</div>

<template id="quote-image-template">
    <div class="col-3 mb-3">
        <input type="file" accept="image/*" name="" id="" class="form-control" />
    </div>
</template>

We’ve reduced the three file input elements to just one and, most importantly, we’ve added a template that we’ll use to add new ones. If you’re not familiar, you can read more about templates in MDN. But the basic idea is that this element will not actually be visible in the page as it is. Instead, we will write some JavaScript that uses it to create new file inputs as the user fills out the currently present ones.

We will create a new JavaScript file that will contain that logic. Let’s first add a reference to it in the View:

<!-- VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml -->

<!-- ... -->

@section scripts {
    <script src="~/js/quotes.new-images.js" asp-append-version="true"></script>
}

And here’s what that JavaScript file looks like:

// VehicleQuotes.AdminPortal/wwwroot/js/quotes.new-images.js

window.addEventListener('DOMContentLoaded', _event => {
    // First of all we attach the "addNewFileInput" function to the change event
    // of the default file input element.
    const newFileInput = document.querySelector("#new-quote-image-file");
    newFileInput.addEventListener("change", addNewFileInput);

    // We use this counter to set the ids of the file input elements as they get
    // added.
    let imageInputCount = 0;

    // This function, which handles the file input's change event, updates the
    // input element that triggered the event so that it can be submitted with
    // the form and also creates a new empty file input ready to accept a new
    // file. See the comments on the "convertToSubmittable" and
    // "createNewFileInput" functions to learn how.
    function addNewFileInput(event) {
        convertToSubmittable(event.target)
        createNewFileInput();

        imageInputCount++;
    }

    // This function takes the file input element that the user interacted with
    // and changes its name to "ImageFiles". That way, when the form is
    // submitted, ASP.NET knows that it needs include the file from this
    // element when populating the "ImageFiles" property.
    // We also detach the change event handler, because once a file
    // is selected on it, we don't want new inputs being created if the user
    // changes the file. It is only the new empty file input that will be
    // responsible of creating another new empty one once it is filled.
    function convertToSubmittable(fileInput) {
        fileInput.setAttribute("name", "ImageFiles");
        fileInput.setAttribute("id", `ImageFiles_${imageInputCount}`);

        fileInput.removeEventListener("change", addNewFileInput);
    }

    // Here we leverage the template to create and attach a new empty file input
    // element. Importantly, we set the addNewFileInput to handle its change
    // event. That way, when the user picks a file to put in the newly created
    // file input, the whole process runs again to get the element ready for
    // submission and create a new empty one.
    function createNewFileInput() {
        const template = document.querySelector("#quote-image-template");
        const clone = template.content.cloneNode(true);

        let fileInput = clone.querySelector("input");
        fileInput.setAttribute("id", "new-quote-image-file");

        appendToContainer(clone);

        fileInput.addEventListener("change", addNewFileInput);
    }

    function appendToContainer(element) {
        const container = document.querySelector("#quote-images-container");
        container.appendChild(element);
    }
});

So the overall strategy is something like this: At page load, we start with an empty file input. Once a file is selected for it, the input gets updated so that it is ready for submission. Then a new empty file input is created which will serve the same purpose as the initial one. This creates a “cycle” where a single new empty file input is always created as a result of users picking files, allowing them to keep adding files one by one.

So the page starts looking like this:

If we pick a file, the page turns into this:

Pick another, and now it looks like this:

And so on.

Now users can pick as many files as they want, hit the “Save” button, and everything just works.

And with that, the initial promise of this blog post is fulfilled. However, there are a few more improvements and features that we can add.

Validating the uploaded images

Commit: 06b547.

Something obvious that’s missing is validation. Right now, our app allows users to upload anything they want. We can limit that a bit with a couple of validation attributes.

For example, here’s one that only allows files with the most common image extensions:

// VehicleQuotes.AdminPortal/Validation/AllFilesHaveImageFileExtensionAttribute.cs

using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.AdminPortal.Validation;

[AttributeUsage(AttributeTargets.Property)]
public class AllFilesHaveImageFileExtensionAttribute : ValidationAttribute
{
    // These are the file extensions that we accept.
    private static readonly string[] imageExtensions =
        [".png", ".jpg", ".jpeg", ".gif", ".bmp"];

    // This method is called by the framework when processing the request to
    // validate the incoming payload.
    protected override ValidationResult? IsValid(
        object? value, ValidationContext context
    ){
        if (value == null) return ValidationResult.Success;

        ThrowIfTypeIsNotSupported(value);

        if (!CheckIfIsValid(value))
            return new ValidationResult(GetErrorMessage());

        return ValidationResult.Success;
    }

    // In this method we make sure that the attribute is only applied to
    // properties of type IEnumerable<IFormFile>. It only works with those.
    private void ThrowIfTypeIsNotSupported(object value)
    {
        if (value is not IEnumerable<IFormFile>)
        {
            throw new ArgumentException($"{GetType().Name} only works with properties of type IEnumerable<IFormFile>.");
        }
    }

    // Here we check that all the files in the collection meet the criteria
    // defined in "HasImageExtension".
    private static bool CheckIfIsValid(object value)
    {
        var files = (IEnumerable<IFormFile>)value;
        return files.All(HasImageExtension);
    }

    // This is the core of our validation logic. This method checks that each
    // given file has one of the extensions from the "imageExtensions" array.
    private static bool HasImageExtension(IFormFile file)
    {
        var extension = Path.GetExtension(file.FileName).ToLowerInvariant();

        if (string.IsNullOrEmpty(extension) || !imageExtensions.Contains(extension))
        {
            return false;
        }

        return true;
    }

    // Here we produce an error message that's easily understandable for users.
    // Letting them know which extensions our app supports.
    private static string GetErrorMessage()
    {
        var allowedExtensions = string.Join(", ", imageExtensions);
        return $"Only the following file extensions are allowed: {allowedExtensions}.";
    }
}

The attribute can be applied to the ImageFiles property in the PageModel like this:

// VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.cs

[BindProperty]
[AllFilesHaveImageFileExtension] // <- Here's the new attribute.
public IEnumerable<IFormFile>? ImageFiles { get; set; }

And here’s another one that checks that none of the incoming files are empty:

// VehicleQuotes.AdminPortal/Validation/AllFilesAreNotEmptyAttribute.cs

using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.AdminPortal.Validation;

[AttributeUsage(AttributeTargets.Property)]
public class AllFilesAreNotEmptyAttribute : ValidationAttribute
{
    protected override ValidationResult? IsValid(object? value, ValidationContext context)
    {
        if (value == null) return ValidationResult.Success;

        ThrowIfTypeIsNotSupported(value);

        if (!CheckIfIsValid(value)) return new ValidationResult(GetErrorMessage());

        return ValidationResult.Success;
    }

    private void ThrowIfTypeIsNotSupported(object value)
    {
        if (value is not IEnumerable<IFormFile>)
        {
            throw new ArgumentException($"{GetType().Name} only works with properties of type IEnumerable<IFormFile>.");
        }
    }

    // This is the only part that's meaningfully different from the other
    // validation attribute we just wrote. Here, we simply check that the
    // uploaded file's length attribute is greater than zero. In other words:
    // we check that the file isn't empty.
    private static bool CheckIfIsValid(object value)
    {
        var files = (IEnumerable<IFormFile>)value;
        return files.All(file => file.Length > 0);
    }

    private static string GetErrorMessage() => "Some of the selected files appear to be empty.";
}

It can be applied similarly:

// VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.cs

[BindProperty]
[AllFilesHaveImageFileExtension]
[AllFilesAreNotEmpty] // <- Here it is.
public IEnumerable<IFormFile>? ImageFiles { get; set; }

The last step is to actually trigger the input validation when the request is received. This can be done with this update to the OnPostSaveAsync handler:

public async Task<IActionResult> OnPostSaveAsync(int id)
{
    var quote = await FindQuote(id);

    if (quote == null)
    {
        return NotFound();
    }

+    if (!ModelState.IsValid)
+    {
+        return Page();
+    }

    if (ImageFiles is not null)
    {
        foreach (var imageFile in ImageFiles)
        {
            var imageFileName = await SaveImageFile(imageFile);
            quote.QuoteImages.Add(new() { FileName = imageFileName });
        }
    }

    await _context.SaveChangesAsync();

    return RedirectToPage("./Edit", new { Id = id });
}

Oh! And don’t forget the necessary using statement near the top of the file:

using VehicleQuotes.AdminPortal.Validation;

We probably also want to add a validation summary in the View so that an error message is shown whenever the validation fails. We put it inside the <form> element:

<form method="post" enctype="multipart/form-data">
+    <div asp-validation-summary="All" class="text-danger"></div>

    <!-- ... -->
</form>

Attempting to upload invalid files produces something like this:

There are more complex validations that can be done. Check out ASP.NET’s official docs to learn more.

Displaying the images

Commit: d15d35

So far we’ve added support for uploading images. We should have a way of displaying the ones that are already in the system as well. Luckily this is quite a straightforward problem to solve. To do that, we need to update our PageModel’s GET request handler so that it fetches the corresponding quote record from the database. Also, we need to create a new property to store this quote, so that it is accessible in the View.

All in all, the PageModel has to be updated to look like this:

// VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.cs

// ...

public class EditModel : PageModel
{
    // ...

    // Here's the property we talked about. We need it so that the View can
    // access the quote record fetched in the handler.
    public Quote Quote { get; set; } = default!;

    // ...

    // public IActionResult OnGet(int id) => Page(); <- This gets removed.

    // This is our new GET request handler.
    public async Task<IActionResult> OnGetAsync(int id)
    {
        var quote = await FindQuote(id);

        // Return a 404 if the quote doesn't exist.
        if (quote == null)
        {
            return NotFound();
        }

        // Don't forget to store the quote in the property.
        Quote = quote;

        return Page();
    }

    // ...

    // We also need to update the query so that the image records are also
    // fetched, along with the particular quote we're interested in.
    private async Task<Quote?> FindQuote(int id) =>
        await _context.Quotes
            .Include(m => m.QuoteImages) // <- We need to add this.
            .FirstOrDefaultAsync(m => m.ID == id);
}

The View will need a way to calculate the images’ URLs. We could put that logic in the .cshtml template itself, but a simple property on QuoteImage would be a bit cleaner. Here’s what it would look like:

// VehicleQuotes.WebApi/Models/QuoteImage.cs

namespace VehicleQuotes.WebApi.Models;

public class QuoteImage
{
    // ...

    // Taking advantage of the fact that we know that the image files are stored
    // in the wwwroot/uploads directory, we can construct a URL that can be used
    // in the src attribute of <img> elements.
    public string Url => $"~/uploads/{FileName}";
}

Finally, in the View, we update the HTML template to iterate over the loaded quote’s QuoteImages and render them. Let’s add this code right above the “Add Images” header:

<!-- VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml -->

<h4>Existing Images</h4>
<hr />

<div class="row mb-4">
    @foreach (var image in Model.Quote.QuoteImages)
    {
        <div class="col-3">
            <img
                src="@image.Url"
                asp-append-version="true"
                alt="Quote image"
                class="mx-auto d-block w-75"
            />
        </div>
    }
</div>

The page should now look something like this:

Deleting the images

Commit: 2526f2.

The last piece of the puzzle that’s missing from our image file management screen is the ability to delete them. The Razor Pages way of doing this is with a new button that submits a form to a named page handler.

The form and button combo looks like this and can be added right below the <img> element that we added inside the foreach loop in the previous section:

<!-- VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml -->

<form method="post">
    <button
        type="submit"
        asp-page-handler="DeleteImage"
        asp-route-imageId="@image.ID"
        class="btn btn-outline-danger mx-auto d-block"
    >
        Delete
    </button>
</form>

The asp-page-handler attribute tells the framework which handler to call when this form gets submitted. We will write that handler shortly! The asp-route-imageId on the other hand, defines a parameter to pass to the handler.

Now for the handler, here’s the code that we need to add to our PageModel:

// VehicleQuotes.AdminPortal/Pages/Quotes/Edit.cshtml.cs

// This is the handler method. Its name matches the asp-page-handler attribute
// on the button that triggers the form submission. The id parameter comes from
// the URL route (i.e. quotes/edit/{id}). The imageId parameter comes from the
// asp-route-imageId attribute in the form's submit button.
public async Task<IActionResult> OnPostDeleteImageAsync(int id, int imageId)
{
    // Find the quote record by id...
    var quote = await FindQuote(id);

    // ...and return a 404 if no quote is found.
    if (quote == null)
    {
        return NotFound();
    }

    // Get the image referenced by imageId. That is, the one for which the
    // delete button was pressed.
    var imageToDelete = quote.QuoteImages.FirstOrDefault(i => i.ID == imageId);

    // Again, return a 404 if no such image record exists.
    if (imageToDelete == null)
    {
        return NotFound();
    }

    // If all goes well, we delete the image file...
    DeleteImageFile(imageToDelete.FileName);
    // ...and also delete the corresponding database record.
    _context.QuoteImages.Remove(imageToDelete);
    await _context.SaveChangesAsync();

    // And finally reload the page.
    return RedirectToPage("./Edit", new { Id = id });
}

// This method uses the typical .NET library features for deleting a file.
private void DeleteImageFile(string fileName)
{
    var filePath = Path.Combine(_imagesPath, fileName);
    System.IO.File.Delete(filePath);
}

With that, each image rendered in the page should now have a working “Delete” button that looks like this:

Alright! Now we have a fully functional image handling page where we can view, add and remove images associated with a particular entity in our system. The neat part is that users can select as many files as they want and upload them all at the same time within a single form submission. We did all this with Razor Pages, using framework features, and a little bit of JavaScript.

How to validate record uniqueness in ASP.NET

2024-05-28T00:00:00+00:00

In ASP.NET, the System.ComponentModel.DataAnnotations namespace includes many attributes that can be used to instruct the framework to perform basic validation tasks for us. There are built-in validators for ensuring that certain fields are present, that they meet character length limits, that they don’t exceed or fall short of certain amounts, that they match certain formats, and more.

One omission however, is checking for record uniqueness: making sure that no other record in the entity’s underlying persistent data storage has the same “name”, or the same “code”, or the same “any other field”.

In this article, we’re going to try to address this shortcoming by implementing this type of uniqueness validation ourselves. We will see two approaches: a simpler solution using a database index, and a more flexible one using a custom validation attribute. Let’s get started.

Throughout this article, I will be using a demo Web API application for code examples. If you’d like to see what the final implementation looks like, you can find all the code on GitHub.

The API is about calculating quotes for used vehicles based on their condition. It runs on .NET 8 and uses a PostgreSQL database, to which it connects using Entity Framework Core. As such it has various endpoints, entities and tables related to vehicle information like makes, models, etc.

Using a database unique index to enforce uniqueness

If your application uses a relational database to store its data, then the easiest solution to enforce record uniqueness is implementing a unique index. This way, we let the database itself enforce the rule.

With Entity Framework Core, we can apply an attribute to an entity class to specify that its underlying database table should define an index.

For example, imagine we have a Make entity class that looks like this:

public class Make
{
    public int ID { get; set; }
    public required string Name { get; set; }
}

If we need every Make record to have a unique Name, we can do so by adding this attribute to the class:

+ using Microsoft.EntityFrameworkCore;

// ...

+ [Index(nameof(Name), IsUnique = true)]
public class Make
{
    // ...
}

The Index attribute from the Microsoft.EntityFrameworkCore namespace, with its IsUnique property set to true takes care of that.

Now, if we were to create a migration with a command like this:

dotnet ef migrations add AddUniqueIndexToMakes

We’ll see something like this in the generated code:

// ...
migrationBuilder.CreateIndex(
    name: "ix_makes_name",
    table: "makes",
    column: "name",
    unique: true);
// ...

See how the framework takes notice of the Index attribute and produces some code to create an index in the database with the column and uniqueness constraint that we specified.

After applying the migration with dotnet ef database update, any attempt at inserting or updating records that violate this constraint will result in an error.

Suppose we have an endpoint for registering new makes that looks like this:

// ...

private readonly VehicleQuotesContext _context;

// ...

[HttpPost]
public async Task<ActionResult<Make>> PostMake(Make make)
{
    _context.Makes.Add(make);

    await _context.SaveChangesAsync();

    return Created();
}

If we POST to it, and give it a payload with a non-unique name…

$ curl -X 'POST' 'http://localhost:5000/api/Makes' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "Toyota"
}'

…here’s what we get back:

Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
 ---> Npgsql.PostgresException (0x80004005): 23505: duplicate key value violates unique constraint "ix_makes_name"

This is only a portion of the output, but sure enough, we see a Microsoft.EntityFrameworkCore.DbUpdateException being thrown with the message indicating that a unique constraint was not met. The request results in a 500 Internal Server Error response.

That gets the job done for sure: it prevents bad data from entering the system. But generally we don’t want to expose such verbose errors to our API clients. It’d be better, for example, to catch that exception and return a 400 series status code. A simple 400 Bad Request seems appropriate in this case. We can make the endpoint behave this way with these tweaks:

[HttpPost]
public async Task<ActionResult<Make>> PostMake(Make make)
{
    _context.Makes.Add(make);

+    try
+    {
        await _context.SaveChangesAsync();
+    }
+    catch (DbUpdateException)
+    {
+        return BadRequest();
+    }

    return Created();
}

Simple enough: we use a try-catch to handle the Microsoft.EntityFrameworkCore.DbUpdateException and return a response of our choosing. Trying the same request again results in a 400 Bad Request response with this body:

{
  "type": "https://tools.ietf.org/html/rfc9110#section-15.5.1",
  "title": "Bad Request",
  "status": 400,
  "traceId": "00-7ea5139de6f4ce471aa096dcf4a79ce5-dd308a30c4dd2b33-00"
}

Writing a custom validation attribute to enforce uniqueness

The database index solution will work in a lot of cases. However, there are some scenarios where it won’t. For example, the validation may need to be applied to input data that is not directly tied to a database table; or, maybe the app does not use a relational database at all. In cases where we can’t rely on unique indexes, a custom validation attribute is the best option.

The validation logic that the attribute would have to implement is simple. All it would have to do is query the underlying data store to see if a record with the given field value already exists, and if it does, trigger a validation error.

We’ll start by implementing a first iteration that works specifically for Make names. Then, we’ll extract the generic parts and create a reusable base attribute that can be used in many situations, with any class and field.

Before we can develop the attribute, consider the same Make class that we worked with before:

public class Make
{
    public int ID { get; set; }
    public required string Name { get; set; }
}

Also, we have a repository class that we use to access the underlying storage. Its public interface looks like this:

public interface IMakeRepository
{
    Make? FindByName(string name);
}

The actual implementation is not important to us right now (although you can find it here). It’s enough to know that we can use objects of this type to search for Make records by name.

Suppose also that we’ve included an instance of this type as a service via Dependency Injection. With something like this in the project’s Program.cs:

services.AddScoped<IMakeRepository, MakeRepository>();

With those pieces in place, a custom validation attribute that prevents Makes with the same name could look like this:

// Annotating the class with the AttributeUsage attribute is our way of letting
// the framework know that our UniqueMakeNameAttribute is meant to be applied to
// classes. It's produce a build error if we try to apply it to anything else.
[AttributeUsage(AttributeTargets.Class)]
// In order for our attribute to integrate correctly with the framework's model
// validation functionality, it needs to inherit from ValidationAttribute.
public class UniqueMakeNameAttribute : ValidationAttribute
{
    // Inheriting from ValidationAttribute requires implementing this method.
    // This is the method that gets called by the framework during model
    // validation.
    // Here, we're using it only to prepare the validation algorithm to run,
    // which is defined in the private Validate method.
    protected override ValidationResult? IsValid(
        // The framework uses this "value" parameter to pass in the full Make
        // object that's being validated.
        object? value,
        // The "context" parameter will contain various objects related to the
        // execution environment. It is used to obtain instances made available
        // by Dependency Injection.
        ValidationContext context
    ) {
        if (value == null) return ValidationResult.Success;

        // Here we use the validation context parameter to obtain an instance of
        // the repository
        var repo = context.GetService<IMakeRepository>();
        if (repo == null) return ValidationResult.Success;

        return Validate((Make)value, repo);
    }

    // This is where we actually run the custom validation logic.
    // Like I mentioned before, the logic is simple...
    private static ValidationResult? Validate(Make input, IMakeRepository repo)
    {
        // ...first it calls on the repository to try and find an existing
        // record wiht the same name...
        var existing = repo.FindByName(input.Name);

        // ...then, if another record with the same name exists, one which is
        // not its not the same being validated right now...
        if (existing != null && input.ID != existing.ID)
        {
            // ...return a ValidationResult which contains an understandable
            // error message.
            return new ValidationResult(
                $"The Make name '{input.Name}' is already in use."
            );
        }

        // If, on the other hand, no problem was found, return a successful
        // ValidationResult.
        return ValidationResult.Success;
    }
}

In order to put this logic to work, we use the attribute to annotate our Make class, like so:

+[UniqueMakeName]
public class Make
{
    public int ID { get; set; }
    public required string Name { get; set; }
}

Now, if we make a request to create a new Make and use an existing name, we get back a 400 Bad Request response with this payload:

{
  "type": "https://tools.ietf.org/html/rfc9110#section-15.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "errors": {
    "": [
      "The Make name 'Toyota' is already in use."
    ]
  },
  "traceId": "00-30bc11fbb22071998c05660fdc454f5b-1ef36a85ecfeaf9b-00"
}

OK, that’s maybe the best response we’ve had so far. It includes not only a proper HTTP status code but also a human readable message that a frontend app can display to its users. Also, with this approach, we don’t have to use a try-catch in the endpoint’s action method like we did before. This is because our attribute plugs right into ASP.NET’s model validation functionality, which triggers automatically at the beginning of every API request.

A generic custom validation attribute to enforce uniqueness

The only disadvantage of our validation attribute is that it only works for one particular class and one particular property. I’m thinking we can make something a little bit more generic which we can reuse in many scenarios.

We could implement an abstract base class that defines the core validation logic as a template method. The overall algorithm will be implemented in the abstract base class. Then, more concrete derived classes will provide the specializations to handle particular objects and fields.

We can refactor our code to make that happen. We should identify the parts of the code that are generic and the ones that are specific. The generic ones won’t change, regardless of what’s being validated. So they will be part of the abstract base class. On the other hand, the specific ones will vary, so they are better defined in derived classes. The base class will provide hooks for the derived ones to implement and supply specializations.

Looking at our current code, here’s what I can identify:

[AttributeUsage(AttributeTargets.Class)]
public class UniqueMakeNameAttribute : ValidationAttribute
{
    protected override ValidationResult? IsValid(
        object? value,
        ValidationContext context
    ) {
        if (value == null) return ValidationResult.Success;

        // Depending on the entity being validated, the repository to use to
        // try to find a would-be duplicated record in the database will be
        // different. So, this line for obtaining a reference to the repository
        // is better implemented in a derived class.
        var repo = context.GetService<IMakeRepository>();
        if (repo == null) return ValidationResult.Success;

        // The type of "value" will be different depending on the class that
        // this attribute is applied to. That means that this casting to "Make"
        // belongs in a derived class.
        return Validate((Make)value, repo);
    }

    private static ValidationResult? Validate(Make input, IMakeRepository repo)
    {
        // Each repository will have its own method for trying to find records.
        // What's more, each entity will have its own field through which its
        // uniqueness is established. This is the responsibility of derived
        // classes.
        var existing = repo.FindByName(input.Name);

        // Every object being validated has its own method for determining if
        // they are equal to or different than what's found in the underlying
        // storage. That can't be part of the generic class.
        if (existing != null && input.ID != existing.ID)
        {
            // Depending on the object being validated, the error message will
            // need to be adjusted. That message belongs in derived classes.
            return new ValidationResult(
                $"The Make name '{input.Name}' is already in use."
            );
        }

        return ValidationResult.Success;
    }
}

With that figured out, here’s an abstract, and generic, base class that can be used as a basis for implementing many concrete custom validation attributes:

[AttributeUsage(AttributeTargets.Class)]
// This class is generic. TEntity is the type of the object that derived classes
// will validate. TRepo is the type of the repository that derived classes will
// use to query the database.
public abstract class BaseUniqueAttribute<TEntity, TRepo> : ValidationAttribute
{
    // These abstract methods are the hooks that derived classes will implement
    // in order to provide specializations for the core algorithm.
    // This one takes care of finding would-be repeated records.
    protected abstract TEntity? FindRecord(TEntity input, TRepo repository);
    // This one checks whether the existing record, if found, is the same being
    // validated.
    protected abstract bool IsSameRecord(TEntity input, TEntity existing);
    // This one produces the error message to return when validation fails.
    protected abstract string GetErrorMessage(TEntity input);

    protected override ValidationResult? IsValid(
        object? input,
        ValidationContext context
    ) {
        if (input == null) return ValidationResult.Success;

        // Here we use TRepo as a type parameter instead of IMakeRepository.
        // Derived classes will be responsible of defining which type the repo
        // will be.
        var repo = context.GetService<TRepo>();
        if (repo == null) return ValidationResult.Success;

        // Here we cast to TEntity instead of Make or any other concrete class.
        // The derived classes will specify the type.
        return Validate((TEntity)input, repo);
    }

    // The overall algorithm in this method remains pretty much identical.
    // Only some of the particular pieces have been replaced by abstract hooks
    // that are to be implemented in derived classes.
    private ValidationResult? Validate(TEntity input, TRepo repo)
    {
        var existing = FindRecord(input, repo);

        if (existing != null && !IsSameRecord(input, existing))
        {
            return new ValidationResult(GetErrorMessage(input));
        }

        return ValidationResult.Success;
    }
}

With that generic abstract class in place, we can develop new specific custom validation attributes very easily. For example, here’s a new version of our UniqueMakeNameAttribute:

public class UniqueMakeNameAttribute : BaseUniqueAttribute<Make, IMakeRepository>
{
    protected override Make? FindRecord(Make input, IMakeRepository repo) =>
        repo.FindByName(input.Name);

    protected override bool IsSameRecord(Make input, Make existing) =>
        input.ID == existing.ID;

    protected override string GetErrorMessage(Make input) =>
        $"The Make name '{input.Name}' is already in use.";
}

Very compact, huh? With this base class, new concrete validation attributes just need to inherit from BaseUniqueAttribute, provide it the proper type parameters (Make and IMakeRepository in this case), and implement the three abstract methods. All of which are simple one liners.

You can test the endpoint again now and should see the same behavior as before. That means the refactoring was successful!

Writing a unit test for the custom validation attribute

Speaking of tests, let’s write a unit test for this. Testing custom validation attributes involves calling their GetValidationResult method and checking that it returns as expected. This is a public method defined in the ValidationAttribute base class. This method is defined by the framework, but it eventually calls our own custom logic that we wrote in the IsValid override.

Since our validator depends on an IMakeRepository, we’re going to have to provide it as a mock object. The validator also needs a ValidationContext instance that it can use to fetch services from the Dependency Injection container. Our test will also have to provide that in some way.

Here’s what such a test could look like:

[Fact]
public void GetValidationResult_ReturnsFailure_WhenAnotherMakeExistsWithTheSameName()
{
    // Arrange

    var attribute = new UniqueMakeNameAttribute();

    var existingMake = new Make() { ID = 10, Name = "test_name" };
    var makeToValidate = new Make() { ID = 20, Name = "test_name" };

    // We have to build a mock of type IMakeRepository for the service provider
    // to return. The attribute uses this class to query the database.
    // This test wants to trigger a validation failure. As such, this mock repo
    // is configured to return an object. This simulates that there is already
    // another record in the database that has the same name as the one being
    // validated.
    var mockRepository = new Mock<IMakeRepository>();
    mockRepository
        .Setup(m => m.FindByName(It.IsAny<string>()))
        .Returns(existingMake);

    // We also have to build a IServiceProvider mock. This will be used to
    // create a ValidationContext instance that the attribute will use to obtain
    // services available via Dependency Injection. Specifically, the repo.
    // With this configuration, it will return the mock repository.
    var mockServiceProvider = new Mock<IServiceProvider>();
    mockServiceProvider
        .Setup(m => m.GetService(typeof(IMakeRepository)))
        .Returns(mockRepository.Object);

    // The validation attribute needs a ValidationContext instance in order to
    // work. So we build this one, giving it the mock we've constructed.
    var context = new ValidationContext(makeToValidate, mockServiceProvider.Object, null);

    // Act

    // Here we invoke the public method that's available for us to run the
    // validation logic. This test expects the validation to fail.
    var result = attribute.GetValidationResult(makeToValidate, context);

    // Assert

    // Two simple assertions here. One checking that the result is not a success
    // and another checking that the expected error message is returned.
    Assert.NotEqual(ValidationResult.Success, result);
    Assert.Equal("The Make name 'test_name' is already in use.", result?.ErrorMessage);
}

And that’s how you can test a custom validation attribute like the one we’ve built. This is very conventional as unit tests go. The only unusual things to keep in mind is the method we need to call in order to exercise the validation logic: GetValidationResult. And also how to construct and configure the ValidationContext object that the method needs as a parameter.

And that’s all for now. We’ve seen two ways of implementing uniqueness validation in ASP.NET. One approach was leveraging an underlying relational database to create unique indexes. A more complicated — but also more flexible — approach leverages framework features to create custom validation attributes. And we even saw how we can test them!

Writing integration tests for an ASP.NET Web API

2024-05-13T00:00:00+00:00

Integration tests exercise a system by instantiating major components and making them interact with each other. They are great for validating important use case scenarios in an end-to-end or close to end-to-end manner.

Full integration tests seldom use mocks or fake objects. Usually, the full stack is tested as if the entire system were running for real. For REST APIs, that generally means tests that involve issuing HTTP requests, validating HTTP responses, and asserting on changes made to a persistent data store, like a database.

In this article, we’re going to discuss how to write such tests for a Web API built using ASP.NET.

Introducing the project

I’ll use an existing ASP.NET Web API project to demonstrate how to write these tests. The API is part of a system that calculates the value of used cars and offers quotes for them. As such, the API has an endpoint for calculating a vehicle quote, given its information and condition: POST /api/Quotes. It also has an endpoint for administration purposes that returns all the quotes that have been stored in the system’s database: GET /api/Quotes. These are the two endpoints that we’ll want to test.

The source code is on GitHub, so feel free to browse. I’ve organized it so that the changes that we’ll make throughout this article are all contained in a single commit. You can see the diff here.

The code base is organized as a .NET solution, as evidenced by the vehicle-quotes.sln file at the root of the repository. The Web API project can be found inside the VehicleQuotes.WebApi directory. The endpoints that we want to test are defined in the controller at VehicleQuotes.WebApi/Controllers/QuotesController.cs.

Our plan is to develop integration tests that exercise the entire stack. That is, the API’s HTTP request handling as well as its database interactions. These are the steps that we’ll take in order to do that:

Create a new xUnit project where we will put our integration tests.
Define a test class fixture that will connect our tests to a test database.
Write some logic to run the tests within their own database transactions. This makes sure they don’t affect one another, and that they encounter the database in a clean state and also leave it that way.
Write some tests that interact with the API over HTTP.

Setting up the integration tests project

The first step is to create a new xUnit project and add it to our solution. This can be done with this pair of commands:

dotnet new xunit -o VehicleQuotes.IntegrationTests
dotnet sln add ./VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj

That will create a new xUnit project under the VehicleQuotes.IntegrationTests directory. It will have an empty test class file that can be deleted.

The project needs the Microsoft.AspNetCore.Mvc.Testing NuGet package. If we move into the VehicleQuotes.IntegrationTests directory, the package can be installed with this command:

dotnet add package Microsoft.AspNetCore.Mvc.Testing --version 8.0.4

This package will allow our tests to issue HTTP requests to the Web API. We’ll see how that’s done soon.

We also need to add a reference to the Web API project. Also from within the VehicleQuotes.IntegrationTests directory, we can do that with:

dotnet add reference ../VehicleQuotes.WebApi/VehicleQuotes.WebApi.csproj

That way our testing project will have access to the classes defined in the Web API project. Specifically, we’ll need the DB context and some entities. We’ll see why soon.

There’s one additional step that we need to do in the Web API project so that it is testable, and that’s explicitly defining its “Program” class. To do that, we add this line at the end of the VehicleQuotes.WebApi/Program.cs file:

// VehicleQuotes.WebApi/Program.cs
// ...
public partial class Program { }

I’ll admit: this is quite strange. But it is a requirement for integration testing. You’ll see how this comes into play when we start writing the tests. You can read more about it in the official docs.

With that, the project is set up. In the end, our VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj should look like this:

<!-- VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj -->
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="coverlet.collector" Version="6.0.0" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="8.0.4" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
    <PackageReference Include="xunit" Version="2.5.3" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.5.3" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Xunit" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\VehicleQuotes.WebApi\VehicleQuotes.WebApi.csproj" />
  </ItemGroup>

</Project>

Writing a database fixture

Now, we need to make it possible for our API to interact with a test instance of our database when running within the context of our tests. This can be done with a properly configured test class fixture. Let’s see what that looks like.

First of all we need an appsettings file for our test project that contains the test database connection string. I created a VehicleQuotes.IntegrationTests/appsettings.Test.json file with these contents:

{
  "ConnectionStrings": {
    "VehicleQuotesContext": "Host=db;Database=vehicle_quotes_test;Username=vehicle_quotes;Password=password;Include Error Detail=True"
  },
  "Jwt": {
    "Key": "this is the secret key for the jwt, it must be kept secure",
    "Issuer": "vehiclequotes.endpointdev.com",
    "Audience": "vehiclequotes.endpointdev.com",
    "Subject": "JWT for vehiclequotes.endpointdev.com"
  },
  "DefaultOffer": 77
}

We also have to tell .NET that it needs to include this file when building the project to run the tests. We do so by adding the following to the VehicleQuotes.IntegrationTests/VehicleQuotes.IntegrationTests.csproj file:

<Project Sdk="Microsoft.NET.Sdk">
  <!-- ... -->
  <ItemGroup>
    <Content Include="appsettings.Test.json" CopyToOutputDirectory="PreserveNewest" />
  </ItemGroup>
  <!-- ... -->
</Project>

The important thing in this appsettings file is the ConnectionStrings.VehicleQuotesContext setting, which contains the test database connection string. Notice the value for Database in the connection string is appended with _test. This is how we make sure the tests run against a different database. The rest of the settings are unrelated to the test database, but need to be defined for the Web API to work. These will obviously be different for every project. All in all, this file is meant to be a test version of the Web API’s own appsettings.json file. You can find it here.

Our Web API supports authentication via Bearer Token. If you want to learn more about how I implemented that, here’s another blog post describing the process.

Next, we develop the test class fixture for enabling database access. We define a VehicleQuotes.IntegrationTests/Fixtures/DatabaseFixture.cs file that looks like this:

using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Configuration;
using VehicleQuotes.WebApi;

namespace VehicleQuotes.IntegrationTests.Fixtures;

// For more info about this class, check:
// https://learn.microsoft.com/en-us/ef/core/testing/testing-with-the-database#creating-seeding-and-managing-a-test-database
public class DatabaseFixture
{
    private static readonly object _lock = new();
    private static bool _databaseInitialized;

    // Initializes the database specified in the connection string defined in
    // the appsettings.Test.json file.
    public DatabaseFixture()
    {
        // Tests can run in parallel. This lock is meant to make this code
        // thread safe.
        lock (_lock)
        {
            if (!_databaseInitialized)
            {
                using (var dbContext = CreateDbContext())
                {
                    // Delete the database and recreate it.
                    dbContext.Database.EnsureDeleted();
                    dbContext.Database.EnsureCreated();
                }

                _databaseInitialized = true;
            }
        }
    }

    // Creates a new VehicleQuotesContext instance configured with the
    // connection string defined in the appsettings.Test.json file.
    public VehicleQuotesContext CreateDbContext()
    {
        // Load up the appsettings.Test.json file
        var config = new ConfigurationBuilder()
            .AddJsonFile("appsettings.Test.json")
            .Build();

        // Create an instance of DbContextOptions using the connection string
        // defined in the appsettings.Test.json file.
        var options = new DbContextOptionsBuilder<VehicleQuotesContext>()
            .UseNpgsql(config.GetConnectionString("VehicleQuotesContext"))
            .UseSnakeCaseNamingConvention()
            .Options;

        var dbContext = new VehicleQuotesContext(options);

        return dbContext;
    }

    // Runs the given "test" within a database transaction created using the
    // given "dbContext". It rolls back the transaction when the "test" is done.
    public async Task WithTransaction(VehicleQuotesContext dbContext, Func<Task> test)
    {
        dbContext.Database.BeginTransaction();

        try
        {
            await test.Invoke();
        }
        catch
        {
            throw;
        }
        finally
        {
            dbContext.Database.RollbackTransaction();
        }
    }
}

I’ve made sure to include some comments on that class trying to explain what it does, so feel free to review. Much of it was inspired by .NET’s official docs.

This class serves the purpose of allowing the tests suite to connect to and interact with the test database. It does so by performing three tasks:

Resetting the database at the beginning of every test run. This happens in the constructor.
Allowing the creation of new VehicleQuotesContext instances which connect to the test database. Tests will use that to interact with the database.
Offering the capability for tests to be run within DB transactions. This makes sure they don’t affect one another, and that they encounter the database in a clean state and also leave it that way.

Writing some integration tests

Now we can finally start writing some tests. Let’s start with a simple one that makes a GET request to the “fetch all quotes” endpoint that I mentioned at the beginning: GET /api/Quotes. The one defined in the GetAll method in the VehicleQuotes.WebApi/Controllers/QuotesController.cs controller.

We create a new VehicleQuotes.IntegrationTests/Controllers/QuotesControllerTests.cs file and write our test in there. It looks like this:

using System.Net;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using VehicleQuotes.IntegrationTests.Fixtures;
using VehicleQuotes.WebApi;

namespace GifBackend.IntegrationTests.WebApi.Controllers;

public class OldQuotesControllerTests : IClassFixture<WebApplicationFactory<Program>>, IClassFixture<DatabaseFixture>
{
    private readonly WebApplicationFactory<Program> _factory;
    private readonly VehicleQuotesContext _dbContext;

    public OldQuotesControllerTests(WebApplicationFactory<Program> factory, DatabaseFixture database)
    {
        _factory = factory;
        _dbContext = database.CreateDbContext();
    }

    protected HttpClient CreateHttpClient()
    {
        return _factory.WithWebHostBuilder(builder =>
        {
            builder.ConfigureTestServices(services =>
            {
                services.AddSingleton(_ => _dbContext);
            });
        })
        .CreateClient();
    }

    [Fact]
    public async Task GetQuotes_ReturnsOK()
    {
        // Arrange
        var client = CreateHttpClient();

        // Act
        var response = await client.GetAsync("/api/Quotes");

        // Assert
        Assert.Equal(HttpStatusCode.OK, response.StatusCode);
    }
}

First, turn your attention to the GetQuotes_ReturnsOK test case. Very simple as tests go, but there are a few interesting things taking place here.

The test case itself is indeed simple. All it does is create an HTTP client, use it to send a GET request to the endpoint that we want to test (using the client’s GetAsync method), and finally validate that the response is a 200 OK. How it does these things is the interesting part.

The HTTP client is created using the CreateHttpClient method. This method leverages _factory, a WebApplicationFactory<Program> instance that’s injected by the framework into our test class. Here, the generic type parameter Program is referring to the “Program” class from the Web API project. The one we defined in the Program.cs file. Notice also how our test class implements the IClassFixture<WebApplicationFactory<Program>> interface. That’s what signals to the framework that a WebApplicationFactory<Program> instance needs to be passed/injected via the constructor. This is the way that the Microsoft.AspNetCore.Mvc.Testing package allows us to express that “this test class contains tests for this web application”.

Full details in the official docs.

Notice also how when creating the HTTP client, a VehicleQuotesContext instance is set up as a singleton service. This is key. This DB context is obtained thanks to our DatabaseFixture. That means that it connects to the test database. We configured it to do so. By setting it up as a service like this, we make sure that the Web API application uses that instance whenever it interacts with the database. And using that instance means that it will use the test database. Since this is the same instance that we will use within our tests, both the tests suite and the application (when running within the context of the tests) will share the same database.

Long story short: This way of constructing the HTTP client and injecting our own DB context into the running application is the secret sauce that allows our tests to utilize the test instance of the database.

Obtaining an instance of the DatabaseFixture is the same as obtaining an instance of the WebApplicationFactory<Program>: all we have to do is make our test class implement the IClassFixture<DatabaseFixture> interface and define the constructor parameter so that the framework passes it in.

Writing some more integration tests

OK, now that we understand the basics, let’s write a few more test cases in order to demonstrate some other common scenarios.

A test that writes to and reads from the database

For example, here’s one that validates that the GET /api/Quotes endpoint actually returns the data that’s stored in the database.

[Fact]
public async Task GetQuotes_ReturnsTheQuotesFromTheDatabase()
{
    await _database.WithTransaction(_dbContext, async () => {
        // Arrange
        await CreateNewQuote("2024", "Toyota", "Corolla");
        await CreateNewQuote("2024", "Honda", "Civic");

        var client = CreateHttpClient();

        // Act
        var response = await client.GetAsync("/api/Quotes");

        // Assert
        Assert.Equal(HttpStatusCode.OK, response.StatusCode);

        var quotes = await response.Content.ReadFromJsonAsync<IEnumerable<SubmittedQuoteRequest>>();

        Assert.NotNull(quotes);
        Assert.Equal(2, quotes.Count());

        Assert.Equal("2024", quotes.First().Year);
        Assert.Equal("Toyota", quotes.First().Make);
        Assert.Equal("Corolla", quotes.First().Model);

        Assert.Equal("2024", quotes.Last().Year);
        Assert.Equal("Honda", quotes.Last().Make);
        Assert.Equal("Civic", quotes.Last().Model);
    });
}

private async Task<Quote> CreateNewQuote(string year, string make, string model)
{
    var bodyType = await _dbContext.BodyTypes.SingleAsync(bt => bt.Name == "Sedan");
    var size = await _dbContext.Sizes.SingleAsync(s => s.Name == "Compact");

    var quote = new Quote {
        Year = year,
        Make = make,
        Model = model,
        BodyTypeID = bodyType.ID,
        SizeID = size.ID,
        ItMoves = true,
        HasAllWheels = true,
        HasAlloyWheels = false,
        HasAllTires = true,
        HasKey = true,
        HasTitle = true,
        RequiresPickup = true,
        HasEngine = true,
        HasTransmission = true,
        HasCompleteInterior = false,
        OfferedQuote = 123,
        Message = "test_message",
        CreatedAt = DateTime.UtcNow
    };

    _dbContext.Quotes.Add(quote);

    _dbContext.SaveChanges();

    return quote;
}

This method introduces a few more interesting features:

It runs within a database transaction. Ensuring that any data changes are rolled back once the test is done.
It uses the singleton DB context to interact with the database. Inserting new records before executing the application under test.
It parses a JSON response body into a .NET object.

In detail, here’s what it does: It uses our DatabaseFixture’s WithTransaction method to run within a database transaction. The test’s strategy is simple: it first inserts new records into the database, leveraging the CreateNewQuote helper method. Then it sends a request to the Web API’s GET /api/Quotes endpoint. Finally, in the assertion section, it validates that the response came back with the correct HTTP status code. Then it parses the JSON response body into an object, and inspects that object to make sure that it has the correct data in it — that is, it contains the database records that were inserted at the beginning of the test case.

A test that makes a POST request

Using all these concepts, we can also write a test for the POST /api/Quotes endpoint. For example, here’s a test that validates that the endpoint stores new records in the database using the given payload:

[Fact]
public async Task PostQuote_CreatesANewQuoteRecord()
{
    await _database.WithTransaction(_dbContext, async () => {
        // Arrange
        var client = CreateHttpClient();

        Assert.Empty(_dbContext.Quotes);

        // Act
        var response = await client.PostAsJsonAsync(
            "/api/Quotes",
            new
            {
                Year = "1990",
                Make = "Toyota",
                Model = "Corolla",
                BodyType = "Sedan",
                Size = "Compact",
                ItMoves = true,
                HasAllWheels = true,
                HasAlloyWheels = false,
                HasAllTires = true,
                HasKey = true,
                HasTitle = true,
                RequiresPickup = false,
                HasEngine = true,
                HasTransmission = true,
                HasCompleteInterior = true
            }
        );

        // Assert
        Assert.Equal(HttpStatusCode.OK, response.StatusCode);

        Assert.Single(_dbContext.Quotes);

        var quote = _dbContext.Quotes.First();
        Assert.NotNull(quote);
        Assert.Equal("1990", quote.Year);
        Assert.Equal("Toyota", quote.Make);
        Assert.Equal("Corolla", quote.Model);
    });
}

The only new concept that this test introduces is the use of the HTTP client’s PostAsJsonAsync method to send POST requests. Notice how we can send any payload we want using an anonymous object. In the assertion phase, the test queries the database to check if the expected record was inserted.

All of this is made possible by the singleton VehicleQuotesContext instance. Both test code and application code are talking to the test database. And thanks to the transactions, each test cleans up after it’s done so that the next test can run with a clean slate.

A test that makes many requests

We can also write tests that span multiple requests. Here’s one for example that registers a new user account and logs in, so that it can then be allowed access to a secure endpoint:

using System.Net.Http.Headers;

// ...

[Fact]
public async Task GetQuotesSecure_ReturnsOK_WhenTheUserHasLoggedIn()
{
    await _database.WithTransaction(_dbContext, async () => {
        // Arrange
        var client = CreateHttpClient();

        await RegisterUser(client);
        var response = await Login(client);
        var authResponse = await response.Content.ReadFromJsonAsync<AuthenticationResponse>();

        Assert.NotNull(authResponse);

        // Act
        using var requestMessage = new HttpRequestMessage(HttpMethod.Get, "/api/Quotes/Secure");
        requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", authResponse.Token);

        response = await client.SendAsync(requestMessage);

        // Assert
        Assert.Equal(HttpStatusCode.OK, response.StatusCode);
    });
}

private async Task<HttpResponseMessage> RegisterUser(HttpClient client)
{
    var response = await client.PostAsJsonAsync(
        "/api/Users",
        new
        {
            UserName = "test_user_name",
            Password = "test_password",
            Email = "test@email.com"
        }
    );

    Assert.Equal(HttpStatusCode.Created, response.StatusCode);

    return response;
}

private async Task<HttpResponseMessage> Login(HttpClient client)
{
    var response = await client.PostAsJsonAsync(
        "/api/Users/BearerToken",
        new
        {
            UserName = "test_user_name",
            Password = "test_password"
        }
    );

    Assert.Equal(HttpStatusCode.OK, response.StatusCode);

    return response;
}

This test is a little more complicated and introduces a few new concepts. All within its own database transaction, this test:

Uses the RegisterUser helper method to register a new user account. It calls the POST /api/Users endpoint for this.
Uses the Login helper method to log in. In our demo Web API project, this means obtaining a token that can be used for Bearer Token authentication. It calls the POST /api/Users/BearerToken endpoint for this.
Extracts the generated token from the response, and prepares a new request for the GET /api/Quotes/Secure endpoint using that token as an authentication header.
Sends the request and validates that it results in a successful HTTP status code.

Notice that we had to use the HTTP client’s more verbose SendAsync method instead of the more convenient GetAsync. This is because GetAsync doesn’t support sending headers, which we needed.

Disabling some application services

Before we’re done here, something useful about this approach is that it is possible to disable some services on the application under test. We’ve been writing full integration tests where all system components are exercised. It could be the case, however, that we would like to exclude some components from testing. For example, if the app under test sends emails, we might want to disable that. Or if it invokes another third party service, we might want the tests to not do that.

Because we’re able to inject services of our choosing to the running application (like we do with the DB context), it’s certainly possible for us to disable parts of the system. For example, imagine an application that sends emails using an IMailer derived class. One could inject a null object in its place. We could do this when creating the test HTTP client. Something like this:

private HttpClient CreateHttpClient()
{
    // Disable emails for integration tests
    var mockMailer = new Mock<IMailer>();
    mockMailer.Setup(m => m.SendMailAsync()).ReturnsAsync(true);

    return _factory.WithWebHostBuilder(builder =>
    {
        builder.ConfigureTestServices(services =>
        {
            services.AddSingleton(_ => _dbContext);
            services.AddTransient(_ => mockMailer.Object);
        });
    })
    .CreateClient();
}

Here, I created a mock object of the same type as the “Mailer” service that the application uses; then I added it to its Dependency Injection container. Now, every time the application calls for an IMailer instance, it will get the mock. A mock that does nothing.

And that’s it! I think a good amount of integration tests will end up utilizing and remixing various combinations of these basic concepts. I invite you to look at the demo app’s source code on GitHub, where I’ve added a few more tests. I also did some refactoring to make these features a little easier to reuse. Happy testing!

I wrote the same app twice, with Hotwire and Blazor Server — here’s what I learned

2023-05-27T00:00:00+00:00

There’s been a very interesting movement that has emerged recently in the world of frontend web development: a rise of little-to-no-JavaScript frontend frameworks.

The promise here is that we would be able to develop web applications with rich interactive capabilities without the need to write a whole lot of JavaScript. As such, these new approaches present themselves as alternatives to the likes of Vue, React, Angular, etc.

Two recent technologies that try to fulfill this promise come from two of the most prolific web application development frameworks of today: Blazor, built on .NET, and Hotwire, built on Ruby on Rails.

Now, I love my JS frameworks as much as the next guy, but these new technologies are intriguing. So I decided to build the same application twice, with Hotwire and with Blazor. I learned a few things along the way that I would like to share in this blog post.

Note that there is a table of contents at the end of this post.

What this article is

I want to present some of my findings when working with these two technologies. I also want to discuss how they work and how they feel. How they are similar and how they are different. How they take different routes to arrive at their ultimately similar destinations. Maybe offer some pros and cons.

This post assumes sufficient familiarity with C#, ASP.NET, Ruby, Ruby on Rails and the current state of the art of web development. I won’t assume any familiarity with either Blazor or Hotwire, but this is not a tutorial for either, so I won’t explain in detail how to fully build apps with these technologies.

So who am I writing this for? Essentially, for anybody who is curious about these technologies and is interested in understanding the big picture of what they are about, how they compare to each other, and building their next project with one of them. So, this article is intended to serve more as an introduction to both, a starting point for a conversation to help you make a decision on what’s best for you and your team.

Spoiler alert: Both are great and you can’t go wrong with either. It all comes down to your team’s preferences and past experience.

One final thing worth noting is that I’m focusing this article on “Blazor Server” specifically. I’ll be using the word “Blazor” moving forward, for short. Blazor as a framework has three variants: Blazor Server, Blazor WebAssembly and Blazor Hybrid. You can learn more about them here.

Most of the examples that I’ll use in this post come from a couple of demo apps that I built in order to get my feet wet with these technologies. You can find both of them in GitHub. Here’s the Blazor one and here’s the Hotwire one. You can study both the source code and the commit history, which I tried my best to keep neatly organized. They are both functionally identical, and based on this excellent Hotwire tutorial.

An overview of Blazor

When it comes to how they are designed and the developer experience they offer, these two are very different. Let’s go over some of the key details of Blazor and then we’ll do the same with Hotwire. With that, the differences between them will become apparent.

The first thing we have to understand about Blazor is that it is a component framework, very much like Vue or React. So, with Blazor, applications are broken up into composable modules called “Razor/Blazor components” that are essentially independent pieces of GUI bundled with their corresponding logic. Each component has three parts to it:

HTML-like markup that describes the layout and UI elements to be rendered,
C# logic that defines the behavior of the component, like what actions to take when users interact with GUI elements, and
CSS for styling the GUI elements.

For example here’s a simple a Blazor component that allows displaying, editing, and deleting a particular type of record called “Quote”. Don’t worry about the details too much; we’ll go over some of them next. For now, I just want us to get a sense of what Blazor components look like:

@using Microsoft.EntityFrameworkCore

@inject IDbContextFactory<QuoteEditorBlazor.Data.QuoteEditorContext> dbContextFactory

@if (isEditing)
{
    <QuoteEditorBlazor.Shared.Quotes.Edit
        QuoteToEdit="QuoteToShow"
        OnCancel="HideEditForm"
    />
}
else
{
    <div class="quote">
        <a href="quotes/@QuoteToShow.ID">@QuoteToShow.Name</a>
        <div class="quote__actions">
            <a class="btn btn--light" @onclick="DeleteQuote">Delete</a>
            <a class="btn btn--light" @onclick="ShowEditForm">Edit</a>
        </div>
    </div>
}

@code {
    [Parameter]
    public QuoteEditorBlazor.Models.Quote QuoteToShow { get; set; }

    [Parameter]
    public EventCallback OnQuoteDeleted { get; set; }

    bool isEditing = false;

    void ShowEditForm()
    {
        isEditing = true;
    }

    void HideEditForm()
    {
        isEditing = false;
    }

    void DeleteQuote()
    {
        using var context = dbContextFactory.CreateDbContext();
        context.Quotes.Remove(QuoteToShow);
        context.SaveChanges();
        OnQuoteDeleted.InvokeAsync();
    }
}

You can see that we have some C# in the file (enclosed in a @code block) with some event handlers and parameters. We also have some markup written with a mixture of HTML and C#. This markup has conditionals, wires up click event handlers, renders data from a given record, renders another component, etc. That markup is really just Razor, a templating language that has been widely used in ASP.NET for a good while now. And we also have some top-level statements like @using and @inject for including classes and objects that the component can use.

As far as CSS goes, here’s how it works: Suppose these are the contents of a file called Example.razor. The CSS for it would have to be defined in an Example.razor.css file sitting right next to it. Syntax-wise, this would just be a plain old CSS file. One cool thing to mention about it, though, is that the CSS within it is visible only to the component. So there’s no risk of conflicting rules between components.

If you’re familiar with modern frontend web development and have used frameworks like Vue or React, this should look very familiar to you. In fact, I would venture to say this is one of Blazor’s most attractive points. If you come from that background, and know .NET, it’s not that big of a leap to get into Blazor. The development experience is very similar as its design shares many concepts with modern JS frameworks; they operate under a very similar mental model.

Of course, C# is not JavaScript and .NET is not a browser. So there are many differences when it comes to the nitty-gritty mechanics of things. So a thorough understanding of the .NET framework and its libraries is also important for mastering Blazor. Which, depending on your team’s background, may be a point in favor or against.

How Blazor works under the hood

As you can probably gather from the previous discussion, Blazor takes control of the entire GUI of the application and puts up a firm layer of abstraction over traditional native web technologies. All notion of JavaScript or code executing on a browser environment is greatly de-emphasized. In fact, most of Blazor executes in the server.

Essentially, all the code that you actually write executes on the server side. When a user begins using the app, a two-way, persistent SignalR connection is established between the browser and server. Whenever the client requests a page, the server renders it and sends it through the connection to the client browser, where there’s a component that interprets and displays it. Likewise, whenever the user interacts with some UI element, like clicking a button or a link, the action is sent to the server via this SignalR connection for it to be processed. If changes to the GUI are necessary, the server calculates them and sends them back to the user’s browser, where they will be interpreted and used to re-render the screen with the new state.

Microsoft’s official documentation has an excellent diagram that helps explain the process:

So, even though there is client side code running and browser DOM being manipulated, this is all happening under the hood. The developer doesn’t need to be concerned with that and can just focus on authoring C# code, for the most part.

This approach has a few implications worth noting. One is that this means higher load on the server compared to more traditional web applications. This is mainly because there needs to be a connection always open between clients and the server, by design. Classic HTTP is purely stateless, and connections are typically opened and closed multiple times throughout a user’s session, as they interact with the web app. Not so for Blazor, where this SignalR connection (most likely via WebSockets) is always alive. So, for scalability concerns, we need to keep in mind that the more concurrent users our app has, the more resources the server will consume, even if the users are somewhat idle.

An abstraction over the request/response cycle

A key element of Blazor is that it abstracts developers from the classic HTTP request/response model. With Blazor, one seldom has to consider that aspect, as all interactions between client and server are managed via the framework itself through the persistent connection we discussed in the last couple of paragraphs. To the developer, there’s no real separation between the two. This is a big departure from classic MVC-like frameworks where things like endpoints, actions, and controllers are front and center. Such concepts are simply not in play on Blazor apps. The idea is to make them feel more like desktop apps: fully integrated, monolithic, simple packages of GUI and functionality.

This could become a double-edged sword as, in general, it is always important to have a clear understanding of the underlying technologies that support the application stack you’re working with. That is, the web is still there, even if you can’t see it. But as long as you’re cognizant of that, this approach can have great advantages too.

For example, thanks to this approach, there’s no need to employ the classic SPA pattern of developing applications in two halves: 1. a backend Web API for domain logic written in some backend programming language, and 2. a frontend application written in JavaScript that implements the user experience and communicates with the backend over HTTP.

Blazor attempts to offer a simpler solution from a developer’s perspective, where everything lives in the same execution environment so there’s no need for inter-process communication over HTTP. The example Blazor component from before demonstrates such a case. Notice how this link has an event handler registered to its click event:

<a @onclick="DeleteQuote">Delete</a>

That DeleteQuote event handler, simply defined as the method below, directly leverages the DbContext to issue the delete command to the underlying database:

void DeleteQuote()
{
    using var context = dbContextFactory.CreateDbContext();
    context.Quotes.Remove(QuoteToShow);
    context.SaveChanges();
}

Pretty simple.

However, this flexibility also requires great discipline from the development team. It falls upon us not to clutter the GUI code with all manner of extraneous things like database calls and domain or application logic that have nothing to do with user interface concerns. The traditional two-halves pattern for SPAs has this separation between domain and GUI logic baked in by necessity. Blazor allows us to break free from it, but that does not mean that the separation is not useful or even necessary. For a small example like this, this works just fine, but for larger applications, a more modularized design should be considered as well. Maybe the introduction of abstractions like repositories or domain services? At the end of the day, the core software design principles still need to be applied.

How Blazor supports common frontend framework features

Something else to consider, which I touched on before, is that Blazor is built on top of .NET. That means that a solid understanding of .NET concepts is all but a necessity in order to be effective with Blazor. Most of the features that are now traditional and expected in frontend JavaScript frameworks exist in Blazor, and they are implemented using age-old .NET concepts. If your team has solid .NET experience, this is a blessing. If not, then Blazor requires a larger investment, one that could be overwhelming depending on your time constraints.

Here are a few examples of how Blazor implements classic frontend framework features:

Handling DOM events

We already saw how event handlers work: they are defined as regular C# methods. The way they are registered to respond to events is via attributes in the GUI elements like @onclick or @onchange. All traditional DOM events are available. Like this:

<a @onclick="DeleteQuote">Delete</a>

Including other Blazor components

We also saw how to render components from within other components. All it takes is adding the component to the template as if it was any other GUI element/HTML tag. The tag itself is the name of the component, which sometimes needs to be fully qualified. We saw an example before:

<QuoteEditorBlazor.Shared.Quotes.Edit
    QuoteToEdit="QuoteToShow"
    OnCancel="HideEditForm"
/>

The fully qualified component name is QuoteEditorBlazor.Shared.Quotes.Edit in this case, and that’s how we reference it. If we were to include the namespace with an @using statement (like @using QuoteEditorBlazor.Shared.Quotes near the top of the file) then we would be able to invoke the component just by its name of Edit.

Passing parameters to components

That previous snippet also demonstrates how to pass parameters to components. In this case, we have two: QuoteToEdit which is an object, and OnCancel which is a custom event. As you can see, parameters are passed as if they were HTML element attributes. In the case of QuoteToEdit, we’re passing it QuoteToShow, which is a C# Property defined in the @code section of the component:

public QuoteEditorBlazor.Models.Quote QuoteToShow { get; set; }

In order for the receiving component to be able to accept the parameter, it needs to define a property itself of the same type, annotated with the Parameter attribute. In this case, the QuoteEditorBlazor.Shared.Quotes.Edit component defines it like this:

[Parameter]
public QuoteEditorBlazor.Models.Quote QuoteToEdit { get; set; }

Notice how the name of the property is the same name used for the “HTML attribute” that was used in markup to pass the parameter to the component:

QuoteToEdit="QuoteToShow"

In this case, the expected parameter is of type QuoteEditorBlazor.Models.Quote.

Defining, handling and triggering custom component events

The other parameter, which is a custom event, accepts a method. It looks like this in the receiving component:

[Parameter]
public EventCallback OnCancel { get; set; }

Again, this is a Property annotated with the Parameter Attribute. The only difference is that the type of this one is EventCallback. That’s what allows it to accept a method. Then, inside the receiving component, the event can be triggered with code like this:

OnCancel.InvokeAsync();

This will execute whatever method the parent component has registered as a handler for this custom event. In this case, that would be our HideEditForm method.

Handling component lifecycle events

Other than DOM and custom events, much like in other frontend frameworks, Blazor components also offer ways of hooking up to their own internal lifecycle events. OnInitialized is one of the most important ones, which runs when the component is first starting up. To hook into it and run some code when it happens, all a Blazor component has to do is implement it as a method override within its code section. Something like this:

protected override void OnInitialized()
{
    // Do something here.
}

There’s more to learn about component lifecycle and the official documentation does a good job in explaining it.

Templates

Like we discussed before, Blazor also offers a rich templating syntax via Razor, which is also a hallmark of modern frontend frameworks. We have conditional rendering, support for loops, interpolation of data, model binding, etc.

Routing

Routing is also included in Blazor and the way that it works is with the @page directive that’s used on top of components like this:

@page "/quotes"

Not all components will use these, only those that correspond to whole pages. Most components will be used as portions of pages and as such, would not include these. A @page statement like the above will make the component that includes it accessible via an URL like http://mydomain.com/quotes. In other words, at the root of the site.

These routes also support parameters, which can be defined like this:

@page "/quotes/{quoteId:int}"

A component with this directive will be accessible through URLs like http://mydomain.com/quotes/123. The code in the component can access the route parameter quoteId by defining it as a Property with the matching type, annotated with the Parameter attribute. Like this:

[Parameter]
public int QuoteId { get; set; }.

State management

Global application state management à la Vuex or Redux is also available in Blazor. The cool thing about how this is implemented in Blazor is that there is no need for any additional library or special components. A global app store can be a simple C# object that’s configured to have a lifetime that spans that of the user’s session. Here’s an example of a class that’s used to store global flash messages:

namespace QuoteEditorBlazor.AppState;

public class FlashStore
{
    public List<string> Messages { get; private set; } = new List<string>();

    public event Action? MessagesChanged;

    public async void AddMessage(string message)
    {
        Messages.Add(message);
        MessagesChanged?.Invoke();

        await Task.Delay(TimeSpan.FromSeconds(5));

        Messages.Remove(message);
        MessagesChanged?.Invoke();
    }
}

Like I said: a plain and simple C# class. It offers a method for displaying a message for a few seconds. It does this by storing the given message into an internal variable and then, after a little while, it gets removed. It also offers a MessagesChanged event that other components can subscribe to which is triggered as messages are added and removed.

The lifetime of the instance is controlled via its dependency injection configuration. In our Program.cs, it would be configured like this:

builder.Services.AddScoped<FlashStore>();

Like I mentioned before, Blazor apps establish a persistent connection for the user throughout their session. This means that the instance of the app that is running on the server is also persistent. So, if we add FlashStore as a scoped service, one single instance of it will also persist throughout the session. That’s why we can rely on its internal variables to store global application state.

You can learn more about state management in Blazor here.

Then, you could have a component that renders those messages that looks like this:

@using QuoteEditorBlazor.AppState

@inject FlashStore flashStore

@implements IDisposable

<div class="flash">
    @foreach (var message in flashStore.Messages)
    {
        <div class="flash__message">
            @message
        </div>
    }
</div>

@code {
    protected override void OnInitialized()
    {
        flashStore.MessagesChanged += StateHasChanged;
    }

    public void Dispose()
    {
        flashStore.MessagesChanged -= StateHasChanged;
    }
}

Very simple too. This Blazor component uses a loop to render all the messages. There are a couple of interesting things about this one. First, the component gets access to the instance of the FlashStore class via the @inject directive near the top of the file. That’s how the flashStore variable is made available for the component to use both in C# code and in the template.

The second interesting element is how this component registers its StateHasChanged method to the MessagesChanged event defined in FlashStore. As you recall, FlashStore will trigger MessagesChanged every time messages are added or removed. By registering StateHasChanged to that event, we make sure that the component re-renders every time the messages list changes; which in turn ensures that the most current messages are always rendered. Kind of a neat trick.

And as you might expect, any piece of code throughout the app can submit new messages to FlashStore by getting hold of the global instance via dependency injection and then calling:

flashStore.AddMessage("Here's a flash message!");

Interoperability with JavaScript

For all the nice abstractions that Blazor presents us with, the reality is that sometimes we actually do have to break through them. In the case of JavaScript, this happens when, for example, we need access to some native browser feature, or when we need to integrate with some library for some widget or other type of capability. In Blazor, this is certainly possible.

There are many details to consider when it comes to JavaScript interoperability in Blazor. Check out the official documentation to learn what all is possible. For our purposes here though, it’s enough to know that calls from .NET to JS and vice versa are supported.

The most basic example of calling JS code from .NET code looks like the following. If we have a JS function like this:

window.showMessagefromServer = (message) => {
    alert(`The server says: ${message}`);
    return "The client says thanks!";
};

Such a method can be invoked from a Blazor component like this:

string interopResult = await js.InvokeAsync<string>("showMessagefromServer", "Have a nice day!");

The Blazor component that runs this code would have to inject an instance of IJSRuntime into the js variable with a statement like @inject IJSRuntime js. A key thing to note is that the JavaScript method needs to be attached to the window object in order for it to be accessible.

Like I said, the other way around also works: JavaScript code is able to call .NET logic defined in a server-side Blazor component. Here’s a simple example for us to get an idea of how it feels. We can have a method that looks like this on a Blazor component:

[JSInvokable]
public static Task<string> getMessageFromServer()
{
    return Task.FromResult("Have a nice day!");
}

This method is public and static. It is also possible to invoke non-static instance methods, but the setup is a little more complicated, and this is enough for our purposes here. Other than that, as you can see, the method needs to be annotated with the JSInvokable attribute and return a type of Task.

Here’s some JavaScript that calls this method:

window.returnArrayAsync = () => {
    DotNet.invokeMethodAsync('{YOUR APP ASSEMBLY NAME}', 'getMessageFromServer')
        .then(data => {
            alert(`I asked the server and it says: ${data}`);
        });
};

There are a few noteworthy things here. First we have the DotNet.invokeMethodAsync function which Blazor makes available to us in JavaScript land. That’s what we use to invoke .NET server side code. Next, the function needs to be given the assembly name of our app as well as the name of the method to invoke within the Blazor component. This is the one that was annotated with JSInvokable. Finally, the function itself is asynchronous and thus returns a promise.

An overview of Hotwire

Hotwire makes a promise similar to Blazor’s. However, the way it goes about it couldn’t be more different.

Hotwire is much simpler and more minimalistic than Blazor. Whereas in Blazor we let the framework take total control of the GUI, just like we do with other popular frontend frameworks, Hotwire feels more like a natural evolution of traditional pre-JavaScript-heavy web development. We still have controllers and views, we still render pages on the server, and we still work in tandem with HTTP’s request/response model.

What Hotwire gives us, if we were to boil it down to a single sentence, is a way to refresh only portions of our pages as a result of user interactions. That is, we’re not forced to reload the entire page, as is the case with non-JavaScript web applications. For example, in Hotwire we have the ability to submit a form or click a link and, as a result of that, only update a particular message, picture, or section.

ASP.NET veterans will find this awfully familiar. That’s because all the way back in version 3.5, ASP.NET AJAX offered a very similar feature: partial page updates with the UpdatePanel component. Indeed, Hotwire presents a very similar concept; only greatly improved and modernized.

When it comes to actual coding, Hotwire’s footprint is minimal. It augments traditional Ruby on Rails controllers and views to produce the desired effect of page refreshes that are partial and targeted. Let’s walk through an example and you’ll see how we can start with a regular looking Rails app and then, through minor adjustments, we end up with a more richly interactive experience.

Imagine we are beginning to develop support for CRUDing a particular type of record called “Quote”, and we have these files:

### app/controllers/quotes_controller.rb
class QuotesController < ApplicationController
  def index
  end

  def new
    @quote = Quote.new
  end
end

<!-- app/views/quotes/index.html.erb -->
<main class="container">
  <div class="header">
    <h1>Quotes</h1>
    <%= link_to "New quote", new_quote_path %>
  </div>
</main>

<!-- app/views/quotes/new.html.erb -->
<main class="container">
  <%= link_to "Back to quotes", quotes_path %>

  <div class="header">
    <h1>New quote</h1>
  </div>

  <%= render "form", quote: @quote %>
</main>

<!-- app/views/quotes/_form.html.erb -->
<%= simple_form_for quote do |f| %>
  <% if quote.errors.any? %>
    <div class="error-message">
      <%= quote.errors.full_messages.to_sentence.capitalize %>
    </div>
  <% end %>

  <%= f.input :name %>
  <%= f.submit %>
<% end %>

This sample is using the simple_form gem, which you can learn more about here.

If you’re familiar with Rails, then you understand what’s happening here. We have a simple index page with a heading and a link to another page. That other page contains a form to create new Quote records. It also contains a link to go back to the index page.

Partial page updates with Turbo Frames

As they are right now, these files would produce a traditional web application user experience. When links are clicked, the whole screen will be reloaded to show the page that the clicked link points to. But what if we wanted, for example, to have the new record creation form appear out of nowhere within the same index page, without a full page reload? Here’s what that would look like with Hotwire:

 <!-- app/views/quotes/index.html.erb -->
 <main class="container">
   <div class="header">
     <h1>Quotes</h1>
     <%= link_to "New quote",
                 new_quote_path,
+                data: { turbo_frame: dom_id(Quote.new) } %>
   </div>

+  <%= turbo_frame_tag Quote.new %>
 </main>

 <!-- app/views/quotes/new.html.erb -->
 <main class="container">
   <%= link_to sanitize("&larr; Back to quotes"), quotes_path %>

   <div class="header">
     <h1>New quote</h1>
   </div>

+  <%= turbo_frame_tag Quote.new do %>
     <%= render "form", quote: @quote %>
+  <% end %>
 </main>

Remember that these examples are taken from a fully working application. Feel free to read through the source code to have a more complete understanding of the context within which these files exist.

And that’s really all it takes. Let’s go over it. With these changes, whenever a user clicks on the “New quote” link, instead of the browser triggering the usual GET request to then reload the screen and show the creation page, Hotwire’s frontend component captures the click event and makes the request itself via XHR. The server then receives this request normally and routes it to the new action in the quotes controller. All that action does is render the new.html.erb template and send that back to the client as a response.

When the Hotwire frontend component receives this, it notices that some of the response is enclosed in a turbo_frame_tag whose ID is that of an empty new Quote object. Now, because the link that the user clicked also had the ID of an empty new Quote object as its data-turbo_frame attribute, Hotwire looks for a turbo_frame_tag with the same ID in the page that’s currently being shown an replaces its contents with the contents from the similarly named turbo_frame_tag from the incoming response.

With that, you’ve seen in action some of the key elements that make Hotwire work. First of all we have the turbo_frame_tag helper, which produces a so-called “Turbo Frame”. Turbo Frames are the main building block that we use for partial page updates. Turbo Frames essentially say: “This section of the page is allowed to be dynamically updated without a full page refresh.” In an app that uses Hotwire, whenever you’re in a page that includes a Turbo Frame, if a request is made (whether it be navigation or form submission), and if the resulting response includes within it another Turbo Frame with the same ID, then Hotwire will notice the match and update the current page’s Turbo Frame with the contents of the Turbo Frame from the response.

Looking back at the code, you can see how we achieved this. We added an empty Turbo Frame on the index page, right below the link to the creation page. We also wrapped the form from the creation page in a similarly named Turbo Frame. Finally, we added the data-turbo_frame attribute to the link in the index page to tell Hotwire that it should kick in for this link and target that specific Turbo Frame. If the link was inside the Turbo Frame, we would not have to do this. Since it is outside, Hotwire needs the little hint that says: “Treat this link as if it was inside this Turbo Frame.”

I feel like this is at the same time a little awkward to wrap your head around and deceptively simple. When compared to Blazor, which builds upon tried and true concepts (as far as the developer experience goes at least), Hotwire almost seems alien, with a much more unusual style. But all in all, one can’t deny just how clean and simple all of this looks, in that “Rails magic” kind of way. And once it clicks, you can begin to see a world of possibilities opening up. The Hotwire developers managed to identify and extract a general design pattern of web application interactions, one that can be leveraged to produce a lot of varying rich interactive user experiences.

One neat aspect worth noting is that, if the user were to disable JavaScript, the app would still fully work. It would gracefully degrade. This is a direct consequence of Hotwire’s paradigm of adding minimal features on top of the existing traditional Rails programming model.

Imperative rendering with Turbo Streams

Let’s consider another example now. This time we’ll see another key component of Hotwire in action which is called Turbo Streams.

Turbo Streams helps solve the problem that emerges when the declarative style of pure Turbo Frames is not enough to obtain the fine grained control and behavior that we need. In these cases, the server needs to issue specific commands to the frontend on how to update the GUI. Via Turbo Streams, we can achieve just that. Coding-wise, these look like regular Rails view templates, albeit with some funky syntax thanks to the Turbo Stream helpers.

In our example, we’ll add a list of quotes in the index page, right below the link and the form. We’ll also complete our quote creation implementation so that we’re actually able to submit the form. As a result of that, we want the GUI to be updated so that the form goes away and the newly created quote is shown in the list. These last two are the things that we’ll use Turbo Streams for. Of course, these updates to the page will be done without a full page refresh.

Let’s start by adding a list of quotes on the index page.

In the controller, we query the database for all the quote records and store them in a variable that the view can later access.

 ### app/controllers/quotes_controller.rb
 class QuotesController < ApplicationController
   def index
+    @quotes = Quote.all
   end

   def new
     @quote = Quote.new
   end
 end

In the index view template, we render the collection of records.

 <!-- app/views/quotes/index.html.erb -->
 <main class="container">
   <div class="header">
     <h1>Quotes</h1>
     <%= link_to "New quote",
                 new_quote_path,
                 data: { turbo_frame: dom_id(Quote.new) } %>
   </div>

   <%= turbo_frame_tag Quote.new %>

+  <%= render @quotes %>
 </main>

Now we need to define a “_quote” partial view so that render statement we added on the index view can work automagically.

<!-- app/views/quotes/_quote.html.erb -->
<div class="quote">
  <%= quote.name %>
</div>

With this, the render @quotes statement will loop through all the records in @quotes and render this partial view for each on of them. This template is very simple. All it does is render the name of the quote.

Now let’s add an action that can accept quote creation form submissions:

 ### app/controllers/quotes_controller.rb
 class QuotesController < ApplicationController
   def index
     @quotes = Quote.all
   end

   def new
     @quote = Quote.new
   end
+
+  def create
+    @quote = Quote.new(quote_params)
+
+    if @quote.save
+      redirect_to quotes_path, notice: "Quote was successfully created."
+    else
+      render :new
+    end
+  end
+
+  def quote_params
+    params.require(:quote).permit(:name)
+  end
 end

A typical Rails recipe for a record creation endpoint. It takes the parameters coming from the request and uses them to create a new record via the Quote Active Record model. If successful, it redirects to the index page; if not, it renders the creation page again (via the new action). The new.html.erb view template has some logic to render error messages when they are present so those are going to show up when that page is rendered as a result of unsuccessful calls to this create endpoint.

At this point, we’re able to view all the quotes on record and create new ones. Now here are the changes to Hotwire-ify this scenario.

First we wrap the list of quotes with a Turbo Frame named “quotes”:

 <!-- app/views/quotes/index.html.erb -->
 <main class="container">
   <div class="header">
     <h1>Quotes</h1>
     <%= link_to "New quote",
                 new_quote_path,
                 data: { turbo_frame: dom_id(Quote.new) } %>
   </div>

   <%= turbo_frame_tag Quote.new %>

+  <%= turbo_frame_tag "quotes" do %>
     <%= render @quotes %>
+  <% end %>
 </main>

Next, we employ Turbo Streams. Like I said, Turbo Streams materialize themselves in code as if they were view templates. So, a new file is added that looks like this:

<!-- app/views/quotes/create.turbo_stream.erb -->
<%= turbo_stream.prepend "quotes", partial: "quotes/quote", locals: { quote: @quote } %>
<%= turbo_stream.update Quote.new, "" %>

It sort of looks like a couple of imperative statements, does it not? The first line instructs Hotwire to prepend, in the "quotes" Turbo Frame, a new render of the app/views/quotes/_quote.html.erb partial view, while passing it the @quote object as a parameter. The second line updates the Quote.new Turbo Frame to be empty. When we remember that the "quotes" Turbo Frame is the one that contains the list of records and the Quote.new Turbo Frame is the one that contains the new quote creation form, this starts to make sense. This Turbo Streams view is making the newly created quote appear in the list; and at the same time, it is making the form disappear. From a user’s perspective, this all takes place after submitting the creation form. So the user experience makes complete sense. All that with no full page refresh.

And finally, the controller action needs to make use of this new view template like so:

 ### app/controllers/quotes_controller.rb

 def create
   @quote = Quote.new(quote_params)

   if @quote.save
-    redirect_to quotes_path, notice: "Quote was successfully created."
+    respond_to do |format|
+      format.html { redirect_to quotes_path, notice: "Quote was successfully created." }
+      format.turbo_stream
+    end
   else
     render :new, status: :unprocessable_entity
   end
 end

This is yet another familiar Rails pattern. This is how we specify different response formats, whether it be HTML, JSON, XML. Now, thanks to Hotwire, we can also specify Turbo Streams. This is one of the great aspects about Hotwire: it seamlessly integrates with Rails’ existing features and concepts.

Anyway, in this case, we invoke format.turbo_stream within the block passed to respond_to and that makes it so the app/views/quotes/create.turbo_stream.erb view template is included in this action’s response. Hotwire’s frontend component sees this coming as part of the response and acts accordingly, updating the GUI how it’s been specified.

Adding JavaScript with Stimulus

The final piece of the Hotwire puzzle is Stimulus, which allows us to integrate JavaScript functionality in a neat way. Stimulus is a very simple JavaScript framework, it does not take control of the entire UI. In fact, it does not render any HTML at all. Stimulus essentially offers a nice way of wiring up JS behavior to existing markup. Let’s look at a quick example of how it could hypothetically be used for showing a confirmation popup before deleting a record.

The JavaScript logic lives inside so-called “Stimulus controllers”. For our example, it could look something like this:

// app/javascript/controllers/confirmations_controller.js
import { Controller } from "@hotwired/stimulus"

export default class extends Controller {
  confirm() {
    if (confirm("Are you sure you want to delete this record?")) {
      // Carry on with the operation...
    }
  }
}

And now, the idea is to wire up this code so that it runs when the hypothetical delete button is clicked. Here’s what that button could like:

<button
  data-controller="confirmations"
  data-action="click->confirmations#confirm"
>
  Delete
</button>

Pretty self-explanatory. We specify the name of the controller to use via the data-controller attribute. We also specify, via data-action, what method to invoke within that controller as a result of which DOM event, click in this case.

And that’s basically all it takes to sprinkle some JavaScript on Hotwire apps. Stimulus does offer a few more useful features, which you can read more about in the official documentation. But for us for now, it’s enough to know that it exists, and what’s the main idea behind it.

So as you can hopefully see, Hotwire is much smaller in scope to Blazor. And yet, it allows us to do just the same: highly interactive applications with little to no JavaScript. This ought to make a lot of Rails developers happy.

A final comparison

I was initially thinking about ending this blog post with a flowchart of sorts to explain the process of deciding which of these two technologies you should use. But really, the decision is very simple: If your team is comfortable with .NET, use Blazor. If your team is comfortable with Ruby on Rails, use Hotwire. It’s obvious, so I won’t claim to have made a great discovery here.

The only thing to add is that if your team is familiar with modern frontend web development framework concepts, you’ll be even better served by Blazor and you’ll hit the ground running. If not, then even for seasoned .NET people, there will be a decent learning curve, but not steep enough to be deterred. Moreover, if your team has no modern frontend development experience at all, then Hotwire is a godsend; thanks to its “augment classic backend-heavy web app development” style.

With that said, let’s close out with a summary of main aspects of both technologies and how they compare to each other.

Overall, Hotwire is much simpler than Blazor. While Blazor is a full-fledged GUI component framework, Hotwire’s approach is more like an augmentation of classic non-JavaScript web development patterns. That said, Hotwire’s style is more unusual than Blazor’s, so if your team is already familiar with modern frontend web development, Blazor can be a great fit.

While both frameworks try to offer enough functionality to allow the development of rich interactive experiences without the need to write any JavaScript, the reality is that sometimes JavaScript does need to be written. Both technologies offer ways to make this happen. And while both are perfectly workable, Blazor’s solution is a bit more clunky than Hotwire’s.

When it comes to classic web technologies like HTTP’s request/response cycle and the separation between server and client, Blazor’s style offers a big deviation from them. It greatly de-emphasizes them and presents instead a completely different programming model, one more akin to desktop application development. The concepts of request, response, client, and server seem to vanish. Not so for Hotwire, which builds upon these classic technologies in a way where they still need to be considered and are in fact in the spotlight. While Blazor attempts to do away with these, Hotwire embraces them.

In Blazor, client events are sent to the server, the server renders the DOM updates and sends them to the client for updates. This happens via the persistent SignalR/WebSockets connection.

Hotwire, on the other hand, intercepts client events and sends classic HTTP requests (via AJAX/XHR) to the server. The server then executes the requests and sends back the responses to the client which carries out the necessary operations, generally speaking, updating sections of the page that’s already being displayed.

That means that at the end of the day, both frameworks do the rendering on the server side and send the rendered markup over the wire to the clients. But in Blazor, the client and server have a persistent connection, while Hotwire’s connections come and go as normal HTTP requests and responses.

A neat aspect of Hotwire’s programming model is that it allows an incremental approach to web development where you can start developing the app like you would a traditional, non-reactive, non-JS app, then augment it with a little code to give it SPA capabilities.

And that’s all for now! I for one am glad to see these types of technologies emerge. While there are many teams out there that are already effective and productive with the current landscape of frontend web development, these two are very interesting and seem capable in their own right.

Besides, having alternatives is never a bad thing. Depending mainly on your previous experience, these could be a great fit for projects new and old. It’s great to know that both .NET and Rails include these types of offerings and that they work pretty well.

Implementing Backend Tasks in ASP.NET Core

2022-08-08T00:00:00+00:00

As we’ve already established, Ruby on Rails is great. The amount and quality of tools that Rails puts at our disposal when it comes to developing web applications is truly outstanding. One aspect of web application development that Rails makes particularly easy is that of creating backend tasks.

These tasks can be anything from database maintenance, file system cleanup, overnight heavy computations, bulk email dispatch, etc. In general, functionality that is typically initiated by a sysadmin in the backend, or scheduled in a cron job, which has no GUI, but rather, is invoked via command line.

By integrating with Rake, Rails allows us to very easily write such tasks as plain old Ruby scrips. These scripts have access to all the domain logic and data that the full-fledged Rails app has access to. The cherry on top is that the command-line interface to invoke such tasks is very straightforward. It looks something like this: bin/rails fulfillment:process_new_orders.

All this is included right out of the box for new Rails projects.

ASP.NET Core, which is also great, doesn’t support this out of the box like Rails does.

However, I think we should be able to implement our own without too much hassle, and have a similar sysadmin experience. Let’s see if we can do it.

There is a Table of contents at the end of this post.

What we want to accomplish

So, to put it in concrete terms, we want to create a backend task that has access to all the logic and data of an existing ASP.NET Core application. The task should be callable via command-line interface, so that it can be easily executed via the likes of cron or other scripts.

In order to meet these requirements, we will create a new .NET console app that:

References the existing ASP.NET Core project.
Loads all the classes from it and makes instances of them available via dependency injection.
Has a usable, Unix-like command-line interface that sysadmins would be familiar with.
Is invokable via the .NET CLI.

We will do all this within the context of an existing web application. One that I’ve been building upon thoughout a few articles.

It is a simple ASP.NET Web API backed by a Postgres database. It has a few endpoints for CRUDing automotive related data and for calculating values of vehicles based on various aspects of them.

You can find the code on GitHub. If you’d like to follow along, clone the repository and check out this commit: 9a078015ce. It represents the project as it was before applying all the changes from this article. The finished product can be found here.

You can follow the instructions in the project’s README file if you want to get the app up and running.

For our demo use case, we will try to develop a backend task that creates new user accounts for our existing application.

Let’s get to it.

Creating a new console app that references the existing web app as a library

The codebase is structured as a solution, as given away by the vehicle-quotes.sln file located at the root of the repository. Within this solution, there are two projects: VehicleQuotes which is the web app itself, and VehicleQuotes.Tests which contains the app’s test suite. For this article, we only care about the web app.

Like I said, the backend task that we will create is nothing fancy in itself. It’s a humble console app. So, we start by asking the dotnet CLI to create a new console app project for us.

From the repository’s root directory, we can do so with this command:

dotnet new console -o VehicleQuotes.CreateUser

That should’ve resulted in a new VehicleQuotes.CreateUser directory being created, and within it, (along with some other nuts and bolts) our new console app’s Program.cs (the code) and VehicleQuotes.CreateUser.csproj (the project definition) files. The name that we’ve chosen is straightforward: the name of the overall solution and the action that this console app is going to perform.

There’s more info regarding the dotnet new command in the official docs.

Now, since we’re using a solution file, let’s add our brand new console app project to it with:

dotnet sln add VehicleQuotes.CreateUser

OK, cool. That should’ve produced the following diff on vehicle-quotes.sln:

diff --git a/vehicle-quotes.sln b/vehicle-quotes.sln
index 537d864..5da277d 100644
--- a/vehicle-quotes.sln
+++ b/vehicle-quotes.sln
@@ -7,6 +7,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VehicleQuotes", "VehicleQuo
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VehicleQuotes.Tests", "VehicleQuotes.Tests\VehicleQuotes.Tests.csproj", "{5F6470E4-12AB-4E30-8879-3664ABAA959D}"
 EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VehicleQuotes.CreateUser", "VehicleQuotes.CreateUser\VehicleQuotes.CreateUser.csproj", "{EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}"
+EndProject
 Global
        GlobalSection(SolutionConfigurationPlatforms) = preSolution
                Debug|Any CPU = Debug|Any CPU
@@ -44,5 +46,17 @@ Global
                {5F6470E4-12AB-4E30-8879-3664ABAA959D}.Release|x64.Build.0 = Release|Any CPU
                {5F6470E4-12AB-4E30-8879-3664ABAA959D}.Release|x86.ActiveCfg = Release|Any CPU
                {5F6470E4-12AB-4E30-8879-3664ABAA959D}.Release|x86.Build.0 = Release|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|Any CPU.Build.0 = Debug|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x64.ActiveCfg = Debug|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x64.Build.0 = Debug|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x86.ActiveCfg = Debug|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x86.Build.0 = Debug|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|Any CPU.ActiveCfg = Release|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|Any CPU.Build.0 = Release|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x64.ActiveCfg = Release|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x64.Build.0 = Release|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x86.ActiveCfg = Release|Any CPU
+               {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x86.Build.0 = Release|Any CPU
        EndGlobalSection
 EndGlobal

This allows the .NET tooling to know that we’ve got some intentional organization going on in our code base. That these projects each form part of a bigger whole.

It’s also nice to add a .gitignore file for our new VehicleQuotes.CreateUser project to keep things manageable. dotnet new can help with that if we were to navigate into the VehicleQuotes.CreateUser directory and run:
dotnet new gitignore

You can learn more about how to work with solutions via the .NET CLI in the official docs.

Now let’s modify our new project’s .csproj file so that it references the main web app project under VehicleQuotes. This will allow our console app to access all of the classes defined in the web app, as if it was a library or package.

If we move to the VehicleQuotes.CreateUser directory, we can do that with the following command:

dotnet add reference ../VehicleQuotes/VehicleQuotes.csproj

The command itself is pretty self-explanatory. It just expects to be given the .csproj file of the project that we want to add as a reference in order to do its magic.

Running that should’ve added the following snippet to VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj:

<ItemGroup>
  <ProjectReference Include="..\VehicleQuotes\VehicleQuotes.csproj" />
</ItemGroup>

This way, .NET allows the code defined in the VehicleQuotes project to be used within the VehicleQuotes.CreateUser project.

You can learn more about the add reference command in the official docs.

Setting up dependency injection in the console app

As a result of the previous steps, our new console app now has access to the classes defined within the web app. However, classes by themselves are no good if we can’t actually create instances of them that we can interact with. The premier method for making instances of classes available throughout a .NET application is via dependency injection. So, we need to set that up for our little console app.

Dependency injection is something that comes out of the box for ASP.NET Core web apps. Luckily for us, .NET makes it fairly easy to set it up in console apps as well by leveraging the same components.

For this app, we want to create user accounts. In the web app, user account management is done via ASP.NET Core Identity. Specifically, the UserManager class is used to create new user accounts. This console app will do the same.

Take a look at VehicleQuotes/Controllers/UsersController.cs to see how the user accounts are created. If you’d like to know more about integrating ASP.NET Core Identity into an existing web app, I wrote an article about it.

Before we do the dependency injection setup, let’s add a new class to our console app project that will encapsulate the logic of leveraging the UserManager for user account creation. This is the actual task that we want to perform. The new class will be defined in VehicleQuotes.CreateUser/UserCreator.cs and these will be its contents:

using Microsoft.AspNetCore.Identity;

namespace VehicleQuotes.CreateUser;

class UserCreator
{
    private readonly UserManager<IdentityUser> _userManager;

    public UserCreator(UserManager<IdentityUser> userManager) {
        _userManager = userManager;
    }

    public IdentityResult Run(string username, string email, string password)
    {
        var userCreateTask = _userManager.CreateAsync(
            new IdentityUser() { UserName = username, Email = email },
            password
        );

        var result = userCreateTask.Result;

        return result;
    }
}

This class is pretty lean. All it does is define a constructor that expects an instance of UserManager<IdentityUser>, which will be supplied via dependency injection; and a simple Run method that, when given a username, email, and password, asks the UserManager<IdentityUser> instance that it was given to create a user account.

Moving on to setting up dependency injection now, we will do it in VehicleQuotes.CreateUser/Program.cs. Replace the contents of that file with this:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using VehicleQuotes.CreateUser;

IHost host = Host.CreateDefaultBuilder(args)
    .UseContentRoot(System.AppContext.BaseDirectory)
    .ConfigureServices((context, services) =>
    {
        var startup = new VehicleQuotes.Startup(context.Configuration);
        startup.ConfigureServices(services);

        services.AddTransient<UserCreator>();
    })
    .Build();

var userCreator = host.Services.GetRequiredService<UserCreator>();
userCreator.Run(args[0], args[1], args[2]);

Let’s dissect this bit by bit.

First off, we’ve got a few using statements that we need in order to access some classes and extension methods that we need down below.

Next, we create and configure a new IHost instance. .NET Core introduced the concept of a “host” as an abstraction for programs; and packed in there a lot of functionality to help with things like configuration, logging and, most importantly for us, dependency injection. To put it simply, the simplest way of enabling dependency injection in a console app is to use a Host and all the goodies that come within.

There’s much more information about hosts in .NET’s official documentation.

Host.CreateDefaultBuilder(args) gives us an IHostBuilder instance that we can use to configure our host. In our case, we’ve chosen to call UseContentRoot(System.AppContext.BaseDirectory) on it, which makes it possible for the app to find assets (like appconfig.json files!) regardless of where its deployed and where its being called from.

This is important for us because, as you will see later, we will install this console app as a .NET Tool. .NET Tools are installed in directories picked by .NET and can be run from anywhere in the system. So we need to make sure that our app can find its assets wherever it has been installed.

After that, we call ConfigureServices where we do a nice trick in order to make sure our console app has all the same configuration as the web app as far as dependency injection goes.

You see, in ASP.NET Core, all the service classes that are to be made available to the application via dependency injection are configured within the web app’s Startup class’ ConfigureServices method. VehicleQuotes is no exception. So, in order for our console app to have access to all of the services (i.e. instances of classes) that the web app does, the console app needs to call that same code. And that’s exactly what’s happening in these two lines:

var startup = new VehicleQuotes.Startup(context.Configuration);
startup.ConfigureServices(services);

We create a new instance of the web app’s Startup class and call its ConfigureServices method. That’s the key element that allows the console app to have access to all the logic that the web app does. Including the services/classes provided by ASP.NET Core Identity like UserManager<IdentityUser>, which UserCreator needs in order to function.

Once that’s done, the rest is straightforward.

We also add our new UserCreator to the dependency injection engine via:

services.AddTransient<UserCreator>();

Curious about what Transient means? The official .NET documentation has the answer.

And that allows us to obtain an instance of it with:

var userCreator = host.Services.GetRequiredService<UserCreator>();

And then, it’s just a matter of calling its Run method like so, passing it the command-line arguments:

userCreator.Run(args[0], args[1], args[2]);

args is a special variable that contains an array with the arguments given by command line. That means that our console app can be called like this:

dotnet run test_username test_email@email.com mysecretpassword

Go ahead, you can try it out and see the app log what it’s doing. Once done, it will also have created a new record in the database.

$ psql -h localhost -U vehicle_quotes
psql (14.3 (Ubuntu 14.3-0ubuntu0.22.04.1), server 14.2 (Debian 14.2-1.pgdg110+1))
Type "help" for help.

vehicle_quotes=# select user_name, email from public."AspNetUsers";
   user_name   |           email            
---------------+----------------------------
 test_username | test_email@email.com
(1 rows)

Pretty neat, huh? At this point we have a console app that creates user accounts for our existing web app. It works, but it could be better. Let’s add a nice command-line interface experience now.

Improving the CLI with CommandLineParser

With help from CommandLineParser, we can develop a Unix-like command-line interface for our app. We can use it to add help text, examples, have strongly typed parameters and useful error messages when said parameters are not correctly provided. Let’s do that now.

First, we need to install the package in our console app project by running the following command from within the project’s directory (VehicleQuotes.CreateUser):

dotnet add package CommandLineParser

After that’s done, a new section will have been added to VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj that looks like this:

<ItemGroup>
  <PackageReference Include="CommandLineParser" Version="2.9.1" />
</ItemGroup>

Now our console app can use the classes provided by the package.

All specifications for CommandLineParser are done via a plain old C# class that we need to define. For this console app, which accepts three mandatory arguments, such a class could look like this:

using CommandLine;
using CommandLine.Text;

namespace VehicleQuotes.CreateUser;

class CliOptions
{
    [Value(0, Required = true, MetaName = "username", HelpText = "The username of the new user account to create.")]
    public string Username { get; set; }

    [Value(1, Required = true, MetaName = "email", HelpText = "The email of the new user account to create.")]
    public string Email { get; set; }

    [Value(2, Required = true, MetaName = "password", HelpText = "The password of the new user account to create.")]
    public string Password { get; set; }

    [Usage(ApplicationAlias = "create_user")]
    public static IEnumerable<Example> Examples
    {
        get
        {
            return new List<Example> {
                new (
                    "Create a new user account",
                    new CliOptions { Username = "name", Email = "email@domain.com", Password = "secret" }
                )
            };
        }
    }
}

I’ve decided to name it CliOptions but really, it could have been anything. Go ahead and create it in VehicleQuotes.CreateUser/CliOptions.cs. There are a few interesting elements to note here.

The key aspect is that we have a few properties: Username, Email, and Password. These represent our three command-line arguments. Thanks to the Value attributes that they have been annotated with, CommandLineParser will know that that’s their purpose. You can see how the attributes themselves also contain each argument’s specification like the order in which they should be supplied, as well as their name and help text.

This class also defines an Examples getter which is used by CommandLineParser to print out usage examples into the console when our app’s help option is invoked.

Other than that, the class itself is unremarkable. In summary, it’s a number of fields annotated with attributes so that CommandLineParser knows what to do with it.

In order to actually put it to work, we update our VehicleQuotes.CreateUser/Program.cs like so:

 using Microsoft.Extensions.DependencyInjection;
 using Microsoft.Extensions.Hosting;
+using CommandLine;
 using VehicleQuotes.CreateUser;
 
+void Run(CliOptions options)
+{
     IHost host = Host.CreateDefaultBuilder(args)
         .UseContentRoot(System.AppContext.BaseDirectory)
         .ConfigureServices((context, services) =>
         {
             var startup = new VehicleQuotes.Startup(context.Configuration);
             startup.ConfigureServices(services);
 
             services.AddTransient<UserCreator>();
         })
         .Build();
 
     var userCreator = host.Services.GetRequiredService<UserCreator>();
-    userCreator.Run(args[0], args[1], args[2]);
+    userCreator.Run(options.Username, options.Email, options.Password);
+}
+
+Parser.Default
+    .ParseArguments<CliOptions>(args)
+    .WithParsed(options => Run(options));

We’ve wrapped Program.cs’s original code into a method simply called Run.

Also, we’ve added this snippet at the bottom of the file:

Parser.Default
    .ParseArguments<CliOptions>(args)
    .WithParsed(options => Run(options));

That’s how we ask CommandLineParser to parse the incoming CLI arguments, as specified by CliOptions and, if it can be done successfully, then execute the rest of the program by calling the Run method.

Neatly, we also no longer have to use the args array directly in order to get the command-line arguments provided to the app, instead we use the options object that CommandLineParser creates for us once it has done its parsing. You can see it in this line:

userCreator.Run(options.Username, options.Email, options.Password);

options is an instance of our very own CliOptions class, so we can access the properties that we defined within it. These contain the arguments that were passed to the program.

If you were to try dotnet run right now, you’d see the following output:

VehicleQuotes.CreateUser 1.0.0
Copyright (C) 2022 VehicleQuotes.CreateUser

ERROR(S):
  A required value not bound to option name is missing.
USAGE:
Create a new user account:
  create_user name email@domain.com secret

  --help               Display this help screen.

  --version            Display version information.

  username (pos. 0)    Required. The username of the new user account to create.

  email (pos. 1)       Required. The email of the new user account to create.

  password (pos. 2)    Required. The password of the new user account to create.

As you can see, CommandLineParser detected that no arguments were given, and as such, it printed out an error message, along with the descriptions, help text and example that we defined. Basically the instructions on how to use our console app.

Deploying the console app as a .NET tool

OK, we now have a console app that does what it needs to do, with a decent interface. The final step is to make it even more accessible by deploying it as a .NET tool. If we do that, we’d be able to invoke it with a command like this:

dotnet create_user Kevin kevin@gmail.com secretpw

.NET makes this easy for us. There’s a caveat that we’ll discuss later, but for now, let’s go through the basic setup.

.NET tools are essentially just glorified NuGet packages. As such, we begin by adding some additional package-related configuration options to VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj. We add them as children elements to the <PropertyGroup>:

<PackAsTool>true</PackAsTool>
<PackageOutputPath>./nupkg</PackageOutputPath>
<ToolCommandName>create_user</ToolCommandName>
<VersionPrefix>1.0.0</VersionPrefix>

With that, we signal .NET that we want the console app to be packed as a tool, the path where it should put the package itself, and what its name will be. That is, how will it be invoked via the console (remember we want to be able to do dotnet create_user).

Finally, we specify a version number. When dealing with NuGet packages, versioning them is very important, as that drives caching and downloading logic in NuGet. More on that later when we talk about the aforementioned caveats.

Now, to build the package, we use:

dotnet pack

That will build the application and produce a VehicleQuotes.CreateUser/nupkg/VehicleQuotes.CreateUser.1.0.0.nupkg file.

We won’t make the tool available for the entire system. Instead, we will make it available from within our solution’s directory only. We can make that happen if we create a tool manifest file in the source code’s root directory. That’s done with this command, run from the root directory:

dotnet new tool-manifest

That should create a new file: .config/dotnet-tools.json.

Now, also from the root directory, we can finally install our tool:

dotnet tool install --add-source ./VehicleQuotes.CreateUser/nupkg VehicleQuotes.CreateUser

This is the regular command to install any tools in .NET. The interesting part is that we use the --add-source option to point it to the path where our freshly built package is located.

After that, .NET shows this output:

$ dotnet tool install --add-source ./VehicleQuotes.CreateUser/nupkg VehicleQuotes.CreateUser
You can invoke the tool from this directory using the following commands: 'dotnet tool run create_user' or 'dotnet create_user'.
Tool 'vehiclequotes.createuser' (version '1.0.0') was successfully installed. Entry is added to the manifest file /path/to/solution/.config/dotnet-tools.json.

It tells us all we need to know. Check out the .config/dotnet-tools.json to see how the tool has been added there. All this means that now we can run our console app as a .NET tool:

$ dotnet create_user --help
VehicleQuotes.CreateUser 1.0.0
Copyright (C) 2022 VehicleQuotes.CreateUser
USAGE:
Create a new user account:
  create_user name email@domain.com secret

  --help               Display this help screen.

  --version            Display version information.

  username (pos. 0)    Required. The username of the new user account to create.

  email (pos. 1)       Required. The email of the new user account to create.

  password (pos. 2)    Required. The password of the new user account to create.

Pretty sweet, huh? And yes, it has taken a lot more effort than what it would’ve taken in Ruby on Rails, but hey, the end result is pretty fabulous I think, and we learned a new thing. Besides, once you’ve done it once, the skeleton can be easily reused for all kinds of different backend tasks.

Now, before we wrap this up, there’s something we need to consider when actively developing these tools. That is, when making changes and re-installing constantly.

The main aspect to understand is that tools are just NuGet packages, and as such are beholden to the NuGet package infrastructure. Which includes caching. If you’re in the process of developing your tool and are quickly making and deploying changes, NuGet won’t update the cache unless you do one of two things:

Manually clear it with a command like dotnet nuget locals all --clear.
Bump up the version of the tool by updating the value of <VersionSuffix> in the project (.csproj) file.

This means that, unless you do one these, the changes that you make to the app between re-builds (with dotnet pack) and re-installs (with dotnet dotnet tool install) won’t ever make their way to the package that’s actually installed in your system. So be sure to keep that in mind.

Implementing Authentication in ASP.NET Core Web APIs

2022-06-17T00:00:00+00:00

Authentication is a complex space. There are many problem scenarios and many more solutions. When it comes to Web APIs written with ASP.NET Core, there are various fully featured options like Duende IdentityServer or Azure Active Directory. These promise to be “everything but the kitchen sink” solutions which are robust and allow you to deal with many complex requirements.

But what if our requirements dictate that we need something simpler? Do we have to roll out our own from scratch? Or does ASP.NET Core offer smaller, customizable, somewhat independent puzzle pieces that we can put together without having to write all the code ourselves and still have a good amount of control?

Spoiler alert: The answer to that last question is yes. And we’re going to talk about it in this very article.

There is a Table of contents at the end of this post.

Two approaches to authentication: JWT and API Keys

In this article, we’ll take an existing ASP.NET Core Web API and add authentication capabilities to it. Specifically, we’ll support two authentication schemes commonly used for Web APIs: JWT and API Keys. Also, we will use our own database for storage of user accounts and credentials.

The project that we will work with is a simple ASP.NET Web API backed by a Postgres database. It has a few endpoints for CRUDing automotive related data and for calculating values of vehicles based on various aspects of them. You can read all about the process of building it here. I also added a few database integration tests for it here.

You can find it on GitHub. If you’d like to follow along, clone the repository and checkout this commit: cd290c765fcd2c6693008d3dc76fa931098dcaa0. It represents the project as it was before applying all the changes from this article.

You can follow the instructions in the project’s README file in order to get the app up and running.

Managing user accounts with ASP.NET Core Identity

Let’s deal first with the requirement of storing the user accounts in our own database.

Luckily for us, ASP.NET Core provides us with Identity. This is an API that offers a comprehensive solution for authentication. It can connect with the aforementioned Duende IdentityServer for OpenID Connect and OAuth 2.0, supports authentication via third parties like Facebook or Google, and can fully integrate with user interfaces on web apps, complete with scaffolding/autogeneration capabilities too.

Most importantly for us, it supports management of user accounts stored in our own database. It includes data for third party logins, passwords, roles for authorization, email confirmation, access tokens, etc.

That’s a lot of of functionality baked in there. But we don’t need all that, so let’s see how we can take only the bits and pieces that we need in order to fulfill our specific requirements.

Specifically, we want:

Tables in our database for storage of user accounts and passwords.
A programmatic way to create, fetch and validate users using those tables.

It’s actually pretty simple.

Install the necessary NuGet packages

First, we need to install a couple of NuGet packages:

Microsoft.AspNetCore.Identity which contains the core library.
Microsoft.AspNetCore.Identity.EntityFrameworkCore which includes the classes that Identity needs in order to properly interact with Entity Framework, which is what we’re using in our Web API for interfacing with the database.

We can install them by running these two commands from the VehicleQuotes directory:

$ dotnet add package Microsoft.AspNetCore.Identity
$ dotnet add package Microsoft.AspNetCore.Identity.EntityFrameworkCore

That will add the following lines to the VehicleQuotes/VehicleQuotes.csproj project file:

 <Project Sdk="Microsoft.NET.Sdk.Web">
   ...
   <ItemGroup>
     ...
+    <PackageReference Include="Microsoft.AspNetCore.Identity" Version="2.2.0" />
+    <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="6.0.5" />
     ...
   </ItemGroup>
 </Project>

Update the DbContext to include the Identity tables

Next step is to configure our DbContext class so that it includes the new tables that we need from the Identity library. So let’s go to VehicleQuotes/Data/VehicleQuotesContext.cs and update it to do so.

We need to include these new using statements at the top of the file so that we have access to the classes that we need from the NuGet packages we just installed:

using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;

Next, instead of DbContext, the VehicleQuotesContext class should inherit from IdentityUserContext<IdentityUser>.

IdentityUserContext is a class provided by ASP.NET Core Identity that’s designed so that our DbContext can inherit from it and gain user management functionality. Namely, it includes a new DbSet (and consequently, a table) for holding user accounts (aptly named Users), among other things that we won’t need.

IdentityUserContext has a more feature rich counterpart called IdentityDbContext, which also includes DbSets to support roles based authorization. We don’t need all that so we use its simpler cousin. Feel free to explore the source code on GitHub to see all it offers.

The generic type parameter that we give it, IdentityUser, is a class that’s also provided by the Identity library. Its purpose is to serve as a default Entity Type for our user model and, as a consequence, our Users DbSet.

In summary, by having our DbContext class inherit from IdentityUserContext<IdentityUser>, we’re telling Identity that we want it to augment our DbContext (and database) to include the core user management tables and that we want our users table to have the same columns as IdentityUser.

If we wanted to include more columns in our users table, what we would have to do is create a new class, make it inherit from IdentityUser, define any additional fields that we want on it, and use that class as a type parameter to IdentityUserContext. For us for now, the default works just fine. You can learn more about customizing Identity in the official docs.

The change looks like this:

-public class VehicleQuotesContext : DbContext
+public class VehicleQuotesContext : IdentityUserContext<IdentityUser>

Finally, IdentityUserContext has some logic that it needs to run when it is being created. In order to allow it to run that logic, let’s add the following line to our VehicleQuotesContext’s OnModelCreating method:

 protected override void OnModelCreating(ModelBuilder modelBuilder)
 {
+    base.OnModelCreating(modelBuilder);
     // ...
 }

This calls IdentityUserContext’s own OnModelCreating implementation so that it can set itself up properly.

The complete file should be looking like this now:

using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
using VehicleQuotes.Models;

namespace VehicleQuotes
{
    public class VehicleQuotesContext : IdentityUserContext<IdentityUser>
    {
        public VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
            : base(options)
        {
        }

        public DbSet<Make> Makes { get; set; }
        public DbSet<Size> Sizes { get; set; }
        public DbSet<BodyType> BodyTypes { get; set; }

        public DbSet<Model> Models { get; set; }
        public DbSet<ModelStyle> ModelStyles { get; set; }
        public DbSet<ModelStyleYear> ModelStyleYears { get; set; }

        public DbSet<QuoteRule> QuoteRules { get; set; }
        public DbSet<QuoteOverride> QuoteOverides { get; set; }

        public DbSet<Quote> Quotes { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);

            modelBuilder.Entity<Size>().HasData(
                new Size { ID = 1, Name = "Subcompact" },
                new Size { ID = 2, Name = "Compact" },
                new Size { ID = 3, Name = "Mid Size" },
                new Size { ID = 5, Name = "Full Size" }
            );

            modelBuilder.Entity<BodyType>().HasData(
                new BodyType { ID = 1, Name = "Coupe" },
                new BodyType { ID = 2, Name = "Sedan" },
                new BodyType { ID = 3, Name = "Hatchback" },
                new BodyType { ID = 4, Name = "Wagon" },
                new BodyType { ID = 5, Name = "Convertible" },
                new BodyType { ID = 6, Name = "SUV" },
                new BodyType { ID = 7, Name = "Truck" },
                new BodyType { ID = 8, Name = "Mini Van" },
                new BodyType { ID = 9, Name = "Roadster" }
            );
        }
    }
}

Applying database changes

The Users DbSet that we added into our data model by inheriting from IdentityUserContext<IdentityUser> will become a table once we create and apply a database migration. That’s what we will do next.

That’s simple in ASP.NET Core. All we need to do is run a command like this:

dotnet ef migrations add AddIdentityTables

That should produce a new migration file named something like 20220605003253_AddIdentityTables.cs. If you explore it, you’ll see how it contains the definitions for a few new database tables. Including the one that we want: AspNetUsers. That’s the one that we will use to store our user account records.

Next, we apply the migration with:

dotnet ef database update

If you now connect to the database, you’ll see the new tables in there.

If you have the database running in a Docker container like we discussed in the beginning of the article, you should be able to connect to it with:

$ psql -h localhost -U vehicle_quotes

Then, to see the tables:

vehicle_quotes=# \dt
                    List of relations
 Schema |         Name          | Type  |     Owner      
--------+-----------------------+-------+----------------
 public | AspNetUserClaims      | table | vehicle_quotes
 public | AspNetUserLogins      | table | vehicle_quotes
 public | AspNetUserTokens      | table | vehicle_quotes
 public | AspNetUsers           | table | vehicle_quotes
 ...
(14 rows)

And that’s pretty much it when it comes to having a sensible storage for user accounts. That was pretty inexpensive, wasn’t it? All we had to do was install some NuGet packages, tweak our existing DbContext, and run some migrations.

The best thing is that we’re not done yet. We can also take advantage of ASP.NET Core Identity to manage users programmatically. Instead of interacting with these tables directly, we will use the service classes provided by Identity to create new users, fetch existing ones and validate their credentials.

Before we can do that though, we must first do some configuration in our app’s Startup class so that said services are properly set up to our liking and are made available to our application.

Configuring the Identity services

We need to add some code to VehicleQuotes/Startup.cs. With it, we configure the Identity system and add a few service classes to ASP.NET Core’s IoC container so that they are available to our app via Dependency Injection.

We need a new using statement:

using Microsoft.AspNetCore.Identity;

And the following code added to the ConfigureServices method:

services
    .AddIdentityCore<IdentityUser>(options => {
        options.SignIn.RequireConfirmedAccount = false;
        options.User.RequireUniqueEmail = true;
        options.Password.RequireDigit = false;
        options.Password.RequiredLength = 6;
        options.Password.RequireNonAlphanumeric = false;
        options.Password.RequireUppercase = false;
        options.Password.RequireLowercase = false;
    })
    .AddEntityFrameworkStores<VehicleQuotesContext>();

The call to AddIdentityCore makes several Identity utility classes available to the application. Among those, UserManager is the only one we will use. We will use it later to… well, manage users. You can also see how we’ve set a few options related to how the user accounts are handled. options.SignIn.RequireConfirmedAccount controls whether new accounts need to be confirmed via email before they are available. With options.User.RequireUniqueEmail, we tell Identity to enforce uniqueness of emails on user accounts. And finally the options.Password.* options configure the password strength requirements.

You can explore all the available options in the official docs.

Then, the call to AddEntityFrameworkStores tells the Identity system that it should use our VehicleQuotesContext for data storage.

Creating users

With that configuration out of the way, we can now write some code to create new user accounts. To keep things simple, we’ll add a new UsersController to our project that will expose a new endpoint that offers that functionality.

Let’s start with this in VehicleQuotes/Controllers/UsersController.cs:

using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Mvc;

namespace VehicleQuotes.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class UsersController : ControllerBase
    {
        private readonly UserManager<IdentityUser> _userManager;

        public UsersController(
            UserManager<IdentityUser> userManager
        ) {
            _userManager = userManager;
        }
    }
}

As you can see, this controller defines a dependency on UserManager<IdentityUser>, an instance of which is injected via the constructor. This is one of the classes made available to us when we configured the Identity core services in our app’s Startup. We will use it to create new user records.

Before that though, we need to define a new class that encapsulates the payload for a request to our endpoint. When it comes to creating user accounts, all we need is a username, a password, and an email. As such, we add the following class in a new VehicleQuotes/ResourceModels/User.cs file:

using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.ResourceModels
{
    public class User
    {
        [Required]
        public string UserName { get; set; }
        [Required]
        public string Password { get; set; }
        [Required]
        public string Email { get; set; }
    }
}

Now, we shall use this type as the parameter for a new PostUser action method in the UserController which will expose the new user account creation endpoint in our API. The method looks like this:

// POST: api/Users
[HttpPost]
public async Task<ActionResult<User>> PostUser(User user)
{
    if (!ModelState.IsValid)
    {
        return BadRequest(ModelState);
    }

    var result = await _userManager.CreateAsync(
        new IdentityUser() { UserName = user.UserName, Email = user.Email },
        user.Password
    );

    if (!result.Succeeded)
    {
        return BadRequest(result.Errors);
    }

    user.Password = null;
    return Created("", user);
}

Be sure to also add the following using statements:

using System.Threading.Tasks;
using VehicleQuotes.ResourceModels;

System.Threading.Tasks allows us to reference the class Task<> which is the return type of the new PostUser method. VehicleQuotes.ResourceModels is where our new User class lives.

Thanks to the instance of UserManager<IdentityUser> that we’re holding onto, this method is very straightforward. The most interesting portion is this:

var result = await _userManager.CreateAsync(
    new IdentityUser() { UserName = user.UserName, Email = user.Email },
    user.Password
);

Here we’re newing up an IdentityUser instance using the given request parameters and passing it on to UserManager<IdentityUser>’s CreateAsync method, along with the password that was also given in the incoming request. This puts the Identity system to work for us and properly create a new user account.

Then, we can inspect its return value (which we capture in the result variable) to determine if the operation was successful. That way we can respond appropriately to our API’s caller.

Finally, with…

user.Password = null;
return Created("", user);

we return the data representing the newly created user account but we’re discreet and make sure not to include the password.

With that, we can test our API. Fire it up with dotnet run (or dotnet watch!) and send a POST request to our new endpoint at http://0.0.0.0:5000/api/Users with a payload like this:

{
  "userName": "newuser000",
  "email": "newuser000@endpointdev.com",
  "password": "password"
}

I just tried it in Postman and this is what it looked like:

Feel free to test it out further. Try repeated emails or usernames. Try passwords that don’t meet the criteria we defined when configuring Identity in the app’s Startup class. It all works as you’d expect.

You can inspect the database and see the newly created record too:

vehicle_quotes=# select id, user_name, email, password_hash from "AspNetUsers";
-[ RECORD 1 ]-+--------------------------------------
id            | aaad07b4-f109-4255-8caa-185fd7694c72
user_name     | newuser000
email         | newuser000@endpointdev.com
password_hash | AQAAAAEAACcQAAAAEJJTV7M2Ejqd3K3iC...

And there it is, that’s our record. With a hashed password and everything.

Now let’s see what would an endpoint to fetch users look like:

Fetching users

UserManager<IdentityUser> offers a FindByNameAsync method that we can use to retrieve users by name. We can add a new endpoint that leverages it like so:

// GET: api/Users/username
[HttpGet("{username}")]
public async Task<ActionResult<User>> GetUser(string username)
{
    IdentityUser user = await _userManager.FindByNameAsync(username);

    if (user == null)
    {
        return NotFound();
    }

    return new User
    {
        UserName = user.UserName,
        Email = user.Email
    };
}

This is even more straightforward than the previous one. We just call the FindByNameAsync method by giving it the username of the account we want, make sure that we actually found it, and then return the data.

For the response, we use the same User class that we created to represent the input for the creation endpoint. If we added more fields to the user profile, we could include them here. Alas, we only have username and email for now.

Restart the app and we can now make a request like this:

Pretty neat, huh? Try a username that does not exist and you should see the API respond with a 404.

One quick improvement that we can do before we move on: let’s change PostUser’s return statement to this:
return CreatedAtAction("GetUser", new { username = user.UserName }, user);
All that does is include a Location header on the POST Users endpoint’s response that contains the URL for the newly created user. That’s just being a good citizen.

Implementing JWT Bearer Token authentication

Now that we have the user management capabilities that we need, let’s implement some actual authentication. We will start by adding JWT-based authentication to our API.

The strategy that we will use is to create a new API endpoint that clients can POST credentials to and that will respond to them with a fresh, short-lived token. They will then be able to use that token for subsequent requests by including it via headers.

We will pick a random endpoint to secure, just to serve as an example. That is, an endpoint that requires authentication in order to be accessed. GET api/BodyTypes is a good candidate. It is defined in VehicleQuotes/Controllers/BodyTypesController.cs’s GetBodyTypes action method. Feel free to test it out.

Creating tokens

Let’s start by creating the new endpoint that clients will call to obtain the auth tokens. In order to do so, we need a few things:

A class that represents incoming request data.
A class that represents outgoing response data.
A class that can generate tokens for a given user.
A new action method in UsersController that does the work.

The request data structure

To deal with step one, let’s define a new class in VehicleQuotes/ResourceModels/AuthenticationRequest.cs that looks like this:

using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.ResourceModels
{
    public class AuthenticationRequest
    {
        [Required]
        public string UserName { get; set; }
        [Required]
        public string Password { get; set; }
    }
}

All users need to authenticate is provide a set of credentials: username and password. So we have this class that contains fields for both and that will be used as input for our endpoint.

The response data structure

Next, we need to define a class to represent that endpoint’s response. I’ve added the following class in VehicleQuotes/ResourceModels/AuthenticationResponse.cs:

using System;

namespace VehicleQuotes.ResourceModels
{
    public class AuthenticationResponse
    {
        public string Token { get; set; }
        public DateTime Expiration { get; set; }
    }
}

Just a simple data structure containing the token itself and a date letting clients know when they can expect it to expire.

The class that creates JWTs

Step 3 is creating a class that can produce the tokens. This is the most interesting part in terms of complexity. Before we can do that though, let’s add the following configurations to VehicleQuotes/appsettings.json:

  "Jwt": {
    "Key": "this is the secret key for the jwt, it must be kept secure",
    "Issuer": "vehiclequotes.endpointdev.com",
    "Audience": "vehiclequotes.endpointdev.com",
    "Subject": "JWT for vehiclequotes.endpointdev.com"
  },

These are values that we’ll need when creating the tokens. We’ll go over the purpose of each them as they come up as we continue writing our code.

Here we’ll gloss over some details on the inner workings of JWTs as a standard. You can learn more about them at jwt.io.

For now, let’s add the following class in VehicleQuotes/Services/JwtService.cs:

using System;
using System.IdentityModel.Tokens.Jwt;
using System.Security.Claims;
using System.Text;
using Microsoft.AspNetCore.Identity;
using Microsoft.Extensions.Configuration;
using Microsoft.IdentityModel.Tokens;
using VehicleQuotes.ResourceModels;

namespace VehicleQuotes.Services
{
    public class JwtService
    {
        private const int EXPIRATION_MINUTES = 1;

        private readonly IConfiguration _configuration;

        public JwtService(IConfiguration configuration)
        {
            _configuration = configuration;
        }

        public AuthenticationResponse CreateToken(IdentityUser user)
        {
            var expiration = DateTime.UtcNow.AddMinutes(EXPIRATION_MINUTES);

            var token = CreateJwtToken(
                CreateClaims(user),
                CreateSigningCredentials(),
                expiration
            );

            var tokenHandler = new JwtSecurityTokenHandler();

            return new AuthenticationResponse {
                Token = tokenHandler.WriteToken(token),
                Expiration = expiration
            };
        }

        private JwtSecurityToken CreateJwtToken(Claim[] claims, SigningCredentials credentials, DateTime expiration) =>
            new JwtSecurityToken(
                _configuration["Jwt:Issuer"],
                _configuration["Jwt:Audience"],
                claims,
                expires: expiration,
                signingCredentials: credentials
            );

        private Claim[] CreateClaims(IdentityUser user) =>
            new[] {
                new Claim(JwtRegisteredClaimNames.Sub, _configuration["Jwt:Subject"]),
                new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
                new Claim(JwtRegisteredClaimNames.Iat, DateTime.UtcNow.ToString()),
                new Claim(ClaimTypes.NameIdentifier, user.Id),
                new Claim(ClaimTypes.Name, user.UserName),
                new Claim(ClaimTypes.Email, user.Email)
            };

        private SigningCredentials CreateSigningCredentials() =>
            new SigningCredentials(
                new SymmetricSecurityKey(
                    Encoding.UTF8.GetBytes(_configuration["Jwt:Key"])
                ),
                SecurityAlgorithms.HmacSha256
            );
    }
}

The CreateToken method is the element that’s most worth discussing here. It receives an instance of IdentityUser as a parameter (which, remember, is the entity class that represents our user accounts), uses it to construct a JWT, and returns it within a AuthenticationResponse object (which is what we decided that our endpoint would return). To do so, we use various classes built into .NET.

The main class that represents the JWT is JwtSecurityToken. We new up one of those in the CreateJwtToken method with this call:

new JwtSecurityToken (
    _configuration["Jwt:Issuer"],
    _configuration["Jwt:Audience"],
    claims,
    expires: expiration,
    signingCredentials: credentials
);

“Issuer” and “Audience” are two important values for how JWTs work. They specify which entity is creating the token (i.e. the “Issuer”) and which entity is the token intended for (i.e. the “Audience”). We use the IConfiguration instance that we got as a dependency to fetch their values from the VehicleQuotes/appsettings.json file.

The “Issuer” and “Audience” parameters in JwtSecurityToken’s constructor correspond to JWT claims iss and aud, respectively. You can learn more about them and other claims in the RFC.

The next parameter that JwtSecurityToken’s constructor needs is the claims array. In JWT terms, a claim is essentially a statement about the entity for which the token is generated, some data that identifies it. For example, if we’re generating a token for a user, what you would expect to see in such a token’s claims are things like username, email, and any other non-secret profile info.

In our case, as you can see in the CreateClaims method, we add a number of claims:

new[] {
    new Claim(JwtRegisteredClaimNames.Sub, _configuration["Jwt:Subject"]),
    new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
    new Claim(JwtRegisteredClaimNames.Iat, DateTime.UtcNow.ToString()),
    new Claim(ClaimTypes.NameIdentifier, user.Id),
    new Claim(ClaimTypes.Name, user.UserName),
    new Claim(ClaimTypes.Email, user.Email)
};

Along with the user id, name and email, we also add sub, jti and iat claims. These are standardized claims of which you can learn more about in the RFC. The data that we put in here will make its way into the encoded token that our API caller eventually sees. We’ll see that later.

The next parameter to JwtSecurityToken is the expiration date of the token. Here we are setting it to just one minute in the future to comply with our original requirement that the token should be short-lived so that it only allows a handful of requests over a short period of time.

Finally there’s the signingCredentials parameter which tells the JwtSecurityToken how to cryptographically sign the token. As you can see in the code:

new SigningCredentials(
    new SymmetricSecurityKey(
        Encoding.UTF8.GetBytes(_configuration["Jwt:Key"])
    ),
    SecurityAlgorithms.HmacSha256
);

That’s a SigningCredentials instance that we create using the “key” that we have configured in VehicleQuotes/appsettings.json, along with the algorithm to use to produce it.

And that’s about it. There isn’t much else to this class. It is somewhat involved but that’s just how JWTs are created in .NET.

The action method that puts it all together

Now all we need to do is actually create an endpoint that exposes this functionality to clients. Since we have the core logic encapsulated in our JwtService class, the actual action method is simple. Here are the changes that we need to make in order to implement it:

First, on VehicleQuotes/Controllers/UsersController.cs, we add a new using statement so that we can reference our new JwtService class:

using VehicleQuotes.Services;

Next, we declare a new parameter of type JwtService in the controller’s constructor so that we signal to ASP.NET Core’s Dependency Injection subsystem that we want to use one of those. We also hold onto it via a new instance variable called _jwtService:

 // ...
 public class UsersController : ControllerBase
 {
     private readonly UserManager<IdentityUser> _userManager;
+    private readonly JwtService _jwtService;
 
     public UsersController(
         UserManager<IdentityUser> userManager,
+        JwtService jwtService
     ) {
         _userManager = userManager;
+        _jwtService = jwtService;
     }
     // ...
 }

We also need to tell ASP.NET Core that JwtService should be available for Dependency Injection. To do so, we can add this line in VehicleQuotes/Startup.cs’s ConfigureServices method:

services.AddScoped<Services.JwtService>();

Note that this way of defining dependencies is not recommended and only done this way here to keep things simple for an illustrative app that’s never going to run in production. Ideally what you want to do here is define the dependency as an abstraction (i.e. an interface) and a concrete implementation that fulfills it. For example:
services.AddScoped<ITokenCreationService, JwtService>();
This way, classes that depend on this service can reference the interface, not the concrete type. In doing that, they adhere to the Dependency Inversion principle and become more easily testable because they allow mocks to be provided as dependencies.

Finally, we write the actual action method:

// POST: api/Users/BearerToken
[HttpPost("BearerToken")]
public async Task<ActionResult<AuthenticationResponse>> CreateBearerToken(AuthenticationRequest request)
{
    if (!ModelState.IsValid)
    {
        return BadRequest("Bad credentials");
    }

    var user = await _userManager.FindByNameAsync(request.UserName);

    if (user == null)
    {
        return BadRequest("Bad credentials");
    }

    var isPasswordValid = await _userManager.CheckPasswordAsync(user, request.Password);

    if (!isPasswordValid)
    {
        return BadRequest("Bad credentials");
    }

    var token = _jwtService.CreateToken(user);

    return Ok(token);
}

Here too we’re leveraging the UserManager instance that ASP.NET Core Identity so graciously provided us with. In summary, we find the user account by name using the incoming request data, check if the given password is correct, then ask our JwtService to create a token for this user, and finally return it wrapped in a 200 response.

If you hit that endpoint with a POST and a set of existing credentials, you should see something like this:

If you take that big string that came back in the "token" field in the response JSON, and paste it in jwt.io’s token decoder, you should be able to see all the claims that we added to the token:

Notice that the keywords here are “Encoded” and “Decoded”. The claims that we put in our JWTs are not protected in any way. As such, we should never put secrets in there.

Securing an endpoint with JWT authentication

Now that we have a way for obtaining tokens, let’s see how we can actually use them to gain access to some resources. To demonstrate that, let’s secure an endpoint in a way that it denies unauthenticated requests.

Applying the Authorize attribute

First we need to signal ASP.NET Core that the endpoint requires auth. We do that by annotating the corresponding action method with the Authorize attribute. We’ve already decided that we were going to use VehicleQuotes/Controllers/BodyTypesController.cs’s GetBodyTypes method as a guinea pig. So, let’s go into that file and add the following using statement:

using Microsoft.AspNetCore.Authorization;

That will allow us access to the attribute, which we can apply to the action method like so:

 // GET: api/BodyTypes
+ [Authorize]
  [HttpGet]
  public async Task<ActionResult<IEnumerable<BodyType>>> GetBodyTypes()
  {
      return await _context.BodyTypes.ToListAsync();
  }

With this, we’ve told ASP.NET Core that we want it to require auth for this endpoint.

Enabling and configuring the JWT authentication

Now, we need to tell it how to actually perform the check. To do that, we need to install the Microsoft.AspNetCore.Authentication.JwtBearer NuGet package. We can do that from the VehicleQuotes directory with the following command:

$ dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer

Once we have that installed, we get access to additional services which we can use to configure the “JwtBearer” authentication scheme. As usual, we do the configuration in VehicleQuotes/Startup.cs’s. Here’s what we need to do:

Add a few new using statements:

using Microsoft.AspNetCore.Authentication.JwtBearer;
using Microsoft.IdentityModel.Tokens;
using System.Text;

Add the following code at the end of the ConfigureServices method:

services
    .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.TokenValidationParameters = new TokenValidationParameters()
        {
            ValidateIssuer = true,
            ValidateAudience = true,
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,
            ValidAudience = Configuration["Jwt:Audience"],
            ValidIssuer = Configuration["Jwt:Issuer"],
            IssuerSigningKey = new SymmetricSecurityKey(
                Encoding.UTF8.GetBytes(Configuration["Jwt:Key"])
            )
        };
    });

The call to AddAuthentication includes all the internal core service classes that are needed to do authentication in our app.

The JwtBearerDefaults.AuthenticationScheme parameter that we give it is the name of the authentication scheme to use as the default. More on that later. For now, know that ASP.NET Core supports multiple authentication schemes to be used at the same time. In fact, our own plan here is to eventually support two auth schemes: JWTs and API Keys. We’re starting with JWT first and as such, that’s the one we specify as the default.

The call to AddJwtBearer configures the JWT authentication scheme. That is, it allows the app to perform authentication checks based on an incoming JWT token.

The most important part of the configuration is the values that we are passing to TokenValidationParameters. As you can see, we are able to specify which aspects of the incoming JWTs to validate. You can see all available options for TokenValidationParameters in the official documentation. Here we’ve chosen to validate obvious things like the issuer, audience, and signing key.

The last configuration step is to add this line right before app.UseAuthorization(); in the Configure method:

app.UseAuthentication();

That will enable the authentication middleware. That way the framework will perform authentication checks as part of the request processing pipeline. In other words, actually put all the configuration that we’ve done to good use.

Alright, now that we have all the configuration ready and have secured an endpoint, let’s try hitting it with a GET on api/BodyTypes and see what happens:

Ok! So far so good. We made an unauthenticated request into an endpoint that requires authentication and as a result we got a 401 back. That’s just what we wanted.

Now, let’s get a token by POSTing to api/Users/BearerToken:

We can copy that token and include as a header in the GET request to api/BodyTypes. The header key should be Authorization and the value should be Bearer <our token>. In Postman, we can setup a similar request if we choose the “Authorization” tab, select “Bearer Token” in the “Type” drop-down list and paste the token in the “Token” text box.

Do that, and you should now see a 200 response from the endpoint:

Neat! Now that we passed a valid token, it wants to talk to us again. Let a few minutes pass, enough for the token to expire and try the request again to see how it’s 401’ing again.

Implementing API Key authentication

Now let’s change gears from JWT and implement an alternate authentication strategy in our Web API: API Keys. Authentication with API Keys is fairly common in the web service world.

The core idea of API Keys is that the API provider (in this case, us) produces a secret string that is given to the clients for safekeeping. The clients then can use that key to access the API and make as many requests as they want. So it’s essentially just a glorified password.

The main functional differences when it comes to JWT as we have implemented it is that the API Keys won’t expire and that we will store them in our database. Then, when processing incoming requests, as part of the authentication step, our app will check if the given API Key exists in our internal records. If it does, then the request is allowed to go through, if it does not, then a 401 is sent back.

This is going to be interesting because ASP.NET Core does not include an authentication scheme out of the box for this kind of behavior. For JWT, we were able to use the Microsoft.AspNetCore.Authentication.JwtBearer package. For this we have to get our hands dirty and actually implement a custom authentication handler that will run the logic that we need.

Let’s get started.

Creating API Keys

Similarly to when we implemented JWTs, we’ll start by creating the new endpoint that clients will call to obtain their keys. In order to make it work, we’ll need:

A new model and database table to store the keys
A service class to create the keys.
A new action method in UsersController that does the work.

The API Key model and database table

For storing API Keys, we just need a simple table that is linked to users via a one-to-many relationship and includes a name and a value. Using Entity Framework, our model could look like this in VehicleQuotes/Models/UserApiKey.cs:

using System.ComponentModel.DataAnnotations;
using System.Text.Json.Serialization;
using Microsoft.AspNetCore.Identity;
using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes.Models
{
    [Index(nameof(Value), IsUnique = true)]
    public class UserApiKey
    {
        [JsonIgnore]
        public int ID { get; set; }

        [Required]
        public string Value { get; set; }

        [JsonIgnore]
        [Required]
        public string UserID { get; set; }

        [JsonIgnore]
        public IdentityUser User { get; set; }
    }
}

Easy enough. We express the relationship between the “keys” and “users” tables via the User navigation property and the UserID field. We also define a unique index on the Value field and annotate some fields with Required because we don’t want them to allow null. Interestingly, we also annotated some of the fields with the JsonIgnore attribute so that when we include objects of this type in API responses (which, as you’ll see, we will), those fields are not included.

We also need to add a new DbSet on our DbContext class at VehicleQuotes/Data/VehicleQuotesContext.cs so that Entity Framework picks it up as part of the data model:

public DbSet<UserApiKey> UserApiKeys { get; set; }

With that, let’s now create and run a migration so that the new table can be added to the database. All it takes is running this to create the migration:

dotnet ef migrations add AddUserApiKeysTable

And this to apply it:

dotnet ef database update

Now the new table should appear in the database:

vehicle_quotes=# \dt
                    List of relations
 Schema |         Name          | Type  |     Owner      
--------+-----------------------+-------+----------------
...
 public | user_api_keys         | table | vehicle_quotes
 (15 rows)

The class that creates API Keys

The next step is very straightforward. All we need is some logic that, when given a user account (in the form of an instance of IdentityUser), can generate a new API Key, insert it in the database, and return it. The following unremarkable class, defined in VehicleQuotes/Services/ApiKeyService.cs, encapsulates said logic:

using System;
using Microsoft.AspNetCore.Identity;
using VehicleQuotes.Models;

namespace VehicleQuotes.Services
{
    public class ApiKeyService
    {
        private readonly VehicleQuotesContext _context;

        public ApiKeyService(VehicleQuotesContext context)
        {
            _context = context;
        }

        public UserApiKey CreateApiKey(IdentityUser user)
        {
            var newApiKey = new UserApiKey
            {
                User = user,
                Value = GenerateApiKeyValue()
            };

            _context.UserApiKeys.Add(newApiKey);

            _context.SaveChanges();

            return newApiKey;
        }

        private string GenerateApiKeyValue() =>
            $"{Guid.NewGuid().ToString()}-{Guid.NewGuid().ToString()}";
    }
}

As you can see, this class’s single public method, aptly named CreateApiKey, just takes the given user and creates a new UserApiKey that’s associated with it and saves it into the database. The key values themselves are just two GUIDs concatenated.

This is an admittedly simplistic and not-all-that-secure method for generating the keys themselves. For simplicity, we’ve gone with this. In a production app however, it’d be better to use a true cryptographically secure string. Here’s an article that explains how to do that with .NET.

Naturally, this class needs to be used elsewhere in the application, so let’s make it available for Dependency Injection by adding the following line to VehicleQuotes/Startup.cs’s ConfigureServices method.

services.AddScoped<Services.ApiKeyService>();

The action method that puts it all together

Now let’s finally write a new action method on VehicleQuotes/Controllers/UsersController.cs so that there’s an endpoint that clients can call to get API Keys. Before adding the action method though, as usual, we need to declare the following dependency as a constructor parameter and hold it in an instance variable:

 public class UsersController : ControllerBase
 {
     private readonly UserManager<IdentityUser> _userManager;
     private readonly JwtService _jwtService;
+    private readonly ApiKeyService _apiKeyService;
 
     public UsersController(
         UserManager<IdentityUser> userManager,
         JwtService jwtService,
+        ApiKeyService apiKeyService 
     ) {
         _userManager = userManager;
         _jwtService = jwtService;
+        _apiKeyService = apiKeyService;
     }}

The action method itself would look like this:

// POST: api/Users/ApiKey
[HttpPost("ApiKey")]
public async Task<ActionResult> CreateApiKey(AuthenticationRequest request)
{
    if (!ModelState.IsValid)
    {
        return BadRequest(ModelState);
    }

    var user = await _userManager.FindByNameAsync(request.UserName);

    if (user == null)
    {
        return BadRequest("Bad credentials");
    }

    var isPasswordValid = await _userManager.CheckPasswordAsync(user, request.Password);

    if (!isPasswordValid)
    {
        return BadRequest("Bad credentials");
    }

    var token = _apiKeyService.CreateApiKey(user);

    return Ok(token);
}

This method is very similar to the other one that generates JWTs. Painfully so. In fact the only difference is that this one calls the ApiKeyService’s CreateApiKey method instead of JwtService’s CreateToken, for obvious reasons.

These two are ripe for some refactoring to eliminate duplication but we won’t do that here. Take a look at the finished project on GitHub to see a neat refactoring option.

Anyway, you can now try a POST into api/Users/ApiKey, pass it a valid set of credentials, and you should see it respond with a brand new API Key:

The key can also be found in the database:

vehicle_quotes=# select id, SUBSTRING(value,1,13), user_id from user_api_keys;
 id |   substring   |               user_id                
----+---------------+--------------------------------------
  1 | efd2133b-cc4a | aaad07b4-f109-4255-8caa-185fd7694c72
(1 row)

Securing an endpoint with API Key authentication

Now that users have a way of getting API Keys, we can start securing endpoints with that authentication scheme. In order to acomplish that, we need to take three steps:

Implement a custom authentication handler that runs the authentication logic.
Configure the new authentication handler, plugging it into ASP.NET Core’s auth mechanisms.
Secure certain endpoints via the Authorization attribute, specifying the scheme that we want to use to do so.

We’ll do that next.

Implementing the API Key authentication handler

Like we’ve touched on before, the “authentication handler” is a class that can plug into ASP.NET Core’s authentication middleware and can run the authentication logic for a given authentication scheme. In practical terms, it’s a class that inherits from Microsoft.AspNetCore.Authentication.AuthenticationHandler and implements the HandleAuthenticateAsync method. Ours will live in VehicleQuotes/Authentication/ApiKey/ApiKeyAuthenticationHandler.cs and it looks like this:

using System.Security.Claims;
using System.Text.Encodings.Web;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Identity;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace VehicleQuotes.Authentication.ApiKey
{
    class ApiKeyAuthenticationHandler : AuthenticationHandler<AuthenticationSchemeOptions>
    {
        private const string API_KEY_HEADER = "Api-Key";

        private readonly VehicleQuotesContext _context;

        public ApiKeyAuthenticationHandler(
            IOptionsMonitor<AuthenticationSchemeOptions> options,
            ILoggerFactory logger,
            UrlEncoder encoder,
            ISystemClock clock,
            VehicleQuotesContext context
        ) : base(options, logger, encoder, clock)
        {
            _context = context;
        }

        protected override async Task<AuthenticateResult> HandleAuthenticateAsync()
        {
            if (!Request.Headers.ContainsKey(API_KEY_HEADER))
            {
                return AuthenticateResult.Fail("Header Not Found.");
            }

            string apiKeyToValidate = Request.Headers[API_KEY_HEADER];

            var apiKey = await _context.UserApiKeys
                .Include(uak => uak.User)
                .SingleOrDefaultAsync(uak => uak.Value == apiKeyToValidate);

            if (apiKey == null)
            {
                return AuthenticateResult.Fail("Invalid key.");
            }

            return AuthenticateResult.Success(CreateTicket(apiKey.User));
        }

        private AuthenticationTicket CreateTicket(IdentityUser user)
        {
            var claims = new[] {
                new Claim(ClaimTypes.NameIdentifier, user.Id),
                new Claim(ClaimTypes.Name, user.UserName),
                new Claim(ClaimTypes.Email, user.Email)
            };

            var identity = new ClaimsIdentity(claims, Scheme.Name);
            var principal = new ClaimsPrincipal(identity);
            var ticket = new AuthenticationTicket(principal, Scheme.Name);

            return ticket;
        }
    }
}

The first thing to note is how we’ve defined the base class: AuthenticationHandler<AuthenticationSchemeOptions>. AuthenticationHandler is a generic, and it allows us to control the type of options object that it accepts. In this case, we’ve used AuthenticationSchemeOptions, which is just a default one already provided by the framework. This is because we don’t really need any custom options.

If we did, then we would define a new class that inherits from AuthenticationSchemeOptions, give it the new fields that we need, and use that as a generic time parameter instead. We would then be able to set values for these new fields during configuration in the app’s Startup class. Just like we’ve done with the “Jwt Bearer” auth scheme via the AddJwtBearer method.

The next point of interest in this class is its constructor. The key aspect of it is that it calls base and passes it a series of parameters that it itself gets. This is because AuthenticationHandler does have some constructor logic that needs to be run for things to work properly, so we make sure to do that.

Finally, there’s the HandleAuthenticateAsync method which does the actual authentication. This method inspects the incoming request looking for an “Api-Key” header. If it finds it, it takes its value and makes a query into the database to try and find a record in the user_api_keys table that matches the incoming value. If it finds that, it creates an AuthenticationTicket which contains the identity of the user that the key belongs to, and returns it wrapped in a AuthenticateResult.Success. That return value signals ASP.NET Core’s authentication middleware that the request is authentic. Otherwise, it returns AuthenticateResult.Fail, which prompts ASP.NET Core to halt the request and return a 401.

Something interesting to note is how the AuthenticationTicket, similarly to the JWT Bearer scheme’s JwtSecurityToken, also includes an array of “claims” that serve to identify the user that has been authenticated. The “claims” concept is central to how auth works in ASP.NET Core.

Enabling and configuring the API Key authentication

Now we need to tell the framework that we want it to use a new authentication scheme for our app, spearheaded by the custom authentication handler that we just wrote. There are two steps to make that happen.

First, we configure our new scheme in the app’s Startup class. For that, we need these two new using statements:

using Microsoft.AspNetCore.Authentication;
using VehicleQuotes.Authentication.ApiKey;

Then we add this right after the AddJwtBearer call in the ConfigureServices method:

 public void ConfigureServices(IServiceCollection services)
 {
     // ...
     services
         .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
         .AddJwtBearer(options =>
         {
             options.TokenValidationParameters = new TokenValidationParameters()
             {
                 ValidateIssuer = true,
                 ValidateAudience = true,
                 ValidateLifetime = true,
                 ValidateIssuerSigningKey = true,
                 ValidAudience = Configuration["Jwt:Audience"],
                 ValidIssuer = Configuration["Jwt:Issuer"],
                 IssuerSigningKey = new SymmetricSecurityKey(
                     Encoding.UTF8.GetBytes(Configuration["Jwt:Key"])
                 )
             };
         })
+        .AddScheme<AuthenticationSchemeOptions, ApiKeyAuthenticationHandler>(
+            "ApiKey",
+            options => { }
+        );
 }

A simple configuration as far as auth schemes go. We specify both the types of the authentication handler and the options object that it accepts via the generic type parameters to the AddScheme method. "ApiKey" is just the name we’ve given our custom auth scheme. It can be anything as long as it’s not “Bearer”, which is the name of the Jwt Bearer scheme (which is the value returned by JwtBearerDefaults.AuthenticationScheme). We’ll refer back to it later. Finally, since we have no special options to give it, we specify a lambda that does nothing with its options.

Updating the Authorize attribute to use our two schemes

Now that the auth scheme is configured, we need to use it to secure an endpoint. We can use the same that we used for JWT: POST api/BodyTypes, defined in BodyTypesController’s GetBodyTypes action method. Remember though, that we wanted to have both auth schemes (JWT and API Key) work on the endpoint. For that, the Authorize attribute allows a comma separated string of scheme names as a parameter. So, we can get both our configured schemes working if we update the attribute like this:

[Authorize(AuthenticationSchemes = $"{JwtBearerDefaults.AuthenticationScheme},ApiKey")]
public async Task<ActionResult<IEnumerable<BodyType>>> GetBodyTypes()
{
    return await _context.BodyTypes.ToListAsync();
}

JwtBearerDefaults.AuthenticationScheme contains the name of the JWT Bearer auth scheme. Next to it, we just put “ApiKey”, which is the name we have given our new custom one.

It’d be nice to put that “ApiKey” string somewhere safe similar to how the “Jwt Bearer” scheme has it in a constant. Take a look at the finished project on GitHub to see one way to organize that and also how to make the Startup configuration a little less verbose by using extension methods.

And finally, let’s test our endpoint and marvel at the fruit of our work.

Make sure to create a new API Key by POSTing to api/Users/ApiKey, and copy the “value” from the response. We can use it as the value for the Api-Key header a GET request to api/BodyTypes. In Postman, we can do that by choosing the “Authorization” tab, selecting “API Key” in the “Type” drop-down list, writing “Api-Key” in the “Key” text box, and putting our key in the “Value” text box.

With that, we can make the request and see how the endpoint now allows authentication via API Keys:

Of course, requests authenticated via JWT will also work for this endpoint.

That’s all

And there you have it! We’ve updated an existing ASP.NET Core Web API application so that it supports authentication using two strategies: JWT and API Keys. We leveraged the Identity libraries to securely store and manage user accounts. We used ASP.NET Core’s built-in authentication capabilities to enable JWT generation and usage. For API Keys, the framework didn’t provide an implementation out of the box. However, it proved to be extensible enough so that we could implement our own.

Monitoring Settings Changes in ASP.NET Core

2021-09-22T00:00:00+00:00

Photo by Linus Nylund on Unsplash

Did you know you can directly respond to config file changes in ASP.NET Core? By using the IOptionsMonitor<T> interface, it’s possible to run a lambda function every time a config file is updated.

For those who aren’t too familiar with it, ASP.NET Core configuration is handled using the Options Pattern. That is, you create a model class that has a field for each property in your config file, and then set up the application to map the configuration to that class, which can be injected into controllers and services. In your controllers, you can request the configuration class T as a dependency by using an options wrapper class like IOptions<T>.

While this works just fine in some cases, it doesn’t allow you to actually run code when the configuration is changed. However, if you request your configuration using IOptionsMonitor<T>, you get the ability to define a lambda function to run every time appsettings.json is changed.

One use case for this functionality would be if you wanted to maintain a list in the config file, and log every time that list was changed. In this article I’ll explain how to set up an ASP.NET Core 5.0 API to run custom code whenever changes are made to appsettings.json.

Setting up the API

In order to use the options pattern in your API, you’ll first need to add the options services to the container using the services.AddOptions() method. Then, you can register your custom configuration class (in this example, MyOptions) to be bound to a specific section in appsettings.json (in this example, "myOptions").

public void ConfigureServices(IServiceCollection services)
{
  // ...

  // Add options services to the container
  services.AddOptions();

  // Configure the app to map the "myOptions" section of
  // the config file to the MyOptions model class
  services.Configure<MyOptions>(
    Configuration.GetSection("myOptions")
  );
}

Monitoring Configuration Changes

Now that the API is set up correctly, in your controllers you can directly request the configuration using IOptionsMonitor<T>. You can also unpack the configuration instance itself by using the IOptionsMonitor<T>.CurrentValue property. This value will automatically get updated by IOptionsMonitor<T> when the configuration is changed, so you only need to retrieve it once in the constructor.

using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Options;

[Route("api/[controller]")]
[ApiController]
public class MyController : ControllerBase
{
  private MyOptions MyOptions;

  public MyController(IOptionsMonitor<MyOptions> MyOptionsMonitor)
  {
    // Stores the actual configuration in the controller
    // to reference later. Changes to the config will be
    // reflected in MyOptions.
    this.MyOptions = MyOptionsMonitor.CurrentValue;

    // Registers a lambda function to be called every time
    // a configuration change is made
    MyOptionsMonitor.OnChange(async (opts) =>
    {
      // Write some code!
    });
  }
}

One small gotcha worth noting is that the OnChange function isn’t debounced. So if you’re using an IDE that automatically saves as you type, it’ll trigger the OnChange function rapidly as you edit appsettings.json.

Conclusion

And that’s it! It doesn’t take much to get your API to run custom code on config file changes.

Have any questions? Feel free to leave a comment!

Building REST APIs with .NET 5, ASP.NET Core, and PostgreSQL

2021-07-09T00:00:00+00:00

Photo by Sam Beasley

This is old news by now, but I’m still amazed by the fact that nowadays .NET is open source and can run on Linux. I truly believe that this new direction can help the technology realize its true potential, since it’s no longer shackled to Windows-based environments. I’ve personally been outside the .NET game for a good while, but with the milestone release that is .NET 5, I think now is a great time to dive back in.

So I thought of taking some time to do just that, really dive in, see what’s new, and get a sense of the general developer experience that the current incarnation of .NET offers. So in this blog post, I’m going to chronicle my experience developing a simple but complete REST API application. Along the way, I’ll touch on the most common problems that one runs into when developing such applications and how they are solved in the .NET world. So think of this piece as a sort of tutorial or overview of the most common framework features when it comes to developing REST APIs.

There’s a table of contents at the bottom.

First, let’s get familiar with what we’re building.

What we’re building

The demo application

You can find the finished product on my GitHub.

The application that we’ll be building throughout this article will address a request from a hypothetical car junker business. Our client wants to automate the process of calculating how much money to offer their customers for their vehicles, given certain information about them. And they want an app to do that. We are building the back-end component that will support that app. It is a REST API that allows users to provide vehicle information (year, make, model, condition, etc.) and will produce a quote of how much money our hypothetical client would be willing to pay for it.

Here’s a short list of features that we need to implement in order to fulfill that requirement:

Given a vehicle model and condition, calculate a price.
Store and manage rules that are used to calculate vehicle prices.
Store and manage pricing overrides on a vehicle model basis. Price overrides are used regardless of the current rules.
CRUD vehicle models so that overrides can be specified for them.

The data model

Here’s what our data model looks like:

The main table in our model is the quotes table. It stores all the requests for quotes received from our client’s customers. It captures all the relevant vehicle information in terms of model and condition. It also captures the offered quote; that is, the money value that our system calculates for their vehicle.

The quotes table includes all the fields that identify a vehicle: year, make, model, body style and size. It also includes a model_style_year_id field which is an optional foreign key to another table. This FK points to the model_style_years table which contains specific vehicle models that our system can store explicitly.

The idea of this is that, when a customer submits a request for a quote, if we have their vehicle registered in our database, then we can populate this foreign key and link the quote with the specific vehicle that it’s quoting. If we don’t have their vehicle registered, then we leave that field unpopulated. Either way, we can offer a quote. The only difference is the level or certainty of the quote.

The records in the model_style_years table represent specific vehicles. That whole hierarchy works like this: A vehicle make (e.g. Honda, Toyota, etc. in the makes table) has many models (e.g. Civic, Corolla, etc. in the models table), each model has many styles (the model_styles table). Styles are combinations of body types (the body_types table) and sizes (e.g. Mid-size Sedan, Compact Coupe, etc. in the sizes table). And finally, each model style has many years in which they were being produced (via the model_style_years table).

This model allows us very fine-grained differentiation between vehicles. For example, we can have a “2008 Honda Civic Hatchback which is a Compact car” and also a “1990 Honda Civic Hatchback which is a Sub-compact”. That is, same model, different year, size or body type.

We also have a quote_rules table which stores the rules that are applied when it comes to calculating a vehicle quote. The rules are pairs of key-values with an associated monetary value. So for example, rules like “a vehicle that has alloy wheels is worth $10 more” can be expressed in the table with a record where feature_type is “has_alloy_wheels”, feature_value is “true” and price_modifier is “10”.

Finally, we have a quote_overrides table which specifies a flat, static price for specific vehicles (via the link to the model_style_years table). The idea here is that if some customer requests a quote for a vehicle for which we have an override, no price calculation rules are applied and they are offered what is specified in the override record.

The development environment

Setting up the PostgreSQL database with Docker

For this project, our database of choice is PostgreSQL. Luckily for us, getting a PostgreSQL instance up and running is very easy thanks to Docker.

If you want to learn more about dockerizing a typical web application, take a look at this article that explains the process in detail.

Once you have Docker installed in your machine, getting a PostgreSQL instance is as simple as running the following command:

$ docker run -d \
    --name vehicle-quote-postgres \
    -p 5432:5432 \
    --network host \
    -e POSTGRES_DB=vehicle_quote \
    -e POSTGRES_USER=vehicle_quote \
    -e POSTGRES_PASSWORD=password \
    postgres

Here we’re asking Docker to run a new container based on the latest postgres image from DockerHub, name it vehicle-quote-postgres, specify the port to use the default PostgreSQL one, make it accessible to the local network (with the --network host option) and finally, specify a few environment variables that the postgres image uses when building our new instance to set up the default database name, user and password (with the three -e options).

After Docker is done working its magic, you should be able to access the database with something like this:

$ docker exec -it vehicle-quote-postgres psql -U vehicle_quote
psql (13.2 (Debian 13.2-1.pgdg100+1))
Type "help" for help.

vehicle_quote=#

This command is connecting to our new vehicle-quote-postgres container and then, from within the container, using the command line client psql in order to connect to the database.

If you have psql installed on your own machine, you can use it directly to connect to the PostgreSQL instance running inside the container:

$ psql -h localhost -U vehicle_quote

This is possible because we specified in our docker run command that the container would be accepting traffic over port 5432 (-p 5432:5432) and that it would be accesible within the same network as our actual machine (--network host).

Installing the .NET 5 SDK

Ok, with that out of the way, let’s install .NET 5.

.NET 5 truly is multi-platform, so whatever environment you prefer to work with, they’ve got you covered. You can go to the .NET 5 download page and pick your desired flavor of the SDK.

On Ubuntu 20.10, which is what I’m running, installation is painless. It’s your typical process with APT and this page from the official docs has all the details.

First step is to add the Microsoft package repository:

$ wget https://packages.microsoft.com/config/ubuntu/20.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb

$ sudo dpkg -i packages-microsoft-prod.deb

Then, install .NET 5 with APT like one would any other software package:

$ sudo apt-get update; \
  sudo apt-get install -y apt-transport-https && \
  sudo apt-get update && \
  sudo apt-get install -y dotnet-sdk-5.0

Run dotnet --version in your console and you should see something like this:

$ dotnet --version
5.0.301

Setting up the project

Creating our ASP.NET Core REST API project

Ok now that we have our requirements, database and SDK, let’s start setting up our project. We do so with the following command:

$ dotnet new webapi -o VehicleQuotes

This instructs the dotnet command line tool to create a new REST API web application project for us in a new VehicleQuotes directory.

As a result, dotnet will give you some messages, including this:

The template "ASP.NET Core Web API" was created successfully.

A new directory was created with our web application files. The newly created VehicleQuotes project looks like this:

.
├── appsettings.Development.json
├── appsettings.json
├── Controllers
│   └── WeatherForecastController.cs
├── obj
│   ├── project.assets.json
│   ├── project.nuget.cache
│   ├── VehicleQuotes.csproj.nuget.dgspec.json
│   ├── VehicleQuotes.csproj.nuget.g.props
│   └── VehicleQuotes.csproj.nuget.g.targets
├── Program.cs
├── Properties
│   └── launchSettings.json
├── Startup.cs
├── VehicleQuotes.csproj
└── WeatherForecast.cs

Important things to note here are the appsettings.json and appsettings.Development.json files which contain environment specific configuration values; the Controllers directory where we define our application controllers and action methods (i.e. our REST API endpoints); the Program.cs and Startup.cs files that contain our application’s entry point and bootstrapping logic; and finally VehicleQuotes.csproj which is the file that contains project-wide configuration that the framework cares about like references, compilation targets, and other options. Feel free to explore.

The dotnet new command has given us quite a bit. These files make up a fully working application that we can run and play around with. It even has a Swagger UI, as I’ll demonstrate shortly. It’s a great place to get started from.

You can also get a pretty comprehensive .gitignore file by running the dotnet new gitignore command.

From inside the VehicleQuotes directory, you can run the application with:

$ dotnet run

Which will start up a development server and give out the following output:

$ dotnet run
Building...
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /home/kevin/projects/endpoint/blog/VehicleQuotes

Open up a browser window and go to https://localhost:5001/swagger to find a Swagger UI listing our API’s endpoints:

As you can see we’ve got a GET /WeatherForecast endpoint in our app. This is included by default in the webapi project template that we specified in our call to dotnet new. You can see it defined in the Controllers/WeatherForecastController.cs file.

Installing packages we’ll need

Now let’s install all the tools and libraries we will need for our application. First, we install the ASP.NET Code Generator tool which we’ll use later for scaffolding controllers:

$ dotnet tool install --global dotnet-aspnet-codegenerator

We also need to install the Entity Framework command line tools which help us with creating and applying database migrations:

$ dotnet tool install --global dotnet-ef

Now, we need to install a few libraries that we’ll use in our project. First are all the packages that allow us to use Entity Framework Core, provide scaffolding support and give us a detailed debugging page for database errors:

$ dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design
$ dotnet add package Microsoft.EntityFrameworkCore.Design
$ dotnet add package Microsoft.EntityFrameworkCore.SqlServer
$ dotnet add package Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore

We also need the EF Core driver for PostgreSQL which will allow us to interact with our database:

$ dotnet add package Npgsql.EntityFrameworkCore.PostgreSQL

Finally, we need another package that will allow us to use the snake case naming convention for our database tables, fields, etc. We need this because EF Core uses capitalized camel case by default, which is not very common in the PostgreSQL world, so this will allow us to play nice. This is the package:

$ dotnet add package EFCore.NamingConventions

Connecting to the database and performing initial app configuration

In order to connect to, query, and modify a database using EF Core, we need to create a DbContext. This is a class that serves as the entry point into the database. Create a new directory called Data in the project root and add this new VehicleQuotesContext.cs file to it:

using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes
{
    public class VehicleQuotesContext : DbContext
    {
        public VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
            : base(options)
        {
        }
    }
}

As you can see this is just a simple class that inherits from EF Core’s DbContext class. That’s all we need for now. We will continue building on this class as we add new tables and configurations.

Now, we need to add this class into ASP.NET Core’s built-in IoC (inversion of control) container so that it’s available to controllers and other classes via Dependency Injection, and tell it how to find our database. Go to Startup.cs and add the following using statement near the top of the file:

using Microsoft.EntityFrameworkCore;

That will allow us to do the following change in the ConfigureServices method:

 public void ConfigureServices(IServiceCollection services)
 {

     // ...

+    services.AddDbContext<VehicleQuotesContext>(options =>
+        options
+            .UseNpgsql(Configuration.GetConnectionString("VehicleQuotesContext"))
+    );
 }

UseNpgsql is an extension method made available to us by the Npgsql.EntityFrameworkCore.PostgreSQL package that we installed in the previous step.

The services variable contains all the objects (known as “services”) that are available in the app for Dependency Injection. So here, we’re adding our newly created DbContext to it, specifying that it will connect to a PostgreSQL database (via the options.UseNpgsql call), and that it will use a connection string named VehicleQuotesContext from the app’s default configuration file. So let’s add the connection string then. To do so, change the appsettings.json like so:

 {
   "Logging": {
     "LogLevel": {
       "Default": "Information",
       "Microsoft": "Warning",
       "Microsoft.Hosting.Lifetime": "Information"
     }
   },
-  "AllowedHosts": "*"
+  "AllowedHosts": "*",
+  "ConnectionStrings": {
+      "VehicleQuotesContext": "Host=localhost;Database=vehicle_quote;Username=vehicle_quote;Password=password"
+  }
 }

This is your typical PostgreSQL connection string. The only gotcha is that it needs to be specified under the ConnectionStrings -> VehicleQuotesContext section so that our call to Configuration.GetConnectionString can find it.

Now let’s put the EFCore.NamingConventions package to good use and configure EF Core to use snake case when naming database objects. Add the following to the ConfigureServices method in Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    // ...

    services.AddDbContext<VehicleQuotesContext>(options =>
        options
            .UseNpgsql(Configuration.GetConnectionString("VehicleQuotesContext"))
+           .UseSnakeCaseNamingConvention()
    );
}

UseSnakeCaseNamingConvention is an extension method made available to us by the EFCore.NamingConventions package that we installed in the previous step.

Now let’s make logging a little bit more verbose with:

public void ConfigureServices(IServiceCollection services)
{
    // ...

    services.AddDbContext<VehicleQuotesContext>(options =>
        options
            .UseNpgsql(Configuration.GetConnectionString("VehicleQuotesContext"))
            .UseSnakeCaseNamingConvention()
+           .UseLoggerFactory(LoggerFactory.Create(builder => builder.AddConsole()))
+           .EnableSensitiveDataLogging()
    );
}

This will make sure full database queries appear in the log in the console, including parameter values. This could expose sensitive data so be careful when using EnableSensitiveDataLogging in production.

We can also add the following service configuration to have the app display detailed error pages when something related to the database or migrations goes wrong:

public void ConfigureServices(IServiceCollection services)
{
    // ...

+   services.AddDatabaseDeveloperPageExceptionFilter();
}

AddDatabaseDeveloperPageExceptionFilter is an extension method made available to us by the Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore package that we installed in the previous step.

Finally, one last configuration I like to do is have the Swagger UI show up at the root URL, so that instead of using https://localhost:5001/swagger, we’re able to just use https://localhost:5001. We do so by by updating the Configure method this time, in the same Startup.cs file that we’ve been working on:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
        app.UseSwagger();
-       app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "VehicleQuotes v1"));
+       app.UseSwaggerUI(c => {
+           c.SwaggerEndpoint("/swagger/v1/swagger.json", "VehicleQuotes v1");
+           c.RoutePrefix = "";
+       });
    }

The magic is done by the c.RoutePrefix = ""; line which makes it so there’s no need to put any prefix in order to access the auto generated Swagger UI.

Try it out. Do dotnet run and navigate to https://localhost:5001 and you should see the Swagger UI there.

Building the application

Creating model entities, migrations and updating the database

Alright, with all that configuration out of the way, let’s implement some of our actual application logic now. Refer back to our data model. We’ll start by defining our three simplest tables: makes, sizes and body_types. With EF Core, we define tables via so-called POCO entities, which are simple C# classes with some properties. The classes become tables and the properties become the tables’ fields. Instances of these classes represent records in the database.

So, create a new Models directory in our project’s root and add these three files:

// Models/BodyType.cs
namespace VehicleQuotes.Models
{
    public class BodyType
    {
        public int ID { get; set; }
        public string Name { get; set; }
    }
}

// Models/Make.cs
namespace VehicleQuotes.Models
{
    public class Make
    {
        public int ID { get; set; }
        public string Name { get; set; }
    }
}

// Models/Size.cs
namespace VehicleQuotes.Models
{
    public class Size
    {
        public int ID { get; set; }
        public string Name { get; set; }
    }
}

Now, we add three corresponding DbSets to our DbContext in Data/VehicleQuoteContext.cs. Here’s the diff:

using Microsoft.EntityFrameworkCore;
+using VehicleQuotes.Models;

namespace VehicleQuotes
{
    public class VehicleQuotesContext : DbContext
    {
        public VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
            : base(options)
        {
        }

+       public DbSet<Make> Makes { get; set; }
+       public DbSet<Size> Sizes { get; set; }
+       public DbSet<BodyType> BodyTypes { get; set; }
    }
}

This is how we tell EF Core to build tables in our database for our entities. You’ll see later how we use those DbSets to access the data in those tables. For now, let’s create a migration script that we can later run to apply changes to our database. Run the following to have EF Core create it for us:

$ dotnet ef migrations add AddLookupTables

Now take a look at the newly created Migrations directory. It contains a few new files, but the one we care about right now is Migrations/{TIMESTAMP}_AddLookupTables.cs. In its Up method, it’s got some code that will modify the database structure when run. The EF Core tooling has inspected our project, identified the new entities, and automatically generated a migration script for us that creates tables for them. Notice also how the tables and fields use the snake case naming convention, just as we specified with the call to UseSnakeCaseNamingConvention in Startup.cs.

Now, to actually run the migration script and apply the changes to the database, we do:

$ dotnet ef database update

That command inspects our project to find any migrations that haven’t been run yet, and applies them. In this case, we only have one, so that’s what it runs. Look at the output in the console to see it working its magic step by step:

$ dotnet ef database update
Build started...
Build succeeded.
warn: Microsoft.EntityFrameworkCore.Model.Validation[10400]
      Sensitive data logging is enabled. Log entries and exception messages may include sensitive application data; this mode should only be enabled during development.

...

info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (4ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE TABLE sizes (
          id integer GENERATED BY DEFAULT AS IDENTITY,
          name text NULL,
          CONSTRAINT pk_sizes PRIMARY KEY (id)
      );
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (1ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      INSERT INTO "__EFMigrationsHistory" (migration_id, product_version)
      VALUES ('20210625212939_AddLookupTables', '5.0.7');
Done.

Notice how it warns us about potential exposure of sensitive data because of that EnableSensitiveDataLogging option we opted into in Startup.cs. Also, EF Core related logs are extra verbose showing all database operations because of another configuration option that we applied there: the UseLoggerFactory(LoggerFactory.Create(builder => builder.AddConsole())) one.

You can connect to the database with the psql command line client and see that the changes took effect:

$ psql -h localhost -U vehicle_quote

...

vehicle_quote=# \c vehicle_quote 
psql (12.7 (Ubuntu 12.7-0ubuntu0.20.10.1), server 13.2 (Debian 13.2-1.pgdg100+1))
You are now connected to database "vehicle_quote" as user "vehicle_quote".
vehicle_quote=# \dt
                   List of relations
 Schema |         Name          | Type  |     Owner     
--------+-----------------------+-------+---------------
 public | __EFMigrationsHistory | table | vehicle_quote
 public | body_types            | table | vehicle_quote
 public | makes                 | table | vehicle_quote
 public | sizes                 | table | vehicle_quote
(4 rows)

vehicle_quote=# \d makes
                            Table "public.makes"
 Column |  Type   | Collation | Nullable |             Default              
--------+---------+-----------+----------+----------------------------------
 id     | integer |           | not null | generated by default as identity
 name   | text    |           |          | 
Indexes:
    "pk_makes" PRIMARY KEY, btree (id)

There are our tables in all their normalized, snake-cased glory. The __EFMigrationsHistory table is used internally by EF Core to keep track of which migrations have been applied.

Creating controllers for CRUDing our tables

Now that we have that, let’s add a few endpoints to support basic CRUD of those tables. We can use the dotnet-aspnet-codegenerator scaffolding tool that we installed earlier. For the three tables that we have, we would do:

$ dotnet aspnet-codegenerator controller \
    -name MakesController \
    -m Make \
    -dc VehicleQuotesContext \
    -async \
    -api \
    -outDir Controllers

$ dotnet aspnet-codegenerator controller \
    -name BodyTypesController \
    -m BodyType \
    -dc VehicleQuotesContext \
    -async \
    -api \
    -outDir Controllers

$ dotnet aspnet-codegenerator controller \
    -name SizesController \
    -m Size \
    -dc VehicleQuotesContext \
    -async \
    -api \
    -outDir Controllers

Those commands tell the scaffolding tool to create new controllers that:

Are named as given by the -name option.
Use the model class specified in the -m option.
Use our VehicleQuotesContext to talk to the database. As per the -dc option.
Define the methods using async/await syntax. Given by the -async option.
Are API controllers. Specified by the -api option.
Are created in the Controllers directory. Via the -outDir option.

Explore the new files that got created in the Controllers directory: MakesController.cs, BodyTypesController.cd and SizesController.cs. The controllers have been generated with the necessary Action Methods to fetch, create, update and delete their corresponding entities. Try dotnet run and navigate to https://localhost:5001 to see the new endpoints in the Swagger UI:

Try it out! You can interact with each of the endpoints from the Swagger UI and it all works as you’d expect.

Adding unique constraints via indexes

Ok, our app is coming along well. Right now though, there’s an issue with the tables that we’ve created. It’s possible to create vehicle makes with the same name. The same is true for body types and sizes. This doesn’t make much sense for these tables. So let’s fix that by adding a uniqueness constraint. We can do it by creating a unique database index using EF Core’s Index attribute. For example, we can modify our Models/Make.cs like so:

+using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes.Models
{
+   [Index(nameof(Name), IsUnique = true)]
    public class Make
    {
        public int ID { get; set; }
        public string Name { get; set; }
    }
}

In fact, do the same for our other entities in Models/BodyType.cs and Models/Size.cs. Don’t forget the using Microsoft.EntityFrameworkCore statement.

With that, we can create a new migration:

$ dotnet ef migrations add AddUniqueIndexesToLookupTables

That will result in a new migration script in Migrations/{TIMESTAMP}_AddUniqueIndexesToLookupTables.cs. Its Up method looks like this:

protected override void Up(MigrationBuilder migrationBuilder)
{
    migrationBuilder.CreateIndex(
        name: "ix_sizes_name",
        table: "sizes",
        column: "name",
        unique: true);

    migrationBuilder.CreateIndex(
        name: "ix_makes_name",
        table: "makes",
        column: "name",
        unique: true);

    migrationBuilder.CreateIndex(
        name: "ix_body_types_name",
        table: "body_types",
        column: "name",
        unique: true);
}

As you can see, new unique indexes are being created on the tables and fields that we specified. Like before, apply the changes to the database structure with:

$ dotnet ef database update

Now if you try to create, for example, a vehicle make with a repeated name, you’ll get an error. Try doing so by POSTing to /api/Makes via the Swagger UI:

Responding with specific HTTP error codes (409 Conflict)

The fact that we can now enforce unique constraints is all well and good. But the error scenario is not very user friendly. Instead of returning a “500 Internal Server Error” status code with a wall of text, we should be responding with something more sensible. Maybe a “409 Conflict” would be more appropriate for this kind of error. We can easily update our controllers to handle that scenario. What we need to do is update the methods that handle the POST and PUT endpoints so that they catch the Microsoft.EntityFrameworkCore.DbUpdateException exception and return the proper response. Here’s how we would do it for the MakesController:

// ...
namespace VehicleQuotes.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class MakesController : ControllerBase
    {
        // ...

        [HttpPut("{id}")]
        public async Task<IActionResult> PutMake(int id, Make make)
        {
            // ...

            try
            {
                await _context.SaveChangesAsync();
            }
            // ...
+           catch (Microsoft.EntityFrameworkCore.DbUpdateException)
+           {
+               return Conflict();
+           }

            return NoContent();
        }

        [HttpPost]
        public async Task<ActionResult<Make>> PostMake(Make make)
        {
            _context.Makes.Add(make);
-           await _context.SaveChangesAsync();

+           try
+           {
+               await _context.SaveChangesAsync();
+           }
+           catch (Microsoft.EntityFrameworkCore.DbUpdateException)
+           {
+               return Conflict();
+           }

            return CreatedAtAction("GetMake", new { id = make.ID }, make);
        }

        // ...
    }
}

Go ahead and do the same for the other two controllers, and try again to POST a repeated make name via the Swagger UI. You should see this now instead:

Much better now, don’t you think?

Adding a more complex entity to the model

Now let’s work on an entity that’s a little bit more complex: the one we will use to represent vehicle models.

For this entity, we don’t want our API to be as low level as the one for the other three, where it was basically a thin wrapper over database tables. We want it to be a little bit more abstract and not expose the entire database structure verbatim.

Refer back to the data model. We’ll add models, model_styles and model_style_years. Let’s start by adding the following classes:

// Models/Model.cs
using System.Collections.Generic;

namespace VehicleQuotes.Models
{
    public class Model
    {
        public int ID { get; set; }
        public string Name { get; set; }
        public int MakeID { get; set; }

        public Make Make { get; set; }

        public ICollection<ModelStyle> ModelStyles { get; set; }
    }
}

// Models/ModelStyle.cs
using System.Collections.Generic;

namespace VehicleQuotes.Models
{
    public class ModelStyle
    {
        public int ID { get; set; }
        public int ModelID { get; set; }
        public int BodyTypeID { get; set; }
        public int SizeID { get; set; }

        public Model Model { get; set; }
        public BodyType BodyType { get; set; }
        public Size Size { get; set; }

        public ICollection<ModelStyleYear> ModelStyleYears { get; set; }
    }
}

// Models/ModelStyleYear.cs
namespace VehicleQuotes.Models
{
    public class ModelStyleYear
    {
        public int ID { get; set; }
        public string Year { get; set; }
        public int ModelStyleID { get; set; }

        public ModelStyle ModelStyle { get; set; }
    }
}

Notice how some of these entities now include properties whose types are other entities. Some of them are collections even. These are called Navigation Properties and are how we tell EF Core that our entities are related to one another. These will result in foreign keys being created in the database.

Take the Model entity for example. It has a property Make of type Make. It also has a MakeID property of type int. EF Core sees this and figures out that there’s a relation between the makes and models tables. Specifically, that models have a make. A many-to-one relation where the models table stores a foreign key to the makes table.

Similarly, the Model entity has a ModelStyles property of type ICollection<ModelStyleYear>. This tells EF Core that models have many model_styles. This one is a one-to-many relation from the perspective of the models table. The foreign key lives in the model_styles table and points back to models.

The official documentation is a great resource to learn more details about how relationships work in EF Core.

After that, same as before, we have to add the corresponding DbSets to our DbContext:

// Data/VehicleQuotesContext.cs
// ...

namespace VehicleQuotes
{
    public class VehicleQuotesContext : DbContext
    {
        // ...

+       public DbSet<Model> Models { get; set; }
+       public DbSet<ModelStyle> ModelStyles { get; set; }
+       public DbSet<ModelStyleYear> ModelStyleYears { get; set; }
    }
}

Don’t forget the migration script. First create it:

$ dotnet ef migrations add AddVehicleModelTables

And then apply it:

$ dotnet ef database update

Adding composite unique indexes

These vehicle model related tables also need some uniqueness enforcement. This time, however, the unique keys are composite. Meaning that they involve multiple fields. For vehicle models, for example, it makes no sense to have multiple records with the same make and name. But it does make sense to have multiple models with the same name, as long as they belong to different makes. We can solve for that with a composite index. Here’s how we create one of those in Model with EF Core:

using System.Collections.Generic;
+using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes.Models
{
+   [Index(nameof(Name), nameof(MakeID), IsUnique = true)]
    public class Model
    {
        // ...
    }
}

Very similar to what we did with the Make, BodyType, and Size entities. The only difference is that this time we included multiple fields in the parameters for the Index attribute.

We should do the same for ModelStyle and ModelStyleYear:

using System.Collections.Generic;
+using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes.Models
{
+   [Index(nameof(ModelID), nameof(BodyTypeID), nameof(SizeID), IsUnique = true)]
    public class ModelStyle
    {
        // ...
    }
}

+using Microsoft.EntityFrameworkCore;

namespace VehicleQuotes.Models
{
+   [Index(nameof(Year), nameof(ModelStyleID), IsUnique = true)]
    public class ModelStyleYear
    {
        // ...
    }
}

Don’t forget the migrations:

$ dotnet ef migrations add AddUniqueIndexesForVehicleModelTables

$ dotnet ef database update

Adding controllers with custom routes

Our data model dictates that vehicle models belong in a make. In other words, a vehicle model has no meaning by itself. It only has meaning within the context of a make. Ideally, we want our API routes to reflect this concept. In other words, instead of URLs for models to look like this: /api/Models/{id}; we’d rather them look like this: /api/Makes/{makeId}/Models/{modelId}. Let’s go ahead and scaffold a controller for this entity:

$ dotnet aspnet-codegenerator controller \
    -name ModelsController \
    -m Model \
    -dc VehicleQuotesContext \
    -async \
    -api \
    -outDir Controllers

Now let’s change the resulting Controllers/ModelsController.cs to use the URL structure that we want. To do so, we modify the Route attribute that’s applied to the ModelsController class to this:

[Route("api/Makes/{makeId}/[controller]/")]

Do a dotnet run and take a peek at the Swagger UI on https://localhost:5001 to see what the Models endpoint routes look like now:

The vehicle model routes are now nested within makes, just like we wanted.

Of course, this is just eye candy for now. We need to actually use this new makeId parameter for the logic in the endpoints. For example, one would expect a GET to /api/Makes/1/Models to return all the vehicle models that belong to the make with id 1. But right now, all vehicle models are returned regardless. All other endpoints behave similarly, there’s no limit to the operations on the vehicle models. The given makeId is not taken into consideration at all.

Let’s update the ModelsController’s GetModels method (which is the one that handles the GET /api/Makes/{makeId}/Models endpoint) to behave like one would expect. It should look like this:

[HttpGet]
public async Task<ActionResult<IEnumerable<Model>>> GetModels([FromRoute] int makeId)
{
    var make = await _context.Makes.FindAsync(makeId);

    if (make == null)
    {
        return NotFound();
    }

    return await _context.Models.Where(m => m.MakeID == makeId).ToListAsync();
}

See how we’ve included a new parameter to the method: [FromRoute] int makeId. This [FromRoute] attribute is how we tell ASP.NET Core that this endpoint will use that makeId parameter coming from the URL route. Then, we use our DbContext to try to find the make that corresponds to the given identifier. This is done in _context.Makes.FindAsync(makeId). Then, if we can’t find the given make, we return a 404 Not Found HTTP status code with the return NotFound(); line. Finally, we query the models table for all the records whose make_id matches the given parameter. That’s done in the last line of the method.

We have access to the DbContext because it has been injected as a dependency into the controller via its constructor by the framework.

The official documentation is a great resource to learn about all the possibilities when querying data with EF Core.

Let’s update the GetModel method, which handles the GET /api/Makes/{makeId}/Models/{id} endpoint, similarly.

[HttpGet("{id}")]
-public async Task<ActionResult<Model>> GetModel(int id)
+public async Task<ActionResult<Model>> GetModel([FromRoute] int makeId, int id)
{
-   var model = await _context.Models.FindAsync(id);
+   var model = await _context.Models.FirstOrDefaultAsync(m =>
+       m.MakeID == makeId && m.ID == id
+   );

    if (model == null)
    {
        return NotFound();
    }

    return model;
}

We’ve once again included the makeId as a parameter to the method and modified the EF Core query to use both the make ID and the vehicle model ID when looking for the record.

And that’s the gist of it. Other methods would need to be updated similarly. The next section will include these methods in their final form, so I won’t go through each one of them here.

Using resource models as DTOs for controllers

Now, I did say at the beginning that we wanted the vehicle model endpoint to be a bit more abstract. Right now it’s operating directly over the EF Core entities and our table. As a result, creating new vehicle models via the POST /api/Makes/{makeId}/Models endpoint is a pain. Take a look at the Swagger UI request schema for that endpoint:

This is way too much. Let’s make it a little bit more user friendly by making it more abstract.

To do that, we will introduce what I like to call a Resource Model (or DTO, Data Transfer Object, or View Model). This is a class whose only purpose is to streamline the API contract of the endpoint by defining a set of fields that clients will use to make requests and interpret responses. Something that’s simpler than our actual database structure, but still captures all the information that’s important for our application. We will update the ModelsController so that it’s able to receive objects of this new class as requests, operate on them, translate them to our EF Core entities and actual database records, and return them as a response. The hope is that, by hiding the details of our database structure, we make it easier for clients to interact with our API.

So let’s create a new ResourceModels directory in our project’s root and add these two classes:

// ResourceModels/ModelSpecification.cs
namespace VehicleQuotes.ResourceModels
{
    public class ModelSpecification
    {
        public int ID { get; set; }
        public string Name { get; set; }

        public ModelSpecificationStyle[] Styles { get; set; }
    }
}

// ResourceModels/ModelSpecificationStyle.cs
namespace VehicleQuotes.ResourceModels
{
    public class ModelSpecificationStyle
    {
        public string BodyType { get; set; }
        public string Size { get; set; }

        public string[] Years { get; set; }
    }
}

Thanks to these two, instead of that mess from above, clients POSTing to /api/Makes/{makeId}/Models will be able to use a request body like this:

{
  "name": "string",
  "styles": [
    {
      "bodyType": "string",
      "size": "string",
      "years": [
        "string"
      ]
    }
  ]
}

Which is much simpler. We have the vehicle model name and an array of styles. Each style has a body type and a size, which we can specify by their names because those are unique keys. We don’t need their integer IDs (i.e. primary keys) in order to find to them. Then, each style has an array of strings that contain the years in which those styles are available for that model. The make is part of the URL already, so we don’t need to also specify it in the request payload.

Let’s update our ModelsController to use these Resource Models instead of the Model EF Core entity. Be sure to include the namespace where the Resource Models are defined by adding the following using statement: using VehicleQuotes.ResourceModels;. Now, let’s update the GetModels method (which handles the GET /api/Makes/{makeId}/Models endpoint) so that it looks like this:

[HttpGet]
// Return a collection of `ModelSpecification`s and expect a `makeId` from the URL.
public async Task<ActionResult<IEnumerable<ModelSpecification>>> GetModels([FromRoute] int makeId)
{
    // Look for the make identified by `makeId`.
    var make = await _context.Makes.FindAsync(makeId);

    // If we can't find the make, then we return a 404.
    if (make == null)
    {
        return NotFound();
    }

    // Build a query to fetch the relevant records from the `models` table and
    // build `ModelSpecification` with the data.
    var modelsToReturn = _context.Models
        .Where(m => m.MakeID == makeId)
        .Select(m => new ModelSpecification {
            ID = m.ID,
            Name = m.Name,
            Styles = m.ModelStyles.Select(ms => new ModelSpecificationStyle {
                BodyType = ms.BodyType.Name,
                Size = ms.Size.Name,
                Years = ms.ModelStyleYears.Select(msy => msy.Year).ToArray()
            }).ToArray()
        });

    // Execute the query and respond with the results.
    return await modelsToReturn.ToListAsync();
}

The first thing that we changed was the return type. Instead of Task<ActionResult<IEnumerable<Model>>>, the method now returns Task<ActionResult<IEnumerable<ModelSpecification>>>. We’re going to use our new Resource Models as these endpoints’ contract, so we need to make sure we are returning those. Next, we considerably changed the LINQ expression that searches the database for the vehicle model records we want. The filtering logic (given by the Where) is the same. That is, we’re still searching for vehicle models within the given make ID. What we changed was the projection logic in the Select. Our Action Method now returns a collection of ModelSpecification objects, so we updated the Select to produce such objects, based on the records from the models table that match our search criteria. We build ModelSpecifications using the data coming from models records and their related model_styles and model_style_years. Finally, we asynchronously execute the query to fetch the data from the database and return it.

Next, let’s move on to the GetModel method, which handles the GET /api/Makes/{makeId}/Models/{id} endpoint. This is what it should look like:

[HttpGet("{id}")]
// Return a `ModelSpecification`s and expect `makeId` and `id` from the URL.
public async Task<ActionResult<ModelSpecification>> GetModel([FromRoute] int makeId, [FromRoute] int id)
{
    // Look for the model specified by the given identifiers and also load
    // all related data that we care about for this method.
    var model = await _context.Models
        .Include(m => m.ModelStyles).ThenInclude(ms => ms.BodyType)
        .Include(m => m.ModelStyles).ThenInclude(ms => ms.Size)
        .Include(m => m.ModelStyles).ThenInclude(ms => ms.ModelStyleYears)
        .FirstOrDefaultAsync(m => m.MakeID == makeId && m.ID == id);

    // If we couldn't find it, respond with a 404.
    if (model == null)
    {
        return NotFound();
    }

    // Use the fetched data to construct a `ModelSpecification` to use in the response.
    return new ModelSpecification {
        ID = model.ID,
        Name = model.Name,
        Styles = model.ModelStyles.Select(ms => new ModelSpecificationStyle {
            BodyType = ms.BodyType.Name,
            Size = ms.Size.Name,
            Years = ms.ModelStyleYears.Select(msy => msy.Year).ToArray()
        }).ToArray()
    };
}

Same as before, we changed the return type of the method to be ModelSpecification. Then, we modified the query so that it loads all the related data for the Model entity via its navigation properties. That’s what the Include and ThenInclude calls do. We need this data loaded because we use it in the method’s return statement to build the ModelSpecification that will be included in the response. The logic to build it is very similar to that of the previous method.

You can learn more about the various available approaches for loading data with EF Core in the official documentation.

Next is the PUT /api/Makes/{makeId}/Models/{id} endpoint, handled by the PutModel method:

[HttpPut("{id}")]
// Expect `makeId` and `id` from the URL and a `ModelSpecification` from the request payload.
public async Task<IActionResult> PutModel([FromRoute] int makeId, int id, ModelSpecification model)
{
    // If the id in the URL and the request payload are different, return a 400.
    if (id != model.ID)
    {
        return BadRequest();
    }

    // Obtain the `models` record that we want to update. Include any related
    // data that we want to update as well.
    var modelToUpdate = await _context.Models
        .Include(m => m.ModelStyles)
        .FirstOrDefaultAsync(m => m.MakeID == makeId && m.ID == id);

    // If we can't find the record, then return a 404.
    if (modelToUpdate == null)
    {
        return NotFound();
    }

    // Update the record with what came in the request payload.
    modelToUpdate.Name = model.Name;

    // Build EF Core entities based on the incoming Resource Model object.
    modelToUpdate.ModelStyles = model.Styles.Select(style => new ModelStyle {
        BodyType = _context.BodyTypes.Single(bodyType => bodyType.Name == style.BodyType),
        Size = _context.Sizes.Single(size => size.Name == style.Size),

        ModelStyleYears = style.Years.Select(year => new ModelStyleYear {
            Year = year
        }).ToList()
    }).ToList();

    try
    {
        // Try saving the changes. This will run the UPDATE statement in the database.
        await _context.SaveChangesAsync();
    }
    catch (Microsoft.EntityFrameworkCore.DbUpdateException)
    {
        // If there's an error updating, respond accordingly.
        return Conflict();
    }

    // Finally return a 204 if everything went well.
    return NoContent();
}

The purpose of this endpoint is to update existing resources. So, it receives a representation of said resource as a parameter that comes from the request body. Before, it expected an instance of the Model entity, but now, we’ve changed it to receive a ModelSpecification. The rest of the method is your usual structure of first obtaining the record to update by the given IDs, then changing its values according to what came in as a parameter, and finally, saving the changes.

You probably get the idea by now: since the API is using the Resource Model, we need to change input and output values for the methods and run some logic to translate between Resource Model objects and Data Model objects that EF Core can understand so that it can perform its database operations.

That said, here’s what the PostModel Action Method, handler of the POST /api/Makes/{makeId}/Models endpoint, should look like:

[HttpPost]
// Return a `ModelSpecification`s and expect `makeId` from the URL and a `ModelSpecification` from the request payload.
public async Task<ActionResult<ModelSpecification>> PostModel([FromRoute] int makeId, ModelSpecification model)
{
    // First, try to find the make specified by the incoming `makeId`.
    var make = await _context.Makes.FindAsync(makeId);

    // Respond with 404 if not found.
    if (make == null)
    {
        return NotFound();
    }

    // Build out a new `Model` entity, complete with all related data, based on
    // the `ModelSpecification` parameter.
    var modelToCreate = new Model {
        Make = make,
        Name = model.Name,

        ModelStyles = model.Styles.Select(style => new ModelStyle {
            // Notice how we search both body type and size by their name field.
            // We can do that because their names are unique.
            BodyType = _context.BodyTypes.Single(bodyType => bodyType.Name == style.BodyType),
            Size = _context.Sizes.Single(size => size.Name == style.Size),

            ModelStyleYears = style.Years.Select(year => new ModelStyleYear {
                Year = year
            }).ToArray()
        }).ToArray()
    };

    // Add it to the DbContext.
    _context.Add(modelToCreate);

    try
    {
        // Try running the INSERTs.
        await _context.SaveChangesAsync();
    }
    catch (Microsoft.EntityFrameworkCore.DbUpdateException)
    {
        // Return accordingly if an error happens.
        return Conflict();
    }

    // Get back the autogenerated ID of the record we just INSERTed.
    model.ID = modelToCreate.ID;

    // Finally, return a 201 including a location header containing the newly
    // created resource's URL and the resource itself in the response payload.
    return CreatedAtAction(
        nameof(GetModel),
        new { makeId = makeId, id = model.ID },
        model
    );
}

All that should be pretty self explanatory by now. Moving on to the DeleteModel method which handles the DELETE /api/Makes/{makeId}/Models/{id} endpoint:

[HttpDelete("{id}")]
// Expect `makeId` and `id` from the URL.
public async Task<IActionResult> DeleteModel([FromRoute] int makeId, int id)
{
    // Try to find the record identified by the ids from the URL.
    var model = await _context.Models.FirstOrDefaultAsync(m => m.MakeID == makeId && m.ID == id);

    // Respond with a 404 if we can't find it.
    if (model == null)
    {
        return NotFound();
    }

    // Mark the entity for removal and run the DELETE.
    _context.Models.Remove(model);
    await _context.SaveChangesAsync();

    // Respond with a 204.
    return NoContent();
}

And that’s all for that controller. Hopefully that demonstrated what it looks like to have endpoints that operate using objects other than the EF Core entities. Fire up the app with dotnet run and explore the Swagger UI and you’ll see the changes that we’ve made reflected in there. Try it out. Try CRUDing some vehicle models. And don’t forget to take a look at our POST endpoint specification which looks much more manageable now:

Which means that you can send in something like this, for example:

{
    "name": "Corolla",
    "styles": [
        {
            "bodyType": "Sedan",
            "size": "Compact",
            "years": [ "2000", "2001" ]
        }
    ]
}

This will work assuming you’ve created at least one make to add the vehicle model to, as well as a body type whose name is Sedan and a size whose name is Compact.

There’s also a ModelExists method in that controller which we don’t need anymore. You can delete it.

Validation using built-in Data Annotations

Depending on how “creative” you were in the previous section when trying to CRUD models, you may have run into an issue or two regarding the data that’s allowed into our database. We solve that by implementing input validation. In ASP.NET Core, the easiest way to implement validation is via Data Annotation attributes on the entities or other objects that controllers receive as request payloads. So let’s see about adding some validation to our app. Since our ModelsController uses the ModelSpecification and ModelSpecificationStyle Resource Models to talk to clients, let’s start there. Here’s the diff:

+using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.ResourceModels
{
    public class ModelSpecification
    {
        public int ID { get; set; }
+       [Required]
        public string Name { get; set; }

+       [Required]
        public ModelSpecificationStyle[] Styles { get; set; }
    }
}

+using System.ComponentModel.DataAnnotations;

namespace VehicleQuotes.ResourceModels
{
    public class ModelSpecificationStyle
    {
+       [Required]
        public string BodyType { get; set; }
+       [Required]
        public string Size { get; set; }

+       [Required]
+       [MinLength(1)]
        public string[] Years { get; set; }
    }
}

And just like that, we get a good amount of functionality. We use the Required and MinLength attributes from the System.ComponentModel.DataAnnotations namespace to specify that some fields are required, and that our Years array needs to contain at least one element. When the app receives a request to the PUT or POST endpoints — which are the ones that expect a ModelSpecification as the payload — validation kicks in. If it fails, the action method is never executed and a 400 status code is returned as a response. Try POSTing to /api/Makes/{makeId}/Models with a payload that violates some of these rules to see for yourself. I tried for example sending this:

{
  "name": null,
  "styles": [
    {
      "bodyType": "Sedan",
      "size": "Full size",
      "years": []
    }
  ]
}

And I got back a 400 response with this payload:

{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-0fd4f00eeb9f2f458ccefc180fcfba1c-79a618f13218394b-00",
  "errors": {
    "Name": [
      "The Name field is required."
    ],
    "Styles[0].Years": [
      "The field Years must be a string or array type with a minimum length of '1'."
    ]
  }
}

Pretty neat, huh? With minimal effort, we have some basic validation rules in place and a pretty usable response for when errors occur.

To learn more about model validation, including all the various validation attributes included in the framework, check the official documentation: Model validation and System.ComponentModel.DataAnnotations Namespace.

Validation using custom attributes

Of course, the framework is never going to cover all possible validation scenarios with the built-in attributes. Case in point, it’d be great to validate that the Years array contains values that look like actual years. That is, four-character, digit-only strings. There are no validation attributes for that. So, we need to create our own. Let’s add this file into a new Validations directory:

// Validations/ContainsYearsAttribute.cs
using System;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Runtime.CompilerServices;

namespace VehicleQuotes.Validation
{
    // In the .NET Framework, attribute classes need to have their name suffixed with the word "Attribute".
    // Validation attributes need to inherit from `System.ComponentModel.DataAnnotations`'s `ValidationAttribute` class
    // and override the `IsValid` method.
    public class ContainsYearsAttribute : ValidationAttribute
    {
        private string propertyName;

        // This constructor is called by the framework when the attribute is applied to some member. In this specific
        // case, we define a `propertyName` parameter annotated with a `CallerMemberName` attribute. This makes it so
        // the framework sends in the name of the member to which our `ContainsYears` attribute is applied to.
        // We store the value to use it later when constructing our validation error message.
        // Check https://docs.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.callermembernameattribute?view=net-5.0
        // for more info on `CallerMemberName`.
        public ContainsYearsAttribute([CallerMemberName] string propertyName = null)
        {
            this.propertyName = propertyName;
        }

        // This method is called by the framework during validation. `value` is the actual value of the field that this
        // attribute will validate.
        protected override ValidationResult IsValid(object value, ValidationContext validationContext)
        {
            // By only applying the validation checks when the value is not null, we make it possible for this
            // attribute to work on optional fields. In other words, this attribute will skip validation if there is no
            // value to validate.
            if (value != null)
            {
                // Check if all the elements of the string array are valid years. Check the `IsValidYear` method below
                // to see what checks are applied for each of the array elements.
                var isValid = (value as string[]).All(IsValidYear);

                if (!isValid)
                {
                    // If not, return an error.
                    return new ValidationResult(GetErrorMessage());
                }
            }

            // Return a successful validation result if no errors were detected.
            return ValidationResult.Success;
        }

        // Determines if a given value is valid by making sure it's not null, nor empty, that its length is 4 and that
        // all its characters are digits.
        private bool IsValidYear(string value) =>
            !String.IsNullOrEmpty(value) && value.Length == 4 && value.All(Char.IsDigit);

        // Builds a user friendly error message which includes the name of the field that this validation attribute has
        // been applied to.
        private string GetErrorMessage() =>
            $"The {propertyName} field must be an array of strings containing four digits.";
    }
}

Check the comments in the code for more details into how that class works. Then, we apply our custom attribute to our ModelSpecificationStyle class in the same way that we applied the built in ones:

using System.ComponentModel.DataAnnotations;
+using VehicleQuotes.Validation;

namespace VehicleQuotes.ResourceModels
{
    public class ModelSpecificationStyle
    {
        // ...

        [Required]
        [MinLength(1)]
+       [ContainsYears]
        public string[] Years { get; set; }
    }
}

Now do a dotnet run and try to POST to /api/Makes/{makeId}/Models this payload:

{
  "name": "Rav4",
  "styles": [
    {
      "bodyType": "SUV",
      "size": "Mid size",
      "years": [ "not_a_year" ]
    }
  ]
}

That should make the API respond with this:

{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-9980325f3e388f48a5975ef382d5b137-2d55da1bb9613e4f-00",
  "errors": {
    "Styles[0].Years": [
      "The Years field must be an array of strings containing four numbers."
    ]
  }
}

That’s our custom validation attribute doing its job.

There’s another aspect that we could validate using a custom validation attribute. What happens if we try to POST a payload with a body type or size that doesn’t exist? These queries from the PostModel method would throw an InvalidOperationException:

BodyType = _context.BodyTypes.Single(bodyType => bodyType.Name == style.BodyType)

and

Size = _context.Sizes.Single(size => size.Name == style.Size)

They do so because we used the Single method, which is designed like that. It tries to find a body type or size whose name is the given value, can’t find it, and thus, throws an exception.

If, for example, we wanted not-founds to return null, we could have used SingleOrDefault instead.

This unhandled exception results in a response that’s quite unbecoming:

So, to prevent that exception and control the error messaging, we need a couple of new validation attributes that go into the body_types and sizes tables and check if the given values exist. Here’s what one would look like:

// Validations/VehicleBodyTypeAttribute.cs
using System;
using System.ComponentModel.DataAnnotations;
using Microsoft.EntityFrameworkCore;
using System.Linq;

namespace VehicleQuotes.Validation
{
    public class VehicleBodyTypeAttribute : ValidationAttribute
    {
        protected override ValidationResult IsValid(object value, ValidationContext validationContext)
        {
            if (value == null) return ValidationResult.Success;

            var dbContext = validationContext.GetService(typeof(VehicleQuotesContext)) as VehicleQuotesContext;

            var bodyTypes = dbContext.BodyTypes.Select(bt => bt.Name).ToList();

            if (!bodyTypes.Contains(value))
            {
                var allowed = String.Join(", ", bodyTypes);
                return new ValidationResult(
                    $"Invalid vehicle body type {value}. Allowed values are {allowed}."
                );
            }

            return ValidationResult.Success;
        }
    }
}

You can find the other one here: Validations/VehicleSizeAttribute.cs.

These two are very similar to one another. The most interesting part is how we use the IsValid method’s second parameter (ValidationContext) to obtain an instance of VehicleQuotesContext that we can use to query the database. The rest should be pretty self-explanatory. These attributes are classes that inherit from System.ComponentModel.DataAnnotations’s ValidationAttribute and implement the IsValid method. The method then checks that the value under scrutiny exists in the corresponding table and if it does not, raises a validation error. The validation error includes a list of all allowed values. They can be applied to our ModelSpecificationStyle class like so:

// ...
namespace VehicleQuotes.ResourceModels
{
    public class ModelSpecificationStyle
    {
        [Required]
+       [VehicleBodyType]
        public string BodyType { get; set; }

        [Required]
+       [VehicleSize]
        public string Size { get; set; }

        //...
    }
}

Now, a request like this:

{
  "name": "Rav4",
  "styles": [
    {
      "bodyType": "not_a_body_type",
      "size": "Mid size",
      "years": [ "2000" ]
    }
  ]
}

Produces a response like this:

{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-9ad59a7aff60944ab54c19a73be73cc7-eeabafe03df74e40-00",
  "errors": {
    "Styles[0].BodyType": [
      "Invalid vehicle body type not_a_body_type. Allowed values are Coupe, Sedan, Convertible, Hatchback, SUV, Truck."
    ]
  }
}

Implementing endpoints for quote rules and overrides

At this point we’ve explored many of the most common features available to us for developing Web APIs. So much so that implementing the next two pieces of functionality for our app doesn’t really introduce any new concepts. So, I wont discuss that here in great detail.

Feel free to browse the source code on GitHub if you want though. These are the relevant files:

The FeatureTypeAttribute class is interesting in that it provides another example of a validation attribute. This time is one that makes sure the value being validated is included in an array of strings that’s defined literally in the code.

Other than that, it’s all stuff we’ve already covered: models, migrations, scaffolding controllers, custom routes, resource models, etc.

If you are following along, be sure to add those files and run a dotnet ef database update to apply the migration.

Implementing the quote model

Let’s now start implementing the main capability of our app: calculating quotes for vehicles. Let’s start with the Quote entity. This is what the new Models/Quote.cs file containing the entity class will look like:

// Models/Quote.cs
using System;
using System.ComponentModel.DataAnnotations.Schema;

namespace VehicleQuotes.Models
{
    public class Quote
    {
        public int ID { get; set; }

        // Directly tie this quote record to a specific vehicle that we have
        // registered in our db, if we have it.
        public int? ModelStyleYearID { get; set; }

        // If we don't have the specific vehicle in our db, then store the
        // vehicle model details independently.
        public string Year { get; set; }
        public string Make { get; set; }
        public string Model { get; set; }
        public int BodyTypeID { get; set; }
        public int SizeID { get; set; }

        public bool ItMoves { get; set; }
        public bool HasAllWheels { get; set; }
        public bool HasAlloyWheels { get; set; }
        public bool HasAllTires { get; set; }
        public bool HasKey { get; set; }
        public bool HasTitle { get; set; }
        public bool RequiresPickup { get; set; }
        public bool HasEngine { get; set; }
        public bool HasTransmission { get; set; }
        public bool HasCompleteInterior { get; set; }

        public int OfferedQuote { get; set; }
        public string Message { get; set; }
        public DateTime CreatedAt { get; set; }

        public ModelStyleYear ModelStyleYear { get; set; }

        public BodyType BodyType { get; set; }
        public Size Size { get; set; }
    }
}

This should be pretty familiar by now. It’s a plain old class that defines a number of properties, one for each of the fields in the resulting table and a few navigation properties that serve to access related data.

The only aspect worth noting is that we’ve defined the ModelStyleYearID property as a nullable integer (with int?). This is because, like we discussed at the beginning, the foreign key from quotes to vehicle_style_years is actually optional. The reason being that we may receive a quote request for a vehicle that we don’t have registered in our database. We need to be able to support quoting those vehicles too, so if we don’t have the requested vehicle registered, then that foreign key will stay unpopulated and we’ll rely on the other fields (i.e. Year, Make, Model, BodyTypeID and SizeID) to identify the vehicle and calculate the quote for it.

Using Dependency Injection

So far we’ve been putting a lot of logic in our controllers. That’s generally not ideal, but fine as long as the logic is simple. The problem is that a design like that can quickly become a hindrance for maintainability and testing as our application grows more complex. For the logic that calculates a quote, we’d be better served by implementing it in its own class, outside of the controller that should only care about defining endpoints and handling HTTP concerns. Then, the controller can be given access to that class and delegate to it all the quote calculation logic. Thankfully, ASP.NET Core includes an IoC container by default, which allows us to use Dependency Injection to solve these kinds of problems. Let’s see what that looks like.

For working with quotes, we want to offer two endpoints:

A POST api/Quotes that captures the vehicle information, calculates the quote, keeps record of the request, and responds with the calculated value.
A GET api/Quotes that returns all the currently registered quotes in the system.

Using the Dependency Injection capabilities, a controller that implements those two could look like this:

using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using VehicleQuotes.ResourceModels;
using VehicleQuotes.Services;

namespace VehicleQuotes.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class QuotesController : ControllerBase
    {
        private readonly QuoteService _service;

        // When intiating the request processing logic, the framework recognizes
        // that this controller has a dependency on QuoteService and expects an
        // instance of it to be injected via the constructor. The framework then
        // does what it needs to do in order to provide that dependency.
        public QuotesController(QuoteService service)
        {
            _service = service;
        }

        // GET: api/Quotes
        [HttpGet]
        // This method returns a collection of a new resource model instead of just the `Quote` entity directly.
        public async Task<ActionResult<IEnumerable<SubmittedQuoteRequest>>> GetAll()
        {
            // Instead of directly implementing the logic in this method, we call on
            // the service class and let it take care of the rest.
            return await _service.GetAllQuotes();
        }

        // POST: api/Quotes
        [HttpPost]
        // This method receives as a paramater a `QuoteRequest` of just the `Quote` entity directly.
        // That way callers of this endpoint don't need to be exposed to the details of our data model implementation.
        public async Task<ActionResult<SubmittedQuoteRequest>> Post(QuoteRequest request)
        {
            // Instead of directly implementing the logic in this method, we call on
            // the service class and let it take care of the rest.
            return await _service.CalculateQuote(request);
        }
    }
}

As you can see, we’ve once again opted to abstract away clients from the implementation details of our data model and used Resource Models for the API contract instead of the Quote entity directly. We have one for input data that’s called QuoteRequest and another one for output: SubmittedQuoteRequest. Not very remarkable by themselves, but feel free to explore the source code in the GitHub repo.

This controller has a dependency on QuoteService, which it uses to perform all of the necessary logic. This class is not defined yet so let’s do that next:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using VehicleQuotes.Models;
using VehicleQuotes.ResourceModels;

namespace VehicleQuotes.Services
{
    public class QuoteService
    {
        private readonly VehicleQuotesContext _context;

        // This constructor defines a dependency on VehicleQuotesContext, similar to most of our controllers.
        // Via the built in dependency injection features, the framework makes sure to provide this parameter when
        // creating new instances of this class.
        public QuoteService(VehicleQuotesContext context)
        {
            _context = context;
        }

        // This method takes all the records from the `quotes` table and constructs `SubmittedQuoteRequest`s with them.
        // Then returns that as a list.
        public async Task<List<SubmittedQuoteRequest>> GetAllQuotes()
        {
            var quotesToReturn = _context.Quotes.Select(q => new SubmittedQuoteRequest
            {
                ID = q.ID,
                CreatedAt = q.CreatedAt,
                OfferedQuote = q.OfferedQuote,
                Message = q.Message,

                Year = q.Year,
                Make = q.Make,
                Model = q.Model,
                BodyType = q.BodyType.Name,
                Size = q.Size.Name,

                ItMoves = q.ItMoves,
                HasAllWheels = q.HasAllWheels,
                HasAlloyWheels = q.HasAlloyWheels,
                HasAllTires = q.HasAllTires,
                HasKey = q.HasKey,
                HasTitle = q.HasTitle,
                RequiresPickup = q.RequiresPickup,
                HasEngine = q.HasEngine,
                HasTransmission = q.HasTransmission,
                HasCompleteInterior = q.HasCompleteInterior,
            });

            return await quotesToReturn.ToListAsync();
        }

        // This method takes an incoming `QuoteRequest` and calculates a quote based on the vehicle described by it.
        // To calculate this quote, it looks for any overrides before trying to use the currently existing rules defined
        // in the `quote_rules` table. It also stores a record on the `quotes` table with all the incoming data and the
        // quote calculation result. It returns back the quote value as well as a message explaining the conditions of
        // the quote.
        public async Task<SubmittedQuoteRequest> CalculateQuote(QuoteRequest request)
        {
            var response = this.CreateResponse(request);
            var quoteToStore = await this.CreateQuote(request);
            var requestedModelStyleYear = await this.FindModelStyleYear(request);
            QuoteOverride quoteOverride = null;

            if (requestedModelStyleYear != null)
            {
                quoteToStore.ModelStyleYear = requestedModelStyleYear;

                quoteOverride = await this.FindQuoteOverride(requestedModelStyleYear);

                if (quoteOverride != null)
                {
                    response.OfferedQuote = quoteOverride.Price;
                }
            }

            if (quoteOverride == null)
            {
                response.OfferedQuote = await this.CalculateOfferedQuote(request);
            }

            if (requestedModelStyleYear == null)
            {
                response.Message = "Offer subject to change upon vehicle inspection.";
            }

            quoteToStore.OfferedQuote = response.OfferedQuote;
            quoteToStore.Message = response.Message;

            _context.Quotes.Add(quoteToStore);
            await _context.SaveChangesAsync();

            response.ID = quoteToStore.ID;
            response.CreatedAt = quoteToStore.CreatedAt;

            return response;
        }

        // Creates a `SubmittedQuoteRequest`, initialized with default values, using the data from the incoming
        // `QuoteRequest`. `SubmittedQuoteRequest` is what gets returned in the response payload of the quote endpoints.
        private SubmittedQuoteRequest CreateResponse(QuoteRequest request)
        {
            return new SubmittedQuoteRequest
            {
                OfferedQuote = 0,
                Message = "This is our final offer.",

                Year = request.Year,
                Make = request.Make,
                Model = request.Model,
                BodyType = request.BodyType,
                Size = request.Size,

                ItMoves = request.ItMoves,
                HasAllWheels = request.HasAllWheels,
                HasAlloyWheels = request.HasAlloyWheels,
                HasAllTires = request.HasAllTires,
                HasKey = request.HasKey,
                HasTitle = request.HasTitle,
                RequiresPickup = request.RequiresPickup,
                HasEngine = request.HasEngine,
                HasTransmission = request.HasTransmission,
                HasCompleteInterior = request.HasCompleteInterior,
            };
        }

        // Creates a `Quote` based on the data from the incoming `QuoteRequest`. This is the object that gets eventually
        // stored in the database.
        private async Task<Quote> CreateQuote(QuoteRequest request)
        {
            return new Quote
            {
                Year = request.Year,
                Make = request.Make,
                Model = request.Model,
                BodyTypeID = (await _context.BodyTypes.SingleAsync(bt => bt.Name == request.BodyType)).ID,
                SizeID = (await _context.Sizes.SingleAsync(s => s.Name == request.Size)).ID,

                ItMoves = request.ItMoves,
                HasAllWheels = request.HasAllWheels,
                HasAlloyWheels = request.HasAlloyWheels,
                HasAllTires = request.HasAllTires,
                HasKey = request.HasKey,
                HasTitle = request.HasTitle,
                RequiresPickup = request.RequiresPickup,
                HasEngine = request.HasEngine,
                HasTransmission = request.HasTransmission,
                HasCompleteInterior = request.HasCompleteInterior,

                CreatedAt = DateTime.Now
            };
        }

        // Tries to find a registered vehicle that matches the one for which the quote is currently being requested.
        private async Task<ModelStyleYear> FindModelStyleYear(QuoteRequest request)
        {
            return await _context.ModelStyleYears.FirstOrDefaultAsync(msy =>
                msy.Year == request.Year &&
                msy.ModelStyle.Model.Make.Name == request.Make &&
                msy.ModelStyle.Model.Name == request.Model &&
                msy.ModelStyle.BodyType.Name == request.BodyType &&
                msy.ModelStyle.Size.Name == request.Size
            );
        }

        // Tries to find an override for the vehicle for which the quote is currently being requested.
        private async Task<QuoteOverride> FindQuoteOverride(ModelStyleYear modelStyleYear)
        {
            return await _context.QuoteOverides
                .FirstOrDefaultAsync(qo => qo.ModelStyleYear == modelStyleYear);
        }

        // Uses the rules stored in the `quote_rules` table to calculate how much money to offer for the vehicle
        // described in the incoming `QuoteRequest`.
        private async Task<int> CalculateOfferedQuote(QuoteRequest request)
        {
            var rules = await _context.QuoteRules.ToListAsync();

            // Given a vehicle feature type, find a rule that applies to that feature type and has the value that
            // matches the condition of the incoming vehicle being quoted.
            Func<string, QuoteRule> theMatchingRule = featureType =>
                rules.FirstOrDefault(r =>
                    r.FeatureType == featureType &&
                    r.FeatureValue == request[featureType]
                );

            // For each vehicle feature that we care about, sum up the the monetary values of all the rules that match
            // the given vehicle condition.
            return QuoteRule.FeatureTypes.All
                .Select(theMatchingRule)
                .Where(r => r != null)
                .Sum(r => r.PriceModifier);
        }
    }
}

Finally, we need to tell the framework that this class is available for Dependency Injection. Similarly to how we did with our VehicleQuotesContext, we do so in the Startup.cs file’s ConfigureServices method. Just add this line at the top:

services.AddScoped<Services.QuoteService>();

The core tenet of Inversion of Control is to depend on abstractions, not on implementations. So ideally, we would not have our controller directly call for a QuoteService instance. Instead, we would have it reference an abstraction, e.g. an interface like IQuoteService. The statement on Startup.cs would then look like this instead: services.AddScoped<Services.IQuoteService, Services.QuoteService>();.

This is important because it would allow us to unit test the component that depends on our service class (i.e. the controller in this case) by passing it a mock object — one that also implements IQuoteService but does not really implement all the functionality of the actual QuoteService class. Since the controller only knows about the interface (that is, it “depends on an abstraction”), the actual object that we give it as a dependency doesn’t matter to it, as long as it implements that interface. This ability to inject mocks as dependencies is invaluable during testing. Testing is beyond the scope of this article though, so I’ll stick with the simpler approach with a static dependency on a concrete class. Know that this is not a good practice when it comes to actual production systems.

And that’s all it takes. Once you add a few rules via POST /api/QuoteRules, you should be able to get some vehicles quoted with POST /api/Quotes. And also see what the system has stored via GET /api/Quotes.

And that’s all the functionality that we set out to build into our REST API! There are a few other neat things that I thought I’d include though.

Adding seed data for lookup tables

Our vehicle size and body type data isn’t meant to really change much. In fact, we could even preload that data when our application starts. EF Core provides a data seeding feature that we can access via configurations on the DbContext itself. For our case, we could add this method to our VehicleQuotesContext:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Size>().HasData(
        new Size { ID = 1, Name = "Subcompact" },
        new Size { ID = 2, Name = "Compact" },
        new Size { ID = 3, Name = "Mid Size" },
        new Size { ID = 5, Name = "Full Size" }
    );

    modelBuilder.Entity<BodyType>().HasData(
        new BodyType { ID = 1, Name = "Coupe" },
        new BodyType { ID = 2, Name = "Sedan" },
        new BodyType { ID = 3, Name = "Hatchback" },
        new BodyType { ID = 4, Name = "Wagon" },
        new BodyType { ID = 5, Name = "Convertible" },
        new BodyType { ID = 6, Name = "SUV" },
        new BodyType { ID = 7, Name = "Truck" },
        new BodyType { ID = 8, Name = "Mini Van" },
        new BodyType { ID = 9, Name = "Roadster" }
    );
}

OnModelCreating is a hook that we can define to run some code at the time the model is being created for the first time. Here, we’re using it to seed some data. In order to apply that, a migration needs to be created and executed. If you’ve added some data to the database, be sure to wipe it before running the migration so that we don’t run into unique constraint violations. Here are the migrations:

$ dotnet ef migrations add AddSeedDataForSizesAndBodyTypes

$ dotnet ef database update

After that’s done, it no longer makes sense to allow creating, updating, deleting and fetching individual sizes and body types, so I would delete those endpoints from the respective controllers.

There are other options for data seeding in EF Core. Take a look: Data Seeding.

Improving the Swagger UI via XML comments

Our current auto-generated Swagger UI is pretty awesome. Especially considering that we got it for free. It’s a little lacking when it comes to more documentation about specific endpoint summaries or expected responses. The good news is that there’s a way to leverage C# XML Comments in order to improve the Swagger UI.

We can add support for that by configuring our project to produce, at build time, an XML file with the docs that we write. In order to do so, we need to update the VehicleQuotes.csproj like this:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <!-- ... -->
+   <GenerateDocumentationFile>true</GenerateDocumentationFile>
+   <NoWarn>$(NoWarn);1591</NoWarn>
  </PropertyGroup>

  <!-- ... -->
</Project>

GenerateDocumentationFile is the flag that tells the .NET 5 build tools to generate the documentation file. The NoWarn element prevents our build output from getting cluttered with a lot of warnings saying that some classes and methods are not properly documented. We don’t want that because we just want to write enough documentation for the Swagger UI. And that includes only the controllers.

You can run dotnet build and look for the new file in bin/Debug/net5.0/VehicleQuotes.xml.

Then, we need to update Startup.cs. First we need to add the following using statements:

using System.IO;
using System.Reflection;

And add the following code to the ConfigureServices method on Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    // ...

    services.AddSwaggerGen(c =>
    {
        c.SwaggerDoc("v1", new OpenApiInfo { Title = "VehicleQuotes", Version = "v1" });

+       c.IncludeXmlComments(
+           Path.Combine(
+               AppContext.BaseDirectory,
+               $"{Assembly.GetExecutingAssembly().GetName().Name}.xml"
+           )
        );
    });

    // ...
}

This makes it so the SwaggerGen service knows to look for the XML documentation file when building up the Open API specification file used for generating the Swagger UI.

Now that all of that is set up, we can actually write some XML comments and attributes that will enhance our Swagger UI. As an example, put this on top of ModelsController’s Post method:

/// <summary>
/// Creates a new vehicle model for the given make.
/// </summary>
/// <param name="makeId">The ID of the vehicle make to add the model to.</param>
/// <param name="model">The data to create the new model with.</param>
/// <response code="201">When the request is valid.</response>
/// <response code="404">When the specified vehicle make does not exist.</response>
/// <response code="409">When there's already another model in the same make with the same name.</response>
[HttpPost]
[ProducesResponseType(StatusCodes.Status201Created)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
[ProducesResponseType(StatusCodes.Status409Conflict)]
public async Task<ActionResult<ModelSpecification>> Post([FromRoute] int makeId, ModelSpecification model)
{
    // ...
}

Then, the Swagger UI now looks like this for this endpoint:

Configuring the app via settings files and environment variables

Another aspect that’s important to web applications is having them be configurable via things like configuration files or environment variables. The framework already has provision for this, we just need to use it. I’m talking about the appsettings files.

We have two of them created for us by default: appsettings.json which is applied in all environments, and appsettings.Development.json that is applied only under development environments. The environment is given by the ASPNETCORE_ENVIRONMENT environment variable, and it can be set to either Development, Staging, or Production by default. That means that if we had, for example, an appsettings.Staging.json file, the settings defined within would be loaded if the ASPNETCORE_ENVIRONMENT environment variable were set to Staging. You get the idea.

You can learn more about configuration and environments in the official documentation.

Anyway, let’s add a new setting on appsettings.json:

{
  // ...
+ "DefaultOffer": 77
}

We’ll use this setting to give default offers when we’re not able to calculate appropriate quotes for vehicles. This can happen if we don’t have rules, or if the ones we have don’t match any of the incoming vehicle features or if for some other reason the final sum ends up in zero or negative number. We can use this setting in our QuoteService like so:

// ...
+using Microsoft.Extensions.Configuration;

namespace VehicleQuotes.Services
{
    public class QuoteService
    {
        // ...
+       private readonly IConfiguration _configuration;

-       public QuoteService(VehicleQuotesContext context)
+       public QuoteService(VehicleQuotesContext context, IConfiguration configuration)
        {
            _context = context;
+           _configuration = configuration;
        }

        // ...

        public async Task<SubmittedQuoteRequest> CalculateQuote(QuoteRequest request)
        {
            // ...

+           if (response.OfferedQuote <= 0)
+           {
+               response.OfferedQuote = _configuration.GetValue<int>("DefaultOffer", 0);
+           }

            quoteToStore.OfferedQuote = response.OfferedQuote;

            // ...
        }

        // ...
    }
}

Here, we’ve added a new parameter to the constructor to specify that VehicleQuotesContext has a dependency on IConfiguration. This prompts the framework to provide an instance of that when instantiating the class. We can use that instance to access the settings that we defined in the appsettings.json file via its GetValue method, like I demonstrated above.

The value of the settings in appsettings.json can be overridden by environment variables as well. On Linux, for example, we can run the app and set an environment value with a line like this:

$ DefaultOffer=123 dotnet run

This will make the application use 123 instead of 77 when it comes to the DefaultOffer setting. This flexibility is great from a DevOps perspective. And we had to do minimal work in order to get that going.

That’s all for now

And that’s it! In this article we’ve gone through many of the features offered in .NET 5, ASP.NET Core, and Entity Framework Core to support some of the most common use cases when it comes to developing Web API applications.

We’ve installed .NET 5 and created an ASP.NET Core Web API project with EF Core and a few bells and whistles, created controllers to support many different endpoints, played a little bit with routes and response codes, created and built upon a data model and updated a database via entities and migrations, implemented database constraints using unique indexes, implemented input validation using both built-in and custom validation attributes, implemented resource models as DTOs for defining the contract of some of our API endpoints, tapped into the built-in dependency injection capabilities, explored and improved the auto-generated Swagger UI, added seed data for our database, learned about configuration via settings files and environment variables.

.NET 5 is looking great.

Decreasing your website load time

2020-01-07T00:00:00+00:00

Photo by Johan Larsson, used under CC BY 2.0

We live in a competitive world, and the web is no different. Improving latency issues is crucial to any Search Engine Optimization (SEO) strategy, increasing the website’s ranking and organic traffic (visitors from search engines) as a result.

There are many factors that can lead to a faster response time, including optimization of your hosting plan, server proximity to your main traffic source, or utilization of a Content Distribution Network (CDN) if you are expecting visitors on an international level. Some of these solutions and many others can be implemented with only a couple hours of coding.

Inline styles and scripts for the topmost content

Nobody enjoys waiting for long load times. When opening a Google search link, being met with a blank page or a loading GIF for several seconds can seem agonizing. That’s why optimizing the initial rendering of your page is crucial.

The content that immediately appears to the user without the need to scroll down is referred to as “above-the-fold”. This is where your optimization efforts should be aimed. So here’s a plan to load and display as quickly as possible:

First, differentiate the critical styles and scripts you need to render the topmost content, and separate them from the rest of our stylesheet and external script references.
Then, minify the separated styles and scripts, and insert them directly on our page template, right before the closing </head> tag.
Finally, take the stylesheet and scripts link references from the <head> tag (where it’s usually located) and move them to the end of the above-the-fold content.

Now, the user won’t have to wait until all references are loaded before seeing content. Tip: Remember to use the async tag on scripts whenever possible.

example.html:

<head>
    <style>{above-the-fold minified inline styles goes here}</style>
    <script type="text/javascript">{above-the-fold critical scripts goes here}</script>
</head>
<body>
    <div class="above-the-fold-content"></div>
    <link rel="stylesheet" href="{below-the-fold minified stylesheet reference goes here}" />
    <script async src="{below-the-fold minified javascript reference goes here}" />
    <div class="below-the-fold-content"></div>
</body>

Deferred loading of ads

If you’re monetizing your website through Google AdSense or another ad agency that uses scripts to load ads, consider loading ads after the content is fully rendered. This may have a small impact on your revenue, but will improve the user’s experience while optimizing the load speed.

Although there are several ways to achieve this, a technique I have successfully used on many websites is removing all of the script references to Google AdSense until your page is fully loaded. A short delay can be added in order to allow some browsing time before showing ads.

Remove script references, the comment, and extra spaces from your original ad code, to convert it from something like this…

<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
<!-- Your ad name -->
<ins class="adsbygoogle"
     style="display:inline-block;width:728px;height:90px"
     data-ad-client="ca-pub-XXXXXXXXXXXXXXXXX"
     data-ad-slot="XXXXXXXXX"></ins>
<script>
     (adsbygoogle = window.adsbygoogle || []).push({});
</script>

… to something like this:

<ins class="adsbygoogle" style="display:inline-block;width:728px;height:90px" data-ad-client="ca-pub-XXXXXXXXXXXXXXXXX" data-ad-slot="XXXXXXXXX"></ins>

A lot shorter, isn’t it? This will create an empty slot in which the ad will be displayed after the page is fully rendered. To accomplish that, a new script like the one below must be added (assuming jQuery is present on the website):

async-ads.js:

// Create a script reference
function addScript(src, async, callback) {
    var js = document.createElement("script");
    js.type = "text/javascript";
    if (async)
        js.async = true;
    if (callback)
        js.onload = callback;
    js.src = src;
    document.body.appendChild(js);
}

// Called when document is ready
$(document).ready(function() {

    // Wait for one second to ensure the user started browsing
    setTimeout(function() {
        (adsbygoogle = window.adsbygoogle || []);
        $("ins.adsbygoogle").each(function() {
            $("<script>(adsbygoogle = window.adsbygoogle || []).push({})</script>").insertAfter($(this));
        });
        addScript("https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js", true);
    }, 1000);

});

This code will wait for one second once the document is ready, and then leave instructions for Google to push a new ad for each slot. Finally, the AdSense external script will be loaded so that Google will read the instructions and start filling all the slots with ads.

Tip: Enabling balancing from your AdSense dashboard may improve the average load speed as well as the user’s experience since ads will not be shown when the expected revenue is deprecable. And if you’re still on the fence about showing fewer ads, try out an experiment like I did. A balance of 50% worked well in my case, but the right balance will depend on your niche and website characteristics.

Lazy load for images

Because the user will most likely spend the majority of the visit reading above-the-fold content (and may even leave before scrolling at all), loading all images from content below-the-fold at first is impractical. Implementing a custom lazy-loading script (also referred to as deferred-loading or loading-on-scroll) for images can be an easy process. Even though changes to the backend would be likely, the concept of this approach is simple:

Replacing the src attributes from all images that will have lazy loading with a custom attribute such as data-src (this part will probably require backend changes) and set a custom class for them, like lazy.
Creating a script that will copy the data-src content into the src attribute as we scroll through the page.

lazy-load.js:

;(function($) {

    $.fn.lazy = function(threshold, callback) {
        var $w = $(window),
        th = threshold || 0,
        attrib = "data-src",
        images = this,
        loaded;
        this.one("lazy", function() {
            var source = this.getAttribute(attrib);
            source = source || this.getAttribute("data-src");
            if (source) {
                this.setAttribute("src", source);
                if (typeof callback === "function") callback.call(this);
            }
        });

        function lazy() {
            var inview = images.filter(function() {
                var $e = $(this);
                if ($e.is(":hidden")) return;
                var wt = $w.scrollTop(),
                wb = wt + $w.height(),
                et = $e.offset().top,
                eb = et + $e.height();
                return eb >= wt - th && et <= wb + th;
            });
            loaded = inview.trigger("lazy");
            images = images.not(loaded);
        }

        $w.scroll(lazy);
        $w.resize(lazy);
        lazy();
        return this;
    };

})(window.jQuery);

$(document).ready(function() {
    $('.lazy').each(function () {
        $(this).lazy(0, function() {
        $(this).load(function() {
            this.style.opacity = 1;
        });
    });
});

// Set the correct attribute when printing
var beforePrint = function() {
    $("img.lazy").each(function() {
        $(this).trigger("lazy");
        this.style.opacity = 1;
    });
};
if (window.matchMedia) {
    var mediaQueryList = window.matchMedia('print');
    mediaQueryList.addListener(function(mql) {
        if (mql.matches)
            beforePrint();
    });
}
window.onbeforeprint = beforePrint;

This script will search for all <img> tags with class lazy, and change the data-src attribute to the src attribute once the image becomes visible due to scrolling. It also includes some additional logic to set the src attribute before printing the page.

Server-side caching

Instead of performing all the backend rendering calculations every time, server-side caching allows you to output the same content to the clients over a period of time from a temporary copy of the response. This not only results in a decreased response time but also saves some resources on the server.

There are several ways to enable server-side caching, depending on factors such as the backend language and hosting platform (e.g. Windows/IIS vs. Linux/Apache), among other things. For this example, we will use ASP.NET (C#) since I’m mostly a Windows user.

The best and most efficient way to do this is by adding a declaration in the top of our ASP.NET page:

<%@ OutputCache Duration="10" VaryByParam="id;date" %>

This declaration is telling the compiler that we want to cache the output from the server for 10 minutes, and we will save different versions based on the id and date URL parameters. So pages like:

https://www.your-url.com/cached-page/?id=1&date=2020-01-01
https://www.your-url.com/cached-page/?id=2&date=2020-01-01
https://www.your-url.com/cached-page/?id=2&date=2020-02-01

will be saved and then served from different cache copies. If we only set the id parameter as a source for caching, pages with different dates will be served from the same cache source (this can be useful as the date parameter is only evaluated on frontend scripts and ignored in the backend).

There are other configurations in ASP.NET to set our output cache policy. The output can be set to be based on the browser, the request headers, or even custom strings. This page has more useful information on this subject.

GZip compression

GZip compression—when the client supports it—allows compressing the response before sending it over the network. In this way, more than 70% of the bandwidth can be saved when loading the website. Enabling GZip compression for dynamic and static content on a Windows Server with IIS is simple: Just go to the “Compression” section on the IIS Manager and check the options “Enable dynamic/static content compression”.

However, if you are running an ASP.NET MVC/WebForms website, this won’t be enough. For all backend responses to be compressed before sending them to the client, some custom code will also need to be added to the global.asax file in the website root:

global.asax:

<%@ Application Language="C#" %>

<script runat="server">

    void Application_PreRequestHandlerExecute(object sender, EventArgs e)
    {
        HttpApplication app = sender as HttpApplication;
        string acceptEncoding = app.Request.Headers["Accept-Encoding"];
        System.IO.Stream prevUncompressedStream = app.Response.Filter;

        if (app.Context.CurrentHandler == null)
            return;

        if (!(app.Context.CurrentHandler is System.Web.UI.Page ||
            app.Context.CurrentHandler.GetType().Name == "SyncSessionlessHandler") ||
            app.Request["HTTP_X_MICROSOFTAJAX"] != null)
            return;

        if (acceptEncoding == null || acceptEncoding.Length == 0)
            return;

        if (Request.ServerVariables["SCRIPT_NAME"].ToLower().Contains(".axd")) return;
        if (Request.ServerVariables["SCRIPT_NAME"].ToLower().Contains(".js")) return;
        if (Request.QueryString.ToString().Contains("_TSM_HiddenField_")) return;

        acceptEncoding = acceptEncoding.ToLower();

        if (acceptEncoding.Contains("deflate") || acceptEncoding == "*")
        {
            app.Response.Filter = new System.IO.Compression.DeflateStream(prevUncompressedStream,
                System.IO.Compression.CompressionMode.Compress);
            app.Response.AppendHeader("Content-Encoding", "deflate");
        }
        else if (acceptEncoding.Contains("gzip"))
        {
            app.Response.Filter = new System.IO.Compression.GZipStream(prevUncompressedStream,
                System.IO.Compression.CompressionMode.Compress);
            app.Response.AppendHeader("Content-Encoding", "gzip");
        }
    }

</script>

To make sure our code is working properly, an external tool like this will inform you if GZip is enabled or not.

Summary

While there are many ways of decreasing the load time of a website, most are common and expensive. However, with a few minor tweaks, we can offer a better user experience in addition to improve our position in the search engine results. Every bit of optimization counts towards the goal with SEO. Load time is a very important factor (to both the developer and the user), especially on mobile platforms where users expect to get what they want instantly.

The image below is a Google Analytics report from one of my websites where, over several months, I implemented most of these formulas. A month ago, I made the latest change of deferring ad loading, which had an observable impact on the average loading speed of the page:

Do you have any other page load optimization techniques? Leave a comment below!

What’s the deal with ASP.NET Core Razor Pages?

2018-11-20T00:00:00+00:00

During the last couple of years I’ve been doing lots of web development with technologies like JavaScript and PHP and Zend Framework, with a strong focus on the front end. Before that, however, the vast majority of the work I did as a web developer was with the .NET Framework.

I love .NET. Particularly the C# language. However, the greatest thing about .NET is the vibrant ecosystem full of tools (the Visual Studio IDE, for example, is outstanding), libraries and frameworks that make one’s life easier and more productive. It has, however, a crucial weakness that prevents it from reaching even greater heights: it’s locked to Windows systems. So, whenever the need comes to develop something outside of a Windows environment, you’re out of luck.

So, naturally, when Microsoft announced their initiative to make the .NET Framework open source and bring it to other platforms a few years ago, I was really excited. Fast forward to today and we have .NET Core as the product of great effort from both Microsoft and the community.

I’ve definitely kept it on my radar since its inception back in 2016, but haven’t had the chance to take a really deep dive and see what the technology is about and get a feel for the ecosystem surrounding it.

Well, that time has come and just last week I decided to go in and see what’s up and how everything works. Suffice it to say, I am impressed. The promise of being able to write .NET code everywhere has been pretty much fulfilled.

In terms of features, .NET Core is still missing some stuff but the feature set is mature enough that it can handle most use cases. Me being first and foremost a web developer though, what has me most excited is ASP.NET Core, .NET Core’s own web development framework inspired by classic ASP.NET.

ASP.NET Core is not just a port of ASP.NET, it is a complete rewrite with a new architecture, new features and ways to develop apps.

At the core of ASP.NET Core (no pun intended) there’s the notion that an HTTP request gets processed by a pipeline composed of several components named middleware, which also produce a response. When the server receives a request, each of the middleware components in the pipeline take their turn in processing the request, are responsible for calling the next middleware in the pipeline and, finally, generating a response to send back to the client.

ASP.NET Core is not the only modern web framework with this pipeline based request processing design. Zend Expressive, for example, also uses this architecture.

Overall, I think this approach is a good solution to the problem of request processing which results in a good mental model for developers to have about how the application is working under the hood.

All of this is more concerned with application startup and initial configuration though. When it comes to actually writing business logic, ASP.NET Core offers two alternatives for web apps with GUIs: MVC and, most recently, Razor Pages.

From the point of view of the developer’s experience, MVC is very similar to classic ASP.NET MVC: there are Controllers which contain Action Methods which return Views which are processed by a templating engine (i.e. Razor) to generate a GUI that a user can see and interact with. Razor Pages though, is a new approach that is exclusive to ASP.NET Core and makes things slightly different. The developers at Microsoft say that this new approach is even preferred to MVC for building web UIs. That is, non-API web applications.

With Razor Pages, Controllers, Actions and Views are gone and in their place we’ve got Pages and so-called Page Models. So, while in MVC we have the router match a URL to a Controller and Action combo that handles the request and returns a processed View to the client; in Razor Pages we have the router match a URL to a Page and an optional Page Model file (via their location in the project directory) that handle the request and finally present some GUI to the client. Another way to look at it is that, while MVC focuses on Controllers and Actions to handle a request, Razor Pages focuses on actual files.

To better illustrate their differences, here’s what the directory structure of each looks like:

As you can see, the main difference is that the Controllers, Models and Views folders are gone and instead we have a Pages folder. The Pages folder contains all of the .cshtml files (which are the Views or templates) together with accompanying .cshtml.cs files (which are the presentation logic behind the views, or Page Models). These Page Model files contain the logic to respond to requests as well as any properties containing values used for presentation in the template. These properties are a substitute for having separate View Model DTO objects that the Actions pass to the Views.

Looking at this directory structure, it’s very easy to see that this model, on the surface at least, looks an awful lot like a Page/Code Behind model that hasn’t been used since the first iteration of ASP.NET: Web Forms. This, together with the fact that MVC’s clean cut separation of Controllers and Views seems to have been merged into some SRP-violating Page Model file, has raised some red flags in the eyes of some people in the community.

Perhaps unsurprisingly, Razor Pages has sparked quite the controversy. There’s a big discussion over on the project’s GitHub page, as I’m sure there is elsewhere in places like StackOverflow.

And, I gotta be honest, I myself, being very familiar with Web Forms and its pitfalls, had a similar initial gut reaction as well. And I think everything is also compounded by the fact that now, seemingly out of nowhere, the ASP.NET Core team are touting this new, inferior (to the eyes of some) method as the preferred method for building new applications moving forward. So, people fear that their preferred style will stop being supported and go away for good.

So, what is the truth? Well, I don’t think it’s as bad as some people are making it out to be. Of course, the technology is fairly new and still needs to be battle tested in large scale real world projects to make sure it’s got what it takes. But as of now, I don’t think the design is fundamentally flawed. In fact, I think it offers a competent way to develop web apps, and I can definitely see why the ASP.NET Core team sees this as the preferred method over MVC.

I’m going to offer some arguments as to why I see Razor Pages as a perfectly fine technology choice. To do that, I’ll address some of the critiques that I’ve seen have been levied against it.

1. It’s a worse design than MVC

Some people are saying that MVC is just better on the grounds that it has better Separation of Concerns. This may look like fact at first because of how the files are structured. Like we saw before, in MVC, presentation logic is distributed among three types of files: Controllers (which contain Action Methods), Views and, generally View Models. In Razor Pages we only have a template and an accompanying file which merges Controller, Action and View Model all together. However, we have to ask ourselves: in MVC, are Controllers, Actions and View Models truly separated? And should they even be?

In the context of MVC, View Models are nothing more than DTOs that serve as containers for data that the Actions in the Controller send to the View for rendering. As such, these are most of the time tightly and statically coupled with the View that uses them and the Action that generates them. Rarely you see a View Model being reused across multiple Actions and Views, or injected as an abstract dependency to the Controller.

So, this means that, with a design like the one Razor Pages uses, we’re not actually losing much in the way of Separation of Concerns. Actually, we gain something in the convenience of having all related presentation logic consolidated and close together instead of dispersed across the directory tree.

Finally, I think one frame of mind that helps further see the value of what Razor Pages offers is thinking of the Page Model as a View Model. And I mean View Model in the sense of MVVM as opposed to its MVC counterpart. As we’ve already discussed, in MVC, a View Model is nothing more than a DTO that serves to carry data from the Controller to the View. In MVVM, a View Model is much more, as it serves not only as a data store but also as a mechanism to respond to user interactions and execution of presentation logic. This is the pattern used by many modern front end development frameworks like React, Angular and Vue.js. The only difference is that, while the MVVM JavaScript frameworks bridge the gap between the DOM and our app’s code; Razor Pages is bridging the gap between an HTTP client and a server. As a consequence, the vocabulary of the messages is completely different. In the front end, we have the View Model responding to click, and hover and mouseEnter events that come from the DOM. In Razor Pages, we have our “View Model” (i.e. the Page Model) responding to events that come from the client via HTTP; so we get things like GET or POST or PUT, with the associated request payloads.

2. It’s just Web Forms all over again

This one is simply not true as both technologies are very different. It is true that, from a file organization point of view, you can definitely see some similarities. After all, both in Razor Pages and Web Forms we have the focus on pages with some code behind them. However, the way that they go about implementing actual websites is completely different.

Just to be clear, I don’t subscribe to the idea that Web Forms is a bad technology or that it was badly designed. For its time, Web Forms was a huge step forward in terms of rapid application development. Also, its model tried to abstract away all the nuance of HTTP and offer a development experience that was very similar to what was seen before in Windows Forms. Now, most modern web development frameworks embrace HTTP’s idiosyncrasies rather than trying to take them out of the way of developers. However, at the time it was an interesting proposition that helped to bridge the gap that developers had to cross in order to move from building desktop apps on Windows to building web apps on a Windows based server.

Anyway, Razor Pages is very different from ASP.NET Web Forms. First of all, in Razor Pages, there’s no abstraction of HTTP whatsoever. Actually, what you see in those Page Model files is not an HTML page’s “Code Behind“, what you see are HTTP-Verb-specific Action Methods of sorts that the framework calls for handling requests. You’re going to see methods like OnGet and OnPost, instead of Button3_OnClick or DropDownList1_OnSelect. Much like MVC, you also have access to the complete Request, Response and HttpContext objects that help out in processing requests. Also, the main pain points in WebFroms: Postbacks and the ViewState, are nowhere to be found. Instead, everything is stateless and you’re 100% in control of the messaging that’s happening between server and client.

Here’s what an empty Page Model file looks like:

In reality, these Page Model files look quite a bit like traditional MVC Controllers in that they contain, essentially, Action Methods. The difference is that, in MVC, we had many Action Methods in one Controller where each Action Method corresponded to a specific route. Through routing, each Action Method was executed depending on the incoming request URL. Then the Action Method was responsible of handling the request depending on the HTTP Verb (i.e. GET, POST, etc) and other incoming data. Finally it would produce a useful response.

Meanwhile, in Razor Pages we have these “Action Methods” of sorts that are geared towards handling the various HTTP Verbs for requests that come to the front facing page that they are associated to. This means that a given Page Model and Page combo will always only serve a particular, specific route (i.e. URL). Where a traditional MVC Controller would potentially server multiple routes, depending on how many Action Methods were defined within it.

So, while it is true that, with Razor Pages, we “went back” to having pairs of files representing one web app functional unit, like we did in ASP.NET Web Forms; the way they go about it and the developer experience that they offer is completely different. Also, Razor Pages is not that different than MVC after all. All the building blocks are still there, they are just slightly rearranged.

3. It’s going to replace MVC completely

I understand the fear that, when the new thing comes out, the old thing is going to be left to the wayside. This is the same feeling that many had (me included!) when ASP.NET MVC for the OG .NET Framework first came out way back in 2009. Web Forms was doomed. Well, guess what? Web Forms is still around. While it’s true that the last major update happened years ago, the technology is still a first class citizen in the .NET ecosystem and perfectly usable. And I attribute the lack of recent big updates with new features to the fact that it is a mature framework that is “pretty complete”, rather than Microsoft pulling the plug. After all, it’s still supported in the newest version of Visual Studio and the .NET Framework and new features and improvements do get developed, albeit small.

Here’s what Visual Studio shows when asked to create a new web application project:

Yup, still alive and kicking.

So yeah, I feel like this is pretty natural human behavior but it’s ultimately baseless, given the history of this ecosystem. Much like Web Forms is still alive after years and years, I think MVC in ASP.NET Core is going to be just fine.

And that concludes my long winded opinion piece about the whole MVC vs Razor Pages controversy and my recent discovery of how cool the .NET Core framework is. As far as I see it, the technology is ready for prime time and I’m looking forward to using it in my projects.

(Reposted from Kevin’s blog.)

Using esbuild and pnpm to Set Up Frontend Asset Bundling in an ASP.NET Core Web App

Installing pnpm and the necessary packages

Authoring JavaScript and SCSS source files

Bundling frontend assets with esbuild

Importing libraries as modules

Importing libraries via HTML tags

Summary

Using Razor templates to render HTML emails in ASP.NET Core

The plan

Step 1: Sending emails with the MailKit NuGet package

Step 2: Rendering Razor templates into strings

Step 3: Defining the email templates

Step 4: Putting it all together: the class that sends the email

Processing payments with Authorize.Net in a Blazor app

The backend

Interacting with Authorize.Net

Defining the API endpoint

The frontend

Payment page initialization

Payment submission

Reducing page load times: Cloudflare and caching with ASP.NET

Adding response caching

Using Cloudflare to distribute cached content

Testing

Using a Containerized Nginx Proxy to Serve a Multi-Application .NET System

Reintroducing the demo project

Adding the proxy service in compose.yaml

Writing the proxy Dockerfile

Configuring the proxy

Configuring the PathBase in ASP.NET Core apps

Deploying

Using Docker Compose to Deploy a Multi-Application .NET System

Getting familiar with the demo project

Including the code base repository as a Git submodule

Deploying the database

Deploying the admin portal web app

Adding a maintenance container

Deploying the web API

Looking at the logs

Persisting data protection keys

Storing sensitive information with secrets

Parameterizing compose.yaml to support multiple deployment environments

Waiting for the database to be ready

Serving the apps with NGINX

That’s all for now

Uploading multiple files in a single request in an ASP.NET Core application

Where to store the images

A Razor Page for uploading many files

Uploading any number of files

Validating the uploaded images

Displaying the images

Deleting the images

How to validate record uniqueness in ASP.NET

Using a database unique index to enforce uniqueness

Writing a custom validation attribute to enforce uniqueness

A generic custom validation attribute to enforce uniqueness

Writing a unit test for the custom validation attribute

Writing integration tests for an ASP.NET Web API

Introducing the project

Setting up the integration tests project

Writing a database fixture

Writing some integration tests

Writing some more integration tests

A test that writes to and reads from the database

A test that makes a POST request

A test that makes many requests

Disabling some application services

I wrote the same app twice, with Hotwire and Blazor Server — here’s what I learned

What this article is

An overview of Blazor

How Blazor works under the hood

An abstraction over the request/response cycle

How Blazor supports common frontend framework features

Handling DOM events

Including other Blazor components

Passing parameters to components

Defining, handling and triggering custom component events

Handling component lifecycle events

Templates

Routing

Adding the proxy service in `compose.yaml`

Writing the proxy `Dockerfile`

Parameterizing `compose.yaml` to support multiple deployment environments