https://www.endpointdev.com/blog/tags/dotnet/2024-01-13T00:00:00+00:00End Point DevBuilding a real-time application with SignalR, .NET, and three.jshttps://www.endpointdev.com/blog/2024/01/building-real-time-app-signalr-dotnet-threejs/2024-01-13T00:00:00+00:00Bimal Gharti Magar
<p><img src="/blog/2024/01/building-real-time-app-signalr-dotnet-threejs/clock-banner.webp" alt="A white watchface, without any other parts, sits on a surface with a dotted texture"><br>
Image by Pixabay on Pexels</p>
<p>When data in a web application is changing in real time, users want to see those updates reflected in real time without refreshing the application. Adding real-time functionality to a .NET application is easy with the <a href="https://dotnet.microsoft.com/en-us/apps/aspnet/signalr">SignalR</a> library.</p>
<p>Depending on the capabilities of the client and server, SignalR can use WebSockets, server-sent events, or long polling to establish a persistent connection between the client and the server. From the server side, SignalR pushes code to connected clients. SignalR is good for applications that require high-frequency updates from the server, such as real-time gaming.</p>
<p>C# code can be used to write SignalR hubs, which easily integrate with other ASP.NET features like dependency injection, authentication, authorization, and scalability.</p>
<h3 id="what-well-build">What we’ll build</h3>
<p>We will learn how to use SignalR for real-time communication to send the camera position of a cube, built using <a href="https://threejs.org/">three.js</a>, to all users on the page.</p>
<p><img src="/blog/2024/01/building-real-time-app-signalr-dotnet-threejs/signal-real-time-cube.gif" alt="Two browser windows are open side by side, to the same web app. On both, under a “Receiver” title is a window showing a 3D cube, after which a “Controller” title is followed by a “Request Control” button. In the left browser window, the mouse clicks the blue “Request Control” button, which turns green and displays a message underneath saying “Control granted: 120 seconds remaining”, with the number counting down every second. Another identical 3D cube window opens beneath. The clicks and drags in this bottom window, which rotates the cube. After a short delay, the other two cubes in the “Receiver” section follow and move to the position where the controller moved the cube."></p>
<p>You can see the <a href="https://nifty.azurewebsites.net/">demo here</a>.</p>
<p>Here is the description of our testing app:</p>
<ul>
<li>A user can request control of a cube for 2 minutes</li>
<li>Control can be released manually, or it will be automatically released after 2 minutes</li>
<li>A user can control the camera position of cube, which will be seen by all users viewing the page</li>
<li>When one user is controlling the cube, other users requesting control will be added to a queue</li>
<li>Queued users will be granted control automatically when the controlling user’s time is over</li>
</ul>
<p>First we will create a web application and create a SignalR communication between the client and server. After that, we can implement the above features.</p>
<h3 id="creating-the-web-application">Creating the web application</h3>
<p>Create a new .NET application.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">> dotnet new webapp -o EPSignalRControl
</code></pre></div><p>I edited the app using VS Code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">> code .\EPSignalRControl\
</code></pre></div><h4 id="installing-and-configuring-signalr">Installing and configuring SignalR</h4>
<p>The SignalR server library is included in the ASP.NET Core shared framework. However, the JavaScript client library isn’t automatically included in the project. You can use Library Manager (LibMan) to get the client library from unpkg. unpkg is a fast global content delivery network for everything on npm.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">> dotnet tool install -g Microsoft.Web.LibraryManager.Cli
> libman install @microsoft/signalr@latest -p unpkg -d wwwroot/js/signalr --files dist/browser/signalr.js
</code></pre></div><h4 id="creating-a-signalr-hub">Creating a SignalR hub</h4>
<p>We’ll create a <code>CameraData</code> model class for passing data between the SignalR server and client.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">CameraData</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">double</span> x { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#00d;font-weight:bold">0</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">double</span> y { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#00d;font-weight:bold">0</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">double</span> z { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#00d;font-weight:bold">5</span>;
}
</code></pre></div><p>We’ll also create a <code>ControlHub</code> class which inherits from <code>Hub</code>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.SignalR</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">EPSignalRControl.Hubs</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ControlHub</span> : Hub
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task SendData(CameraData data)
{
<span style="color:#888">// Broadcast the data to all clients
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">await</span> Clients.All.SendAsync(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, data);
}
}
</code></pre></div><p>The <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub?view=aspnetcore-8.0">Hub</a> class has Clients, Context, and Groups properties:</p>
<ul>
<li><strong><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub.clients?view=aspnetcore-8.0">Clients</a></strong> can be used to invoke methods on the clients connected to this hub.</li>
<li><strong><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub.context?view=aspnetcore-8.0">Context</a></strong> is the hub caller context for accessing information about the hub caller connection.</li>
<li><strong><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub.groups?view=aspnetcore-8.0">Groups</a></strong> is the group manager to manage connections in groups.</li>
</ul>
<blockquote>
<p>One thing to note is that hubs are transient, so we cannot store state in a property of the Hub class. Each hub method call is executed on a new Hub instance.</p>
</blockquote>
<p><code>Clients.All</code> calls a method on all connected clients. As a result, the control hub sends the data received from one client back to all connected clients. Similarly, we can use <code>Clients.Caller</code> to send back data to the client that invoked the hub method.</p>
<p>Then, we need to register the services using <code>AddSignalR()</code> and configure the endpoints using <code>MapHub()</code> required by SignalR in <code>Program.cs</code></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">EPSignalRControl.Hubs</span>;
<span style="color:#888;font-weight:bold">var</span> builder = WebApplication.CreateBuilder(args);
builder.Services.AddRazorPages(); <span style="color:#888">// Add services to the container
</span><span style="color:#888"></span>builder.Services.AddSignalR(); <span style="color:#888">// Register SignalR service
</span><span style="color:#888"></span>
<span style="color:#888;font-weight:bold">var</span> app = builder.Build();
<span style="color:#080;font-weight:bold">if</span> (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler(<span style="color:#d20;background-color:#fff0f0">"/Error"</span>);
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();
app.MapRazorPages();
app.MapHub<ControlHub>(<span style="color:#d20;background-color:#fff0f0">"/controlHub"</span>); <span style="color:#888">// Map ControlHub to the '/controlHub' endpoint
</span><span style="color:#888"></span>
app.Run();
</code></pre></div><h4 id="connecting-to-signalr-hub-using-the-signalr-javascript-client-library">Connecting to SignalR Hub using the SignalR JavaScript client library</h4>
<p>We’ll reference the SignalR JavaScript library as well as create a <code>site.js</code> file and reference it in <code>Index.cshtml</code>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/signalr/dist/browser/signalr.js"</span>></<span style="color:#b06;font-weight:bold">script</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">type</span>=<span style="color:#d20;background-color:#fff0f0">"module"</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/site.js"</span> <span style="color:#369">asp-append-version</span>=<span style="color:#d20;background-color:#fff0f0">"true"</span>></<span style="color:#b06;font-weight:bold">script</span>>
</code></pre></div><p>In the <code>site.js</code> file, we will:</p>
<ul>
<li>Connect to the control hub using the endpoint <code>/connecthub</code></li>
<li>Start the connection with our control hub</li>
<li>Send data to the server by invoking the <code>SendData</code> method of ControlHub</li>
<li>Add a listener for <code>ReceiveData</code> to receive the data sent from SignalR Hub</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">const</span> controlHubConnection = <span style="color:#080;font-weight:bold">new</span> signalR.HubConnectionBuilder()
.withUrl(<span style="color:#d20;background-color:#fff0f0">"/controlhub"</span>)
.build();
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, <span style="color:#080;font-weight:bold">function</span> (cameraData) {
console.log(<span style="color:#d20;background-color:#fff0f0">`position.x = </span><span style="color:#33b;background-color:#fff0f0">${</span>cameraData.x<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
console.log(<span style="color:#d20;background-color:#fff0f0">`position.y = </span><span style="color:#33b;background-color:#fff0f0">${</span>cameraData.y<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
console.log(<span style="color:#d20;background-color:#fff0f0">`position.z = </span><span style="color:#33b;background-color:#fff0f0">${</span>cameraData.z<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
});
controlHubConnection.start()
.then(<span style="color:#080;font-weight:bold">function</span> () {
console.log(<span style="color:#d20;background-color:#fff0f0">"Connected to ControlHub"</span>);
console.log(<span style="color:#d20;background-color:#fff0f0">"Invoking SendData"</span>);
controlHubConnection.invoke(<span style="color:#d20;background-color:#fff0f0">"SendData"</span>, {
x: <span style="color:#00d;font-weight:bold">100</span>,
y: <span style="color:#00d;font-weight:bold">100</span>,
z: <span style="color:#00d;font-weight:bold">100</span>
})
.<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
<span style="color:#080;font-weight:bold">return</span> console.error(err.toString());
});
}).<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
console.error(<span style="color:#d20;background-color:#fff0f0">"Error connecting to ControlHub: "</span>, err);
});
</code></pre></div><p>At this point if we run the web application, open the browser, and look at the developer tools console, we will see:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Connected to ControlHub
Invoking SendData
position.x = 100
position.y = 100
position.z = 100
</code></pre></div><p>If we open the web application in a different tab, then in the first tab’s dev tools console we will see the position values are appended. This is because the new tab invoked <code>SendData</code> and the server received and sent back data to all connected clients.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Connected to ControlHub
Invoking SendData
position.x = 100
position.y = 100
position.z = 100
position.x = 100
position.y = 100
position.z = 100
</code></pre></div><p>This is the basic way of implementing SignalR in the project to achieve real-time functionality. Now, we’ll dive deep into creating more features to extend the app to more practical usage.</p>
<h3 id="adding-features">Adding features</h3>
<h4 id="backend">Backend</h4>
<p>Add a <code>ControlRequest.cs</code> model to hold the SignalR connection ID and the time of the request.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ControlRequest</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> ConnectionId { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#888;font-weight:bold">string</span>.Empty;
<span style="color:#080;font-weight:bold">public</span> DateTime RequestTime { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
</code></pre></div><p>Add two variables in <code>ControlHub.cs</code>: <code>currentControl</code> to hold the details of the current active request, and <code>controlQueue</code> to hold the requests in queue.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">static</span> ControlRequest? currentControl;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">static</span> Queue<ControlRequest> controlQueue = <span style="color:#080;font-weight:bold">new</span> Queue<ControlRequest>();
</code></pre></div><p>Add a <code>ControlTimer.cs</code> class with properties for a ConcurrentDictionary mapping of the connection ID, the ControlHub clients, the ControlHub context, and the ControlTimer object.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Collections.Concurrent</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.SignalR</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ControlTimer</span> : System.Timers.Timer
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> ConcurrentDictionary<<span style="color:#888;font-weight:bold">string</span>, ControlTimer> ControlTimers = <span style="color:#080;font-weight:bold">new</span>();
<span style="color:#080;font-weight:bold">public</span> HubCallerContext hubContext { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> IHubCallerClients hubCallerClients { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ControlTimer(<span style="color:#888;font-weight:bold">double</span> interval) : <span style="color:#080;font-weight:bold">base</span>(interval) { }
}
</code></pre></div><p>Add two more variables in <code>ControlHub.cs</code>:</p>
<ul>
<li><code>controlTimer</code> to store control timer dictionary, control hub context, and control hub clients that can be accessed during the interval of the timer that runs for the active request.</li>
<li><code>CONTROL_TIME</code> holds the time in seconds for which the active request has control.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> ControlTimer? controlTimer;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">const</span> <span style="color:#888;font-weight:bold">int</span> CONTROL_TIME = <span style="color:#00d;font-weight:bold">120</span>; <span style="color:#888">//seconds
</span></code></pre></div><p>Add a <code>RequestControl</code> method to <code>ControlHub</code> that is invoked by the client to request the control. The method adds the connection ID of the invoked request to the queue, and creates and adds the control timer for the currently invoked request. If the queue has only one request, it gives control to the current request, starts the timer and sends a message to the requesting client notifying that access is granted. If the queue has more than one request, the requesting client is notified that their request is queued.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task RequestControl()
{
<span style="color:#888;font-weight:bold">var</span> controlRequest = <span style="color:#080;font-weight:bold">new</span> ControlRequest
{
ConnectionId = Context.ConnectionId,
RequestTime = DateTime.Now
};
<span style="color:#888">// Add the control request to the queue
</span><span style="color:#888"></span> controlQueue.Enqueue(controlRequest);
<span style="color:#888">// Add timer to dictionary for request
</span><span style="color:#888"></span> controlTimer = <span style="color:#080;font-weight:bold">new</span> ControlTimer(<span style="color:#00d;font-weight:bold">500</span>)
{
hubContext = Context,
hubCallerClients = Clients
};
controlTimer = ControlTimer.ControlTimers.GetOrAdd(controlRequest.ConnectionId, controlTimer);
<span style="color:#888">// Grant control if the queue is empty or it's the first request
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (controlQueue.Count == <span style="color:#00d;font-weight:bold">1</span> || controlQueue.Peek() == controlRequest)
{
currentControl = controlRequest;
SetupTimerForRelease(controlRequest);
<span style="color:#080;font-weight:bold">await</span> Clients.Client(controlRequest.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlGranted"</span>);
}
<span style="color:#080;font-weight:bold">else</span>
{
<span style="color:#080;font-weight:bold">await</span> Clients.Client(controlRequest.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlQueued"</span>);
}
}
</code></pre></div><p>Add a <code>SetupTimerForRelease</code> method to add calls to the <code>ReleaseControlMiddleware</code> method on a regular interval from the running timer.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> SetupTimerForRelease(ControlRequest controlRequest)
{
<span style="color:#080;font-weight:bold">if</span> (controlRequest != <span style="color:#080;font-weight:bold">null</span> && controlTimer == <span style="color:#080;font-weight:bold">null</span>)
{
controlTimer = ControlTimer.ControlTimers.GetOrAdd(controlRequest.ConnectionId, controlTimer);
}
controlTimer.Elapsed += <span style="color:#080;font-weight:bold">new</span> ElapsedEventHandler(ReleaseControlMiddleware);
controlTimer.Enabled = <span style="color:#080;font-weight:bold">true</span>;
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> ReleaseControlMiddleware(<span style="color:#888;font-weight:bold">object</span> source, ElapsedEventArgs e)
{
<span style="color:#00d;font-weight:bold">_</span> = AutoReleaseControl(source, e);
}
</code></pre></div><p>Add an <code>AutoReleaseControl</code> method, to be called by a running timer, which checks if the time for the current control has elapsed. The method sends the time remaining message to the user with control. If the time has elapsed, it calls <code>ClearTimerAndControl</code> to clear the timer and release control.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task AutoReleaseControl(<span style="color:#888;font-weight:bold">object</span> source, ElapsedEventArgs e)
{
<span style="color:#888;font-weight:bold">var</span> controlTimer = (ControlTimer)source;
HubCallerContext hcallerContext = controlTimer.hubContext;
IHubCallerClients hubClients = controlTimer.hubCallerClients;
<span style="color:#080;font-weight:bold">if</span> (currentControl != <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#888;font-weight:bold">var</span> elapsedSeconds = Math.Ceiling(DateTime.Now.Subtract(currentControl.RequestTime).TotalSeconds);
<span style="color:#080;font-weight:bold">await</span> hubClients.Client(currentControl.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlRemaining"</span>, CONTROL_TIME - elapsedSeconds);
<span style="color:#080;font-weight:bold">if</span> (elapsedSeconds >= CONTROL_TIME)
{
<span style="color:#080;font-weight:bold">await</span> ClearTimerAndControl(hubClients, hcallerContext);
}
}
}
</code></pre></div><p>Add <code>ClearTimerAndControl</code> method which:</p>
<ul>
<li>Clears the timer for a given connection ID</li>
<li>Sends a message to the controlling client informing that their control time is released</li>
<li>Sends default camera position data to all the clients to reset the cube camera position</li>
<li>Dequeues the current control from the queue, which is the first item</li>
<li>Gives control to the next request in the queue</li>
<li>Sends a message to the client who is granted the control</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">async</span> Task ClearTimerAndControl(IHubCallerClients hubClients, HubCallerContext context)
{
<span style="color:#080;font-weight:bold">try</span>
{
<span style="color:#888">// Clear the timer when control is explicitly released
</span><span style="color:#888"></span> ClearControlTimer(context.ConnectionId);
<span style="color:#080;font-weight:bold">await</span> hubClients.Client(context.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlReleased"</span>);
<span style="color:#080;font-weight:bold">await</span> hubClients.All.SendAsync(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, <span style="color:#080;font-weight:bold">new</span> CameraData());
<span style="color:#888">// Release control
</span><span style="color:#888"></span> currentControl = <span style="color:#080;font-weight:bold">null</span>;
<span style="color:#888">// Remove the first request from the queue
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (controlQueue.Count > <span style="color:#00d;font-weight:bold">0</span>)
controlQueue.Dequeue();
<span style="color:#888">// Grant control to the next in the queue
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (controlQueue.Count > <span style="color:#00d;font-weight:bold">0</span>)
{
currentControl = controlQueue.Peek();
currentControl.RequestTime = DateTime.Now;
SetupTimerForRelease(currentControl);
<span style="color:#080;font-weight:bold">await</span> hubClients.Client(currentControl.ConnectionId).SendAsync(<span style="color:#d20;background-color:#fff0f0">"ControlGranted"</span>, currentControl.RequestTime.ToString());
}
}
<span style="color:#080;font-weight:bold">catch</span> (Exception ex)
{
<span style="color:#080;font-weight:bold">await</span> Clients.All.SendAsync(ex.ToString());
}
}
</code></pre></div><p>Add a <code>ClearControlTimer</code> method which gets the control timer of the given connection ID. Removes the <code>ReleaseControlMiddleware</code> from the control timer and disables the control timer.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> ClearControlTimer(<span style="color:#888;font-weight:bold">string</span> connectionId)
{
controlTimer = ControlTimer.ControlTimers.GetOrAdd(connectionId, <span style="color:#080;font-weight:bold">new</span> ControlTimer(<span style="color:#00d;font-weight:bold">500</span>));
<span style="color:#080;font-weight:bold">if</span> (controlTimer != <span style="color:#080;font-weight:bold">null</span>)
{
controlTimer.Elapsed -= <span style="color:#080;font-weight:bold">new</span> ElapsedEventHandler(ReleaseControlMiddleware);
controlTimer.Enabled = <span style="color:#080;font-weight:bold">false</span>;
controlTimer = <span style="color:#080;font-weight:bold">null</span>;
}
}
</code></pre></div><p>Add a <code>ReleaseControl</code> method to <code>ControlHub</code>, that is invoked manually by the controlling client to release the control. It checks if the requesting client has the current control access and calls the method to clear timer.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task ReleaseControl()
{
<span style="color:#080;font-weight:bold">if</span> (currentControl != <span style="color:#080;font-weight:bold">null</span> && currentControl.ConnectionId == Context.ConnectionId)
{
<span style="color:#080;font-weight:bold">await</span> ClearTimerAndControl(Clients, Context);
}
}
</code></pre></div><p>Add a <code>SendData</code> method, which receives the data from the controlling user and then sends the received data to all the clients.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task SendData(CameraData cameraData)
{
controlTimer = ControlTimer.ControlTimers.GetOrAdd(Context.ConnectionId, <span style="color:#080;font-weight:bold">new</span> ControlTimer(<span style="color:#00d;font-weight:bold">500</span>));
<span style="color:#080;font-weight:bold">if</span> (controlTimer != <span style="color:#080;font-weight:bold">null</span> && controlTimer.hubContext != <span style="color:#080;font-weight:bold">null</span> && Context.ConnectionId == controlTimer.hubContext.ConnectionId)
{
<span style="color:#888">// Broadcast the received sensor data to all clients
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">await</span> controlTimer.hubCallerClients.All.SendAsync(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, cameraData);
}
}
</code></pre></div><p>That’s all the code required for the backend. Now let’s add to the frontend.</p>
<h4 id="frontend">Frontend</h4>
<p>In <code>Index.cshtml</code>, we’ll include:</p>
<ul>
<li>A receiver section to show the three.js cube and update the camera position using the received data from SignalR connection.</li>
<li>A section to show the message from backend.</li>
<li>A controller section with button to request the control using SignalR connection and show the three.js cube. The cube can be rotated with the mouse which sends the data to the backend using the SignalR connection. The backend then sends the data back to all the clients to update the cube camera position in receiver section.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html">@page
@model IndexModel
@{
ViewData["Title"] = "Home page";
}
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"wrapper"</span>>
<<span style="color:#b06;font-weight:bold">h3</span>>Receiver</<span style="color:#b06;font-weight:bold">h3</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"receiver-wrapper"</span>></<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">hr</span> />
<<span style="color:#b06;font-weight:bold">h3</span>>Controller</<span style="color:#b06;font-weight:bold">h3</span>>
<<span style="color:#b06;font-weight:bold">button</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"btn btn-primary"</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"requestCtrlBtn"</span> <span style="color:#369">disabled</span>></<span style="color:#b06;font-weight:bold">button</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"alert alert-info d-none"</span> <span style="color:#369">role</span>=<span style="color:#d20;background-color:#fff0f0">"alert"</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"message"</span>></<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">id</span>=<span style="color:#d20;background-color:#fff0f0">"controller-wrapper"</span>></<span style="color:#b06;font-weight:bold">div</span>>
</<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/signalr/dist/browser/signalr.js"</span>></<span style="color:#b06;font-weight:bold">script</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">type</span>=<span style="color:#d20;background-color:#fff0f0">"module"</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"~/js/site.js"</span> <span style="color:#369">asp-append-version</span>=<span style="color:#d20;background-color:#fff0f0">"true"</span>></<span style="color:#b06;font-weight:bold">script</span>>
</code></pre></div><p>Then we can add JavaScript code to implement the app functionality in the template. We’ll add:</p>
<ul>
<li>A click handler to invoke the <code>RequestControl</code> method using the SignalR connection.</li>
<li>Listeners for the <code>ControlGranted</code>, <code>ControlQueued</code>, <code>ControlReleased</code>, and <code>ControlRemaining</code> events to update the UI based on the message included in the corresponding event.</li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js"><span style="color:#080;font-weight:bold">let</span> controlRequested = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">let</span> controlGranted = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">let</span> destroyController = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">const</span> requestCtrlBtn = <span style="color:#038">document</span>.getElementById(<span style="color:#d20;background-color:#fff0f0">'requestCtrlBtn'</span>)
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">true</span>;
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Request Control"</span>;
<span style="color:#080;font-weight:bold">const</span> messageCtrl = <span style="color:#038">document</span>.getElementById(<span style="color:#d20;background-color:#fff0f0">'message'</span>)
requestCtrlBtn.addEventListener(<span style="color:#d20;background-color:#fff0f0">'click'</span>, () => {
<span style="color:#080;font-weight:bold">let</span> action = <span style="color:#d20;background-color:#fff0f0">'RequestControl'</span>;
<span style="color:#080;font-weight:bold">if</span> (controlGranted) {
action = <span style="color:#d20;background-color:#fff0f0">'ReleaseControl'</span>;
}
controlHubConnection.invoke(action).<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
<span style="color:#080;font-weight:bold">return</span> console.error(err.toString());
});
})
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlGranted"</span>, <span style="color:#080;font-weight:bold">function</span> () {
controlGranted = <span style="color:#080;font-weight:bold">true</span>;
controlRequested = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.classList.remove(...[<span style="color:#d20;background-color:#fff0f0">'btn-primary'</span>, <span style="color:#d20;background-color:#fff0f0">'btn-warning'</span>]);
requestCtrlBtn.classList.add(<span style="color:#d20;background-color:#fff0f0">'btn-success'</span>);
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Release Control"</span>;
destroyController = createController();
});
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlQueued"</span>, <span style="color:#080;font-weight:bold">function</span> () {
controlRequested = <span style="color:#080;font-weight:bold">true</span>;
controlGranted = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">true</span>;
requestCtrlBtn.classList.remove(<span style="color:#d20;background-color:#fff0f0">'btn-primary'</span>);
requestCtrlBtn.classList.add(<span style="color:#d20;background-color:#fff0f0">'btn-warning'</span>);
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Waiting in Queue"</span>;
});
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlReleased"</span>, <span style="color:#080;font-weight:bold">function</span> () {
controlRequested = <span style="color:#080;font-weight:bold">false</span>;
controlGranted = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.disabled = <span style="color:#080;font-weight:bold">false</span>;
requestCtrlBtn.innerText = <span style="color:#d20;background-color:#fff0f0">"Request Control"</span>;
requestCtrlBtn.classList.remove(...[<span style="color:#d20;background-color:#fff0f0">'btn-success'</span>, <span style="color:#d20;background-color:#fff0f0">'btn-warning'</span>]);
requestCtrlBtn.classList.add(<span style="color:#d20;background-color:#fff0f0">'btn-primary'</span>);
messageCtrl.innerText = <span style="color:#d20;background-color:#fff0f0">''</span>;
messageCtrl.classList.add(<span style="color:#d20;background-color:#fff0f0">'d-none'</span>);
<span style="color:#080;font-weight:bold">if</span> (destroyController != <span style="color:#080;font-weight:bold">null</span>) {
destroyController();
destroyController = <span style="color:#080;font-weight:bold">null</span>;
}
});
controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ControlRemaining"</span>, <span style="color:#080;font-weight:bold">function</span> (seconds) {
messageCtrl.classList.remove(<span style="color:#d20;background-color:#fff0f0">'d-none'</span>);
messageCtrl.innerText = <span style="color:#d20;background-color:#fff0f0">`Control Granted: </span><span style="color:#33b;background-color:#fff0f0">${</span>seconds<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0"> seconds remaining`</span>;
});
</code></pre></div><p>At the top of the file, import the <code>three.js</code> library along with the <code>OrbitalControl</code> add-on to control the cube camera position.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">import</span> * as THREE from <span style="color:#d20;background-color:#fff0f0">'three'</span>;
<span style="color:#080;font-weight:bold">import</span> { OrbitControls } from <span style="color:#d20;background-color:#fff0f0">'three/addons/controls/OrbitControls.js'</span>;
</code></pre></div><p>Add <code>createRenderer</code>, <code>createScene</code>, <code>createCamera</code>, and <code>createCube</code> helper methods that are required for creating cube and attaching it to the receiver as well as the controller section to show a cube.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">function</span> createRenderer(canvasDOMId) {
<span style="color:#888">// Load a Renderer
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">let</span> renderer = <span style="color:#080;font-weight:bold">new</span> THREE.WebGLRenderer({ alpha: <span style="color:#080;font-weight:bold">false</span>, antialias: <span style="color:#080;font-weight:bold">true</span> });
renderer.setClearColor(<span style="color:#00d;font-weight:bold">0xC5C5C3</span>);
renderer.setPixelRatio(<span style="color:#038">window</span>.devicePixelRatio);
renderer.setSize(<span style="color:#00d;font-weight:bold">250</span>, <span style="color:#00d;font-weight:bold">250</span>);
<span style="color:#038">document</span>.getElementById(canvasDOMId).appendChild(renderer.domElement);
<span style="color:#080;font-weight:bold">return</span> renderer;
}
<span style="color:#080;font-weight:bold">function</span> createScene() {
<span style="color:#888">// Load 3D Scene
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">let</span> scene = <span style="color:#080;font-weight:bold">new</span> THREE.Scene();
<span style="color:#888">// Load Light
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">var</span> ambientLight = <span style="color:#080;font-weight:bold">new</span> THREE.AmbientLight(<span style="color:#00d;font-weight:bold">0xcccccc</span>);
scene.add(ambientLight);
<span style="color:#080;font-weight:bold">var</span> directionalLight = <span style="color:#080;font-weight:bold">new</span> THREE.DirectionalLight(<span style="color:#00d;font-weight:bold">0xffffff</span>);
directionalLight.position.set(<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">1</span>).normalize();
scene.add(directionalLight);
<span style="color:#080;font-weight:bold">return</span> scene;
}
<span style="color:#080;font-weight:bold">function</span> createCamera(cameraZPosition = <span style="color:#00d;font-weight:bold">10</span>) {
<span style="color:#888">// Load Camera Perspective
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">let</span> camera = <span style="color:#080;font-weight:bold">new</span> THREE.PerspectiveCamera(<span style="color:#00d;font-weight:bold">50</span>, <span style="color:#00d;font-weight:bold">250</span> / <span style="color:#00d;font-weight:bold">250</span>, <span style="color:#00d;font-weight:bold">1</span>, <span style="color:#00d;font-weight:bold">200</span>);
camera.position.z = <span style="color:#00d;font-weight:bold">5</span>;
<span style="color:#080;font-weight:bold">return</span> camera;
}
<span style="color:#080;font-weight:bold">function</span> createCube() {
<span style="color:#080;font-weight:bold">const</span> geometry = <span style="color:#080;font-weight:bold">new</span> THREE.BoxGeometry(<span style="color:#00d;font-weight:bold">2</span>, <span style="color:#00d;font-weight:bold">2</span>, <span style="color:#00d;font-weight:bold">2</span>).toNonIndexed();;
<span style="color:#080;font-weight:bold">const</span> positionAttribute = geometry.getAttribute(<span style="color:#d20;background-color:#fff0f0">'position'</span>);
<span style="color:#080;font-weight:bold">const</span> colors = [];
<span style="color:#080;font-weight:bold">const</span> color = <span style="color:#080;font-weight:bold">new</span> THREE.Color();
<span style="color:#080;font-weight:bold">for</span> (<span style="color:#080;font-weight:bold">let</span> i = <span style="color:#00d;font-weight:bold">0</span>; i < positionAttribute.count; i += <span style="color:#00d;font-weight:bold">3</span>) {
<span style="color:#080;font-weight:bold">if</span> (i >= <span style="color:#00d;font-weight:bold">0</span> && i <= <span style="color:#00d;font-weight:bold">11</span>) {
color.set(<span style="color:#00d;font-weight:bold">0xffff00</span>); <span style="color:#888">// x facing yellow
</span><span style="color:#888"></span> }
<span style="color:#080;font-weight:bold">else</span> <span style="color:#080;font-weight:bold">if</span> (i >= <span style="color:#00d;font-weight:bold">12</span> && i <= <span style="color:#00d;font-weight:bold">23</span>) {
color.set(<span style="color:#00d;font-weight:bold">0xff0000</span>); <span style="color:#888">// y facing red
</span><span style="color:#888"></span> }
<span style="color:#080;font-weight:bold">else</span> {
color.set(<span style="color:#00d;font-weight:bold">0x0000ff</span>); <span style="color:#888">// z facing blue
</span><span style="color:#888"></span> }
<span style="color:#888">// define the same color for each vertex of a triangle
</span><span style="color:#888"></span> colors.push(color.r, color.g, color.b);
colors.push(color.r, color.g, color.b);
colors.push(color.r, color.g, color.b);
}
geometry.setAttribute(<span style="color:#d20;background-color:#fff0f0">'color'</span>, <span style="color:#080;font-weight:bold">new</span> THREE.Float32BufferAttribute(colors, <span style="color:#00d;font-weight:bold">3</span>));
<span style="color:#080;font-weight:bold">const</span> material = <span style="color:#080;font-weight:bold">new</span> THREE.MeshBasicMaterial({ vertexColors: <span style="color:#080;font-weight:bold">true</span> });
<span style="color:#080;font-weight:bold">let</span> cube = <span style="color:#080;font-weight:bold">new</span> THREE.Mesh(geometry, material);
<span style="color:#888">// wireframe
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">var</span> geo = <span style="color:#080;font-weight:bold">new</span> THREE.EdgesGeometry(cube.geometry); <span style="color:#888">// or WireframeGeometry
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">var</span> mat = <span style="color:#080;font-weight:bold">new</span> THREE.LineBasicMaterial({ color: <span style="color:#00d;font-weight:bold">0xffffff</span> });
<span style="color:#080;font-weight:bold">var</span> wireframe = <span style="color:#080;font-weight:bold">new</span> THREE.LineSegments(geo, mat);
cube.add(wireframe);
<span style="color:#080;font-weight:bold">return</span> cube;
}
</code></pre></div><p>Use the helper methods in the receiver section to create the cube and display it.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#888">// Define variables
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">const</span> receiverRenderer = createRenderer(<span style="color:#d20;background-color:#fff0f0">'receiver-wrapper'</span>);
<span style="color:#080;font-weight:bold">const</span> receiverScene = createScene();
<span style="color:#080;font-weight:bold">const</span> receiverCamera = createCamera();
<span style="color:#080;font-weight:bold">const</span> receiverCube = createCube();
receiverScene.add(receiverCube);
<span style="color:#080;font-weight:bold">let</span> receiverControls = <span style="color:#080;font-weight:bold">new</span> OrbitControls(receiverCamera, receiverRenderer.domElement);
receiverControls.enabled = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">function</span> animate() {
requestAnimationFrame(animate);
receiverControls.update();
receiverRenderer.render(receiverScene, receiverCamera);
}
animate();
</code></pre></div><p>Listen to the <code>ReceiveData</code> event to update the receiver section cube with updated camera position data.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">controlHubConnection.on(<span style="color:#d20;background-color:#fff0f0">"ReceiveData"</span>, <span style="color:#080;font-weight:bold">function</span> (cameraData) {
receiverCamera.position.x = cameraData.x;
receiverCamera.position.y = cameraData.y;
receiverCamera.position.z = cameraData.z;
});
</code></pre></div><p>Add a <code>createController</code> method which is called when a user is granted control of the cube. It creates a cube and renders on the controller section and listens to the camera position change.</p>
<p>When the cube is rotated, it calls the server method <code>SendData</code> using the SignalR connection. It also returns a method that cancels the cube renderer and removes the listener from the controller cube. This returned method is called when the <code>ControlReleased</code> event is called from server.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#080;font-weight:bold">function</span> createController() {
<span style="color:#080;font-weight:bold">const</span> controllerRenderer = createRenderer(<span style="color:#d20;background-color:#fff0f0">'controller-wrapper'</span>);
<span style="color:#080;font-weight:bold">const</span> controllerScene = createScene();
<span style="color:#080;font-weight:bold">const</span> controllerCamera = createCamera();
<span style="color:#080;font-weight:bold">const</span> controllerCube = createCube();
controllerScene.add(controllerCube);
<span style="color:#080;font-weight:bold">let</span> controls = <span style="color:#080;font-weight:bold">new</span> OrbitControls(controllerCamera, controllerRenderer.domElement);
<span style="color:#080;font-weight:bold">function</span> onPositionChange(o) {
controlHubConnection.invoke(<span style="color:#d20;background-color:#fff0f0">"SendData"</span>, controllerCamera.position).<span style="color:#080;font-weight:bold">catch</span>(<span style="color:#080;font-weight:bold">function</span> (err) {
<span style="color:#080;font-weight:bold">return</span> console.error(err.toString());
});
}
controls.addEventListener(<span style="color:#d20;background-color:#fff0f0">'change'</span>, onPositionChange);
<span style="color:#080;font-weight:bold">let</span> request;
<span style="color:#080;font-weight:bold">function</span> controllerAnimate() {
request = requestAnimationFrame(controllerAnimate);
controls.update();
controllerRenderer.render(controllerScene, controllerCamera);
}
controllerAnimate();
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">function</span> () {
<span style="color:#038">document</span>.getElementById(<span style="color:#d20;background-color:#fff0f0">'controller-wrapper'</span>).innerHTML = <span style="color:#d20;background-color:#fff0f0">''</span>;
cancelAnimationFrame(request);
controls.removeEventListener(<span style="color:#d20;background-color:#fff0f0">'change'</span>, onPositionChange);
}
}
</code></pre></div><h3 id="wrapping-up">Wrapping Up</h3>
<p>We learned the basics of SignalR and how to build a simple application to communicate between the backend and frontend using a SignalR connection. We also looked into a practical implementation on how can we use SignalR to control a graphical object built with three.js. We can extend the application to build apps like a multi-player game, or to send messages to the frontend from long-running backend background jobs.</p>
<p>You can get the code at <a href="https://github.com/bimalghartimagar/EPSignalRControl">GitHub</a>.</p>
Client Profile: J.G. Title Companyhttps://www.endpointdev.com/blog/2023/12/client-profile-j-g-title-company/2023-12-27T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2023/12/client-profile-j-g-title-company/j-g-title-company.webp" alt="The J.G. Title Company logo sits in the center of an image on a blue background with futuristic designs reminiscent of circuit boards. Text on either side of the logo reads “Nationwide titling”, with text under reading “info@jgtitleco.com” and “https://jgtitleco.com”"></p>
<p><a href="https://jgtitleco.com/">J.G. Title</a> is a prominent service company specializing in automotive dealership titling and registration processes across the country. They simplify operations for their dealers, businesses, and individuals by giving tax/fee quotes, document validation, and checklists for all jurisdictions.</p>
<p>Jordan Kivett, the owner of J.G. Title, envisioned an innovative web application to simplify dealership operations. He contacted us to request a quote for developing this solution and making his vision a reality. This project became a great opportunity for our <a href="/team/">.NET team</a> to build a robust system with state-of-the-art technologies functioning seamlessly together.</p>
<h3 id="the-solution">The solution</h3>
<p>J.G. Title submitted a detailed document containing a description of the requirements for the J.G. Title Suite app along with workflow diagrams explaining their current business processes. After analyzing all the documents and having some initial meetings, we decided to divide the project into different stages, and estimate each phase of work separately. We ended up with three phases:</p>
<ul>
<li>Phase I: Create a product that allows users to enter deals into the system and retrieve the quote estimation, including taxes, fees, and charges for services across 50 states.</li>
<li>Phase II: Implement several UI improvements such as a detailed dealership dashboard, financial management, and integration with <a href="https://quickbooks.intuit.com/">QuickBooks</a>.</li>
<li>Phase III: Add an ambitious PDF document management section featuring document recognition, analysis, and manipulation, to perform automatic validations on the document’s contents and automatically generate PDF outputs from the deal’s details.</li>
</ul>
<h3 id="tech-stack">Tech stack</h3>
<ul>
<li><a href="https://dotnet.microsoft.com/en-us/learn/dotnet/what-is-dotnet">.NET</a> 6 with <a href="https://learn.microsoft.com/en-us/dotnet/csharp/tour-of-csharp/">C#</a></li>
<li><a href="https://www.postgresql.org/">PostgreSQL</a> 14</li>
<li><a href="https://rockylinux.org/">Rocky Linux</a> 9 with <a href="https://www.nginx.com/">Nginx</a></li>
</ul>
<p>We had already worked with this combination of technologies for several clients, and we are confident in the combined power in a robust database such as Postgres along with .NET 6 running in a Linux environment. It has proven to be a stable, fast, and reliable setup for our web solutions.</p>
<p>We also used <a href="https://trello.com/">Trello</a> for tracking the project’s progress and tasks, <a href="https://github.com/">GitHub</a> for version control, and <a href="https://visualstudio.microsoft.com/vs/">Visual Studio 2022</a> or <a href="https://code.visualstudio.com/">VS Code</a> with <a href="https://learn.microsoft.com/en-us/dotnet/core/tools/">.NET CLI</a> as a coding environment. Some of us also used <a href="https://code.visualstudio.com/docs/devcontainers/tutorial">VS Code dev containers</a>. <a href="https://www.pgadmin.org/">pgAdmin</a> is our usual tool to manage the local Postgres database, as well as <a href="https://www.postman.com/">Postman</a> to test our API endpoints.</p>
<h3 id="the-team">The team</h3>
<p>Most of our .NET team is involved on this project. We also receive valuable help from our hosting team and our Postgres developers.</p>
<ul>
<li><a href="/team/bimal-gharti-magar/">Bimal Gharti Magar</a></li>
<li><a href="/team/dan-briones/">Dan Briones</a></li>
<li><a href="/team/dylan-wooters/">Dylan Wooters</a></li>
<li><a href="/team/juan-pablo-ventoso/">Juan Pablo Ventoso</a></li>
<li><a href="/team/kevin-campusano/">Kevin Campusano</a></li>
<li><a href="/team/mike-delange/">Mike DeLange</a></li>
</ul>
<p>The client is also highly involved in all aspects of the development process, from requirements gathering to documenting, testing, and providing feedback. We have bi-weekly standup calls, and the entire team (End Point + J.G. Title) interacts actively through Trello, working together and updating each task as the work progresses.</p>
<h3 id="results">Results</h3>
<p>We began working on the project on October 25, 2022, and completed Phase I within the expected timeline, on June 7, 2023. The next phases were launched iteratively, with new deployments usually scheduled twice a week.</p>
<p>The application is now widely used by dealerships and the J.G. Title team. They have over 200 users registered in the system, with more than 8,000 deals quoted, and new deals being added at an increasing rate.</p>
<p><img src="/blog/2023/12/client-profile-j-g-title-company/j-g-title-suite-featured.webp" alt="On the right side is an image of a laptop and phone, each running the J.G. Title Suite application. Text on the left reads “From chaos to clarity. Simplify, streamline, succeed: Embrace clarity in your dealership’s tax and title operations. Our dedicated team of specialists conducted extensive research and established partnerships with local DMVs and state Treasurers to become the trusted partner for automotive dealerships and businesses nationwide, offering efficient and reliable title and registration services. We continue transforming automotive title management on a national scale and welcome you to experience the future of automotive titling.""><br>
The J.G. Title Suite application, featured on the company’s website.</p>
<p>We are continuing work on several tasks related to Phase III of the project. As the application keeps growing and users send feedback, new phases of work will surely be added to the project in the future.</p>
<p>This partnership is a testament to our shared vision for efficiency and innovation, and we’re excited to continue reshaping the industry with J.G. Title Company!</p>
I wrote the same app twice, with Hotwire and Blazor Server — here’s what I learnedhttps://www.endpointdev.com/blog/2023/05/i-wrote-the-same-app-twice-with-hotwire-and-blazor-server/2023-05-27T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2023/05/i-wrote-the-same-app-twice-with-hotwire-and-blazor-server/2022-09-14_193717.webp" alt="A dark sky sprawls over a tall canyon. Misty clouds hang on jagged peaks on the hill to the left. The other hill on the right rises at a steep angle, making a “V” shape. Both hills are covered in different shades of green, mostly a deep, dark green due to the late hour. Some sun peeks through the overcast sky to let through a bit of bluer light."></p>
<!-- Photo by Seth Jensen, 2022. -->
<p>There’s been a very interesting movement that has emerged recently in the world of frontend web development: a rise of little-to-no-JavaScript frontend frameworks.</p>
<p>The promise here is that we would be able to develop web applications with rich interactive capabilities without the need to write a whole lot of JavaScript. As such, these new approaches present themselves as alternatives to the likes of Vue, React, Angular, etc.</p>
<p>Two recent technologies that try to fulfill this promise come from two of the most prolific web application development frameworks of today: <a href="https://dotnet.microsoft.com/en-us/apps/aspnet/web-apps/blazor">Blazor</a>, built on .NET, and <a href="https://hotwired.dev/">Hotwire</a>, built on Ruby on Rails.</p>
<p>Now, I love my JS frameworks as much as the next guy, but these new technologies are intriguing. So I decided to build the same application twice, with Hotwire and with Blazor. I learned a few things along the way that I would like to share in this blog post.</p>
<blockquote>
<p>Note that there is a <a href="#table-of-contents">table of contents</a> at the end of this post.</p>
</blockquote>
<h3 id="what-this-article-is">What this article is</h3>
<p>I want to present some of my findings when working with these two technologies. I also want to discuss how they work and how they feel. How they are similar and how they are different. How they take different routes to arrive at their ultimately similar destinations. Maybe offer some pros and cons.</p>
<p>This post assumes sufficient familiarity with <a href="https://learn.microsoft.com/en-us/dotnet/csharp/">C#</a>, <a href="https://dotnet.microsoft.com/en-us/apps/aspnet">ASP.NET</a>, <a href="https://www.ruby-lang.org/en/">Ruby</a>, <a href="https://rubyonrails.org/">Ruby on Rails</a> and the current state of the art of web development. I won’t assume any familiarity with either Blazor or Hotwire, but this is not a tutorial for either, so I won’t explain in detail how to fully build apps with these technologies.</p>
<p>So who am I writing this for? Essentially, for anybody who is curious about these technologies and is interested in understanding the big picture of what they are about, how they compare to each other, and building their next project with one of them. So, this article is intended to serve more as an introduction to both, a starting point for a conversation to help you make a decision on what’s best for you and your team.</p>
<p>Spoiler alert: Both are great and you can’t go wrong with either. It all comes down to your team’s preferences and past experience.</p>
<p>One final thing worth noting is that I’m focusing this article on “<a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/?view=aspnetcore-7.0#blazor-server">Blazor Server</a>” specifically. I’ll be using the word “Blazor” moving forward, for short. Blazor as a framework has three variants: Blazor Server, Blazor WebAssembly and Blazor Hybrid. You can learn more about them <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/?view=aspnetcore-7.0">here</a>.</p>
<blockquote>
<p>Most of the examples that I’ll use in this post come from a couple of demo apps that I built in order to get my feet wet with these technologies. You can find both of them in GitHub. <a href="https://github.com/megakevin/quote-editor-blazor">Here’s the Blazor one</a> and <a href="https://github.com/megakevin/quote-editor-hotwire">here’s the Hotwire one</a>. You can study both the source code and the commit history, which I tried my best to keep neatly organized. They are both functionally identical, and based on <a href="https://www.hotrails.dev/turbo-rails">this excellent Hotwire tutorial</a>.</p>
</blockquote>
<h3 id="an-overview-of-blazor">An overview of Blazor</h3>
<p>When it comes to how they are designed and the developer experience they offer, these two are very different. Let’s go over some of the key details of Blazor and then we’ll do the same with Hotwire. With that, the differences between them will become apparent.</p>
<p>The first thing we have to understand about Blazor is that it is a component framework, very much like Vue or React. So, with Blazor, applications are broken up into composable modules called “<a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/components/?view=aspnetcore-7.0">Razor/Blazor components</a>” that are essentially independent pieces of GUI bundled with their corresponding logic. Each component has three parts to it:</p>
<ol>
<li>HTML-like markup that describes the layout and UI elements to be rendered,</li>
<li>C# logic that defines the behavior of the component, like what actions to take when users interact with GUI elements, and</li>
<li>CSS for styling the GUI elements.</li>
</ol>
<p>For example here’s a simple a Blazor component that allows displaying, editing, and deleting a particular type of record called “Quote”. Don’t worry about the details too much; we’ll go over some of them next. For now, I just want us to get a sense of what Blazor components look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">@using Microsoft.EntityFrameworkCore
@inject IDbContextFactory<QuoteEditorBlazor.Data.QuoteEditorContext> dbContextFactory
@if (isEditing)
{
<QuoteEditorBlazor.Shared.Quotes.Edit
QuoteToEdit=<span style="color:#d20;background-color:#fff0f0">"QuoteToShow"</span>
OnCancel=<span style="color:#d20;background-color:#fff0f0">"HideEditForm"</span>
/>
}
<span style="color:#080;font-weight:bold">else</span>
{
<div class=<span style="color:#d20;background-color:#fff0f0">"quote"</span>>
<a href=<span style="color:#d20;background-color:#fff0f0">"quotes/@QuoteToShow.ID"</span>>@QuoteToShow.Name</a>
<div class=<span style="color:#d20;background-color:#fff0f0">"quote__actions"</span>>
<a class=<span style="color:#d20;background-color:#fff0f0">"btn btn--light"</span> @onclick=<span style="color:#d20;background-color:#fff0f0">"DeleteQuote"</span>>Delete</a>
<a class=<span style="color:#d20;background-color:#fff0f0">"btn btn--light"</span> @onclick=<span style="color:#d20;background-color:#fff0f0">"ShowEditForm"</span>>Edit</a>
</div>
</div>
}
@code {
<span style="color:#369"> [Parameter]</span>
<span style="color:#080;font-weight:bold">public</span> QuoteEditorBlazor.Models.Quote QuoteToShow { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [Parameter]</span>
<span style="color:#080;font-weight:bold">public</span> EventCallback OnQuoteDeleted { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888;font-weight:bold">bool</span> isEditing = <span style="color:#080;font-weight:bold">false</span>;
<span style="color:#080;font-weight:bold">void</span> ShowEditForm()
{
isEditing = <span style="color:#080;font-weight:bold">true</span>;
}
<span style="color:#080;font-weight:bold">void</span> HideEditForm()
{
isEditing = <span style="color:#080;font-weight:bold">false</span>;
}
<span style="color:#080;font-weight:bold">void</span> DeleteQuote()
{
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">var</span> context = dbContextFactory.CreateDbContext();
context.Quotes.Remove(QuoteToShow);
context.SaveChanges();
OnQuoteDeleted.InvokeAsync();
}
}
</code></pre></div><p>You can see that we have some C# in the file (enclosed in a <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0#code"><code>@code</code></a> block) with some event handlers and parameters. We also have some markup written with a mixture of HTML and C#. This markup has conditionals, wires up click event handlers, renders data from a given record, renders another component, etc. That markup is really just <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0">Razor</a>, a templating language that has been widely used in ASP.NET for a good while now. And we also have some top-level statements like <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0#using"><code>@using</code></a> and <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0#inject"><code>@inject</code></a> for including classes and objects that the component can use.</p>
<p>As far as CSS goes, <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/components/css-isolation?view=aspnetcore-7.0">here’s how it works</a>: Suppose these are the contents of a file called <code>Example.razor</code>. The CSS for it would have to be defined in an <code>Example.razor.css</code> file sitting right next to it. Syntax-wise, this would just be a plain old CSS file. One cool thing to mention about it, though, is that the CSS within it is visible only to the component. So there’s no risk of conflicting rules between components.</p>
<p>If you’re familiar with modern frontend web development and have used frameworks like Vue or React, this should look very familiar to you. In fact, I would venture to say this is one of Blazor’s most attractive points. If you come from that background, and know .NET, it’s not that big of a leap to get into Blazor. The development experience is very similar as its design shares many concepts with modern JS frameworks; they operate under a very similar mental model.</p>
<p>Of course, C# is not JavaScript and .NET is not a browser. So there are many differences when it comes to the nitty-gritty mechanics of things. So a thorough understanding of the .NET framework and its <a href="https://learn.microsoft.com/en-us/dotnet/standard/framework-libraries">libraries</a> is also important for mastering Blazor. Which, depending on your team’s background, may be a point in favor or against.</p>
<h4 id="how-blazor-works-under-the-hood">How Blazor works under the hood</h4>
<p>As you can probably gather from the previous discussion, Blazor takes control of the entire GUI of the application and puts up a firm layer of abstraction over traditional native web technologies. All notion of JavaScript or code executing on a browser environment is greatly de-emphasized. In fact, most of Blazor executes in the server.</p>
<p>Essentially, all the code that you actually write executes on the server side. When a user begins using the app, a two-way, persistent <a href="https://dotnet.microsoft.com/en-us/apps/aspnet/signalr">SignalR</a> connection is established between the browser and server. Whenever the client requests a page, the server renders it and sends it through the connection to the client browser, where there’s a component that interprets and displays it. Likewise, whenever the user interacts with some UI element, like clicking a button or a link, the action is sent to the server via this SignalR connection for it to be processed. If changes to the GUI are necessary, the server calculates them and sends them back to the user’s browser, where they will be interpreted and used to re-render the screen with the new state.</p>
<p><a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/?view=aspnetcore-7.0#blazor-server">Microsoft’s official documentation</a> has an excellent diagram that helps explain the process:</p>
<p><img src="/blog/2023/05/i-wrote-the-same-app-twice-with-hotwire-and-blazor-server/blazor-server.png" alt="On the left, a cloud icon sits in front of a server icon. The cloud is labeled “ASP.NET Core”, and has two smaller boxes labeled “Razor Components” and “.NET”. There are two arrows pointing to and from a web browser diagram, with the arrows labeled “SignalR”. The web browser diagram has a smaller box in it labeled “DOM”."></p>
<p>So, even though there is client side code running and browser DOM being manipulated, this is all happening under the hood. The developer doesn’t need to be concerned with that and can just focus on authoring C# code, for the most part.</p>
<p>This approach has a few implications worth noting. One is that this means higher load on the server compared to more traditional web applications. This is mainly because there needs to be a connection always open between clients and the server, by design. Classic HTTP is purely stateless, and connections are typically opened and closed multiple times throughout a user’s session, as they interact with the web app. Not so for Blazor, where this SignalR connection (most likely via <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API">WebSockets</a>) is always alive. So, for scalability concerns, we need to keep in mind that the more concurrent users our app has, the more resources the server will consume, even if the users are somewhat idle.</p>
<h4 id="an-abstraction-over-the-requestresponse-cycle">An abstraction over the request/response cycle</h4>
<p>A key element of Blazor is that it abstracts developers from the classic <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview#http_messages">HTTP request/response model</a>. With Blazor, one seldom has to consider that aspect, as all interactions between client and server are managed via the framework itself through the persistent connection we discussed in the last couple of paragraphs. To the developer, there’s no real separation between the two. This is a big departure from classic <a href="https://dotnet.microsoft.com/en-us/apps/aspnet/mvc">MVC</a>-like frameworks where things like endpoints, actions, and controllers are front and center. Such concepts are simply not in play on Blazor apps. The idea is to make them feel more like desktop apps: fully integrated, monolithic, simple packages of GUI and functionality.</p>
<p>This could become a double-edged sword as, in general, it is always important to have a clear understanding of the underlying technologies that support the application stack you’re working with. That is, the web is still there, even if you can’t see it. But as long as you’re cognizant of that, this approach can have great advantages too.</p>
<p>For example, thanks to this approach, there’s no need to employ the classic <a href="https://developer.mozilla.org/en-US/docs/Glossary/SPA">SPA pattern</a> of developing applications in two halves: 1. a backend Web API for domain logic written in some backend programming language, and 2. a frontend application written in JavaScript that implements the user experience and communicates with the backend over HTTP.</p>
<p>Blazor attempts to offer a simpler solution from a developer’s perspective, where everything lives in the same execution environment so there’s no need for inter-process communication over HTTP. The example Blazor component from before demonstrates such a case. Notice how this link has an event handler registered to its click event:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">a</span> <span style="color:#a61717;background-color:#e3d2d2">@</span><span style="color:#369">onclick</span>=<span style="color:#d20;background-color:#fff0f0">"DeleteQuote"</span>>Delete</<span style="color:#b06;font-weight:bold">a</span>>
</code></pre></div><p>That <code>DeleteQuote</code> event handler, simply defined as the method below, directly leverages the <a href="https://learn.microsoft.com/en-us/ef/ef6/fundamentals/working-with-dbcontext"><code>DbContext</code></a> to issue the delete command to the underlying database:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">void</span> DeleteQuote()
{
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">var</span> context = dbContextFactory.CreateDbContext();
context.Quotes.Remove(QuoteToShow);
context.SaveChanges();
}
</code></pre></div><p>Pretty simple.</p>
<p>However, this flexibility also requires great discipline from the development team. It falls upon us not to clutter the GUI code with all manner of extraneous things like database calls and domain or application logic that have nothing to do with user interface concerns. The traditional two-halves pattern for SPAs has this separation between domain and GUI logic baked in by necessity. Blazor allows us to break free from it, but that does not mean that the separation is not useful or even necessary. For a small example like this, this works just fine, but for larger applications, a more modularized design should be considered as well. Maybe the introduction of abstractions like <a href="https://martinfowler.com/eaaCatalog/repository.html">repositories</a> or <a href="https://martinfowler.com/bliki/EvansClassification.html">domain services</a>? At the end of the day, the core software design principles still need to be applied.</p>
<h4 id="how-blazor-supports-common-frontend-framework-features">How Blazor supports common frontend framework features</h4>
<p>Something else to consider, which I touched on before, is that Blazor is built on top of .NET. That means that a solid understanding of .NET concepts is all but a necessity in order to be effective with Blazor. Most of the features that are now traditional and expected in frontend JavaScript frameworks exist in Blazor, and they are implemented using age-old .NET concepts. If your team has solid .NET experience, this is a blessing. If not, then Blazor requires a larger investment, one that could be overwhelming depending on your time constraints.</p>
<p>Here are a few examples of how Blazor implements classic frontend framework features:</p>
<h5 id="handling-dom-events">Handling DOM events</h5>
<p>We already saw how <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/components/event-handling?view=aspnetcore-7.0">event handlers</a> work: they are defined as regular C# methods. The way they are registered to respond to events is via attributes in the GUI elements like <code>@onclick</code> or <code>@onchange</code>. All traditional DOM events are available. Like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">a</span> <span style="color:#a61717;background-color:#e3d2d2">@</span><span style="color:#369">onclick</span>=<span style="color:#d20;background-color:#fff0f0">"DeleteQuote"</span>>Delete</<span style="color:#b06;font-weight:bold">a</span>>
</code></pre></div><h5 id="including-other-blazor-components">Including other Blazor components</h5>
<p>We also saw how to render components from within other components. All it takes is adding the component to the template as if it was any other GUI element/HTML tag. The tag itself is the name of the component, which sometimes needs to be fully qualified. We saw an example before:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">QuoteEditorBlazor.Shared.Quotes.Edit</span>
<span style="color:#369">QuoteToEdit</span>=<span style="color:#d20;background-color:#fff0f0">"QuoteToShow"</span>
<span style="color:#369">OnCancel</span>=<span style="color:#d20;background-color:#fff0f0">"HideEditForm"</span>
/>
</code></pre></div><p>The fully qualified component name is <code>QuoteEditorBlazor.Shared.Quotes.Edit</code> in this case, and that’s how we reference it. If we were to include the namespace with an <code>@using</code> statement (like <code>@using QuoteEditorBlazor.Shared.Quotes</code> near the top of the file) then we would be able to invoke the component just by its name of <code>Edit</code>.</p>
<h5 id="passing-parameters-to-components">Passing parameters to components</h5>
<p>That previous snippet also demonstrates how to pass parameters to components. In this case, we have two: <code>QuoteToEdit</code> which is an object, and <code>OnCancel</code> which is a custom event. As you can see, parameters are passed as if they were HTML element attributes. In the case of <code>QuoteToEdit</code>, we’re passing it <code>QuoteToShow</code>, which is a <a href="https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/properties">C# Property</a> defined in the <code>@code</code> section of the component:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> QuoteEditorBlazor.Models.Quote QuoteToShow { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
</code></pre></div><p>In order for the receiving component to be able to accept the parameter, it needs to define a property itself of the same type, annotated with the <code>Parameter</code> <a href="https://learn.microsoft.com/en-us/dotnet/csharp/advanced-topics/reflection-and-attributes/">attribute</a>. In this case, the <code>QuoteEditorBlazor.Shared.Quotes.Edit</code> component defines it like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#369">[Parameter]</span>
<span style="color:#080;font-weight:bold">public</span> QuoteEditorBlazor.Models.Quote QuoteToEdit { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
</code></pre></div><p>Notice how the name of the property is the same name used for the “HTML attribute” that was used in markup to pass the parameter to the component:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">QuoteToEdit=<span style="color:#d20;background-color:#fff0f0">"QuoteToShow"</span>
</code></pre></div><p>In this case, the expected parameter is of type <code>QuoteEditorBlazor.Models.Quote</code>.</p>
<h5 id="defining-handling-and-triggering-custom-component-events">Defining, handling and triggering custom component events</h5>
<p>The other parameter, which is a custom event, accepts a method. It looks like this in the receiving component:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#369">[Parameter]</span>
<span style="color:#080;font-weight:bold">public</span> EventCallback OnCancel { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
</code></pre></div><p>Again, this is a Property annotated with the <code>Parameter</code> Attribute. The only difference is that the type of this one is <code>EventCallback</code>. That’s what allows it to accept a method. Then, inside the receiving component, the event can be triggered with code like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">OnCancel.InvokeAsync();
</code></pre></div><p>This will execute whatever method the parent component has registered as a handler for this custom event. In this case, that would be our <code>HideEditForm</code> method.</p>
<h5 id="handling-component-lifecycle-events">Handling component lifecycle events</h5>
<p>Other than DOM and custom events, much like in other frontend frameworks, Blazor components also offer ways of hooking up to their own internal lifecycle events. <code>OnInitialized</code> is one of the most important ones, which runs when the component is first starting up. To hook into it and run some code when it happens, all a Blazor component has to do is implement it as a <a href="https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/override">method override</a> within its code section. Something like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> OnInitialized()
{
<span style="color:#888">// Do something here.
</span><span style="color:#888"></span>}
</code></pre></div><p>There’s more to learn about component lifecycle and <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/components/lifecycle?view=aspnetcore-7.0">the official documentation</a> does a good job in explaining it.</p>
<h5 id="templates">Templates</h5>
<p>Like we discussed before, Blazor also offers a rich templating syntax via Razor, which is also a hallmark of modern frontend frameworks. We have conditional rendering, support for loops, interpolation of data, model binding, etc.</p>
<h5 id="routing">Routing</h5>
<p>Routing is also included in Blazor and the way that it works is with the <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0#page"><code>@page</code></a> directive that’s used on top of components like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">@page <span style="color:#d20;background-color:#fff0f0">"/quotes"</span>
</code></pre></div><p>Not all components will use these, only those that correspond to whole pages. Most components will be used as portions of pages and as such, would not include these. A <code>@page</code> statement like the above will make the component that includes it accessible via an URL like <code>http://mydomain.com/quotes</code>. In other words, at the root of the site.</p>
<p>These routes also support parameters, which can be defined like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">@page <span style="color:#d20;background-color:#fff0f0">"/quotes/{quoteId:int}"</span>
</code></pre></div><p>A component with this directive will be accessible through URLs like <code>http://mydomain.com/quotes/123</code>. The code in the component can access the route parameter <code>quoteId</code> by defining it as a Property with the matching type, annotated with the <code>Parameter</code> attribute. Like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#369">[Parameter]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> QuoteId { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }.
</code></pre></div><h5 id="state-management">State management</h5>
<p>Global application state management à la <a href="https://vuex.vuejs.org/">Vuex</a> or <a href="https://redux.js.org/">Redux</a> is also available in Blazor. The cool thing about how this is implemented in Blazor is that there is no need for any additional library or special components. A global app store can be a simple C# object that’s configured to have a lifetime that spans that of the user’s session. Here’s an example of a class that’s used to store global flash messages:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">QuoteEditorBlazor.AppState</span>;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">FlashStore</span>
{
<span style="color:#080;font-weight:bold">public</span> List<<span style="color:#888;font-weight:bold">string</span>> Messages { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">set</span>; } = <span style="color:#080;font-weight:bold">new</span> List<<span style="color:#888;font-weight:bold">string</span>>();
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> Action? MessagesChanged;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> <span style="color:#080;font-weight:bold">void</span> AddMessage(<span style="color:#888;font-weight:bold">string</span> message)
{
Messages.Add(message);
MessagesChanged?.Invoke();
<span style="color:#080;font-weight:bold">await</span> Task.Delay(TimeSpan.FromSeconds(<span style="color:#00d;font-weight:bold">5</span>));
Messages.Remove(message);
MessagesChanged?.Invoke();
}
}
</code></pre></div><p>Like I said: a plain and simple C# class. It offers a method for displaying a message for a few seconds. It does this by storing the given message into an internal variable and then, after a little while, it gets removed. It also offers a <code>MessagesChanged</code> <a href="https://learn.microsoft.com/en-us/dotnet/standard/events/">event</a> that other components can subscribe to which is triggered as messages are added and removed.</p>
<p>The lifetime of the instance is controlled via its <a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/dependency-injection?view=aspnetcore-7.0">dependency injection</a> configuration. In our <code>Program.cs</code>, it would be configured like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">builder.Services.AddScoped<FlashStore>();
</code></pre></div><p>Like I mentioned before, Blazor apps establish a persistent connection for the user throughout their session. This means that the instance of the app that is running on the server is also persistent. So, if we add <code>FlashStore</code> as a <a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#scoped">scoped</a> service, one single instance of it will also persist throughout the session. That’s why we can rely on its internal variables to store global application state.</p>
<p>You can learn more about state management in Blazor <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/state-management?view=aspnetcore-7.0&pivots=server">here</a>.</p>
<p>Then, you could have a component that renders those messages that looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">@using QuoteEditorBlazor.AppState
@inject FlashStore flashStore
@implements IDisposable
<div class=<span style="color:#d20;background-color:#fff0f0">"flash"</span>>
@foreach (<span style="color:#888;font-weight:bold">var</span> message <span style="color:#080;font-weight:bold">in</span> flashStore.Messages)
{
<div class=<span style="color:#d20;background-color:#fff0f0">"flash__message"</span>>
@message
</div>
}
</div>
@code {
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> OnInitialized()
{
flashStore.MessagesChanged += StateHasChanged;
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Dispose()
{
flashStore.MessagesChanged -= StateHasChanged;
}
}
</code></pre></div><p>Very simple too. This Blazor component uses a loop to render all the messages. There are a couple of interesting things about this one. First, the component gets access to the instance of the <code>FlashStore</code> class via the <code>@inject</code> directive near the top of the file. That’s how the <code>flashStore</code> variable is made available for the component to use both in C# code and in the template.</p>
<p>The second interesting element is how this component registers its <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/components/lifecycle?view=aspnetcore-7.0#state-changes-statehaschanged"><code>StateHasChanged</code></a> method to the <code>MessagesChanged</code> event defined in <code>FlashStore</code>. As you recall, <code>FlashStore</code> will trigger <code>MessagesChanged</code> every time messages are added or removed. By registering <code>StateHasChanged</code> to that event, we make sure that the component re-renders every time the messages list changes; which in turn ensures that the most current messages are always rendered. Kind of a neat trick.</p>
<p>And as you might expect, any piece of code throughout the app can submit new messages to <code>FlashStore</code> by getting hold of the global instance via dependency injection and then calling:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">flashStore.AddMessage(<span style="color:#d20;background-color:#fff0f0">"Here's a flash message!"</span>);
</code></pre></div><h5 id="interoperability-with-javascript">Interoperability with JavaScript</h5>
<p>For all the nice abstractions that Blazor presents us with, the reality is that sometimes we actually do have to break through them. In the case of JavaScript, this happens when, for example, we need access to some native browser feature, or when we need to integrate with some library for some widget or other type of capability. In Blazor, this is certainly possible.</p>
<p>There are many details to consider when it comes to JavaScript interoperability in Blazor. Check out <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/javascript-interoperability/?view=aspnetcore-7.0">the official documentation</a> to learn what all is possible. For our purposes here though, it’s enough to know that <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/javascript-interoperability/call-javascript-from-dotnet?view=aspnetcore-7.0">calls from .NET to JS</a> and <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/javascript-interoperability/call-dotnet-from-javascript?view=aspnetcore-7.0">vice versa</a> are supported.</p>
<p>The most basic example of calling JS code from .NET code looks like the following. If we have a JS function like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#038">window</span>.showMessagefromServer = (message) => {
alert(<span style="color:#d20;background-color:#fff0f0">`The server says: </span><span style="color:#33b;background-color:#fff0f0">${</span>message<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
<span style="color:#080;font-weight:bold">return</span> <span style="color:#d20;background-color:#fff0f0">"The client says thanks!"</span>;
};
</code></pre></div><p>Such a method can be invoked from a Blazor component like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">string</span> interopResult = <span style="color:#080;font-weight:bold">await</span> js.InvokeAsync<<span style="color:#888;font-weight:bold">string</span>>(<span style="color:#d20;background-color:#fff0f0">"showMessagefromServer"</span>, <span style="color:#d20;background-color:#fff0f0">"Have a nice day!"</span>);
</code></pre></div><p>The Blazor component that runs this code would have to inject an instance of <code>IJSRuntime</code> into the <code>js</code> variable with a statement like <code>@inject IJSRuntime js</code>. A key thing to note is that the JavaScript method needs to be attached to the <code>window</code> object in order for it to be accessible.</p>
<p>Like I said, the other way around also works: JavaScript code is able to call .NET logic defined in a server-side Blazor component. Here’s a simple example for us to get an idea of how it feels. We can have a method that looks like this on a Blazor component:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#369">[JSInvokable]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> Task<<span style="color:#888;font-weight:bold">string</span>> getMessageFromServer()
{
<span style="color:#080;font-weight:bold">return</span> Task.FromResult(<span style="color:#d20;background-color:#fff0f0">"Have a nice day!"</span>);
}
</code></pre></div><p>This method is public and <a href="https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/static-classes-and-static-class-members">static</a>. It is also possible to invoke non-static instance methods, but the setup is a little more complicated, and this is enough for our purposes here. Other than that, as you can see, the method needs to be annotated with the <code>JSInvokable</code> attribute and return a type of <code>Task</code>.</p>
<p>Here’s some JavaScript that calls this method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-js" data-lang="js"><span style="color:#038">window</span>.returnArrayAsync = () => {
DotNet.invokeMethodAsync(<span style="color:#d20;background-color:#fff0f0">'{YOUR APP ASSEMBLY NAME}'</span>, <span style="color:#d20;background-color:#fff0f0">'getMessageFromServer'</span>)
.then(data => {
alert(<span style="color:#d20;background-color:#fff0f0">`I asked the server and it says: </span><span style="color:#33b;background-color:#fff0f0">${</span>data<span style="color:#33b;background-color:#fff0f0">}</span><span style="color:#d20;background-color:#fff0f0">`</span>);
});
};
</code></pre></div><p>There are a few noteworthy things here. First we have the <code>DotNet.invokeMethodAsync</code> function which Blazor makes available to us in JavaScript land. That’s what we use to invoke .NET server side code. Next, the function needs to be given the <a href="https://learn.microsoft.com/en-us/dotnet/standard/assembly/">assembly</a> name of our app as well as the name of the method to invoke within the Blazor component. This is the one that was annotated with <code>JSInvokable</code>. Finally, the function itself is asynchronous and thus returns a <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise">promise</a>.</p>
<h3 id="an-overview-of-hotwire">An overview of Hotwire</h3>
<p><a href="https://hotwired.dev/">Hotwire</a> makes a promise similar to Blazor’s. However, the way it goes about it couldn’t be more different.</p>
<p>Hotwire is much simpler and more minimalistic than Blazor. Whereas in Blazor we let the framework take total control of the GUI, just like we do with other popular frontend frameworks, Hotwire feels more like a natural evolution of traditional pre-JavaScript-heavy web development. We still have <a href="https://guides.rubyonrails.org/action_controller_overview.html">controllers</a> and <a href="https://guides.rubyonrails.org/action_view_overview.html">views</a>, we still render pages on the server, and we still work in tandem with HTTP’s request/response model.</p>
<p>What Hotwire gives us, if we were to boil it down to a single sentence, is a way to refresh only portions of our pages as a result of user interactions. That is, we’re not forced to reload the entire page, as is the case with non-JavaScript web applications. For example, in Hotwire we have the ability to submit a form or click a link and, as a result of that, only update a particular message, picture, or section.</p>
<blockquote>
<p>ASP.NET veterans will find this awfully familiar. That’s because all the way back in version 3.5, <a href="https://learn.microsoft.com/en-us/aspnet/web-forms/overview/older-versions-getting-started/aspnet-ajax/">ASP.NET AJAX</a> offered a very similar feature: partial page updates with the UpdatePanel component. Indeed, Hotwire presents a very similar concept; only greatly improved and modernized.</p>
</blockquote>
<p>When it comes to actual coding, Hotwire’s footprint is minimal. It augments traditional Ruby on Rails controllers and views to produce the desired effect of page refreshes that are partial and targeted. Let’s walk through an example and you’ll see how we can start with a regular looking Rails app and then, through minor adjustments, we end up with a more richly interactive experience.</p>
<p>Imagine we are beginning to develop support for CRUDing a particular type of record called “Quote”, and we have these files:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ruby" data-lang="ruby"><span style="color:#888">### app/controllers/quotes_controller.rb</span>
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuotesController</span> < <span style="color:#036;font-weight:bold">ApplicationController</span>
<span style="color:#080;font-weight:bold">def</span> <span style="color:#06b;font-weight:bold">index</span>
<span style="color:#080;font-weight:bold">end</span>
<span style="color:#080;font-weight:bold">def</span> <span style="color:#06b;font-weight:bold">new</span>
<span style="color:#33b">@quote</span> = <span style="color:#036;font-weight:bold">Quote</span>.new
<span style="color:#080;font-weight:bold">end</span>
<span style="color:#080;font-weight:bold">end</span>
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><span style="color:#888"><!-- app/views/quotes/index.html.erb --></span>
<<span style="color:#b06;font-weight:bold">main</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"container"</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"header"</span>>
<<span style="color:#b06;font-weight:bold">h1</span>>Quotes</<span style="color:#b06;font-weight:bold">h1</span>>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= link_to "New quote", new_quote_path %>
</<span style="color:#b06;font-weight:bold">div</span>>
</<span style="color:#b06;font-weight:bold">main</span>>
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><span style="color:#888"><!-- app/views/quotes/new.html.erb --></span>
<<span style="color:#b06;font-weight:bold">main</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"container"</span>>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= link_to "Back to quotes", quotes_path %>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"header"</span>>
<<span style="color:#b06;font-weight:bold">h1</span>>New quote</<span style="color:#b06;font-weight:bold">h1</span>>
</<span style="color:#b06;font-weight:bold">div</span>>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= render "form", quote: @quote %>
</<span style="color:#b06;font-weight:bold">main</span>>
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><span style="color:#888"><!-- app/views/quotes/_form.html.erb --></span>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= simple_form_for quote do |f| %>
<span style="color:#a61717;background-color:#e3d2d2"><</span>% if quote.errors.any? %>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"error-message"</span>>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= quote.errors.full_messages.to_sentence.capitalize %>
</<span style="color:#b06;font-weight:bold">div</span>>
<span style="color:#a61717;background-color:#e3d2d2"><</span>% end %>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= f.input :name %>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= f.submit %>
<span style="color:#a61717;background-color:#e3d2d2"><</span>% end %>
</code></pre></div><blockquote>
<p>This sample is using the <code>simple_form</code> gem, which you can learn more about <a href="https://github.com/heartcombo/simple_form">here</a>.</p>
</blockquote>
<p>If you’re familiar with Rails, then you understand what’s happening here. We have a simple <code>index</code> page with a heading and a link to another page. That other page contains a form to create new Quote records. It also contains a link to go back to the <code>index</code> page.</p>
<h4 id="partial-page-updates-with-turbo-frames">Partial page updates with Turbo Frames</h4>
<p>As they are right now, these files would produce a traditional web application user experience. When links are clicked, the whole screen will be reloaded to show the page that the clicked link points to. But what if we wanted, for example, to have the new record creation form appear out of nowhere within the same <code>index</code> page, without a full page reload? Here’s what that would look like with Hotwire:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> <!-- app/views/quotes/index.html.erb -->
<main class="container">
<div class="header">
<h1>Quotes</h1>
<%= link_to "New quote",
new_quote_path,
<span style="color:#000;background-color:#dfd">+ data: { turbo_frame: dom_id(Quote.new) } %>
</span><span style="color:#000;background-color:#dfd"></span> </div>
<span style="color:#000;background-color:#dfd">+ <%= turbo_frame_tag Quote.new %>
</span><span style="color:#000;background-color:#dfd"></span> </main>
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> <!-- app/views/quotes/new.html.erb -->
<main class="container">
<%= link_to sanitize("&larr; Back to quotes"), quotes_path %>
<div class="header">
<h1>New quote</h1>
</div>
<span style="color:#000;background-color:#dfd">+ <%= turbo_frame_tag Quote.new do %>
</span><span style="color:#000;background-color:#dfd"></span> <%= render "form", quote: @quote %>
<span style="color:#000;background-color:#dfd">+ <% end %>
</span><span style="color:#000;background-color:#dfd"></span> </main>
</code></pre></div><blockquote>
<p>Remember that these examples are taken from a fully working application. Feel free to read through <a href="https://github.com/megakevin/quote-editor-hotwire">the source code</a> to have a more complete understanding of the context within which these files exist.</p>
</blockquote>
<p>And that’s really all it takes. Let’s go over it. With these changes, whenever a user clicks on the “New quote” link, instead of the browser triggering the usual GET request to then reload the screen and show the creation page, Hotwire’s frontend component captures the click event and makes the request itself via <a href="https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest">XHR</a>. The server then receives this request normally and routes it to the <code>new</code> action in the <code>quotes</code> controller. All that action does is render the <code>new.html.erb</code> template and send that back to the client as a response.</p>
<p>When the Hotwire frontend component receives this, it notices that some of the response is enclosed in a <code>turbo_frame_tag</code> whose ID is that of an empty new Quote object. Now, because the link that the user clicked also had the ID of an empty new Quote object as its <code>data-turbo_frame</code> attribute, Hotwire looks for a <code>turbo_frame_tag</code> with the same ID in the page that’s currently being shown an replaces its contents with the contents from the similarly named <code>turbo_frame_tag</code> from the incoming response.</p>
<p>With that, you’ve seen in action some of the key elements that make Hotwire work. First of all we have the <code>turbo_frame_tag</code> helper, which produces a so-called “<a href="https://turbo.hotwired.dev/handbook/frames">Turbo Frame</a>”. Turbo Frames are the main building block that we use for partial page updates. Turbo Frames essentially say: “This section of the page is allowed to be dynamically updated without a full page refresh.” In an app that uses Hotwire, whenever you’re in a page that includes a Turbo Frame, if a request is made (whether it be navigation or form submission), and if the resulting response includes within it another Turbo Frame with the same ID, then Hotwire will notice the match and update the current page’s Turbo Frame with the contents of the Turbo Frame from the response.</p>
<p>Looking back at the code, you can see how we achieved this. We added an empty Turbo Frame on the index page, right below the link to the creation page. We also wrapped the form from the creation page in a similarly named Turbo Frame. Finally, we added the <code>data-turbo_frame</code> attribute to the link in the index page to tell Hotwire that it should kick in for this link and target that specific Turbo Frame. If the link was inside the Turbo Frame, we would not have to do this. Since it is outside, Hotwire needs the little hint that says: “Treat this link as if it was inside this Turbo Frame.”</p>
<p>I feel like this is at the same time a little awkward to wrap your head around and deceptively simple. When compared to Blazor, which builds upon tried and true concepts (as far as the developer experience goes at least), Hotwire almost seems alien, with a much more unusual style. But all in all, one can’t deny just how clean and simple all of this looks, in that “Rails magic” kind of way. And once it clicks, you can begin to see a world of possibilities opening up. The Hotwire developers managed to identify and extract a general design pattern of web application interactions, one that can be leveraged to produce a lot of varying rich interactive user experiences.</p>
<p>One neat aspect worth noting is that, if the user were to disable JavaScript, the app would still fully work. It would <a href="https://developer.mozilla.org/en-US/docs/Glossary/Graceful_degradation">gracefully degrade</a>. This is a direct consequence of Hotwire’s paradigm of adding minimal features on top of the existing traditional Rails programming model.</p>
<h4 id="imperative-rendering-with-turbo-streams">Imperative rendering with Turbo Streams</h4>
<p>Let’s consider another example now. This time we’ll see another key component of Hotwire in action which is called <a href="https://turbo.hotwired.dev/handbook/streams">Turbo Streams</a>.</p>
<p>Turbo Streams helps solve the problem that emerges when the declarative style of pure Turbo Frames is not enough to obtain the fine grained control and behavior that we need. In these cases, the server needs to issue specific commands to the frontend on how to update the GUI. Via Turbo Streams, we can achieve just that. Coding-wise, these look like regular Rails view templates, albeit with some funky syntax thanks to the Turbo Stream helpers.</p>
<p>In our example, we’ll add a list of quotes in the index page, right below the link and the form. We’ll also complete our quote creation implementation so that we’re actually able to submit the form. As a result of that, we want the GUI to be updated so that the form goes away and the newly created quote is shown in the list. These last two are the things that we’ll use Turbo Streams for. Of course, these updates to the page will be done without a full page refresh.</p>
<p>Let’s start by adding a list of quotes on the index page.</p>
<p>In the controller, we query the database for all the quote records and store them in a variable that the view can later access.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> ### app/controllers/quotes_controller.rb
class QuotesController < ApplicationController
def index
<span style="color:#000;background-color:#dfd">+ @quotes = Quote.all
</span><span style="color:#000;background-color:#dfd"></span> end
def new
@quote = Quote.new
end
end
</code></pre></div><p>In the index view template, we render the collection of records.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> <!-- app/views/quotes/index.html.erb -->
<main class="container">
<div class="header">
<h1>Quotes</h1>
<%= link_to "New quote",
new_quote_path,
data: { turbo_frame: dom_id(Quote.new) } %>
</div>
<%= turbo_frame_tag Quote.new %>
<span style="color:#000;background-color:#dfd">+ <%= render @quotes %>
</span><span style="color:#000;background-color:#dfd"></span> </main>
</code></pre></div><p>Now we need to define a “_quote” partial view so that render statement we added on the index view can work <a href="https://thesaurus.plus/img/synonyms/128/automagically.png">automagically</a>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><span style="color:#888"><!-- app/views/quotes/_quote.html.erb --></span>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"quote"</span>>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= quote.name %>
</<span style="color:#b06;font-weight:bold">div</span>>
</code></pre></div><p>With this, the <code>render @quotes</code> statement will loop through all the records in <code>@quotes</code> and render this partial view for each on of them. This template is very simple. All it does is render the name of the quote.</p>
<p>Now let’s add an action that can accept quote creation form submissions:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> ### app/controllers/quotes_controller.rb
class QuotesController < ApplicationController
def index
@quotes = Quote.all
end
def new
@quote = Quote.new
end
<span style="color:#000;background-color:#dfd">+
</span><span style="color:#000;background-color:#dfd">+ def create
</span><span style="color:#000;background-color:#dfd">+ @quote = Quote.new(quote_params)
</span><span style="color:#000;background-color:#dfd">+
</span><span style="color:#000;background-color:#dfd">+ if @quote.save
</span><span style="color:#000;background-color:#dfd">+ redirect_to quotes_path, notice: "Quote was successfully created."
</span><span style="color:#000;background-color:#dfd">+ else
</span><span style="color:#000;background-color:#dfd">+ render :new
</span><span style="color:#000;background-color:#dfd">+ end
</span><span style="color:#000;background-color:#dfd">+ end
</span><span style="color:#000;background-color:#dfd">+
</span><span style="color:#000;background-color:#dfd">+ def quote_params
</span><span style="color:#000;background-color:#dfd">+ params.require(:quote).permit(:name)
</span><span style="color:#000;background-color:#dfd">+ end
</span><span style="color:#000;background-color:#dfd"></span> end
</code></pre></div><p>A typical Rails recipe for a record creation endpoint. It takes the parameters coming from the request and uses them to create a new record via the <code>Quote</code> <a href="https://guides.rubyonrails.org/active_record_basics.html">Active Record</a> model. If successful, it redirects to the index page; if not, it renders the creation page again (via the <code>new</code> action). The <code>new.html.erb</code> view template has some logic to render error messages when they are present so those are going to show up when that page is rendered as a result of unsuccessful calls to this <code>create</code> endpoint.</p>
<p>At this point, we’re able to view all the quotes on record and create new ones. Now here are the changes to Hotwire-ify this scenario.</p>
<p>First we wrap the list of quotes with a Turbo Frame named “<code>quotes</code>":</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> <!-- app/views/quotes/index.html.erb -->
<main class="container">
<div class="header">
<h1>Quotes</h1>
<%= link_to "New quote",
new_quote_path,
data: { turbo_frame: dom_id(Quote.new) } %>
</div>
<%= turbo_frame_tag Quote.new %>
<span style="color:#000;background-color:#dfd">+ <%= turbo_frame_tag "quotes" do %>
</span><span style="color:#000;background-color:#dfd"></span> <%= render @quotes %>
<span style="color:#000;background-color:#dfd">+ <% end %>
</span><span style="color:#000;background-color:#dfd"></span> </main>
</code></pre></div><p>Next, we employ Turbo Streams. Like I said, Turbo Streams materialize themselves in code as if they were view templates. So, a new file is added that looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><span style="color:#888"><!-- app/views/quotes/create.turbo_stream.erb --></span>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= turbo_stream.prepend "quotes", partial: "quotes/quote", locals: { quote: @quote } %>
<span style="color:#a61717;background-color:#e3d2d2"><</span>%= turbo_stream.update Quote.new, "" %>
</code></pre></div><p>It sort of looks like a couple of imperative statements, does it not? The first line instructs Hotwire to prepend, in the <code>"quotes"</code> Turbo Frame, a new render of the <code>app/views/quotes/_quote.html.erb</code> partial view, while passing it the <code>@quote</code> object as a parameter. The second line updates the <code>Quote.new</code> Turbo Frame to be empty. When we remember that the <code>"quotes"</code> Turbo Frame is the one that contains the list of records and the <code>Quote.new</code> Turbo Frame is the one that contains the new quote creation form, this starts to make sense. This Turbo Streams view is making the newly created quote appear in the list; and at the same time, it is making the form disappear. From a user’s perspective, this all takes place after submitting the creation form. So the user experience makes complete sense. All that with no full page refresh.</p>
<p>And finally, the controller action needs to make use of this new view template like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> ### app/controllers/quotes_controller.rb
def create
@quote = Quote.new(quote_params)
if @quote.save
<span style="color:#000;background-color:#fdd">- redirect_to quotes_path, notice: "Quote was successfully created."
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ respond_to do |format|
</span><span style="color:#000;background-color:#dfd">+ format.html { redirect_to quotes_path, notice: "Quote was successfully created." }
</span><span style="color:#000;background-color:#dfd">+ format.turbo_stream
</span><span style="color:#000;background-color:#dfd">+ end
</span><span style="color:#000;background-color:#dfd"></span> else
render :new, status: :unprocessable_entity
end
end
</code></pre></div><p>This is yet another familiar Rails pattern. This is how we specify different <a href="https://guides.rubyonrails.org/action_controller_overview.html#rendering-xml-and-json-data">response formats</a>, whether it be HTML, JSON, XML. Now, thanks to Hotwire, we can also specify Turbo Streams. This is one of the great aspects about Hotwire: it seamlessly integrates with Rails' existing features and concepts.</p>
<p>Anyway, in this case, we invoke <code>format.turbo_stream</code> within the block passed to <code>respond_to</code> and that makes it so the <code>app/views/quotes/create.turbo_stream.erb</code> view template is included in this action’s response. Hotwire’s frontend component sees this coming as part of the response and acts accordingly, updating the GUI how it’s been specified.</p>
<h4 id="adding-javascript-with-stimulus">Adding JavaScript with Stimulus</h4>
<p>The final piece of the Hotwire puzzle is <a href="https://stimulus.hotwired.dev/">Stimulus</a>, which allows us to integrate JavaScript functionality in a neat way. Stimulus is a very simple JavaScript framework, it does not take control of the entire UI. In fact, it does not render any HTML at all. Stimulus essentially offers a nice way of wiring up JS behavior to existing markup. Let’s look at a quick example of how it could hypothetically be used for showing a confirmation popup before deleting a record.</p>
<p>The JavaScript logic lives inside so-called “Stimulus controllers”. For our example, it could look something like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#888">// app/javascript/controllers/confirmations_controller.js
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">import</span> { Controller } from <span style="color:#d20;background-color:#fff0f0">"@hotwired/stimulus"</span>
<span style="color:#080;font-weight:bold">export</span> <span style="color:#080;font-weight:bold">default</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#080;font-weight:bold">extends</span> Controller {
confirm() {
<span style="color:#080;font-weight:bold">if</span> (confirm(<span style="color:#d20;background-color:#fff0f0">"Are you sure you want to delete this record?"</span>)) {
<span style="color:#888">// Carry on with the operation...
</span><span style="color:#888"></span> }
}
}
</code></pre></div><p>And now, the idea is to wire up this code so that it runs when the hypothetical delete button is clicked. Here’s what that button could like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">button</span>
<span style="color:#369">data-controller</span>=<span style="color:#d20;background-color:#fff0f0">"confirmations"</span>
<span style="color:#369">data-action</span>=<span style="color:#d20;background-color:#fff0f0">"click->confirmations#confirm"</span>
>
Delete
</<span style="color:#b06;font-weight:bold">button</span>>
</code></pre></div><p>Pretty self-explanatory. We specify the name of the controller to use via the <code>data-controller</code> attribute. We also specify, via <code>data-action</code>, what method to invoke within that controller as a result of which DOM event, <code>click</code> in this case.</p>
<p>And that’s basically all it takes to sprinkle some JavaScript on Hotwire apps. Stimulus does offer a few more useful features, which you can read more about in <a href="https://stimulus.hotwired.dev/handbook/introduction">the official documentation</a>. But for us for now, it’s enough to know that it exists, and what’s the main idea behind it.</p>
<p>So as you can hopefully see, Hotwire is much smaller in scope to Blazor. And yet, it allows us to do just the same: highly interactive applications with little to no JavaScript. This ought to make a lot of Rails developers happy.</p>
<h3 id="a-final-comparison">A final comparison</h3>
<p>I was initially thinking about ending this blog post with a flowchart of sorts to explain the process of deciding which of these two technologies you should use. But really, the decision is very simple: If your team is comfortable with .NET, use Blazor. If your team is comfortable with Ruby on Rails, use Hotwire. It’s obvious, so I won’t claim to have made a great discovery here.</p>
<p>The only thing to add is that if your team is familiar with modern frontend web development framework concepts, you’ll be even better served by Blazor and you’ll hit the ground running. If not, then even for seasoned .NET people, there will be a decent learning curve, but not steep enough to be deterred. Moreover, if your team has no modern frontend development experience at all, then Hotwire is a godsend; thanks to its “augment classic backend-heavy web app development” style.</p>
<p>With that said, let’s close out with a summary of main aspects of both technologies and how they compare to each other.</p>
<p>Overall, Hotwire is much simpler than Blazor. While Blazor is a full-fledged GUI component framework, Hotwire’s approach is more like an augmentation of classic non-JavaScript web development patterns. That said, Hotwire’s style is more unusual than Blazor’s, so if your team is already familiar with modern frontend web development, Blazor can be a great fit.</p>
<p>While both frameworks try to offer enough functionality to allow the development of rich interactive experiences without the need to write any JavaScript, the reality is that sometimes JavaScript does need to be written. Both technologies offer ways to make this happen. And while both are perfectly workable, Blazor’s solution is a bit more clunky than Hotwire’s.</p>
<p>When it comes to classic web technologies like HTTP’s request/response cycle and the separation between server and client, Blazor’s style offers a big deviation from them. It greatly de-emphasizes them and presents instead a completely different programming model, one more akin to desktop application development. The concepts of request, response, client, and server seem to vanish. Not so for Hotwire, which builds upon these classic technologies in a way where they still need to be considered and are in fact in the spotlight. While Blazor attempts to do away with these, Hotwire embraces them.</p>
<p>In Blazor, client events are sent to the server, the server renders the DOM updates and sends them to the client for updates. This happens via the persistent SignalR/WebSockets connection.</p>
<p>Hotwire, on the other hand, intercepts client events and sends classic HTTP requests (via AJAX/XHR) to the server. The server then executes the requests and sends back the responses to the client which carries out the necessary operations, generally speaking, updating sections of the page that’s already being displayed.</p>
<p>That means that at the end of the day, both frameworks do the rendering on the server side and send the rendered markup over the wire to the clients. But in Blazor, the client and server have a persistent connection, while Hotwire’s connections come and go as normal HTTP requests and responses.</p>
<p>A neat aspect of Hotwire’s programming model is that it allows an incremental approach to web development where you can start developing the app like you would a traditional, non-reactive, non-JS app, then augment it with a little code to give it SPA capabilities.</p>
<p>And that’s all for now! I for one am glad to see these types of technologies emerge. While there are many teams out there that are already effective and productive with the current landscape of frontend web development, these two are very interesting and seem capable in their own right.</p>
<p>Besides, having alternatives is never a bad thing. Depending mainly on your previous experience, these could be a great fit for projects new and old. It’s great to know that both .NET and Rails include these types of offerings and that they work pretty well.</p>
<h3 id="table-of-contents">Table of contents</h3>
<ul>
<li><a href="#what-this-article-is">What this article is</a></li>
<li><a href="#an-overview-of-blazor">An overview of Blazor</a>
<ul>
<li><a href="#how-blazor-works-under-the-hood">How Blazor works under the hood</a></li>
<li><a href="#an-abstraction-over-the-requestresponse-cycle">An abstraction over the request/response cycle</a></li>
<li><a href="#how-blazor-supports-common-frontend-framework-features">How Blazor supports common frontend framework features</a>
<ul>
<li><a href="#handling-dom-events">Handling DOM events</a></li>
<li><a href="#including-other-blazor-components">Including other Blazor components</a></li>
<li><a href="#passing-parameters-to-components">Passing parameters to components</a></li>
<li><a href="#defining-handling-and-triggering-custom-component-events">Defining, handling and triggering custom component events</a></li>
<li><a href="#handling-component-lifecycle-events">Handling component lifecycle events</a></li>
<li><a href="#templates">Templates</a></li>
<li><a href="#routing">Routing</a></li>
<li><a href="#state-management">State management</a></li>
<li><a href="#interoperability-with-javascript">Interoperability with JavaScript</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#an-overview-of-hotwire">An overview of Hotwire</a>
<ul>
<li><a href="#partial-page-updates-with-turbo-frames">Partial page updates with Turbo Frames</a></li>
<li><a href="#imperative-rendering-with-turbo-streams">Imperative rendering with Turbo Streams</a></li>
<li><a href="#adding-javascript-with-stimulus">Adding JavaScript with Stimulus</a></li>
</ul>
</li>
<li><a href="#a-final-comparison">A final comparison</a></li>
</ul>
Building and Hosting a Web App with .NET 6, Postgres and Linuxhttps://www.endpointdev.com/blog/2022/11/dotnet6-postgresql-and-linux/2022-11-03T00:00:00+00:00Dylan Wooters
<p><img src="/blog/2022/11/dotnet6-postgresql-and-linux/boat-dar-es-salaam.webp" alt="Fishing boat in Dar es Salaam. A traditional fishing boat sits on the beach at low tide, with the fading light of sunset behind. In the background, other boats float on the Msasani Bay, and several high-rise buildings are visible to the right on the Masaki peninsula."></p>
<!-- Photo by Dylan Wooters, 2022 -->
<p>For well over a decade, working with the .NET framework meant running Windows. With the release of .NET Core in 2016, developers were granted the freedom to choose their OS, including Linux; no longer were we bound to Windows. However, few took the plunge, at least in my experience. Why? Well, we are comfortable with what we know, and afraid of what we don’t.</p>
<p>The truth is that building a .NET application on Linux is not that hard, once you get over a few minor bumps in the road. And there are many advantages to this approach, including flexibility, simplicity, and lower costs.</p>
<p>To demonstrate this, we will create a simple .NET MVC web application that connects to Postgres. Then, we will host the app on Linux with Nginx. Shall we start?</p>
<h3 id="preparing-the-database">Preparing the database</h3>
<p>First, you’ll want to install Postgres locally. If you’re using a Mac, this step is very easy. Simply install <a href="https://postgresapp.com/">Postgres.app</a> and you’ll be ready to go.</p>
<p>If you’re using Windows, check out the <a href="https://www.postgresql.org/download/windows/">Windows Installers page</a> on the Postgres website to download the latest installer.</p>
<h3 id="creating-the-projects">Creating the projects</h3>
<p>To develop .NET 6 apps, you will need to install Visual Studio 2022. Check out the Visual Studio <a href="https://visualstudio.microsoft.com/downloads/">downloads page</a> for options for both Windows and Mac.</p>
<p>Start by opening up Visual Studio and creating a new Web Application (MVC) project, and choosing .NET 6.0 as the target framework. I’ve named my project “DotNetSix.Demo”. Here are the steps as they look in Visual Studio on my Mac.</p>
<p><img src="/blog/2022/11/dotnet6-postgresql-and-linux/create-web-project-1.webp" alt="Visual Studio. A window is displayed called New Project. It shows two templates, with the Web Application (Model-View-Controller) template selected. A button on the bottom right shows “Continue”."></p>
<p><img src="/blog/2022/11/dotnet6-postgresql-and-linux/create-web-project-2.webp" alt="Visual Studio. A window is displayed called New Project. At the top is reads “Configure your new Web Application (Model-View-Controller)”. There is a Target Framework dropdown with “.NET 6.0” selected and second Authentication dropdown with “No Authentication” selected. Under Advanced, the “Configure for HTTPS” checkbox is checked, and the “Do not use top-level statements” checkbox is unchecked."></p>
<p>On the final screen, go ahead and give your solution a name, and then click Create. Visual Studio should create a new solution for you, with a single web project. It will automatically create the necessary MVC folders, including Controllers, Models, and Views.</p>
<h3 id="setting-up-the-database-connection">Setting up the database connection</h3>
<p>For this demo, we’ll create a simple app that tracks books that you’ve recently read. Let’s go ahead and add a few simple models for the database: Author and Book. You can create these files in the pre-existing Models folder.</p>
<p><strong>Book Model:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Text.Json.Serialization</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Book</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> Id { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Title { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [Display(Name = "Publish Date")]</span>
<span style="color:#369"> [DisplayFormat(DataFormatString = "{0:d}")]</span>
<span style="color:#080;font-weight:bold">public</span> DateTime PublishDate { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Author? Author { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p><strong>Author Model:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Text.Json.Serialization</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Author</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> Id { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> FirstName { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> LastName { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ICollection<Book>? Books { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Next, add a folder named “Data” in your project. This will hold a few classes necessary for connecting to Postgres and creating the database. Create a new class file in the folder called <code>AppDBContext.cs</code>. This class will use EF Core to setup a database connection.</p>
<p><strong>AppDBContext.cs:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Models</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Data</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">AppDBContext</span> : DbContext
{
<span style="color:#080;font-weight:bold">public</span> AppDBContext(DbContextOptions<AppDBContext> options) : <span style="color:#080;font-weight:bold">base</span>(options)
{
Database.EnsureCreated();
DBInitializer.Initialize(<span style="color:#080;font-weight:bold">this</span>);
}
<span style="color:#080;font-weight:bold">public</span> DbSet<Book> Books { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<Author> Authors { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Then create another class file in the folder called <code>DBInitializer.cs</code>. This class will initialize the Postgres database with test data.</p>
<p><strong>DBInitializer.cs:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Models</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Data</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">DBInitializer</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> <span style="color:#080;font-weight:bold">void</span> Initialize(AppDBContext context)
{
<span style="color:#888">// Look for any existing Book records.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (context.Books.Any())
{
<span style="color:#080;font-weight:bold">return</span>; <span style="color:#888">// DB has been seeded
</span><span style="color:#888"></span> }
<span style="color:#888;font-weight:bold">var</span> books = <span style="color:#080;font-weight:bold">new</span> Book[]
{
<span style="color:#080;font-weight:bold">new</span> Book
{
Title=<span style="color:#d20;background-color:#fff0f0">"Blood Meridian"</span>,
PublishDate=DateTime.Parse(<span style="color:#d20;background-color:#fff0f0">"04-01-1985"</span>),
Author=<span style="color:#080;font-weight:bold">new</span> Author{FirstName=<span style="color:#d20;background-color:#fff0f0">"Cormac"</span>,LastName=<span style="color:#d20;background-color:#fff0f0">"McCarthy"</span>}
},
<span style="color:#080;font-weight:bold">new</span> Book
{
Title=<span style="color:#d20;background-color:#fff0f0">"The Dog of the South"</span>,
PublishDate=DateTime.Parse(<span style="color:#d20;background-color:#fff0f0">"01-31-1979"</span>),
Author=<span style="color:#080;font-weight:bold">new</span> Author{FirstName=<span style="color:#d20;background-color:#fff0f0">"Charles"</span>,LastName=<span style="color:#d20;background-color:#fff0f0">"Portis"</span>}
},
<span style="color:#080;font-weight:bold">new</span> Book
{
Title=<span style="color:#d20;background-color:#fff0f0">"Outline"</span>,
PublishDate=DateTime.Parse(<span style="color:#d20;background-color:#fff0f0">"05-15-2014"</span>),
Author=<span style="color:#080;font-weight:bold">new</span> Author{FirstName=<span style="color:#d20;background-color:#fff0f0">"Rachel"</span>,LastName=<span style="color:#d20;background-color:#fff0f0">"Cusk"</span>}
},
};
context.Books.AddRange(books);
context.SaveChanges();
}
}
}
</code></pre></div><p>At this point, you may notice some errors in your IDE. This is because we need to add two important Nuget packages: <code>Microsoft.EntityFrameworkCore</code> and <code>Npgsql.EntityFrameworkCore.PostgreSQL</code>. You can add these by right-clicking on the Dependencies folder and clicking “Manage Nuget Packages…”. Here’s how it looks in Visual Studio.</p>
<p><img src="/blog/2022/11/dotnet6-postgresql-and-linux/add-nuget-dependency.webp" alt="Visual Studio. The Nuget Packages window shows several packages in a column on the left, with the Microsoft.EntityFrameworkCore package selected. On the top right is a search bar to use to search for dependencies, and below it is an informational window on the dependency, as well as a “Add Package” button."></p>
<p>To round out the database connection, you’ll want to update your <code>Program.cs</code> file, adding the DB Context and the initializing the database. Also, you may encounter an error when you first run your application, citing incompatible dates between .NET and Postgres. To fix this, we will set the <code>Npgsql.EnableLegacyTimestampBehavior</code> to true.</p>
<p>Here is the complete <code>Program.cs</code> for your reference. Lines 7–8, 14, and 24–34 are what was added to the default Program.cs that is created as part of the web project.</p>
<p><strong>Program.cs:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Data</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">DBContext</span> = DotNetSix.Demo.Data.AppDBContext;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#888;font-weight:bold">var</span> builder = WebApplication.CreateBuilder(args);
builder.Services.AddDbContext<DBContext>(options =>
options.UseNpgsql(builder.Configuration.GetConnectionString(<span style="color:#d20;background-color:#fff0f0">"DemoDbContext"</span>)));
<span style="color:#888">// Add services to the container.
</span><span style="color:#888"></span>builder.Services.AddControllersWithViews();
<span style="color:#888;font-weight:bold">var</span> app = builder.Build();
AppContext.SetSwitch(<span style="color:#d20;background-color:#fff0f0">"Npgsql.EnableLegacyTimestampBehavior"</span>, <span style="color:#080;font-weight:bold">true</span>);
<span style="color:#888">// Configure the HTTP request pipeline.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">if</span> (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler(<span style="color:#d20;background-color:#fff0f0">"/Home/Error"</span>);
<span style="color:#888">// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
</span><span style="color:#888"></span> app.UseHsts();
}
<span style="color:#080;font-weight:bold">using</span> (<span style="color:#888;font-weight:bold">var</span> scope = app.Services.CreateScope())
{
<span style="color:#888;font-weight:bold">var</span> services = scope.ServiceProvider;
<span style="color:#888;font-weight:bold">var</span> context = services.GetRequiredService<DBContext>();
<span style="color:#888">// Note: if you're having trouble with EF, database schema, etc.,
</span><span style="color:#888"></span> <span style="color:#888">// uncomment the line below to re-create the database upon each run.
</span><span style="color:#888"></span> <span style="color:#888">//context.Database.EnsureDeleted();
</span><span style="color:#888"></span> context.Database.EnsureCreated();
DBInitializer.Initialize(context);
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();
app.MapControllerRoute(
name: <span style="color:#d20;background-color:#fff0f0">"default"</span>,
pattern: <span style="color:#d20;background-color:#fff0f0">"{controller=Books}/{action=Index}/{id?}"</span>);
app.Run();
</code></pre></div><p>Note that the pattern in <code>MapControllerRoute</code> points to a new controller and action that we will create in the next section.</p>
<h3 id="updating-the-config-file">Updating the config file</h3>
<p>The final task to connect to Postgres is to update the <code>appsettings.json</code> file with a connection string. Since I used Postgres.app to install Postgres, my connection string is simple. The username is the same as my Mac, and there is no password. Here is the full file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"ConnectionStrings": {
"DemoDbContext": "Host=localhost;Database=demo;Username=dylan"
}
}
</code></pre></div><h3 id="displaying-the-test-data">Displaying the test data</h3>
<p>Now that we have the database connection setup, let’s make some small changes to the controllers and views in order to see the data in Postgres.</p>
<p>Add a new file in the Controllers folder called <code>BookController.cs</code>. This file provides an Index controller action that queries the book data from Postgres using EFCore.</p>
<p><strong>Controllers/BooksController.cs:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Mvc</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Data</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">DotNetSix.Demo.Controllers</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">BooksController</span> : Controller
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> AppDBContext <span style="color:#00d;font-weight:bold">_</span>context;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> IConfiguration <span style="color:#00d;font-weight:bold">_</span>config;
<span style="color:#080;font-weight:bold">public</span> BooksController(AppDBContext context, IConfiguration config)
{
<span style="color:#00d;font-weight:bold">_</span>context = context;
<span style="color:#00d;font-weight:bold">_</span>config = config;
}
<span style="color:#888">// GET: Papers
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<IActionResult> Index()
{
<span style="color:#888;font-weight:bold">var</span> appDBContext = <span style="color:#00d;font-weight:bold">_</span>context.Books.Include(b => b.Author);
<span style="color:#080;font-weight:bold">return</span> View(<span style="color:#080;font-weight:bold">await</span> appDBContext.ToListAsync());
}
}
}
</code></pre></div><p>Now create an accompanying view. Add a “Books” folder under Views, and then add a new file to the Books folder called Index.cshtml. This view will receive the data from the controller and display a simple table of recently read books.</p>
<p><strong>Books/Index.cshtml:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">@model IEnumerable<DotNetSix.Demo.Models.Book>
<head>
<meta charset=<span style="color:#d20;background-color:#fff0f0">"utf-8"</span> />
<meta name=<span style="color:#d20;background-color:#fff0f0">"viewport"</span> content=<span style="color:#d20;background-color:#fff0f0">"width=device-width, initial-scale=1.0"</span> />
<link rel=<span style="color:#d20;background-color:#fff0f0">"stylesheet"</span> href=<span style="color:#d20;background-color:#fff0f0">"./css/CreateStyleSheet.css"</span> asp-append-version=<span style="color:#d20;background-color:#fff0f0">"true"</span> />
<link rel=<span style="color:#d20;background-color:#fff0f0">"stylesheet"</span> href=<span style="color:#d20;background-color:#fff0f0">"~/css/site.css"</span> asp-append-version=<span style="color:#d20;background-color:#fff0f0">"true"</span> />
</head>
<span style="color:#a61717;background-color:#e3d2d2">@</span>{
ViewData[<span style="color:#d20;background-color:#fff0f0">"Title"</span>] = <span style="color:#d20;background-color:#fff0f0">"Index"</span>;
}
<h1>Books Recently Read</h1>
<div class=<span style="color:#d20;background-color:#fff0f0">"table-wrapper"</span>>
<div class=<span style="color:#d20;background-color:#fff0f0">"table-container"</span>>
<table class=<span style="color:#d20;background-color:#fff0f0">"table"</span>>
<thead>
<tr>
<th>
@Html.DisplayNameFor(model => model.Title)
</th>
<th>
@Html.DisplayNameFor(model => model.Author)
</th>
<th>
@Html.DisplayNameFor(model => model.PublishDate)
</th>
</tr>
</thead>
<tbody>
@foreach (<span style="color:#888;font-weight:bold">var</span> item <span style="color:#080;font-weight:bold">in</span> Model)
{
<tr>
<td>
@Html.DisplayFor(modelItem => item.Title)
</td>
<td>
<span style="color:#a61717;background-color:#e3d2d2">@</span>{
<span style="color:#888;font-weight:bold">var</span> authorFullName = item.Author.FirstName + <span style="color:#d20;background-color:#fff0f0">" "</span> + item.Author.LastName;
@Html.DisplayFor(modelItem => authorFullName);
}
</td>
<td>
@Html.DisplayFor(modelItem => item.PublishDate)
</td>
</tr>
}
</tbody>
</table>
</div>
</div>
</code></pre></div><p>With the above files in place, you are now ready to run your app. Go ahead and run the project in Visual Studio. You should then see the Books index page in your browser. Hopefully it looks like this:</p>
<p><img src="/blog/2022/11/dotnet6-postgresql-and-linux/books-index-page.webp" alt="Demo page in the web browser. A new window in Brave is pointing to https://localhost:7281/ and displays a top-level navigation with our app name (DotNetsix.Demo) and Home and Privacy links. Below the navigation is a simple table displaying the book data that is saved in Postgres."></p>
<h3 id="installing-and-configuring-nginx">Installing and configuring Nginx</h3>
<blockquote>
<p>Just a heads up that the following sections borrow from the <a href="https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-nginx?view=aspnetcore-6.0">MSDN article</a> on hosting ASP.NET Core on Linux. Be sure to check out the article if you’re in need of more info or additional help!</p>
</blockquote>
<p>Now that we have the application running, we can prepare to deploy it to a Linux server. The first step in preparing your server to host the application is to install Nginx. Nginx will act as a reverse proxy to your .NET application running on localhost. To install Nginx, use the appropriate package manager for your Linux distro. For example, if you’re running Debian, use apt-get to install Nginx:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">sudo apt-get update
sudo apt-get install nginx
</code></pre></div><p>You can verify the installation by running <code>sudo nginx -v</code>, which should display the version. Finally, start Nginx using the command <code>sudo service nginx start</code>.</p>
<p>Once Nginx is installed, you’ll need to configure it to host the .NET Core application. Go ahead and create a new Nginx configuration file in <code>/etc/nginx/conf.d</code>. In our case, we’ll name it <code>dotnetsixdemo.conf</code>. Within this configuration file, we’ll do a few things:</p>
<ol>
<li>Redirect HTTP traffic to HTTPS.</li>
<li>Use HTTPS with an SSL cert installed by Let’s Encrypt <a href="https://certbot.eff.org/">certbot</a>.</li>
<li>Configure Nginx to forward HTTP requests to the .NET application, which by default will run locally at <code>http://127.0.0.1:5000</code>.</li>
</ol>
<p>Here is what our configuration file ends up looking like.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">server {
if ($host = dotnetsixdemo.org) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
listen [::]:80;
server_name dotnetsixdemo.org;
return 404; # managed by Certbot
}
server {
server_name dotnetsixdemo.org;
location / {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
listen [::]:443 ssl; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/dotnetsixdemo.org/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/dotnetsixdemo.org/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
</code></pre></div><h3 id="final-application-adjustments">Final application adjustments</h3>
<p>Before we publish the app, we need to make a small change to the <code>Program.cs</code> file. This will allow redirect and security policies to work correctly given the Nginx reverse proxy setup.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.HttpOverrides</span>;
...
app.UseForwardedHeaders(<span style="color:#080;font-weight:bold">new</span> ForwardedHeadersOptions
{
ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
});
</code></pre></div><p>Additionally, you’ll want to update your <code>appsettings.json</code> file to reflect your target environment, in particular the connection string for Postgres.</p>
<h3 id="time-to-publish">Time to publish!</h3>
<p>We are now ready to publish the app to the server. In the command prompt, navigate to the root directory of the project, and run the following command.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet publish --configuration release
</code></pre></div><p>This will build the app and create a new output folder at <code>/your-project-root/bin/Release/net6.0/publish</code>.</p>
<p>At this point, all that’s left to do is to copy the contents of the <code>publish</code> folder to the server. You can do this using a variety of tools including SCP, SFTP, etc.</p>
<h3 id="running-the-app-on-the-server">Running the app on the server</h3>
<p>Once you’ve copied the app to the server, you can start the app by navigating to the directory where the app was copied, and then running the command <code>dotnet [app_assembly].dll</code>. For our demo app, the target DLL would be <code>DotNetSix.Demo.dll</code>.</p>
<p>This will run the app at <code>http://127.0.0.1:5000</code>, and it should now be accessible via the URL that you configured using Nginx. Based on the Nginx configuration provided above, that would be <code>https://dotnetsixdemo.org</code>. Go ahead and test your site in the browser to make sure it is accessible and the reverse proxy is working properly.</p>
<h3 id="using-systemd-to-run-the-app-as-a-service">Using systemd to run the app as a service</h3>
<p>As you might have noticed, there are a few problems with running the app directly using the <code>dotnet</code> command above. The app could easily stop running if the server encounters an issue, and the app is not monitorable. To fix this, let’s use systemd to run and monitor the app process.</p>
<p>Create a new service definition file by running <code>sudo nano /etc/systemd/system/dot-net-six-demo.service</code> and entering the following.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">[Unit]
Description=Dotnet Six Demo App
[Service]
WorkingDirectory=/home/dotnetsix/publish
ExecStart=/usr/bin/dotnet /home/dotnetsix/publish/DotNetSix.Demo.dll
Restart=always
# Restart service after 10 seconds if the dotnet service crashes:
RestartSec=10
KillSignal=SIGINT
SyslogIdentifier=dotnetsixdemo
User=dotnetsix
Environment=ASPNETCORE_ENVIRONMENT=Production
Environment=DOTNET_PRINT_TELEMETRY_MESSAGE=false
[Install]
WantedBy=multi-user.target
</code></pre></div><p>Note that the paths in <code>WorkingDirectory</code> and <code>ExecStart</code> should match where you copied the application build files on the server, as part of the publish step above.</p>
<p>Also, the <code>User</code> option specifies the user that manages the service and runs the process. In our example, this is the <code>dotnetsix</code> user. You’ll want to create your own user, and importantly grant that user proper ownership of the application files.</p>
<p>To finalize the new service, save the service file and then start the service with <code>sudo systemctl enable --now dot-net-six-demo.service</code>. This will run the app now in the background via systemd and ensure it also starts up after the next reboot.</p>
<h3 id="viewing-logs">Viewing logs</h3>
<p>Since we are now running the app via systemd, we can use the <code>journalctl</code> to view logs. To view the logs for the demo app, you would run the command <code>sudo journalctl -fu dot-net-six-demo.service</code>.</p>
<h3 id="wrapping-up">Wrapping up</h3>
<p>That is all! We now have a .NET application running on our Linux server, hosted with Nginx via a reverse proxy, and connecting to a local PostgreSQL database. As we can see, there are several steps to take into account, but the process itself is not particularly complex.</p>
<p>See also my co-worker Kevin Campusano’s blog post for <a href="/blog/2021/07/dotnet-5-web-api/">more tips on using .NET with PostgreSQL</a>.</p>
<p>Have you had problems working with .NET and Linux? Do you have any alternative solutions to the ones proposed here? We await your comment.</p>
CI/CD with Azure DevOpshttps://www.endpointdev.com/blog/2022/08/cicd-with-azure-devops/2022-08-28T00:00:00+00:00Dylan Wooters
<p><img src="/blog/2022/08/cicd-with-azure-devops/moonrise-in-sibley.webp" alt="Moonrise in Sibley Volcanic Park. The sun casts a shadow over everything but the tops of the trees and a brown, grassy hill. The moon has risen just above the hill, against the backdrop of a light blue cloudless evening sky."><br>
Photo by Dylan Wooters, 2022.</p>
<p>A development process that includes manual builds, tests, and deployments can work for a small-scale project, but as your codebase and team grow, it can quickly become time-consuming and unwieldy. This can be particularly true if you’re a .NET developer. We all know the struggle of merging in the latest feature and clicking build in Visual Studio, only to have it fail, citing cryptic errors. “Maybe a clean will fix it?”</p>
<p>If this sounds like your current situation, it’s likely time to consider building a Continuous Integration and Continuous Deployment pipeline, commonly known as “CI/CD”. A good CI/CD pipeline will help you automate the painful areas of building, testing, and deploying your code, as well as help to enforce best practices like pull requests and build verification.</p>
<p>There are many great options to choose from when selecting a CI/CD tool. There are self-hosted options like <a href="https://www.jenkins.io/">Jenkins</a> and <a href="https://www.jetbrains.com/teamcity/">TeamCity</a>. There are also providers like <a href="https://github.com/features/actions">GitHub</a> and Azure DevOps, which offer CI/CD alongside cloud-hosted source control. All of these options have pros and cons, but if you’re looking for a large feature set, flexibility, and in particular good support for .NET solutions, you should consider Azure DevOps.</p>
<p>In this post, I’ll show you how to set up an Azure CI/CD pipeline for a self-hosted .NET MVC web solution targeting two different environments.</p>
<h3 id="creating-environments">Creating Environments</h3>
<p>The first step is to sign up for a free account at <a href="https://azure.microsoft.com/en-us/services/devops/">Azure DevOps</a>. Once you have your account created, you can then create your Environments. The Environments are the places where you want your app to be deployed.</p>
<p>In the case of a recent project here at End Point Dev, we had three different environments—UAT, Stage, and Production—all running as IIS websites on self-hosted Windows VMs. The UAT environment was on the UAT server, and the Stage and Production environments were on the Production server. So you’ll want to plan out your environments accordingly.</p>
<p>Once you do so, head to Azure DevOps and then click on Environments under the Pipelines section, then click Create Environment.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/create-env.webp" alt="Azure DevOps. A menu in the drawer on the left has Pipelines expanded, with Environments selected in its sub-menu. On the right, outside the menu, is a button that reads Create environment."></p>
<p>Then enter a name from the environment and select Virtual Machines and click Next.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/new-env.webp" alt="A dialog box called “New environment.” The Name field is filled out as “UAT,” and the Description field is filled out with “Testing environment.” The Resource field has three radio buttons, with Virtual machines selected. There is a “Next” button at the bottom of the dialog, which is highlighted."></p>
<p>In the following window, select Generic Provider, and then select your OS, which in our case is Windows. You will see a registration script with a copy icon next to it. Click the icon to copy the (rather lengthy) PowerShell registration script to the clipboard, and then paste it into a text file.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/registration-script.webp" alt="The same “New environment” dialog. Under the “Virtual machine resource” section, “Provider” has “Generic provider” selected, “Operating system” has “Windows” selected, and under “Registration script” is a copy icon with instructions to run it in PowerShell."></p>
<p>Next, connect to the target environment. Open a PowerShell window as Administrator, copy and paste the registration script, and then press Enter. The PowerShell script will then do its magic and register the environment with Azure. It may take a minute to run, but afterwards you should see a success message.</p>
<p>Now, if you head back to Azure DevOps and click on the Environment, you should see the server name of the machine that you ran the PowerShell script on. The server name is referred to as a Resource in Azure.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/uat-env.webp" alt="The UAT Environment, Resources tab. It shows a server with a redacted name and a latest job ID with a green check mark."></p>
<p>You will then want to complete the above steps for each of your additional target environments, for example, Staging and Production. In our case, Staging and Production are hosted on the same web server, so we only need to create one additional environment (Production) in Azure.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/prod-env.webp" alt="The Production Enviroment, Resources tab."></p>
<h3 id="creating-the-pipelines">Creating the Pipelines</h3>
<p>Now it’s time to create the actual Pipeline. The Pipeline will be responsible for building and deploying your app to the Environments that you created in the previous step. To start, Click on Pipelines in Azure DevOps, and then click the Create Pipeline button.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/create-pipeline.webp" alt="Azure DevOps. A menu in the drawer on the left has Pipelines expanded, with Pipelines selected in its sub-menu. Outside the menu is a button that reads Create Pipeline."></p>
<p>You’ll then be asked where your source code lives. In our case, the source code exists in a repo in Azure DevOps. The nice thing about Azure is that you can also target source code that exists on another provider, for example, GitHub, Bitbucket, or even a self-hosted Git repo (Other Git). Click on your hosting provider and follow the instructions.</p>
<p>Once you’re connected to your source provider, then you can configure the Pipeline. The “Pipeline” is actually just a YAML file within your target repo. You can read all about the Azure YAML syntax <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/?view=azure-pipelines">here</a>. We’ll choose a Starter Pipeline, which opens up a text editor and enables you to create your YAML file.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/configure-your-pipeline.webp" alt="Azure DevOps. A menu in the drawer on the left has Pipelines expanded. On the right a configure tab is selcted. A heading reads “Configure your pipeline,” under which are a couple options. highlighted is “Starter pipeline.""></p>
<p>Delete the sample text that appears in the editor, and then enter the following YAML. This is a pre-baked YAML pipeline that restores and builds a .NET web project, and then publishes the build assets to your three different target environments. In the next section, we will take a deep dive into how the YAML works so that you can edit it to fit your needs.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">trigger</span>:<span style="color:#bbb">
</span><span style="color:#bbb"></span>- uat<span style="color:#bbb">
</span><span style="color:#bbb"></span>- staging<span style="color:#bbb">
</span><span style="color:#bbb"></span>- main<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">pool</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">vmImage</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'windows-latest'</span><span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">variables</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">projectPath</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial/ci_tutorial_web.csproj'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">packageName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_web.zip'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">solution</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'**/*.sln'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildPlatform</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'AnyCPU'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">artifactName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'AzureDrop'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>${{ if eq(variables['Build.SourceBranchName'], 'uat') }}:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">websiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_uat'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildConfiguration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'UAT'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environmentName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'UAT'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetVM</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'UATWEBAPP1'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>${{ if eq(variables['Build.SourceBranchName'], 'staging') }}:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">websiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_staging'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildConfiguration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Staging'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environmentName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Production'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetVM</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'WEBAPP1'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>${{ if eq(variables['Build.SourceBranchName'], 'main') }}:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">websiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_prod'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildConfiguration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Release'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environmentName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Production'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetVM</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'WEBAPP1'</span><span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">stages</span>:<span style="color:#bbb">
</span><span style="color:#bbb"></span>- <span style="color:#b06;font-weight:bold">stage</span>:<span style="color:#bbb"> </span>build<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">jobs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">job</span>:<span style="color:#bbb"> </span>RestoreAndBuild<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">steps</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>NuGetToolInstaller@1<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>NuGetCommand@2<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">restoreSolution</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(solution)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>VSBuild@1<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">solution</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(projectPath)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">msbuildArgs</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">platform</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(buildPlatform)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configuration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(buildConfiguration)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#888">#- task: VSTest@2</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#888"># inputs:</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#888"># platform: '$(buildPlatform)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#888"># configuration: '$(buildConfiguration)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>PublishBuildArtifacts@1<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">PathtoPublish</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(Build.ArtifactStagingDirectory)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ArtifactName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(artifactName)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">publishLocation</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Container'</span><span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span>- <span style="color:#b06;font-weight:bold">stage</span>:<span style="color:#bbb"> </span>Deploy<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">displayName</span>:<span style="color:#bbb"> </span>Deploy to IIS<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">dependsOn</span>:<span style="color:#bbb"> </span>Build<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">jobs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">deployment</span>:<span style="color:#bbb"> </span>DeploytoIIS<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">displayName</span>:<span style="color:#bbb"> </span>Deploy the web application to dev environment<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environment</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>${{ variables.environmentName }}<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resourceName</span>:<span style="color:#bbb"> </span>${{ variables.targetVM }}<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resourceType</span>:<span style="color:#bbb"> </span>VirtualMachine<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">strategy</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">runOnce</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">deploy</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">steps</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>DownloadBuildArtifacts@0<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildType</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'current'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">downloadType</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'specific'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">downloadPath</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(System.ArtifactsDirectory)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>IISWebAppManagementOnMachineGroup@0<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">displayName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Create App Pool and Website'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">WebsiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(websiteName)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">WebsitePhysicalPath</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'%SystemDrive%\inetpub\wwwroot\$(websiteName)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">CreateOrUpdateAppPoolForWebsite</span>:<span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">true</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">AppPoolNameForWebsite</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(websiteName)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#888"># For testing, to confirm target website</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>PowerShell@2<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">displayName</span>:<span style="color:#bbb"> </span>Display target websitename<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetType</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'inline'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">script</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Write-Host "Target Website Name: $(websiteName)"'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">task</span>:<span style="color:#bbb"> </span>IISWebAppDeploymentOnMachineGroup@0<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">displayName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Deploy IIS Website'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">inputs</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">WebSiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(websiteName)'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">Package</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'$(System.ArtifactsDirectory)\$(artifactName)\$(packageName)'</span><span style="color:#bbb">
</span></code></pre></div><h3 id="understanding-the-yaml">Understanding the YAML</h3>
<p>The first section of the YAML, <code>trigger</code>, determines which branches will trigger the pipeline when they receive a push. In our case, we have three branches: <code>uat</code>, <code>staging</code>, and <code>main</code>.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">trigger</span>:<span style="color:#bbb">
</span><span style="color:#bbb"></span>- uat<span style="color:#bbb">
</span><span style="color:#bbb"></span>- staging<span style="color:#bbb">
</span><span style="color:#bbb"></span>- main<span style="color:#bbb">
</span></code></pre></div><p>The <code>variables</code> section defines variables that are used later in the pipeline YAML. An important part of this section is the use of <code>if</code> statements to assign values to certain variables (i.e., <code>environmentName</code>) based on the source branch for the build. <strong>This mechanism allows the pipeline to deploy to several different environments using a single YAML file.</strong></p>
<p>Note that the <code>if</code> statements also switch the build configuration. If you have config transforms, this will be crucial for ensuring that the correct config files are deployed to each environment.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml">${{ if eq(variables['Build.SourceBranchName'], 'uat') }}:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">websiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_uat'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildConfiguration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'UAT'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environmentName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'UAT'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetVM</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'UATWEBAPP1'</span><span style="color:#bbb">
</span><span style="color:#bbb"></span>${{ if eq(variables['Build.SourceBranchName'], 'staging') }}:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">websiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_staging'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildConfiguration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Staging'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environmentName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Production'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetVM</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'WEBAPP1'</span><span style="color:#bbb">
</span><span style="color:#bbb"></span>${{ if eq(variables['Build.SourceBranchName'], 'main') }}:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">websiteName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'ci_tutorial_prod'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">buildConfiguration</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Release'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">environmentName</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'Production'</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetVM</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">'WEBAPP1'</span><span style="color:#bbb">
</span></code></pre></div><p>Next, we get into the actual actions, or stages, of the pipeline. This pipeline has two stages: build and deploy. The build stage runs a NuGet restore and then a VS build, and then publishes the build output/artifacts to an Azure cloud storage location. The last bit is something that took me a while to wrap my head around: <strong>The build does not occur on the target environment via the runner—it actually occurs up in “Azure land”, on a Windows cloud VM.</strong></p>
<p>The VM on which the build occurs can be defined using the <code>pool: vmImage</code> section. In our case, we are running the build on <code>windows-latest</code>, which currently translates to the brand new Windows Server 2022.</p>
<p>The deploy stage then downloads the build artifacts from Azure and deploys them to IIS on the target machine, using the runner that you installed as part of the environment setup above. Note that the variables defined in the <code>if</code> statements come into play here—this is how the pipeline knows which environment/VM to target.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">environment</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>${{ variables.environmentName }}<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resourceName</span>:<span style="color:#bbb"> </span>${{ variables.targetVM }}<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resourceType</span>:<span style="color:#bbb"> </span>VirtualMachine<span style="color:#bbb">
</span></code></pre></div><p>The <code>IISWebAppManagementOnMachineGroup</code> task will automatically create an app pool and website in IIS for the application. This feature can be toggled using the <code>CreateOrUpdateAppPoolForWebsite</code> setting. If a website already exists based on the WebsiteName, Azure will simply deploy to that existing app pool and website.</p>
<h3 id="adding-branch-protection-and-build-validation">Adding Branch Protection and Build Validation</h3>
<p>In order to ensure successful deployments to Production, you will likely want to add in branch protection for your main or master branch, and also consider build validation.</p>
<p>Branch protection allows you to enforce reviews for pull requests into certain branches. Build validation is a feature that pre-compiles your .NET code to ensure that it builds prior to merging a pull request. Both features help to improve code quality and prevent broken deployments.</p>
<p>To add branch protection, go to Repo → Branches → <em>your target branch</em> → <em>right side menu</em>, then select Branch Policies.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/branch-policies.webp" alt="Under the Branches header, the “main” branch has a menu expanded on the far right of its row. Highlighted is “Branch policies.""></p>
<p>To enforce reviews, turn on “Require a minimum number of reviewers”, and adjust the number of reviewers and settings as required.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/minimum-reviewers.webp" alt="A dialog box titled “Branch Policies.” A switch next to “Require a minimum number of reviewers” is turned on."></p>
<p>For build validation, scroll down to the Build Validation section, and click the plus button to add a new policy. Select the target build pipeline, adjust the policy settings as necessary (see below), and then give it a display name. Click save, and the build validation policy will be applied to the branch.</p>
<blockquote>
<p>An important note here is that the build validation essentially runs the CI pipeline. In our case, that means it will run the build and the deployment. This is not ideal; we only want it to run the build (NuGet restore and VS build). So, it’s advisable to create a separate pipeline with only the build stage. This can be done easily by copying our example YAML and removing the “Deploy” stage, then saving it as a new pipeline. Then, choose that new pipeline as the target build pipeline in the validation policy.</p>
</blockquote>
<p><img src="/blog/2022/08/cicd-with-azure-devops/add-build-policy.webp" alt="A header reads “Add build policy.” under that is a form, with “Build pipeline (required)” highlighted. Highlight text reads “Add your new ‘build-only’ pipeline here.” “ci-tutorial” is selected."></p>
<p>Now when you open up a PR, it can only be merged once a review has been applied, and the build validation has run successfully.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/pr-view.webp" alt="Pull request view, “Overview” tab. Next to green check marks are messages saying “Required check succeeded” with the successful build validation under it, and “1 reviewer approved.""></p>
<h3 id="monitoring-and-troubleshooting-the-pipeline">Monitoring and Troubleshooting the Pipeline</h3>
<p>You can see the overall status of your pipeline by clicking on the Pipelines section in the left-side navigation. Green means good!</p>
<p>To dive into more details, click on the pipeline name. This will show all the recent pipeline runs. Then click on a particular run. This will bring up the run details, including a flowchart for each stage.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/pipeline-flowchart.webp" alt="Pipeline flowchart, with a completed “build” leading into a completed “Deploy to IIS.""></p>
<p>To dive further, click on a specific stage. You’ll then see all of the tasks in the stage, with actual output details on the right, similar to what you would see in the output window in Visual Studio.</p>
<p><img src="/blog/2022/08/cicd-with-azure-devops/stage-details.webp" alt="Stage details. On the left are the successful completed jobs from the “build” and “Deploy to IIS” sections. On the right is more verbose output for a selected job."></p>
<p><strong>If you encounter an error in your pipeline, the stage details page is a great place to troubleshoot.</strong> Azure will show you the task that generated the error by displaying a red error icon next to the task. You can then click on the task and trace the output on the right to determine the exact error message.</p>
<h3 id="wrapping-up">Wrapping Up</h3>
<p>Hopefully, this post gives you a good overview of CI/CD options for a self-hosted .NET application. Azure is almost infinitely configurable, so there are many other options to explore that are not covered in this post, whether it be within the pipeline YAML, or through other settings like branch policies/protection. If you have any questions, please feel free to leave a comment. Happy automating!</p>
Implementing Backend Tasks in ASP.NET Corehttps://www.endpointdev.com/blog/2022/08/implementing-backend-tasks-in-asp.net-core/2022-08-08T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2022/08/implementing-backend-tasks-in-asp.net-core/mountain-path.webp" alt=""></p>
<!-- Photo by Seth Jensen -->
<p>As we’ve <a href="https://www.endpointdev.com/blog/2022/01/database-integration-testing-with-dotnet/">already established</a>, <a href="https://rubyonrails.org/">Ruby on Rails</a> is great. The amount and quality of tools that Rails puts at our disposal when it comes to developing web applications is truly outstanding. One aspect of web application development that Rails makes particularly easy is that of creating backend tasks.</p>
<p>These tasks can be anything from database maintenance, file system cleanup, overnight heavy computations, bulk email dispatch, etc. In general, functionality that is typically initiated by a sysadmin in the backend, or scheduled in a <a href="https://en.wikipedia.org/wiki/Cron">cron</a> job, which has no GUI, but rather, is invoked via command line.</p>
<p>By integrating with <a href="https://github.com/ruby/rake">Rake</a>, Rails allows us to <a href="https://guides.rubyonrails.org/command_line.html#custom-rake-tasks">very easily write such tasks</a> as plain old <a href="https://ruby-doc.org/">Ruby</a> scrips. These scripts have access to all the domain logic and data that the full-fledged Rails app has access to. The cherry on top is that the command-line interface to invoke such tasks is very straightforward. It looks something like this: <code>bin/rails fulfillment:process_new_orders</code>.</p>
<p>All this is included right out of the box for new Rails projects.</p>
<p><a href="https://dotnet.microsoft.com/en-us/apps/aspnet">ASP.NET Core</a>, which is also great, doesn’t support this out of the box like Rails does.</p>
<p>However, I think we should be able to implement our own without too much hassle, and have a similar sysadmin experience. Let’s see if we can do it.</p>
<blockquote>
<p>There is a <a href="#table-of-contents">Table of contents</a> at the end of this post.</p>
</blockquote>
<h3 id="what-we-want-to-accomplish">What we want to accomplish</h3>
<p>So, to put it in concrete terms, we want to create a backend task that has access to all the logic and data of an existing ASP.NET Core application. The task should be callable via command-line interface, so that it can be easily executed via the likes of cron or other scripts.</p>
<p>In order to meet these requirements, we will create a new .NET console app that:</p>
<ol>
<li>References the existing ASP.NET Core project.</li>
<li>Loads all the classes from it and makes instances of them available via <a href="https://en.wikipedia.org/wiki/Dependency_injection">dependency injection</a>.</li>
<li>Has a usable, Unix-like command-line interface that sysadmins would be familiar with.</li>
<li>Is invokable via the <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/">.NET CLI</a>.</li>
</ol>
<p>We will do all this within the context of an existing web application. One that I’ve been <a href="https://www.endpointdev.com/blog/2021/07/dotnet-5-web-api/">building upon</a> <a href="https://www.endpointdev.com/blog/2022/01/database-integration-testing-with-dotnet/">thoughout a few</a> <a href="https://www.endpointdev.com/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/">articles</a>.</p>
<p>It is a simple ASP.NET Web API backed by a <a href="https://www.postgresql.org/">Postgres</a> database. It has a few endpoints for <a href="https://en.wikipedia.org/wiki/Create,_read,_update_and_delete">CRUD</a>ing automotive related data and for calculating values of vehicles based on various aspects of them.</p>
<p>You can find the code <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">on GitHub</a>. If you’d like to follow along, clone the repository and check out this commit: <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/tree/9a078015cec2e8a42a4898e203a1a50db69731e8">9a078015ce</a>. It represents the project as it was before applying all the changes from this article. The finished product can be found <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/tree/backend-task">here</a>.</p>
<p>You can follow the instructions in <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/json-web-token/VehicleQuotes/README.md">the project’s README file</a> if you want to get the app up and running.</p>
<p>For our demo use case, we will try to develop a backend task that creates new user accounts for our existing application.</p>
<p>Let’s get to it.</p>
<h3 id="creating-a-new-console-app-that-references-the-existing-web-app-as-a-library">Creating a new console app that references the existing web app as a library</h3>
<p>The codebase is structured as a <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-sln">solution</a>, as given away by the <code>vehicle-quotes.sln</code> file located at the root of the repository. Within this solution, there are two projects: <code>VehicleQuotes</code> which is the web app itself, and <code>VehicleQuotes.Tests</code> which contains the app’s test suite. For this article, we only care about the web app.</p>
<p>Like I said, the backend task that we will create is nothing fancy in itself. It’s a humble console app. So, we start by asking the <code>dotnet</code> CLI to create a new console app project for us.</p>
<p>From the repository’s root directory, we can do so with this command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet new console -o VehicleQuotes.CreateUser
</code></pre></div><p>That should’ve resulted in a new <code>VehicleQuotes.CreateUser</code> directory being created, and within it, (along with some other nuts and bolts) our new console app’s <code>Program.cs</code> (the code) and <code>VehicleQuotes.CreateUser.csproj</code> (the project definition) files. The name that we’ve chosen is straightforward: the name of the overall solution and the action that this console app is going to perform.</p>
<blockquote>
<p>There’s more info regarding the <code>dotnet new</code> command in <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-new">the official docs</a>.</p>
</blockquote>
<p>Now, since we’re using a solution file, let’s add our brand new console app project to it with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet sln add VehicleQuotes.CreateUser
</code></pre></div><p>OK, cool. That should’ve produced the following diff on <code>vehicle-quotes.sln</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#333">diff --git a/vehicle-quotes.sln b/vehicle-quotes.sln
</span><span style="color:#333">index 537d864..5da277d 100644
</span><span style="color:#333"></span><span style="color:#000;background-color:#fdd">--- a/vehicle-quotes.sln
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+++ b/vehicle-quotes.sln
</span><span style="color:#000;background-color:#dfd"></span><span style="color:#666">@@ -7,6 +7,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VehicleQuotes", "VehicleQuo
</span><span style="color:#666"></span> EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VehicleQuotes.Tests", "VehicleQuotes.Tests\VehicleQuotes.Tests.csproj", "{5F6470E4-12AB-4E30-8879-3664ABAA959D}"
EndProject
<span style="color:#000;background-color:#dfd">+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "VehicleQuotes.CreateUser", "VehicleQuotes.CreateUser\VehicleQuotes.CreateUser.csproj", "{EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}"
</span><span style="color:#000;background-color:#dfd">+EndProject
</span><span style="color:#000;background-color:#dfd"></span> Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
<span style="color:#666">@@ -44,5 +46,17 @@ Global
</span><span style="color:#666"></span> {5F6470E4-12AB-4E30-8879-3664ABAA959D}.Release|x64.Build.0 = Release|Any CPU
{5F6470E4-12AB-4E30-8879-3664ABAA959D}.Release|x86.ActiveCfg = Release|Any CPU
{5F6470E4-12AB-4E30-8879-3664ABAA959D}.Release|x86.Build.0 = Release|Any CPU
<span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|Any CPU.Build.0 = Debug|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x64.ActiveCfg = Debug|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x64.Build.0 = Debug|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x86.ActiveCfg = Debug|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Debug|x86.Build.0 = Debug|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|Any CPU.ActiveCfg = Release|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|Any CPU.Build.0 = Release|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x64.ActiveCfg = Release|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x64.Build.0 = Release|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x86.ActiveCfg = Release|Any CPU
</span><span style="color:#000;background-color:#dfd">+ {EDBB33E3-DCCE-4957-8A69-DC905D1BEAA4}.Release|x86.Build.0 = Release|Any CPU
</span><span style="color:#000;background-color:#dfd"></span> EndGlobalSection
EndGlobal
</code></pre></div><p>This allows the .NET tooling to know that we’ve got some intentional organization going on in our code base. That these projects each form part of a bigger whole.</p>
<blockquote>
<p>It’s also nice to add a <a href="https://git-scm.com/docs/gitignore"><code>.gitignore</code></a> file for our new <code>VehicleQuotes.CreateUser</code> project to keep things manageable. <code>dotnet new</code> can help with that if we were to navigate into the <code>VehicleQuotes.CreateUser</code> directory and run:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet new gitignore
</code></pre></div></blockquote>
<blockquote>
<p>You can learn more about how to work with solutions via the .NET CLI in <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-sln">the official docs</a>.</p>
</blockquote>
<p>Now let’s modify our new project’s <code>.csproj</code> file so that it references the main web app project under <code>VehicleQuotes</code>. This will allow our console app to access all of the classes defined in the web app, as if it was a library or package.</p>
<p>If we move to the <code>VehicleQuotes.CreateUser</code> directory, we can do that with the following command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet add reference ../VehicleQuotes/VehicleQuotes.csproj
</code></pre></div><p>The command itself is pretty self-explanatory. It just expects to be given the <code>.csproj</code> file of the project that we want to add as a reference in order to do its magic.</p>
<p>Running that should’ve added the following snippet to <code>VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#b06;font-weight:bold"><ItemGroup></span>
<span style="color:#b06;font-weight:bold"><ProjectReference</span> <span style="color:#369">Include=</span><span style="color:#d20;background-color:#fff0f0">"..\VehicleQuotes\VehicleQuotes.csproj"</span> <span style="color:#b06;font-weight:bold">/></span>
<span style="color:#b06;font-weight:bold"></ItemGroup></span>
</code></pre></div><p>This way, .NET allows the code defined in the <code>VehicleQuotes</code> project to be used within the <code>VehicleQuotes.CreateUser</code> project.</p>
<blockquote>
<p>You can learn more about the add reference command in <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-add-reference">the official docs</a>.</p>
</blockquote>
<h3 id="setting-up-dependency-injection-in-the-console-app">Setting up dependency injection in the console app</h3>
<p>As a result of the previous steps, our new console app now has access to the classes defined within the web app. However, classes by themselves are no good if we can’t actually create instances of them that we can interact with. The premier method for making instances of classes available throughout a .NET application is via <a href="https://docs.microsoft.com/en-us/dotnet/core/extensions/dependency-injection">dependency injection</a>. So, we need to set that up for our little console app.</p>
<p>Dependency injection is something that comes out of the box for ASP.NET Core web apps. Luckily for us, .NET makes it fairly easy to set it up in console apps as well by leveraging the same components.</p>
<p>For this app, we want to create user accounts. In the web app, user account management is done via <a href="https://docs.microsoft.com/en-us/aspnet/core/security/authentication/identity?view=aspnetcore-6.0&tabs=netcore-cli">ASP.NET Core Identity</a>. Specifically, the <code>UserManager</code> class is used to create new user accounts. This console app will do the same.</p>
<blockquote>
<p>Take a look at <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/backend-task/VehicleQuotes/Controllers/UsersController.cs"><code>VehicleQuotes/Controllers/UsersController.cs</code></a> to see how the user accounts are created. If you’d like to know more about integrating ASP.NET Core Identity into an existing web app, I wrote <a href="https://www.endpointdev.com/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/">an article</a> about it.</p>
</blockquote>
<p>Before we do the dependency injection setup, let’s add a new class to our console app project that will encapsulate the logic of leveraging the <code>UserManager</code> for user account creation. This is the actual task that we want to perform. The new class will be defined in <code>VehicleQuotes.CreateUser/UserCreator.cs</code> and these will be its contents:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.CreateUser</span>;
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">UserCreator</span>
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> UserManager<IdentityUser> <span style="color:#00d;font-weight:bold">_</span>userManager;
<span style="color:#080;font-weight:bold">public</span> UserCreator(UserManager<IdentityUser> userManager) {
<span style="color:#00d;font-weight:bold">_</span>userManager = userManager;
}
<span style="color:#080;font-weight:bold">public</span> IdentityResult Run(<span style="color:#888;font-weight:bold">string</span> username, <span style="color:#888;font-weight:bold">string</span> email, <span style="color:#888;font-weight:bold">string</span> password)
{
<span style="color:#888;font-weight:bold">var</span> userCreateTask = <span style="color:#00d;font-weight:bold">_</span>userManager.CreateAsync(
<span style="color:#080;font-weight:bold">new</span> IdentityUser() { UserName = username, Email = email },
password
);
<span style="color:#888;font-weight:bold">var</span> result = userCreateTask.Result;
<span style="color:#080;font-weight:bold">return</span> result;
}
}
</code></pre></div><p>This class is pretty lean. All it does is define a constructor that expects an instance of <code>UserManager<IdentityUser></code>, which will be supplied via dependency injection; and a simple <code>Run</code> method that, when given a username, email, and password, asks the <code>UserManager<IdentityUser></code> instance that it was given to create a user account.</p>
<p>Moving on to setting up dependency injection now, we will do it in <code>VehicleQuotes.CreateUser/Program.cs</code>. Replace the contents of that file with this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.DependencyInjection</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Hosting</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.CreateUser</span>;
IHost host = Host.CreateDefaultBuilder(args)
.UseContentRoot(System.AppContext.BaseDirectory)
.ConfigureServices((context, services) =>
{
<span style="color:#888;font-weight:bold">var</span> startup = <span style="color:#080;font-weight:bold">new</span> VehicleQuotes.Startup(context.Configuration);
startup.ConfigureServices(services);
services.AddTransient<UserCreator>();
})
.Build();
<span style="color:#888;font-weight:bold">var</span> userCreator = host.Services.GetRequiredService<UserCreator>();
userCreator.Run(args[<span style="color:#00d;font-weight:bold">0</span>], args[<span style="color:#00d;font-weight:bold">1</span>], args[<span style="color:#00d;font-weight:bold">2</span>]);
</code></pre></div><p>Let’s dissect this bit by bit.</p>
<p>First off, we’ve got a few <code>using</code> statements that we need in order to access some classes and extension methods that we need down below.</p>
<p>Next, we create and configure a new <code>IHost</code> instance. .NET Core introduced the concept of a “host” as an abstraction for programs; and packed in there a lot of functionality to help with things like configuration, logging and, most importantly for us, dependency injection. To put it simply, the simplest way of enabling dependency injection in a console app is to use a <code>Host</code> and all the goodies that come within.</p>
<blockquote>
<p>There’s much more information about hosts in <a href="https://docs.microsoft.com/en-us/dotnet/core/extensions/generic-host">.NET’s official documentation</a>.</p>
</blockquote>
<p><code>Host.CreateDefaultBuilder(args)</code> gives us an <code>IHostBuilder</code> instance that we can use to configure our host. In our case, we’ve chosen to call <code>UseContentRoot(System.AppContext.BaseDirectory)</code> on it, which makes it possible for the app to find assets (like <code>appconfig.json</code> files!) regardless of where its deployed and where its being called from.</p>
<p>This is important for us because, as you will see later, we will install this console app as a <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools">.NET Tool</a>. .NET Tools are installed in directories picked by .NET and can be run from anywhere in the system. So we need to make sure that our app can find its assets wherever it has been installed.</p>
<p>After that, we call <code>ConfigureServices</code> where we do a nice trick in order to make sure our console app has all the same configuration as the web app as far as dependency injection goes.</p>
<p>You see, in ASP.NET Core, all the service classes that are to be made available to the application via dependency injection are configured within the web app’s <code>Startup</code> class' <code>ConfigureServices</code> method. <code>VehicleQuotes</code> is no exception. So, in order for our console app to have access to all of the services (i.e. instances of classes) that the web app does, the console app needs to call that same code. And that’s exactly what’s happening in these two lines:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">var</span> startup = <span style="color:#080;font-weight:bold">new</span> VehicleQuotes.Startup(context.Configuration);
startup.ConfigureServices(services);
</code></pre></div><p>We create a new instance of the web app’s <code>Startup</code> class and call its <code>ConfigureServices</code> method. That’s the key element that allows the console app to have access to all the logic that the web app does. Including the services/classes provided by ASP.NET Core Identity like <code>UserManager<IdentityUser></code>, which <code>UserCreator</code> needs in order to function.</p>
<p>Once that’s done, the rest is straightforward.</p>
<p>We also add our new <code>UserCreator</code> to the dependency injection engine via:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">services.AddTransient<UserCreator>();
</code></pre></div><blockquote>
<p>Curious about what <code>Transient</code> means? <a href="https://docs.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#service-lifetimes">The official .NET documentation</a> has the answer.</p>
</blockquote>
<p>And that allows us to obtain an instance of it with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">var</span> userCreator = host.Services.GetRequiredService<UserCreator>();
</code></pre></div><p>And then, it’s just a matter of calling its <code>Run</code> method like so, passing it the command-line arguments:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">userCreator.Run(args[<span style="color:#00d;font-weight:bold">0</span>], args[<span style="color:#00d;font-weight:bold">1</span>], args[<span style="color:#00d;font-weight:bold">2</span>]);
</code></pre></div><p><code>args</code> is a special variable that contains an array with the arguments given by command line. That means that our console app can be called like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet run test_username test_email@email.com mysecretpassword
</code></pre></div><p>Go ahead, you can try it out and see the app log what it’s doing. Once done, it will also have created a new record in the database.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ psql -h localhost -U vehicle_quotes
psql (14.3 (Ubuntu 14.3-0ubuntu0.22.04.1), server 14.2 (Debian 14.2-1.pgdg110+1))
Type "help" for help.
vehicle_quotes=# select user_name, email from public."AspNetUsers";
user_name | email
---------------+----------------------------
test_username | test_email@email.com
(1 rows)
</code></pre></div><p>Pretty neat, huh? At this point we have a console app that creates user accounts for our existing web app. It works, but it could be better. Let’s add a nice command-line interface experience now.</p>
<h3 id="improving-the-cli-with-commandlineparser">Improving the CLI with CommandLineParser</h3>
<p>With help from <a href="https://github.com/commandlineparser/commandline">CommandLineParser</a>, we can develop a Unix-like command-line interface for our app. We can use it to add help text, examples, have strongly typed parameters and useful error messages when said parameters are not correctly provided. Let’s do that now.</p>
<p>First, we need to install the package in our console app project by running the following command from within the project’s directory (<code>VehicleQuotes.CreateUser</code>):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet add package CommandLineParser
</code></pre></div><p>After that’s done, a new section will have been added to <code>VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj</code> that looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#b06;font-weight:bold"><ItemGroup></span>
<span style="color:#b06;font-weight:bold"><PackageReference</span> <span style="color:#369">Include=</span><span style="color:#d20;background-color:#fff0f0">"CommandLineParser"</span> <span style="color:#369">Version=</span><span style="color:#d20;background-color:#fff0f0">"2.9.1"</span> <span style="color:#b06;font-weight:bold">/></span>
<span style="color:#b06;font-weight:bold"></ItemGroup></span>
</code></pre></div><p>Now our console app can use the classes provided by the package.</p>
<p>All specifications for <code>CommandLineParser</code> are done via a plain old C# class that we need to define. For this console app, which accepts three mandatory arguments, such a class could look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">CommandLine</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">CommandLine.Text</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.CreateUser</span>;
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">CliOptions</span>
{
<span style="color:#369"> [Value(0, Required = true, MetaName = "username", HelpText = "The username of the new user account to create.")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Username { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [Value(1, Required = true, MetaName = "email", HelpText = "The email of the new user account to create.")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Email { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [Value(2, Required = true, MetaName = "password", HelpText = "The password of the new user account to create.")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Password { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [Usage(ApplicationAlias = "create_user")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> IEnumerable<Example> Examples
{
<span style="color:#080;font-weight:bold">get</span>
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> List<Example> {
<span style="color:#080;font-weight:bold">new</span> (
<span style="color:#d20;background-color:#fff0f0">"Create a new user account"</span>,
<span style="color:#080;font-weight:bold">new</span> CliOptions { Username = <span style="color:#d20;background-color:#fff0f0">"name"</span>, Email = <span style="color:#d20;background-color:#fff0f0">"email@domain.com"</span>, Password = <span style="color:#d20;background-color:#fff0f0">"secret"</span> }
)
};
}
}
}
</code></pre></div><p>I’ve decided to name it <code>CliOptions</code> but really, it could have been anything. Go ahead and create it in <code>VehicleQuotes.CreateUser/CliOptions.cs</code>. There are a few interesting elements to note here.</p>
<p>The key aspect is that we have a few properties: <code>Username</code>, <code>Email</code>, and <code>Password</code>. These represent our three command-line arguments. Thanks to the <code>Value</code> <a href="https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/attributes/">attributes</a> that they have been annotated with, <code>CommandLineParser</code> will know that that’s their purpose. You can see how the attributes themselves also contain each argument’s specification like the order in which they should be supplied, as well as their name and help text.</p>
<p>This class also defines an <code>Examples</code> getter which is used by <code>CommandLineParser</code> to print out usage examples into the console when our app’s help option is invoked.</p>
<p>Other than that, the class itself is unremarkable. In summary, it’s a number of fields annotated with attributes so that <code>CommandLineParser</code> knows what to do with it.</p>
<p>In order to actually put it to work, we update our <code>VehicleQuotes.CreateUser/Program.cs</code> like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
<span style="color:#000;background-color:#dfd">+using CommandLine;
</span><span style="color:#000;background-color:#dfd"></span> using VehicleQuotes.CreateUser;
<span style="color:#000;background-color:#dfd">+void Run(CliOptions options)
</span><span style="color:#000;background-color:#dfd">+{
</span><span style="color:#000;background-color:#dfd"></span> IHost host = Host.CreateDefaultBuilder(args)
.UseContentRoot(System.AppContext.BaseDirectory)
.ConfigureServices((context, services) =>
{
var startup = new VehicleQuotes.Startup(context.Configuration);
startup.ConfigureServices(services);
services.AddTransient<UserCreator>();
})
.Build();
var userCreator = host.Services.GetRequiredService<UserCreator>();
<span style="color:#000;background-color:#fdd">- userCreator.Run(args[0], args[1], args[2]);
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ userCreator.Run(options.Username, options.Email, options.Password);
</span><span style="color:#000;background-color:#dfd">+}
</span><span style="color:#000;background-color:#dfd">+
</span><span style="color:#000;background-color:#dfd">+Parser.Default
</span><span style="color:#000;background-color:#dfd">+ .ParseArguments<CliOptions>(args)
</span><span style="color:#000;background-color:#dfd">+ .WithParsed(options => Run(options));
</span></code></pre></div><p>We’ve wrapped <code>Program.cs</code>’s original code into a method simply called <code>Run</code>.</p>
<p>Also, we’ve added this snippet at the bottom of the file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">Parser.Default
.ParseArguments<CliOptions>(args)
.WithParsed(options => Run(options));
</code></pre></div><p>That’s how we ask <code>CommandLineParser</code> to parse the incoming CLI arguments, as specified by <code>CliOptions</code> and, if it can be done successfully, then execute the rest of the program by calling the <code>Run</code> method.</p>
<p>Neatly, we also no longer have to use the <code>args</code> array directly in order to get the command-line arguments provided to the app, instead we use the <code>options</code> object that <code>CommandLineParser</code> creates for us once it has done its parsing. You can see it in this line:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">userCreator.Run(options.Username, options.Email, options.Password);
</code></pre></div><p><code>options</code> is an instance of our very own <code>CliOptions</code> class, so we can access the properties that we defined within it. These contain the arguments that were passed to the program.</p>
<p>If you were to try <code>dotnet run</code> right now, you’d see the following output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">VehicleQuotes.CreateUser 1.0.0
Copyright (C) 2022 VehicleQuotes.CreateUser
ERROR(S):
A required value not bound to option name is missing.
USAGE:
Create a new user account:
create_user name email@domain.com secret
--help Display this help screen.
--version Display version information.
username (pos. 0) Required. The username of the new user account to create.
email (pos. 1) Required. The email of the new user account to create.
password (pos. 2) Required. The password of the new user account to create.
</code></pre></div><p>As you can see, <code>CommandLineParser</code> detected that no arguments were given, and as such, it printed out an error message, along with the descriptions, help text and example that we defined. Basically the instructions on how to use our console app.</p>
<h3 id="deploying-the-console-app-as-a-net-tool">Deploying the console app as a .NET tool</h3>
<p>OK, we now have a console app that does what it needs to do, with a decent interface. The final step is to make it even more accessible by deploying it as a .NET tool. If we do that, we’d be able to invoke it with a command like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet create_user Kevin kevin@gmail.com secretpw
</code></pre></div><p>.NET makes this easy for us. There’s a caveat that we’ll discuss later, but for now, let’s go through the basic setup.</p>
<p>.NET tools are essentially just glorified NuGet packages. As such, we begin by adding some additional package-related configuration options to <code>VehicleQuotes.CreateUser/VehicleQuotes.CreateUser.csproj</code>. We add them as children elements to the <code><PropertyGroup></code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#b06;font-weight:bold"><PackAsTool></span>true<span style="color:#b06;font-weight:bold"></PackAsTool></span>
<span style="color:#b06;font-weight:bold"><PackageOutputPath></span>./nupkg<span style="color:#b06;font-weight:bold"></PackageOutputPath></span>
<span style="color:#b06;font-weight:bold"><ToolCommandName></span>create_user<span style="color:#b06;font-weight:bold"></ToolCommandName></span>
<span style="color:#b06;font-weight:bold"><VersionPrefix></span>1.0.0<span style="color:#b06;font-weight:bold"></VersionPrefix></span>
</code></pre></div><p>With that, we signal .NET that we want the console app to be packed as a tool, the path where it should put the package itself, and what its name will be. That is, how will it be invoked via the console (remember we want to be able to do <code>dotnet create_user</code>).</p>
<p>Finally, we specify a version number. When dealing with NuGet packages, versioning them is very important, as that drives caching and downloading logic in NuGet. More on that later when we talk about the aforementioned caveats.</p>
<p>Now, to build the package, we use:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet pack
</code></pre></div><p>That will build the application and produce a <code>VehicleQuotes.CreateUser/nupkg/VehicleQuotes.CreateUser.1.0.0.nupkg</code> file.</p>
<p>We won’t make the tool available for the entire system. Instead, we will make it available from within our solution’s directory only. We can make that happen if we create a tool manifest file in the source code’s root directory. That’s done with this command, run from the root directory:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet new tool-manifest
</code></pre></div><p>That should create a new file: <code>.config/dotnet-tools.json</code>.</p>
<p>Now, also from the root directory, we can finally install our tool:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet tool install --add-source ./VehicleQuotes.CreateUser/nupkg VehicleQuotes.CreateUser
</code></pre></div><p>This is the regular command to install any tools in .NET. The interesting part is that we use the <code>--add-source</code> option to point it to the path where our freshly built package is located.</p>
<p>After that, .NET shows this output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ dotnet tool install --add-source ./VehicleQuotes.CreateUser/nupkg VehicleQuotes.CreateUser
You can invoke the tool from this directory using the following commands: 'dotnet tool run create_user' or 'dotnet create_user'.
Tool 'vehiclequotes.createuser' (version '1.0.0') was successfully installed. Entry is added to the manifest file /path/to/solution/.config/dotnet-tools.json.
</code></pre></div><p>It tells us all we need to know. Check out the <code>.config/dotnet-tools.json</code> to see how the tool has been added there. All this means that now we can run our console app as a .NET tool:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ dotnet create_user --help
VehicleQuotes.CreateUser 1.0.0
Copyright (C) 2022 VehicleQuotes.CreateUser
USAGE:
Create a new user account:
create_user name email@domain.com secret
--help Display this help screen.
--version Display version information.
username (pos. 0) Required. The username of the new user account to create.
email (pos. 1) Required. The email of the new user account to create.
password (pos. 2) Required. The password of the new user account to create.
</code></pre></div><p>Pretty sweet, huh? And yes, it has taken a lot more effort than what it would’ve taken in Ruby on Rails, but hey, the end result is pretty fabulous I think, and we learned a new thing. Besides, once you’ve done it once, the skeleton can be easily reused for all kinds of different backend tasks.</p>
<p>Now, before we wrap this up, there’s something we need to consider when actively developing these tools. That is, when making changes and re-installing constantly.</p>
<p>The main aspect to understand is that tools are just NuGet packages, and as such are beholden to the NuGet package infrastructure. Which includes caching. If you’re in the process of developing your tool and are quickly making and deploying changes, NuGet won’t update the cache unless you do one of two things:</p>
<ol>
<li>Manually clear it with a command like <code>dotnet nuget locals all --clear</code>.</li>
<li>Bump up the version of the tool by updating the value of <code><VersionSuffix></code> in the project (<code>.csproj</code>) file.</li>
</ol>
<p>This means that, unless you do one these, the changes that you make to the app between re-builds (with <code>dotnet pack</code>) and re-installs (with <code>dotnet dotnet tool install</code>) won’t ever make their way to the package that’s actually installed in your system. So be sure to keep that in mind.</p>
<h3 id="table-of-contents">Table of contents</h3>
<ul>
<li><a href="#main-content">Implementing Backend Tasks in ASP.NET Core</a></li>
<li><a href="#what-we-want-to-accomplish">What we want to accomplish</a></li>
<li><a href="#creating-a-new-console-app-that-references-the-existing-web-app-as-a-library">Creating a new console app that references the existing web app as a library</a></li>
<li><a href="#setting-up-dependency-injection-in-the-console-app">Setting up dependency injection in the console app</a></li>
<li><a href="#improving-the-cli-with-commandlineparser">Improving the CLI with CommandLineParser</a></li>
<li><a href="#deploying-the-console-app-as-a-net-tool">Deploying the console app as a .NET tool</a></li>
</ul>
Implementing Authentication in ASP.NET Core Web APIshttps://www.endpointdev.com/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/2022-06-17T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/stockholm-buildings.webp" alt="Several buildings with a lightly cloudy blue sky"></p>
<!-- Photo by Seth Jensen -->
<p>Authentication is a complex space. There are many problem scenarios and many more solutions. When it comes to <a href="https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api?view=aspnetcore-6.0&tabs=visual-studio-code">Web APIs</a> written with <a href="https://github.com/dotnet/aspnetcore">ASP.NET Core</a>, there are various fully featured options like <a href="https://duendesoftware.com/">Duende IdentityServer</a> or <a href="https://docs.microsoft.com/en-us/azure/api-management/api-management-howto-protect-backend-with-aad">Azure Active Directory</a>. These promise to be “everything but the kitchen sink” solutions which are robust and allow you to deal with many complex requirements.</p>
<p>But what if our requirements dictate that we need something simpler? Do we have to roll out our own from scratch? Or does ASP.NET Core offer smaller, customizable, somewhat independent puzzle pieces that we can put together without having to write all the code ourselves and still have a good amount of control?</p>
<p>Spoiler alert: The answer to that last question is yes. And we’re going to talk about it in this very article.</p>
<blockquote>
<p>There is a <a href="#table-of-contents">Table of contents</a> at the end of this post.</p>
</blockquote>
<h3 id="two-approaches-to-authentication-jwt-and-api-keys">Two approaches to authentication: JWT and API Keys</h3>
<p>In this article, we’ll take an existing ASP.NET Core Web API and add authentication capabilities to it. Specifically, we’ll support two authentication schemes commonly used for Web APIs: JWT and API Keys. Also, we will use our own database for storage of user accounts and credentials.</p>
<p>The project that we will work with is a simple ASP.NET Web API backed by a <a href="https://www.postgresql.org/">Postgres</a> database. It has a few endpoints for <a href="https://en.wikipedia.org/wiki/Create,_read,_update_and_delete">CRUD</a>ing automotive related data and for calculating values of vehicles based on various aspects of them. You can read all about the process of building it <a href="https://www.endpointdev.com/blog/2021/07/dotnet-5-web-api/">here</a>. I also added a few database integration tests for it <a href="https://www.endpointdev.com/blog/2022/01/database-integration-testing-with-dotnet/">here</a>.</p>
<p>You can find it <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">on GitHub</a>. If you’d like to follow along, clone the repository and checkout this commit: <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/tree/cd290c765fcd2c6693008d3dc76fa931098dcaa0">cd290c765fcd2c6693008d3dc76fa931098dcaa0</a>. It represents the project as it was before applying all the changes from this article.</p>
<p>You can follow the instructions in <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/json-web-token/VehicleQuotes/README.md">the project’s README file</a> in order to get the app up and running.</p>
<h3 id="managing-user-accounts-with-aspnet-core-identity">Managing user accounts with ASP.NET Core Identity</h3>
<p>Let’s deal first with the requirement of storing the user accounts in our own database.</p>
<p>Luckily for us, ASP.NET Core provides us with <a href="https://docs.microsoft.com/en-us/aspnet/core/security/authentication/identity?view=aspnetcore-6.0&tabs=netcore-cli">Identity</a>. This is an API that offers a comprehensive solution for authentication. It can connect with the aforementioned Duende
IdentityServer for <a href="https://openid.net/connect/">OpenID Connect</a> and <a href="https://oauth.net/2/">OAuth 2.0</a>, supports authentication via third parties like Facebook or Google, and can fully integrate with user interfaces on web apps, complete with scaffolding/autogeneration capabilities too.</p>
<p>Most importantly for us, it supports management of user accounts stored in our own database. It includes data for third party logins, passwords, roles for authorization, email confirmation, access tokens, etc.</p>
<p>That’s a lot of of functionality baked in there. But we don’t need all that, so let’s see how we can take only the bits and pieces that we need in order to fulfill our specific requirements.</p>
<p>Specifically, we want:</p>
<ol>
<li>Tables in our database for storage of user accounts and passwords.</li>
<li>A programmatic way to create, fetch and validate users using those tables.</li>
</ol>
<p>It’s actually pretty simple.</p>
<h4 id="install-the-necessary-nuget-packages">Install the necessary NuGet packages</h4>
<p>First, we need to install a couple of <a href="https://www.nuget.org/">NuGet</a> packages:</p>
<ul>
<li><a href="https://www.nuget.org/packages/Microsoft.AspNetCore.Identity/">Microsoft.AspNetCore.Identity</a> which contains the core library.</li>
<li><a href="https://www.nuget.org/packages/Microsoft.AspNetCore.Identity.EntityFrameworkCore">Microsoft.AspNetCore.Identity.EntityFrameworkCore</a> which includes the classes that Identity needs in order to properly interact with <a href="https://docs.microsoft.com/en-us/ef/">Entity Framework</a>, which is what we’re using in our Web API for interfacing with the database.</li>
</ul>
<p>We can install them by running these two commands from the <code>VehicleQuotes</code> directory:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ dotnet add package Microsoft.AspNetCore.Identity
$ dotnet add package Microsoft.AspNetCore.Identity.EntityFrameworkCore
</code></pre></div><p>That will add the following lines to the <code>VehicleQuotes/VehicleQuotes.csproj</code> project file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> <Project Sdk="Microsoft.NET.Sdk.Web">
...
<ItemGroup>
...
<span style="color:#000;background-color:#dfd">+ <PackageReference Include="Microsoft.AspNetCore.Identity" Version="2.2.0" />
</span><span style="color:#000;background-color:#dfd">+ <PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="6.0.5" />
</span><span style="color:#000;background-color:#dfd"></span> ...
</ItemGroup>
</Project>
</code></pre></div><h4 id="update-the-dbcontext-to-include-the-identity-tables">Update the DbContext to include the Identity tables</h4>
<p>Next step is to configure our <code>DbContext</code> class so that it includes the new tables that we need from the Identity library. So let’s go to <code>VehicleQuotes/Data/VehicleQuotesContext.cs</code> and update it to do so.</p>
<p>We need to include these new <code>using</code> statements at the top of the file so that we have access to the classes that we need from the NuGet packages we just installed:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity.EntityFrameworkCore</span>;
</code></pre></div><p>Next, instead of <code>DbContext</code>, the <code>VehicleQuotesContext</code> class should inherit from <code>IdentityUserContext<IdentityUser></code>.</p>
<p><code>IdentityUserContext</code> is a class provided by ASP.NET Core Identity that’s designed so that our <code>DbContext</code> can inherit from it and gain user management functionality. Namely, it includes a new <code>DbSet</code> (and consequently, a table) for holding user accounts (aptly named <code>Users</code>), among other things that we won’t need.</p>
<blockquote>
<p><code>IdentityUserContext</code> has a more feature rich counterpart called <code>IdentityDbContext</code>, which also includes <code>DbSet</code>s to support roles based authorization. We don’t need all that so we use its simpler cousin. Feel free to explore the <a href="https://github.com/dotnet/aspnetcore/blob/main/src/Identity/EntityFrameworkCore/src/IdentityDbContext.cs">source code on GitHub</a> to see all it offers.</p>
</blockquote>
<p>The <a href="https://docs.microsoft.com/en-us/dotnet/standard/generics/">generic type parameter</a> that we give it, <code>IdentityUser</code>, is a class that’s also provided by the Identity library. Its purpose is to serve as a default <a href="https://docs.microsoft.com/en-us/ef/core/modeling/entity-types?tabs=data-annotations">Entity Type</a> for our user model and, as a consequence, our <code>Users</code> <code>DbSet</code>.</p>
<p>In summary, by having our <code>DbContext</code> class inherit from <code>IdentityUserContext<IdentityUser></code>, we’re telling Identity that we want it to augment our <code>DbContext</code> (and database) to include the core user management tables and that we want our users table to have the same columns as <code>IdentityUser</code>.</p>
<blockquote>
<p>If we wanted to include more columns in our users table, what we would have to do is create a new class, make it inherit from <code>IdentityUser</code>, define any additional fields that we want on it, and use that class as a type parameter to <code>IdentityUserContext</code>. For us for now, the default works just fine. You can learn more about customizing Identity in <a href="https://docs.microsoft.com/en-us/aspnet/core/security/authentication/customize-identity-model?view=aspnetcore-6.0">the official docs</a>.</p>
</blockquote>
<p>The change looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#fdd">-public class VehicleQuotesContext : DbContext
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+public class VehicleQuotesContext : IdentityUserContext<IdentityUser>
</span></code></pre></div><p>Finally, <code>IdentityUserContext</code> has some logic that it needs to run when it is being created. In order to allow it to run that logic, let’s add the following line to our <code>VehicleQuotesContext</code>’s <code>OnModelCreating</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> protected override void OnModelCreating(ModelBuilder modelBuilder)
{
<span style="color:#000;background-color:#dfd">+ base.OnModelCreating(modelBuilder);
</span><span style="color:#000;background-color:#dfd"></span> // ...
}
</code></pre></div><p>This calls <code>IdentityUserContext</code>’s own <code>OnModelCreating</code> implementation so that it can set itself up properly.</p>
<p>The complete file should be looking like this now:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">VehicleQuotesContext</span> : IdentityUserContext<IdentityUser>
{
<span style="color:#080;font-weight:bold">public</span> VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
: <span style="color:#080;font-weight:bold">base</span>(options)
{
}
<span style="color:#080;font-weight:bold">public</span> DbSet<Make> Makes { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<Size> Sizes { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<BodyType> BodyTypes { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<Model> Models { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<ModelStyle> ModelStyles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<ModelStyleYear> ModelStyleYears { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<QuoteRule> QuoteRules { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<QuoteOverride> QuoteOverides { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DbSet<Quote> Quotes { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> OnModelCreating(ModelBuilder modelBuilder)
{
<span style="color:#080;font-weight:bold">base</span>.OnModelCreating(modelBuilder);
modelBuilder.Entity<Size>().HasData(
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">1</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Subcompact"</span> },
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">2</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Compact"</span> },
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">3</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Mid Size"</span> },
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">5</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Full Size"</span> }
);
modelBuilder.Entity<BodyType>().HasData(
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">1</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Coupe"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">2</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Sedan"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">3</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Hatchback"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">4</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Wagon"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">5</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Convertible"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">6</span>, Name = <span style="color:#d20;background-color:#fff0f0">"SUV"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">7</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Truck"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">8</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Mini Van"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">9</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Roadster"</span> }
);
}
}
}
</code></pre></div><h4 id="applying-database-changes">Applying database changes</h4>
<p>The <code>Users</code> <code>DbSet</code> that we added into our data model by inheriting from <code>IdentityUserContext<IdentityUser></code> will become a table once we create and apply a database migration. That’s what we will do next.</p>
<p>That’s simple in ASP.NET Core. All we need to do is run a command like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet ef migrations add AddIdentityTables
</code></pre></div><p>That should produce a new migration file named something like <code>20220605003253_AddIdentityTables.cs</code>. If you explore it, you’ll see how it contains the definitions for a few new database tables. Including the one that we want: <code>AspNetUsers</code>. That’s the one that we will use to store our user account records.</p>
<p>Next, we apply the migration with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet ef database update
</code></pre></div><p>If you now connect to the database, you’ll see the new tables in there.</p>
<p>If you have the database running in a Docker container like we discussed in the beginning of the article, you should be able to connect to it with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ psql -h localhost -U vehicle_quotes
</code></pre></div><p>Then, to see the tables:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vehicle_quotes=# \dt
List of relations
Schema | Name | Type | Owner
--------+-----------------------+-------+----------------
public | AspNetUserClaims | table | vehicle_quotes
public | AspNetUserLogins | table | vehicle_quotes
public | AspNetUserTokens | table | vehicle_quotes
public | AspNetUsers | table | vehicle_quotes
...
(14 rows)
</code></pre></div><p>And that’s pretty much it when it comes to having a sensible storage for user accounts. That was pretty inexpensive, wasn’t it? All we had to do was install some NuGet packages, tweak our existing DbContext, and run some migrations.</p>
<p>The best thing is that we’re not done yet. We can also take advantage of ASP.NET Core Identity to manage users programmatically. Instead of interacting with these tables directly, we will use the service classes provided by Identity to create new users, fetch existing ones and validate their credentials.</p>
<p>Before we can do that though, we must first do some configuration in our app’s <code>Startup</code> class so that said services are properly set up to our liking and are made available to our application.</p>
<!-- Next step for us is creating and fetching users. Let's see how we can make that happen. -->
<h4 id="configuring-the-identity-services">Configuring the Identity services</h4>
<p>We need to add some code to <code>VehicleQuotes/Startup.cs</code>. With it, we configure the Identity system and add a few service classes to ASP.NET Core’s <a href="https://en.wikipedia.org/wiki/Inversion_of_control">IoC</a> <a href="https://www.tutorialsteacher.com/ioc/ioc-container">container</a> so that they are available to our app via <a href="https://en.wikipedia.org/wiki/Dependency_injection">Dependency Injection</a>.</p>
<p>We need a new <code>using</code> statement:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
</code></pre></div><p>And the following code added to the <code>ConfigureServices</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">services
.AddIdentityCore<IdentityUser>(options => {
options.SignIn.RequireConfirmedAccount = <span style="color:#080;font-weight:bold">false</span>;
options.User.RequireUniqueEmail = <span style="color:#080;font-weight:bold">true</span>;
options.Password.RequireDigit = <span style="color:#080;font-weight:bold">false</span>;
options.Password.RequiredLength = <span style="color:#00d;font-weight:bold">6</span>;
options.Password.RequireNonAlphanumeric = <span style="color:#080;font-weight:bold">false</span>;
options.Password.RequireUppercase = <span style="color:#080;font-weight:bold">false</span>;
options.Password.RequireLowercase = <span style="color:#080;font-weight:bold">false</span>;
})
.AddEntityFrameworkStores<VehicleQuotesContext>();
</code></pre></div><p>The call to <code>AddIdentityCore</code> makes several Identity utility classes available to the application. Among those, <code>UserManager</code> is the only one we will use. We will use it later to… well, manage users. You can also see how we’ve set a few options related to how the user accounts are handled. <code>options.SignIn.RequireConfirmedAccount</code> controls whether new accounts need to be confirmed via email before they are available. With <code>options.User.RequireUniqueEmail</code>, we tell Identity to enforce uniqueness of emails on user accounts. And finally the <code>options.Password.*</code> options configure the password strength requirements.</p>
<blockquote>
<p>You can explore all the available options in <a href="https://docs.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.identity.identityoptions?view=aspnetcore-6.0">the official docs</a>.</p>
</blockquote>
<p>Then, the call to <code>AddEntityFrameworkStores</code> tells the Identity system that it should use our <code>VehicleQuotesContext</code> for data storage.</p>
<h4 id="creating-users">Creating users</h4>
<p>With that configuration out of the way, we can now write some code to create new user accounts. To keep things simple, we’ll add a new <code>UsersController</code> to our project that will expose a new endpoint that offers that functionality.</p>
<p>Let’s start with this in <code>VehicleQuotes/Controllers/UsersController.cs</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Mvc</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Controllers</span>
{
<span style="color:#369"> [Route("api/[controller]</span><span style="color:#d20;background-color:#fff0f0">")]
</span><span style="color:#d20;background-color:#fff0f0"></span><span style="color:#369"> [ApiController]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">UsersController</span> : ControllerBase
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> UserManager<IdentityUser> <span style="color:#00d;font-weight:bold">_</span>userManager;
<span style="color:#080;font-weight:bold">public</span> UsersController(
UserManager<IdentityUser> userManager
) {
<span style="color:#00d;font-weight:bold">_</span>userManager = userManager;
}
}
}
</code></pre></div><p>As you can see, this controller defines a dependency on <code>UserManager<IdentityUser></code>, an instance of which is injected via the constructor. This is one of the classes made available to us when we configured the Identity core services in our app’s <code>Startup</code>. We will use it to create new user records.</p>
<p>Before that though, we need to define a new class that encapsulates the payload for a request to our endpoint. When it comes to creating user accounts, all we need is a username, a password, and an email. As such, we add the following class in a new <code>VehicleQuotes/ResourceModels/User.cs</code> file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">User</span>
{
<span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> UserName { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Password { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Email { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Now, we shall use this type as the parameter for a new <code>PostUser</code> action method in the <code>UserController</code> which will expose the new user account creation endpoint in our API. The method looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// POST: api/Users
</span><span style="color:#888"></span><span style="color:#369">[HttpPost]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<User>> PostUser(User user)
{
<span style="color:#080;font-weight:bold">if</span> (!ModelState.IsValid)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(ModelState);
}
<span style="color:#888;font-weight:bold">var</span> result = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.CreateAsync(
<span style="color:#080;font-weight:bold">new</span> IdentityUser() { UserName = user.UserName, Email = user.Email },
user.Password
);
<span style="color:#080;font-weight:bold">if</span> (!result.Succeeded)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(result.Errors);
}
user.Password = <span style="color:#080;font-weight:bold">null</span>;
<span style="color:#080;font-weight:bold">return</span> Created(<span style="color:#d20;background-color:#fff0f0">""</span>, user);
}
</code></pre></div><p>Be sure to also add the following <code>using</code> statements:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Threading.Tasks</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>;
</code></pre></div><p><code>System.Threading.Tasks</code> allows us to reference the class <code>Task<></code> which is the return type of the new <code>PostUser</code> method. <code>VehicleQuotes.ResourceModels</code> is where our new <code>User</code> class lives.</p>
<p>Thanks to the instance of <code>UserManager<IdentityUser></code> that we’re holding onto, this method is very straightforward. The most interesting portion is this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">var</span> result = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.CreateAsync(
<span style="color:#080;font-weight:bold">new</span> IdentityUser() { UserName = user.UserName, Email = user.Email },
user.Password
);
</code></pre></div><p>Here we’re <code>new</code>ing up an <code>IdentityUser</code> instance using the given request parameters and passing it on to <code>UserManager<IdentityUser></code>’s <code>CreateAsync</code> method, along with the password that was also given in the incoming request. This puts the Identity system to work for us and properly create a new user account.</p>
<p>Then, we can inspect its return value (which we capture in the <code>result</code> variable) to determine if the operation was successful. That way we can respond appropriately to our API’s caller.</p>
<p>Finally, with…</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">user.Password = <span style="color:#080;font-weight:bold">null</span>;
<span style="color:#080;font-weight:bold">return</span> Created(<span style="color:#d20;background-color:#fff0f0">""</span>, user);
</code></pre></div><p>we return the data representing the newly created user account but we’re discreet and make sure not to include the password.</p>
<p>With that, we can test our API. Fire it up with <code>dotnet run</code> (or <a href="https://docs.microsoft.com/en-us/aspnet/core/test/hot-reload?view=aspnetcore-6.0"><code>dotnet watch</code></a>!) and send a <code>POST</code> request to our new endpoint at <code>http://0.0.0.0:5000/api/Users</code> with a payload like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"userName"</span>: <span style="color:#d20;background-color:#fff0f0">"newuser000"</span>,
<span style="color:#b06;font-weight:bold">"email"</span>: <span style="color:#d20;background-color:#fff0f0">"newuser000@endpointdev.com"</span>,
<span style="color:#b06;font-weight:bold">"password"</span>: <span style="color:#d20;background-color:#fff0f0">"password"</span>
}
</code></pre></div><p>I just tried it in <a href="https://www.postman.com/">Postman</a> and this is what it looked like:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/post-user.webp" alt="Successful user creation request in Postman"></p>
<p>Feel free to test it out further. Try repeated emails or usernames. Try passwords that don’t meet the criteria we defined when configuring Identity in the app’s <code>Startup</code> class. It all works as you’d expect.</p>
<p>You can inspect the database and see the newly created record too:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vehicle_quotes=# select id, user_name, email, password_hash from "AspNetUsers";
-[ RECORD 1 ]-+--------------------------------------
id | aaad07b4-f109-4255-8caa-185fd7694c72
user_name | newuser000
email | newuser000@endpointdev.com
password_hash | AQAAAAEAACcQAAAAEJJTV7M2Ejqd3K3iC...
</code></pre></div><p>And there it is, that’s our record. With a hashed password and everything.</p>
<p>Now let’s see what would an endpoint to fetch users look like:</p>
<h4 id="fetching-users">Fetching users</h4>
<p><code>UserManager<IdentityUser></code> offers a <code>FindByNameAsync</code> method that we can use to retrieve users by name. We can add a new endpoint that leverages it like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// GET: api/Users/username
</span><span style="color:#888"></span><span style="color:#369">[HttpGet("{username}")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<User>> GetUser(<span style="color:#888;font-weight:bold">string</span> username)
{
IdentityUser user = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.FindByNameAsync(username);
<span style="color:#080;font-weight:bold">if</span> (user == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> User
{
UserName = user.UserName,
Email = user.Email
};
}
</code></pre></div><p>This is even more straightforward than the previous one. We just call the <code>FindByNameAsync</code> method by giving it the username of the account we want, make sure that we actually found it, and then return the data.</p>
<p>For the response, we use the same <code>User</code> class that we created to represent the input for the creation endpoint. If we added more fields to the user profile, we could include them here. Alas, we only have username and email for now.</p>
<p>Restart the app and we can now make a request like this:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/get-user.webp" alt="Successful user fetch request in Postman"></p>
<p>Pretty neat, huh? Try a username that does not exist and you should see the API respond with a 404.</p>
<blockquote>
<p>One quick improvement that we can do before we move on: let’s change <code>PostUser</code>’s return statement to this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">return</span> CreatedAtAction(<span style="color:#d20;background-color:#fff0f0">"GetUser"</span>, <span style="color:#080;font-weight:bold">new</span> { username = user.UserName }, user);
</code></pre></div><p>All that does is include a <code>Location</code> header on the POST Users endpoint’s response that contains the URL for the newly created user. <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/201">That’s just being a good citizen</a>.</p>
</blockquote>
<h3 id="implementing-jwt-bearer-token-authentication">Implementing JWT Bearer Token authentication</h3>
<p>Now that we have the user management capabilities that we need, let’s implement some actual authentication. We will start by adding <a href="https://jwt.io/">JWT</a>-based authentication to our API.</p>
<p>The strategy that we will use is to create a new API endpoint that clients can <code>POST</code> credentials to and that will respond to them with a fresh, short-lived token. They will then be able to use that token for subsequent requests by including it via headers.</p>
<p>We will pick a random endpoint to secure, just to serve as an example. That is, an endpoint that requires authentication in order to be accessed. <code>GET api/BodyTypes</code> is a good candidate. It is defined in <code>VehicleQuotes/Controllers/BodyTypesController.cs</code>’s <code>GetBodyTypes</code> action method. Feel free to test it out.</p>
<h4 id="creating-tokens">Creating tokens</h4>
<p>Let’s start by creating the new endpoint that clients will call to obtain the auth tokens. In order to do so, we need a few things:</p>
<ol>
<li>A class that represents incoming request data.</li>
<li>A class that represents outgoing response data.</li>
<li>A class that can generate tokens for a given user.</li>
<li>A new action method in <code>UsersController</code> that does the work.</li>
</ol>
<h5 id="the-request-data-structure">The request data structure</h5>
<p>To deal with step one, let’s define a new class in <code>VehicleQuotes/ResourceModels/AuthenticationRequest.cs</code> that looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">AuthenticationRequest</span>
{
<span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> UserName { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Password { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>All users need to authenticate is provide a set of credentials: username and password. So we have this class that contains fields for both and that will be used as input for our endpoint.</p>
<h5 id="the-response-data-structure">The response data structure</h5>
<p>Next, we need to define a class to represent that endpoint’s response. I’ve added the following class in <code>VehicleQuotes/ResourceModels/AuthenticationResponse.cs</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">AuthenticationResponse</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Token { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DateTime Expiration { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Just a simple data structure containing the token itself and a date letting clients know when they can expect it to expire.</p>
<h5 id="the-class-that-creates-jwts">The class that creates JWTs</h5>
<p>Step 3 is creating a class that can produce the tokens. This is the most interesting part in terms of complexity. Before we can do that though, let’s add the following configurations to <code>VehicleQuotes/appsettings.json</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json"> <span style="color:#d20;background-color:#fff0f0">"Jwt"</span><span style="color:#a61717;background-color:#e3d2d2">:</span> {
<span style="color:#b06;font-weight:bold">"Key"</span>: <span style="color:#d20;background-color:#fff0f0">"this is the secret key for the jwt, it must be kept secure"</span>,
<span style="color:#b06;font-weight:bold">"Issuer"</span>: <span style="color:#d20;background-color:#fff0f0">"vehiclequotes.endpointdev.com"</span>,
<span style="color:#b06;font-weight:bold">"Audience"</span>: <span style="color:#d20;background-color:#fff0f0">"vehiclequotes.endpointdev.com"</span>,
<span style="color:#b06;font-weight:bold">"Subject"</span>: <span style="color:#d20;background-color:#fff0f0">"JWT for vehiclequotes.endpointdev.com"</span>
}<span style="color:#a61717;background-color:#e3d2d2">,</span>
</code></pre></div><p>These are values that we’ll need when creating the tokens. We’ll go over the purpose of each them as they come up as we continue writing our code.</p>
<blockquote>
<p>Here we’ll gloss over some details on the inner workings of JWTs as a standard. You can learn more about them at <a href="https://jwt.io/introduction">jwt.io</a>.</p>
</blockquote>
<p>For now, let’s add the following class in <code>VehicleQuotes/Services/JwtService.cs</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.IdentityModel.Tokens.Jwt</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Security.Claims</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Text</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Configuration</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.IdentityModel.Tokens</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">JwtService</span>
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">const</span> <span style="color:#888;font-weight:bold">int</span> EXPIRATION_MINUTES = <span style="color:#00d;font-weight:bold">1</span>;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> IConfiguration <span style="color:#00d;font-weight:bold">_</span>configuration;
<span style="color:#080;font-weight:bold">public</span> JwtService(IConfiguration configuration)
{
<span style="color:#00d;font-weight:bold">_</span>configuration = configuration;
}
<span style="color:#080;font-weight:bold">public</span> AuthenticationResponse CreateToken(IdentityUser user)
{
<span style="color:#888;font-weight:bold">var</span> expiration = DateTime.UtcNow.AddMinutes(EXPIRATION_MINUTES);
<span style="color:#888;font-weight:bold">var</span> token = CreateJwtToken(
CreateClaims(user),
CreateSigningCredentials(),
expiration
);
<span style="color:#888;font-weight:bold">var</span> tokenHandler = <span style="color:#080;font-weight:bold">new</span> JwtSecurityTokenHandler();
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> AuthenticationResponse {
Token = tokenHandler.WriteToken(token),
Expiration = expiration
};
}
<span style="color:#080;font-weight:bold">private</span> JwtSecurityToken CreateJwtToken(Claim[] claims, SigningCredentials credentials, DateTime expiration) =>
<span style="color:#080;font-weight:bold">new</span> JwtSecurityToken(
<span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Issuer"</span>],
<span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Audience"</span>],
claims,
expires: expiration,
signingCredentials: credentials
);
<span style="color:#080;font-weight:bold">private</span> Claim[] CreateClaims(IdentityUser user) =>
<span style="color:#080;font-weight:bold">new</span>[] {
<span style="color:#080;font-weight:bold">new</span> Claim(JwtRegisteredClaimNames.Sub, <span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Subject"</span>]),
<span style="color:#080;font-weight:bold">new</span> Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
<span style="color:#080;font-weight:bold">new</span> Claim(JwtRegisteredClaimNames.Iat, DateTime.UtcNow.ToString()),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.NameIdentifier, user.Id),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.Name, user.UserName),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.Email, user.Email)
};
<span style="color:#080;font-weight:bold">private</span> SigningCredentials CreateSigningCredentials() =>
<span style="color:#080;font-weight:bold">new</span> SigningCredentials(
<span style="color:#080;font-weight:bold">new</span> SymmetricSecurityKey(
Encoding.UTF8.GetBytes(<span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Key"</span>])
),
SecurityAlgorithms.HmacSha256
);
}
}
</code></pre></div><p>The <code>CreateToken</code> method is the element that’s most worth discussing here. It receives an instance of <code>IdentityUser</code> as a parameter (which, remember, is the entity class that represents our user accounts), uses it to construct a JWT, and returns it within a <code>AuthenticationResponse</code> object (which is what we decided that our endpoint would return). To do so, we use various classes built into .NET.</p>
<p>The main class that represents the JWT is <code>JwtSecurityToken</code>. We <code>new</code> up one of those in the <code>CreateJwtToken</code> method with this call:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">new</span> JwtSecurityToken (
<span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Issuer"</span>],
<span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Audience"</span>],
claims,
expires: expiration,
signingCredentials: credentials
);
</code></pre></div><p>“Issuer” and “Audience” are two important values for how JWTs work. They specify which entity is creating the token (i.e. the “Issuer”) and which entity is the token intended for (i.e. the “Audience”). We use the <code>IConfiguration</code> instance that we got as a dependency to fetch their values from the <code>VehicleQuotes/appsettings.json</code> file.</p>
<blockquote>
<p>The “Issuer” and “Audience” parameters in <code>JwtSecurityToken</code>’s constructor correspond to JWT claims <code>iss</code> and <code>aud</code>, respectively. You can learn more about them and other claims in <a href="https://datatracker.ietf.org/doc/html/rfc7519#section-4.1">the RFC</a>.</p>
</blockquote>
<p>The next parameter that <code>JwtSecurityToken</code>’s constructor needs is the claims array. In JWT terms, a claim is essentially a statement about the entity for which the token is generated, some data that identifies it. For example, if we’re generating a token for a user, what you would expect to see in such a token’s claims are things like username, email, and any other non-secret profile info.</p>
<p>In our case, as you can see in the <code>CreateClaims</code> method, we add a number of claims:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">new</span>[] {
<span style="color:#080;font-weight:bold">new</span> Claim(JwtRegisteredClaimNames.Sub, <span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Subject"</span>]),
<span style="color:#080;font-weight:bold">new</span> Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
<span style="color:#080;font-weight:bold">new</span> Claim(JwtRegisteredClaimNames.Iat, DateTime.UtcNow.ToString()),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.NameIdentifier, user.Id),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.Name, user.UserName),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.Email, user.Email)
};
</code></pre></div><p>Along with the user id, name and email, we also add <code>sub</code>, <code>jti</code> and <code>iat</code> claims. These are standardized claims of which you can learn more about in <a href="https://datatracker.ietf.org/doc/html/rfc7519#section-4.1">the RFC</a>. The data that we put in here will make its way into the encoded token that our API caller eventually sees. We’ll see that later.</p>
<p>The next parameter to <code>JwtSecurityToken</code> is the expiration date of the token. Here we are setting it to just one minute in the future to comply with our original requirement that the token should be short-lived so that it only allows a handful of requests over a short period of time.</p>
<p>Finally there’s the <code>signingCredentials</code> parameter which tells the <code>JwtSecurityToken</code> how to cryptographically sign the token. As you can see in the code:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">new</span> SigningCredentials(
<span style="color:#080;font-weight:bold">new</span> SymmetricSecurityKey(
Encoding.UTF8.GetBytes(<span style="color:#00d;font-weight:bold">_</span>configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Key"</span>])
),
SecurityAlgorithms.HmacSha256
);
</code></pre></div><p>That’s a <code>SigningCredentials</code> instance that we create using the “key” that we have configured in <code>VehicleQuotes/appsettings.json</code>, along with the algorithm to use to produce it.</p>
<p>And that’s about it. There isn’t much else to this class. It is somewhat involved but that’s just how JWTs are created in .NET.</p>
<h5 id="the-action-method-that-puts-it-all-together">The action method that puts it all together</h5>
<p>Now all we need to do is actually create an endpoint that exposes this functionality to clients. Since we have the core logic encapsulated in our <code>JwtService</code> class, the actual action method is simple. Here are the changes that we need to make in order to implement it:</p>
<p>First, on <code>VehicleQuotes/Controllers/UsersController.cs</code>, we add a new using statement so that we can reference our new <code>JwtService</code> class:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Services</span>;
</code></pre></div><p>Next, we declare a new parameter of type <code>JwtService</code> in the controller’s constructor so that we signal to ASP.NET Core’s Dependency Injection subsystem that we want to use one of those. We also hold onto it via a new instance variable called <code>_jwtService</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> // ...
public class UsersController : ControllerBase
{
private readonly UserManager<IdentityUser> _userManager;
<span style="color:#000;background-color:#dfd">+ private readonly JwtService _jwtService;
</span><span style="color:#000;background-color:#dfd"></span>
public UsersController(
UserManager<IdentityUser> userManager,
<span style="color:#000;background-color:#dfd">+ JwtService jwtService
</span><span style="color:#000;background-color:#dfd"></span> ) {
_userManager = userManager;
<span style="color:#000;background-color:#dfd">+ _jwtService = jwtService;
</span><span style="color:#000;background-color:#dfd"></span> }
// ...
}
</code></pre></div><p>We also need to tell ASP.NET Core that <code>JwtService</code> should be available for Dependency Injection. To do so, we can add this line in <code>VehicleQuotes/Startup.cs</code>’s <code>ConfigureServices</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">services.AddScoped<Services.JwtService>();
</code></pre></div><blockquote>
<p>Note that this way of defining dependencies is not recommended and only done this way here to keep things simple for an illustrative app that’s never going to run in production. Ideally what you want to do here is define the dependency as an abstraction (i.e. an interface) and a concrete implementation that fulfills it. For example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">services.AddScoped<ITokenCreationService, JwtService>();
</code></pre></div><p>This way, classes that depend on this service can reference the interface, not the concrete type. In doing that, they adhere to the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle">Dependency Inversion</a> principle and become more easily testable because they allow mocks to be provided as dependencies.</p>
</blockquote>
<p>Finally, we write the actual action method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// POST: api/Users/BearerToken
</span><span style="color:#888"></span><span style="color:#369">[HttpPost("BearerToken")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<AuthenticationResponse>> CreateBearerToken(AuthenticationRequest request)
{
<span style="color:#080;font-weight:bold">if</span> (!ModelState.IsValid)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(<span style="color:#d20;background-color:#fff0f0">"Bad credentials"</span>);
}
<span style="color:#888;font-weight:bold">var</span> user = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.FindByNameAsync(request.UserName);
<span style="color:#080;font-weight:bold">if</span> (user == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(<span style="color:#d20;background-color:#fff0f0">"Bad credentials"</span>);
}
<span style="color:#888;font-weight:bold">var</span> isPasswordValid = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.CheckPasswordAsync(user, request.Password);
<span style="color:#080;font-weight:bold">if</span> (!isPasswordValid)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(<span style="color:#d20;background-color:#fff0f0">"Bad credentials"</span>);
}
<span style="color:#888;font-weight:bold">var</span> token = <span style="color:#00d;font-weight:bold">_</span>jwtService.CreateToken(user);
<span style="color:#080;font-weight:bold">return</span> Ok(token);
}
</code></pre></div><p>Here too we’re leveraging the <code>UserManager</code> instance that ASP.NET Core Identity so graciously provided us with. In summary, we find the user account by name using the incoming request data, check if the given password is correct, then ask our <code>JwtService</code> to create a token for this user, and finally return it wrapped in a 200 response.</p>
<p>If you hit that endpoint with a POST and a set of existing credentials, you should see something like this:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/post-bearer-token.webp" alt="Successful JTW creation in Postman"></p>
<p>If you take that big string that came back in the <code>"token"</code> field in the response JSON, and paste it in <a href="https://jwt.io/">jwt.io</a>’s token decoder, you should be able to see all the claims that we added to the token:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/jwt-decoded.webp" alt="The JTW decoded showing all the claims"></p>
<blockquote>
<p>Notice that the keywords here are “Encoded” and “Decoded”. The claims that we put in our JWTs are not protected in any way. As such, we should never put secrets in there.</p>
</blockquote>
<h4 id="securing-an-endpoint-with-jwt-authentication">Securing an endpoint with JWT authentication</h4>
<p>Now that we have a way for obtaining tokens, let’s see how we can actually use them to gain access to some resources. To demonstrate that, let’s secure an endpoint in a way that it denies unauthenticated requests.</p>
<h5 id="applying-the-authorize-attribute">Applying the Authorize attribute</h5>
<p>First we need to signal ASP.NET Core that the endpoint requires auth. We do that by annotating the corresponding action method with the <code>Authorize</code> attribute. We’ve already decided that we were going to use <code>VehicleQuotes/Controllers/BodyTypesController.cs</code>’s <code>GetBodyTypes</code> method as a guinea pig. So, let’s go into that file and add the following <code>using</code> statement:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Authorization</span>;
</code></pre></div><p>That will allow us access to the attribute, which we can apply to the action method like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> // GET: api/BodyTypes
<span style="color:#000;background-color:#dfd">+ [Authorize]
</span><span style="color:#000;background-color:#dfd"></span> [HttpGet]
public async Task<ActionResult<IEnumerable<BodyType>>> GetBodyTypes()
{
return await _context.BodyTypes.ToListAsync();
}
</code></pre></div><p>With this, we’ve told ASP.NET Core that we want it to require auth for this endpoint.</p>
<h5 id="enabling-and-configuring-the-jwt-authentication">Enabling and configuring the JWT authentication</h5>
<p>Now, we need to tell it how to actually perform the check. To do that, we need to install the <a href="https://www.nuget.org/packages/Microsoft.AspNetCore.Authentication.JwtBearer"><code>Microsoft.AspNetCore.Authentication.JwtBearer</code></a> NuGet package. We can do that from the <code>VehicleQuotes</code> directory with the following command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">$ dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer
</code></pre></div><p>Once we have that installed, we get access to additional services which we can use to configure the “JwtBearer” authentication scheme. As usual, we do the configuration in <code>VehicleQuotes/Startup.cs</code>’s. Here’s what we need to do:</p>
<p>Add a few new <code>using</code> statements:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Authentication.JwtBearer</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.IdentityModel.Tokens</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Text</span>;
</code></pre></div><p>Add the following code at the end of the <code>ConfigureServices</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">services
.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
.AddJwtBearer(options =>
{
options.TokenValidationParameters = <span style="color:#080;font-weight:bold">new</span> TokenValidationParameters()
{
ValidateIssuer = <span style="color:#080;font-weight:bold">true</span>,
ValidateAudience = <span style="color:#080;font-weight:bold">true</span>,
ValidateLifetime = <span style="color:#080;font-weight:bold">true</span>,
ValidateIssuerSigningKey = <span style="color:#080;font-weight:bold">true</span>,
ValidAudience = Configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Audience"</span>],
ValidIssuer = Configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Issuer"</span>],
IssuerSigningKey = <span style="color:#080;font-weight:bold">new</span> SymmetricSecurityKey(
Encoding.UTF8.GetBytes(Configuration[<span style="color:#d20;background-color:#fff0f0">"Jwt:Key"</span>])
)
};
});
</code></pre></div><p>The call to <code>AddAuthentication</code> includes all the internal core service classes that are needed to do authentication in our app.</p>
<p>The <code>JwtBearerDefaults.AuthenticationScheme</code> parameter that we give it is the name of the authentication scheme to use as the default. More on that later. For now, know that ASP.NET Core supports multiple authentication schemes to be used at the same time. In fact, our own plan here is to eventually support two auth schemes: JWTs and API Keys. We’re starting with JWT first and as such, that’s the one we specify as the default.</p>
<p>The call to <code>AddJwtBearer</code> configures the JWT authentication scheme. That is, it allows the app to perform authentication checks based on an incoming JWT token.</p>
<p>The most important part of the configuration is the values that we are passing to <code>TokenValidationParameters</code>. As you can see, we are able to specify which aspects of the incoming JWTs to validate. You can see all available options for <code>TokenValidationParameters</code> in <a href="https://docs.microsoft.com/en-us/dotnet/api/microsoft.identitymodel.tokens.tokenvalidationparameters?view=azure-dotnet">the official documentation</a>. Here we’ve chosen to validate obvious things like the issuer, audience, and signing key.</p>
<p>The last configuration step is to add this line right before <code>app.UseAuthorization();</code> in the <code>Configure</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">app.UseAuthentication();
</code></pre></div><p>That will enable the authentication <a href="https://docs.microsoft.com/en-us/aspnet/core/fundamentals/middleware/?view=aspnetcore-6.0">middleware</a>. That way the framework will perform authentication checks as part of the request processing pipeline. In other words, actually put all the configuration that we’ve done to good use.</p>
<p>Alright, now that we have all the configuration ready and have secured an endpoint, let’s try hitting it with a GET on <code>api/BodyTypes</code> and see what happens:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/unauthorized-response.webp" alt="Unauthorized response for unauthenticated request"></p>
<p>Ok! So far so good. We made an unauthenticated request into an endpoint that requires authentication and as a result we got a 401 back. That’s just what we wanted.</p>
<p>Now, let’s get a token by POSTing to <code>api/Users/BearerToken</code>:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/post-bearer-token-2.webp" alt="Another successful JTW creation in Postman"></p>
<p>We can copy that token and include as a header in the GET request to <code>api/BodyTypes</code>. The header key should be <code>Authorization</code> and the value should be <code>Bearer <our token></code>. In Postman, we can setup a similar request if we choose the “Authorization” tab, select “Bearer Token” in the “Type” drop-down list and paste the token in the “Token” text box.</p>
<p>Do that, and you should now see a 200 response from the endpoint:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/authorized-response-jwt.webp" alt="Successful response for request authenticated with JWT"></p>
<p>Neat! Now that we passed a valid token, it wants to talk to us again. Let a few minutes pass, enough for the token to expire and try the request again to see how it’s 401’ing again.</p>
<h3 id="implementing-api-key-authentication">Implementing API Key authentication</h3>
<p>Now let’s change gears from JWT and implement an alternate authentication strategy in our Web API: API Keys. Authentication with API Keys is fairly common in the web service world.</p>
<p>The core idea of API Keys is that the API provider (in this case, us) produces a secret string that is given to the clients for safekeeping. The clients then can use that key to access the API and make as many requests as they want. So it’s essentially just a glorified password.</p>
<p>The main functional differences when it comes to JWT as we have implemented it is that the API Keys won’t expire and that we will store them in our database. Then, when processing incoming requests, as part of the authentication step, our app will check if the given API Key exists in our internal records. If it does, then the request is allowed to go through, if it does not, then a 401 is sent back.</p>
<p>This is going to be interesting because ASP.NET Core does not include an authentication scheme out of the box for this kind of behavior. For JWT, we were able to use the <code>Microsoft.AspNetCore.Authentication.JwtBearer</code> package. For this we have to get our hands dirty and actually implement a custom authentication handler that will run the logic that we need.</p>
<p>Let’s get started.</p>
<h4 id="creating-api-keys">Creating API Keys</h4>
<p>Similarly to when we implemented JWTs, we’ll start by creating the new endpoint that clients will call to obtain their keys. In order to make it work, we’ll need:</p>
<ol>
<li>A new model and database table to store the keys</li>
<li>A service class to create the keys.</li>
<li>A new action method in <code>UsersController</code> that does the work.</li>
</ol>
<h5 id="the-api-key-model-and-database-table">The API Key model and database table</h5>
<p>For storing API Keys, we just need a simple table that is linked to users via a one-to-many relationship and includes a name and a value. Using Entity Framework, our model could look like this in <code>VehicleQuotes/Models/UserApiKey.cs</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Text.Json.Serialization</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#369"> [Index(nameof(Value), IsUnique = true)]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">UserApiKey</span>
{
<span style="color:#369"> [JsonIgnore]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Value { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [JsonIgnore]</span>
<span style="color:#369"> [Required]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> UserID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#369">
</span><span style="color:#369"> [JsonIgnore]</span>
<span style="color:#080;font-weight:bold">public</span> IdentityUser User { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Easy enough. We express the relationship between the “keys” and “users” tables via the <code>User</code> navigation property and the <code>UserID</code> field. We also define a unique index on the <code>Value</code> field and annotate some fields with <code>Required</code> because we don’t want them to allow null. Interestingly, we also annotated some of the fields with the <code>JsonIgnore</code> attribute so that when we include objects of this type in API responses (which, as you’ll see, we will), those fields are not included.</p>
<p>We also need to add a new <code>DbSet</code> on our DbContext class at <code>VehicleQuotes/Data/VehicleQuotesContext.cs</code> so that Entity Framework picks it up as part of the data model:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> DbSet<UserApiKey> UserApiKeys { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
</code></pre></div><p>With that, let’s now create and run a migration so that the new table can be added to the database. All it takes is running this to create the migration:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">dotnet ef migrations add AddUserApiKeysTable
</code></pre></div><p>And this to apply it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">dotnet ef database update
</code></pre></div><p>Now the new table should appear in the database:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vehicle_quotes=# \dt
List of relations
Schema | Name | Type | Owner
--------+-----------------------+-------+----------------
...
public | user_api_keys | table | vehicle_quotes
(15 rows)
</code></pre></div><h5 id="the-class-that-creates-api-keys">The class that creates API Keys</h5>
<p>The next step is very straightforward. All we need is some logic that, when given a user account (in the form of an instance of <code>IdentityUser</code>), can generate a new API Key, insert it in the database, and return it. The following unremarkable class, defined in <code>VehicleQuotes/Services/ApiKeyService.cs</code>, encapsulates said logic:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ApiKeyService</span>
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> VehicleQuotesContext <span style="color:#00d;font-weight:bold">_</span>context;
<span style="color:#080;font-weight:bold">public</span> ApiKeyService(VehicleQuotesContext context)
{
<span style="color:#00d;font-weight:bold">_</span>context = context;
}
<span style="color:#080;font-weight:bold">public</span> UserApiKey CreateApiKey(IdentityUser user)
{
<span style="color:#888;font-weight:bold">var</span> newApiKey = <span style="color:#080;font-weight:bold">new</span> UserApiKey
{
User = user,
Value = GenerateApiKeyValue()
};
<span style="color:#00d;font-weight:bold">_</span>context.UserApiKeys.Add(newApiKey);
<span style="color:#00d;font-weight:bold">_</span>context.SaveChanges();
<span style="color:#080;font-weight:bold">return</span> newApiKey;
}
<span style="color:#080;font-weight:bold">private</span> <span style="color:#888;font-weight:bold">string</span> GenerateApiKeyValue() =>
<span style="color:#d20;background-color:#fff0f0">$"{Guid.NewGuid().ToString()}-{Guid.NewGuid().ToString()}"</span>;
}
}
</code></pre></div><p>As you can see, this class’s single public method, aptly named <code>CreateApiKey</code>, just takes the given user and creates a new <code>UserApiKey</code> that’s associated with it and saves it into the database. The key values themselves are just two <a href="https://en.wikipedia.org/wiki/Universally_unique_identifier">GUIDs</a> concatenated.</p>
<blockquote>
<p>This is an admittedly simplistic and not-all-that-secure method for generating the keys themselves. For simplicity, we’ve gone with this. In a production app however, it’d be better to use a true cryptographically secure string. <a href="https://jonathancrozier.com/blog/how-to-generate-a-cryptographically-secure-random-string-in-dot-net-with-c-sharp">Here’s an article</a> that explains how to do that with .NET.</p>
</blockquote>
<p>Naturally, this class needs to be used elsewhere in the application, so let’s make it available for Dependency Injection by adding the following line to <code>VehicleQuotes/Startup.cs</code>’s <code>ConfigureServices</code> method.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">services.AddScoped<Services.ApiKeyService>();
</code></pre></div><h5 id="the-action-method-that-puts-it-all-together-1">The action method that puts it all together</h5>
<p>Now let’s finally write a new action method on <code>VehicleQuotes/Controllers/UsersController.cs</code> so that there’s an endpoint that clients can call to get API Keys. Before adding the action method though, as usual, we need to declare the following dependency as a constructor parameter and hold it in an instance variable:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> public class UsersController : ControllerBase
{
private readonly UserManager<IdentityUser> _userManager;
private readonly JwtService _jwtService;
<span style="color:#000;background-color:#dfd">+ private readonly ApiKeyService _apiKeyService;
</span><span style="color:#000;background-color:#dfd"></span>
public UsersController(
UserManager<IdentityUser> userManager,
JwtService jwtService,
<span style="color:#000;background-color:#dfd">+ ApiKeyService apiKeyService
</span><span style="color:#000;background-color:#dfd"></span> ) {
_userManager = userManager;
_jwtService = jwtService;
<span style="color:#000;background-color:#dfd">+ _apiKeyService = apiKeyService;
</span><span style="color:#000;background-color:#dfd"></span> }}
</code></pre></div><p>The action method itself would look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// POST: api/Users/ApiKey
</span><span style="color:#888"></span><span style="color:#369">[HttpPost("ApiKey")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult> CreateApiKey(AuthenticationRequest request)
{
<span style="color:#080;font-weight:bold">if</span> (!ModelState.IsValid)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(ModelState);
}
<span style="color:#888;font-weight:bold">var</span> user = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.FindByNameAsync(request.UserName);
<span style="color:#080;font-weight:bold">if</span> (user == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(<span style="color:#d20;background-color:#fff0f0">"Bad credentials"</span>);
}
<span style="color:#888;font-weight:bold">var</span> isPasswordValid = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>userManager.CheckPasswordAsync(user, request.Password);
<span style="color:#080;font-weight:bold">if</span> (!isPasswordValid)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest(<span style="color:#d20;background-color:#fff0f0">"Bad credentials"</span>);
}
<span style="color:#888;font-weight:bold">var</span> token = <span style="color:#00d;font-weight:bold">_</span>apiKeyService.CreateApiKey(user);
<span style="color:#080;font-weight:bold">return</span> Ok(token);
}
</code></pre></div><p>This method is very similar to the other one that generates JWTs. Painfully so. In fact the only difference is that this one calls the <code>ApiKeyService</code>’s <code>CreateApiKey</code> method instead of <code>JwtService</code>’s <code>CreateToken</code>, for obvious reasons.</p>
<blockquote>
<p>These two are ripe for some refactoring to eliminate duplication but we won’t do that here. Take a look at the <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/tree/json-web-token">finished project on GitHub</a> to see <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/commit/9a078015cec2e8a42a4898e203a1a50db69731e8">a neat refactoring option</a>.</p>
</blockquote>
<p>Anyway, you can now try a POST into <code>api/Users/ApiKey</code>, pass it a valid set of credentials, and you should see it respond with a brand new API Key:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/post-api-key.webp" alt="Successful response for authenticated request"></p>
<p>The key can also be found in the database:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vehicle_quotes=# select id, SUBSTRING(value,1,13), user_id from user_api_keys;
id | substring | user_id
----+---------------+--------------------------------------
1 | efd2133b-cc4a | aaad07b4-f109-4255-8caa-185fd7694c72
(1 row)
</code></pre></div><h4 id="securing-an-endpoint-with-api-key-authentication">Securing an endpoint with API Key authentication</h4>
<p>Now that users have a way of getting API Keys, we can start securing endpoints with that authentication scheme. In order to acomplish that, we need to take three steps:</p>
<ol>
<li>Implement a custom authentication handler that runs the authentication logic.</li>
<li>Configure the new authentication handler, plugging it into ASP.NET Core’s auth mechanisms.</li>
<li>Secure certain endpoints via the Authorization attribute, specifying the scheme that we want to use to do so.</li>
</ol>
<p>We’ll do that next.</p>
<h5 id="implementing-the-api-key-authentication-handler">Implementing the API Key authentication handler</h5>
<p>Like we’ve touched on before, the “authentication handler” is a class that can plug into ASP.NET Core’s authentication middleware and can run the authentication logic for a given authentication scheme. In practical terms, it’s a class that inherits from <code>Microsoft.AspNetCore.Authentication.AuthenticationHandler</code> and implements the <code>HandleAuthenticateAsync</code> method. Ours will live in <code>VehicleQuotes/Authentication/ApiKey/ApiKeyAuthenticationHandler.cs</code> and it looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Security.Claims</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Text.Encodings.Web</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Threading.Tasks</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Authentication</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Identity</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Logging</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Options</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Authentication.ApiKey</span>
{
<span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ApiKeyAuthenticationHandler</span> : AuthenticationHandler<AuthenticationSchemeOptions>
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">const</span> <span style="color:#888;font-weight:bold">string</span> API_KEY_HEADER = <span style="color:#d20;background-color:#fff0f0">"Api-Key"</span>;
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> VehicleQuotesContext <span style="color:#00d;font-weight:bold">_</span>context;
<span style="color:#080;font-weight:bold">public</span> ApiKeyAuthenticationHandler(
IOptionsMonitor<AuthenticationSchemeOptions> options,
ILoggerFactory logger,
UrlEncoder encoder,
ISystemClock clock,
VehicleQuotesContext context
) : <span style="color:#080;font-weight:bold">base</span>(options, logger, encoder, clock)
{
<span style="color:#00d;font-weight:bold">_</span>context = context;
}
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">async</span> Task<AuthenticateResult> HandleAuthenticateAsync()
{
<span style="color:#080;font-weight:bold">if</span> (!Request.Headers.ContainsKey(API_KEY_HEADER))
{
<span style="color:#080;font-weight:bold">return</span> AuthenticateResult.Fail(<span style="color:#d20;background-color:#fff0f0">"Header Not Found."</span>);
}
<span style="color:#888;font-weight:bold">string</span> apiKeyToValidate = Request.Headers[API_KEY_HEADER];
<span style="color:#888;font-weight:bold">var</span> apiKey = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.UserApiKeys
.Include(uak => uak.User)
.SingleOrDefaultAsync(uak => uak.Value == apiKeyToValidate);
<span style="color:#080;font-weight:bold">if</span> (apiKey == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> AuthenticateResult.Fail(<span style="color:#d20;background-color:#fff0f0">"Invalid key."</span>);
}
<span style="color:#080;font-weight:bold">return</span> AuthenticateResult.Success(CreateTicket(apiKey.User));
}
<span style="color:#080;font-weight:bold">private</span> AuthenticationTicket CreateTicket(IdentityUser user)
{
<span style="color:#888;font-weight:bold">var</span> claims = <span style="color:#080;font-weight:bold">new</span>[] {
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.NameIdentifier, user.Id),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.Name, user.UserName),
<span style="color:#080;font-weight:bold">new</span> Claim(ClaimTypes.Email, user.Email)
};
<span style="color:#888;font-weight:bold">var</span> identity = <span style="color:#080;font-weight:bold">new</span> ClaimsIdentity(claims, Scheme.Name);
<span style="color:#888;font-weight:bold">var</span> principal = <span style="color:#080;font-weight:bold">new</span> ClaimsPrincipal(identity);
<span style="color:#888;font-weight:bold">var</span> ticket = <span style="color:#080;font-weight:bold">new</span> AuthenticationTicket(principal, Scheme.Name);
<span style="color:#080;font-weight:bold">return</span> ticket;
}
}
}
</code></pre></div><p>The first thing to note is how we’ve defined the base class: <code>AuthenticationHandler<AuthenticationSchemeOptions></code>. <code>AuthenticationHandler</code> is a generic, and it allows us to control the type of options object that it accepts. In this case, we’ve used <code>AuthenticationSchemeOptions</code>, which is just a default one already provided by the framework. This is because we don’t really need any custom options.</p>
<blockquote>
<p>If we did, then we would define a new class that inherits from <code>AuthenticationSchemeOptions</code>, give it the new fields that we need, and use that as a generic time parameter instead. We would then be able to set values for these new fields during configuration in the app’s <code>Startup</code> class. Just like we’ve done with the “Jwt Bearer” auth scheme via the <code>AddJwtBearer</code> method.</p>
</blockquote>
<p>The next point of interest in this class is its constructor. The key aspect of it is that it calls base and passes it a series of parameters that it itself gets. This is because <code>AuthenticationHandler</code> does have some constructor logic that needs to be run for things to work properly, so we make sure to do that.</p>
<p>Finally, there’s the <code>HandleAuthenticateAsync</code> method which does the actual authentication. This method inspects the incoming request looking for an “Api-Key” header. If it finds it, it takes its value and makes a query into the database to try and find a record in the <code>user_api_keys</code> table that matches the incoming value. If it finds that, it creates an <code>AuthenticationTicket</code> which contains the identity of the user that the key belongs to, and returns it wrapped in a <code>AuthenticateResult.Success</code>. That return value signals ASP.NET Core’s authentication middleware that the request is authentic. Otherwise, it returns <code>AuthenticateResult.Fail</code>, which prompts ASP.NET Core to halt the request and return a 401.</p>
<p>Something interesting to note is how the <code>AuthenticationTicket</code>, similarly to the JWT Bearer scheme’s <code>JwtSecurityToken</code>, also includes an array of “claims” that serve to identify the user that has been authenticated. The “claims” concept is central to how auth works in ASP.NET Core.</p>
<h5 id="enabling-and-configuring-the-api-key-authentication">Enabling and configuring the API Key authentication</h5>
<p>Now we need to tell the framework that we want it to use a new authentication scheme for our app, spearheaded by the custom authentication handler that we just wrote. There are two steps to make that happen.</p>
<p>First, we configure our new scheme in the app’s <code>Startup</code> class. For that, we need these two new <code>using</code> statements:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Authentication</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Authentication.ApiKey</span>;
</code></pre></div><p>Then we add this right after the <code>AddJwtBearer</code> call in the <code>ConfigureServices</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> public void ConfigureServices(IServiceCollection services)
{
// ...
services
.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
.AddJwtBearer(options =>
{
options.TokenValidationParameters = new TokenValidationParameters()
{
ValidateIssuer = true,
ValidateAudience = true,
ValidateLifetime = true,
ValidateIssuerSigningKey = true,
ValidAudience = Configuration["Jwt:Audience"],
ValidIssuer = Configuration["Jwt:Issuer"],
IssuerSigningKey = new SymmetricSecurityKey(
Encoding.UTF8.GetBytes(Configuration["Jwt:Key"])
)
};
})
<span style="color:#000;background-color:#dfd">+ .AddScheme<AuthenticationSchemeOptions, ApiKeyAuthenticationHandler>(
</span><span style="color:#000;background-color:#dfd">+ "ApiKey",
</span><span style="color:#000;background-color:#dfd">+ options => { }
</span><span style="color:#000;background-color:#dfd">+ );
</span><span style="color:#000;background-color:#dfd"></span> }
</code></pre></div><p>A simple configuration as far as auth schemes go. We specify both the types of the authentication handler and the options object that it accepts via the generic type parameters to the <code>AddScheme</code> method. <code>"ApiKey"</code> is just the name we’ve given our custom auth scheme. It can be anything as long as it’s not “Bearer”, which is the name of the Jwt Bearer scheme (which is the value returned by <code>JwtBearerDefaults.AuthenticationScheme</code>). We’ll refer back to it later. Finally, since we have no special options to give it, we specify a lambda that does nothing with its options.</p>
<h5 id="updating-the-authorize-attribute-to-use-our-two-schemes">Updating the Authorize attribute to use our two schemes</h5>
<p>Now that the auth scheme is configured, we need to use it to secure an endpoint. We can use the same that we used for JWT: <code>POST api/BodyTypes</code>, defined in <code>BodyTypesController</code>’s <code>GetBodyTypes</code> action method. Remember though, that we wanted to have both auth schemes (JWT and API Key) work on the endpoint. For that, the <code>Authorize</code> attribute allows a comma separated string of scheme names as a parameter. So, we can get both our configured schemes working if we update the attribute like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#369">[Authorize(AuthenticationSchemes = $"{JwtBearerDefaults.AuthenticationScheme},ApiKey")]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<IEnumerable<BodyType>>> GetBodyTypes()
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.BodyTypes.ToListAsync();
}
</code></pre></div><p><code>JwtBearerDefaults.AuthenticationScheme</code> contains the name of the JWT Bearer auth scheme. Next to it, we just put “ApiKey”, which is the name we have given our new custom one.</p>
<blockquote>
<p>It’d be nice to put that “ApiKey” string somewhere safe similar to how the “Jwt Bearer” scheme has it in a constant. Take a look at the <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/tree/json-web-token">finished project on GitHub</a> to see <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/commit/8e9432068bfad977980c7c28702773a25ee293b8">one way to organize that</a> and also how to <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/commit/05138e927986fc191d4a042a3455ed1ee1947b8f">make the <code>Startup</code> configuration a little less verbose</a> by using extension methods.</p>
</blockquote>
<p>And finally, let’s test our endpoint and marvel at the fruit of our work.</p>
<p>Make sure to create a new API Key by POSTing to <code>api/Users/ApiKey</code>, and copy the “value” from the response. We can use it as the value for the <code>Api-Key</code> header a GET request to <code>api/BodyTypes</code>. In Postman, we can do that by choosing the “Authorization” tab, selecting “API Key” in the “Type” drop-down list, writing “Api-Key” in the “Key” text box, and putting our key in the “Value” text box.</p>
<p>With that, we can make the request and see how the endpoint now allows authentication via API Keys:</p>
<p><img src="/blog/2022/06/implementing-authentication-in-asp.net-core-web-apis/authorized-response-api-key.webp" alt="Successful response for request authenticated with API Key"></p>
<p>Of course, requests authenticated via JWT will also work for this endpoint.</p>
<h3 id="thats-all">That’s all</h3>
<p>And there you have it! We’ve updated an existing ASP.NET Core Web API application so that it supports authentication using two strategies: JWT and API Keys. We leveraged the Identity libraries to securely store and manage user accounts. We used ASP.NET Core’s built-in authentication capabilities to enable JWT generation and usage. For API Keys, the framework didn’t provide an implementation out of the box. However, it proved to be extensible enough so that we could implement our own.</p>
<h3 id="table-of-contents">Table of contents</h3>
<ul>
<li><a href="#two-approaches-to-authentication-jwt-and-api-keys">Two approaches to authentication: JWT and API Keys</a></li>
<li><a href="#managing-user-accounts-with-aspnet-core-identity">Managing user accounts with ASP.NET Core Identity</a>
<ul>
<li><a href="#install-the-necessary-nuget-packages">Install the necessary NuGet packages</a></li>
<li><a href="#update-the-dbcontext-to-include-the-identity-tables">Update the DbContext to include the Identity tables</a></li>
<li><a href="#applying-database-changes">Applying database changes</a></li>
<li><a href="#configuring-the-identity-services">Configuring the Identity services</a></li>
<li><a href="#creating-users">Creating users</a></li>
<li><a href="#fetching-users">Fetching users</a></li>
</ul>
</li>
<li><a href="#implementing-jwt-bearer-token-authentication">Implementing JWT Bearer Token authentication</a>
<ul>
<li><a href="#creating-tokens">Creating tokens</a>
<ul>
<li><a href="#the-request-data-structure">The request data structure</a></li>
<li><a href="#the-response-data-structure">The response data structure</a></li>
<li><a href="#the-class-that-creates-jwts">The class that creates JWTs</a></li>
<li><a href="#the-action-method-that-puts-it-all-together">The action method that puts it all together</a></li>
</ul>
</li>
<li><a href="#securing-an-endpoint-with-jwt-authentication">Securing an endpoint with JWT authentication</a>
<ul>
<li><a href="#applying-the-authorize-attribute">Applying the Authorize attribute</a></li>
<li><a href="#enabling-and-configuring-the-jwt-authentication">Enabling and configuring the JWT authentication</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#implementing-api-key-authentication">Implementing API Key authentication</a>
<ul>
<li><a href="#creating-api-keys">Creating API Keys</a>
<ul>
<li><a href="#the-api-key-model-and-database-table">The API Key model and database table</a></li>
<li><a href="#the-class-that-creates-api-keys">The class that creates API Keys</a></li>
<li><a href="#the-action-method-that-puts-it-all-together-1">The action method that puts it all together</a></li>
</ul>
</li>
<li><a href="#securing-an-endpoint-with-api-key-authentication">Securing an endpoint with API Key authentication</a>
<ul>
<li><a href="#implementing-the-api-key-authentication-handler">Implementing the API Key authentication handler</a></li>
<li><a href="#enabling-and-configuring-the-api-key-authentication">Enabling and configuring the API Key authentication</a></li>
<li><a href="#updating-the-authorize-attribute-to-use-our-two-schemes">Updating the Authorize attribute to use our two schemes</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#thats-all">That’s all</a></li>
</ul>
Database integration testing with .NEThttps://www.endpointdev.com/blog/2022/01/database-integration-testing-with-dotnet/2022-01-12T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2022/01/database-integration-testing-with-dotnet/banner.jpg" alt="Sunset over lake in mountains"></p>
<!-- Image by Zed Jensen, 2021 -->
<p><a href="https://rubyonrails.org/">Ruby on Rails</a> is great. We use it at End Point for many projects with great success. One of Rails’ cool features is how easy it is to write database integration tests. Out of the box, Rails projects come with all the configuration necessary to set up a database that’s exclusive for the automated test suite. This database includes all the tables and other objects that exist within the regular database that the app uses during its normal execution. So, it is very easy to write automated tests that cover the application components that interact with the database.</p>
<p><a href="https://dotnet.microsoft.com/en-us/apps/aspnet">ASP.NET Core</a> is also great! However, it doesn’t have this feature out of the box. Let’s see if we can’t do it ourselves.</p>
<h3 id="the-sample-project">The sample project</h3>
<p>As a sample project we will use a REST API that I wrote for <a href="/blog/2021/07/dotnet-5-web-api/">another article</a>. Check it out if you want to learn more about the ins and outs of developing REST APIs with .NET. You can find the source code <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">on GitHub</a>.</p>
<p>The API is very straightforward. It provides a few endpoints for <a href="https://developer.mozilla.org/en-US/docs/Glossary/CRUD">CRUDing</a> some database tables. It also provides an endpoint which, when given some vehicle information, will calculate a monetary value for that vehicle. That’s a feature that would be interesting for us to cover with some tests.</p>
<p>The logic for that feature is backed by a specific class and it depends heavily on database interactions. As such, that class is a great candidate for writing a few automated integration tests against. The class in question is <code>QuotesService</code> which is defined in <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Services/QuoteService.cs">Services/QuoteService.cs</a>. The class provides features for fetching records from the database (the <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Services/QuoteService.cs#L23"><code>GetAllQuotes</code></a> method) as well as creating new records based on data from the incoming request and a set of rules stored in the database itself (the <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Services/QuoteService.cs#L53"><code>CalculateQuote</code></a> method).</p>
<p>In order to add automated tests, the first step is to organize our project so that it supports them. Let’s do that next.</p>
<h3 id="organizing-the-source-code-to-allow-for-automated-testing">Organizing the source code to allow for automated testing</h3>
<p>In general, the source code of most real world .NET applications is organized as one or more “projects” under one “solution”. A solution is a collection of related projects, and a project is something that produces a deployment artifact. An artifact is a library (i.e. a <code>*.dll</code> file) or something that can be executed like a console or web app.</p>
<p>Our sample app is a stand-alone <a href="https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api?view=aspnetcore-6.0&tabs=visual-studio-code">“webapi” project</a>, meaning that it’s not within a solution. For automated tests, however, we need to create a new project for tests, parallel to our main one. Now that we have two projects instead of one, we need to reorganize the sample app’s source code to comply with the “projects in a solution” structure I mentioned earlier.</p>
<p>Let’s start by moving all the files in the root directory into a new <code>VehicleQuotes</code> directory. That’s one project. Then, we create a new automated tests project by running the following, still from the root directory:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">dotnet new xunit -o VehicleQuotes.Tests
</code></pre></div><p>That creates a new automated tests project named <code>VehicleQuotes.Tests</code> (under a new aptly-named <code>VehicleQuotes.Tests</code> directory) which uses the <a href="https://xunit.net">xUnit.net</a> test framework. There are other options when it comes to test frameworks in .NET, such as <a href="https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-mstest">MSTest</a> and <a href="https://nunit.org/">NUnit</a>. We’re going to use xUnit.net, but the others should work just as well for our purposes.</p>
<p>Now, we need to create a new solution to contain those two projects. Solutions come in the form of <code>*.sln</code> files and we can create ours like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">dotnet new sln -o vehicle-quotes
</code></pre></div><p>That should’ve created a new <code>vehicle-quotes.sln</code> file for us. We should now have a file structure like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">.
├── vehicle-quotes.sln
├── VehicleQuotes
│ ├── VehicleQuotes.csproj
│ └── ...
└── VehicleQuotes.Tests
├── VehicleQuotes.Tests.csproj
└── ...
</code></pre></div><p>Like I said, the <code>*.sln</code> file indicates that this is a solution. The <code>*.csproj</code> files identify the individual projects that make up the solution.</p>
<p>Now, we need to tell dotnet that those two projects belong in the same solution. These commands do that:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">dotnet sln add ./VehicleQuotes/VehicleQuotes.csproj
dotnet sln add ./VehicleQuotes.Tests/VehicleQuotes.Tests.csproj
</code></pre></div><p>Finally, we update the <code>VehicleQuotes.Tests</code> project so that it references the <code>VehicleQuotes</code> project. That way, the test suite will have access to all the classes defined in the REST API. Here’s the command for that:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">dotnet add ./VehicleQuotes.Tests/VehicleQuotes.Tests.csproj reference ./VehicleQuotes/VehicleQuotes.csproj
</code></pre></div><p>With all that setup out of the way, we can now start writing some tests.</p>
<blockquote>
<p>You can learn more about project organization in the <a href="https://docs.microsoft.com/en-us/dotnet/core/tutorials/testing-with-cli">official online documentation</a>.</p>
</blockquote>
<h3 id="creating-a-dbcontext-instance-to-talk-to-the-database">Creating a DbContext instance to talk to the database</h3>
<p>The <code>VehicleQuotes.Tests</code> automated tests project got created with a default test file named <code>UnitTest1.cs</code>. You can delete it or ignore it, since we will not use it.</p>
<p>In general, it’s a good idea for the test project to mimic the directory structure of the project that it will be testing. Also, we already decided that we would focus our test efforts on the <code>QuoteService</code> class from the <code>VehicleQuotes</code> project. That class is defined in <code>VehicleQuotes/Services/QuoteService.cs</code>, so let’s create a similarly located file within the test project which will contain the test cases for that class. Here: <code>VehicleQuotes.Tests/Services/QuoteServiceTests.cs</code>. These would be the contents:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// VehicleQuotes.Tests/Services/QuoteServiceTests.cs
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Xunit</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Tests.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuoteServiceTests</span>
{
<span style="color:#369"> [Fact]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> GetAllQuotesReturnsEmptyWhenThereIsNoDataStored()
{
<span style="color:#888">// Given
</span><span style="color:#888"></span>
<span style="color:#888">// When
</span><span style="color:#888"></span>
<span style="color:#888">// Then
</span><span style="color:#888"></span> }
}
}
</code></pre></div><p>This is the basic structure for tests using xUnit.net. Any method annotated with a <code>[Fact]</code> attribute will be picked up and run by the test framework. In this case, I’ve created one such method called <code>GetAllQuotesReturnsEmptyWhenThereIsNoDataStored</code> which should give away its intention. This test case will validate that <code>QuoteService</code>’s <code>GetAllQuotes</code> method returns an empty set when called with no data in the database.</p>
<p>Before we can write this test case, though, the suite needs access to the test database. Our app uses <a href="https://docs.microsoft.com/en-us/ef/core/">Entity Framework Core</a> for database interaction, which means that the database is accessed via a <code>DbContext</code> class. Looking at the source code of our sample app, we can see that the <code>DbContext</code> being used is <code>VehicleQuotesContext</code>, defined in <code>VehicleQuotes/Data/VehicleQuotesContext.cs</code>. Let’s add a utility method to the <code>QuoteServiceTests</code> class which can be used to create new instances of <code>VehicleQuotesContext</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// VehicleQuotes.Tests/Services/QuoteServiceTests.cs
</span><span style="color:#888"></span>
<span style="color:#888">// ...
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Services</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Tests.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuoteServiceTests</span>
{
<span style="color:#080;font-weight:bold">private</span> VehicleQuotesContext CreateDbContext()
{
<span style="color:#888;font-weight:bold">var</span> options = <span style="color:#080;font-weight:bold">new</span> DbContextOptionsBuilder<VehicleQuotesContext>()
.UseNpgsql(<span style="color:#d20;background-color:#fff0f0">"Host=db;Database=vehicle_quotes_test;Username=vehicle_quotes;Password=password"</span>)
.UseSnakeCaseNamingConvention()
.Options;
<span style="color:#888;font-weight:bold">var</span> context = <span style="color:#080;font-weight:bold">new</span> VehicleQuotesContext(options);
context.Database.EnsureCreated();
<span style="color:#080;font-weight:bold">return</span> context;
}
<span style="color:#888">// ...
</span><span style="color:#888"></span> }
}
</code></pre></div><p>As you can see, we need to go through three steps to create the <code>VehicleQuotesContext</code> instance and get a database that’s ready for testing:</p>
<p>First, we create a <code>DbContextOptionsBuilder</code> and use that to obtain the <code>options</code> object that the <code>VehicleQuotesContext</code> needs as a constructor parameter. We needed to include the <code>Microsoft.EntityFrameworkCore</code> namespace in order to have access to the <code>DbContextOptionsBuilder</code>. For this, I just copied and slightly modified this statement from the <code>ConfigureServices</code> method in the REST API’s <code>VehicleQuotes/Startup.cs</code> file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// VehicleQuotes/Startup.cs
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> ConfigureServices(IServiceCollection services)
{
<span style="color:#888">// ...
</span><span style="color:#888"></span>
services.AddDbContext<VehicleQuotesContext>(options =>
options
.UseNpgsql(Configuration.GetConnectionString(<span style="color:#d20;background-color:#fff0f0">"VehicleQuotesContext"</span>))
.UseSnakeCaseNamingConvention()
.UseLoggerFactory(LoggerFactory.Create(builder => builder.AddConsole()))
.EnableSensitiveDataLogging()
);
<span style="color:#888">// ...
</span><span style="color:#888"></span>}
</code></pre></div><p>This is a method that runs when the application is starting up to set up all the services that the app uses to work. Here, it’s setting up the <code>DbContext</code> to enable database interaction. For the test suite, I took this statement as a starting point and removed the logging configurations and specified a hardcoded connection string that specifically points to a new <code>vehicle_quotes_test</code> database that will be used for testing.</p>
<p>If you’re following along, then you need a PostgreSQL instance that you can use to run the tests. In my case, I have one running that is reachable via the connection string I specified: <code>Host=db;Database=vehicle_quotes_test;Username=vehicle_quotes;Password=password</code>.</p>
<p>If you have Docker, a quick way to get a Postgres database up and running is with this command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sh" data-lang="sh">docker run -d <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> --name vehicle-quotes-db <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -p 5432:5432 <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> --network host <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -e <span style="color:#369">POSTGRES_DB</span>=vehicle_quotes <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -e <span style="color:#369">POSTGRES_USER</span>=vehicle_quotes <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -e <span style="color:#369">POSTGRES_PASSWORD</span>=password <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> postgres
</code></pre></div><p>That’ll spin up a new Postgres instance that’s reachable via <code>localhost</code>.</p>
<p>Secondly, now that we have the options parameter ready, we can quite simply instantiate a new <code>VehicleQuotesContext</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888;font-weight:bold">var</span> context = <span style="color:#080;font-weight:bold">new</span> VehicleQuotesContext(options);
</code></pre></div><p>Finally, we call the <code>EnsureCreated</code> method so that the database that we specified in the connection string is actually created.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp">context.Database.EnsureCreated();
</code></pre></div><p>This is the database that our test suite will use.</p>
<h3 id="defining-the-test-database-connection-string-in-the-appsettingsjson-file">Defining the test database connection string in the appsettings.json file</h3>
<p>One quick improvement that we can do to the code we’ve written so far is move the connection string for the test database into a separate configuration file, instead of having it hardcoded. Let’s do that next.</p>
<p>We need to create a new <code>appsettings.json</code> file under the <code>VehicleQuotes.Tests</code> directory. Then we have to add the connection string like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"ConnectionStrings"</span>: {
<span style="color:#b06;font-weight:bold">"VehicleQuotesContext"</span>: <span style="color:#d20;background-color:#fff0f0">"Host=db;Database=vehicle_quotes_test;Username=vehicle_quotes;Password=password"</span>
}
}
</code></pre></div><p>This is the standard way of configuring connection strings in .NET. Now, to actually fetch this value from within our test suite code, we make the following changes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">// ...
<span style="color:#000;background-color:#dfd">+using Microsoft.Extensions.Hosting;
</span><span style="color:#000;background-color:#dfd">+using Microsoft.Extensions.Configuration;
</span><span style="color:#000;background-color:#dfd">+using Microsoft.Extensions.DependencyInjection;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Tests.Services
{
public class QuoteServiceTests
{
private VehicleQuotesContext CreateDbContext()
{
<span style="color:#000;background-color:#dfd">+ var host = Host.CreateDefaultBuilder().Build();
</span><span style="color:#000;background-color:#dfd">+ var config = host.Services.GetRequiredService<IConfiguration>();
</span><span style="color:#000;background-color:#dfd"></span>
var options = new DbContextOptionsBuilder<VehicleQuotesContext>()
<span style="color:#000;background-color:#fdd">- .UseNpgsql("Host=db;Database=vehicle_quotes_test;Username=vehicle_quotes;Password=password")
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ .UseNpgsql(config.GetConnectionString("VehicleQuotesContext"))
</span><span style="color:#000;background-color:#dfd"></span> .UseSnakeCaseNamingConvention()
.Options;
var context = new VehicleQuotesContext(options);
context.Database.EnsureCreated();
return context;
}
// ...
}
}
</code></pre></div><p>First we add a few <code>using</code> statements. We need <code>Microsoft.Extensions.Hosting</code> so that we can have access to the <code>Host</code> class through which we obtain access to the application’s execution context. This allows us to access the built-in configuration service. We also need <code>Microsoft.Extensions.Configuration</code> to have access to the <code>IConfiguration</code> interface which is how we reference the configuration service which allows us access to the <code>appsettings.json</code> config file. And we also need the <code>Microsoft.Extensions.DependencyInjection</code> namespace which allows us to tap into the built-in dependency injection mechanism, through which we can access the default configuration service I mentioned before. Specifically, that namespace is where the <code>GetRequiredService</code> extension method lives.</p>
<p>All this translates into the few code changes that you see in the previous diff: first getting the app’s host, then getting the configuration service, then using that to fetch our connection string.</p>
<blockquote>
<p>You can refer to <a href="https://docs.microsoft.com/en-us/dotnet/core/extensions/configuration">the official documentation</a> to learn more about configuration in .NET.</p>
</blockquote>
<h3 id="writing-a-simple-test-case-that-fetches-data">Writing a simple test case that fetches data</h3>
<p>Now that we have a way to access the database from within the test suite, we can finally write an actual test case. Here’s the <code>GetAllQuotesReturnsEmptyWhenThereIsNoDataStored</code> one that I alluded to earlier:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// ...
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Tests.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuoteServiceTests</span>
{
<span style="color:#888">// ...
</span><span style="color:#888"></span><span style="color:#369">
</span><span style="color:#369"> [Fact]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> <span style="color:#080;font-weight:bold">void</span> GetAllQuotesReturnsEmptyWhenThereIsNoDataStored()
{
<span style="color:#888">// Given
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> dbContext = CreateDbContext();
<span style="color:#888;font-weight:bold">var</span> service = <span style="color:#080;font-weight:bold">new</span> QuoteService(dbContext, <span style="color:#080;font-weight:bold">null</span>);
<span style="color:#888">// When
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> result = <span style="color:#080;font-weight:bold">await</span> service.GetAllQuotes();
<span style="color:#888">// Then
</span><span style="color:#888"></span> Assert.Empty(result);
}
}
}
</code></pre></div><p>This one is a very simple test. We obtain a new <code>VehicleQuotesContext</code> instance that we can use to pass as a parameter when instantiating the component that we want to test: the <code>QuoteService</code>. We then call the <code>GetAllQuotes</code> method and assert that it returned an empty set. The test database was just created, so there should be no data in it, hence the empty resource set.</p>
<p>To run this test, we do <code>dotnet test</code>. I personally like a more verbose output so I like to use this variant of the command: <code>dotnet test --logger "console;verbosity=detailed"</code>. Here’s what the output looks like.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plaintext" data-lang="plaintext">$ dotnet test --logger "console;verbosity=detailed"
Determining projects to restore...
All projects are up-to-date for restore.
VehicleQuotes -> /app/VehicleQuotes/bin/Debug/net5.0/VehicleQuotes.dll
VehicleQuotes.Tests -> /app/VehicleQuotes.Tests/bin/Debug/net5.0/VehicleQuotes.Tests.dll
Test run for /app/VehicleQuotes.Tests/bin/Debug/net5.0/VehicleQuotes.Tests.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 16.11.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
/app/VehicleQuotes.Tests/bin/Debug/net5.0/VehicleQuotes.Tests.dll
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit .NET 5.0.12)
[xUnit.net 00:00:01.03] Discovering: VehicleQuotes.Tests
[xUnit.net 00:00:01.06] Discovered: VehicleQuotes.Tests
[xUnit.net 00:00:01.06] Starting: VehicleQuotes.Tests
[xUnit.net 00:00:03.25] Finished: VehicleQuotes.Tests
Passed VehicleQuotes.Tests.Services.QuoteServiceTests.GetAllQuotesReturnsEmptyWhenThereIsNoDataStored [209 ms]
Test Run Successful.
Total tests: 1
Passed: 1
Total time: 3.7762 Seconds
</code></pre></div><h3 id="resetting-the-state-of-the-database-after-each-test">Resetting the state of the database after each test</h3>
<p>Now we need to write a test that actually writes data into the database. However, every test case needs to start with the database in its original state. In other words, the changes that one test case does to the test database should not be seen, affect, or be expected by any subsequent test. That will make it so our test cases are isolated and repeatable. That’s not possible with our current implementation, though.</p>
<blockquote>
<p>You can read more about the FIRST principles of testing <a href="https://medium.com/@tasdikrahman/f-i-r-s-t-principles-of-testing-1a497acda8d6">here</a>.</p>
</blockquote>
<p>Luckily, that’s a problem that’s easily solved with Entity Framework Core. All we need to do is call a method that ensures that the database is deleted just before it ensures that it is created. Here’s what it looks like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> private VehicleQuotesContext CreateDbContext()
{
var host = Host.CreateDefaultBuilder().Build();
var config = host.Services.GetRequiredService<IConfiguration>();
var options = new DbContextOptionsBuilder<VehicleQuotesContext>()
.UseNpgsql(config.GetConnectionString("VehicleQuotesContext"))
.UseSnakeCaseNamingConvention()
.Options;
var context = new VehicleQuotesContext(options);
<span style="color:#000;background-color:#dfd">+ context.Database.EnsureDeleted();
</span><span style="color:#000;background-color:#dfd"></span> context.Database.EnsureCreated();
return context;
}
</code></pre></div><p>And that’s all. Now every test case that calls <code>CreateDbContext</code> in order to obtain a <code>DbContext</code> instance will effectively trigger a database reset. Feel free to <code>dotnet test</code> again to validate that the test suite is still working.</p>
<p>Now, depending on the size of the database, this can be quite expensive. For integration tests, performance is not as big of a concern as for unit tests. This is because integration tests should be fewer in number and less frequently run.</p>
<p>We can make it better though. Instead of deleting and recreating the database before each test case, we’ll take a page out of Ruby on Rails’ book and run each test case within a database transaction which gets rolled back after the test is done. For now though, let’s write another test case: this time, one where we insert new records into the database.</p>
<blockquote>
<p>If you want to hear a more in-depth discussion about automated testing in general, I go into further detail on the topic in this article: <a href="/blog/2020/09/automated-testing-with-symfony/">An introduction to automated testing for web applications with Symfony</a>.</p>
</blockquote>
<h3 id="writing-another-simple-test-case-that-stores-data">Writing another simple test case that stores data</h3>
<p>Now let’s write another test that exercises <code>QuoteService</code>’s <code>GetAllQuotes</code> method. This time though, let’s add a new record to the database before calling it so that the method’s result is not empty. Here’s what the test looks like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// ...
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Linq</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Tests.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuoteServiceTests</span>
{
<span style="color:#888">// ...
</span><span style="color:#888"></span><span style="color:#369">
</span><span style="color:#369"> [Fact]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> <span style="color:#080;font-weight:bold">void</span> GetAllQuotesReturnsTheStoredData()
{
<span style="color:#888">// Given
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> dbContext = CreateDbContext();
<span style="color:#888;font-weight:bold">var</span> quote = <span style="color:#080;font-weight:bold">new</span> Quote
{
OfferedQuote = <span style="color:#00d;font-weight:bold">100</span>,
Message = <span style="color:#d20;background-color:#fff0f0">"test_quote_message"</span>,
Year = <span style="color:#d20;background-color:#fff0f0">"2000"</span>,
Make = <span style="color:#d20;background-color:#fff0f0">"Toyota"</span>,
Model = <span style="color:#d20;background-color:#fff0f0">"Corolla"</span>,
BodyTypeID = dbContext.BodyTypes.Single(bt => bt.Name == <span style="color:#d20;background-color:#fff0f0">"Sedan"</span>).ID,
SizeID = dbContext.Sizes.Single(s => s.Name == <span style="color:#d20;background-color:#fff0f0">"Compact"</span>).ID,
ItMoves = <span style="color:#080;font-weight:bold">true</span>,
HasAllWheels = <span style="color:#080;font-weight:bold">true</span>,
HasAlloyWheels = <span style="color:#080;font-weight:bold">true</span>,
HasAllTires = <span style="color:#080;font-weight:bold">true</span>,
HasKey = <span style="color:#080;font-weight:bold">true</span>,
HasTitle = <span style="color:#080;font-weight:bold">true</span>,
RequiresPickup = <span style="color:#080;font-weight:bold">true</span>,
HasEngine = <span style="color:#080;font-weight:bold">true</span>,
HasTransmission = <span style="color:#080;font-weight:bold">true</span>,
HasCompleteInterior = <span style="color:#080;font-weight:bold">true</span>,
CreatedAt = DateTime.Now
};
dbContext.Quotes.Add(quote);
dbContext.SaveChanges();
<span style="color:#888;font-weight:bold">var</span> service = <span style="color:#080;font-weight:bold">new</span> QuoteService(dbContext, <span style="color:#080;font-weight:bold">null</span>);
<span style="color:#888">// When
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> result = <span style="color:#080;font-weight:bold">await</span> service.GetAllQuotes();
<span style="color:#888">// Then
</span><span style="color:#888"></span> Assert.NotEmpty(result);
Assert.Single(result);
Assert.Equal(quote.ID, result.First().ID);
Assert.Equal(quote.OfferedQuote, result.First().OfferedQuote);
Assert.Equal(quote.Message, result.First().Message);
}
}
}
</code></pre></div><p>First we include the <code>VehicleQuotes.Models</code> namespace so that we can use the <code>Quotes</code> model class. In our REST API, this is the class that represents the data from the <code>quotes</code> table. This is the main table that <code>GetAllQuotes</code> queries. We also include the <code>System.Linq</code> namespace, which allows us to use various collection extension methods (like <code>Single</code> and <code>First</code>) which we leverage throughout the test case to query lookup tables and assert on the test results.</p>
<p>Other than that, the test case itself is pretty self-explanatory. We start by obtaining an instance of <code>VehicleQuotesContext</code> via the <code>CreateDbContext</code> method. Remember that this also resets the whole database so that the test case can run over a clean slate. Then, we create a new <code>Quote</code> object and use our <code>VehicleQuotesContext</code> to insert it as a record into the database. We do this so that the later call to <code>QuoteService</code>’s <code>GetAllQuotes</code> method actually finds some data to return this time. Finally, the test case validates that the result contains a record and that its data is identical to what we set manually.</p>
<p>Neat! At this point we have what I think is the bare minimum infrastructure when it comes to serviceable and effective database integration tests, namely, access to a test database. We can take it one step further, though, and make things more reusable and a little bit better performing.</p>
<h3 id="refactoring-into-a-fixture-for-reusability">Refactoring into a fixture for reusability</h3>
<p>We can use the test fixture functionality offered by xUnit.net in order to make the database interactivity aspect of our test suite into a reusable component. That way, if we had other test classes focused on other components that interact with the database, we could just plug that code in. We can define a fixture by creating a new file called, for example, <code>VehicleQuotes.Tests/Fixtures/DatabaseFixture.cs</code> with these contents:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Hosting</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Configuration</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.DependencyInjection</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Tests.Fixtures</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">DatabaseFixture</span> : IDisposable
{
<span style="color:#080;font-weight:bold">public</span> VehicleQuotesContext DbContext { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DatabaseFixture()
{
DbContext = CreateDbContext();
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Dispose()
{
DbContext.Dispose();
}
<span style="color:#080;font-weight:bold">private</span> VehicleQuotesContext CreateDbContext()
{
<span style="color:#888;font-weight:bold">var</span> host = Host.CreateDefaultBuilder().Build();
<span style="color:#888;font-weight:bold">var</span> config = host.Services.GetRequiredService<IConfiguration>();
<span style="color:#888;font-weight:bold">var</span> options = <span style="color:#080;font-weight:bold">new</span> DbContextOptionsBuilder<VehicleQuotesContext>()
.UseNpgsql(config.GetConnectionString(<span style="color:#d20;background-color:#fff0f0">"VehicleQuotesContext"</span>))
.UseSnakeCaseNamingConvention()
.Options;
<span style="color:#888;font-weight:bold">var</span> context = <span style="color:#080;font-weight:bold">new</span> VehicleQuotesContext(options);
context.Database.EnsureDeleted();
context.Database.EnsureCreated();
<span style="color:#080;font-weight:bold">return</span> context;
}
}
}
</code></pre></div><p>All this class does is define the <code>CreateDbContext</code> method that we’re already familiar with but puts it in a nice reusable package. Upon instantiation, as seen in the constructor, it stores a reference to the <code>VehicleQuotesContext</code> in its <code>DbContext</code> property.</p>
<p>With that, our <code>QuoteServiceTests</code> test class can use it if we make the following changes to it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> using System;
using Xunit;
<span style="color:#000;background-color:#fdd">-using Microsoft.EntityFrameworkCore;
</span><span style="color:#000;background-color:#fdd"></span> using VehicleQuotes.Services;
<span style="color:#000;background-color:#fdd">-using Microsoft.Extensions.Hosting;
</span><span style="color:#000;background-color:#fdd">-using Microsoft.Extensions.Configuration;
</span><span style="color:#000;background-color:#fdd">-using Microsoft.Extensions.DependencyInjection;
</span><span style="color:#000;background-color:#fdd"></span> using VehicleQuotes.Models;
using System.Linq;
<span style="color:#000;background-color:#dfd">+using VehicleQuotes.Tests.Fixtures;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Tests.Services
{
<span style="color:#000;background-color:#fdd">- public class QuoteServiceTests
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ public class QuoteServiceTests : IClassFixture<DatabaseFixture>
</span><span style="color:#000;background-color:#dfd"></span> {
<span style="color:#000;background-color:#dfd">+ private VehicleQuotesContext dbContext;
</span><span style="color:#000;background-color:#dfd"></span>
<span style="color:#000;background-color:#dfd">+ public QuoteServiceTests(DatabaseFixture fixture)
</span><span style="color:#000;background-color:#dfd">+ {
</span><span style="color:#000;background-color:#dfd">+ dbContext = fixture.DbContext;
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd"></span>
<span style="color:#000;background-color:#fdd">- private VehicleQuotesContext CreateDbContext()
</span><span style="color:#000;background-color:#fdd">- {
</span><span style="color:#000;background-color:#fdd">- var host = Host.CreateDefaultBuilder().Build();
</span><span style="color:#000;background-color:#fdd">- var config = host.Services.GetRequiredService<IConfiguration>();
</span><span style="color:#000;background-color:#fdd"></span>
<span style="color:#000;background-color:#fdd">- var options = new DbContextOptionsBuilder<VehicleQuotesContext>()
</span><span style="color:#000;background-color:#fdd">- .UseNpgsql(config.GetConnectionString("VehicleQuotesContext"))
</span><span style="color:#000;background-color:#fdd">- .UseSnakeCaseNamingConvention()
</span><span style="color:#000;background-color:#fdd">- .Options;
</span><span style="color:#000;background-color:#fdd"></span>
<span style="color:#000;background-color:#fdd">- var context = new VehicleQuotesContext(options);
</span><span style="color:#000;background-color:#fdd"></span>
<span style="color:#000;background-color:#fdd">- context.Database.EnsureDeleted();
</span><span style="color:#000;background-color:#fdd">- context.Database.EnsureCreated();
</span><span style="color:#000;background-color:#fdd"></span>
<span style="color:#000;background-color:#fdd">- return context;
</span><span style="color:#000;background-color:#fdd">- }
</span><span style="color:#000;background-color:#fdd"></span>
[Fact]
public async void GetAllQuotesReturnsEmptyWhenThereIsNoDataStored()
{
// Given
<span style="color:#000;background-color:#fdd">- var dbContext = CreateDbContext();
</span><span style="color:#000;background-color:#fdd"></span>
// ...
}
[Fact]
public async void GetAllQuotesReturnsTheStoredData()
{
// Given
<span style="color:#000;background-color:#fdd">- var dbContext = CreateDbContext();
</span><span style="color:#000;background-color:#fdd"></span>
// ...
}
}
}
</code></pre></div><p>Here we’ve updated the <code>QuoteServiceTests</code> class definition so that it inherits from <a href="https://github.com/xunit/xunit/blob/main/src/xunit.v3.core/IClassFixture.cs"><code>IClassFixture<DatabaseFixture></code></a>. This is how we tell xUnit.net that our tests use the new fixture that we created. Next, we define a constructor that receives a <code>DatabaseFixture</code> object as a parameter. That’s how xUnit.net allows our test class to access the capabilities provided by the fixture. In this case, we take the fixture’s <code>DbContext</code> instance, and store it for later use in all of the test cases that need database interaction. We also removed the <code>CreateDbContext</code> method because now that’s defined within the fixture. We also removed a few <code>using</code> statements that became unnecessary.</p>
<p>One important aspect to note about this fixture is that it is initialized once per whole test suite run, not once per test case. Specifically, the code within the <code>DatabaseFixture</code>’s constructor gets executed once, before all of the test cases. Similarly, the code in <code>DatabaseFixture</code>’s <code>Dispose</code> method get executed once at the end, after all test cases have been run.</p>
<p>This means that our test database deletion and recreation step now happens only once for the entire test suite. This is not good with our current implementation because that means that individual test cases no longer run with a fresh, empty database. This can be good for performance though, as long as we update our test cases to run within database transactions. Let’s do just that.</p>
<h3 id="using-transactions-to-reset-the-state-of-the-database">Using transactions to reset the state of the database</h3>
<p>Here’s how we update out test class so that each test case runs within a transaction:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> // ...
namespace VehicleQuotes.Tests.Services
{
<span style="color:#000;background-color:#fdd">- public class QuoteServiceTests : IClassFixture<DatabaseFixture>
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ public class QuoteServiceTests : IClassFixture<DatabaseFixture>, IDisposable
</span><span style="color:#000;background-color:#dfd"></span> {
private VehicleQuotesContext dbContext;
public QuoteServiceTests(DatabaseFixture fixture)
{
dbContext = fixture.DbContext;
<span style="color:#000;background-color:#dfd">+ dbContext.Database.BeginTransaction();
</span><span style="color:#000;background-color:#dfd"></span> }
<span style="color:#000;background-color:#dfd">+ public void Dispose()
</span><span style="color:#000;background-color:#dfd">+ {
</span><span style="color:#000;background-color:#dfd">+ dbContext.Database.RollbackTransaction();
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd"></span>
# ...
}
}
</code></pre></div><p>The first thing to note here is that we added a call to <code>BeginTransaction</code> in the test class constructor. xUnit.net creates a new instance of the test class for each test case. This means that this constructor is run before each and every test case. We use that opportunity to begin a database transaction.</p>
<p>The other interesting point is that we’ve updated the class to implement the <a href="https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/implementing-dispose"><code>IDisposable</code> interface’s <code>Dispose</code> method</a>. xUnit.net will run this code after each test case, so we rollback the transaction.</p>
<p>Put those two together and we’ve updated our test suite so that every test case runs within the context of its own database transaction. Try it out with <code>dotnet test</code> and see what happens.</p>
<blockquote>
<p>To learn more about database transactions with Entity Framework Core, you can look at <a href="https://docs.microsoft.com/en-us/ef/core/saving/transactions">the official docs</a>.</p>
<p>You can learn more about xUnit.net’s test class fixtures in <a href="https://github.com/xunit/samples.xunit/tree/main/ClassFixtureExample">the samples repository</a>.</p>
</blockquote>
<p>Alright, that’s all for now. It is great to see that implementing automated database integration tests is actually fairly straightforward using .NET, xUnit.net, and Entity Framework. Even if it isn’t quite as easy as it is in Rails, it is perfectly doable.</p>
Kubernetes 101: Deploying a web application and databasehttps://www.endpointdev.com/blog/2022/01/kubernetes-101/2022-01-08T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2022/01/kubernetes-101/2021-11-26_092823-sm.webp" alt="Groups of birds on a telephone pole"></p>
<!-- Photo by Seth Jensen -->
<p>The DevOps world seems to have been taken over by <a href="https://kubernetes.io/">Kubernetes</a> during the past few years. And rightfully so, I believe, as it is a great piece of software that promises and delivers when it comes to managing deployments of complex systems.</p>
<p>Kubernetes is hard though. But it’s all good, I’m not a DevOps engineer. As a software developer, I shouldn’t care about any of that. Or should I? Well… Yes. I know that very well after being thrown head first into a project that heavily involves Kubernetes, without knowing the first thing about it.</p>
<p>Even if I wasn’t in the role of a DevOps engineer, as a software developer, I had to work with it in order to set up dev environments, troubleshoot system issues, and make sound design and architectural decisions.</p>
<p>After a healthy amount of struggle, I eventually gained some understanding on the subject. In this blog post I’ll share what I learned. My hope is to put out there the things I wish I knew when I first encountered and had to work with Kubernetes.</p>
<p>So, I’m going to introduce the basic concepts and building blocks of Kubernetes. Then, I’m going to walk you through the process of containerizing a sample application, developing all the Kubernetes configuration files necessary for deploying it into a Kubernetes cluster, and actually deploying it into a local development cluster. We will end up with an application and its associated database running completely on and being managed by Kubernetes.</p>
<p>In short: If you know nothing about Kubernetes, and are interested in learning, read on. This post is for you.</p>
<h3 id="what-is-kubernetes">What is Kubernetes?</h3>
<p>Simply put, <a href="https://kubernetes.io/">Kubernetes</a> (or K8s) is software for managing <a href="https://en.wikipedia.org/wiki/Computer_cluster">computer clusters</a>. That is, groups of computers that are working together in order to process some workload or offer a service. Kubernetes does this by leveraging <a href="https://www.docker.com/resources/what-container">application containers</a>. Kubernetes will help you out in automating the deployment, scaling, and management of containerized applications.</p>
<p>Once you’ve designed an application’s complete execution environment and associated components, using Kubernetes you can specify all that declaratively via configuration files. Then, you’ll be able to deploy that application with a single command. Once deployed, Kubernetes will give you tools to check on the health of your application, recover from issues, keep it running, scale it, etc.</p>
<p>There are a few basic concepts that we need to be familiar with in order to effectively work with Kubernetes. I think the <a href="https://kubernetes.io/docs/concepts/">official documentation</a> does a great job in explaining this, but I’ll try to summarize.</p>
<h4 id="nodes-pods-and-containers">Nodes, pods, and containers</h4>
<p>First up are <a href="https://kubernetes.io/docs/concepts/containers/">containers</a>. If you’re interested in Kubernetes, chances are that you’ve already been exposed to some sort of container technology like <a href="https://www.docker.com/">Docker</a>. If not, no worries. For our purposes here, we can think of a container as an isolated process with its own resources and file system in which an application can run.</p>
<p>A container has all the software dependencies that an application needs to run, including the application itself. From the application’s perspective, the container is its execution environment: the “machine” in which it’s running. In more practical terms, a container is a form of packaging, delivering, and executing an application. What’s the advantage? Instead of installing the application and its dependencies directly into the machine that’s going to run it, having it containerized allows for a container runtime (like Docker) to just run it as a self-contained unit. This makes it possible for the application to run anywhere that has the container runtime installed, with minimal configuration.</p>
<p>Something very closely related to containers is the concept of <a href="https://kubernetes.io/docs/concepts/containers/images/">images</a>. You can think of images as the blueprint for containers. An image is the spec, and the container is the instance that’s actually running.</p>
<p>When deploying applications into Kubernetes, this is how it runs them: via containers. In other words, for Kubernetes to be able to run an application, it needs to be delivered within a container.</p>
<p>Next is the concept of a <a href="https://kubernetes.io/docs/concepts/architecture/">node</a>. This is very straightforward and not even specific to Kubernetes. A node is a computer within the cluster. That’s it. Like I said before, Kubernetes is built to manage computer clusters. A node is just one computer, either virtual or physical, within that cluster.</p>
<p>Then there are <a href="https://kubernetes.io/docs/concepts/workloads/pods/">pods</a>. Pods are the main executable units in Kubernetes. When we deploy an application or service into a Kubernetes cluster, it runs within a pod. Kubernetes works with containerized applications though, so it is the pods that take care of running said containers within them.</p>
<p>These three work very closely together within Kubernetes. To summarize: containers run within pods which in turn exist within nodes in the cluster.</p>
<p>There are other key components to talk about like <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">deployments</a>, <a href="https://kubernetes.io/docs/concepts/services-networking/service/">services</a>, <a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/">replica sets</a>, and <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/">persistent volumes</a>. But I think that’s enough theory for now. We’ll learn more about all these as we get our hands dirty working though our example. So let’s get started with our demo and we’ll be discovering and discussing them organically as we go through it.</p>
<h3 id="installing-and-setting-up-kubernetes">Installing and setting up Kubernetes</h3>
<p>The first thing we need is a Kubernetes environment. There are many Kubernetes implementations out there. <a href="https://cloud.google.com/kubernetes-engine">Google</a>, <a href="https://azure.microsoft.com/en-us/services/kubernetes-service/">Microsoft</a>, and <a href="https://aws.amazon.com/eks">Amazon</a> offer Kubernetes solutions on their respective cloud platforms, for example. There are also implementations that one can install and run on their own, like <a href="https://kind.sigs.k8s.io/docs/">kind</a>, <a href="https://minikube.sigs.k8s.io/docs/">minikube</a>, and <a href="https://microk8s.io/">MicroK8s</a>. We are going to use MicroK8s for our demo, for no particular reason other than “this is the one I know”.</p>
<p>When done installing, MicroK8s will have set up a whole Kubernetes cluster, with your machine as its one and only node.</p>
<h4 id="installing-microk8s">Installing MicroK8s</h4>
<p>So, if you’re in Ubuntu and have <a href="https://snapcraft.io/docs/installing-snapd">snapd</a>, installing MicroK8s is easy. The <a href="https://microk8s.io/docs">official documentation</a> explains it best. You install it with a command like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ sudo snap install microk8s --classic --channel=1.21
</code></pre></div><p>MicroK8s will create a user group which is best to add your user account to so you can execute commands that would otherwise require admin privileges. You can do so with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ sudo usermod -a -G microk8s $USER
$ sudo chown -f -R $USER ~/.kube
</code></pre></div><p>With that, our very own Kubernetes cluster, courtesy of MicroK8s, should be ready to go. Check its status with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ microk8s status --wait-ready
</code></pre></div><p>You should see a “MicroK8s is running” message along with some specifications on your cluster. Including the available add-ons, which ones are enabled and which ones are disabled.</p>
<p>You can also shut down your cluster anytime with <code>microk8s stop</code>. Use <code>microk8s start</code> to bring it back up.</p>
<h4 id="introducing-kubectl">Introducing kubectl</h4>
<p>MicroK8s also comes with <a href="https://kubectl.docs.kubernetes.io/guides/introduction/kubectl/">kubectl</a>. This is our gateway into Kubernetes, as this is the command line tool that we use to interact with it. By default, MicroK8s makes it so we can call it using <code>microk8s kubectl ...</code>. That is, namespaced. This is useful if you have multiple Kubernetes implementations running at the same time, or another, separate kubectl. I don’t, so I like to create an alias for it, so that I can call it without having to use the <code>microk8s</code> prefix. You can do it like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ sudo snap alias microk8s.kubectl kubectl
</code></pre></div><p>Now that all that’s done, we can start talking to our Kubernetes cluster. We can ask it for example to tell us which are the nodes in the cluster with this command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get nodes
</code></pre></div><p>That will result in something like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">NAME STATUS ROLES AGE VERSION
pop-os Ready <none> 67d v1.21.4-3+e5758f73ed2a04
</code></pre></div><p>The only node in the cluster is your own machine. In my case, my machine is called “pop-os” so that’s what shows up. You can get more information out of this command by using <code>kubectl get nodes -o wide</code>.</p>
<h4 id="installing-add-ons">Installing add-ons</h4>
<p>MicroK8s supports many add-ons that we can use to enhance our Kubernetes installation. We are going to need a few of them so let’s install them now. They are:</p>
<ol>
<li>The <a href="https://microk8s.io/docs/addon-dashboard">dashboard</a>, which gives us a nice web GUI which serves as a window into our cluster. In there we can see everything that’s running, read logs, run commands, etc.</li>
<li><a href="https://microk8s.io/docs/addon-dns">dns</a>, which sets up DNS for within the cluster. In general it’s a good idea to enable this one because other add-ons use it.</li>
<li>storage, which allows the cluster to access the host machine’s disk for storage. The application that we will deploy needs a persistent database so we need this plugin to make it happen.</li>
<li>registry, which sets up a <a href="https://kubernetes.io/docs/concepts/containers/images/">container image</a> registry that Kubernetes can access. Kubernetes runs containerized applications, containers are based on images. So, having this add-on allows us to define an image for our application and make it available to Kubernetes.</li>
</ol>
<p>To install these, just run the following commands:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ microk8s enable dashboard
$ microk8s enable dns
$ microk8s enable storage
$ microk8s enable registry
</code></pre></div><p>Those are all the add-ons that we’ll use.</p>
<h4 id="introducing-the-dashboard">Introducing the Dashboard</h4>
<p>The dashboard is one we can play with right now. In order to access it, first run this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ microk8s dashboard-proxy
</code></pre></div><p>That will start up a proxy into the dashboard. The command will give you a URL and login token that you can use to access the dashboard. It results in an output like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">Checking if Dashboard is running.
Dashboard will be available at https://127.0.0.1:10443
Use the following token to login:
<YOUR LOGIN TOKEN>
</code></pre></div><p>Now you can navigate to that URL in your browser and you’ll find a screen like this:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-login.webp" alt="Dashboard login"></p>
<p>Make sure the “Token” option is selected and take the login token generated by the <code>microk8s dashboard-proxy</code> command from before and paste it in the field in the page. Click the “Sign In” button and you’ll be able to see the dashboard, allowing you access to many aspects of your cluster. It should look like this:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-home.webp" alt="Dashboard home"></p>
<p>Feel free to play around with it a little bit. You don’t have to understand everything yet. As we work through our example, we’ll see how the dashboard and the other add-ons come into play.</p>
<blockquote>
<p>There’s also a very useful command line tool called <a href="https://k9scli.io/">K9s</a>, which helps in interacting with our cluster. We will not be discussing it further in this article but feel free to explore it if you need or want a command line alternative to the built-in dashboard.</p>
</blockquote>
<h3 id="deploying-applications-into-a-kubernetes-cluster">Deploying applications into a Kubernetes cluster</h3>
<p>With all that setup out of the way, we can start using our K8s cluster for what it was designed: running applications.</p>
<h4 id="deployments">Deployments</h4>
<p>Pods are very much the stars of the show when it comes to Kubernetes. However, most of the time we don’t create them directly. We usually do so through “<a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">deployments</a>”. Deployments are a more abstract concept in Kubernetes. They basically control pods and make sure they behave as specified. You can think of them as wrappers for pods which make our lives easier than if we had to handle pods directly. Let’s go ahead and create a deployment so things will be clearer.</p>
<p>In Kubernetes, there are various ways of managing objects like deployments. For this post, I’m going to focus exclusively on the configuration-file-driven declarative approach as that’s the one better suited for real world scenarios.</p>
<blockquote>
<p>You can learn more about the different ways of interacting with Kubernetes objects in <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/object-management/">the official documentation</a>.</p>
</blockquote>
<p>So, simply put, if we want to create a deployment, then we need to author a file that defines it. A simple deployment specification looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># nginx-deployment.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>apps/v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Deployment<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>nginx-deployment<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">replicas</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">3</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">matchLabels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">template</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">containers</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>nginx:1.14.2<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">containerPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">80</span><span style="color:#bbb">
</span></code></pre></div><blockquote>
<p>This example is taken straight from <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">the official documentation</a>.</p>
</blockquote>
<p>Don’t worry if most of that doesn’t make sense at this point. I’ll explain it in detail later. First, let’s actually do something with it.</p>
<p>Save that in a new file. You can call it <code>nginx-deployment.yaml</code>. Once that’s done, you can actually create the deployment (and its associated objects) in your K8s cluster with this command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -f nginx-deployment.yaml
</code></pre></div><p>Which should result in the following message:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">deployment.apps/nginx-deployment created
</code></pre></div><p>And that’s it for creating deployments! (Or any other type of object in Kubernetes for that matter.) We define the object in a file and then invoke <code>kubectl</code>’s <code>apply</code> command. Pretty simple.</p>
<blockquote>
<p>If you want to delete the deployment, then this command will do it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl delete -f nginx-deployment.yaml
deployment.apps "nginx-deployment" deleted
</code></pre></div></blockquote>
<h4 id="using-kubectl-to-explore-a-deployment">Using kubectl to explore a deployment</h4>
<p>Now, let’s inspect our cluster to see what this command has done for us.</p>
<p>First, we can ask it directly for the deployment with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get deployments
</code></pre></div><p>Which outputs:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 3/3 3 3 2m54s
</code></pre></div><p>Now you can see that the deployment that we just created is right there with the name that we gave it.</p>
<p>As I said earlier, deployments are used to manage pods, and that’s just what the <code>READY</code>, <code>UP-TO-DATE</code> and <code>AVAILABLE</code> columns allude to with those values of <code>3</code>. This deployment has three pods because, in our YAML file, we specified we wanted three replicas with the <code>replicas: 3</code> line. Each “replica” is a pod. For our example, that means that we will have three instances of <a href="https://www.nginx.com/">NGINX</a> running side by side.</p>
<p>We can see the pods that have been created for us with this command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get pods
</code></pre></div><p>Which gives us something like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">NAME READY STATUS RESTARTS AGE
nginx-deployment-66b6c48dd5-fs5rq 1/1 Running 0 55m
nginx-deployment-66b6c48dd5-xmnl2 1/1 Running 0 55m
nginx-deployment-66b6c48dd5-sfzxm 1/1 Running 0 55m
</code></pre></div><p>The exact names will vary, as the IDs are auto-generated. But as you can see, this command gives us some basic information about our pods. Remember that pods are the ones that actually run our workloads via containers. The <code>READY</code> field is particularly interesting in this sense because it tells us how many containers are running in the pods vs how many are supposed to run. So, <code>1/1</code> means that the pod has one container ready out of 1. In other words, the pod is fully ready.</p>
<h4 id="using-the-dashboard-to-explore-a-deployment">Using the dashboard to explore a deployment</h4>
<p>Like I said before, the dashboard offers us a window into our cluster. Let’s see how we can use it to see the information that we just saw via <code>kubectl</code>. Navigate into the dashboard via your browser and you should now see that some new things have appeared:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-home-with-deployment.webp" alt="Dashboard home with deployment"></p>
<p>We now have new “CPU Usage” and “Memory Usage” sections that give us insight into the utilization of our machine’s resources.</p>
<p>There’s also “Workload status” that has some nice graphs giving us a glance at the status of our deployments, pods, and <a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/">replica sets</a>.</p>
<blockquote>
<p>Don’t worry too much about replica sets right now, as we seldom interact with them directly. Suffice it to say, replica sets are objects that deployments rely on to make sure that the number of specified replica pods is maintained. As always, there’s more info in <a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/">the official documentation</a>.</p>
</blockquote>
<p>Scroll down a little bit more and you’ll find the “Deployments” and “Pods” sections, which contain the information that we’ve already seen via <code>kubectl</code> before.</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-home-deployments-and-pods.webp" alt="Dashboard home: deployments and pods"></p>
<p>Feel free to click around and explore the capabilities of the dashboard.</p>
<h4 id="dissecting-the-deployment-configuration-file">Dissecting the deployment configuration file</h4>
<p>Now that we have a basic understanding of deployments and pods and how to create them, let’s look more closely into the configuration file that defines it. This is what we had:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># nginx-deployment.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>apps/v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Deployment<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>nginx-deployment<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">replicas</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">3</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">matchLabels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">template</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">containers</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>nginx:1.14.2<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">containerPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">80</span><span style="color:#bbb">
</span></code></pre></div><p>This example is very simple, but it touches on the key aspects of deployment configuration. We will be building more complex deployments as we work through this article, but this is a great start. Let’s start at the top:</p>
<ul>
<li><code>apiVersion</code>: Under the hood, a Kubernetes cluster exposes its functionality via a REST API. We seldom interact with this API directly because we have <code>kubectl</code> that takes care of it for us. <code>kubectl</code> takes our commands, translates them into HTTP requests that the K8s REST API can understand, sends them, and gives us back the results. So, this <code>apiVersion</code> field specifies which version of the K8s REST API we are expecting to talk to.</li>
<li><code>kind</code>: It represents the type of object that the configuration file defines. All objects in Kubernetes can be managed via YAML configuration files and <code>kubectl apply</code>. So, this field specifies which one we are managing at any given time.</li>
<li><code>metadata.name</code>: Quite simply, the name of the object. It’s how we and Kubernetes refer to it.</li>
<li><code>metadata.labels</code>: These help us further categorize cluster objects. These have no real effect in the system so they are useful for user help more than anything else.</li>
<li><code>spec</code>: This contains the actual functional specification for the behavior of the deployment. More details below.</li>
<li><code>spec.replicas</code>: The number of replica pods that the deployment should create. We already talked a bit about this before.</li>
<li><code>spec.selector.labels</code>: This is one case when labels are actually important. Remember that when we create deployments, replica sets and pods are created with it. Within the K8s cluster, they each are their own individual objects though. This field is the mechanism that K8s uses to associate a given deployment with its replica set and pods. In practice, that means that whatever labels are in this field need to match the labels in <code>spec.template.metadata.labels</code>. More on that one below.</li>
<li><code>spec.template</code>: Specifies the configuration of the pods that will be part of the deployment.</li>
<li><code>spec.template.metadata.labels</code>: Very similar to <code>metadata.labels</code>. The only difference is that those labels are added to the deployment while these ones are added to the pods. The only notable thing is that these labels are key for the deployment to know which pods it should care about (as explained in above in <code>spec.selector.labels</code>).</li>
<li><code>spec.template.spec</code>: This section specifies the actual functional configuration of the pods.</li>
<li><code>spec.template.spec.containers</code>: This section specifies the configuration of the containers that will be running inside the pods. It’s an array so there can be many. In our example we have only one.</li>
<li><code>spec.template.spec.containers[0].name</code>: The name of the container.</li>
<li><code>spec.template.spec.containers[0].image</code>: The image that will be used to build the container.</li>
<li><code>spec.template.spec.containers[0].ports[0].containerPort</code>: A port through which the container will accept traffic from the outside. In this case, <code>80</code>.</li>
</ul>
<blockquote>
<p>You can find a detailed description of all the fields supported by deployment configuration files <a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#deployment-v1-apps">in the official API reference documentation</a>. And much more!</p>
</blockquote>
<h4 id="connecting-to-the-containers-in-the-pods">Connecting to the containers in the pods</h4>
<p>Kubernetes allows us to connect to the containers running inside a pod. This is pretty easy to do with <code>kubectl</code>. All we need to know is the name of the pod and the container that we want to connect to. If the pod is running only one container (like our NGINX one does) then we don’t need the container name. We can find out the names of our pods with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-deployment-66b6c48dd5-85nwq 1/1 Running 0 25s
nginx-deployment-66b6c48dd5-x5b4x 1/1 Running 0 25s
nginx-deployment-66b6c48dd5-wvkhc 1/1 Running 0 25s
</code></pre></div><p>Pick one of those and we can open a bash session in it with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl exec -it nginx-deployment-66b6c48dd5-85nwq -- bash
</code></pre></div><p>Which results in a prompt like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">root@nginx-deployment-66b6c48dd5-85nwq:/#
</code></pre></div><p>We’re now connected to the container in one of our NGINX pods. There isn’t a lot to do with this right now, but feel free to explore it. It’s got its own processes and file system which are isolated from the other replica pods and your actual machine.</p>
<p>We can also connect to containers via the dashboard. Go back to the dashboard in your browser, log in again if the session expired, and scroll down to the “Pods” section. Each pod in the list has an action menu with an “Exec” command. See it here:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-pod-exec.webp" alt="Dashboard pod exec"></p>
<p>Click it and you’ll be taken to a screen with a console just like the one we obtained via <code>kubectl exec</code>:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-pod-bash.webp" alt="Dashboard pod bash"></p>
<p>The dashboard is quite useful, right?</p>
<h4 id="services">Services</h4>
<p>So far, we’ve learned quite a bit about deployments. How to specify and create them, how to explore them via command line and the dashboard, how to interact with the pods, etc. We haven’t seen a very important part yet, though: actually accessing the application that has been deployed. That’s where <a href="https://kubernetes.io/docs/concepts/services-networking/service/">services</a> come in. We use services to expose an application running in a set of pods to the world outside the cluster.</p>
<p>Here’s what a configuration file for a service that exposes access to our NGINX deployment could look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># nginx-service.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Service<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>nginx-service<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">type</span>:<span style="color:#bbb"> </span>NodePort<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>nginx<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"http"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">port</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">80</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">80</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">nodePort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">30080</span><span style="color:#bbb">
</span></code></pre></div><p>Same as with the deployment’s configuration file, this one also has a <code>kind</code> field that specifies what it is; and a name given to it via the <code>metadata.name</code> field. The <code>spec</code> section is where things get interesting.</p>
<ul>
<li><code>spec.type</code> specifies, well… The type of the service. Kubernetes supports many <a href="https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types">types of services</a>. For now, we want a <code>NodePort</code>. This type of service makes sure to expose itself as a static port (given by <code>spec.ports[0].nodePort</code>) on every node in the cluster. In our setup, we only have one node, which is our own machine.</li>
<li><code>spec.ports</code> defines which ports of the pods’ containers the service will expose.</li>
<li><code>spec.ports[0].name</code>: The name of the port. To be used elsewhere to reference the specific port.</li>
<li><code>spec.ports[0].port</code>: The port that will be exposed by the service.</li>
<li><code>spec.ports[0].targetPort</code>: The port that the service will target in the container.</li>
<li><code>spec.ports[0].nodePort</code>: The port that the service will expose in all the nodes of the cluster.</li>
</ul>
<p>Same as with deployments, we can create such a service with the <code>kubectl apply</code> command. If you save the contents from the YAML above into a <code>nginx-service.yaml</code> file, you can run the following to create it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -f nginx-service.yaml
</code></pre></div><p>And to inspect it and validate that it was in fact created:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 68d
nginx-service NodePort 10.152.183.22 <none> 80:30080/TCP 27s
</code></pre></div><p>The dashboard also has a section for services. It looks like this:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-services.webp" alt="Dashboard: services"></p>
<h4 id="accessing-an-application-via-a-service">Accessing an application via a service</h4>
<p>We can access our service in a few different ways. We can use its “cluster IP” which we obtain from the output of the <code>kubectl get services</code> command. As given by the example above, that would be <code>10.152.183.22</code> in my case. Browsing to that IP gives us the familiar NGINX default welcome page:</p>
<p><img src="/blog/2022/01/kubernetes-101/nginx-via-cluster-ip.webp" alt="NGINX via Cluster IP"></p>
<p>Another way is by using the “NodePort”. Remember that the “NodePort” specifies the port in which the service will be available on every node of the cluster. With our current MicroK8s setup, our own machine is a node in the cluster, so we can also access the NGINX that’s running in our Kubernetes cluster using <code>localhost:30080</code>. <code>30080</code> is given by the <code>spec.ports[0].nodePort</code> field in the service configuration file from before. Try it out:</p>
<p><img src="/blog/2022/01/kubernetes-101/nginx-via-nodeport.webp" alt="NGINX via NodePort"></p>
<p>How cool is that? We have identical, replicated NGINX instances running in a Kubernetes cluster that’s installed locally in our machine.</p>
<h3 id="deploying-our-own-custom-application">Deploying our own custom application</h3>
<p>Alright, by deploying NGINX, we’ve learned a lot about nodes, pods, deployments, services, and how they all work together to run and serve an application from a Kubernetes cluster. Now, let’s take all that knowledge and try to do the same for a completely custom application of our own.</p>
<h4 id="what-are-we-building">What are we building</h4>
<p>The application that we are going to deploy into our cluster is a simple one with only two components: a REST API written with <a href="https://dotnet.microsoft.com/">.NET 5</a> and a <a href="https://www.postgresql.org/">Postgres</a> database. You can find the source code <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">in GitHub</a>. It’s an API for supporting a hypothetical front end application for capturing used vehicle information and calculating their value in dollars.</p>
<blockquote>
<p>If you’re interested in learning more about the process of actually writing that app, it’s all documented in another blog post: <a href="/blog/2021/07/dotnet-5-web-api/">Building REST APIs with .NET 5, ASP.NET Core, and PostgreSQL</a>.</p>
</blockquote>
<blockquote>
<p>If you’re following along, now would be a good time to download the source code of the web application that we’re going to be playing with. You can find it on <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">GitHub</a>. From now on, we’ll use that as the root directory of all the files we create and modify.</p>
</blockquote>
<blockquote>
<p>Also, be sure to delete or put aside the <code>k8s</code> directory. We’ll be building that throughout the rest of this post.</p>
</blockquote>
<h3 id="deploying-the-database">Deploying the database</h3>
<p>Let’s begin with the Postgres database. Similar as before, we start by setting up a deployment with one pod and one container. We can do so with a deployment configuration YAML file like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># db-deployment.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>apps/v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Deployment<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-db<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">matchLabels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-db<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">replicas</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">1</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">template</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-db<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">containers</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-db<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>postgres:13<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">containerPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5432</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"postgres"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">env</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_DB<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>vehicle_quotes<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_USER<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>vehicle_quotes<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_PASSWORD<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>password<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">limits</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">memory</span>:<span style="color:#bbb"> </span>4Gi<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">cpu</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"2"</span><span style="color:#bbb">
</span></code></pre></div><p>This deployment configuration YAML file is similar to the one we used for NGINX before, but it introduces a few new elements:</p>
<ul>
<li><code>spec.template.spec.containers[0].ports[0].name</code>: We can give specific names to ports which we can reference later, elsewhere in the K8s configurations, which is what this field is for.</li>
<li><code>spec.template.spec.containers[0].env</code>: This is a list of environment variables that will be defined in the container inside the pod. In this case, we’ve specified a few variables that are necessary to configure the Postgres instance that will be running. We’re using <a href="https://hub.docker.com/_/postgres">the official Postgres image from Dockerhub</a>, and it calls for these variables. Their purpose is straightforward: they specify database name, username, and password.</li>
<li><code>spec.template.spec.containers[0].resources</code>: This field defines the hardware resources that the container needs in order to function. We can specify upper limits with <code>limits</code> and lower ones with <code>requests</code>. You can learn more about resource management in <a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/">the official documentation</a>. In our case, we’ve kept it simple and used <code>limits</code> to prevent the container from using more than 4Gi of memory and 2 CPU cores.</li>
</ul>
<p>Now, let’s save that YAML into a new file called <code>db-deployment.yaml</code> and run the following:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -f db-deployment.yaml
</code></pre></div><p>Which should output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">deployment.apps/vehicle-quotes-db created
</code></pre></div><p>After a few seconds, you should be able to see the new deployment and pod via <code>kubectl</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get deployment -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
...
default vehicle-quotes-db 1/1 1 1 9m20s
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
...
default vehicle-quotes-db-5fb576778-gx7j6 1/1 Running 0 9m22s
</code></pre></div><p>Remember you can also see them in the dashboard:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-db-deployment-and-pod.webp" alt="Dashboard DB deployment and pod"></p>
<h4 id="connecting-to-the-database">Connecting to the database</h4>
<p>Let’s try connecting to the Postgres instance that we just deployed. Take note of the pod’s name and try:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl exec -it <DB_POD_NAME> -- bash
</code></pre></div><p>You’ll get a bash session on the container that’s running the database. For me, given the pod’s auto-generated name, it looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">root@vehicle-quotes-db-5fb576778-gx7j6:/#
</code></pre></div><p>From here, you can connect to the database using the <a href="https://www.postgresql.org/docs/13/app-psql.html">psql</a> command line client. Remember that we told the Postgres instance to create a <code>vehicle_quotes</code> user. We set it up via the container environment variables on our deployment configuration. As a result, we can do <code>psql -U vehicle_quotes</code> to connect to the database. Put together, it all looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl exec -it vehicle-quotes-db-5fb576778-gx7j6 -- bash
root@vehicle-quotes-db-5fb576778-gx7j6:/# psql -U vehicle_quotes
psql (13.3 (Debian 13.3-1.pgdg100+1))
Type "help" for help.
vehicle_quotes=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
----------------+----------------+----------+------------+------------+-----------------------------------
postgres | vehicle_quotes | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | vehicle_quotes | UTF8 | en_US.utf8 | en_US.utf8 | =c/vehicle_quotes +
| | | | | vehicle_quotes=CTc/vehicle_quotes
template1 | vehicle_quotes | UTF8 | en_US.utf8 | en_US.utf8 | =c/vehicle_quotes +
| | | | | vehicle_quotes=CTc/vehicle_quotes
vehicle_quotes | vehicle_quotes | UTF8 | en_US.utf8 | en_US.utf8 |
(4 rows)
</code></pre></div><p>Pretty cool, don’t you think? We have a database running on our cluster now with minimal effort. There’s a slight problem though…</p>
<h4 id="persistent-volumes-and-claims">Persistent volumes and claims</h4>
<p>The problem in our database is that any changes are lost if the pod or container were to shut down or reset for some reason. This is because all the database files live inside the container’s file system, so when the container is gone, the data is also gone.</p>
<p>In Kubernetes, pods are supposed to be treated as ephemeral entities. The idea is that pods should easily be brought down and replaced by new pods and users and clients shouldn’t even notice. This is all Kubernetes working as expected. That is to say, pods should be as stateless as possible to work well with this behavior.</p>
<p>However, a database is, by definition, not stateless. So what we need to do to solve this problem is have some available disk space from outside the cluster that can be used by our database to store its files. Something persistent that won’t go away if the pod or container goes away. That’s where <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/">persistent volumes and persistent volume claims</a> come in.</p>
<p>We will use a persistent volume (PV) to define a directory in our host machine that we will allow our Postgres container to use to store data files. Then, a persistent volume claim (PVC) is used to define a “request” for some of that available disk space that a specific container can make. In short, a persistent volume says to K8s “here’s some storage that the cluster can use” and a persistent volume claim says “here’s a portion of that storage that’s available for containers to use”.</p>
<h4 id="configuration-files-for-the-pv-and-pvc">Configuration files for the PV and PVC</h4>
<p>Start by tearing down our currently broken Postgres deployment:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl delete -f db-deployment.yaml
</code></pre></div><p>Now let’s add two new YAML configuration files. One is for the persistent volume:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># db-persistent-volume.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>PersistentVolume<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-postgres-data-persisent-volume<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">type</span>:<span style="color:#bbb"> </span>local<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">claimRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">namespace</span>:<span style="color:#bbb"> </span>default<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-postgres-data-persisent-volume-claim<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storageClassName</span>:<span style="color:#bbb"> </span>manual<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">capacity</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storage</span>:<span style="color:#bbb"> </span>5Gi<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">accessModes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- ReadWriteOnce<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">hostPath</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">path</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/home/kevin/projects/vehicle-quotes-postgres-data"</span><span style="color:#bbb">
</span></code></pre></div><p>In this config file, we already know about the <code>kind</code> and <code>metadata</code> fields. A few of the other elements are interesting though:</p>
<ul>
<li><code>spec.claimRef</code>: Contains identifying information about the claim that’s associated with the PV. Used to bind the PVC with a specific PVC. Notice how it matches the name defined in the PVC config file from below.</li>
<li><code>spec.capacity.storage</code>: Is pretty straightforward in that it specifies the size of the persistent volume.</li>
<li><code>spec.accessModes</code>: Defines how the PV can be accessed. In this case, we’re using <code>ReadWriteOnce</code> so that it can only be used by a single node in the cluster which is allowed to read from and write into the PV.</li>
<li><code>spec.hostPath.path</code>: Specifies the directory in the host machine’s file system where the PV will be mounted. Simply put, the containers in the cluster will have access to the specific directory defined here. I’ve used <code>/home/kevin/projects/vehicle-quotes-postgres-data</code> because that makes sense on my own machine. If you’re following along, make sure to set it to something that makes sense in your environment.</li>
</ul>
<blockquote>
<p><code>hostPath</code> is just one type of persistent volume which works well for development deployments. Managed Kubernetes implementations like the ones from Google or Amazon have their own types which are more appropriate for production.</p>
</blockquote>
<p>We also need another config file for the persistent volume claim:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># db-persistent-volume-claim.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>PersistentVolumeClaim<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-postgres-data-persisent-volume-claim<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumeName</span>:<span style="color:#bbb"> </span>vehicle-quotes-postgres-data-persisent-volume<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storageClassName</span>:<span style="color:#bbb"> </span>manual<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">accessModes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- ReadWriteOnce<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">requests</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storage</span>:<span style="color:#bbb"> </span>5Gi<span style="color:#bbb">
</span></code></pre></div><p>Like I said, PVCs are essentially usage requests for PVs. So, the config file is simple in that it’s mostly specified to match the PV.</p>
<ul>
<li><code>spec.volumeName</code>: The name of the PV that this PVC is going to access. Notice how it matches the name that we defined in the PV’s config file.</li>
<li><code>spec.resources.requests</code>: Defines how much space this PVC requests from the PV. In this case, we’re just requesting all the space that the PV has available to it, as given by its config file: <code>5Gi</code>.</li>
</ul>
<h4 id="configuring-the-deployment-to-use-the-pvc">Configuring the deployment to use the PVC</h4>
<p>After saving those files, all that’s left is to update the database deployment configuration to use the PVC. Here’s what the updated config file would look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> apiVersion: apps/v1
kind: Deployment
metadata:
name: vehicle-quotes-db
spec:
selector:
matchLabels:
app: vehicle-quotes-db
replicas: 1
template:
metadata:
labels:
app: vehicle-quotes-db
spec:
containers:
- name: vehicle-quotes-db
image: postgres:13
ports:
- containerPort: 5432
name: "postgres"
<span style="color:#000;background-color:#dfd">+ volumeMounts:
</span><span style="color:#000;background-color:#dfd">+ - mountPath: "/var/lib/postgresql/data"
</span><span style="color:#000;background-color:#dfd">+ name: vehicle-quotes-postgres-data-storage
</span><span style="color:#000;background-color:#dfd"></span> env:
- name: POSTGRES_DB
value: vehicle_quotes
- name: POSTGRES_USER
value: vehicle_quotes
- name: POSTGRES_PASSWORD
value: password
resources:
limits:
memory: 4Gi
cpu: "2"
<span style="color:#000;background-color:#dfd">+ volumes:
</span><span style="color:#000;background-color:#dfd">+ - name: vehicle-quotes-postgres-data-storage
</span><span style="color:#000;background-color:#dfd">+ persistentVolumeClaim:
</span><span style="color:#000;background-color:#dfd">+ claimName: vehicle-quotes-postgres-data-persisent-volume-claim
</span></code></pre></div><p>First, notice the <code>volumes</code> section at the bottom of the file. Here’s where we define the volume that will be available to the container, give it a name and specify which PVC it will use. The <code>spec.template.volumes[0].persistentVolumeClaim.claimName</code> needs to match the name of the PVC that we defined in <code>db-persistent-volume-claim.yaml</code>.</p>
<p>Then, up in the <code>containers</code> section, we define a <code>volumeMounts</code> element. We use that to specify which directory within the container will map to our PV. In this case, we’ve set the container’s <code>/var/lib/postgresql/data</code> directory to use the volume that we defined at the bottom of the file. That volume is backed by our persistent volume claim, which is in turn backed by our persistent volume. The significance of the <code>/var/lib/postgresql/data</code> directory is that this is where Postgres stores database files by default.</p>
<p>In summary: We created a persistent volume that defines some disk space in our machine that’s available to the cluster; then we defined a persistent volume claim that represents a request of some of that space that a container can have access to; after that we defined a volume within our pod configuration in our deployment to point to that persistent volume claim; and finally we defined a volume mount in our container that uses that volume to store the Postgres database files.</p>
<p>By setting it up this way, we’ve made it so that regardless of how many Postgres pods come and go, the database files will always be persisted, because the files now live outside of the container. They are stored in our host machine instead.</p>
<blockquote>
<p>There’s another limitation that’s important to note. Just using the approach that we discussed, it’s not possible to deploy multiple replicas of Postgres which work in tandem and operate on the same data. Even though the data files can be defined outside of the cluster and persisted that way, there can only be one single Postgres instance running against it at any given time.</p>
<p>In production, the high availability problem is better solved leveraging the features provided by the database software itself. <a href="https://www.postgresql.org/docs/9.1/different-replication-solutions.html">Postgres offers various options in that area</a>. Or, if you are deploying to the cloud, the best strategy may be to use a relational database service managed by your cloud provider. Examples are <a href="https://aws.amazon.com/rds/">Amazon’s RDS</a> and <a href="https://azure.microsoft.com/en-us/products/azure-sql/database/">Microsoft’s Azure SQL Database</a>.</p>
</blockquote>
<h4 id="applying-changes">Applying changes</h4>
<p>Now let’s see it in action. Run the following three commands to create the objects:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -f db-persistent-volume.yaml
$ kubectl apply -f db-persistent-volume-claim.yaml
$ kubectl apply -f db-deployment.yaml
</code></pre></div><p>After a while, they will show up in the dashboard. You already know how to look for deployments and pods. For persistent volumes, click the “Persistent Volumes” option under the “Cluster” section in the sidebar:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-persistent-volume.webp" alt="Dashboard persistent volume"></p>
<p>Persistent volume claims can be found in the “Persistent Volume Claims” option under the “Config and Storage” section in the sidebar:</p>
<p><img src="/blog/2022/01/kubernetes-101/dashboard-persistent-volume-claim.webp" alt="Dashboard persistent volume claim"></p>
<p>Now, try connecting to the database (using <code>kubectl exec -it <VEHICLE_QUOTES_DB_POD_NAME> -- bash</code> and then <code>psql -U vehicle_quotes</code>) and creating some tables. Something simple like this would work:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-sql" data-lang="sql"><span style="color:#080;font-weight:bold">CREATE</span><span style="color:#bbb"> </span><span style="color:#080;font-weight:bold">TABLE</span><span style="color:#bbb"> </span>test<span style="color:#bbb"> </span>(test_field<span style="color:#bbb"> </span><span style="color:#038">varchar</span>);<span style="color:#bbb">
</span></code></pre></div><p>Now, close <code>psql</code> and the <code>bash</code> in the pod and delete the objects:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl delete -f db-deployment.yaml
$ kubectl delete -f db-persistent-volume-claim.yaml
$ kubectl delete -f db-persistent-volume.yaml
</code></pre></div><p>Create them again:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -f db-persistent-volume.yaml
$ kubectl apply -f db-persistent-volume-claim.yaml
$ kubectl apply -f db-deployment.yaml
</code></pre></div><p>Connect to the database again and you should see that the table is still there:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vehicle_quotes=# \c vehicle_quotes
You are now connected to database "vehicle_quotes" as user "vehicle_quotes".
vehicle_quotes=# \dt
List of relations
Schema | Name | Type | Owner
--------+------+-------+----------------
public | test | table | vehicle_quotes
(1 row)
</code></pre></div><p>That’s just what we wanted: the database is persisting independently of what happens to the pods and containers.</p>
<h4 id="exposing-the-database-as-a-service">Exposing the database as a service</h4>
<p>Lastly, we need to expose the database as a service so that the rest of the cluster can access it without having to use explicit pod names. We don’t need this for our testing, but we do need it for later when we deploy our web app, so that it can reach the database. As you’ve seen, services are easy to create. Here’s the YAML config file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># db-service.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Service<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-db-service<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">type</span>:<span style="color:#bbb"> </span>NodePort<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-db<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"postgres"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">protocol</span>:<span style="color:#bbb"> </span>TCP<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">port</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5432</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5432</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">nodePort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">30432</span><span style="color:#bbb">
</span></code></pre></div><p>Save that into a new <code>db-service.yaml</code> file and don’t forget to <code>kubectl apply -f db-service.yaml</code>.</p>
<h3 id="deploying-the-web-application">Deploying the web application</h3>
<p>Now that we’ve got the database sorted out, let’s turn our attention to the app itself. As you’ve seen, Kubernetes runs apps as containers. That means that we need images to build those containers. A custom web application is no exception. We need to build a custom image that contains our application so that it can be deployed into Kubernetes.</p>
<h4 id="building-the-web-application-image">Building the web application image</h4>
<p>The first step for building a container image is writing a <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile</a>. Since our application is a <a href="https://dotnet.microsoft.com/apps/aspnet/apis">Web API</a> built using .NET 5, I’m going to use a slightly modified version of the Dockerfile used by <a href="https://code.visualstudio.com/">Visual Studio Code</a>’s <a href="https://github.com/microsoft/vscode-remote-try-dotnetcore/blob/main/.devcontainer/Dockerfile">development container demo for .NET</a>. These development containers are excellent for, well… development. You can see the original in the link above, but here’s mine:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-dockerfile" data-lang="dockerfile"><span style="color:#888"># [Choice] .NET version: 5.0, 3.1, 2.1</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ARG</span> <span style="color:#369">VARIANT</span>=<span style="color:#d20;background-color:#fff0f0">"5.0"</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">FROM</span><span style="color:#d20;background-color:#fff0f0"> mcr.microsoft.com/vscode/devcontainers/dotnet:0-${VARIANT}</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#888"># [Option] Install Node.js</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ARG</span> <span style="color:#369">INSTALL_NODE</span>=<span style="color:#d20;background-color:#fff0f0">"false"</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#888"># [Option] Install Azure CLI</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ARG</span> <span style="color:#369">INSTALL_AZURE_CLI</span>=<span style="color:#d20;background-color:#fff0f0">"false"</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#888"># Install additional OS packages.</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> apt-get update && <span style="color:#038">export</span> <span style="color:#369">DEBIAN_FRONTEND</span>=noninteractive <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> && apt-get -y install --no-install-recommends postgresql-client-common postgresql-client<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#888"># Run the remaining commands as the "vscode" user</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">USER</span><span style="color:#d20;background-color:#fff0f0"> vscode</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#888"># Install EF and code generator development tools</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> dotnet tool install --global dotnet-ef<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> dotnet tool install --global dotnet-aspnet-codegenerator<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> <span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">'export PATH="$PATH:/home/vscode/.dotnet/tools"'</span> >> /home/vscode/.bashrc<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">WORKDIR</span><span style="color:#d20;background-color:#fff0f0"> /app</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#888"># Prevent the container from closing automatically</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ENTRYPOINT</span> [<span style="color:#d20;background-color:#fff0f0">"tail"</span>, <span style="color:#d20;background-color:#fff0f0">"-f"</span>, <span style="color:#d20;background-color:#fff0f0">"/dev/null"</span>]<span style="color:#a61717;background-color:#e3d2d2">
</span></code></pre></div><blockquote>
<p>There are <a href="https://github.com/microsoft/vscode-dev-containers/tree/main/containers">many other</a> development containers for other languages and frameworks. Take a look at the <a href="https://github.com/microsoft/vscode-dev-containers">microsoft/vscode-dev-containers GitHub repo</a> to learn more.</p>
</blockquote>
<p>An interesting thing about this Dockerfile is that we install the <code>psql</code> command line client so that we can connect to our Postgres database from within the web application container. The rest is stuff specific to .NET and the particular image we’re basing this Dockerfile on, so don’t sweat it too much.</p>
<p>If you’ve downloaded the source code, this Dockerfile should already be there as <code>Dockerfile.dev</code>.</p>
<h4 id="making-the-image-accessible-to-kubernetes">Making the image accessible to Kubernetes</h4>
<p>Now that we have a Dockerfile, we can use it to build an image that Kubernetes is able to use to build a container to deploy. So that Kubernetes can see it, we need to build it in a specific way and push it into a registry that’s accessible to Kubernetes. Remember how we ran <code>microk8s enable registry</code> to install the registry add-on when we were setting up MicroK8s? That will pay off now, as that’s the registry to which we’ll push our image.</p>
<p>First, we build the image:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ docker build . -f Dockerfile.dev -t localhost:32000/vehicle-quotes-dev:registry
</code></pre></div><p>That will take some time to download and set up everything. Once that’s done, we push the image to the registry:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ docker push localhost:32000/vehicle-quotes-dev:registry
</code></pre></div><p>That will also take a little while.</p>
<h4 id="deploying-the-web-application-1">Deploying the web application</h4>
<p>The next step is to create a deployment for the web app. Like usual, we start with a deployment YAML configuration file. Let’s call it <code>web-deployment.yaml</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># web-deployment.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>apps/v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Deployment<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">matchLabels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">replicas</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">1</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">template</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">containers</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>localhost:32000/vehicle-quotes-dev:registry<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">containerPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5000</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"http"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">containerPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5001</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"https"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumeMounts</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">mountPath</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/app"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-storage<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">env</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_DB<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>vehicle_quotes<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_USER<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>vehicle_quotes<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_PASSWORD<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>password<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>CUSTOMCONNSTR_VehicleQuotesContext<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>Host=$(VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST);Database=$(POSTGRES_DB);Username=$(POSTGRES_USER);Password=$(POSTGRES_PASSWORD)<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">limits</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">memory</span>:<span style="color:#bbb"> </span>2Gi<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">cpu</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"1"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-storage<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">persistentVolumeClaim</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">claimName</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-persisent-volume-claim<span style="color:#bbb">
</span></code></pre></div><p>This deployment configuration should look very familiar to you by now as it is very similar to the ones we’ve already seen. There are a few notable elements though:</p>
<ul>
<li>Notice how we specified <code>localhost:32000/vehicle-quotes-dev:registry</code> as the container image. This is the exact same name of the image that we built and pushed into the registry before.</li>
<li>In the environment variables section, the one named <code>CUSTOMCONNSTR_VehicleQuotesContext</code> is interesting for a couple of reasons:
<ul>
<li>First, the value is a Postgres connection string being built off of other environment variables using the following format: <code>$(ENV_VAR_NAME)</code>. That’s a neat feature of Kubernetes config files that allows us to reference variables to build other ones.</li>
<li>Second, the <code>VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST</code> environment variable used within that connection string is not defined anywhere in our configuration files. That’s an automatic environment variable that Kubernetes injects on all containers when there are services available. In this case, it contains the hostname of the <code>vehicle-quotes-db-service</code> that we created a few sections ago. The automatic injection of this <code>*_SERVICE_HOST</code> variable always happens as long as the service is already created by the time that the pod gets created. We have already created the service so we should be fine using the variable here. As usual, there’s more info in the <a href="https://kubernetes.io/docs/concepts/services-networking/service/#environment-variables">official documentation</a>.</li>
</ul>
</li>
</ul>
<p>As you may have noticed, this deployment has a persistent volume. That’s to store the application’s source code. Or, more accurately, to make the source code, which lives in our machine, available to the container. This is a development setup after all, so we want to be able to edit the code from the comfort of our own file system, and have the container inside the cluster be aware of that.</p>
<p>Anyway, let’s create the associated persistent volume and persistent volume claim. Here’s the PV (save it as <code>web-persistent-volume.yaml</code>):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># web-persistent-volume.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>PersistentVolume<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-persisent-volume<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">type</span>:<span style="color:#bbb"> </span>local<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">claimRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">namespace</span>:<span style="color:#bbb"> </span>default<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-persisent-volume-claim<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storageClassName</span>:<span style="color:#bbb"> </span>manual<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">capacity</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storage</span>:<span style="color:#bbb"> </span>1Gi<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">accessModes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- ReadWriteOnce<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">hostPath</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">path</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/home/kevin/projects/vehicle-quotes"</span><span style="color:#bbb">
</span></code></pre></div><p>And here’s the PVC (save it as <code>web-persistent-volume-claim.yaml</code>):</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># web-persistent-volume-claim.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>PersistentVolumeClaim<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-persisent-volume-claim<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumeName</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-persisent-volume<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storageClassName</span>:<span style="color:#bbb"> </span>manual<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">accessModes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- ReadWriteOnce<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">requests</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">storage</span>:<span style="color:#bbb"> </span>1Gi<span style="color:#bbb">
</span></code></pre></div><p>The only notable element here is the PV’s <code>hostPath</code>. I have it pointing to the path where I downloaded the app’s source code from <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">GitHub</a>. Make sure to do the same on your end.</p>
<p>Finally, tie it all up with a service that will expose the development build of our REST API. Here’s the config file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># web-service.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Service<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web-service<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">type</span>:<span style="color:#bbb"> </span>NodePort<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"http"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">protocol</span>:<span style="color:#bbb"> </span>TCP<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">port</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5000</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5000</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">nodePort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">30000</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"https"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">protocol</span>:<span style="color:#bbb"> </span>TCP<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">port</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5001</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">targetPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5001</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">nodePort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">30001</span><span style="color:#bbb">
</span><span style="color:#bbb">
</span></code></pre></div><p>Should be pretty self-explanatory at this point. In this case, we expose two ports, one for HTTP and another for HTTPS. Our .NET 5 Web API works with both so that’s why we specify them here. This configuration says that the service should expose port <code>30000</code> and send traffic that comes into that port from the outside world into port <code>5000</code> on the container. Likewise, outside traffic coming to port <code>30001</code> will be sent to port <code>5001</code> in the container.</p>
<p>Save that file as <code>web-service.yaml</code> and we’re ready to apply the changes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -f web-persistent-volume.yaml
$ kubectl apply -f web-persistent-volume-claim.yaml
$ kubectl apply -f web-deployment.yaml
$ kubectl apply -f web-service.yaml
</code></pre></div><p>Feel free to explore the dashboard’s “Deployments”, “Pods”, “Services”, “Persistent Volumes”, and “Persistent Volume Claims” sections to see the fruits of our labor.</p>
<h4 id="starting-the-application">Starting the application</h4>
<p>Let’s now do some final setup and start up our application. Start by connecting to the web application pod:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl exec -it vehicle-quotes-web-86cbc65c7f-5cpg8 -- bash
</code></pre></div><blockquote>
<p>Remember that the pod name will be different for you, so copy it from the dashboard or <code>kubectl get pods -A</code>.</p>
</blockquote>
<p>You’ll get a prompt like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vscode ➜ /app (master ✗) $
</code></pre></div><p>Try <code>ls</code> to see all of the app’s source code files courtesy of the PV that we set up before:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vscode ➜ /app (master ✗) $ ls
Controllers Dockerfile.prod Models README.md Startup.cs appsettings.Development.json k8s queries.sql
Data K8S_README.md Program.cs ResourceModels Validations appsettings.json k8s_wip
Dockerfile.dev Migrations Properties Services VehicleQuotes.csproj database.dbml obj
</code></pre></div><p>Now it’s just a few .NET commands to get the app up and running. First, compile and download packages:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ dotnet build
</code></pre></div><p>That will take a while. Once done, let’s build the database schema:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ dotnet ef database update
</code></pre></div><p>And finally, run the development web server:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ dotnet run
</code></pre></div><blockquote>
<p>If you get the error message “System.InvalidOperationException: Unable to configure HTTPS endpoint.” while trying <code>dotnet run</code>, follow the error message’s instructions and run <code>dotnet dev-certs https --trust</code>. This will generate a development certificate so that the dev server can serve HTTPS.</p>
</blockquote>
<p>As a result, you should see this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">vscode ➜ /app (master ✗) $ dotnet run
Building...
info: Microsoft.Hosting.Lifetime[0]
Now listening on: https://0.0.0.0:5001
info: Microsoft.Hosting.Lifetime[0]
Now listening on: http://0.0.0.0:5000
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
Content root path: /app
</code></pre></div><p>It indicates that the application is up and running. Now, navigate to <code>http://localhost:30000</code> in your browser of choice and you should see our REST API’s Swagger UI:</p>
<p><img src="/blog/2022/01/kubernetes-101/swagger.webp" alt="Swagger!"></p>
<p>Notice that <code>30000</code> is the port we specified in the <code>web-service.yaml</code>’s <code>nodePort</code> for the <code>http</code> port. That’s the port that the service exposes to the world outside the cluster. Notice also how our .NET web app’s development server listens to traffic coming from ports <code>5000</code> and <code>5001</code> for HTTP and HTTPS respectively. That’s why we configured <code>web-service.yaml</code> like we did.</p>
<p>Outstanding! All our hard work has paid off and we have a full-fledged web application running in our Kubernetes cluster. This is quite a momentous occasion. We’ve built a custom image that can be used to create containers to run a .NET web application, pushed that image into our local registry so that K8s could use it, and deployed a functioning application. As a cherry on top, we made it so the source code is super easy to edit, as it lives within our own machine’s file system and the container in the cluster accesses it directly from there. Quite an accomplishment.</p>
<p>Now it’s time to go the extra mile and organize things a bit. Let’s talk about Kustomize next.</p>
<h3 id="putting-it-all-together-with-kustomize">Putting it all together with Kustomize</h3>
<p><a href="https://kubectl.docs.kubernetes.io/guides/introduction/kustomize/">Kustomize</a> is a tool that helps us improve Kubernetes’ declarative object management with configuration files (which is what we’ve been doing throughout this post). Kustomize has useful features that help with better organizing configuration files, managing configuration variables, and support for deployment variants (for things like dev vs. test vs. prod environments). Let’s explore what Kustomize has to offer.</p>
<p>First, be sure to tear down all the objects that we have created so far as we will be replacing them later once we have a setup with Kustomize. This will work for that:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl delete -f db-service.yaml
$ kubectl delete -f db-deployment.yaml
$ kubectl delete -f db-persistent-volume-claim.yaml
$ kubectl delete -f db-persistent-volume.yaml
$ kubectl delete -f web-service.yaml
$ kubectl delete -f web-deployment.yaml
$ kubectl delete -f web-persistent-volume-claim.yaml
$ kubectl delete -f web-persistent-volume.yaml
</code></pre></div><p>Next, let’s reorganize our <code>db-*</code> and <code>web-*</code> YAML files like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">k8s
├── db
│ ├── db-deployment.yaml
│ ├── db-persistent-volume-claim.yaml
│ ├── db-persistent-volume.yaml
│ └── db-service.yaml
└── web
├── web-deployment.yaml
├── web-persistent-volume-claim.yaml
├── web-persistent-volume.yaml
└── web-service.yaml
</code></pre></div><p>As you can see, we’ve put them all inside a new <code>k8s</code> directory, and further divided them into <code>db</code> and <code>web</code> sub-directories. <code>web-*</code> files went into the <code>web</code> directory and <code>db-*</code> files went into <code>db</code>. At this point, the prefixes on the files are a bit redundant so we can remove them. After all, we know what component they belong to because of the name of their respective sub-directories.</p>
<blockquote>
<p>There’s already a <code>k8s</code> directory in the repo. Feel free to get rid of it as we will build it back up from scratch now.</p>
</blockquote>
<p>So it should end up looking like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">k8s
├── db
│ ├── deployment.yaml
│ ├── persistent-volume-claim.yaml
│ ├── persistent-volume.yaml
│ └── service.yaml
└── web
├── deployment.yaml
├── persistent-volume-claim.yaml
├── persistent-volume.yaml
└── service.yaml
</code></pre></div><blockquote>
<p>kubectl’s <code>apply</code> and <code>delete</code> commands support directories as well, not only individual files. That means that, at this point, to build up all of our objects you could simply do <code>kubectl apply -f k8s/db</code> and <code>kubectl apply -f k8s/web</code>. This is much better than what we’ve been doing until now where we had to specify every single file. Still, with Kustomize, we can do better than that…</p>
</blockquote>
<h4 id="the-kustomization-file">The Kustomization file</h4>
<p>We can bring everything together with a <code>kustomization.yaml</code> file. For our setup, here’s what it could look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/kustomization.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Kustomization<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/persistent-volume.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/persistent-volume-claim.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/service.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/deployment.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/persistent-volume.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/persistent-volume-claim.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/service.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/deployment.yaml<span style="color:#bbb">
</span></code></pre></div><p>This first iteration of the Kustomization file is simple. It just lists all of our other config files in the <a href="https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/resource/"><code>resources</code></a> section in their relative locations. Save that as <code>k8s/kustomization.yaml</code> and you can apply it with the following:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -k k8s
</code></pre></div><p>The <code>-k</code> option tells <code>kubectl apply</code> to look for a Kustomization within the given directory and use that to build the cluster objects. After running it, you should see familiar output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">service/vehicle-quotes-db-service created
service/vehicle-quotes-web-service created
persistentvolume/vehicle-quotes-postgres-data-persisent-volume created
persistentvolume/vehicle-quotes-source-code-persisent-volume created
persistentvolumeclaim/vehicle-quotes-postgres-data-persisent-volume-claim created
persistentvolumeclaim/vehicle-quotes-source-code-persisent-volume-claim created
deployment.apps/vehicle-quotes-db created
deployment.apps/vehicle-quotes-web created
</code></pre></div><p>Feel free to explore the dashboard or <code>kubectl get</code> commands to see the objects that got created. You can connect to pods, run the app, query the database, everything. Just like we did before. The only difference is that now everything is neatly organized and there’s a single file that serves as bootstrap for the whole setup. All thanks to Kustomize and the <code>-k</code> option.</p>
<p><code>kubectl delete -k k8s</code> can be used to tear everything down.</p>
<h4 id="defining-reusable-configuration-values-with-configmaps">Defining reusable configuration values with ConfigMaps</h4>
<p>Another useful feature of Kustomize is <a href="https://kubernetes.io/docs/concepts/configuration/configmap/">ConfigMaps</a>. These allow us to specify configuration variables in the Kustomization and use them throughout the rest of the resource config files. A good candidate to demonstrate their use are the environment variables that configure our Postgres database and the connection string in our web application.</p>
<p>We’re going to make changes to the config so be sure to tear everything down with <code>kubectl delete -k k8s</code>.</p>
<p>We can start by adding the following to the <code>kustomization.yaml</code> file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> # k8s/kustomization.yaml
kind: Kustomization
resources:
- db/persistent-volume.yaml
- db/persistent-volume-claim.yaml
- db/service.yaml
- db/deployment.yaml
- web/persistent-volume.yaml
- web/persistent-volume-claim.yaml
- web/service.yaml
- web/deployment.yaml
<span style="color:#000;background-color:#dfd">+configMapGenerator:
</span><span style="color:#000;background-color:#dfd">+ - name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ literals:
</span><span style="color:#000;background-color:#dfd">+ - POSTGRES_DB=vehicle_quotes
</span><span style="color:#000;background-color:#dfd">+ - POSTGRES_USER=vehicle_quotes
</span><span style="color:#000;background-color:#dfd">+ - POSTGRES_PASSWORD=password
</span></code></pre></div><p>The <code>configMapGenerator</code> section is where the magic happens. We’ve kept it simple and defined the variables as literals. <code>configMapGenerator</code> is much more flexible than that though, accepting external configuration files. <a href="https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/configmapgenerator/">The official documentation</a> has more details.</p>
<p>Now, let’s see what we have to do to actually use those values in our configuration.</p>
<p>First up is the database deployment configuration file, <code>k8s/db/deployment.yaml</code>. Update its <code>env</code> section like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"># k8s/db/deployment.yaml
# ...
env:
<span style="color:#000;background-color:#fdd">- - name: POSTGRES_DB
</span><span style="color:#000;background-color:#fdd">- value: vehicle_quotes
</span><span style="color:#000;background-color:#fdd">- - name: POSTGRES_USER
</span><span style="color:#000;background-color:#fdd">- value: vehicle_quotes
</span><span style="color:#000;background-color:#fdd">- - name: POSTGRES_PASSWORD
</span><span style="color:#000;background-color:#fdd">- value: password
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ - name: POSTGRES_DB
</span><span style="color:#000;background-color:#dfd">+ valueFrom:
</span><span style="color:#000;background-color:#dfd">+ configMapKeyRef:
</span><span style="color:#000;background-color:#dfd">+ name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ key: POSTGRES_DB
</span><span style="color:#000;background-color:#dfd">+ - name: POSTGRES_USER
</span><span style="color:#000;background-color:#dfd">+ valueFrom:
</span><span style="color:#000;background-color:#dfd">+ configMapKeyRef:
</span><span style="color:#000;background-color:#dfd">+ name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ key: POSTGRES_USER
</span><span style="color:#000;background-color:#dfd">+ - name: POSTGRES_PASSWORD
</span><span style="color:#000;background-color:#dfd">+ valueFrom:
</span><span style="color:#000;background-color:#dfd">+ configMapKeyRef:
</span><span style="color:#000;background-color:#dfd">+ name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ key: POSTGRES_PASSWORD
</span><span style="color:#000;background-color:#dfd"></span># ...
</code></pre></div><p>Notice how we’ve replaced the simple key-value pairs with new, more complex objects. Their <code>name</code>s are still the same, they have to be because that’s what the Postgres database container expects. But instead of a literal, hard-coded value, we have changed them to these <code>valueFrom.configMapKeyRef</code> objects. Their <code>name</code>s match the <code>name</code> of the <code>configMapGenerator</code> we configured in the Kustomization. Their <code>key</code>s match the keys of the literal values that we specified in the <code>configMapGenerator</code>’s <code>literals</code> field. That’s how it all ties together.</p>
<p>Similarly, we can update the web application deployment configuration file, <code>k8s/web/deployment.yaml</code>. Its <code>env</code> section would look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"># k8s/web/deployment.yaml
# ...
env:
<span style="color:#000;background-color:#fdd">- - name: POSTGRES_DB
</span><span style="color:#000;background-color:#fdd">- value: vehicle_quotes
</span><span style="color:#000;background-color:#fdd">- - name: POSTGRES_USER
</span><span style="color:#000;background-color:#fdd">- value: vehicle_quotes
</span><span style="color:#000;background-color:#fdd">- - name: POSTGRES_PASSWORD
</span><span style="color:#000;background-color:#fdd">- value: password
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ - name: POSTGRES_DB
</span><span style="color:#000;background-color:#dfd">+ valueFrom:
</span><span style="color:#000;background-color:#dfd">+ configMapKeyRef:
</span><span style="color:#000;background-color:#dfd">+ name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ key: POSTGRES_DB
</span><span style="color:#000;background-color:#dfd">+ - name: POSTGRES_USER
</span><span style="color:#000;background-color:#dfd">+ valueFrom:
</span><span style="color:#000;background-color:#dfd">+ configMapKeyRef:
</span><span style="color:#000;background-color:#dfd">+ name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ key: POSTGRES_USER
</span><span style="color:#000;background-color:#dfd">+ - name: POSTGRES_PASSWORD
</span><span style="color:#000;background-color:#dfd">+ valueFrom:
</span><span style="color:#000;background-color:#dfd">+ configMapKeyRef:
</span><span style="color:#000;background-color:#dfd">+ name: postgres-config
</span><span style="color:#000;background-color:#dfd">+ key: POSTGRES_PASSWORD
</span><span style="color:#000;background-color:#dfd"></span> - name: CUSTOMCONNSTR_VehicleQuotesContext
value: Host=$(VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST);Database=$(POSTGRES_DB);Username=$(POSTGRES_USER);Password=$(POSTGRES_PASSWORD)
# ...
</code></pre></div><p>This is the exact same change as with the database deployment. Out with the hard coded values and in with the new ConfigMap-driven ones.</p>
<p>Try <code>kubectl apply -k k8s</code> and you’ll see that things are still working well. Try to connect to the web application pod and build and run the app.</p>
<blockquote>
<p>For data that’s important to secure like passwords, tokens, and keys, Kubernetes and Kustomize also offer <a href="https://kubernetes.io/docs/concepts/configuration/secret/">Secrets</a> and <a href="https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kustomize/"><code>secretGenerator</code></a>. Secrets are very similar to ConfigMaps in how they work, but are tailored specifically for handling secret data. You can learn more about them in the official documentation.</p>
</blockquote>
<h3 id="creating-variants-for-production-and-development-environments">Creating variants for production and development environments</h3>
<p>The crowning achievement of Kustomize is its ability to facilitate multiple deployment variants. Variants, as the name suggests, are variations of deployment configurations that are ideal for setting up various execution environments for an application. Think development, staging, production, etc., all based on a common set of reusable configurations to avoid superfluous repetition.</p>
<p>Kustomize does this by introducing the concepts of <a href="https://kubectl.docs.kubernetes.io/guides/introduction/kustomize/#2-create-variants-using-overlays">bases and overlays</a>. A base is a set of configs that can be reused but not deployed on its own, and overlays are the actual configurations that use and extend the base and can be deployed.</p>
<p>To demonstrate this, let’s build two variants: one for development and another for production. Let’s consider the one we’ve already built to be the development variant and work towards properly specifying it as so, and then building a new production variant.</p>
<blockquote>
<p>Note that the so-called “production” variant we’ll build is not actually meant to be production worthy. It’s just an example to illustrate the concepts and process of building bases and overlays. It does not meet the rigors of a proper production system.</p>
</blockquote>
<h4 id="creating-the-base-and-overlays">Creating the base and overlays</h4>
<p>The strategy I like to use is to just copy everything over from one variant to another, implement the differences, identify the common elements, and extract them into a base that both use.</p>
<p>Let’s begin by creating a new <code>k8s/dev</code> directory and move all of our YAML files into it. That will be our “development overlay”. Then, make a copy the <code>k8s/dev</code> directory and all of its contents and call it <code>k8s/prod</code>. That will be our “production overlay”. Let’s also create a <code>k8s/base</code> directory to store the common files. That will be our “base”. It should be like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">k8s
├── base
├── dev
│ ├── kustomization.yaml
│ ├── db
│ │ ├── deployment.yaml
│ │ ├── persistent-volume-claim.yaml
│ │ ├── persistent-volume.yaml
│ │ └── service.yaml
│ └── web
│ ├── deployment.yaml
│ ├── persistent-volume-claim.yaml
│ ├── persistent-volume.yaml
│ └── service.yaml
└── prod
├── kustomization.yaml
├── db
│ ├── deployment.yaml
│ ├── persistent-volume-claim.yaml
│ ├── persistent-volume.yaml
│ └── service.yaml
└── web
├── deployment.yaml
├── persistent-volume-claim.yaml
├── persistent-volume.yaml
└── service.yaml
</code></pre></div><p>Now we have two variants, but they don’t do us any good because they aren’t any different. We’ll now go through each file one by one and identify which aspects need to be the same and which need to be different between our two variants:</p>
<ol>
<li><code>db/deployment.yaml</code>: I want the same database instance configuration for both our variants. So we copy the file into <code>base/db/deployment.yaml</code> and delete <code>dev/db/deployment.yaml</code> and <code>prod/db/deployment.yaml</code>.</li>
<li><code>db/persistent-volume-claim.yaml</code>: This one is also the same for both variants. So we copy the file into <code>base/db/persistent-volume-claim.yaml</code> and delete <code>dev/db/persistent-volume-claim.yaml</code> and <code>prod/db/persistent-volume-claim.yaml</code>.</li>
<li><code>db/persistent-volume.yaml</code>: This file defines the location in the host machine that will be available for the Postgres instance that’s running in the cluster to store its data files. I do want this path to be different between variants. So let’s leave them where they are and do the following changes to them: For <code>dev/db/persistent-volume.yaml</code>, change its <code>spec.hostPath.path</code> to <code>"/path/to/vehicle-quotes-postgres-data-dev"</code>. For <code>prod/db/persistent-volume.yaml</code>, change its <code>spec.hostPath.path</code> to <code>"/path/to/vehicle-quotes-postgres-data-prod"</code>. Of course, adjust the paths to something that makes sense in your environment.</li>
<li><code>db/service.yaml</code>: There doesn’t need to be any difference in this file between the variants so we copy it into <code>base/db/service.yaml</code> and delete <code>dev/db/service.yaml</code> and <code>prod/db/service.yaml</code>.</li>
<li><code>web/deployment.yaml</code>: There are going to be quite a few differences between the dev and prod deployments of the web application. So we leave them as they are. Later we’ll see the differences in detail.</li>
<li><code>web/persistent-volume-claim.yaml</code>: This is also going to be different. Let’s leave it be now and we’ll come back to it later.</li>
<li><code>web/persistent-volume.yaml</code>: Same as <code>web/persistent-volume-claim.yaml</code>. Leave it be for now.</li>
<li><code>wev/service.yaml</code>: This one is going to be the same for both dev and prod so let’s do the usual and copy it into <code>base/web/service.yaml</code> and remove <code>dev/web/service.yaml</code> and <code>prod/web/service.yaml</code></li>
</ol>
<blockquote>
<p>The decisions made when designing these overlays and the base may seem arbitrary. That’s because they totally are. The purpose of this article is to demonstrate Kustomize’s features, not produce a real-world, production-worthy setup.</p>
</blockquote>
<p>Once all those changes are done, you should have the following file structure:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">k8s
├── base
│ ├── db
│ │ ├── deployment.yaml
│ │ ├── persistent-volume-claim.yaml
│ │ └── service.yaml
│ └── web
│ └── service.yaml
├── dev
│ ├── kustomization.yaml
│ ├── db
│ │ └── persistent-volume.yaml
│ └── web
│ ├── deployment.yaml
│ ├── persistent-volume-claim.yaml
│ └── persistent-volume.yaml
└── prod
├── kustomization.yaml
├── db
│ └── persistent-volume.yaml
└── web
├── deployment.yaml
├── persistent-volume-claim.yaml
└── persistent-volume.yaml
</code></pre></div><p>Much better, huh? We’ve gotten rid of quite a bit of repetition. But we’re not done just yet. The base also needs a Kustomization file. Let’s create it as <code>k8s/base/kustomization.yaml</code> and add these contents:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/base/kustomization.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Kustomization<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/persistent-volume-claim.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/service.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/deployment.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/service.yaml<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">configMapGenerator</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">literals</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- POSTGRES_DB=vehicle_quotes<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- POSTGRES_USER=vehicle_quotes<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- POSTGRES_PASSWORD=password<span style="color:#bbb">
</span></code></pre></div><p>As you can see, the file is very similar to the other one we created. We just list the resources that we moved into the <code>base</code> directory and define the database environment variables via the <code>configMapGenerator</code>. We need to define the <code>configMapGenerator</code> here because we’ve moved all the other files that use them into here.</p>
<p>Now that we have the base defined, we need to update the <code>kustomization.yaml</code> files of the overlays to use it. We also need to update them so that they only point to the resources that they need to.</p>
<p>Here’s how the changes to the “dev” overlay’s <code>kustomization.yaml</code> file look:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> # dev/kustomization.yaml
kind: Kustomization
<span style="color:#000;background-color:#dfd">+bases:
</span><span style="color:#000;background-color:#dfd">+ - ../base
</span><span style="color:#000;background-color:#dfd"></span>
resources:
- db/persistent-volume.yaml
<span style="color:#000;background-color:#fdd">- - db/persistent-volume-claim.yaml
</span><span style="color:#000;background-color:#fdd">- - db/service.yaml
</span><span style="color:#000;background-color:#fdd">- - db/deployment.yaml
</span><span style="color:#000;background-color:#fdd"></span> - web/persistent-volume.yaml
- web/persistent-volume-claim.yaml
<span style="color:#000;background-color:#fdd">- - web/service.yaml
</span><span style="color:#000;background-color:#fdd"></span> - web/deployment.yaml
<span style="color:#000;background-color:#fdd">-configMapGenerator:
</span><span style="color:#000;background-color:#fdd">- - name: postgres-config
</span><span style="color:#000;background-color:#fdd">- literals:
</span><span style="color:#000;background-color:#fdd">- - POSTGRES_DB=vehicle_quotes
</span><span style="color:#000;background-color:#fdd">- - POSTGRES_USER=vehicle_quotes
</span><span style="color:#000;background-color:#fdd">- - POSTGRES_PASSWORD=password
</span></code></pre></div><p>As you can see we removed the <code>configMapGenerator</code> and the individual resources that were already defined in the base. Most importantly, we’ve added a <code>bases</code> element that indicates that our Kustomization over on the <code>base</code> directory is this overlay’s base.</p>
<p>The changes to the “prod” overlay’s <code>kustomization.yaml</code> file are identical. Go ahead and make them.</p>
<p>At this point, you can run <code>kubectl apply -k k8s/dev</code> or <code>kubectl apply -k k8s/prod</code> and things should work just like before.</p>
<blockquote>
<p>Don’t forget to also do <code>kubectl delete -k k8s/dev</code> or <code>kubectl delete -k k8s/prod</code> when you’re done testing the previous commands, as we’ll continue doing changes to the configs. Keep in mind also that both variants can’t be deployed at the same time. So be sure <code>delete</code> one before <code>apply</code>ing the other.</p>
</blockquote>
<h4 id="developing-the-production-variant">Developing the production variant</h4>
<p>I want our production variant to use a different image for the web application. That means a new Dockerfile. If you downloaded the source code from the GitHub repo, you should see the production Dockerfile in the root directory of the repo. It’s called <code>Dockerfile.prod</code>.</p>
<p>Here’s what it looks like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-Dockerfile" data-lang="Dockerfile"><span style="color:#888"># Dockerfile.prod</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ARG</span> <span style="color:#369">VARIANT</span>=<span style="color:#d20;background-color:#fff0f0">"5.0"</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">FROM</span><span style="color:#d20;background-color:#fff0f0"> mcr.microsoft.com/dotnet/sdk:${VARIANT}</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> apt-get update && <span style="color:#038">export</span> <span style="color:#369">DEBIAN_FRONTEND</span>=noninteractive <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> && apt-get -y install --no-install-recommends postgresql-client-common postgresql-client<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> dotnet tool install --global dotnet-ef<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ENV</span> PATH <span style="color:#369">$PATH</span>:/root/.dotnet/tools<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">RUN</span> dotnet dev-certs https<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">WORKDIR</span><span style="color:#d20;background-color:#fff0f0"> /source</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">COPY</span> . .<span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2">
</span><span style="color:#a61717;background-color:#e3d2d2"></span><span style="color:#080;font-weight:bold">ENTRYPOINT</span> [<span style="color:#d20;background-color:#fff0f0">"tail"</span>, <span style="color:#d20;background-color:#fff0f0">"-f"</span>, <span style="color:#d20;background-color:#fff0f0">"/dev/null"</span>]<span style="color:#a61717;background-color:#e3d2d2">
</span></code></pre></div><p>The first takeaway from this production Dockerfile is that it is simpler, when compared to the development one. The image here is based on the official <code>dotnet/sdk</code> instead of the dev-ready one from <code>vscode/devcontainers/dotnet</code>. Also, this Dockerfile just copies all the source code into a <code>/source</code> directory within the image. This is because we want to “ship” the image with everything it needs to work without too much manual intervention. Also, we won’t be editing code live on the container, as opposed to how we set up the dev variant which allowed that. So we just copy the files over instead of leaving them out to provision them later via volumes. We’ll see how that pans out later.</p>
<p>Now that we have our production Dockerfile, we can build an image with it and push it to the registry so that Kubernetes can use it. So, save that file as <code>Dockerfile.prod</code> (or just use the one that’s already in the repo), and run the following commands:</p>
<p>Build the image with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ docker build . -f Dockerfile.prod -t localhost:32000/vehicle-quotes-prod:registry
</code></pre></div><p>And push it to the registry with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ docker push localhost:32000/vehicle-quotes-prod:registry
</code></pre></div><p>Now, we need to modify our prod variant’s deployment configuration so that it can work well with this new prod image. Here’s how the new <code>k8s/prod/web/deployment.yaml</code> should look:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/prod/web/deployment.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>apps/v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Deployment<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">selector</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">matchLabels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">replicas</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">1</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">template</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">labels</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">app</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">initContainers</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>await-db-ready<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>postgres:13<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">command</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"/bin/sh"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">args</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"-c"</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"until pg_isready -h $(VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST) -p 5432; do echo waiting for database; sleep 2; done;"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>build<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>localhost:32000/vehicle-quotes-dev:registry<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">workingDir</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/source"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">command</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"/bin/sh"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">args</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"-c"</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"dotnet restore -v n && dotnet ef database update && dotnet publish -c release -o /app --no-restore"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumeMounts</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">mountPath</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/app"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-storage<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">env</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_DB<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">valueFrom</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configMapKeyRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">key</span>:<span style="color:#bbb"> </span>POSTGRES_DB<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_USER<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">valueFrom</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configMapKeyRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">key</span>:<span style="color:#bbb"> </span>POSTGRES_USER<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_PASSWORD<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">valueFrom</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configMapKeyRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">key</span>:<span style="color:#bbb"> </span>POSTGRES_PASSWORD<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>CUSTOMCONNSTR_VehicleQuotesContext<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>Host=$(VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST);Database=$(POSTGRES_DB);Username=$(POSTGRES_USER);Password=$(POSTGRES_PASSWORD)<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">containers</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">image</span>:<span style="color:#bbb"> </span>localhost:32000/vehicle-quotes-dev:registry<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">workingDir</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/app"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">command</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"/bin/sh"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">args</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"-c"</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"dotnet VehicleQuotes.dll --urls=https://0.0.0.0:5001/"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">ports</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">containerPort</span>:<span style="color:#bbb"> </span><span style="color:#00d;font-weight:bold">5001</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"https"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumeMounts</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">mountPath</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/app"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-storage<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">env</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_DB<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">valueFrom</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configMapKeyRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">key</span>:<span style="color:#bbb"> </span>POSTGRES_DB<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_USER<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">valueFrom</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configMapKeyRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">key</span>:<span style="color:#bbb"> </span>POSTGRES_USER<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>POSTGRES_PASSWORD<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">valueFrom</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">configMapKeyRef</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>postgres-config<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">key</span>:<span style="color:#bbb"> </span>POSTGRES_PASSWORD<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>CUSTOMCONNSTR_VehicleQuotesContext<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">value</span>:<span style="color:#bbb"> </span>Host=$(VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST);Database=$(POSTGRES_DB);Username=$(POSTGRES_USER);Password=$(POSTGRES_PASSWORD)<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">limits</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">memory</span>:<span style="color:#bbb"> </span>2Gi<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">cpu</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"1"</span><span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">volumes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-storage<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">emptyDir</span>:<span style="color:#bbb"> </span>{}<span style="color:#bbb">
</span></code></pre></div><p>This deployment config is similar to the one from the dev variant, but we’ve changed a few elements on it.</p>
<h4 id="init-containers">Init containers</h4>
<p>The most notable change is that we added an <code>initContainers</code> section. Init Containers are one-and-done containers that run specific processes during pod initialization. They are good for doing any sort of initialization tasks that need to be run once, before a pod is ready to work. After they’ve done their task, they go away, and the pod is left with the containers specified in the <code>containers</code> section, like usual. In this case, we’ve added two init containers.</p>
<p>First is the <code>await-db-ready</code> one. This is a simple container based on the <code>postgres:13</code> image that just sits there waiting for the database to become available. This is thanks to its <code>command</code> and <code>args</code>, which make up a simple shell script that leverages the <code>pg_isready</code> tool to continuously check if connections can be made to our database:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">command</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"/bin/sh"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">args</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"-c"</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"until pg_isready -h $(VEHICLE_QUOTES_DB_SERVICE_SERVICE_HOST) -p 5432; do echo waiting for database; sleep 2; done;"</span>]<span style="color:#bbb">
</span></code></pre></div><p>This will cause pod initialization to stop until the database is ready.</p>
<blockquote>
<p>Thanks to <a href="https://medium.com/@xcoulon/initializing-containers-in-order-with-kubernetes-18173b9cc222">this blog post</a> for the very useful recipe.</p>
</blockquote>
<p>We need to wait for the database to be ready before continuing because of what the second init container does. Among other things, the <code>build</code> init container sets up the database. The database needs to be available for it to be able to to do that. The init container also downloads dependencies, builds the app, produces the deployable artifacts and copies them over to the directory from which the app will run: <code>/app</code>. You can see that all that is specified in the <code>command</code> and <code>args</code> elements, which define a few shell commands to do those tasks.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">command</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"/bin/sh"</span>]<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">args</span>:<span style="color:#bbb"> </span>[<span style="color:#d20;background-color:#fff0f0">"-c"</span>,<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"dotnet restore -v n && dotnet ef database update && dotnet publish -c release -o /app --no-restore"</span>]<span style="color:#bbb">
</span></code></pre></div><p>Another interesting aspect of this deployment is the volume that we’ve defined. It’s at the bottom of the file, take a quick look:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">volumes</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-source-code-storage<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">emptyDir</span>:<span style="color:#bbb"> </span>{}<span style="color:#bbb">
</span></code></pre></div><p>This one is different from the ones we’ve seen before which relied on persistent volumes and persistent volume claims. This one uses <code>emptyDir</code>. This means that this volume will provide storage that is persistent throughout the lifetime of the pod. That is, not tied to any specific container. In other words, even when the container goes away, the files in this volume will stay. This is a mechanism that’s useful when we want one container to produce some files that another container will use. In our case, the <code>build</code> init container produces the artifacts/binaries that the main <code>vehicle-quotes-web</code> container will use to actually run the web app.</p>
<p>The only other notable difference of this deployment is how its containers use the new prod image that we built before, instead of the dev one. That is, it uses <code>localhost:32000/vehicle-quotes-prod:registry</code> instead of <code>localhost:32000/vehicle-quotes-dev:registry</code>.</p>
<p>The rest of the deployment doesn’t have anything we haven’t already seen. Feel free to explore it.</p>
<p>As you saw, this prod variant doesn’t need to access the source code via a persistent volume. So, we don’t need PV and PVC definitions for it. So feel free to delete <code>k8s/prod/web/persistent-volume.yaml</code> and <code>k8s/prod/web/persistent-volume-claim.yaml</code>. Remember to also remove them from the <code>resources</code> section in <code>k8s/prod/kustomization.yaml</code>.</p>
<p>With those changes done, we can fire up our prod variants with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ kubectl apply -k k8s/prod
</code></pre></div><p>The web pod will take quite a while to properly start up because it’s downloading a lot of dependencies. Remember that you can use <code>kubectl get pods -A</code> to see their current status. Take note of their names and you would also be able to see container specific logs.</p>
<ul>
<li>Use <code>kubectl logs -f <vehicle-quotes-web-pod-name> await-db-ready</code> to see the logs from the <code>await-db-ready</code> init container.</li>
<li>Use <code>kubectl logs -f <vehicle-quotes-web-pod-name> build</code> to see the logs from the <code>build</code> init container.</li>
</ul>
<blockquote>
<p>If this were an actual production setup, and we were worried about pod startup time, there’s one way we could make it faster. We could perform the “download dependencies” step when building the production image instead of when deploying the pods. So, we could have our <code>Dockerfile.prod</code> call <code>dotnet restore -v n</code>, instead of the <code>build</code> init container. That way building the image would take more time, but it would have all dependencies already baked in by the time Kubernetes tries to use it to build containers. Then the web pod would start up much faster.</p>
</blockquote>
<p>This deployment automatically starts the web app, so after the pods are in the “Running” status (or green in the dashboard!), we can just navigate to the app via a web browser. We’ve configured the deployment to only work over HTTPS (as given by the <code>ports</code> section in the <code>vehicle-quotes-web</code> container), so this is the only URL that’s available to us: https://localhost:30001. We can navigate to it and see the familiar screen:</p>
<p><img src="/blog/2022/01/kubernetes-101/swagger-prod.webp" alt="SwaggerUI on prod"></p>
<p>At this point, we finally have fully working, distinct variants. However, we can take our configuration a few steps further by leveraging some additional Kustomize features.</p>
<h4 id="using-patches-for-small-precise-changes">Using patches for small, precise changes</h4>
<p>Right now, the persistent volume configurations for the databases of both variants are pretty much identical. The only difference is the <code>hostPath</code>. With patches, we can focus in on that property and vary it specifically.</p>
<p>To do it, we first copy either of the variant’s <code>db/persistent-volume.yaml</code> into <code>k8s/base/db/persistent-volume.yaml</code>. We also need to add it under <code>resources</code> on <code>k8s/base/kustomization.yaml</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> # k8s/base/kustomization.yaml
# ...
resources:
<span style="color:#000;background-color:#dfd">+ - db/persistent-volume.yaml
</span><span style="color:#000;background-color:#dfd"></span> - db/persistent-volume-claim.yaml
# ...
</code></pre></div><p>That will serve as the common ground for overlays to “patch over”. Now we can create the patches. First, the one for the one for the dev variant:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/dev/db/persistent-volume-host-path.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>PersistentVolume<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-postgres-data-persisent-volume<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">hostPath</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">path</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/home/kevin/projects/vehicle-quotes-postgres-data-dev"</span><span style="color:#bbb">
</span></code></pre></div><p>And then the one for the prod variant:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/prod/db/persistent-volume-host-path.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">apiVersion</span>:<span style="color:#bbb"> </span>v1<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>PersistentVolume<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">metadata</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-postgres-data-persisent-volume<span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">spec</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">hostPath</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">path</span>:<span style="color:#bbb"> </span><span style="color:#d20;background-color:#fff0f0">"/home/kevin/projects/vehicle-quotes-postgres-data-prod"</span><span style="color:#bbb">
</span></code></pre></div><p>As you can see, these patches are sort of truncated persistent volume configs which only include the <code>kind</code>, <code>metadata.name</code>, and the value that actually changes: the <code>hostPath</code>.</p>
<p>Once those are saved, we need to include them in their respective <code>kustomization.yaml</code>. It’s the same modification to both <code>k8s/dev/kustomization.yaml</code> and <code>k8s/prod/kustomization.yaml</code>. Just remove the <code>db/persistent-volume.yaml</code> item from their <code>resources</code> sections and add the following to both of them:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">patches</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/persistent-volume-host-path.yaml<span style="color:#bbb">
</span></code></pre></div><p>Right now, <code>k8s/dev/kustomization.yaml</code> should be:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/dev/kustomization.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Kustomization<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">bases</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- ../base<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/persistent-volume.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/persistent-volume-claim.yaml<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/deployment.yaml<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">patches</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/persistent-volume-host-path.yaml<span style="color:#bbb">
</span></code></pre></div><p>And <code>k8s/prod/kustomization.yaml</code> should be:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#888"># k8s/prod/kustomization.yaml</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">kind</span>:<span style="color:#bbb"> </span>Kustomization<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">bases</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- ../base<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">resources</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- web/deployment.yaml<span style="color:#bbb">
</span><span style="color:#bbb">
</span><span style="color:#bbb"></span><span style="color:#b06;font-weight:bold">patches</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- db/persistent-volume-host-path.yaml<span style="color:#bbb">
</span></code></pre></div><h4 id="overriding-container-images">Overriding container images</h4>
<p>Another improvement we can make is to use the <code>images</code> element in the <code>kustomization.yaml</code> files to control the web app images used by the deployments in the variants. This is easier for maintenance as it’s defined in one single, expected place. Also, the full name of the image doesn’t have to be used throughout the configs so it reduces repetition.</p>
<p>To put it in practice, add the following at the end of the <code>k8s/dev/kustomization.yaml</code> file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">images</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">newName</span>:<span style="color:#bbb"> </span>localhost:32000/vehicle-quotes-dev<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">newTag</span>:<span style="color:#bbb"> </span>registry<span style="color:#bbb">
</span></code></pre></div><p>Similar thing with <code>k8s/prod/kustomization.yaml</code>, only use the prod image for this one:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#b06;font-weight:bold">images</span>:<span style="color:#bbb">
</span><span style="color:#bbb"> </span>- <span style="color:#b06;font-weight:bold">name</span>:<span style="color:#bbb"> </span>vehicle-quotes-web<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">newName</span>:<span style="color:#bbb"> </span>localhost:32000/vehicle-quotes-prod<span style="color:#bbb">
</span><span style="color:#bbb"> </span><span style="color:#b06;font-weight:bold">newTag</span>:<span style="color:#bbb"> </span>registry<span style="color:#bbb">
</span></code></pre></div><p>Now, we can replace any mention of <code>localhost:32000/vehicle-quotes-dev</code> in the dev variant, and any mention of <code>localhost:32000/vehicle-quotes-prod</code> in the prod variant with <code>vehicle-quotes-web</code>. Which is simpler.</p>
<p>In <code>k8s/dev/web/deployment.yaml</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> # ...
spec:
containers:
- name: vehicle-quotes-web
<span style="color:#000;background-color:#fdd">- image: localhost:32000/vehicle-quotes-dev:registry
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ image: vehicle-quotes-web
</span><span style="color:#000;background-color:#dfd"></span> ports:
# ...
</code></pre></div><p>And in <code>k8s/prod/web/deployment.yaml</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> # ...
- name: build
<span style="color:#000;background-color:#fdd">- image: localhost:32000/vehicle-quotes-prod:registry
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ image: vehicle-quotes-web
</span><span style="color:#000;background-color:#dfd"></span> workingDir: "/source"
command: ["/bin/sh"]
# ...
containers:
- name: vehicle-quotes-web
<span style="color:#000;background-color:#fdd">- image: localhost:32000/vehicle-quotes-prod:registry
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ image: vehicle-quotes-web
</span><span style="color:#000;background-color:#dfd"></span> workingDir: "/app"
# ...
</code></pre></div><p>Once that’s all done, you should be able to <code>kubectl apply -k k8s/dev</code> or <code>kubectl apply -k k8s/prod</code> and everything should work fine. Be sure to <code>kubectl delete</code> before <code>kubectl apply</code>ing a different variant though, as both of them cannot coexist in the same cluster, due to many objects having the same name.</p>
<h3 id="closing-thoughts">Closing thoughts</h3>
<p>Wow! That was a good one. In this post I’ve captured all the knowledge that I wish I had when I first encountered Kubernetes. We went from knowing nothing to being able to put together a competent environment. We figured out how to install Kubernetes locally via MicroK8s, along with a few useful add-ons. We learned about the main concepts in Kubernetes like nodes, pods, images, containers, deployments, services, and persistent volumes. Most importantly, we learned how to define and create them using a declarative configuration file approach.</p>
<p>Then, we learned about Kustomize and how to use it to implement variants of our configurations. And we did all that by actually getting our hands dirty and, step by step, deploying a real web application and its backing database system. When all was said and done, a simple <code>kubeclt apply -k <kustomization></code> was all it took to get the app up and running fully. Not bad, eh?</p>
<h3 id="useful-commands">Useful commands</h3>
<ul>
<li>Start up MicroK8s: <code>microk8s start</code></li>
<li>Shut down MicroK8s: <code>microk8s stop</code></li>
<li>Check MicroK8s info: <code>microk8s status</code></li>
<li>Start up the K8s dashboard: <code>microk8s dashboard-proxy</code></li>
<li>Get available pods on all namespaces: <code>kubectl get pods -A</code></li>
<li>Watch and follow the logs on a specific container in a pod: <code>kubectl logs -f <POD_NAME> <CONTAINER_NAME></code></li>
<li>Open a shell into the default container in a pod: <code>kubectl exec -it <POD_NAME> -- bash</code></li>
<li>Create a K8s resource given a YAML file or directory: <code>kubectl apply -f <FILE_OR_DIRECTORY_NAME></code></li>
<li>Delete a K8s resource given a YAML file or directory: <code>kubectl delete -f <FILE_OR_DIRECTORY_NAME></code></li>
<li>Create K8s resources with Kustomize: <code>kubectl apply -k <KUSTOMIZATION_DIR></code></li>
<li>Delete K8s resources with Kustomize: <code>kubectl delete -k <KUSTOMIZATION_DIR></code></li>
<li>Build custom images for the K8s registry: <code>docker build . -f <DOCKERFILE> -t localhost:32000/<IMAGE_NAME>:registry</code></li>
<li>Push custom images to the K8s registry: <code>docker push localhost:32000/<IMAGE_NAME>:registry</code></li>
</ul>
<h3 id="table-of-contents">Table of contents</h3>
<ul>
<li><a href="#what-is-kubernetes">What is Kubernetes?</a>
<ul>
<li><a href="#nodes-pods-and-containers">Nodes, pods and containers</a></li>
</ul>
</li>
<li><a href="#installing-and-setting-up-kubernetes">Installing and setting up Kubernetes</a>
<ul>
<li><a href="#installing-microk8s">Installing MicroK8s</a></li>
<li><a href="#introducing-kubectl">Introducing kubectl</a></li>
<li><a href="#installing-add-ons">Installing add-ons</a></li>
<li><a href="#introducing-the-dashboard">Introducing the Dashboard</a></li>
</ul>
</li>
<li><a href="#deploying-applications-into-a-kubernetes-cluster">Deploying applications into a Kubernetes cluster</a>
<ul>
<li><a href="#deployments">Deployments</a></li>
<li><a href="#using-kubectl-to-explore-a-deployment">Using kubectl to explore a deployment</a></li>
<li><a href="#using-the-dashboard-to-explore-a-deployment">Using the dashboard to explore a deployment</a></li>
<li><a href="#dissecting-the-deployment-configuration-file">Dissecting the deployment configuration file</a></li>
<li><a href="#connecting-to-the-containers-in-the-pods">Connecting to the containers in the pods</a></li>
<li><a href="#services">Services</a></li>
<li><a href="#accessing-an-application-via-a-service">Accessing an application via a service</a></li>
</ul>
</li>
<li><a href="#deploying-our-own-custom-application">Deploying our own custom application</a>
<ul>
<li><a href="#what-are-we-building">What are we building</a></li>
</ul>
</li>
<li><a href="#deploying-the-database">Deploying the database</a>
<ul>
<li><a href="#connecting-to-the-database">Connecting to the database</a></li>
<li><a href="#persistent-volumes-and-claims">Persistent volumes and claims</a></li>
<li><a href="#configuration-files-for-the-pv-and-pvc">Configuration files for the PV and PVC</a></li>
<li><a href="#configuring-the-deployment-to-use-the-pvc">Configuring the deployment to use the PVC</a></li>
<li><a href="#applying-changes">Applying changes</a></li>
<li><a href="#exposing-the-database-as-a-service">Exposing the database as a service</a></li>
</ul>
</li>
<li><a href="#deploying-the-web-application">Deploying the web application</a>
<ul>
<li><a href="#building-the-web-application-image">Building the web application image</a></li>
<li><a href="#making-the-image-accessible-to-kubernetes">Making the image accessible to Kubernetes</a></li>
<li><a href="#deploying-the-web-application-1">Deploying the web application</a></li>
<li><a href="#starting-the-application">Starting the application</a></li>
</ul>
</li>
<li><a href="#putting-it-all-together-with-kustomize">Putting it all together with Kustomize</a>
<ul>
<li><a href="#the-kustomization-file">The Kustomization file</a></li>
<li><a href="#defining-reusable-configuration-values-with-configmaps">Defining reusable configuration values with ConfigMaps</a></li>
</ul>
</li>
<li><a href="#creating-variants-for-production-and-development-environments">Creating variants for production and development environments</a>
<ul>
<li><a href="#creating-the-base-and-overlays">Creating the base and overlays</a></li>
<li><a href="#developing-the-production-variant">Developing the production variant</a></li>
<li><a href="#init-containers">Init containers</a></li>
<li><a href="#using-patches-for-small-precise-changes">Using patches for small, precise changes</a></li>
<li><a href="#overriding-container-images">Overriding container images</a></li>
</ul>
</li>
<li><a href="#closing-thoughts">Closing thoughts</a></li>
<li><a href="#useful-commands">Useful commands</a></li>
<li><a href="#table-of-contents">Table of contents</a></li>
</ul>
.NET/C# developer job openinghttps://www.endpointdev.com/blog/2021/11/dotnet-developer-job/2021-11-09T00:00:00+00:00Jon Jensen
<p><img src="/blog/2018/08/job-opening-dotnet-csharp-javascript-developer/25677299412_9934b2ec91_o-mod.jpg" alt="programmer at keyboard on desk" /><br><a href="https://www.wocintechchat.com/">Photo by #WOCinTech Chat</a> · <a href="https://www.flickr.com/photos/wocintechchat/25677299412/">CC BY 2.0, modified</a></p>
<p>We are seeking a full-time <strong>.NET/C# software developer based in the United States</strong> to work with us on our clients’ applications.</p>
<p>End Point Dev is an Internet technology consulting company based in New York City, with 50 employees serving many clients ranging from small family businesses to large corporations. The company is going strong after 26 years in business!</p>
<p>Even before the pandemic most of us worked remotely from home offices. We collaborate using SSH, Git, project tracking tools, Zulip chat, video conferencing, and of course email and phones.</p>
<h3 id="what-you-will-be-doing">What you will be doing:</h3>
<ul>
<li>Develop new web applications and support existing ones for our clients.</li>
<li>Work together with End Point Dev co-workers and our clients’ in-house staff.</li>
<li>Use your desktop operating system of choice: Windows, macOS, or Linux.</li>
<li>Enhance open source software and contribute back as opportunity arises.</li>
</ul>
<h3 id="youll-need-professional-development-experience-with">You’ll need professional development experience with:</h3>
<ul>
<li><strong>3+ years of development with .NET and C#</strong></li>
<li>Databases such as SQL Server, PostgreSQL, Redis, Solr, Elasticsearch, etc.</li>
<li>Front-end web development with HTML, CSS, JavaScript and frameworks such as Vue, React, Angular</li>
<li>Security consciousness such as under PCI-DSS for ecommerce or HIPAA medical data</li>
<li>Git version control</li>
<li>Automated testing</li>
<li>HTTP, REST APIs</li>
</ul>
<h3 id="you-have-these-important-work-traits">You have these important work traits:</h3>
<ul>
<li>Strong verbal and written communication skills</li>
<li>An eye for detail</li>
<li>Tenacity in solving problems and focusing on customer needs</li>
<li>A feeling of ownership of your projects</li>
<li>Work both independently and as part of a team</li>
<li>A good remote work environment</li>
</ul>
<p><strong>For some of our clients you will need to submit to and pass a criminal background check.</strong></p>
<h3 id="what-work-here-offers">What work here offers:</h3>
<ul>
<li>Collaboration with knowledgeable, friendly, helpful, and diligent co-workers around the world</li>
<li>Work from your home office, or from our offices in New York City and the Tennessee Tri-Cities area</li>
<li>Flexible, sane work hours</li>
<li>Paid holidays and vacation</li>
<li>Health insurance subsidy and 401(k) retirement savings plan</li>
<li>Annual bonus opportunity</li>
</ul>
<h3 id="get-in-touch-with-us">Get in touch with us:</h3>
<p><del>Please email us an introduction to jobs@endpointdev.com to apply.</del>
<strong>(This job has been filled.)</strong></p>
<p>Include your location, a resume/CV, your Git repository or LinkedIn URLs, and whatever else may help us get to know you.</p>
<p>We look forward to hearing from you! Direct work seekers only, please—this role is not for agencies or subcontractors.</p>
<p>We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of sex/gender, race, religion, color, national origin, sexual orientation, age, marital status, veteran status, or disability status.</p>
.NET Conf 2021 is coming!https://www.endpointdev.com/blog/2021/10/net-conf-2021-is-coming/2021-10-27T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2021/10/net-conf-2021-is-coming/dotnet-conf-2021.jpg" alt=".NET Conf 2021 is coming!"></p>
<p>It’s that time of the year again! It has been almost one year since .NET 5 was launched at <a href="/blog/2020/11/dotnet-5-released-net-conf-2020/">.NET Conf 2020</a>, unifying .NET Framework and .NET Core into a single open-source and cross-platform framework. With .NET 6 around the corner, it’s time to prepare for the new edition, <a href="https://www.dotnetconf.net/">.NET Conf 2021</a>, starting on November 9th.</p>
<p>This edition will be the 11th online conference, and the <a href="https://www.dotnetconf.net/agenda">Agenda</a> will mainly focus on the .NET 6 launch and the new C# 10, along with some coding challenges and community sessions. The event is organized by both the .NET community and Microsoft. The main changes that will likely be discussed at the conference, based on what we’ve seen through the previews, might include:</p>
<h3 id="mobile-development">Mobile development</h3>
<p>We will see several changes to how web mobile app development works under .NET 6. With a smaller, optimized, and optional SDK for mobile based on <a href="https://dotnet.microsoft.com/apps/xamarin">Xamarin</a>, now called Multi-platform App UI or MAUI, developing an app that targets multiple mobile platforms should be simpler than ever.</p>
<p><img src="/blog/2021/10/net-conf-2021-is-coming/dotnet-maui.jpg" alt=".NET 6 and MAUI"></p>
<h3 id="hot-reload-after-some-struggling">Hot reload (after some struggling)</h3>
<p>The <a href="https://devblogs.microsoft.com/dotnet/introducing-net-hot-reload/">Hot Reload feature</a>, which will allow us to make and apply changes to the code at execution time, will be finally available in .NET 6. There was some noise about it in the past days since Microsoft made a decision to lock it to Visual Studio 2022, which is a Windows-limited mostly-paid product. But after the open source community made it clear they were angry about it, Microsoft <a href="https://devblogs.microsoft.com/dotnet/update-on-net-hot-reload-progress-and-visual-studio-2022-highlights/">reversed the change</a> and the feature will be available for all platforms.</p>
<h3 id="blazor-6">Blazor 6</h3>
<p>The new version of Blazor will also have improvements. One of the main features is the implementation of <a href="https://devblogs.microsoft.com/aspnet/asp-net-core-updates-in-net-6-preview-4/#blazor-webassembly-ahead-of-time-aot-compilation">ahead-of-time compilation</a>, that allows generating WebAssembly code during the publishing process. That way, the application performance is increased, since it can run natively instead of needing a .NET IL interpreter.</p>
<h3 id="lts-long-term-support">LTS (Long Term Support)</h3>
<p>.NET 6 is a Long Term Support version, which means that it will have support for at least 3 years after its release. The current version, .NET 5, is a General Availability (GA) version, which means its support will likely end six months after the next release is available.</p>
<p>Several improvements were introduced to different aspects of the framework, including <a href="https://www.reddit.com/r/dotnet/comments/o21i5k/webassembly_aot_support_is_now_available_with_net/">AOT compilation</a> or the <a href="https://dotnetcoretutorials.com/2021/07/16/building-minimal-apis-in-net-6/">Minimal API Framework</a>. There are also improvements to the build speed as we can see in the chart below. A complete list of breaking changes can be seen <a href="https://docs.microsoft.com/en-us/dotnet/core/compatibility/6.0">here</a>.</p>
<p><img src="/blog/2021/10/net-conf-2021-is-coming/dotnet-5-vs-dotnet-6-improvements.jpg" alt=".NET 5 vs. .NET 6 build improvements"></p>
<p>Bonus: Although not part of the event itself, the new version of Visual Studio will also be available from November 8th, and it will be launched on an <a href="https://visualstudio.microsoft.com/launch/">online event</a> that Microsoft is preparing with several presentations from the development team.</p>
<p>As we did last year, the .NET team at End Point Dev will be listening to the talks and presentations from the conference, and trying out the new features that will be publicly available on November 9th. Exciting days ahead! We hope to see you there.</p>
Deploying a .NET 5 app on IIShttps://www.endpointdev.com/blog/2021/09/deploying-dotnet-5-app-iis/2021-09-27T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2021/09/deploying-dotnet-5-app-iis/puzzle-cropped.jpg" alt="Puzzle">
<a href="https://flic.kr/p/4riBj8">Photo</a> by <a href="https://www.flickr.com/photos/nexus_icon/">Christian Cable</a>, CC BY 2.0</p>
<p>.NET 5 has been around for a few years now, after being <a href="/blog/2020/11/dotnet-5-released-net-conf-2020/">released at .NET Conf 2020</a>, containing the best of both worlds: .NET Core, including multi-platform support and several performance improvements, and .NET Framework, including Windows desktop development support with WPF and Windows Forms (UWP is also supported, but not officially yet).</p>
<p>A .NET Core-based project can be published into any platform (as long as we’re not depending on libraries targeted to .NET Framework), allowing us to save costs by hosting on Linux servers and increasing performance by having cheaper scalability options. But most developers are still using Windows with Internet Information Services (IIS) as the publishing target, likely due to the almost 20 years of history of .NET Framework, compared to the relatively short history of .NET Core, launched in 2016.</p>
<h3 id="our-net-project">Our .NET project</h3>
<p>We won’t review the <a href="/blog/2021/07/dotnet-5-web-api/">steps needed to set up a new .NET 5 project</a>, since this time we are only focusing on publishing what we already have developed. But to understand how our application will integrate with IIS and the framework, it’s important to note a fundamental change any .NET 5 project has in comparison with a .NET Framework one:</p>
<p>Since .NET 5 is .NET Core in its foundation, our project output will actually be a console application. If we create a new .NET Core project, no matter which version we are using, we will find a <code>Program.cs</code> file in the root with an application entry point in it, that will look somewhat similar to the one below:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-c#" data-lang="c#"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Program</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> <span style="color:#080;font-weight:bold">void</span> Main(<span style="color:#888;font-weight:bold">string</span>[] args)
{
BuildWebHost(args).Run();
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> IWebHost BuildWebHost(<span style="color:#888;font-weight:bold">string</span>[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.Build();
}
</code></pre></div><blockquote>
<p>The <a href="https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/web-host?view=aspnetcore-5.0"><code>WebHost</code></a> object will be the one processing requests to the app, as well as setting configuration like the content root, accessing environment variables, and logging.</p>
</blockquote>
<p>This application needs to be executed by the <code>dotnet</code> process, which comes with any .NET 5 runtime.</p>
<h3 id="installing-a-net-5-runtime">Installing a .NET 5 runtime</h3>
<p>The first step we need to do in our destination server, is to prepare the environment to run .NET 5 apps by installing the <a href="https://dotnet.microsoft.com/download/dotnet/5.0">.NET 5 Hosting Bundle</a>, a 65 MB setup file which has all the basic stuff needed to run .NET 5 on Windows. Since .NET can also run on Linux and macOS, installer executables are available for these operating systems as well.</p>
<p><img src="/blog/2021/09/deploying-dotnet-5-app-iis/dotnet-hosting-bundle-screenshot.jpg" alt="Installing the hosting bundle"></p>
<p>Once the installation finishes, we will need to restart IIS by typing <code>iisreset</code> on an elevated command prompt.</p>
<h3 id="creating-the-application-pool">Creating the application pool</h3>
<p>It’s always recommended to create a new application pool for a new website that will be published. That allows us to run the website in a separate IIS process, which is safer and prevents other websites from crashing if an application throws an unhandled exception. To create a new application pool, right-click on the “Application pools” section on the IIS Manager sidebar and choose the option “Add application pool”.</p>
<p>Since .NET 5 is based on .NET Core, the application pool we create will not be loaded inside the <a href="https://docs.microsoft.com/en-us/dotnet/standard/clr">.NET Framework runtime environment</a>. All .NET 5 applications will run by calling the external <code>dotnet</code> process, which is the reason why we need to install a separate hosting bundle in the first place.</p>
<p>That means that, when we are creating our application pool, we will need to set the .NET CLR version to “No managed code” before saving changes, as shown below:</p>
<p><img src="/blog/2021/09/deploying-dotnet-5-app-iis/iis-new-app-pool.jpg" alt="App pool settings"></p>
<h3 id="creating-the-new-website">Creating the new website</h3>
<p>With the bundle installed and a new application pool created, it’s time to add the new website where our application will be published to. Right-click on the “Sites” section on the IIS Manager sidebar and choose the “Add Website” option.</p>
<p>We can choose any name to identify the new website. The important thing is to point the website to the newly created application pool, and bind it to the correct IP address and domain/host name, as shown below:</p>
<p><img src="/blog/2021/09/deploying-dotnet-5-app-iis/iis-new-website.jpg" alt="Setting up a new website"></p>
<p>Once we accept the changes, the new website will be automatically started, which means we should be able to reach the IP address/hostname we entered. We will get a default page or a 404 response, depending on how our IIS instance is configured, since we haven’t published our application yet.</p>
<h3 id="publishing-our-project">Publishing our project</h3>
<p>Finally, it’s time to publish our .NET application into the new website. If we have Visual Studio, we can have the IDE automatically upload our content to IIS and publish it by right-clicking on our project and choosing “Publish”. Or we can use the <code>dotnet</code> command with the <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish"><code>publish</code></a> parameter to do it ourselves.</p>
<p>I usually prefer to do a manual publish into a local folder, and then decide which content I need to copy into the destination. Sometimes we only update a portion of the backend logic, in which case copying the output DLLs is all we need to do.</p>
<p>If we choose to let Visual Studio handle the publishing process, we need to choose “Web Server (IIS) / Web deploy” as our publish destination and enter the information needed for Visual Studio to connect to the server and copy the files:</p>
<p><img src="/blog/2021/09/deploying-dotnet-5-app-iis/visual-studio-publish-screen.jpg" alt="Publishing the website"></p>
<blockquote>
<p>We need to make sure that the “site name” we enter here corresponds to the site name we entered when we created the new website on the server.</p>
</blockquote>
<p>If you prefer to manually copy the files like me, on the destination screen choose “Folder” instead of “Web Server (IIS)”. That option will copy the project’s output into the specified folder, so it can be manually copied into our website’s root later.</p>
<p>And that’s it! Our project is now published into the website we created. We can hit our IP address/hostname again and we should now get our website’s default response.</p>
<p>To sum it up, the main difference between publishing a .NET Framework app and a .NET 5 app is that we need to install a runtime and do a couple of tweaks when setting up the new application pool. But in general, it’s a pretty straightforward process.</p>
Monitoring Settings Changes in ASP.NET Corehttps://www.endpointdev.com/blog/2021/09/monitoring-settings-changes-in-asp-net-core/2021-09-22T00:00:00+00:00Daniel Gomm
<p><img src="/blog/2021/09/monitoring-settings-changes-in-asp-net-core/ripples.jpg" alt="Ripples in water"><br>
<a href="https://unsplash.com/photos/Q5QspluNZmM">Photo</a> by <a href="https://unsplash.com/@dreamsoftheoceans">Linus Nylund</a> on Unsplash</p>
<p>Did you know you can directly respond to config file changes in ASP.NET Core? By using the <code>IOptionsMonitor<T></code> interface, it’s possible to run a lambda function every time a config file is updated.</p>
<p>For those who aren’t too familiar with it, ASP.NET Core configuration is handled using the <a href="https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-5.0">Options Pattern</a>. That is, you create a model class that has a field for each property in your config file, and then set up the application to map the configuration to that class, which can be injected into controllers and services. In your controllers, you can request the configuration class <code>T</code> as a dependency by using an options wrapper class like <code>IOptions<T></code>.</p>
<p>While this works just fine in some cases, it doesn’t allow you to actually run code when the configuration is changed. However, if you request your configuration using <code>IOptionsMonitor<T></code>, you get the ability to define a lambda function to run every time <code>appsettings.json</code> is changed.</p>
<p>One use case for this functionality would be if you wanted to maintain a list in the config file, and log every time that list was changed. In this article I’ll explain how to set up an ASP.NET Core 5.0 API to run custom code whenever changes are made to <code>appsettings.json</code>.</p>
<h3 id="setting-up-the-api">Setting up the API</h3>
<p>In order to use the options pattern in your API, you’ll first need to add the options services to the container using the <code>services.AddOptions()</code> method. Then, you can register your custom configuration class (in this example, <code>MyOptions</code>) to be bound to a specific section in <code>appsettings.json</code> (in this example, <code>"myOptions"</code>).</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> ConfigureServices(IServiceCollection services)
{
<span style="color:#888">// ...
</span><span style="color:#888"></span>
<span style="color:#888">// Add options services to the container
</span><span style="color:#888"></span> services.AddOptions();
<span style="color:#888">// Configure the app to map the "myOptions" section of
</span><span style="color:#888"></span> <span style="color:#888">// the config file to the MyOptions model class
</span><span style="color:#888"></span> services.Configure<MyOptions>(
Configuration.GetSection(<span style="color:#d20;background-color:#fff0f0">"myOptions"</span>)
);
}
</code></pre></div><h3 id="monitoring-configuration-changes">Monitoring Configuration Changes</h3>
<p>Now that the API is set up correctly, in your controllers you can directly request the configuration using <code>IOptionsMonitor<T></code>. You can also unpack the configuration instance itself by using the <code>IOptionsMonitor<T>.CurrentValue</code> property. This value will automatically get updated by <code>IOptionsMonitor<T></code> when the configuration is changed, so you only need to retrieve it once in the constructor.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Threading.Tasks</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Mvc</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.Extensions.Options</span>;
<span style="color:#369">
</span><span style="color:#369">[Route("api/[controller]</span><span style="color:#d20;background-color:#fff0f0">")]
</span><span style="color:#d20;background-color:#fff0f0"></span><span style="color:#369">[ApiController]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">MyController</span> : ControllerBase
{
<span style="color:#080;font-weight:bold">private</span> MyOptions MyOptions;
<span style="color:#080;font-weight:bold">public</span> MyController(IOptionsMonitor<MyOptions> MyOptionsMonitor)
{
<span style="color:#888">// Stores the actual configuration in the controller
</span><span style="color:#888"></span> <span style="color:#888">// to reference later. Changes to the config will be
</span><span style="color:#888"></span> <span style="color:#888">// reflected in MyOptions.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">this</span>.MyOptions = MyOptionsMonitor.CurrentValue;
<span style="color:#888">// Registers a lambda function to be called every time
</span><span style="color:#888"></span> <span style="color:#888">// a configuration change is made
</span><span style="color:#888"></span> MyOptionsMonitor.OnChange(<span style="color:#080;font-weight:bold">async</span> (opts) =>
{
<span style="color:#888">// Write some code!
</span><span style="color:#888"></span> });
}
}
</code></pre></div><p>One small gotcha worth noting is that the OnChange function isn’t debounced. So if you’re using an IDE that automatically saves as you type, it’ll trigger the <code>OnChange</code> function rapidly as you edit <code>appsettings.json</code>.</p>
<h3 id="conclusion">Conclusion</h3>
<p>And that’s it! It doesn’t take much to get your API to run custom code on config file changes.</p>
<p>Have any questions? Feel free to leave a comment!</p>
Building REST APIs with .NET 5, ASP.NET Core, and PostgreSQLhttps://www.endpointdev.com/blog/2021/07/dotnet-5-web-api/2021-07-09T00:00:00+00:00Kevin Campusano
<p><img src="/blog/2021/07/dotnet-5-web-api/market-cropped.jpg" alt="A market at night">
<a href="https://unsplash.com/photos/cpbWNtkKoiU">Photo</a> by <a href="https://unsplash.com/@sam_beasley">Sam Beasley</a></p>
<p>This is old news by now, but I’m still amazed by the fact that nowadays <a href="https://dotnet.microsoft.com/platform/open-source">.NET is open source and can run on Linux</a>. I truly believe that this new direction can help the technology realize its true potential, since it’s no longer shackled to Windows-based environments. I’ve personally been outside the .NET game for a good while, but with <a href="https://docs.microsoft.com/en-us/dotnet/core/dotnet-five">the milestone release that is .NET 5</a>, I think now is a great time to dive back in.</p>
<p>So I thought of taking some time to do just that, really dive in, see what’s new, and get a sense of the general developer experience that the current incarnation of .NET offers. So in this blog post, I’m going to chronicle my experience developing a simple but complete <a href="https://www.redhat.com/en/topics/api/what-is-a-rest-api">REST API</a> application. Along the way, I’ll touch on the most common problems that one runs into when developing such applications and how they are solved in the .NET world. So think of this piece as a sort of tutorial or overview of the most common framework features when it comes to developing REST APIs.</p>
<blockquote>
<p>There’s a <a href="#table-of-contents">table of contents</a> at the bottom.</p>
</blockquote>
<p>First, let’s get familiar with what we’re building.</p>
<h3 id="what-were-building">What we’re building</h3>
<h4 id="the-demo-application">The demo application</h4>
<blockquote>
<p>You can find the finished product on my <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">GitHub</a>.</p>
</blockquote>
<p>The application that we’ll be building throughout this article will address a request from a hypothetical car junker business. Our client wants to automate the process of calculating how much money to offer their customers for their vehicles, given certain information about them. And they want an app to do that. We are building the back-end component that will support that app. It is a REST API that allows users to provide vehicle information (year, make, model, condition, etc.) and will produce a quote of how much money our hypothetical client would be willing to pay for it.</p>
<p>Here’s a short list of features that we need to implement in order to fulfill that requirement:</p>
<ol>
<li>Given a vehicle model and condition, calculate a price.</li>
<li>Store and manage rules that are used to calculate vehicle prices.</li>
<li>Store and manage pricing overrides on a vehicle model basis. Price overrides are used regardless of the current rules.</li>
<li>CRUD vehicle models so that overrides can be specified for them.</li>
</ol>
<h4 id="the-data-model">The data model</h4>
<p>Here’s what our data model looks like:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/data-model.png" alt="Data Model"></p>
<p>The main table in our model is the <code>quotes</code> table. It stores all the requests for quotes received from our client’s customers. It captures all the relevant vehicle information in terms of model and condition. It also captures the offered quote; that is, the money value that our system calculates for their vehicle.</p>
<p>The <code>quotes</code> table includes all the fields that identify a vehicle: year, make, model, body style and size. It also includes a <code>model_style_year_id</code> field which is an optional foreign key to another table. This FK points to the <code>model_style_years</code> table which contains specific vehicle models that our system can store explicitly.</p>
<p>The idea of this is that, when a customer submits a request for a quote, if we have their vehicle registered in our database, then we can populate this foreign key and link the quote with the specific vehicle that it’s quoting. If we don’t have their vehicle registered, then we leave that field unpopulated. Either way, we can offer a quote. The only difference is the level or certainty of the quote.</p>
<p>The records in the <code>model_style_years</code> table represent specific vehicles. That whole hierarchy works like this: A vehicle make (e.g. Honda, Toyota, etc. in the <code>makes</code> table) has many models (e.g. Civic, Corolla, etc. in the <code>models</code> table), each model has many styles (the <code>model_styles</code> table). Styles are combinations of body types (the <code>body_types</code> table) and sizes (e.g. Mid-size Sedan, Compact Coupe, etc. in the <code>sizes</code> table). And finally, each model style has many years in which they were being produced (via the <code>model_style_years</code> table).</p>
<p>This model allows us very fine-grained differentiation between vehicles. For example, we can have a “2008 Honda Civic Hatchback which is a Compact car” and also a “1990 Honda Civic Hatchback which is a Sub-compact”. That is, same model, different year, size or body type.</p>
<p>We also have a <code>quote_rules</code> table which stores the rules that are applied when it comes to calculating a vehicle quote. The rules are pairs of key-values with an associated monetary value. So for example, rules like “a vehicle that has alloy wheels is worth $10 more” can be expressed in the table with a record where <code>feature_type</code> is “has_alloy_wheels”, <code>feature_value</code> is “true” and <code>price_modifier</code> is “10”.</p>
<p>Finally, we have a <code>quote_overrides</code> table which specifies a flat, static price for specific vehicles (via the link to the <code>model_style_years</code> table). The idea here is that if some customer requests a quote for a vehicle for which we have an override, no price calculation rules are applied and they are offered what is specified in the override record.</p>
<h3 id="the-development-environment">The development environment</h3>
<h4 id="setting-up-the-postgresql-database-with-docker">Setting up the PostgreSQL database with Docker</h4>
<p>For this project, our database of choice is <a href="https://www.postgresql.org/">PostgreSQL</a>. Luckily for us, getting a PostgreSQL instance up and running is very easy thanks to <a href="https://www.docker.com/">Docker</a>.</p>
<blockquote>
<p>If you want to learn more about dockerizing a typical web application, take a look at <a href="/blog/2020/08/containerizing-magento-with-docker-compose-elasticsearch-mysql-and-magento/">this article</a> that explains the process in detail.</p>
</blockquote>
<p>Once you have <a href="https://docs.docker.com/get-docker/">Docker installed</a> in your machine, getting a PostgreSQL instance is as simple as running the following command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ docker run -d <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> --name vehicle-quote-postgres <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -p 5432:5432 <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> --network host <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -e <span style="color:#369">POSTGRES_DB</span>=vehicle_quote <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -e <span style="color:#369">POSTGRES_USER</span>=vehicle_quote <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -e <span style="color:#369">POSTGRES_PASSWORD</span>=password <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> postgres
</code></pre></div><p>Here we’re asking Docker to run a new <a href="https://docs.docker.com/get-started/#what-is-a-container">container</a> based on the latest <code>postgres</code> <a href="https://docs.docker.com/get-started/#what-is-a-container-image">image</a> from <a href="https://hub.docker.com/_/postgres">DockerHub</a>, name it <code>vehicle-quote-postgres</code>, specify the port to use the default PostgreSQL one, make it accessible to the local network (with the <code>--network host</code> option) and finally, specify a few environment variables that the <code>postgres</code> image uses when building our new instance to set up the default database name, user and password (with the three <code>-e</code> options).</p>
<p>After Docker is done working its magic, you should be able to access the database with something like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">$ docker exec -it vehicle-quote-postgres psql -U vehicle_quote
psql (13.2 (Debian 13.2-1.pgdg100+1))
Type "help" for help.
vehicle_quote=#
</code></pre></div><p>This command is connecting to our new <code>vehicle-quote-postgres</code> container and then, from within the container, using the <a href="https://www.postgresql.org/docs/current/app-psql.html">command line client psql</a> in order to connect to the database.</p>
<p>If you have <a href="https://www.compose.com/articles/postgresql-tips-installing-the-postgresql-client/">psql installed</a> on your own machine, you can use it directly to connect to the PostgreSQL instance running inside the container:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ psql -h localhost -U vehicle_quote
</code></pre></div><p>This is possible because we specified in our <code>docker run</code> command that the container would be accepting traffic over port 5432 (<code>-p 5432:5432</code>) and that it would be accesible within the same network as our actual machine (<code>--network host</code>).</p>
<h4 id="installing-the-net-5-sdk">Installing the .NET 5 SDK</h4>
<p>Ok, with that out of the way, let’s install .NET 5.</p>
<p>.NET 5 truly is multi-platform, so whatever environment you prefer to work with, they’ve got you covered. You can go to <a href="https://dotnet.microsoft.com/download/dotnet/5.0">the .NET 5 download page</a> and pick your desired flavor of the SDK.</p>
<p>On Ubuntu 20.10, which is what I’m running, installation is painless. It’s your typical process with <a href="https://en.wikipedia.org/wiki/APT_(software)">APT</a> and <a href="https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#2010-">this page from the official docs</a> has all the details.</p>
<p>First step is to add the Microsoft package repository:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ wget https://packages.microsoft.com/config/ubuntu/20.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
$ sudo dpkg -i packages-microsoft-prod.deb
</code></pre></div><p>Then, install .NET 5 with APT like one would any other software package:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ sudo apt-get update; <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> sudo apt-get install -y apt-transport-https && <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> sudo apt-get update && <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> sudo apt-get install -y dotnet-sdk-5.0
</code></pre></div><p>Run <code>dotnet --version</code> in your console and you should see something like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet --version
5.0.301
</code></pre></div><h3 id="setting-up-the-project">Setting up the project</h3>
<h4 id="creating-our-aspnet-core-rest-api-project">Creating our ASP.NET Core REST API project</h4>
<p>Ok now that we have our requirements, database and SDK, let’s start setting up our project. We do so with the following command:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet new webapi -o VehicleQuotes
</code></pre></div><p>This instructs the <code>dotnet</code> command line tool to create a new REST API web application project for us in a new <code>VehicleQuotes</code> directory.</p>
<p>As a result, <code>dotnet</code> will give you some messages, including this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">The template "ASP.NET Core Web API" was created successfully.
</code></pre></div><p>A new directory was created with our web application files. The newly created <code>VehicleQuotes</code> project looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-plain" data-lang="plain">.
├── appsettings.Development.json
├── appsettings.json
├── Controllers
│ └── WeatherForecastController.cs
├── obj
│ ├── project.assets.json
│ ├── project.nuget.cache
│ ├── VehicleQuotes.csproj.nuget.dgspec.json
│ ├── VehicleQuotes.csproj.nuget.g.props
│ └── VehicleQuotes.csproj.nuget.g.targets
├── Program.cs
├── Properties
│ └── launchSettings.json
├── Startup.cs
├── VehicleQuotes.csproj
└── WeatherForecast.cs
</code></pre></div><p>Important things to note here are the <code>appsettings.json</code> and <code>appsettings.Development.json</code> files which contain environment specific configuration values; the <code>Controllers</code> directory where we define our application controllers and action methods (i.e. our REST API endpoints); the <code>Program.cs</code> and <code>Startup.cs</code> files that contain our application’s entry point and bootstrapping logic; and finally <code>VehicleQuotes.csproj</code> which is the file that contains project-wide configuration that the framework cares about like references, compilation targets, and other options. Feel free to explore.</p>
<p>The <code>dotnet new</code> command has given us quite a bit. These files make up a fully working application that we can run and play around with. It even has a <a href="https://swagger.io/tools/swagger-ui/">Swagger UI</a>, as I’ll demonstrate shortly. It’s a great place to get started from.</p>
<blockquote>
<p>You can also get a pretty comprehensive <code>.gitignore</code> file by running the <code>dotnet new gitignore</code> command.</p>
</blockquote>
<p>From inside the <code>VehicleQuotes</code> directory, you can run the application with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet run
</code></pre></div><p>Which will start up a development server and give out the following output:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet run
Building...
info: Microsoft.Hosting.Lifetime[0]
Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
Content root path: /home/kevin/projects/endpoint/blog/VehicleQuotes
</code></pre></div><p>Open up a browser window and go to <code>https://localhost:5001/swagger</code> to find a Swagger UI listing our API’s endpoints:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/initial-swagger.png" alt="Initial Swagger UI"></p>
<p>As you can see we’ve got a <code>GET /WeatherForecast</code> endpoint in our app. This is included by default in the <code>webapi</code> project template that we specified in our call to <code>dotnet new</code>. You can see it defined in the <code>Controllers/WeatherForecastController.cs</code> file.</p>
<h4 id="installing-packages-well-need">Installing packages we’ll need</h4>
<p>Now let’s install all the tools and libraries we will need for our application. First, we install the <a href="https://www.nuget.org/packages/dotnet-aspnet-codegenerator/">ASP.NET Code Generator</a> tool which we’ll use later for scaffolding controllers:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet tool install --global dotnet-aspnet-codegenerator
</code></pre></div><p>We also need to install the <a href="https://www.nuget.org/packages/dotnet-ef/">Entity Framework command line tools</a> which help us with creating and applying database migrations:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet tool install --global dotnet-ef
</code></pre></div><p>Now, we need to install a few libraries that we’ll use in our project. First are all the packages that allow us to use <a href="https://docs.microsoft.com/en-us/ef/">Entity Framework Core</a>, provide scaffolding support and give us a detailed debugging page for database errors:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design
$ dotnet add package Microsoft.EntityFrameworkCore.Design
$ dotnet add package Microsoft.EntityFrameworkCore.SqlServer
$ dotnet add package Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore
</code></pre></div><p>We also need the <a href="https://www.npgsql.org/efcore/">EF Core driver for PostgreSQL</a> which will allow us to interact with our database:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet add package Npgsql.EntityFrameworkCore.PostgreSQL
</code></pre></div><p>Finally, we need <a href="https://github.com/efcore/EFCore.NamingConventions">another package</a> that will allow us to use the <a href="https://en.wikipedia.org/wiki/Snake_case">snake case</a> naming convention for our database tables, fields, etc. We need this because EF Core uses <a href="https://wiki.c2.com/?UpperCamelCase">capitalized camel case</a> by default, which is not very common in the PostgreSQL world, so this will allow us to play nice. This is the package:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet add package EFCore.NamingConventions
</code></pre></div><h4 id="connecting-to-the-database-and-performing-initial-app-configuration">Connecting to the database and performing initial app configuration</h4>
<p>In order to connect to, query, and modify a database using EF Core, we need to create a <a href="https://docs.microsoft.com/en-us/ef/core/dbcontext-configuration/"><code>DbContext</code></a>. This is a class that serves as the entry point into the database. Create a new directory called <code>Data</code> in the project root and add this new <code>VehicleQuotesContext.cs</code> file to it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">VehicleQuotesContext</span> : DbContext
{
<span style="color:#080;font-weight:bold">public</span> VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
: <span style="color:#080;font-weight:bold">base</span>(options)
{
}
}
}
</code></pre></div><p>As you can see this is just a simple class that inherits from EF Core’s <a href="https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext?view=efcore-5.0"><code>DbContext</code></a> class. That’s all we need for now. We will continue building on this class as we add new tables and configurations.</p>
<p>Now, we need to add this class into <a href="https://docs.microsoft.com/en-us/aspnet/core/?view=aspnetcore-5.0">ASP.NET Core’s</a> built-in <a href="https://martinfowler.com/articles/injection.html">IoC (inversion of control) container</a> so that it’s available to controllers and other classes via <a href="https://en.wikipedia.org/wiki/Dependency_injection">Dependency Injection</a>, and tell it how to find our database. Go to <code>Startup.cs</code> and add the following using statement near the top of the file:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
</code></pre></div><p>That will allow us to do the following change in the <code>ConfigureServices</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> public void ConfigureServices(IServiceCollection services)
{
// ...
<span style="color:#000;background-color:#dfd">+ services.AddDbContext<VehicleQuotesContext>(options =>
</span><span style="color:#000;background-color:#dfd">+ options
</span><span style="color:#000;background-color:#dfd">+ .UseNpgsql(Configuration.GetConnectionString("VehicleQuotesContext"))
</span><span style="color:#000;background-color:#dfd">+ );
</span><span style="color:#000;background-color:#dfd"></span> }
</code></pre></div><blockquote>
<p><code>UseNpgsql</code> is an <a href="https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/extension-methods">extension method</a> made available to us by the <code>Npgsql.EntityFrameworkCore.PostgreSQL</code> package that we installed in the previous step.</p>
</blockquote>
<p>The <code>services</code> variable contains all the objects (known as “services”) that are available in the app for Dependency Injection. So here, we’re adding our newly created <code>DbContext</code> to it, specifying that it will connect to a PostgreSQL database (via the <code>options.UseNpgsql</code> call), and that it will use a connection string named <code>VehicleQuotesContext</code> from the app’s default configuration file. So let’s add the connection string then. To do so, change the <code>appsettings.json</code> like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"> {
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
<span style="color:#000;background-color:#fdd">- "AllowedHosts": "*"
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ "AllowedHosts": "*",
</span><span style="color:#000;background-color:#dfd">+ "ConnectionStrings": {
</span><span style="color:#000;background-color:#dfd">+ "VehicleQuotesContext": "Host=localhost;Database=vehicle_quote;Username=vehicle_quote;Password=password"
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd"></span> }
</code></pre></div><p>This is your typical PostgreSQL connection string. The only gotcha is that it needs to be specified under the <code>ConnectionStrings</code> -> <code>VehicleQuotesContext</code> section so that our call to <code>Configuration.GetConnectionString</code> can find it.</p>
<p>Now let’s put the <code>EFCore.NamingConventions</code> package to good use and configure EF Core to use snake case when naming database objects. Add the following to the <code>ConfigureServices</code> method in <code>Startup.cs</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">public void ConfigureServices(IServiceCollection services)
{
// ...
services.AddDbContext<VehicleQuotesContext>(options =>
options
.UseNpgsql(Configuration.GetConnectionString("VehicleQuotesContext"))
<span style="color:#000;background-color:#dfd">+ .UseSnakeCaseNamingConvention()
</span><span style="color:#000;background-color:#dfd"></span> );
}
</code></pre></div><blockquote>
<p><code>UseSnakeCaseNamingConvention</code> is an extension method made available to us by the <code>EFCore.NamingConventions</code> package that we installed in the previous step.</p>
</blockquote>
<p>Now let’s make logging a little bit more verbose with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">public void ConfigureServices(IServiceCollection services)
{
// ...
services.AddDbContext<VehicleQuotesContext>(options =>
options
.UseNpgsql(Configuration.GetConnectionString("VehicleQuotesContext"))
.UseSnakeCaseNamingConvention()
<span style="color:#000;background-color:#dfd">+ .UseLoggerFactory(LoggerFactory.Create(builder => builder.AddConsole()))
</span><span style="color:#000;background-color:#dfd">+ .EnableSensitiveDataLogging()
</span><span style="color:#000;background-color:#dfd"></span> );
}
</code></pre></div><p>This will make sure full database queries appear in the log in the console, including parameter values. This could expose sensitive data so be careful when using <code>EnableSensitiveDataLogging</code> in production.</p>
<p>We can also add the following service configuration to have the app display detailed error pages when something related to the database or migrations goes wrong:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">public void ConfigureServices(IServiceCollection services)
{
// ...
<span style="color:#000;background-color:#dfd">+ services.AddDatabaseDeveloperPageExceptionFilter();
</span><span style="color:#000;background-color:#dfd"></span>}
</code></pre></div><blockquote>
<p><code>AddDatabaseDeveloperPageExceptionFilter</code> is an extension method made available to us by the <code>Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore</code> package that we installed in the previous step.</p>
</blockquote>
<p>Finally, one last configuration I like to do is have the Swagger UI show up at the root URL, so that instead of using <code>https://localhost:5001/swagger</code>, we’re able to just use <code>https://localhost:5001</code>. We do so by by updating the <code>Configure</code> method this time, in the same <code>Startup.cs</code> file that we’ve been working on:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
app.UseSwagger();
<span style="color:#000;background-color:#fdd">- app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "VehicleQuotes v1"));
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ app.UseSwaggerUI(c => {
</span><span style="color:#000;background-color:#dfd">+ c.SwaggerEndpoint("/swagger/v1/swagger.json", "VehicleQuotes v1");
</span><span style="color:#000;background-color:#dfd">+ c.RoutePrefix = "";
</span><span style="color:#000;background-color:#dfd">+ });
</span><span style="color:#000;background-color:#dfd"></span> }
</code></pre></div><p>The magic is done by the <code>c.RoutePrefix = "";</code> line which makes it so there’s no need to put any prefix in order to access the auto generated Swagger UI.</p>
<p>Try it out. Do <code>dotnet run</code> and navigate to <code>https://localhost:5001</code> and you should see the Swagger UI there.</p>
<h3 id="building-the-application">Building the application</h3>
<h4 id="creating-model-entities-migrations-and-updating-the-database">Creating model entities, migrations and updating the database</h4>
<p>Alright, with all that configuration out of the way, let’s implement some of our actual application logic now. Refer back to our data model. We’ll start by defining our three simplest tables: <code>makes</code>, <code>sizes</code> and <code>body_types</code>. With EF Core, we define tables via so-called <a href="https://en.wikipedia.org/wiki/Plain_old_CLR_object">POCO</a> entities, which are simple C# classes with some properties. The classes become tables and the properties become the tables’ fields. Instances of these classes represent records in the database.</p>
<p>So, create a new <code>Models</code> directory in our project’s root and add these three files:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/BodyType.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">BodyType</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Name { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/Make.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Make</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Name { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/Size.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Size</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Name { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Now, we add three corresponding <a href="https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbset-1?view=efcore-5.0"><code>DbSet</code></a>s to our <code>DbContext</code> in <code>Data/VehicleQuoteContext.cs</code>. Here’s the diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">using Microsoft.EntityFrameworkCore;
<span style="color:#000;background-color:#dfd">+using VehicleQuotes.Models;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes
{
public class VehicleQuotesContext : DbContext
{
public VehicleQuotesContext (DbContextOptions<VehicleQuotesContext> options)
: base(options)
{
}
<span style="color:#000;background-color:#dfd">+ public DbSet<Make> Makes { get; set; }
</span><span style="color:#000;background-color:#dfd">+ public DbSet<Size> Sizes { get; set; }
</span><span style="color:#000;background-color:#dfd">+ public DbSet<BodyType> BodyTypes { get; set; }
</span><span style="color:#000;background-color:#dfd"></span> }
}
</code></pre></div><p>This is how we tell EF Core to build tables in our database for our entities. You’ll see later how we use those <code>DbSet</code>s to access the data in those tables. For now, let’s create a <a href="https://docs.microsoft.com/en-us/ef/core/managing-schemas/migrations/?tabs=dotnet-core-cli">migration</a> script that we can later run to apply changes to our database. Run the following to have EF Core create it for us:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef migrations add AddLookupTables
</code></pre></div><p>Now take a look at the newly created <code>Migrations</code> directory. It contains a few new files, but the one we care about right now is <code>Migrations/{TIMESTAMP}_AddLookupTables.cs</code>. In its <code>Up</code> method, it’s got some code that will modify the database structure when run. The EF Core tooling has inspected our project, identified the new entities, and automatically generated a migration script for us that creates tables for them. Notice also how the tables and fields use the snake case naming convention, just as we specified with the call to <code>UseSnakeCaseNamingConvention</code> in <code>Startup.cs</code>.</p>
<p>Now, to actually run the migration script and apply the changes to the database, we do:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef database update
</code></pre></div><p>That command inspects our project to find any migrations that haven’t been run yet, and applies them. In this case, we only have one, so that’s what it runs. Look at the output in the console to see it working its magic step by step:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef database update
Build started...
Build succeeded.
warn: Microsoft.EntityFrameworkCore.Model.Validation[10400]
Sensitive data logging is enabled. Log entries and exception messages may include sensitive application data; this mode should only be enabled during development.
...
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
Executed DbCommand (4ms) [<span style="color:#369">Parameters</span>=[], <span style="color:#369">CommandType</span>=<span style="color:#d20;background-color:#fff0f0">'Text'</span>, <span style="color:#369">CommandTimeout</span>=<span style="color:#d20;background-color:#fff0f0">'30'</span>]
CREATE TABLE sizes (
id integer GENERATED BY DEFAULT AS IDENTITY,
name text NULL,
CONSTRAINT pk_sizes PRIMARY KEY (id)
);
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
Executed DbCommand (1ms) [<span style="color:#369">Parameters</span>=[], <span style="color:#369">CommandType</span>=<span style="color:#d20;background-color:#fff0f0">'Text'</span>, <span style="color:#369">CommandTimeout</span>=<span style="color:#d20;background-color:#fff0f0">'30'</span>]
INSERT INTO <span style="color:#d20;background-color:#fff0f0">"__EFMigrationsHistory"</span> (migration_id, product_version)
VALUES (<span style="color:#d20;background-color:#fff0f0">'20210625212939_AddLookupTables'</span>, <span style="color:#d20;background-color:#fff0f0">'5.0.7'</span>);
Done.
</code></pre></div><p>Notice how it warns us about potential exposure of sensitive data because of that <code>EnableSensitiveDataLogging</code> option we opted into in <code>Startup.cs</code>. Also, EF Core related logs are extra verbose showing all database operations because of another configuration option that we applied there: the <code>UseLoggerFactory(LoggerFactory.Create(builder => builder.AddConsole()))</code> one.</p>
<p>You can connect to the database with the <code>psql</code> command line client and see that the changes took effect:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ psql -h localhost -U vehicle_quote
...
<span style="color:#369">vehicle_quote</span>=<span style="color:#888"># \c vehicle_quote </span>
psql (12.7 (Ubuntu 12.7-0ubuntu0.20.10.1), server 13.2 (Debian 13.2-1.pgdg100+1))
You are now connected to database <span style="color:#d20;background-color:#fff0f0">"vehicle_quote"</span> as user <span style="color:#d20;background-color:#fff0f0">"vehicle_quote"</span>.
<span style="color:#369">vehicle_quote</span>=<span style="color:#888"># \dt</span>
List of relations
Schema | Name | Type | Owner
--------+-----------------------+-------+---------------
public | __EFMigrationsHistory | table | vehicle_quote
public | body_types | table | vehicle_quote
public | makes | table | vehicle_quote
public | sizes | table | vehicle_quote
(<span style="color:#00d;font-weight:bold">4</span> rows)
<span style="color:#369">vehicle_quote</span>=<span style="color:#888"># \d makes</span>
Table <span style="color:#d20;background-color:#fff0f0">"public.makes"</span>
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+----------------------------------
id | integer | | not null | generated by default as identity
name | text | | |
Indexes:
<span style="color:#d20;background-color:#fff0f0">"pk_makes"</span> PRIMARY KEY, btree (id)
</code></pre></div><p>There are our tables in all their normalized, snake-cased glory. The <code>__EFMigrationsHistory</code> table is used internally by EF Core to keep track of which migrations have been applied.</p>
<h4 id="creating-controllers-for-cruding-our-tables">Creating controllers for CRUDing our tables</h4>
<p>Now that we have that, let’s add a few endpoints to support basic CRUD of those tables. We can use the <code>dotnet-aspnet-codegenerator</code> scaffolding tool that we installed earlier. For the three tables that we have, we would do:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet aspnet-codegenerator controller <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -name MakesController <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -m Make <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -dc VehicleQuotesContext <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -async <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -api <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -outDir Controllers
$ dotnet aspnet-codegenerator controller <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -name BodyTypesController <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -m BodyType <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -dc VehicleQuotesContext <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -async <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -api <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -outDir Controllers
$ dotnet aspnet-codegenerator controller <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -name SizesController <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -m Size <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -dc VehicleQuotesContext <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -async <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -api <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -outDir Controllers
</code></pre></div><p>Those commands tell the scaffolding tool to create new controllers that:</p>
<ol>
<li>Are named as given by the <code>-name</code> option.</li>
<li>Use the model class specified in the <code>-m</code> option.</li>
<li>Use our <code>VehicleQuotesContext</code> to talk to the database. As per the <code>-dc</code> option.</li>
<li>Define the methods using <code>async</code>/<code>await</code> syntax. Given by the <code>-async</code> option.</li>
<li>Are API controllers. Specified by the <code>-api</code> option.</li>
<li>Are created in the <code>Controllers</code> directory. Via the <code>-outDir</code> option.</li>
</ol>
<p>Explore the new files that got created in the <code>Controllers</code> directory: <code>MakesController.cs</code>, <code>BodyTypesController.cd</code> and <code>SizesController.cs</code>. The controllers have been generated with the necessary <a href="https://docs.microsoft.com/en-us/aspnet/mvc/overview/older-versions-1/controllers-and-routing/aspnet-mvc-controllers-overview-cs#understanding-controller-actions">Action Methods</a> to fetch, create, update and delete their corresponding entities. Try <code>dotnet run</code> and navigate to <code>https://localhost:5001</code> to see the new endpoints in the Swagger UI:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/swagger-lookup-tables.png" alt="Swagger UI with lookup tables"></p>
<p>Try it out! You can interact with each of the endpoints from the Swagger UI and it all works as you’d expect.</p>
<h4 id="adding-unique-constraints-via-indexes">Adding unique constraints via indexes</h4>
<p>Ok, our app is coming along well. Right now though, there’s an issue with the tables that we’ve created. It’s possible to create vehicle makes with the same name. The same is true for body types and sizes. This doesn’t make much sense for these tables. So let’s fix that by adding a uniqueness constraint. We can do it by creating a unique database index using EF Core’s <code>Index</code> attribute. For example, we can modify our <code>Models/Make.cs</code> like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#dfd">+using Microsoft.EntityFrameworkCore;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Models
{
<span style="color:#000;background-color:#dfd">+ [Index(nameof(Name), IsUnique = true)]
</span><span style="color:#000;background-color:#dfd"></span> public class Make
{
public int ID { get; set; }
public string Name { get; set; }
}
}
</code></pre></div><p>In fact, do the same for our other entities in <code>Models/BodyType.cs</code> and <code>Models/Size.cs</code>. Don’t forget the <code>using Microsoft.EntityFrameworkCore</code> statement.</p>
<p>With that, we can create a new migration:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef migrations add AddUniqueIndexesToLookupTables
</code></pre></div><p>That will result in a new migration script in <code>Migrations/{TIMESTAMP}_AddUniqueIndexesToLookupTables.cs</code>. Its <code>Up</code> method looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateIndex(
name: <span style="color:#d20;background-color:#fff0f0">"ix_sizes_name"</span>,
table: <span style="color:#d20;background-color:#fff0f0">"sizes"</span>,
column: <span style="color:#d20;background-color:#fff0f0">"name"</span>,
unique: <span style="color:#080;font-weight:bold">true</span>);
migrationBuilder.CreateIndex(
name: <span style="color:#d20;background-color:#fff0f0">"ix_makes_name"</span>,
table: <span style="color:#d20;background-color:#fff0f0">"makes"</span>,
column: <span style="color:#d20;background-color:#fff0f0">"name"</span>,
unique: <span style="color:#080;font-weight:bold">true</span>);
migrationBuilder.CreateIndex(
name: <span style="color:#d20;background-color:#fff0f0">"ix_body_types_name"</span>,
table: <span style="color:#d20;background-color:#fff0f0">"body_types"</span>,
column: <span style="color:#d20;background-color:#fff0f0">"name"</span>,
unique: <span style="color:#080;font-weight:bold">true</span>);
}
</code></pre></div><p>As you can see, new unique indexes are being created on the tables and fields that we specified. Like before, apply the changes to the database structure with:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef database update
</code></pre></div><p>Now if you try to create, for example, a vehicle make with a repeated name, you’ll get an error. Try doing so by <code>POST</code>ing to <code>/api/Makes</code> via the Swagger UI:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/unique-constraint-violation.png" alt="Unique constraint violation"></p>
<h4 id="responding-with-specific-http-error-codes-409-conflict">Responding with specific HTTP error codes (409 Conflict)</h4>
<p>The fact that we can now enforce unique constraints is all well and good. But the error scenario is not very user friendly. Instead of returning a “500 Internal Server Error” status code with a wall of text, we should be responding with something more sensible. Maybe a “409 Conflict” would be more appropriate for this kind of error. We can easily update our controllers to handle that scenario. What we need to do is update the methods that handle the <code>POST</code> and <code>PUT</code> endpoints so that they catch the <code>Microsoft.EntityFrameworkCore.DbUpdateException</code> exception and return the proper response. Here’s how we would do it for the <code>MakesController</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">// ...
namespace VehicleQuotes.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class MakesController : ControllerBase
{
// ...
[HttpPut("{id}")]
public async Task<IActionResult> PutMake(int id, Make make)
{
// ...
try
{
await _context.SaveChangesAsync();
}
// ...
<span style="color:#000;background-color:#dfd">+ catch (Microsoft.EntityFrameworkCore.DbUpdateException)
</span><span style="color:#000;background-color:#dfd">+ {
</span><span style="color:#000;background-color:#dfd">+ return Conflict();
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd"></span>
return NoContent();
}
[HttpPost]
public async Task<ActionResult<Make>> PostMake(Make make)
{
_context.Makes.Add(make);
<span style="color:#000;background-color:#fdd">- await _context.SaveChangesAsync();
</span><span style="color:#000;background-color:#fdd"></span>
<span style="color:#000;background-color:#dfd">+ try
</span><span style="color:#000;background-color:#dfd">+ {
</span><span style="color:#000;background-color:#dfd">+ await _context.SaveChangesAsync();
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd">+ catch (Microsoft.EntityFrameworkCore.DbUpdateException)
</span><span style="color:#000;background-color:#dfd">+ {
</span><span style="color:#000;background-color:#dfd">+ return Conflict();
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd"></span>
return CreatedAtAction("GetMake", new { id = make.ID }, make);
}
// ...
}
}
</code></pre></div><p>Go ahead and do the same for the other two controllers, and try again to POST a repeated make name via the Swagger UI. You should see this now instead:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/http-409-conflict.png" alt="HTTP 409 Conflict"></p>
<p>Much better now, don’t you think?</p>
<h4 id="adding-a-more-complex-entity-to-the-model">Adding a more complex entity to the model</h4>
<p>Now let’s work on an entity that’s a little bit more complex: the one we will use to represent vehicle models.</p>
<p>For this entity, we don’t want our API to be as low level as the one for the other three, where it was basically a thin wrapper over database tables. We want it to be a little bit more abstract and not expose the entire database structure verbatim.</p>
<p>Refer back to the data model. We’ll add <code>models</code>, <code>model_styles</code> and <code>model_style_years</code>. Let’s start by adding the following classes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/Model.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Collections.Generic</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Model</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Name { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> MakeID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Make Make { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ICollection<ModelStyle> ModelStyles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/ModelStyle.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Collections.Generic</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ModelStyle</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ModelID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> BodyTypeID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> SizeID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Model Model { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> BodyType BodyType { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Size Size { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ICollection<ModelStyleYear> ModelStyleYears { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/ModelStyleYear.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ModelStyleYear</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Year { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ModelStyleID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ModelStyle ModelStyle { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Notice how some of these entities now include properties whose types are other entities. Some of them are collections even. These are called Navigation Properties and are how we tell EF Core that our entities are related to one another. These will result in foreign keys being created in the database.</p>
<p>Take the <code>Model</code> entity for example. It has a property <code>Make</code> of type <code>Make</code>. It also has a <code>MakeID</code> property of type <code>int</code>. EF Core sees this and figures out that there’s a relation between the <code>makes</code> and <code>models</code> tables. Specifically, that <code>models</code> have a <code>make</code>. A many-to-one relation where the <code>models</code> table stores a foreign key to the <code>makes</code> table.</p>
<p>Similarly, the <code>Model</code> entity has a <code>ModelStyles</code> property of type <code>ICollection<ModelStyleYear></code>. This tells EF Core that <code>models</code> have many <code>model_styles</code>. This one is a one-to-many relation from the perspective of the <code>models</code> table. The foreign key lives in the <code>model_styles</code> table and points back to <code>models</code>.</p>
<blockquote>
<p>The <a href="https://docs.microsoft.com/en-us/ef/core/modeling/relationships?tabs=fluent-api%2Cfluent-api-simple-key%2Csimple-key#single-navigation-property-1">official documentation</a> is a great resource to learn more details about how relationships work in EF Core.</p>
</blockquote>
<p>After that, same as before, we have to add the corresponding <code>DbSet</code>s to our <code>DbContext</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">// Data/VehicleQuotesContext.cs
// ...
namespace VehicleQuotes
{
public class VehicleQuotesContext : DbContext
{
// ...
<span style="color:#000;background-color:#dfd">+ public DbSet<Model> Models { get; set; }
</span><span style="color:#000;background-color:#dfd">+ public DbSet<ModelStyle> ModelStyles { get; set; }
</span><span style="color:#000;background-color:#dfd">+ public DbSet<ModelStyleYear> ModelStyleYears { get; set; }
</span><span style="color:#000;background-color:#dfd"></span> }
}
</code></pre></div><p>Don’t forget the migration script. First create it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef migrations add AddVehicleModelTables
</code></pre></div><p>And then apply it:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef database update
</code></pre></div><h4 id="adding-composite-unique-indexes">Adding composite unique indexes</h4>
<p>These vehicle model related tables also need some uniqueness enforcement. This time, however, the unique keys are composite. Meaning that they involve multiple fields. For vehicle models, for example, it makes no sense to have multiple records with the same make and name. But it does make sense to have multiple models with the same name, as long as they belong to different makes. We can solve for that with a composite index. Here’s how we create one of those in <code>Model</code> with EF Core:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">using System.Collections.Generic;
<span style="color:#000;background-color:#dfd">+using Microsoft.EntityFrameworkCore;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Models
{
<span style="color:#000;background-color:#dfd">+ [Index(nameof(Name), nameof(MakeID), IsUnique = true)]
</span><span style="color:#000;background-color:#dfd"></span> public class Model
{
// ...
}
}
</code></pre></div><p>Very similar to what we did with the <code>Make</code>, <code>BodyType</code>, and <code>Size</code> entities. The only difference is that this time we included multiple fields in the parameters for the <code>Index</code> attribute.</p>
<p>We should do the same for <code>ModelStyle</code> and <code>ModelStyleYear</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">using System.Collections.Generic;
<span style="color:#000;background-color:#dfd">+using Microsoft.EntityFrameworkCore;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Models
{
<span style="color:#000;background-color:#dfd">+ [Index(nameof(ModelID), nameof(BodyTypeID), nameof(SizeID), IsUnique = true)]
</span><span style="color:#000;background-color:#dfd"></span> public class ModelStyle
{
// ...
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#dfd">+using Microsoft.EntityFrameworkCore;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Models
{
<span style="color:#000;background-color:#dfd">+ [Index(nameof(Year), nameof(ModelStyleID), IsUnique = true)]
</span><span style="color:#000;background-color:#dfd"></span> public class ModelStyleYear
{
// ...
}
}
</code></pre></div><p>Don’t forget the migrations:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef migrations add AddUniqueIndexesForVehicleModelTables
$ dotnet ef database update
</code></pre></div><h4 id="adding-controllers-with-custom-routes">Adding controllers with custom routes</h4>
<p>Our data model dictates that vehicle models belong in a make. In other words, a vehicle model has no meaning by itself. It only has meaning within the context of a make. Ideally, we want our API routes to reflect this concept. In other words, instead of URLs for models to look like this: <code>/api/Models/{id}</code>; we’d rather them look like this: <code>/api/Makes/{makeId}/Models/{modelId}</code>. Let’s go ahead and scaffold a controller for this entity:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet aspnet-codegenerator controller <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -name ModelsController <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -m Model <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -dc VehicleQuotesContext <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -async <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -api <span style="color:#04d;background-color:#fff0f0">\
</span><span style="color:#04d;background-color:#fff0f0"></span> -outDir Controllers
</code></pre></div><p>Now let’s change the resulting <code>Controllers/ModelsController.cs</code> to use the URL structure that we want. To do so, we modify the <code>Route</code> attribute that’s applied to the <code>ModelsController</code> class to this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[Route("api/Makes/{makeId}/[controller]</span>/<span style="color:#d20;background-color:#fff0f0">")]
</span></code></pre></div><p>Do a <code>dotnet run</code> and take a peek at the Swagger UI on <code>https://localhost:5001</code> to see what the <code>Models</code> endpoint routes look like now:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/nested-routes.png" alt="Nested routes"></p>
<p>The vehicle model routes are now nested within makes, just like we wanted.</p>
<p>Of course, this is just eye candy for now. We need to actually use this new <code>makeId</code> parameter for the logic in the endpoints. For example, one would expect a <code>GET</code> to <code>/api/Makes/1/Models</code> to return all the vehicle models that belong to the make with <code>id</code> 1. But right now, all vehicle models are returned regardless. All other endpoints behave similarly, there’s no limit to the operations on the vehicle models. The given <code>makeId</code> is not taken into consideration at all.</p>
<p>Let’s update the <code>ModelsController</code>’s <code>GetModels</code> method (which is the one that handles the <code>GET /api/Makes/{makeId}/Models</code> endpoint) to behave like one would expect. It should look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[HttpGet]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<IEnumerable<Model>>> GetModels([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId)
{
<span style="color:#888;font-weight:bold">var</span> make = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Makes.FindAsync(makeId);
<span style="color:#080;font-weight:bold">if</span> (make == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Models.Where(m => m.MakeID == makeId).ToListAsync();
}
</code></pre></div><p>See how we’ve included a new parameter to the method: <code>[FromRoute] int makeId</code>. This <code>[FromRoute]</code> <a href="https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/attributes/">attribute</a> is how we tell ASP.NET Core that this endpoint will use that <code>makeId</code> parameter coming from the URL route. Then, we use our <code>DbContext</code> to try to find the make that corresponds to the given identifier. This is done in <code>_context.Makes.FindAsync(makeId)</code>. Then, if we can’t find the given make, we return a <code>404 Not Found</code> HTTP status code with the <code>return NotFound();</code> line. Finally, we query the <code>models</code> table for all the records whose <code>make_id</code> matches the given parameter. That’s done in the last line of the method.</p>
<blockquote>
<p>We have access to the <code>DbContext</code> because it has been injected as a dependency into the controller via its constructor by the framework.</p>
</blockquote>
<blockquote>
<p><a href="https://docs.microsoft.com/en-us/ef/core/querying/">The official documentation</a> is a great resource to learn about all the possibilities when querying data with EF Core.</p>
</blockquote>
<p>Let’s update the <code>GetModel</code> method, which handles the <code>GET /api/Makes/{makeId}/Models/{id}</code> endpoint, similarly.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">[HttpGet("{id}")]
<span style="color:#000;background-color:#fdd">-public async Task<ActionResult<Model>> GetModel(int id)
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+public async Task<ActionResult<Model>> GetModel([FromRoute] int makeId, int id)
</span><span style="color:#000;background-color:#dfd"></span>{
<span style="color:#000;background-color:#fdd">- var model = await _context.Models.FindAsync(id);
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ var model = await _context.Models.FirstOrDefaultAsync(m =>
</span><span style="color:#000;background-color:#dfd">+ m.MakeID == makeId && m.ID == id
</span><span style="color:#000;background-color:#dfd">+ );
</span><span style="color:#000;background-color:#dfd"></span>
if (model == null)
{
return NotFound();
}
return model;
}
</code></pre></div><p>We’ve once again included the <code>makeId</code> as a parameter to the method and modified the EF Core query to use both the make ID and the vehicle model ID when looking for the record.</p>
<p>And that’s the gist of it. Other methods would need to be updated similarly. The next section will include these methods in their final form, so I won’t go through each one of them here.</p>
<h4 id="using-resource-models-as-dtos-for-controllers">Using resource models as DTOs for controllers</h4>
<p>Now, I did say at the beginning that we wanted the vehicle model endpoint to be a bit more abstract. Right now it’s operating directly over the EF Core entities and our table. As a result, creating new vehicle models via the <code>POST /api/Makes/{makeId}/Models</code> endpoint is a pain. Take a look at the Swagger UI request schema for that endpoint:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/raw-model-request-schema.png" alt="Raw model request schema"></p>
<p>This is way too much. Let’s make it a little bit more user friendly by making it more abstract.</p>
<p>To do that, we will introduce what I like to call a Resource Model (or <a href="https://martinfowler.com/eaaCatalog/dataTransferObject.html">DTO, Data Transfer Object</a>, or View Model). This is a class whose only purpose is to streamline the API contract of the endpoint by defining a set of fields that clients will use to make requests and interpret responses. Something that’s simpler than our actual database structure, but still captures all the information that’s important for our application. We will update the <code>ModelsController</code> so that it’s able to receive objects of this new class as requests, operate on them, translate them to our EF Core entities and actual database records, and return them as a response. The hope is that, by hiding the details of our database structure, we make it easier for clients to interact with our API.</p>
<p>So let’s create a new <code>ResourceModels</code> directory in our project’s root and add these two classes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// ResourceModels/ModelSpecification.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ModelSpecification</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Name { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ModelSpecificationStyle[] Styles { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// ResourceModels/ModelSpecificationStyle.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ModelSpecificationStyle</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> BodyType { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Size { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span>[] Years { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>Thanks to these two, instead of that mess from above, clients <code>POST</code>ing to <code>/api/Makes/{makeId}/Models</code> will be able to use a request body like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"name"</span>: <span style="color:#d20;background-color:#fff0f0">"string"</span>,
<span style="color:#b06;font-weight:bold">"styles"</span>: [
{
<span style="color:#b06;font-weight:bold">"bodyType"</span>: <span style="color:#d20;background-color:#fff0f0">"string"</span>,
<span style="color:#b06;font-weight:bold">"size"</span>: <span style="color:#d20;background-color:#fff0f0">"string"</span>,
<span style="color:#b06;font-weight:bold">"years"</span>: [
<span style="color:#d20;background-color:#fff0f0">"string"</span>
]
}
]
}
</code></pre></div><p>Which is much simpler. We have the vehicle model name and an array of styles. Each style has a body type and a size, which we can specify by their names because those are unique keys. We don’t need their integer IDs (i.e. primary keys) in order to find to them. Then, each style has an array of strings that contain the years in which those styles are available for that model. The make is part of the URL already, so we don’t need to also specify it in the request payload.</p>
<p>Let’s update our <code>ModelsController</code> to use these Resource Models instead of the <code>Model</code> EF Core entity. Be sure to include the namespace where the Resource Models are defined by adding the following using statement: <code>using VehicleQuotes.ResourceModels;</code>. Now, let’s update the <code>GetModels</code> method (which handles the <code>GET /api/Makes/{makeId}/Models</code> endpoint) so that it looks like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[HttpGet]</span>
<span style="color:#888">// Return a collection of `ModelSpecification`s and expect a `makeId` from the URL.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<IEnumerable<ModelSpecification>>> GetModels([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId)
{
<span style="color:#888">// Look for the make identified by `makeId`.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> make = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Makes.FindAsync(makeId);
<span style="color:#888">// If we can't find the make, then we return a 404.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (make == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#888">// Build a query to fetch the relevant records from the `models` table and
</span><span style="color:#888"></span> <span style="color:#888">// build `ModelSpecification` with the data.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> modelsToReturn = <span style="color:#00d;font-weight:bold">_</span>context.Models
.Where(m => m.MakeID == makeId)
.Select(m => <span style="color:#080;font-weight:bold">new</span> ModelSpecification {
ID = m.ID,
Name = m.Name,
Styles = m.ModelStyles.Select(ms => <span style="color:#080;font-weight:bold">new</span> ModelSpecificationStyle {
BodyType = ms.BodyType.Name,
Size = ms.Size.Name,
Years = ms.ModelStyleYears.Select(msy => msy.Year).ToArray()
}).ToArray()
});
<span style="color:#888">// Execute the query and respond with the results.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> modelsToReturn.ToListAsync();
}
</code></pre></div><p>The first thing that we changed was the return type. Instead of <code>Task<ActionResult<IEnumerable<Model>>></code>, the method now returns <code>Task<ActionResult<IEnumerable<ModelSpecification>>></code>. We’re going to use our new Resource Models as these endpoints’ contract, so we need to make sure we are returning those. Next, we considerably changed the LINQ expression that searches the database for the vehicle model records we want. The filtering logic (given by the <code>Where</code>) is the same. That is, we’re still searching for vehicle models within the given make ID. What we changed was the projection logic in the <code>Select</code>. Our Action Method now returns a collection of <code>ModelSpecification</code> objects, so we updated the <code>Select</code> to produce such objects, based on the records from the <code>models</code> table that match our search criteria. We build <code>ModelSpecification</code>s using the data coming from <code>models</code> records and their related <code>model_styles</code> and <code>model_style_years</code>. Finally, we asynchronously execute the query to fetch the data from the database and return it.</p>
<p>Next, let’s move on to the <code>GetModel</code> method, which handles the <code>GET /api/Makes/{makeId}/Models/{id}</code> endpoint. This is what it should look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[HttpGet("{id}")]</span>
<span style="color:#888">// Return a `ModelSpecification`s and expect `makeId` and `id` from the URL.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<ModelSpecification>> GetModel([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId, [FromRoute] <span style="color:#888;font-weight:bold">int</span> id)
{
<span style="color:#888">// Look for the model specified by the given identifiers and also load
</span><span style="color:#888"></span> <span style="color:#888">// all related data that we care about for this method.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> model = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Models
.Include(m => m.ModelStyles).ThenInclude(ms => ms.BodyType)
.Include(m => m.ModelStyles).ThenInclude(ms => ms.Size)
.Include(m => m.ModelStyles).ThenInclude(ms => ms.ModelStyleYears)
.FirstOrDefaultAsync(m => m.MakeID == makeId && m.ID == id);
<span style="color:#888">// If we couldn't find it, respond with a 404.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (model == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#888">// Use the fetched data to construct a `ModelSpecification` to use in the response.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> ModelSpecification {
ID = model.ID,
Name = model.Name,
Styles = model.ModelStyles.Select(ms => <span style="color:#080;font-weight:bold">new</span> ModelSpecificationStyle {
BodyType = ms.BodyType.Name,
Size = ms.Size.Name,
Years = ms.ModelStyleYears.Select(msy => msy.Year).ToArray()
}).ToArray()
};
}
</code></pre></div><p>Same as before, we changed the return type of the method to be <code>ModelSpecification</code>. Then, we modified the query so that it loads all the related data for the <code>Model</code> entity via its navigation properties. That’s what the <code>Include</code> and <code>ThenInclude</code> calls do. We need this data loaded because we use it in the method’s return statement to build the <code>ModelSpecification</code> that will be included in the response. The logic to build it is very similar to that of the previous method.</p>
<blockquote>
<p>You can learn more about the various available approaches for loading data with EF Core in <a href="https://docs.microsoft.com/en-us/ef/core/querying/related-data/">the official documentation</a>.</p>
</blockquote>
<p>Next is the <code>PUT /api/Makes/{makeId}/Models/{id}</code> endpoint, handled by the <code>PutModel</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[HttpPut("{id}")]</span>
<span style="color:#888">// Expect `makeId` and `id` from the URL and a `ModelSpecification` from the request payload.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<IActionResult> PutModel([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId, <span style="color:#888;font-weight:bold">int</span> id, ModelSpecification model)
{
<span style="color:#888">// If the id in the URL and the request payload are different, return a 400.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (id != model.ID)
{
<span style="color:#080;font-weight:bold">return</span> BadRequest();
}
<span style="color:#888">// Obtain the `models` record that we want to update. Include any related
</span><span style="color:#888"></span> <span style="color:#888">// data that we want to update as well.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> modelToUpdate = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Models
.Include(m => m.ModelStyles)
.FirstOrDefaultAsync(m => m.MakeID == makeId && m.ID == id);
<span style="color:#888">// If we can't find the record, then return a 404.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (modelToUpdate == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#888">// Update the record with what came in the request payload.
</span><span style="color:#888"></span> modelToUpdate.Name = model.Name;
<span style="color:#888">// Build EF Core entities based on the incoming Resource Model object.
</span><span style="color:#888"></span> modelToUpdate.ModelStyles = model.Styles.Select(style => <span style="color:#080;font-weight:bold">new</span> ModelStyle {
BodyType = <span style="color:#00d;font-weight:bold">_</span>context.BodyTypes.Single(bodyType => bodyType.Name == style.BodyType),
Size = <span style="color:#00d;font-weight:bold">_</span>context.Sizes.Single(size => size.Name == style.Size),
ModelStyleYears = style.Years.Select(year => <span style="color:#080;font-weight:bold">new</span> ModelStyleYear {
Year = year
}).ToList()
}).ToList();
<span style="color:#080;font-weight:bold">try</span>
{
<span style="color:#888">// Try saving the changes. This will run the UPDATE statement in the database.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.SaveChangesAsync();
}
<span style="color:#080;font-weight:bold">catch</span> (Microsoft.EntityFrameworkCore.DbUpdateException)
{
<span style="color:#888">// If there's an error updating, respond accordingly.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> Conflict();
}
<span style="color:#888">// Finally return a 204 if everything went well.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> NoContent();
}
</code></pre></div><p>The purpose of this endpoint is to update existing resources. So, it receives a representation of said resource as a parameter that comes from the request body. Before, it expected an instance of the <code>Model</code> entity, but now, we’ve changed it to receive a <code>ModelSpecification</code>. The rest of the method is your usual structure of first obtaining the record to update by the given IDs, then changing its values according to what came in as a parameter, and finally, saving the changes.</p>
<p>You probably get the idea by now: since the API is using the Resource Model, we need to change input and output values for the methods and run some logic to translate between Resource Model objects and Data Model objects that EF Core can understand so that it can perform its database operations.</p>
<p>That said, here’s what the <code>PostModel</code> Action Method, handler of the <code>POST /api/Makes/{makeId}/Models</code> endpoint, should look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[HttpPost]</span>
<span style="color:#888">// Return a `ModelSpecification`s and expect `makeId` from the URL and a `ModelSpecification` from the request payload.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<ModelSpecification>> PostModel([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId, ModelSpecification model)
{
<span style="color:#888">// First, try to find the make specified by the incoming `makeId`.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> make = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Makes.FindAsync(makeId);
<span style="color:#888">// Respond with 404 if not found.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (make == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#888">// Build out a new `Model` entity, complete with all related data, based on
</span><span style="color:#888"></span> <span style="color:#888">// the `ModelSpecification` parameter.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> modelToCreate = <span style="color:#080;font-weight:bold">new</span> Model {
Make = make,
Name = model.Name,
ModelStyles = model.Styles.Select(style => <span style="color:#080;font-weight:bold">new</span> ModelStyle {
<span style="color:#888">// Notice how we search both body type and size by their name field.
</span><span style="color:#888"></span> <span style="color:#888">// We can do that because their names are unique.
</span><span style="color:#888"></span> BodyType = <span style="color:#00d;font-weight:bold">_</span>context.BodyTypes.Single(bodyType => bodyType.Name == style.BodyType),
Size = <span style="color:#00d;font-weight:bold">_</span>context.Sizes.Single(size => size.Name == style.Size),
ModelStyleYears = style.Years.Select(year => <span style="color:#080;font-weight:bold">new</span> ModelStyleYear {
Year = year
}).ToArray()
}).ToArray()
};
<span style="color:#888">// Add it to the DbContext.
</span><span style="color:#888"></span> <span style="color:#00d;font-weight:bold">_</span>context.Add(modelToCreate);
<span style="color:#080;font-weight:bold">try</span>
{
<span style="color:#888">// Try running the INSERTs.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.SaveChangesAsync();
}
<span style="color:#080;font-weight:bold">catch</span> (Microsoft.EntityFrameworkCore.DbUpdateException)
{
<span style="color:#888">// Return accordingly if an error happens.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> Conflict();
}
<span style="color:#888">// Get back the autogenerated ID of the record we just INSERTed.
</span><span style="color:#888"></span> model.ID = modelToCreate.ID;
<span style="color:#888">// Finally, return a 201 including a location header containing the newly
</span><span style="color:#888"></span> <span style="color:#888">// created resource's URL and the resource itself in the response payload.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> CreatedAtAction(
nameof(GetModel),
<span style="color:#080;font-weight:bold">new</span> { makeId = makeId, id = model.ID },
model
);
}
</code></pre></div><p>All that should be pretty self explanatory by now. Moving on to the <code>DeleteModel</code> method which handles the <code>DELETE /api/Makes/{makeId}/Models/{id}</code> endpoint:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#369">[HttpDelete("{id}")]</span>
<span style="color:#888">// Expect `makeId` and `id` from the URL.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<IActionResult> DeleteModel([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId, <span style="color:#888;font-weight:bold">int</span> id)
{
<span style="color:#888">// Try to find the record identified by the ids from the URL.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> model = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Models.FirstOrDefaultAsync(m => m.MakeID == makeId && m.ID == id);
<span style="color:#888">// Respond with a 404 if we can't find it.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (model == <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">return</span> NotFound();
}
<span style="color:#888">// Mark the entity for removal and run the DELETE.
</span><span style="color:#888"></span> <span style="color:#00d;font-weight:bold">_</span>context.Models.Remove(model);
<span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.SaveChangesAsync();
<span style="color:#888">// Respond with a 204.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> NoContent();
}
</code></pre></div><p>And that’s all for that controller. Hopefully that demonstrated what it looks like to have endpoints that operate using objects other than the EF Core entities. Fire up the app with <code>dotnet run</code> and explore the Swagger UI and you’ll see the changes that we’ve made reflected in there. Try it out. Try CRUDing some vehicle models. And don’t forget to take a look at our POST endpoint specification which looks much more manageable now:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/post-model-endpoint.png" alt="POST Models endpoint"></p>
<p>Which means that you can send in something like this, for example:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"name"</span>: <span style="color:#d20;background-color:#fff0f0">"Corolla"</span>,
<span style="color:#b06;font-weight:bold">"styles"</span>: [
{
<span style="color:#b06;font-weight:bold">"bodyType"</span>: <span style="color:#d20;background-color:#fff0f0">"Sedan"</span>,
<span style="color:#b06;font-weight:bold">"size"</span>: <span style="color:#d20;background-color:#fff0f0">"Compact"</span>,
<span style="color:#b06;font-weight:bold">"years"</span>: [ <span style="color:#d20;background-color:#fff0f0">"2000"</span>, <span style="color:#d20;background-color:#fff0f0">"2001"</span> ]
}
]
}
</code></pre></div><blockquote>
<p>This will work assuming you’ve created at least one make to add the vehicle model to, as well as a body type whose name is <code>Sedan</code> and a size whose name is <code>Compact</code>.</p>
</blockquote>
<blockquote>
<p>There’s also a <code>ModelExists</code> method in that controller which we don’t need anymore. You can delete it.</p>
</blockquote>
<h4 id="validation-using-built-in-data-annotations">Validation using built-in Data Annotations</h4>
<p>Depending on how “creative” you were in the previous section when trying to CRUD models, you may have run into an issue or two regarding the data that’s allowed into our database. We solve that by implementing input validation. In ASP.NET Core, the easiest way to implement validation is via Data Annotation attributes on the entities or other objects that controllers receive as request payloads. So let’s see about adding some validation to our app. Since our <code>ModelsController</code> uses the <code>ModelSpecification</code> and <code>ModelSpecificationStyle</code> Resource Models to talk to clients, let’s start there. Here’s the diff:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#dfd">+using System.ComponentModel.DataAnnotations;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.ResourceModels
{
public class ModelSpecification
{
public int ID { get; set; }
<span style="color:#000;background-color:#dfd">+ [Required]
</span><span style="color:#000;background-color:#dfd"></span> public string Name { get; set; }
<span style="color:#000;background-color:#dfd">+ [Required]
</span><span style="color:#000;background-color:#dfd"></span> public ModelSpecificationStyle[] Styles { get; set; }
}
}
</code></pre></div><div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><span style="color:#000;background-color:#dfd">+using System.ComponentModel.DataAnnotations;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.ResourceModels
{
public class ModelSpecificationStyle
{
<span style="color:#000;background-color:#dfd">+ [Required]
</span><span style="color:#000;background-color:#dfd"></span> public string BodyType { get; set; }
<span style="color:#000;background-color:#dfd">+ [Required]
</span><span style="color:#000;background-color:#dfd"></span> public string Size { get; set; }
<span style="color:#000;background-color:#dfd">+ [Required]
</span><span style="color:#000;background-color:#dfd">+ [MinLength(1)]
</span><span style="color:#000;background-color:#dfd"></span> public string[] Years { get; set; }
}
}
</code></pre></div><p>And just like that, we get a good amount of functionality. We use the <code>Required</code> and <code>MinLength</code> attributes from the <code>System.ComponentModel.DataAnnotations</code> namespace to specify that some fields are required, and that our <code>Years</code> array needs to contain at least one element. When the app receives a request to the PUT or POST endpoints — which are the ones that expect a <code>ModelSpecification</code> as the payload — validation kicks in. If it fails, the action method is never executed and a 400 status code is returned as a response. Try POSTing to <code>/api/Makes/{makeId}/Models</code> with a payload that violates some of these rules to see for yourself. I tried for example sending this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"name"</span>: <span style="color:#080;font-weight:bold">null</span>,
<span style="color:#b06;font-weight:bold">"styles"</span>: [
{
<span style="color:#b06;font-weight:bold">"bodyType"</span>: <span style="color:#d20;background-color:#fff0f0">"Sedan"</span>,
<span style="color:#b06;font-weight:bold">"size"</span>: <span style="color:#d20;background-color:#fff0f0">"Full size"</span>,
<span style="color:#b06;font-weight:bold">"years"</span>: []
}
]
}
</code></pre></div><p>And I got back a 400 response with this payload:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"type"</span>: <span style="color:#d20;background-color:#fff0f0">"https://tools.ietf.org/html/rfc7231#section-6.5.1"</span>,
<span style="color:#b06;font-weight:bold">"title"</span>: <span style="color:#d20;background-color:#fff0f0">"One or more validation errors occurred."</span>,
<span style="color:#b06;font-weight:bold">"status"</span>: <span style="color:#00d;font-weight:bold">400</span>,
<span style="color:#b06;font-weight:bold">"traceId"</span>: <span style="color:#d20;background-color:#fff0f0">"00-0fd4f00eeb9f2f458ccefc180fcfba1c-79a618f13218394b-00"</span>,
<span style="color:#b06;font-weight:bold">"errors"</span>: {
<span style="color:#b06;font-weight:bold">"Name"</span>: [
<span style="color:#d20;background-color:#fff0f0">"The Name field is required."</span>
],
<span style="color:#b06;font-weight:bold">"Styles[0].Years"</span>: [
<span style="color:#d20;background-color:#fff0f0">"The field Years must be a string or array type with a minimum length of '1'."</span>
]
}
}
</code></pre></div><p>Pretty neat, huh? With minimal effort, we have some basic validation rules in place and a pretty usable response for when errors occur.</p>
<blockquote>
<p>To learn more about model validation, including all the various validation attributes included in the framework, check the official documentation: <a href="https://docs.microsoft.com/en-us/aspnet/core/mvc/models/validation?view=aspnetcore-5.0">Model validation</a> and <a href="https://docs.microsoft.com/en-us/dotnet/api/system.componentmodel.dataannotations?view=net-5.0">System.ComponentModel.DataAnnotations Namespace</a>.</p>
</blockquote>
<h4 id="validation-using-custom-attributes">Validation using custom attributes</h4>
<p>Of course, the framework is never going to cover all possible validation scenarios with the built-in attributes. Case in point, it’d be great to validate that the <code>Years</code> array contains values that look like actual years. That is, four-character, digit-only strings. There are no validation attributes for that. So, we need to create our own. Let’s add this file into a new <code>Validations</code> directory:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Validations/ContainsYearsAttribute.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Linq</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Runtime.CompilerServices</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Validation</span>
{
<span style="color:#888">// In the .NET Framework, attribute classes need to have their name suffixed with the word "Attribute".
</span><span style="color:#888"></span> <span style="color:#888">// Validation attributes need to inherit from `System.ComponentModel.DataAnnotations`'s `ValidationAttribute` class
</span><span style="color:#888"></span> <span style="color:#888">// and override the `IsValid` method.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">ContainsYearsAttribute</span> : ValidationAttribute
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#888;font-weight:bold">string</span> propertyName;
<span style="color:#888">// This constructor is called by the framework when the attribute is applied to some member. In this specific
</span><span style="color:#888"></span> <span style="color:#888">// case, we define a `propertyName` parameter annotated with a `CallerMemberName` attribute. This makes it so
</span><span style="color:#888"></span> <span style="color:#888">// the framework sends in the name of the member to which our `ContainsYears` attribute is applied to.
</span><span style="color:#888"></span> <span style="color:#888">// We store the value to use it later when constructing our validation error message.
</span><span style="color:#888"></span> <span style="color:#888">// Check https://docs.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.callermembernameattribute?view=net-5.0
</span><span style="color:#888"></span> <span style="color:#888">// for more info on `CallerMemberName`.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> ContainsYearsAttribute([CallerMemberName] <span style="color:#888;font-weight:bold">string</span> propertyName = <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#080;font-weight:bold">this</span>.propertyName = propertyName;
}
<span style="color:#888">// This method is called by the framework during validation. `value` is the actual value of the field that this
</span><span style="color:#888"></span> <span style="color:#888">// attribute will validate.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> ValidationResult IsValid(<span style="color:#888;font-weight:bold">object</span> <span style="color:#080;font-weight:bold">value</span>, ValidationContext validationContext)
{
<span style="color:#888">// By only applying the validation checks when the value is not null, we make it possible for this
</span><span style="color:#888"></span> <span style="color:#888">// attribute to work on optional fields. In other words, this attribute will skip validation if there is no
</span><span style="color:#888"></span> <span style="color:#888">// value to validate.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (<span style="color:#080;font-weight:bold">value</span> != <span style="color:#080;font-weight:bold">null</span>)
{
<span style="color:#888">// Check if all the elements of the string array are valid years. Check the `IsValidYear` method below
</span><span style="color:#888"></span> <span style="color:#888">// to see what checks are applied for each of the array elements.
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> isValid = (<span style="color:#080;font-weight:bold">value</span> <span style="color:#080;font-weight:bold">as</span> <span style="color:#888;font-weight:bold">string</span>[]).All(IsValidYear);
<span style="color:#080;font-weight:bold">if</span> (!isValid)
{
<span style="color:#888">// If not, return an error.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> ValidationResult(GetErrorMessage());
}
}
<span style="color:#888">// Return a successful validation result if no errors were detected.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> ValidationResult.Success;
}
<span style="color:#888">// Determines if a given value is valid by making sure it's not null, nor empty, that its length is 4 and that
</span><span style="color:#888"></span> <span style="color:#888">// all its characters are digits.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#888;font-weight:bold">bool</span> IsValidYear(<span style="color:#888;font-weight:bold">string</span> <span style="color:#080;font-weight:bold">value</span>) =>
!String.IsNullOrEmpty(<span style="color:#080;font-weight:bold">value</span>) && <span style="color:#080;font-weight:bold">value</span>.Length == <span style="color:#00d;font-weight:bold">4</span> && <span style="color:#080;font-weight:bold">value</span>.All(Char.IsDigit);
<span style="color:#888">// Builds a user friendly error message which includes the name of the field that this validation attribute has
</span><span style="color:#888"></span> <span style="color:#888">// been applied to.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#888;font-weight:bold">string</span> GetErrorMessage() =>
<span style="color:#d20;background-color:#fff0f0">$"The {propertyName} field must be an array of strings containing four digits."</span>;
}
}
</code></pre></div><p>Check the comments in the code for more details into how that class works. Then, we apply our custom attribute to our <code>ModelSpecificationStyle</code> class in the same way that we applied the built in ones:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">using System.ComponentModel.DataAnnotations;
<span style="color:#000;background-color:#dfd">+using VehicleQuotes.Validation;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.ResourceModels
{
public class ModelSpecificationStyle
{
// ...
[Required]
[MinLength(1)]
<span style="color:#000;background-color:#dfd">+ [ContainsYears]
</span><span style="color:#000;background-color:#dfd"></span> public string[] Years { get; set; }
}
}
</code></pre></div><p>Now do a <code>dotnet run</code> and try to POST to <code>/api/Makes/{makeId}/Models</code> this payload:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"name"</span>: <span style="color:#d20;background-color:#fff0f0">"Rav4"</span>,
<span style="color:#b06;font-weight:bold">"styles"</span>: [
{
<span style="color:#b06;font-weight:bold">"bodyType"</span>: <span style="color:#d20;background-color:#fff0f0">"SUV"</span>,
<span style="color:#b06;font-weight:bold">"size"</span>: <span style="color:#d20;background-color:#fff0f0">"Mid size"</span>,
<span style="color:#b06;font-weight:bold">"years"</span>: [ <span style="color:#d20;background-color:#fff0f0">"not_a_year"</span> ]
}
]
}
</code></pre></div><p>That should make the API respond with this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"type"</span>: <span style="color:#d20;background-color:#fff0f0">"https://tools.ietf.org/html/rfc7231#section-6.5.1"</span>,
<span style="color:#b06;font-weight:bold">"title"</span>: <span style="color:#d20;background-color:#fff0f0">"One or more validation errors occurred."</span>,
<span style="color:#b06;font-weight:bold">"status"</span>: <span style="color:#00d;font-weight:bold">400</span>,
<span style="color:#b06;font-weight:bold">"traceId"</span>: <span style="color:#d20;background-color:#fff0f0">"00-9980325f3e388f48a5975ef382d5b137-2d55da1bb9613e4f-00"</span>,
<span style="color:#b06;font-weight:bold">"errors"</span>: {
<span style="color:#b06;font-weight:bold">"Styles[0].Years"</span>: [
<span style="color:#d20;background-color:#fff0f0">"The Years field must be an array of strings containing four numbers."</span>
]
}
}
</code></pre></div><p>That’s our custom validation attribute doing its job.</p>
<p>There’s another aspect that we could validate using a custom validation attribute. What happens if we try to POST a payload with a body type or size that doesn’t exist? These queries from the <code>PostModel</code> method would throw an <code>InvalidOperationException</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs">BodyType = <span style="color:#00d;font-weight:bold">_</span>context.BodyTypes.Single(bodyType => bodyType.Name == style.BodyType)
</code></pre></div><p>and</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs">Size = <span style="color:#00d;font-weight:bold">_</span>context.Sizes.Single(size => size.Name == style.Size)
</code></pre></div><p>They do so because we used the <code>Single</code> method, which is designed like that. It tries to find a body type or size whose name is the given value, can’t find it, and thus, throws an exception.</p>
<blockquote>
<p>If, for example, we wanted not-founds to return <code>null</code>, we could have used <code>SingleOrDefault</code> instead.</p>
</blockquote>
<p>This unhandled exception results in a response that’s quite unbecoming:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/invalid-operation-exception.png" alt="InvalidOperationException during POST"></p>
<p>So, to prevent that exception and control the error messaging, we need a couple of new validation attributes that go into the <code>body_types</code> and <code>sizes</code> tables and check if the given values exist. Here’s what one would look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Validations/VehicleBodyTypeAttribute.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Linq</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Validation</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">VehicleBodyTypeAttribute</span> : ValidationAttribute
{
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> ValidationResult IsValid(<span style="color:#888;font-weight:bold">object</span> <span style="color:#080;font-weight:bold">value</span>, ValidationContext validationContext)
{
<span style="color:#080;font-weight:bold">if</span> (<span style="color:#080;font-weight:bold">value</span> == <span style="color:#080;font-weight:bold">null</span>) <span style="color:#080;font-weight:bold">return</span> ValidationResult.Success;
<span style="color:#888;font-weight:bold">var</span> dbContext = validationContext.GetService(<span style="color:#080;font-weight:bold">typeof</span>(VehicleQuotesContext)) <span style="color:#080;font-weight:bold">as</span> VehicleQuotesContext;
<span style="color:#888;font-weight:bold">var</span> bodyTypes = dbContext.BodyTypes.Select(bt => bt.Name).ToList();
<span style="color:#080;font-weight:bold">if</span> (!bodyTypes.Contains(<span style="color:#080;font-weight:bold">value</span>))
{
<span style="color:#888;font-weight:bold">var</span> allowed = String.Join(<span style="color:#d20;background-color:#fff0f0">", "</span>, bodyTypes);
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> ValidationResult(
<span style="color:#d20;background-color:#fff0f0">$"Invalid vehicle body type {value}. Allowed values are {allowed}."</span>
);
}
<span style="color:#080;font-weight:bold">return</span> ValidationResult.Success;
}
}
}
</code></pre></div><blockquote>
<p>You can find the other one here: <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Validations/VehicleSizeAttribute.cs">Validations/VehicleSizeAttribute.cs</a>.</p>
</blockquote>
<p>These two are very similar to one another. The most interesting part is how we use the <code>IsValid</code> method’s second parameter (<code>ValidationContext</code>) to obtain an instance of <code>VehicleQuotesContext</code> that we can use to query the database. The rest should be pretty self-explanatory. These attributes are classes that inherit from <code>System.ComponentModel.DataAnnotations</code>’s <code>ValidationAttribute</code> and implement the <code>IsValid</code> method. The method then checks that the value under scrutiny exists in the corresponding table and if it does not, raises a validation error. The validation error includes a list of all allowed values. They can be applied to our <code>ModelSpecificationStyle</code> class like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">// ...
namespace VehicleQuotes.ResourceModels
{
public class ModelSpecificationStyle
{
[Required]
<span style="color:#000;background-color:#dfd">+ [VehicleBodyType]
</span><span style="color:#000;background-color:#dfd"></span> public string BodyType { get; set; }
[Required]
<span style="color:#000;background-color:#dfd">+ [VehicleSize]
</span><span style="color:#000;background-color:#dfd"></span> public string Size { get; set; }
//...
}
}
</code></pre></div><p>Now, a request like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"name"</span>: <span style="color:#d20;background-color:#fff0f0">"Rav4"</span>,
<span style="color:#b06;font-weight:bold">"styles"</span>: [
{
<span style="color:#b06;font-weight:bold">"bodyType"</span>: <span style="color:#d20;background-color:#fff0f0">"not_a_body_type"</span>,
<span style="color:#b06;font-weight:bold">"size"</span>: <span style="color:#d20;background-color:#fff0f0">"Mid size"</span>,
<span style="color:#b06;font-weight:bold">"years"</span>: [ <span style="color:#d20;background-color:#fff0f0">"2000"</span> ]
}
]
}
</code></pre></div><p>Produces a response like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-json" data-lang="json">{
<span style="color:#b06;font-weight:bold">"type"</span>: <span style="color:#d20;background-color:#fff0f0">"https://tools.ietf.org/html/rfc7231#section-6.5.1"</span>,
<span style="color:#b06;font-weight:bold">"title"</span>: <span style="color:#d20;background-color:#fff0f0">"One or more validation errors occurred."</span>,
<span style="color:#b06;font-weight:bold">"status"</span>: <span style="color:#00d;font-weight:bold">400</span>,
<span style="color:#b06;font-weight:bold">"traceId"</span>: <span style="color:#d20;background-color:#fff0f0">"00-9ad59a7aff60944ab54c19a73be73cc7-eeabafe03df74e40-00"</span>,
<span style="color:#b06;font-weight:bold">"errors"</span>: {
<span style="color:#b06;font-weight:bold">"Styles[0].BodyType"</span>: [
<span style="color:#d20;background-color:#fff0f0">"Invalid vehicle body type not_a_body_type. Allowed values are Coupe, Sedan, Convertible, Hatchback, SUV, Truck."</span>
]
}
}
</code></pre></div><h4 id="implementing-endpoints-for-quote-rules-and-overrides">Implementing endpoints for quote rules and overrides</h4>
<p>At this point we’ve explored many of the most common features available to us for developing Web APIs. So much so that implementing the next two pieces of functionality for our app doesn’t really introduce any new concepts. So, I wont discuss that here in great detail.</p>
<p>Feel free to browse the source code <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api">on GitHub</a> if you want though. These are the relevant files:</p>
<ul>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Controllers/QuoteOverridesController.cs">Controllers/QuoteOverridesController.cs</a></li>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Controllers/QuoteRulesController.cs">Controllers/QuoteRulesController.cs</a></li>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Models/QuoteOverride.cs">Models/QuoteOverride.cs</a></li>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Models/QuoteRule.cs">Models/QuoteRule.cs</a></li>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/ResourceModels/QuoteOverrideSpecification.cs">ResourceModels/QuoteOverrideSpecification.cs</a></li>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Validations/FeatureTypeAttribute.cs">Validation/FeatureTypeAttribute.cs</a></li>
<li><a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/Migrations/20210627204444_AddQuoteRulesAndOverridesTables.cs">Migrations/20210627204444_AddQuoteRulesAndOverridesTables.cs</a></li>
</ul>
<p>The <code>FeatureTypeAttribute</code> class is interesting in that it provides another example of a validation attribute. This time is one that makes sure the value being validated is included in an array of strings that’s defined literally in the code.</p>
<p>Other than that, it’s all stuff we’ve already covered: models, migrations, scaffolding controllers, custom routes, resource models, etc.</p>
<p>If you are following along, be sure to add those files and run a <code>dotnet ef database update</code> to apply the migration.</p>
<h4 id="implementing-the-quote-model">Implementing the quote model</h4>
<p>Let’s now start implementing the main capability of our app: calculating quotes for vehicles. Let’s start with the <code>Quote</code> entity. This is what the new <code>Models/Quote.cs</code> file containing the entity class will look like:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">// Models/Quote.cs
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.ComponentModel.DataAnnotations.Schema</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">Quote</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> ID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888">// Directly tie this quote record to a specific vehicle that we have
</span><span style="color:#888"></span> <span style="color:#888">// registered in our db, if we have it.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int?</span> ModelStyleYearID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#888">// If we don't have the specific vehicle in our db, then store the
</span><span style="color:#888"></span> <span style="color:#888">// vehicle model details independently.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Year { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Make { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Model { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> BodyTypeID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> SizeID { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> ItMoves { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasAllWheels { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasAlloyWheels { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasAllTires { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasKey { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasTitle { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> RequiresPickup { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasEngine { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasTransmission { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">bool</span> HasCompleteInterior { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">int</span> OfferedQuote { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">string</span> Message { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> DateTime CreatedAt { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> ModelStyleYear ModelStyleYear { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> BodyType BodyType { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
<span style="color:#080;font-weight:bold">public</span> Size Size { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
}
</code></pre></div><p>This should be pretty familiar by now. It’s a plain old class that defines a number of properties, one for each of the fields in the resulting table and a few navigation properties that serve to access related data.</p>
<p>The only aspect worth noting is that we’ve defined the <code>ModelStyleYearID</code> property as a nullable integer (with <code>int?</code>). This is because, like we discussed at the beginning, the foreign key from <code>quotes</code> to <code>vehicle_style_years</code> is actually optional. The reason being that we may receive a quote request for a vehicle that we don’t have registered in our database. We need to be able to support quoting those vehicles too, so if we don’t have the requested vehicle registered, then that foreign key will stay unpopulated and we’ll rely on the other fields (i.e. <code>Year</code>, <code>Make</code>, <code>Model</code>, <code>BodyTypeID</code> and <code>SizeID</code>) to identify the vehicle and calculate the quote for it.</p>
<h4 id="using-dependency-injection">Using Dependency Injection</h4>
<p>So far we’ve been putting a lot of logic in our controllers. That’s generally not ideal, but fine as long as the logic is simple. The problem is that a design like that can quickly become a hindrance for maintainability and testing as our application grows more complex. For the logic that calculates a quote, we’d be better served by implementing it in its own class, outside of the controller that should only care about defining endpoints and handling HTTP concerns. Then, the controller can be given access to that class and delegate to it all the quote calculation logic. Thankfully, ASP.NET Core includes an IoC container by default, which allows us to use Dependency Injection to solve these kinds of problems. Let’s see what that looks like.</p>
<p>For working with quotes, we want to offer two endpoints:</p>
<ol>
<li>A <code>POST api/Quotes</code> that captures the vehicle information, calculates the quote, keeps record of the request, and responds with the calculated value.</li>
<li>A <code>GET api/Quotes</code> that returns all the currently registered quotes in the system.</li>
</ol>
<p>Using the Dependency Injection capabilities, a controller that implements those two could look like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Collections.Generic</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Threading.Tasks</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.AspNetCore.Mvc</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Services</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Controllers</span>
{
<span style="color:#369"> [Route("api/[controller]</span><span style="color:#d20;background-color:#fff0f0">")]
</span><span style="color:#d20;background-color:#fff0f0"></span><span style="color:#369"> [ApiController]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuotesController</span> : ControllerBase
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> QuoteService <span style="color:#00d;font-weight:bold">_</span>service;
<span style="color:#888">// When intiating the request processing logic, the framework recognizes
</span><span style="color:#888"></span> <span style="color:#888">// that this controller has a dependency on QuoteService and expects an
</span><span style="color:#888"></span> <span style="color:#888">// instance of it to be injected via the constructor. The framework then
</span><span style="color:#888"></span> <span style="color:#888">// does what it needs to do in order to provide that dependency.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> QuotesController(QuoteService service)
{
<span style="color:#00d;font-weight:bold">_</span>service = service;
}
<span style="color:#888">// GET: api/Quotes
</span><span style="color:#888"></span><span style="color:#369"> [HttpGet]</span>
<span style="color:#888">// This method returns a collection of a new resource model instead of just the `Quote` entity directly.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<IEnumerable<SubmittedQuoteRequest>>> GetAll()
{
<span style="color:#888">// Instead of directly implementing the logic in this method, we call on
</span><span style="color:#888"></span> <span style="color:#888">// the service class and let it take care of the rest.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>service.GetAllQuotes();
}
<span style="color:#888">// POST: api/Quotes
</span><span style="color:#888"></span><span style="color:#369"> [HttpPost]</span>
<span style="color:#888">// This method receives as a paramater a `QuoteRequest` of just the `Quote` entity directly.
</span><span style="color:#888"></span> <span style="color:#888">// That way callers of this endpoint don't need to be exposed to the details of our data model implementation.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<SubmittedQuoteRequest>> Post(QuoteRequest request)
{
<span style="color:#888">// Instead of directly implementing the logic in this method, we call on
</span><span style="color:#888"></span> <span style="color:#888">// the service class and let it take care of the rest.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>service.CalculateQuote(request);
}
}
}
</code></pre></div><p>As you can see, we’ve once again opted to abstract away clients from the implementation details of our data model and used Resource Models for the API contract instead of the <code>Quote</code> entity directly. We have one for input data that’s called <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/ResourceModels/QuoteRequest.cs"><code>QuoteRequest</code></a> and another one for output: <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/blob/master/ResourceModels/SubmittedQuoteRequest.cs"><code>SubmittedQuoteRequest</code></a>. Not very remarkable by themselves, but feel free to explore the source code in <a href="https://github.com/megakevin/end-point-blog-dotnet-5-web-api/tree/master/ResourceModels">the GitHub repo</a>.</p>
<p>This controller has a dependency on <code>QuoteService</code>, which it uses to perform all of the necessary logic. This class is not defined yet so let’s do that next:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Collections.Generic</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Linq</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Threading.Tasks</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">Microsoft.EntityFrameworkCore</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Models</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.ResourceModels</span>;
<span style="color:#080;font-weight:bold">namespace</span> <span style="color:#b06;font-weight:bold">VehicleQuotes.Services</span>
{
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">QuoteService</span>
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> VehicleQuotesContext <span style="color:#00d;font-weight:bold">_</span>context;
<span style="color:#888">// This constructor defines a dependency on VehicleQuotesContext, similar to most of our controllers.
</span><span style="color:#888"></span> <span style="color:#888">// Via the built in dependency injection features, the framework makes sure to provide this parameter when
</span><span style="color:#888"></span> <span style="color:#888">// creating new instances of this class.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> QuoteService(VehicleQuotesContext context)
{
<span style="color:#00d;font-weight:bold">_</span>context = context;
}
<span style="color:#888">// This method takes all the records from the `quotes` table and constructs `SubmittedQuoteRequest`s with them.
</span><span style="color:#888"></span> <span style="color:#888">// Then returns that as a list.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<List<SubmittedQuoteRequest>> GetAllQuotes()
{
<span style="color:#888;font-weight:bold">var</span> quotesToReturn = <span style="color:#00d;font-weight:bold">_</span>context.Quotes.Select(q => <span style="color:#080;font-weight:bold">new</span> SubmittedQuoteRequest
{
ID = q.ID,
CreatedAt = q.CreatedAt,
OfferedQuote = q.OfferedQuote,
Message = q.Message,
Year = q.Year,
Make = q.Make,
Model = q.Model,
BodyType = q.BodyType.Name,
Size = q.Size.Name,
ItMoves = q.ItMoves,
HasAllWheels = q.HasAllWheels,
HasAlloyWheels = q.HasAlloyWheels,
HasAllTires = q.HasAllTires,
HasKey = q.HasKey,
HasTitle = q.HasTitle,
RequiresPickup = q.RequiresPickup,
HasEngine = q.HasEngine,
HasTransmission = q.HasTransmission,
HasCompleteInterior = q.HasCompleteInterior,
});
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> quotesToReturn.ToListAsync();
}
<span style="color:#888">// This method takes an incoming `QuoteRequest` and calculates a quote based on the vehicle described by it.
</span><span style="color:#888"></span> <span style="color:#888">// To calculate this quote, it looks for any overrides before trying to use the currently existing rules defined
</span><span style="color:#888"></span> <span style="color:#888">// in the `quote_rules` table. It also stores a record on the `quotes` table with all the incoming data and the
</span><span style="color:#888"></span> <span style="color:#888">// quote calculation result. It returns back the quote value as well as a message explaining the conditions of
</span><span style="color:#888"></span> <span style="color:#888">// the quote.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<SubmittedQuoteRequest> CalculateQuote(QuoteRequest request)
{
<span style="color:#888;font-weight:bold">var</span> response = <span style="color:#080;font-weight:bold">this</span>.CreateResponse(request);
<span style="color:#888;font-weight:bold">var</span> quoteToStore = <span style="color:#080;font-weight:bold">await</span> <span style="color:#080;font-weight:bold">this</span>.CreateQuote(request);
<span style="color:#888;font-weight:bold">var</span> requestedModelStyleYear = <span style="color:#080;font-weight:bold">await</span> <span style="color:#080;font-weight:bold">this</span>.FindModelStyleYear(request);
QuoteOverride quoteOverride = <span style="color:#080;font-weight:bold">null</span>;
<span style="color:#080;font-weight:bold">if</span> (requestedModelStyleYear != <span style="color:#080;font-weight:bold">null</span>)
{
quoteToStore.ModelStyleYear = requestedModelStyleYear;
quoteOverride = <span style="color:#080;font-weight:bold">await</span> <span style="color:#080;font-weight:bold">this</span>.FindQuoteOverride(requestedModelStyleYear);
<span style="color:#080;font-weight:bold">if</span> (quoteOverride != <span style="color:#080;font-weight:bold">null</span>)
{
response.OfferedQuote = quoteOverride.Price;
}
}
<span style="color:#080;font-weight:bold">if</span> (quoteOverride == <span style="color:#080;font-weight:bold">null</span>)
{
response.OfferedQuote = <span style="color:#080;font-weight:bold">await</span> <span style="color:#080;font-weight:bold">this</span>.CalculateOfferedQuote(request);
}
<span style="color:#080;font-weight:bold">if</span> (requestedModelStyleYear == <span style="color:#080;font-weight:bold">null</span>)
{
response.Message = <span style="color:#d20;background-color:#fff0f0">"Offer subject to change upon vehicle inspection."</span>;
}
quoteToStore.OfferedQuote = response.OfferedQuote;
quoteToStore.Message = response.Message;
<span style="color:#00d;font-weight:bold">_</span>context.Quotes.Add(quoteToStore);
<span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.SaveChangesAsync();
response.ID = quoteToStore.ID;
response.CreatedAt = quoteToStore.CreatedAt;
<span style="color:#080;font-weight:bold">return</span> response;
}
<span style="color:#888">// Creates a `SubmittedQuoteRequest`, initialized with default values, using the data from the incoming
</span><span style="color:#888"></span> <span style="color:#888">// `QuoteRequest`. `SubmittedQuoteRequest` is what gets returned in the response payload of the quote endpoints.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> SubmittedQuoteRequest CreateResponse(QuoteRequest request)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> SubmittedQuoteRequest
{
OfferedQuote = <span style="color:#00d;font-weight:bold">0</span>,
Message = <span style="color:#d20;background-color:#fff0f0">"This is our final offer."</span>,
Year = request.Year,
Make = request.Make,
Model = request.Model,
BodyType = request.BodyType,
Size = request.Size,
ItMoves = request.ItMoves,
HasAllWheels = request.HasAllWheels,
HasAlloyWheels = request.HasAlloyWheels,
HasAllTires = request.HasAllTires,
HasKey = request.HasKey,
HasTitle = request.HasTitle,
RequiresPickup = request.RequiresPickup,
HasEngine = request.HasEngine,
HasTransmission = request.HasTransmission,
HasCompleteInterior = request.HasCompleteInterior,
};
}
<span style="color:#888">// Creates a `Quote` based on the data from the incoming `QuoteRequest`. This is the object that gets eventually
</span><span style="color:#888"></span> <span style="color:#888">// stored in the database.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">async</span> Task<Quote> CreateQuote(QuoteRequest request)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> Quote
{
Year = request.Year,
Make = request.Make,
Model = request.Model,
BodyTypeID = (<span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.BodyTypes.SingleAsync(bt => bt.Name == request.BodyType)).ID,
SizeID = (<span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.Sizes.SingleAsync(s => s.Name == request.Size)).ID,
ItMoves = request.ItMoves,
HasAllWheels = request.HasAllWheels,
HasAlloyWheels = request.HasAlloyWheels,
HasAllTires = request.HasAllTires,
HasKey = request.HasKey,
HasTitle = request.HasTitle,
RequiresPickup = request.RequiresPickup,
HasEngine = request.HasEngine,
HasTransmission = request.HasTransmission,
HasCompleteInterior = request.HasCompleteInterior,
CreatedAt = DateTime.Now
};
}
<span style="color:#888">// Tries to find a registered vehicle that matches the one for which the quote is currently being requested.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">async</span> Task<ModelStyleYear> FindModelStyleYear(QuoteRequest request)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.ModelStyleYears.FirstOrDefaultAsync(msy =>
msy.Year == request.Year &&
msy.ModelStyle.Model.Make.Name == request.Make &&
msy.ModelStyle.Model.Name == request.Model &&
msy.ModelStyle.BodyType.Name == request.BodyType &&
msy.ModelStyle.Size.Name == request.Size
);
}
<span style="color:#888">// Tries to find an override for the vehicle for which the quote is currently being requested.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">async</span> Task<QuoteOverride> FindQuoteOverride(ModelStyleYear modelStyleYear)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.QuoteOverides
.FirstOrDefaultAsync(qo => qo.ModelStyleYear == modelStyleYear);
}
<span style="color:#888">// Uses the rules stored in the `quote_rules` table to calculate how much money to offer for the vehicle
</span><span style="color:#888"></span> <span style="color:#888">// described in the incoming `QuoteRequest`.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">async</span> Task<<span style="color:#888;font-weight:bold">int</span>> CalculateOfferedQuote(QuoteRequest request)
{
<span style="color:#888;font-weight:bold">var</span> rules = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_</span>context.QuoteRules.ToListAsync();
<span style="color:#888">// Given a vehicle feature type, find a rule that applies to that feature type and has the value that
</span><span style="color:#888"></span> <span style="color:#888">// matches the condition of the incoming vehicle being quoted.
</span><span style="color:#888"></span> Func<<span style="color:#888;font-weight:bold">string</span>, QuoteRule> theMatchingRule = featureType =>
rules.FirstOrDefault(r =>
r.FeatureType == featureType &&
r.FeatureValue == request[featureType]
);
<span style="color:#888">// For each vehicle feature that we care about, sum up the the monetary values of all the rules that match
</span><span style="color:#888"></span> <span style="color:#888">// the given vehicle condition.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">return</span> QuoteRule.FeatureTypes.All
.Select(theMatchingRule)
.Where(r => r != <span style="color:#080;font-weight:bold">null</span>)
.Sum(r => r.PriceModifier);
}
}
}
</code></pre></div><p>Finally, we need to tell the framework that this class is available for Dependency Injection. Similarly to how we did with our <code>VehicleQuotesContext</code>, we do so in the <code>Startup.cs</code> file’s <code>ConfigureServices</code> method. Just add this line at the top:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs">services.AddScoped<Services.QuoteService>();
</code></pre></div><blockquote>
<p>The core tenet of Inversion of Control is to depend on abstractions, not on implementations. So ideally, we would not have our controller directly call for a <code>QuoteService</code> instance. Instead, we would have it reference an abstraction, e.g. an interface like <code>IQuoteService</code>. The statement on <code>Startup.cs</code> would then look like this instead: <code>services.AddScoped<Services.IQuoteService, Services.QuoteService>();</code>.</p>
<p>This is important because it would allow us to unit test the component that depends on our service class (i.e. the controller in this case) by passing it a <a href="https://en.wikipedia.org/wiki/Mock_object">mock object</a> — one that also implements <code>IQuoteService</code> but does not really implement all the functionality of the actual <code>QuoteService</code> class. Since the controller only knows about the interface (that is, it “depends on an abstraction”), the actual object that we give it as a dependency doesn’t matter to it, as long as it implements that interface. This ability to inject mocks as dependencies is invaluable during testing. Testing is beyond the scope of this article though, so I’ll stick with the simpler approach with a static dependency on a concrete class. Know that this is not a good practice when it comes to actual production systems.</p>
</blockquote>
<p>And that’s all it takes. Once you add a few rules via <code>POST /api/QuoteRules</code>, you should be able to get some vehicles quoted with <code>POST /api/Quotes</code>. And also see what the system has stored via <code>GET /api/Quotes</code>.</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/a-quote.png" alt="A Quote"></p>
<p>And that’s all the functionality that we set out to build into our REST API! There are a few other neat things that I thought I’d include though.</p>
<h4 id="adding-seed-data-for-lookup-tables">Adding seed data for lookup tables</h4>
<p>Our vehicle size and body type data isn’t meant to really change much. In fact, we could even preload that data when our application starts. EF Core provides a data seeding feature that we can access via configurations on the <code>DbContext</code> itself. For our case, we could add this method to our <code>VehicleQuotesContext</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Size>().HasData(
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">1</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Subcompact"</span> },
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">2</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Compact"</span> },
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">3</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Mid Size"</span> },
<span style="color:#080;font-weight:bold">new</span> Size { ID = <span style="color:#00d;font-weight:bold">5</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Full Size"</span> }
);
modelBuilder.Entity<BodyType>().HasData(
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">1</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Coupe"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">2</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Sedan"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">3</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Hatchback"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">4</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Wagon"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">5</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Convertible"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">6</span>, Name = <span style="color:#d20;background-color:#fff0f0">"SUV"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">7</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Truck"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">8</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Mini Van"</span> },
<span style="color:#080;font-weight:bold">new</span> BodyType { ID = <span style="color:#00d;font-weight:bold">9</span>, Name = <span style="color:#d20;background-color:#fff0f0">"Roadster"</span> }
);
}
</code></pre></div><p><a href="https://docs.microsoft.com/en-us/ef/core/modeling/#use-fluent-api-to-configure-a-model"><code>OnModelCreating</code></a> is a hook that we can define to run some code at the time the model is being created for the first time. Here, we’re using it to seed some data. In order to apply that, a migration needs to be created and executed. If you’ve added some data to the database, be sure to wipe it before running the migration so that we don’t run into unique constraint violations. Here are the migrations:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ dotnet ef migrations add AddSeedDataForSizesAndBodyTypes
$ dotnet ef database update
</code></pre></div><p>After that’s done, it no longer makes sense to allow creating, updating, deleting and fetching individual sizes and body types, so I would delete those endpoints from the respective controllers.</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/body-types-get-all-only.png" alt="Body Types, GET all only"></p>
<p><img src="/blog/2021/07/dotnet-5-web-api/sizes-get-all-only.png" alt="Sizes, GET all only"></p>
<blockquote>
<p>There are other options for data seeding in EF Core. Take a look: <a href="https://docs.microsoft.com/en-us/ef/core/modeling/data-seeding">Data Seeding</a>.</p>
</blockquote>
<h4 id="improving-the-swagger-ui-via-xml-comments">Improving the Swagger UI via XML comments</h4>
<p>Our current auto-generated Swagger UI is pretty awesome. Especially considering that we got it for free. It’s a little lacking when it comes to more documentation about specific endpoint summaries or expected responses. The good news is that there’s a way to leverage <a href="https://docs.microsoft.com/en-us/dotnet/csharp/codedoc">C# XML Comments</a> in order to improve the Swagger UI.</p>
<p>We can add support for that by configuring our project to produce, at build time, an XML file with the docs that we write. In order to do so, we need to update the <code>VehicleQuotes.csproj</code> like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff"><Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<!-- ... -->
<span style="color:#000;background-color:#dfd">+ <GenerateDocumentationFile>true</GenerateDocumentationFile>
</span><span style="color:#000;background-color:#dfd">+ <NoWarn>$(NoWarn);1591</NoWarn>
</span><span style="color:#000;background-color:#dfd"></span> </PropertyGroup>
<!-- ... -->
</Project>
</code></pre></div><p><code>GenerateDocumentationFile</code> is the flag that tells the .NET 5 build tools to generate the documentation file. The <code>NoWarn</code> element prevents our build output from getting cluttered with a lot of warnings saying that some classes and methods are not properly documented. We don’t want that because we just want to write enough documentation for the Swagger UI. And that includes only the controllers.</p>
<p>You can run <code>dotnet build</code> and look for the new file in <code>bin/Debug/net5.0/VehicleQuotes.xml</code>.</p>
<p>Then, we need to update <code>Startup.cs</code>. First we need to add the following <code>using</code> statements:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.IO</span>;
<span style="color:#080;font-weight:bold">using</span> <span style="color:#b06;font-weight:bold">System.Reflection</span>;
</code></pre></div><p>And add the following code to the <code>ConfigureServices</code> method on <code>Startup.cs</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">public void ConfigureServices(IServiceCollection services)
{
// ...
services.AddSwaggerGen(c =>
{
c.SwaggerDoc("v1", new OpenApiInfo { Title = "VehicleQuotes", Version = "v1" });
<span style="color:#000;background-color:#dfd">+ c.IncludeXmlComments(
</span><span style="color:#000;background-color:#dfd">+ Path.Combine(
</span><span style="color:#000;background-color:#dfd">+ AppContext.BaseDirectory,
</span><span style="color:#000;background-color:#dfd">+ $"{Assembly.GetExecutingAssembly().GetName().Name}.xml"
</span><span style="color:#000;background-color:#dfd">+ )
</span><span style="color:#000;background-color:#dfd"></span> );
});
// ...
}
</code></pre></div><p>This makes it so the <code>SwaggerGen</code> service knows to look for the XML documentation file when building up the Open API specification file used for generating the Swagger UI.</p>
<p>Now that all of that is set up, we can actually write some XML comments and attributes that will enhance our Swagger UI. As an example, put this on top of <code>ModelsController</code>’s <code>Post</code> method:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cs" data-lang="cs"><span style="color:#888">/// <summary>
</span><span style="color:#888">/// Creates a new vehicle model for the given make.
</span><span style="color:#888">/// </summary>
</span><span style="color:#888">/// <param name="makeId">The ID of the vehicle make to add the model to.</param>
</span><span style="color:#888">/// <param name="model">The data to create the new model with.</param>
</span><span style="color:#888">/// <response code="201">When the request is valid.</response>
</span><span style="color:#888">/// <response code="404">When the specified vehicle make does not exist.</response>
</span><span style="color:#888">/// <response code="409">When there's already another model in the same make with the same name.</response>
</span><span style="color:#888"></span><span style="color:#369">[HttpPost]</span>
<span style="color:#369">[ProducesResponseType(StatusCodes.Status201Created)]</span>
<span style="color:#369">[ProducesResponseType(StatusCodes.Status404NotFound)]</span>
<span style="color:#369">[ProducesResponseType(StatusCodes.Status409Conflict)]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<ActionResult<ModelSpecification>> Post([FromRoute] <span style="color:#888;font-weight:bold">int</span> makeId, ModelSpecification model)
{
<span style="color:#888">// ...
</span><span style="color:#888"></span>}
</code></pre></div><p>Then, the Swagger UI now looks like this for this endpoint:</p>
<p><img src="/blog/2021/07/dotnet-5-web-api/fully-documented-post-models.png" alt="Fully documented POST Models endpoint"></p>
<h4 id="configuring-the-app-via-settings-files-and-environment-variables">Configuring the app via settings files and environment variables</h4>
<p>Another aspect that’s important to web applications is having them be configurable via things like configuration files or environment variables. The framework already has provision for this, we just need to use it. I’m talking about the <code>appsettings</code> files.</p>
<p>We have two of them created for us by default: <code>appsettings.json</code> which is applied in all environments, and <code>appsettings.Development.json</code> that is applied only under development environments. The environment is given by the <code>ASPNETCORE_ENVIRONMENT</code> environment variable, and it can be set to either <code>Development</code>, <code>Staging</code>, or <code>Production</code> by default. That means that if we had, for example, an <code>appsettings.Staging.json</code> file, the settings defined within would be loaded if the <code>ASPNETCORE_ENVIRONMENT</code> environment variable were set to <code>Staging</code>. You get the idea.</p>
<blockquote>
<p>You can learn more about <a href="https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-5.0">configuration</a> and <a href="https://docs.microsoft.com/en-us/aspnet/core/fundamentals/environments?view=aspnetcore-5.0">environments</a> in the official documentation.</p>
</blockquote>
<p>Anyway, let’s add a new setting on <code>appsettings.json</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">{
// ...
<span style="color:#000;background-color:#dfd">+ "DefaultOffer": 77
</span><span style="color:#000;background-color:#dfd"></span>}
</code></pre></div><p>We’ll use this setting to give default offers when we’re not able to calculate appropriate quotes for vehicles. This can happen if we don’t have rules, or if the ones we have don’t match any of the incoming vehicle features or if for some other reason the final sum ends up in zero or negative number. We can use this setting in our <code>QuoteService</code> like so:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-diff" data-lang="diff">// ...
<span style="color:#000;background-color:#dfd">+using Microsoft.Extensions.Configuration;
</span><span style="color:#000;background-color:#dfd"></span>
namespace VehicleQuotes.Services
{
public class QuoteService
{
// ...
<span style="color:#000;background-color:#dfd">+ private readonly IConfiguration _configuration;
</span><span style="color:#000;background-color:#dfd"></span>
<span style="color:#000;background-color:#fdd">- public QuoteService(VehicleQuotesContext context)
</span><span style="color:#000;background-color:#fdd"></span><span style="color:#000;background-color:#dfd">+ public QuoteService(VehicleQuotesContext context, IConfiguration configuration)
</span><span style="color:#000;background-color:#dfd"></span> {
_context = context;
<span style="color:#000;background-color:#dfd">+ _configuration = configuration;
</span><span style="color:#000;background-color:#dfd"></span> }
// ...
public async Task<SubmittedQuoteRequest> CalculateQuote(QuoteRequest request)
{
// ...
<span style="color:#000;background-color:#dfd">+ if (response.OfferedQuote <= 0)
</span><span style="color:#000;background-color:#dfd">+ {
</span><span style="color:#000;background-color:#dfd">+ response.OfferedQuote = _configuration.GetValue<int>("DefaultOffer", 0);
</span><span style="color:#000;background-color:#dfd">+ }
</span><span style="color:#000;background-color:#dfd"></span>
quoteToStore.OfferedQuote = response.OfferedQuote;
// ...
}
// ...
}
}
</code></pre></div><p>Here, we’ve added a new parameter to the constructor to specify that <code>VehicleQuotesContext</code> has a dependency on <code>IConfiguration</code>. This prompts the framework to provide an instance of that when instantiating the class. We can use that instance to access the settings that we defined in the <code>appsettings.json</code> file via its <code>GetValue</code> method, like I demonstrated above.</p>
<p>The value of the settings in <code>appsettings.json</code> can be overridden by environment variables as well. On Linux, for example, we can run the app and set an environment value with a line like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#369">DefaultOffer</span>=<span style="color:#00d;font-weight:bold">123</span> dotnet run
</code></pre></div><p>This will make the application use <code>123</code> instead of <code>77</code> when it comes to the <code>DefaultOffer</code> setting. This flexibility is great from a DevOps perspective. And we had to do minimal work in order to get that going.</p>
<h3 id="thats-all-for-now">That’s all for now</h3>
<p>And that’s it! In this article we’ve gone through many of the features offered in <a href="https://docs.microsoft.com/en-us/dotnet/core/dotnet-five">.NET 5</a>, <a href="https://docs.microsoft.com/en-us/aspnet/core/introduction-to-aspnet-core?view=aspnetcore-5.0">ASP.NET Core</a>, and <a href="https://docs.microsoft.com/en-us/ef/core/">Entity Framework Core</a> to support some of the most common use cases when it comes to developing Web API applications.</p>
<p>We’ve installed .NET 5 and created an ASP.NET Core Web API project with EF Core and a few bells and whistles, created controllers to support many different endpoints, played a little bit with routes and response codes, created and built upon a data model and updated a database via entities and migrations, implemented database constraints using unique indexes, implemented input validation using both built-in and custom validation attributes, implemented resource models as DTOs for defining the contract of some of our API endpoints, tapped into the built-in dependency injection capabilities, explored and improved the auto-generated Swagger UI, added seed data for our database, learned about configuration via settings files and environment variables.</p>
<p>.NET 5 is looking great.</p>
<h3 id="table-of-contents">Table of contents</h3>
<ul>
<li><a href="#what-were-building">What we’re building</a>
<ul>
<li><a href="#the-demo-application">The demo application</a></li>
<li><a href="#the-data-model">The data model</a></li>
</ul>
</li>
<li><a href="#the-development-environment">The development environment</a>
<ul>
<li><a href="#setting-up-the-postgresql-database-with-docker">Setting up the PostgreSQL database with Docker</a></li>
<li><a href="#installing-the-net-5-sdk">Installing the .NET 5 SDK</a></li>
</ul>
</li>
<li><a href="#setting-up-the-project">Setting up the project</a>
<ul>
<li><a href="#creating-our-aspnet-core-rest-api-project">Creating our ASP.NET Core REST API project</a></li>
<li><a href="#installing-packages-well-need">Installing packages we’ll need</a></li>
<li><a href="#connecting-to-the-database-and-performing-initial-app-configuration">Connecting to the database and performing initial app configuration</a></li>
</ul>
</li>
<li><a href="#building-the-application">Building the application</a>
<ul>
<li><a href="#creating-model-entities-migrations-and-updating-the-database">Creating model entities, migrations and updating the database</a></li>
<li><a href="#creating-controllers-for-cruding-our-tables">Creating controllers for CRUDing our tables</a></li>
<li><a href="#adding-unique-constraints-via-indexes">Adding unique constraints via indexes</a></li>
<li><a href="#responding-with-specific-http-error-codes-409-conflict">Responding with specific HTTP error codes (409 Conflict)</a></li>
<li><a href="#adding-a-more-complex-entity-to-the-model">Adding a more complex entity to the model</a></li>
<li><a href="#adding-composite-unique-indexes">Adding composite unique indexes</a></li>
<li><a href="#adding-controllers-with-custom-routes">Adding controllers with custom routes</a></li>
<li><a href="#using-resource-models-as-dtos-for-controllers">Using resource models as DTOs for controllers</a></li>
<li><a href="#validation-using-built-in-data-annotations">Validation using built-in Data Annotations</a></li>
<li><a href="#validation-using-custom-attributes">Validation using custom attributes</a></li>
<li><a href="#implementing-endpoints-for-quote-rules-and-overrides">Implementing endpoints for quote rules and overrides</a></li>
<li><a href="#implementing-the-quote-model">Implementing the quote model</a></li>
<li><a href="#using-dependency-injection">Using Dependency Injection</a></li>
<li><a href="#adding-seed-data-for-lookup-tables">Adding seed data for lookup tables</a></li>
<li><a href="#improving-the-swagger-ui-via-xml-comments">Improving the Swagger UI via XML comments</a></li>
<li><a href="#configuring-the-app-via-settings-files-and-environment-variables">Configuring the app via settings files and environment variables</a></li>
</ul>
</li>
<li><a href="#thats-all-for-now">That’s all for now</a></li>
</ul>
Automating Windows Service Installationhttps://www.endpointdev.com/blog/2021/04/automating-windows-service-installation/2021-04-23T00:00:00+00:00Daniel Gomm
<p><img src="/blog/2021/04/automating-windows-service-installation/assembly-line.jpg" alt="Assembly line">
<a href="https://unsplash.com/photos/pAzSrQF3XUQ">Photo</a> by <a href="https://unsplash.com/@scienceinhd">Science in HD</a> on Unsplash</p>
<p>For me, setting up a service started as a clean one-liner that used <code>InstallUtil.exe</code>, but as time went on, I accumulated additional steps. Adding external files & folders, setting a custom <strong>Service Logon Account</strong>, and even an SSL cert had to be configured first before the service could be used. An entire checklist was needed just to make sure the service would start successfully. That’s when I realized a proper installation method was needed here. This article will go over how to make a dedicated <code>.msi</code> installer for a Windows Service that can do all these things and more.</p>
<p>Creating an installer can be tricky, because not all the available features are easy to find. In fact, the setup project itself is not included by default in Visual Studio; you need to install an extension in order to create one. But once the installer is created, we can use it to do things like:</p>
<ul>
<li>Configure the installer to copy the build output of a project to the <code>C:\Program Files (x86)</code> folder, as well as add custom files & folders to the installation</li>
<li>Add custom CLI flags to the installer to specify the <strong>Service Logon Account</strong> at install time</li>
<li>Add an installer class to the service and use the installation lifecycle hooks to write custom code that gets run at any stage of the installation.</li>
</ul>
<h3 id="a-note-on-compatibility">A Note On Compatibility</h3>
<p>For .NET Core and .NET 5.0 projects, you won’t be able to add an installer class. To use either .NET Core or .NET 5.0 to make a service instead, you’d need to make a different kind of project called a <strong>Worker Service</strong>. A Worker Service differs from a traditional <strong>Windows Service</strong> in that it’s more like a console application that spawns off a worker process on a new thread. It <em>can</em> be configured to run as a Windows service, but doesn’t have to be. So instead of using an installer, for a Worker Service you’d publish the project to an output directory and then use the <code>SC.exe</code> utility to add it as a Windows service:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bat" data-lang="bat">dotnet publish -o C:\<PUBLISH_PATH>
SC CREATE <WORKER_NAME> C:\<PUBLISH_PATH>
</code></pre></div><h3 id="creating-a-windows-setup-project">Creating a Windows Setup Project</h3>
<p>In order to create a .msi installer in Visual Studio 2019, you’ll need to install the <a href="https://marketplace.visualstudio.com/items?itemName=VisualStudioClient.MicrosoftVisualStudio2017InstallerProjects">Microsoft Visual Studio Installer Projects</a> extension. While it’s not provided with a default installation of Visual Studio 2019, it’s an official Microsoft extension. Once you’ve installed it, you’ll be able to create any of the following projects:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/setup-projects-list.jpg" alt="Setup Project Templates screenshot: Setup Project, Web Setup Projet, Merge Module Project, Setup Wizard"></p>
<p>To create an installer, you can create a new <strong>Setup Project</strong>. The build output from this project will be your <code>.msi</code> installer. The setup project has a few different views, which you can use to configure what the installer needs to accomplish. These views can be accessed by right-clicking on the project in the Solution Explorer and expanding View from the context menu:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/installer-views.jpg" alt="Setup Project Views screenshot"></p>
<h3 id="configuring-the-installation-file-system">Configuring the Installation File System</h3>
<p>To configure what files need to be installed, you can use the <strong>File System</strong> view, which provides a UI with some folders added to it:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/file-system-view.jpg" alt="File System View screenshot: Application Folder, User’s Desktop, User’s Programs Menu"></p>
<p>Here, clicking on any folder on the left shows its contents over on the right. It also populates the <strong>Properties Window</strong> with the information about the folder:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/application-folder-properties-window.jpg" alt="Application Folder Properties screenshot: AlwaysCreate, Condition, DefaultLocation, Property, Transitive"></p>
<p>In the above example, we can see that the <strong>Application Folder</strong> is being output to a folder inside <code>C:\Program Files (x86)</code>. You can add any folders you want to the file system by right-clicking on the file system to open the Special Folders context menu:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/special-folder-context-menu.jpg" alt="Special Folder Context Menu screenshot"></p>
<p>Some default folders are shown here for convenience. But let’s say we wanted to make some files get added to the <code>C:\ProgramData</code> folder. To do this, select “Custom Folder” and give it a name. Then, in the <strong>Properties Window</strong>, set the value of <code>DefaultLocation</code> to the correct path:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/program-data-properties-window.jpg" alt="ProgramData Properties screenshot"></p>
<p>From here, you can use the right half of the view to add additional folders within <code>C:\ProgramData\DotNetDemoService</code> based on your needs.</p>
<p>Another thing you’ll likely want to do is put the DLLs from your application into a folder within <code>C:\Program Files (x86)</code>. You can easily do this by mapping the primary build output of your project to the Application Folder in the installer’s file system. To do this, right-click on the <strong>Application Folder</strong>, and add project output:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/add-project-output.jpg" alt="Adding Project Output screenshot"></p>
<p>From there you’ll be prompted to select the project and output type. Select your project, and “Primary Output”:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/add-project-output-dialog.jpg" alt="Add Project Output Dialog screenshot"></p>
<p>This will copy over the DLLs for your project and all of its dependencies.</p>
<h3 id="creating-an-installer-class">Creating an Installer class</h3>
<p>You may be wondering if it’s possible to define custom code to be run during the installation process. It is! For any project targeting .NET Framework 4.8 and under, you can add a class that extends <code>System.Configuration.Install.Installer</code>, and has the <code>[RunInstaller(true)]</code> attribute applied to it. After doing so, you’ll then be able to hook in and override any of the installation lifecycle methods. Taking a look into the definition of the <code>System.Configuration.Install.Installer</code> class reveals the list of overridable lifecycle hook methods you can use to add custom logic to the installation:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> Commit(IDictionary savedState);
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> Install(IDictionary stateSaver);
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> Rollback(IDictionary savedState);
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> Uninstall(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnAfterInstall(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnAfterRollback(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnAfterUninstall(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnBeforeInstall(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnBeforeRollback(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnBeforeUninstall(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnCommitted(IDictionary savedState);
<span style="color:#080;font-weight:bold">protected</span> <span style="color:#080;font-weight:bold">virtual</span> <span style="color:#080;font-weight:bold">void</span> OnCommitting(IDictionary savedState);
</code></pre></div><p>It also defines event handlers for each of these steps as well:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler BeforeInstall;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler Committing;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler AfterUninstall;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler AfterRollback;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler AfterInstall;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler Committed;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler BeforeRollback;
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">event</span> InstallEventHandler BeforeUninstall;
</code></pre></div><p>To add an installer class to the Windows Service project, there’s a helper you can use by right clicking on the designer view of the service and selecting “Add Installer” from the context menu:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/add-installer.jpg" alt="Adding an Installer screenshot"></p>
<p>This will add a new file called <code>ProjectInstaller.cs</code> to your project, which has its own designer view. The designer view has a corresponding <code>ProjectInstaller.Designer.cs</code> file that amends the <code>ProjectInstaller</code> class with the code generated by the designer. You’ll notice that this designer view already defines two objects, <code>serviceInstaller1</code> and <code>serviceProcessInstaller1</code>.</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/installer-designer-view.jpg" alt="Installer Designer View screenshot"></p>
<p>These are special installer classes that will handle all the default installation tasks for your service. <code>serviceInstaller1</code> is of type <code>ServiceInstaller</code> and handles defining the service name and if it should auto start when the machine boots up. <code>serviceProcessInstaller1</code> is of type <code>ServiceProcessInstaller</code> and handles setting up the <strong>Service Logon Account</strong>, which the service will run with once installed. Both of these are already set up and invoked by the designer generated code in <code>ProjectInstaller.Designer.cs</code>.</p>
<p>Since both of these special service installers extend <code>System.Configuration.Install.Installer</code>, you can add custom code to occur at any point of the installation on these as well. The designer view again provides a GUI helper to add this in. Double-clicking on <code>serviceInstaller1</code> will automatically add a new method to <code>ProjectInstaller</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">void</span> serviceInstaller1_AfterInstall(<span style="color:#888;font-weight:bold">object</span> sender, InstallEventArgs e) { }
</code></pre></div><p>It will also put some code into <code>ProjectInstaller.Designer.cs</code> which adds this method to the <code>AfterInstall</code> event of <code>serviceInstaller1</code>.</p>
<h3 id="adding-installer-cli-options">Adding Installer CLI Options</h3>
<p>It’s also possible to add custom properties that you can pass to the installer as command line arguments. These can be done by defining <strong>Custom Actions</strong> on the primary build output of your project. To do this, go to the <strong>Custom Actions</strong> view of the installer project, right click on “Install” and select “Add Custom Action” from the context menu:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/add-custom-action.jpg" alt="Add a Custom Action screenshot"></p>
<p>This will open up a dialog that prompts you to select a file in the installer’s file system to define a custom action for. In this case, we want to define a custom action on the primary build output. This way, the custom CLI options we are about to define will be passed to the project’s installer class.</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/add-custom-action-dialog.jpg" alt="Add Custom Action Dialog screenshot"></p>
<p>After you click “OK”, the primary build output will show up in the <strong>Custom Actions</strong> view. When you click on it, you’ll notice that the properties window has a property called <a href="https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2010/2w2fhwzz(v=vs.100)?redirectedfrom=MSDN">CustomActionData</a>. In short, you can use it to define custom CLI arguments like this:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/custom-action-data.jpg" alt="CustomActionData Definining CLI Arguments screenshot"></p>
<p><code>CustomActionData</code> has its own syntax, so let’s dive deeper into what this actually does. We’re mapping the value of <code>USERNAME</code> and <code>PASSWORD</code> from the installer’s <strong>Properties Collection</strong> to the <code>InstallContext</code> of the installer class of your project under the <code>Username</code> and <code>Password</code> keys, respectively. The square brackets denote that the value is to be taken from the <strong>Properties Collection</strong>, and the quotes allow the value of the property to contain spaces. The forward slash denotes that we are adding a new key to the context. Any command line arguments passed to the installer are added to the <strong>Properties Collection</strong> by default.</p>
<h3 id="using-custom-cli-options-in-the-installer-class">Using Custom CLI Options in the Installer Class</h3>
<p>Now that we have defined our custom action with the CLI arguments, we can go over to the project’s Installer class and access them via the <code>Context</code> property. In this example, we’re using the custom properties to define the Logon account for the service, which needs to be set right before the installation happens. We can use the <code>Install(IDictionary stateSaver)</code> lifecycle hook method for this purpose:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">override</span> <span style="color:#080;font-weight:bold">void</span> Install(IDictionary stateSaver)
{
<span style="color:#888">// If no username or password is specified, fall back to installing the service to run as the SYSTEM account
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (
<span style="color:#888;font-weight:bold">string</span>.IsNullOrEmpty(<span style="color:#080;font-weight:bold">this</span>.Context.Parameters[<span style="color:#d20;background-color:#fff0f0">"Username"</span>])
|| <span style="color:#888;font-weight:bold">string</span>.IsNullOrEmpty(<span style="color:#080;font-weight:bold">this</span>.Context.Parameters[<span style="color:#d20;background-color:#fff0f0">"Password"</span>])
) {
<span style="color:#080;font-weight:bold">this</span>.serviceProcessInstaller1.Account = System.ServiceProcess.ServiceAccount.LocalSystem;
}
<span style="color:#888">// Otherwise, configure the service to run under the specified account.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">else</span>
{
<span style="color:#080;font-weight:bold">this</span>.serviceProcessInstaller1.Username = <span style="color:#080;font-weight:bold">this</span>.Context.Parameters[<span style="color:#d20;background-color:#fff0f0">"Username"</span>];
<span style="color:#080;font-weight:bold">this</span>.serviceProcessInstaller1.Password = <span style="color:#080;font-weight:bold">this</span>.Context.Parameters[<span style="color:#d20;background-color:#fff0f0">"Password"</span>];
}
<span style="color:#888">// Run the base class install after the service has been configured.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">base</span>.Install(stateSaver);
}
</code></pre></div><h3 id="conditionally-installing-files">Conditionally Installing Files</h3>
<p>It’s also possible to make the installer conditionally install files based on a value from the <strong>Properties Collection</strong>. One example of how this can be useful would be swapping in the production or development configuration file based on the value of a command line argument. We don’t need to write any additional code to do this, we just have to add a value for the <a href="https://docs.microsoft.com/en-us/windows/win32/msi/conditional-statement-syntax">Condition</a> property in the <strong>Properties Window</strong> for the file:</p>
<p><img src="/blog/2021/04/automating-windows-service-installation/conditional-file-installation.jpg" alt="Conditionally Installing a File screenshot"></p>
<p>The above condition will make the file <code>settings.production.config</code> be installed only if the <code>DEBUG</code> command line argument is not defined or is set to “false”. Like the custom actions, this property is also sourced from the <strong>Properties Collection</strong>.</p>
<h3 id="conclusion">Conclusion</h3>
<p>And that’s it! I found that having a dedicated <code>.msi</code> installer was handy for making the setup of my Windows Service completely hands-free. While some of the features you need might seem buried within context menus, the flexibility of having the installer handle the service setup is well worth the effort.</p>
<p>Have any questions? Feel free to leave a comment!</p>
Demonstrating the QuickBooks Desktop SDKhttps://www.endpointdev.com/blog/2020/12/demonstrating-quickbooks-desktop-sdk/2020-12-04T00:00:00+00:00Daniel Gomm
<p><img src="/blog/2020/12/demonstrating-quickbooks-desktop-sdk/rock-arch.jpg" alt="Rock arch">
<a href="https://unsplash.com/photos/KWvSPOQvAaw">Photo</a> by <a href="https://unsplash.com/@clarissemeyer">Clarisse Meyer</a></p>
<p>Is your client or company thinking about switching to <a href="https://quickbooks.intuit.com/">QuickBooks</a>? If so, you might be discovering that migrating your existing financial and sales data out of your old system and into QuickBooks is both time consuming and tedious. You might even have an existing ecommerce site or database with tons of data and no clear way of getting the orders into QuickBooks without manual entry.</p>
<p>Recently I was tasked with solving this problem. Our client needed to migrate data from an existing MySQL database into QuickBooks, and automatically add orders from our ecommerce site directly into QuickBooks going forward.</p>
<p>In this article I’ll go over how to use the <strong>QuickBooks Desktop SDK</strong> (also referred to as QBFC for “QuickBooks Foundation Classes” in the API documentation) to send and receive data from QuickBooks.</p>
<h3 id="quickbooks-primer-for-developers">QuickBooks primer for developers</h3>
<p>For the uninitiated, QuickBooks is an accounting software made by <a href="https://www.intuit.com/">Intuit</a>. It can be used to manage lots of data, including lists of customers, inventory items, sales orders, and invoices. All of this data is stored in a “company file”, which is a file with a .qbw extension that uses a proprietary data format. This file gets created when setting up QuickBooks for the first time, and may be served to multiple machines across a network depending on the “open mode”.</p>
<p>The open mode determines how the company file can be accessed. In <strong>multi-user mode</strong>, there is one company file stored in a central location on the network which all users can access. You can use multi-user mode if your license supports multiple users, otherwise you will only be able to use <strong>single-user mode</strong>. In single-user mode, only one user can access the company file at a time.</p>
<p>There are two methods of communicating with QuickBooks: The QuickBooks Desktop SDK, which I mentioned earlier, and the <strong>QuickBooks Web Connector</strong>. Both use <strong>qbXML</strong>, an XML data format that is used to communicate with QuickBooks, and defines tags for each type of request it can process.</p>
<p>The QuickBooks Web Connector is a utility that comes included with QuickBooks and can send/receive SOAP messages from a web server. In order to set up the web connector, you need to make a web service that implements the SOAP methods specified in the documentation (using, for example, ASP.NET). The web connector is configured by the user to check in with your web service at regular intervals, during which the web service can respond with qbXML requests for QuickBooks to process and respond back with.</p>
<p>The QuickBooks Desktop SDK is a Windows COM-based library that allows you to write code that can communicate directly with a local installation of QuickBooks. In order to use the SDK, you’ll need to make an application that runs on a machine that has QuickBooks installed locally, and is on the same local network where the company file is being served. It can be used to send qbXML requests to QuickBooks at any time, and provides a library of foundation classes that abstract away qbXML into an object-oriented API. For migrating data into a new installation of QuickBooks, this SDK is the best solution.</p>
<h3 id="getting-started">Getting started</h3>
<p>To get started, you’ll need to download the latest version of the <a href="https://developer.intuit.com/app/developer/qbdesktop/docs/get-started/download-and-install-the-sdk">QuickBooks Desktop C# SDK</a>, and Visual Studio on your computer. In order to download the SDK, you’ll also need to create a developer account with Intuit.</p>
<p>After the installer is done, the SDK is located in <code>C:\Program Files (x86)\Intuit\IDN</code>. Included in this folder are the assemblies for the SDK, and a local copy of the documentation. You can navigate to this documentation in your browser by going to the following URL:</p>
<p><a href="http://localhost:2211/QBSDK13.0/doc/html/GettingStarted.html">http://localhost:2211/QBSDK13.0/doc/html/GettingStarted.html</a></p>
<p>To add the SDK to your project in Visual Studio, right click on <code>References -> Add Reference</code>, and in the COM tab scroll down to qbFC13 1.0 Type Library, and click add. This should already be in the list once you’ve installed the SDK.</p>
<p><img src="/blog/2020/12/demonstrating-quickbooks-desktop-sdk/adding-reference.png" alt="Adding the reference to your project"></p>
<h3 id="opening-a-session-with-quickbooks">Opening a session with QuickBooks</h3>
<p>In order to start communicating with QuickBooks, you’ll have to open a new session using the <code>QBSessionManager</code> class provided in the SDK. This is the class you will use to send to and receive data from QuickBooks. Before you write any code, make sure that you have a copy of QuickBooks installed on your machine, and that it’s currently running. The below code snippet shows how to connect to QuickBooks via the <code>QBSessionManager</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cpp" data-lang="cpp"><span style="color:#888">// Open a new connection to QuickBooks within a try-catch. An exception
</span><span style="color:#888">// will be thrown if the connection fails.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">try</span>
{
<span style="color:#888">// Create a new QBSessionManager. This object is the entry point to
</span><span style="color:#888"></span> <span style="color:#888">// interfacing with QuickBooks.
</span><span style="color:#888"></span> QBSessionManager sessMgr = <span style="color:#080;font-weight:bold">new</span> QBSessionManager();
<span style="color:#888">// Opens a new connection, which needs to happen before starting
</span><span style="color:#888"></span> <span style="color:#888">// a session. The second argument is the application name, which
</span><span style="color:#888"></span> <span style="color:#888">// QuickBooks will use to identify your application in the future.
</span><span style="color:#888"></span> sessMgr.OpenConnection(<span style="color:#d20;background-color:#fff0f0">""</span>, <span style="color:#d20;background-color:#fff0f0">"QuickBooks Web Api"</span>);
sessMgr.BeginSession(<span style="color:#d20;background-color:#fff0f0">""</span>, ENOpenMode.omDontCare);
<span style="color:#080;font-weight:bold">return</span> <span style="color:#d20;background-color:#fff0f0">"Connected"</span>;
}
<span style="color:#080;font-weight:bold">catch</span> (Exception ex)
{
<span style="color:#080;font-weight:bold">return</span> ex.Message;
}
</code></pre></div><p>Now, make sure you aren’t running Visual Studio as administrator, then build and run the project. QuickBooks and your application need to be running on the same user level, otherwise you’ll receive the error message “Could not start QuickBooks.”</p>
<p>Once you’ve run your application, a popup should appear from QuickBooks, asking you if you want to allow this application to access your company file. Select “Yes, always; allow access even if QuickBooks is not running”, then click “Continue.”</p>
<p><img src="/blog/2020/12/demonstrating-quickbooks-desktop-sdk/verifying-the-application.png" alt="Verifying the application with QuickBooks"></p>
<p>Now, when you run the code example, it should return “Connected.” QuickBooks will make you verify your application the first time it tries to connect. In the sample, we’ve put “QuickBooks Web Api” as the application name. If you change this later, you’ll be prompted to verify your application again.</p>
<h3 id="communicating-with-quickbooks">Communicating with QuickBooks</h3>
<p>Now that you can successfully open a connection and start a session, we’ll look at some code to send requests to QuickBooks. We’ll again be using the <code>QBSessionManager</code> class from the SDK to accomplish this. The key methods to be aware of here are:</p>
<ul>
<li><code>IMsgSetRequest QBSessionManager::CreateMsgSetRequest(string, short, short)</code></li>
<li><code>IMsgSetResponse QBSessionManager::DoRequests(IMsgSetRequest)</code></li>
</ul>
<p>The <code>QBSessionManager</code> sends a set of requests, contained in an SDK object of type <code>IMsgSetRequest</code>, to QuickBooks, which processes each of these requests and responds to each. All the responses are returned in another SDK object of type <code>IMsgSetResponse</code>.</p>
<p>The <code>IMsgSetRequest</code> object contains methods to create each type of request. For example, if you wanted to get a list of all customers from QuickBooks, you could use the <code>IMsgSetRequest.AppendCustomerQueryRq()</code> method, which creates a new <code>ICustomerQuery</code> request object, appends it to the message set, and returns the request object. Try creating a new <code>IMsgSetRequest</code> object in Visual Studio and typing <code>.</code> to see a list of all the Append methods available. There’s one for each request listed in the API reference!</p>
<h3 id="sending-a-request-to-quickbooks">Sending a request to QuickBooks</h3>
<p>Next, we’ll write some code that gets all the customers in the QuickBooks company file. To do this, we’ll create an <code>ICustomerQuery</code> object, add it to the request message set, and then parse the response to gain a list of the customers.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cpp" data-lang="cpp"><span style="color:#888">// Create a QBSessionManager
</span><span style="color:#888"></span>QBSessionManager sessMgr = <span style="color:#080;font-weight:bold">new</span> QBSessionManager();
<span style="color:#888">// Put your code in a try-catch, as the session manager will throw an
</span><span style="color:#888">// exception if an error occurs while sending the request or opening a
</span><span style="color:#888">// connection to QuickBooks.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">try</span>
{
<span style="color:#888">// Connect to QuickBooks and open a new session using the open mode
</span><span style="color:#888"></span> <span style="color:#888">// currently in use by your local QuickBooks installation.
</span><span style="color:#888"></span> sessMgr.OpenConnection(<span style="color:#d20;background-color:#fff0f0">""</span>, <span style="color:#d20;background-color:#fff0f0">"QuickBooks Web Api"</span>);
sessMgr.BeginSession(<span style="color:#d20;background-color:#fff0f0">""</span>, ENOpenMode.omDontCare);
<span style="color:#888">// Create a request message set, which will contain the customer
</span><span style="color:#888"></span> <span style="color:#888">// query. The arguments specify the country (should match your
</span><span style="color:#888"></span> <span style="color:#888">// QuickBooks version) and qbXML version.
</span><span style="color:#888"></span> IMsgSetRequest requestMessageSet = sessMgr.CreateMsgSetRequest(<span style="color:#d20;background-color:#fff0f0">"US"</span>, <span style="color:#00d;font-weight:bold">13</span>, <span style="color:#00d;font-weight:bold">0</span>);
<span style="color:#888">// Create the customer query and add to the request message set. The
</span><span style="color:#888"></span> <span style="color:#888">// append method adds the request to the set, and returns the new
</span><span style="color:#888"></span> <span style="color:#888">// request.
</span><span style="color:#888"></span> ICustomerQuery customerQuery = requestMessageSet.AppendCustomerQueryRq();
<span style="color:#888">// Add a filter to the request that limits the number of items in
</span><span style="color:#888"></span> <span style="color:#888">// the response to 50.
</span><span style="color:#888"></span> customerQuery.ORCustomerListQuery
.CustomerListFilter
.MaxReturned
.SetValue(<span style="color:#00d;font-weight:bold">50</span>);
<span style="color:#888">// Execute all requests in the session manager’s request message
</span><span style="color:#888"></span> <span style="color:#888">// set. The response list contains the responses for each request
</span><span style="color:#888"></span> <span style="color:#888">// sent to QuickBooks, in the order they were sent.
</span><span style="color:#888"></span> IMsgSetResponse resp = sessMgr.DoRequests(requestMessageSet);
IResponseList respList = resp.ResponseList;
<span style="color:#888">// Since we only made one request, our data is in the first (and
</span><span style="color:#888"></span> <span style="color:#888">// only) item in the response list.
</span><span style="color:#888"></span> IResponse curResp = respList.GetAt(<span style="color:#00d;font-weight:bold">0</span>);
<span style="color:#888">// Make sure response code is not less than 0 (which would denote an
</span><span style="color:#888"></span> <span style="color:#888">// error).
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">if</span> (curResp.StatusCode >= <span style="color:#00d;font-weight:bold">0</span>) {
<span style="color:#888">// Get the customer list from the response Detail property
</span><span style="color:#888"></span> <span style="color:#888">// (see OSR) and cast to the expected type.
</span><span style="color:#888"></span> ICustomerRetList custList = (ICustomerRetList)curResp.Detail;
<span style="color:#888">// Iterate through all customers and process.
</span><span style="color:#888"></span> <span style="color:#080;font-weight:bold">for</span> (<span style="color:#888;font-weight:bold">int</span> i = <span style="color:#00d;font-weight:bold">0</span>; i < custList.Count; i++)
{
ICustomerRet cust = custList.GetAt(i);
<span style="color:#888">// < Process the response data >
</span><span style="color:#888"></span> }
}
<span style="color:#080;font-weight:bold">else</span> {
<span style="color:#888">// < Handle error response >
</span><span style="color:#888"></span> }
}
<span style="color:#888">// Catch any exceptions that occur and report them in the response.
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">catch</span> (Exception ex)
{
<span style="color:#888">// < Handle the exception here >
</span><span style="color:#888"></span>}
<span style="color:#888">// Finally close connection & session no matter what happens.
</span><span style="color:#888"></span>finally
{
sessMgr.EndSession();
sessMgr.CloseConnection();
}
</code></pre></div><p>There are a few things to note here. First, the response list type <code>IResponseList</code> is defined by the SDK and doesn’t implement IEnumerable or IQueryable; we have to use the <code>Count</code> property and the <code>GetAt(int)</code> method to get data from it. All list types defined by the SDK are structured in this manner.</p>
<p>Second, each <code>IResponse</code> contains a <code>Details</code> property which has the data. The type of the <code>Details</code> property differs depending on which request was sent. In this case, its type is <code>ICustomerRetList</code>. When getting the response, you’ll need to cast the Details property to the expected type, as it’s defined as the base SDK object type, <code>IQBBase</code>, in the <code>IResponse</code> interface:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cpp" data-lang="cpp">ICustomerRetList custList = (ICustomerRetList)curResp.Detail;
</code></pre></div><p>Additionally, you can get this type programmatically from the response using the <code>IResponse.Type</code> property. Refer to the online documentation for the type of the response details received for each type of request.</p>
<p>The second argument passed to the <code>QBSessionManager.BeginSession()</code> method is the open mode, whose possible values are defined by the <code>ENOpenMode</code> enum. This open mode determines how users can access the company file, and has values for multi-user mode, single-user mode, and a “don’t care” mode, which defaults to whatever mode the local QuickBooks is currently using.</p>
<p>After creating the customer query, this next line adds a filter to the request:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-cpp" data-lang="cpp">customerQuery.ORCustomerListQuery.CustomerListFilter.MaxReturned.SetValue(<span style="color:#00d;font-weight:bold">50</span>);
</code></pre></div><p>In this case we’re limiting the amount of customers in the response to 50. Each request is a unique object with different properties, but many of the requests that <em>receive</em> data from QuickBooks provide a filter object that can be used to control the data that’s included in the response.</p>
<h3 id="conclusion">Conclusion</h3>
<p>The QuickBooks Desktop SDK makes it simple to move data around between your application and your company’s QuickBooks installation. If you end up doing extensive work with the SDK, I recommend wrapping the <code>QBSessionManager</code> in a new class that can convert the responses into a <code>List<IResponse></code>. This way, you can easily get any response’s <code>Detail</code> property by its type in a single line using a LINQ query.</p>
<p>If you have any questions about the SDK, feel free to leave a comment!</p>
<h3 id="further-reading">Further reading</h3>
<ul>
<li><a href="https://www.youtube.com/watch?v=3AjM5ocTgDY">Official Intuit QuickBooks Desktop SDK Training Course</a></li>
<li><a href="https://developer.intuit.com/app/developer/qbdesktop/docs/api-reference/qbdesktop">QuickBooks SDK Online API Documentation</a></li>
</ul>
.NET 5 will be released at .NET Conf 2020https://www.endpointdev.com/blog/2020/11/dotnet-5-released-net-conf-2020/2020-11-04T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2020/11/dotnet-5-released-net-conf-2020/dotnet-5-platform.png" alt=".NET 5 Platform"></p>
<p><a href="https://devblogs.microsoft.com/dotnet/introducing-net-5/">.NET — A unified platform</a> by Microsoft</p>
<p>Last year, at .NET Conf 2019, Microsoft announced that the new .NET 5 will have a base class library that will allow creation of any type of application for any platform — Windows, Linux, Android, iOS, and IoT. And that’s finally about to happen: .NET 5 will be launched at <a href="https://www.dotnetconf.net/">.NET Conf 2020</a>, starting November 10th!</p>
<p>According to Microsoft, <a href="https://devblogs.microsoft.com/dotnet/introducing-net-5/">.NET 5</a> will take the best of .NET Core, .NET Framework, Xamarin, and Mono and merge them into one framework, offering the same experience for all developers, regardless of the type of application or platform targeted. It will also include two different compiler models: just-in-time (JIT, prepared for client-server and desktop apps) and static compilation with ahead-of-time (AOT, optimized to decrease startup times, ideal for mobile and IoT devices).</p>
<p>Some of the key features will be:</p>
<h3 id="windows-desktop-development-wpfwindows-formsuwp">Windows Desktop development (WPF/Windows Forms/UWP)</h3>
<p>This was already a part of the current .NET Core 3 release, and it will stay while getting some updates like a Chromium-based WebView control, improvements to the visual designer, and customizable task dialogs. It will also include all the latest features of C# 8.</p>
<h3 id="full-stack-web-development-cblazor">Full-stack web development (C#/Blazor)</h3>
<p>This feature is also present on the current .NET Core 3 version, and will be updated for this release. With Blazor, we can write full-stack web applications only using C#, removing the need to use a separate language for the frontend. While it’s still a discussed feature, the advantage of using it will depend on the type of web application we’re working on. But it’s there and it will be fully supported on .NET 5.</p>
<h3 id="c-8">C# 8</h3>
<p>This version tries to reduce the well-known (by all of us!) null reference exceptions as a source of program failures by <a href="https://docs.microsoft.com/en-us/archive/msdn-magazine/2018/february/essential-net-csharp-8-0-and-nullable-reference-types">introducing a nullability modifier for nullable reference types</a>. Other main improvements include asynchronous streams, default interface methods and pattern matching enhancements. A full reference of all the new features can be found <a href="https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-8">here</a>.</p>
<h3 id="updates-to-entity-framework">Updates to Entity Framework</h3>
<p>EF will now integrate with the new C# 8, allowing us to consume the query results as <a href="https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-8#asynchronous-streams">asynchronous streams</a>, and automatically mapping the new non-nullable types to non-nullable fields on the database, among other improvements.</p>
<h3 id="updates-to-mlnet">Updates to ML.NET</h3>
<p>.NET 5 will also include the latest version of <a href="https://dotnet.microsoft.com/learn/ml-dotnet/what-is-mldotnet">ML.NET</a>, a free, open-source, and cross-platform machine learning framework for .NET, with some key features like the <code>DatabaseLoader</code> class, which allows loading the data from any relational database with a connection string, and improvements to the object detection capabilities.</p>
<h3 id="and-more">And more</h3>
<p>And of course, everything that was already on .NET Core 3 will be a part of .NET 5: multi-platform mobile development, Azure Cloud access tools, gaming development with Unity and traditional ASP.NET development, etc.</p>
<p>The long-term support (LTS) version of .NET Core is aimed to be launched in 2021, and it will be called .NET 6, hopefully recollecting everything that might be improved based on the feedback for this General Availability (GA) release.</p>
<p>The .NET development team at End Point will be attending the main talks that will happen at the .NET Conf 2020. On Day 1, the focus will be put on this new release, so stay tuned for more to come!</p>
Decreasing your website load timehttps://www.endpointdev.com/blog/2020/01/decreasing-website-load-time/2020-01-07T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2020/01/decreasing-website-load-time/mobile-desktop-browsing.jpg" alt="Decreasing our website load time" /> <a href="https://www.flickr.com/photos/johanl/6798184016/">Photo</a> by <a href="https://www.flickr.com/photos/johanl/">Johan Larsson</a>, used under <a href="https://creativecommons.org/licenses/by/2.0/">CC BY 2.0</a></p>
<p>We live in a competitive world, and the web is no different. Improving latency issues is crucial to any Search Engine Optimization (SEO) strategy, increasing the website’s ranking and organic traffic (visitors from search engines) as a result.</p>
<p>There are many factors that can lead to a faster response time, including optimization of your hosting plan, server proximity to your main traffic source, or utilization of a Content Distribution Network (CDN) if you are expecting visitors on an international level. Some of these solutions and many others can be implemented with only a couple hours of coding.</p>
<h3 id="inline-styles-and-scripts-for-the-topmost-content">Inline styles and scripts for the topmost content</h3>
<p>Nobody enjoys waiting for long load times. When opening a Google search link, being met with a blank page or a loading GIF for several seconds can seem agonizing. That’s why optimizing the initial rendering of your page is crucial.</p>
<p>The content that immediately appears to the user without the need to scroll down is referred to as “above-the-fold”. This is where your optimization efforts should be aimed. So here’s a plan to load and display as quickly as possible:</p>
<ul>
<li>
<p>First, differentiate the critical styles and scripts you need to render the topmost content, and separate them from the rest of our stylesheet and external script references.</p>
</li>
<li>
<p>Then, <a href="https://www.imperva.com/learn/performance/minification/">minify</a> the separated <a href="https://csscompressor.com/">styles</a> and <a href="https://jscompress.com/">scripts</a>, and insert them directly on our page template, right before the closing <code></head></code> tag.</p>
</li>
<li>
<p>Finally, take the stylesheet and scripts link references from the <code><head></code> tag (where it’s usually located) and move them to the end of the above-the-fold content.</p>
</li>
</ul>
<p>Now, the user won’t have to wait until all references are loaded before seeing content. <b>Tip</b>: Remember to use the <a href="https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script#attr-async">async</a> tag on scripts whenever possible.</p>
<p><strong>example.html:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">head</span>>
<<span style="color:#b06;font-weight:bold">style</span>>{<span style="color:#a61717;background-color:#e3d2d2">above-the-fold</span> <span style="color:#a61717;background-color:#e3d2d2">minified</span> <span style="color:#a61717;background-color:#e3d2d2">inline</span> <span style="color:#a61717;background-color:#e3d2d2">styles</span> <span style="color:#a61717;background-color:#e3d2d2">goes</span> <span style="color:#a61717;background-color:#e3d2d2">here</span>}</<span style="color:#b06;font-weight:bold">style</span>>
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">type</span>=<span style="color:#d20;background-color:#fff0f0">"text/javascript"</span>>{above-the-fold critical scripts goes here}</<span style="color:#b06;font-weight:bold">script</span>>
</<span style="color:#b06;font-weight:bold">head</span>>
<<span style="color:#b06;font-weight:bold">body</span>>
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"above-the-fold-content"</span>></<span style="color:#b06;font-weight:bold">div</span>>
<<span style="color:#b06;font-weight:bold">link</span> <span style="color:#369">rel</span>=<span style="color:#d20;background-color:#fff0f0">"stylesheet"</span> <span style="color:#369">href</span>=<span style="color:#d20;background-color:#fff0f0">"{below-the-fold minified stylesheet reference goes here}"</span> />
<<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">async</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"{below-the-fold minified javascript reference goes here}"</span> />
<<span style="color:#b06;font-weight:bold">div</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"below-the-fold-content"</span>></<span style="color:#b06;font-weight:bold">div</span>>
</<span style="color:#b06;font-weight:bold">body</span>>
</code></pre></div><h3 id="deferred-loading-of-ads">Deferred loading of ads</h3>
<p>If you’re monetizing your website through Google AdSense or another ad agency that uses scripts to load ads, consider loading ads after the content is fully rendered. This may have a small impact on your revenue, but will improve the user’s experience while optimizing the load speed.</p>
<p>Although there are several ways to achieve this, a technique I have successfully used on many websites is removing all of the script references to Google AdSense until your page is fully loaded. A short delay can be added in order to allow some browsing time before showing ads.</p>
<p>Remove script references, the comment, and extra spaces from your original ad code, to convert it from something like this…</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">script</span> <span style="color:#369">async</span> <span style="color:#369">src</span>=<span style="color:#d20;background-color:#fff0f0">"https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"</span>></<span style="color:#b06;font-weight:bold">script</span>>
<span style="color:#888"><!-- Your ad name --></span>
<<span style="color:#b06;font-weight:bold">ins</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"adsbygoogle"</span>
<span style="color:#369">style</span>=<span style="color:#d20;background-color:#fff0f0">"display:inline-block;width:728px;height:90px"</span>
<span style="color:#369">data-ad-client</span>=<span style="color:#d20;background-color:#fff0f0">"ca-pub-XXXXXXXXXXXXXXXXX"</span>
<span style="color:#369">data-ad-slot</span>=<span style="color:#d20;background-color:#fff0f0">"XXXXXXXXX"</span>></<span style="color:#b06;font-weight:bold">ins</span>>
<<span style="color:#b06;font-weight:bold">script</span>>
(adsbygoogle = <span style="color:#038">window</span>.adsbygoogle || []).push({});
</<span style="color:#b06;font-weight:bold">script</span>>
</code></pre></div><p>… to something like this:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-html" data-lang="html"><<span style="color:#b06;font-weight:bold">ins</span> <span style="color:#369">class</span>=<span style="color:#d20;background-color:#fff0f0">"adsbygoogle"</span> <span style="color:#369">style</span>=<span style="color:#d20;background-color:#fff0f0">"display:inline-block;width:728px;height:90px"</span> <span style="color:#369">data-ad-client</span>=<span style="color:#d20;background-color:#fff0f0">"ca-pub-XXXXXXXXXXXXXXXXX"</span> <span style="color:#369">data-ad-slot</span>=<span style="color:#d20;background-color:#fff0f0">"XXXXXXXXX"</span>></<span style="color:#b06;font-weight:bold">ins</span>>
</code></pre></div><p>A lot shorter, isn’t it? This will create an empty slot in which the ad will be displayed after the page is fully rendered. To accomplish that, a new script like the one below must be added (assuming jQuery is present on the website):</p>
<p><strong>async-ads.js:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript"><span style="color:#888">// Create a script reference
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">function</span> addScript(src, <span style="color:#080;font-weight:bold">async</span>, callback) {
<span style="color:#080;font-weight:bold">var</span> js = <span style="color:#038">document</span>.createElement(<span style="color:#d20;background-color:#fff0f0">"script"</span>);
js.type = <span style="color:#d20;background-color:#fff0f0">"text/javascript"</span>;
<span style="color:#080;font-weight:bold">if</span> (<span style="color:#080;font-weight:bold">async</span>)
js.<span style="color:#080;font-weight:bold">async</span> = <span style="color:#080;font-weight:bold">true</span>;
<span style="color:#080;font-weight:bold">if</span> (callback)
js.onload = callback;
js.src = src;
<span style="color:#038">document</span>.body.appendChild(js);
}
<span style="color:#888">// Called when document is ready
</span><span style="color:#888"></span>$(<span style="color:#038">document</span>).ready(<span style="color:#080;font-weight:bold">function</span>() {
<span style="color:#888">// Wait for one second to ensure the user started browsing
</span><span style="color:#888"></span> setTimeout(<span style="color:#080;font-weight:bold">function</span>() {
(adsbygoogle = <span style="color:#038">window</span>.adsbygoogle || []);
$(<span style="color:#d20;background-color:#fff0f0">"ins.adsbygoogle"</span>).each(<span style="color:#080;font-weight:bold">function</span>() {
$(<span style="color:#d20;background-color:#fff0f0">"<script>(adsbygoogle = window.adsbygoogle || []).push({})</script>"</span>).insertAfter($(<span style="color:#080;font-weight:bold">this</span>));
});
addScript(<span style="color:#d20;background-color:#fff0f0">"https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"</span>, <span style="color:#080;font-weight:bold">true</span>);
}, <span style="color:#00d;font-weight:bold">1000</span>);
});
</code></pre></div><p>This code will wait for one second once the document is ready, and then leave instructions for Google to push a new ad for each slot. Finally, the AdSense external script will be loaded so that Google will read the instructions and start filling all the slots with ads.</p>
<p><b>Tip</b>: Enabling balancing from your AdSense dashboard may improve the average load speed as well as the user’s experience since ads will not be shown when the expected revenue is deprecable. And if you’re still on the fence about showing fewer ads, <a href="https://fatstacksblog.com/adsense-ad-balance-experiment/">try out an experiment</a> like I did. A balance of 50% worked well in my case, but the right balance will depend on your niche and website characteristics.</p>
<h3 id="lazy-load-for-images">Lazy load for images</h3>
<p>Because the user will most likely spend the majority of the visit reading above-the-fold content (and may even leave before scrolling at all), loading all images from content below-the-fold at first is impractical. Implementing a custom lazy-loading script (also referred to as deferred-loading or loading-on-scroll) for images can be an easy process. Even though changes to the backend would be likely, the concept of this approach is simple:</p>
<ul>
<li>
<p>Replacing the <code>src</code> attributes from all images that will have lazy loading with a custom attribute such as <code>data-src</code> (this part will probably require backend changes) and set a custom class for them, like <code>lazy</code>.</p>
</li>
<li>
<p>Creating a script that will copy the <code>data-src</code> content into the <code>src</code> attribute as we scroll through the page.</p>
</li>
</ul>
<p><strong>lazy-load.js:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">;(<span style="color:#080;font-weight:bold">function</span>($) {
$.fn.lazy = <span style="color:#080;font-weight:bold">function</span>(threshold, callback) {
<span style="color:#080;font-weight:bold">var</span> $w = $(<span style="color:#038">window</span>),
th = threshold || <span style="color:#00d;font-weight:bold">0</span>,
attrib = <span style="color:#d20;background-color:#fff0f0">"data-src"</span>,
images = <span style="color:#080;font-weight:bold">this</span>,
loaded;
<span style="color:#080;font-weight:bold">this</span>.one(<span style="color:#d20;background-color:#fff0f0">"lazy"</span>, <span style="color:#080;font-weight:bold">function</span>() {
<span style="color:#080;font-weight:bold">var</span> source = <span style="color:#080;font-weight:bold">this</span>.getAttribute(attrib);
source = source || <span style="color:#080;font-weight:bold">this</span>.getAttribute(<span style="color:#d20;background-color:#fff0f0">"data-src"</span>);
<span style="color:#080;font-weight:bold">if</span> (source) {
<span style="color:#080;font-weight:bold">this</span>.setAttribute(<span style="color:#d20;background-color:#fff0f0">"src"</span>, source);
<span style="color:#080;font-weight:bold">if</span> (<span style="color:#080;font-weight:bold">typeof</span> callback === <span style="color:#d20;background-color:#fff0f0">"function"</span>) callback.call(<span style="color:#080;font-weight:bold">this</span>);
}
});
<span style="color:#080;font-weight:bold">function</span> lazy() {
<span style="color:#080;font-weight:bold">var</span> inview = images.filter(<span style="color:#080;font-weight:bold">function</span>() {
<span style="color:#080;font-weight:bold">var</span> $e = $(<span style="color:#080;font-weight:bold">this</span>);
<span style="color:#080;font-weight:bold">if</span> ($e.is(<span style="color:#d20;background-color:#fff0f0">":hidden"</span>)) <span style="color:#080;font-weight:bold">return</span>;
<span style="color:#080;font-weight:bold">var</span> wt = $w.scrollTop(),
wb = wt + $w.height(),
et = $e.offset().top,
eb = et + $e.height();
<span style="color:#080;font-weight:bold">return</span> eb >= wt - th && et <= wb + th;
});
loaded = inview.trigger(<span style="color:#d20;background-color:#fff0f0">"lazy"</span>);
images = images.not(loaded);
}
$w.scroll(lazy);
$w.resize(lazy);
lazy();
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">this</span>;
};
})(<span style="color:#038">window</span>.jQuery);
$(<span style="color:#038">document</span>).ready(<span style="color:#080;font-weight:bold">function</span>() {
$(<span style="color:#d20;background-color:#fff0f0">'.lazy'</span>).each(<span style="color:#080;font-weight:bold">function</span> () {
$(<span style="color:#080;font-weight:bold">this</span>).lazy(<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#080;font-weight:bold">function</span>() {
$(<span style="color:#080;font-weight:bold">this</span>).load(<span style="color:#080;font-weight:bold">function</span>() {
<span style="color:#080;font-weight:bold">this</span>.style.opacity = <span style="color:#00d;font-weight:bold">1</span>;
});
});
});
<span style="color:#888">// Set the correct attribute when printing
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">var</span> beforePrint = <span style="color:#080;font-weight:bold">function</span>() {
$(<span style="color:#d20;background-color:#fff0f0">"img.lazy"</span>).each(<span style="color:#080;font-weight:bold">function</span>() {
$(<span style="color:#080;font-weight:bold">this</span>).trigger(<span style="color:#d20;background-color:#fff0f0">"lazy"</span>);
<span style="color:#080;font-weight:bold">this</span>.style.opacity = <span style="color:#00d;font-weight:bold">1</span>;
});
};
<span style="color:#080;font-weight:bold">if</span> (<span style="color:#038">window</span>.matchMedia) {
<span style="color:#080;font-weight:bold">var</span> mediaQueryList = <span style="color:#038">window</span>.matchMedia(<span style="color:#d20;background-color:#fff0f0">'print'</span>);
mediaQueryList.addListener(<span style="color:#080;font-weight:bold">function</span>(mql) {
<span style="color:#080;font-weight:bold">if</span> (mql.matches)
beforePrint();
});
}
<span style="color:#038">window</span>.onbeforeprint = beforePrint;
</code></pre></div><p>This script will search for all <code><img></code> tags with class <code>lazy</code>, and change the <code>data-src</code> attribute to the <code>src</code> attribute once the image becomes visible due to scrolling. It also includes some additional logic to set the <code>src</code> attribute before printing the page.</p>
<h3 id="server-side-caching">Server-side caching</h3>
<p>Instead of performing all the backend rendering calculations every time, server-side caching allows you to output the same content to the clients over a period of time from a temporary copy of the response. This not only results in a decreased response time but also saves some resources on the server.</p>
<p>There are several ways to enable server-side caching, depending on factors such as the backend language and hosting platform (e.g. Windows/IIS vs. Linux/Apache), among other things. For this example, we will use ASP.NET (C#) since I’m mostly a Windows user.</p>
<p>The best and most efficient way to do this is by adding a declaration in the top of our ASP.NET page:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><%<span style="color:#a61717;background-color:#e3d2d2">@</span> OutputCache Duration=<span style="color:#d20;background-color:#fff0f0">"10"</span> VaryByParam=<span style="color:#d20;background-color:#fff0f0">"id;date"</span> %>
</code></pre></div><p>This declaration is telling the compiler that we want to cache the output from the server for 10 minutes, and we will save different versions based on the <code>id</code> and <code>date</code> URL parameters. So pages like:</p>
<ul>
<li>https://www.your-url.com/cached-page/?id=1&date=2020-01-01</li>
<li>https://www.your-url.com/cached-page/?id=2&date=2020-01-01</li>
<li>https://www.your-url.com/cached-page/?id=2&date=2020-02-01</li>
</ul>
<p>will be saved and then served from different cache copies. If we only set the <code>id</code> parameter as a source for caching, pages with different dates will be served from the same cache source (this can be useful as the <code>date</code> parameter is only evaluated on frontend scripts and ignored in the backend).</p>
<p>There are other configurations in ASP.NET to set our output cache policy. The output can be set to be based on the browser, the request headers, or even custom strings. <a href="https://www.c-sharpcorner.com/UploadFile/chinnasrihari/Asp-Net-mvc-framework-server-side-html-caching-techniques/">This page</a> has more useful information on this subject.</p>
<h3 id="gzip-compression">GZip compression</h3>
<p>GZip compression—when the client supports it—allows compressing the response before sending it over the network. In this way, more than 70% of the bandwidth can be saved when loading the website. Enabling GZip compression for dynamic and static content on a Windows Server with IIS is simple: Just go to the “Compression” section on the IIS Manager and check the options “Enable dynamic/static content compression”.</p>
<p><img src="/blog/2020/01/decreasing-website-load-time/enabling-compression-iis.jpg" alt="Enabling compression in IIS"></p>
<p>However, if you are running an ASP.NET MVC/WebForms website, this won’t be enough. For all backend responses to be compressed before sending them to the client, some custom code will also need to be added to the <code>global.asax</code> file in the website root:</p>
<p><strong>global.asax:</strong></p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><%<span style="color:#a61717;background-color:#e3d2d2">@</span> Application Language=<span style="color:#d20;background-color:#fff0f0">"C#"</span> %>
<script runat=<span style="color:#d20;background-color:#fff0f0">"server"</span>>
<span style="color:#080;font-weight:bold">void</span> Application_PreRequestHandlerExecute(<span style="color:#888;font-weight:bold">object</span> sender, EventArgs e)
{
HttpApplication app = sender <span style="color:#080;font-weight:bold">as</span> HttpApplication;
<span style="color:#888;font-weight:bold">string</span> acceptEncoding = app.Request.Headers[<span style="color:#d20;background-color:#fff0f0">"Accept-Encoding"</span>];
System.IO.Stream prevUncompressedStream = app.Response.Filter;
<span style="color:#080;font-weight:bold">if</span> (app.Context.CurrentHandler == <span style="color:#080;font-weight:bold">null</span>)
<span style="color:#080;font-weight:bold">return</span>;
<span style="color:#080;font-weight:bold">if</span> (!(app.Context.CurrentHandler <span style="color:#080;font-weight:bold">is</span> System.Web.UI.Page ||
app.Context.CurrentHandler.GetType().Name == <span style="color:#d20;background-color:#fff0f0">"SyncSessionlessHandler"</span>) ||
app.Request[<span style="color:#d20;background-color:#fff0f0">"HTTP_X_MICROSOFTAJAX"</span>] != <span style="color:#080;font-weight:bold">null</span>)
<span style="color:#080;font-weight:bold">return</span>;
<span style="color:#080;font-weight:bold">if</span> (acceptEncoding == <span style="color:#080;font-weight:bold">null</span> || acceptEncoding.Length == <span style="color:#00d;font-weight:bold">0</span>)
<span style="color:#080;font-weight:bold">return</span>;
<span style="color:#080;font-weight:bold">if</span> (Request.ServerVariables[<span style="color:#d20;background-color:#fff0f0">"SCRIPT_NAME"</span>].ToLower().Contains(<span style="color:#d20;background-color:#fff0f0">".axd"</span>)) <span style="color:#080;font-weight:bold">return</span>;
<span style="color:#080;font-weight:bold">if</span> (Request.ServerVariables[<span style="color:#d20;background-color:#fff0f0">"SCRIPT_NAME"</span>].ToLower().Contains(<span style="color:#d20;background-color:#fff0f0">".js"</span>)) <span style="color:#080;font-weight:bold">return</span>;
<span style="color:#080;font-weight:bold">if</span> (Request.QueryString.ToString().Contains(<span style="color:#d20;background-color:#fff0f0">"_TSM_HiddenField_"</span>)) <span style="color:#080;font-weight:bold">return</span>;
acceptEncoding = acceptEncoding.ToLower();
<span style="color:#080;font-weight:bold">if</span> (acceptEncoding.Contains(<span style="color:#d20;background-color:#fff0f0">"deflate"</span>) || acceptEncoding == <span style="color:#d20;background-color:#fff0f0">"*"</span>)
{
app.Response.Filter = <span style="color:#080;font-weight:bold">new</span> System.IO.Compression.DeflateStream(prevUncompressedStream,
System.IO.Compression.CompressionMode.Compress);
app.Response.AppendHeader(<span style="color:#d20;background-color:#fff0f0">"Content-Encoding"</span>, <span style="color:#d20;background-color:#fff0f0">"deflate"</span>);
}
<span style="color:#080;font-weight:bold">else</span> <span style="color:#080;font-weight:bold">if</span> (acceptEncoding.Contains(<span style="color:#d20;background-color:#fff0f0">"gzip"</span>))
{
app.Response.Filter = <span style="color:#080;font-weight:bold">new</span> System.IO.Compression.GZipStream(prevUncompressedStream,
System.IO.Compression.CompressionMode.Compress);
app.Response.AppendHeader(<span style="color:#d20;background-color:#fff0f0">"Content-Encoding"</span>, <span style="color:#d20;background-color:#fff0f0">"gzip"</span>);
}
}
</script>
</code></pre></div><p>To make sure our code is working properly, an external tool like <a href="https://www.giftofspeed.com/gzip-test/">this</a> will inform you if GZip is enabled or not.</p>
<p><img src="/blog/2020/01/decreasing-website-load-time/gzip-compression-enabled.jpg" alt="It works!"></p>
<h3 id="summary">Summary</h3>
<p>While there are many ways of decreasing the load time of a website, most are common and expensive. However, with a few minor tweaks, we can offer a better user experience in addition to improve our position in the search engine results. Every bit of optimization counts towards the goal with SEO. Load time is a very important factor (to both the developer and the user), especially on mobile platforms where users expect to get what they want instantly.</p>
<p>The image below is a Google Analytics report from one of my websites where, over several months, I implemented most of these formulas. A month ago, I made the latest change of deferring ad loading, which had an observable impact on the average loading speed of the page:</p>
<p><img src="/blog/2020/01/decreasing-website-load-time/analytics-average-page-load.jpg" alt="Report from Google Analytics"></p>
<p>Do you have any other page load optimization techniques? <b>Leave a comment below!</b></p>
Prepare for .NET Core 3 and .NET 5https://www.endpointdev.com/blog/2019/08/prepare-for-dotnet-core-3-dotnet-5/2019-08-03T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2019/08/prepare-for-dotnet-core-3-dotnet-5/image-0.jpg" alt="Prepare for .NET 5" /> <a href="https://unsplash.com/photos/fPkvU7RDmCo">Photo</a> by Caspar Camille Rubin on Unsplash, edited by Juan Pablo Ventoso</p>
<h3 id="introduction">Introduction</h3>
<p>It’s been a while now since .NET Core is out there: It was released back in June 2016 and it kept growing since then. The main advantages when comparing with .NET Framework is that .NET Core is free, open-source and cross-platform. It also has several <a href="https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-core/" target="_blank">performance improvements</a> that gains up to 600% increase for some particular functions like converting elements to a string, or more than 200% for some LINQ queries, and a general performance boost in application startup.</p>
<p>But it also has drawbacks: Some third party libraries are still not fully supported, so while they can still be used, the compiled results will only be portable to Windows. Also, .NET Core 2.2 (the latest release to date) doesn’t yet have support for Windows Presentation Foundation (WPF) or Windows Forms applications… <strong>But it looks like this is going to change soon.</strong></p>
<h3 id="next-stop-net-core-3">Next stop: .NET Core 3</h3>
<p>In the 2019 Build Conference that took place in May, <a href="https://devblogs.microsoft.com/dotnet/announcing-net-core-3/" target="_blank">.NET Core 3 was announced</a>: It’s expected to be released in November this year, and it will finally include support for Windows desktop development (WPF, UWP and Windows Forms). It will also include Entity Framework 6, since most existing Windows Forms and WPF applications use that framework to access data.</p>
<p><img src="/blog/2019/08/prepare-for-dotnet-core-3-dotnet-5/image-1.jpg" alt=".NET Core 3 features and tools"></p>
<p>.NET Core 3 will also include support for IoT (Internet-of-things) applications, aiming for internet-enabled devices like smart locks or glasses, and will allow AI development through ML.NET (an open-source machine learning framework built for .NET). For those who can’t wait and want to try it out before its release, previews are <a href="https://devblogs.microsoft.com/dotnet/announcing-net-core-3-0-preview-7/" target="_blank">available to download</a>.</p>
<h3 id="game-changer-net-5">Game changer: .NET 5</h3>
<p>In the same conference, on May 7, Microsoft also announced that they are planning to release a new platform for building applications targetting all devices and operative systems in november 2020: <a href="https://devblogs.microsoft.com/dotnet/introducing-net-5/" target="_blank">.NET 5</a>. And while it will be based on .NET Core, the word “core” will be removed because Microsoft is aiming to unify .NET Framework into this new platform as well (that’s why its version number will be 5: to avoid confusion with existing 4.x versions of .NET Framework).</p>
<p><img src="/blog/2019/08/prepare-for-dotnet-core-3-dotnet-5/image-2.jpg" alt=".NET 5 features and tools"></p>
<p>Microsoft defines .NET 5 as a “game changer”: In one hand, we can expect some important features to be improved. So far, Microsoft has stated that cross-platform will be mantained and even improved, and we will have two runtimes to choose (<a href="https://github.com/mono/mono" target="_blank">Mono</a> or <a href="https://github.com/dotnet/coreclr" target="_blank">CoreCLR</a>). Java, Objective-C and Swift interoperability will be also supported for all platforms.</p>
<p>But, on the other hand, <strong>other features will be discontinued</strong>: ASP.NET Web Forms will not be included, so legacy apps with this technology will have to stick with .NET Framework, or migrated to an alternative like the new client-based <a href="https://dotnet.microsoft.com/apps/aspnet/web-apps/client" target="_blank">Blazor</a>, or <a href="/blog/2018/11/whats-the-deal-with-asp-net-core-razor-pages/" target="_blank">Razor Pages</a> (both development frameworks will be fully supported). And on desktop development, we can expect WCF (Windows Communication Foundation) and WWF (Windows Workflow Foundation) to be out of the equation too.</p>
<p>But having .NET 5 built upon .NET Core doesn’t mean that .NET Framework will cease to exist: John Montgomery, Microsoft’s corporate VP in Developer Tools, <a href="https://www.theregister.co.uk/2019/05/16/will_net_5_really_unify_microsoft_development_stack/" target="_blank">stated in an interview</a> that “The big thing (.NET 5) it’s doing is unifying Mono and Xamarin with .NET Core. It is not bringing all the technology from full .NET Framework, which remains the thing that’s going to be in Windows for a long time and will not move forward very much”.</p>
<h3 id="and-beyond">And beyond</h3>
<p>After .NET 5 is released, Microsoft plans to ship a new major release of .NET once a year, as we can see from the schedule roadmap posted on their .NET blog.</p>
<p><img src="/blog/2019/08/prepare-for-dotnet-core-3-dotnet-5/image-3.jpg" alt=".NET schedule roadmap"></p>
<h3 id="conclusion">Conclusion</h3>
<p><b>Microsoft is clearly taking a new direction</b>. They are planning to use .NET 3 as an intermediate step to merge .NET Core architecture with Windows desktop development. And one year after, they are planning to have multiple devices, architectures, operative systems and application types covered with a single unified framework, .NET 5.</p>
<p>Is clear that many existing applications will not have an easy way to be migrated into .NET Core, so we can expect .NET Framework to remain supported for many years. But Microsoft is showing us the path that .NET will take as the main road in the future: To grow steadily based on the community .NET Core has built itself upon.</p>
Mocking asynchronous database calls in .NET Corehttps://www.endpointdev.com/blog/2019/07/mocking-asynchronous-database-calls-net-core/2019-07-16T00:00:00+00:00Juan Pablo Ventoso
<p><img src="/blog/2019/07/mocking-asynchronous-database-calls-net-core/image-0.jpg" alt="Mocking asynchronous database calls in .NET Core" /> <a href="https://flic.kr/p/59cz7W">Photo</a> by <a href="https://www.flickr.com/photos/kapten/">Björn Söderqvist</a>, used under <a href="https://creativecommons.org/licenses/by/2.0/">CC BY 2.0</a></p>
<h3 id="introduction">Introduction</h3>
<p>Whenever we like—or are forced!—to develop using a TDD (test-driven development) approach or a more traditional practice, unit testing should be always a part of our development process. And when it comes to .NET Core, no matter what framework we choose (xUnit, MSTest or other), we will probably need to use a mock library.</p>
<p><strong>What is mocking?</strong> By definition, it’s making an imitation. Objects usually depend on other objects, which could rely on database connections, local files or external APIs. So when we need to test the behavior of a particular object, we will have to isolate its dependencies, replacing those objects with fake versions. And that’s where mocking—and the Moq library—comes into place: it’s a set of classes that allows us to easily create and handle those fake objects.</p>
<p>So, assuming we already have a .NET Core solution with a test project included, all we need to do to add Moq is install the package from the NuGet console in Visual Studio:</p>
<pre tabindex="0"><code>Install-Package Moq
</code></pre><h3 id="mocking-data-with-async-calls-support">Mocking data with async calls support</h3>
<p>One of the first showstoppers I’ve encountered when trying to add unit tests to an existing project was to mock objects that contain asynchronous calls to the database: If we want to run offline (in-memory) tests against a set of code that performs an asynchronous query over a <code>DbSet<T></code>, we’ll have to set up some helpers first.</p>
<p>So for example, let’s suppose we have a simple function called <code>GetUserIDByEmail()</code> that returns the ID of the user that matches an email passed by parameter, and that function uses an asyncronous query to the database to find that user:</p>
<ul>
<li><strong>UserHandler.cs</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> Task<<span style="color:#888;font-weight:bold">int?</span>> GetUserIDByEmail(<span style="color:#888;font-weight:bold">string</span> Email)
{
<span style="color:#888;font-weight:bold">var</span> User = <span style="color:#080;font-weight:bold">await</span> <span style="color:#00d;font-weight:bold">_D</span>bContext.Users.Where(x => x.Email.Equals(Email)).FirstOrDefaultAsync();
<span style="color:#080;font-weight:bold">if</span> (User != <span style="color:#080;font-weight:bold">null</span>)
<span style="color:#080;font-weight:bold">return</span> User.ID;
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">null</span>;
}
</code></pre></div><p>Where _DbContext is a reference to the interface <code>IMockProjectDbContext</code>, defined as:</p>
<ul>
<li><strong>MockProjectDbContext.cs</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">interface</span> IMockProjectDbContext
{
DbSet<User> Users { <span style="color:#080;font-weight:bold">get</span>; <span style="color:#080;font-weight:bold">set</span>; }
}
</code></pre></div><p><code>GetUserIDByEmail()</code> will return a <code>Task<int?></code> that will be created by the <code>FirstOrDefaultAsync()</code> method from Entity Framework. If we want to test this function with a in-memory set of data, the test will fail because our test data won’t support the interfaces needed to make the asynchronous call.</p>
<p>Why? Because the traditional provider for <code>IQueryable</code> and <code>IEnumerable</code> (the interfaces used for traditional sets) doesn’t implement the <code>IAsyncQueryProvider</code> interface needed for the Entity Framework asynchronous extension methods. So what we need to do is create a set of classes that will allow us to mock asynchronous calls to our in-memory lists.</p>
<ul>
<li><strong>TestClasses.cs</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// Async query provider for unit testing
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">internal</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">TestAsyncQueryProvider</span><TEntity> : IAsyncQueryProvider
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> IQueryProvider <span style="color:#00d;font-weight:bold">_</span>inner;
<span style="color:#080;font-weight:bold">internal</span> TestAsyncQueryProvider(IQueryProvider inner)
{
<span style="color:#00d;font-weight:bold">_</span>inner = inner;
}
<span style="color:#080;font-weight:bold">public</span> IQueryable CreateQuery(Expression expression)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> TestAsyncEnumerable<TEntity>(expression);
}
<span style="color:#080;font-weight:bold">public</span> IQueryable<TElement> CreateQuery<TElement>(Expression expression)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> TestAsyncEnumerable<TElement>(expression);
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#888;font-weight:bold">object</span> Execute(Expression expression)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#00d;font-weight:bold">_</span>inner.Execute(expression);
}
<span style="color:#080;font-weight:bold">public</span> TResult Execute<TResult>(Expression expression)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#00d;font-weight:bold">_</span>inner.Execute<TResult>(expression);
}
<span style="color:#080;font-weight:bold">public</span> IAsyncEnumerable<TResult> ExecuteAsync<TResult>(Expression expression)
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> TestAsyncEnumerable<TResult>(expression);
}
<span style="color:#080;font-weight:bold">public</span> Task<TResult> ExecuteAsync<TResult>(Expression expression, CancellationToken cancellationToken)
{
<span style="color:#080;font-weight:bold">return</span> Task.FromResult(Execute<TResult>(expression));
}
}
<span style="color:#888">// Async enumerable for unit testing
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">internal</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">TestAsyncEnumerable</span><T> : EnumerableQuery<T>, IAsyncEnumerable<T>, IQueryable<T>
{
<span style="color:#080;font-weight:bold">public</span> TestAsyncEnumerable(IEnumerable<T> enumerable)
: <span style="color:#080;font-weight:bold">base</span>(enumerable)
{ }
<span style="color:#080;font-weight:bold">public</span> TestAsyncEnumerable(Expression expression)
: <span style="color:#080;font-weight:bold">base</span>(expression)
{ }
<span style="color:#080;font-weight:bold">public</span> IAsyncEnumerator<T> GetEnumerator()
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> TestAsyncEnumerator<T>(<span style="color:#080;font-weight:bold">this</span>.AsEnumerable().GetEnumerator());
}
IQueryProvider IQueryable.Provider
{
<span style="color:#080;font-weight:bold">get</span> { <span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> TestAsyncQueryProvider<T>(<span style="color:#080;font-weight:bold">this</span>); }
}
}
<span style="color:#888">// Async enumerator for unit testing
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">internal</span> <span style="color:#080;font-weight:bold">class</span> <span style="color:#b06;font-weight:bold">TestAsyncEnumerator</span><T> : IAsyncEnumerator<T>
{
<span style="color:#080;font-weight:bold">private</span> <span style="color:#080;font-weight:bold">readonly</span> IEnumerator<T> <span style="color:#00d;font-weight:bold">_</span>inner;
<span style="color:#080;font-weight:bold">public</span> TestAsyncEnumerator(IEnumerator<T> inner)
{
<span style="color:#00d;font-weight:bold">_</span>inner = inner;
}
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">void</span> Dispose()
{
<span style="color:#00d;font-weight:bold">_</span>inner.Dispose();
}
<span style="color:#080;font-weight:bold">public</span> T Current
{
<span style="color:#080;font-weight:bold">get</span>
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#00d;font-weight:bold">_</span>inner.Current;
}
}
<span style="color:#080;font-weight:bold">public</span> Task<<span style="color:#888;font-weight:bold">bool</span>> MoveNext(CancellationToken cancellationToken)
{
<span style="color:#080;font-weight:bold">return</span> Task.FromResult(<span style="color:#00d;font-weight:bold">_</span>inner.MoveNext());
}
}
</code></pre></div><ul>
<li>The <code>TestAsyncQueryProvider</code> class implements the <code>IAsyncQueryProvider</code> interface supplying the provider methods needed to make asynchronous calls.</li>
<li>The <code>TestAsyncEnumerable</code> class implements the <code>IAsyncEnumerable</code> interface, returning our provider class when required by the framework.</li>
<li>Finally, the <code>TestAsyncEnumerator</code> class implements the <code>IAsyncEnumerator</code> interface, returning a Task when the <code>MoveNext()</code> function is called.</li>
</ul>
<p>Now we can create a function (taking advantage of <a href="https://www.geeksforgeeks.org/c-sharp-generics-introduction/">generics</a>) that will return a mock version of a <code>DbSet<T></code> containing the data we pass as the TestData parameter. The resulting <code>DbSet<T></code> will have support to asynchronous calls because it will implement our custom classes.</p>
<ul>
<li><strong>TestFunctions.cs</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// Return a DbSet of the specified generic type with support for async operations
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> Mock<DbSet<T>> GetDbSet<T>(IQueryable<T> TestData) <span style="color:#080;font-weight:bold">where</span> T : <span style="color:#080;font-weight:bold">class</span>
{
<span style="color:#888;font-weight:bold">var</span> MockSet = <span style="color:#080;font-weight:bold">new</span> Mock<DbSet<T>>();
MockSet.As<IAsyncEnumerable<T>>().Setup(x => x.GetEnumerator()).Returns(<span style="color:#080;font-weight:bold">new</span> TestAsyncEnumerator<T>(TestData.GetEnumerator()));
MockSet.As<IQueryable<T>>().Setup(x => x.Provider).Returns(<span style="color:#080;font-weight:bold">new</span> TestAsyncQueryProvider<T>(TestData.Provider));
MockSet.As<IQueryable<T>>().Setup(x => x.Expression).Returns(TestData.Expression);
MockSet.As<IQueryable<T>>().Setup(x => x.ElementType).Returns(TestData.ElementType);
MockSet.As<IQueryable<T>>().Setup(x => x.GetEnumerator()).Returns(TestData.GetEnumerator());
<span style="color:#080;font-weight:bold">return</span> MockSet;
}
</code></pre></div><h3 id="defining-the-data">Defining the data</h3>
<p>Next step will be to declare the data that we want to use for our tests. This can be declared as a set of properties that will return <code>IQueryable<T></code> interfaces.</p>
<ul>
<li><strong>TestData.cs</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// Test data for the DbSet<User> getter
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">static</span> IQueryable<User> Users
{
<span style="color:#080;font-weight:bold">get</span>
{
<span style="color:#080;font-weight:bold">return</span> <span style="color:#080;font-weight:bold">new</span> List<User>
{
<span style="color:#080;font-weight:bold">new</span> User { ID = <span style="color:#00d;font-weight:bold">1</span>, Username = <span style="color:#d20;background-color:#fff0f0">"admin"</span>, Email = <span style="color:#d20;background-color:#fff0f0">"admin@host.com"</span> },
<span style="color:#080;font-weight:bold">new</span> User { ID = <span style="color:#00d;font-weight:bold">2</span>, Username = <span style="color:#d20;background-color:#fff0f0">"guest"</span>, Email = <span style="color:#d20;background-color:#fff0f0">"guest@host.com"</span> }
}
.AsQueryable();
}
}
</code></pre></div><p>In this example, we’re adding two users to the set because that’s the minimum data we need to properly use our unit test.</p>
<h3 id="putting-it-all-together">Putting it all together</h3>
<p>Now that we have the test data and the helper functions needed for that data to be accessed asynchronously by Entity Framework, we’re ready to write our test function. So given our function <code>GetUserIDByEmail()</code>, we want to test it to make sure it works and it’s returning the correct value.</p>
<ul>
<li><strong>UserHandlerTests.cs</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-csharp" data-lang="csharp"><span style="color:#888">// Should return a user with ID 1
</span><span style="color:#888"></span><span style="color:#369">[Fact]</span>
<span style="color:#080;font-weight:bold">public</span> <span style="color:#080;font-weight:bold">async</span> <span style="color:#080;font-weight:bold">void</span> GetUserIDByEmailTest()
{
<span style="color:#888">// Create a mock version of the DbContext
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> DbContext = <span style="color:#080;font-weight:bold">new</span> Mock<IMockProjectDbContext>();
<span style="color:#888">// Users getter will return our mock DbSet with test data
</span><span style="color:#888"></span> <span style="color:#888">// (Here is where we call our helper function)
</span><span style="color:#888"></span> DbContext.SetupGet(x => x.Users).Returns(TestFunctions.GetDbSet<User>(TestData.Users).Object);
<span style="color:#888">// Call the function to test
</span><span style="color:#888"></span> <span style="color:#888;font-weight:bold">var</span> UserHandler = <span style="color:#080;font-weight:bold">new</span> MockProject.Common.UserHandler(DbContext.Object);
<span style="color:#888;font-weight:bold">var</span> Result = <span style="color:#080;font-weight:bold">await</span> UserHandler.GetUserIDByEmail(<span style="color:#d20;background-color:#fff0f0">"admin@host.com"</span>);
<span style="color:#888">// Verify the results
</span><span style="color:#888"></span> Assert.Equal(<span style="color:#00d;font-weight:bold">1</span>, Result);
}
</code></pre></div><p>And now we have it ready to go. Visual Studio will first compile both projects and then run all the tests. The result looks good:</p>
<p><img src="/blog/2019/07/mocking-asynchronous-database-calls-net-core/image-1.jpg" alt="MockProject test results"></p>
<p>Finally, I would recommend checking how much code is actually being covered by unit tests. There are different tools to achieve this, but I like using <a href="https://dev.to/nlowe/easy-automated-code-coverage-for-net-core-1khh">MiniCover</a>. It’s one of the simplest tools and we can prepare a simple batch file to get a detailed list of lines covered per file. This batch file is located in the root of the <code>MockProject.Tests</code> project.</p>
<p>This batch file will first clean and build the project, and then it calls <code>MiniCover</code> to evaluate the source code and report the current code coverage in lines and percentage.</p>
<ul>
<li><strong>CodeCoverage.bat</strong></li>
</ul>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bat" data-lang="bat">@<span style="color:#080;font-weight:bold">echo</span> off
<span style="color:#888">REM Clean and build the project</span>
dotnet clean
dotnet build /p:DebugType=Full
<span style="color:#888">REM Instrument assemblies in our test project to detect hits to source files on our main project</span>
dotnet minicover instrument --workdir ../ --assemblies MockProject.Tests/**/bin/**/*.dll --sources MockProject/**/*.cs --exclude-sources MockProject/*.cs
<span style="color:#888">REM Reset previous counters</span>
dotnet minicover reset --workdir ../
<span style="color:#888">REM Run the tests</span>
dotnet test --no-build
<span style="color:#888">REM Uninstrument assemblies in case we want to deploy</span>
dotnet minicover uninstrument --workdir ../
<span style="color:#888">REM Print the console report</span>
dotnet minicover report --workdir ../ --threshold 70
</code></pre></div><p>After running the script, we’ll come to these results:</p>
<p><img src="/blog/2019/07/mocking-asynchronous-database-calls-net-core/image-2.jpg" alt="MiniCover results"></p>
<p>The percentage of code that should be covered by tests depends greatly on the type of application (for example, a WebAPI project will probably have more testable code than a Razor Pages project), but as a general rule, we can expect that a well-tested project will exceed 70% of code coverage.</p>
<h3 id="summary">Summary</h3>
<p>We can take advantage of mocking and <a href="https://www.geeksforgeeks.org/c-sharp-generics-introduction/">generics</a> to create an easy way to test our app using sample data and resources. We can emulate connecting and querying data asynchronously from a database with test data and a couple of helper classes that will support all interfaces needed by Entity Framework to run asynchronous operations.</p>
<p>I’ve uploaded the main and test projects (<code>MockProject</code> and <code>MockProject.Tests</code>) into a <a href="https://github.com/juanpabloventoso/MockProject">GitHub repository</a> if you want to try it. And please leave any comment or suggestion below!</p>
What’s the deal with ASP.NET Core Razor Pages?https://www.endpointdev.com/blog/2018/11/whats-the-deal-with-asp-net-core-razor-pages/2018-11-20T00:00:00+00:00Kevin Campusano
<p><img src="whats-the-deal-with-asp-net-core-razor-pages/banner.png" alt="Banner Image"></p>
<p>During the last couple of years I’ve been doing lots of web development with technologies like <a href="https://en.wikipedia.org/wiki/JavaScript">JavaScript</a> and <a href="http://php.net/">PHP</a> and <a href="https://framework.zend.com/">Zend Framework</a>, with a strong focus on the front end. Before that, however, the vast majority of the work I did as a web developer was with the <a href="https://www.microsoft.com/net">.NET Framework</a>.</p>
<p>I love .NET. Particularly the <a href="https://docs.microsoft.com/en-us/dotnet/csharp/">C# language</a>. However, the greatest thing about .NET is the vibrant ecosystem full of tools (the <a href="https://visualstudio.microsoft.com/">Visual Studio IDE</a>, for example, is outstanding), libraries and frameworks that make one’s life easier and more productive. It has, however, a crucial weakness that prevents it from reaching even greater heights: it’s locked to Windows systems. So, whenever the need comes to develop something outside of a Windows environment, you’re out of luck.</p>
<p>So, naturally, when Microsoft announced their initiative to make the .NET Framework open source and bring it to other platforms a few years ago, I was really excited. Fast forward to today and we have <a href="https://dotnet.github.io/">.NET Core</a> as the product of great effort from both Microsoft and the community.</p>
<p>I’ve definitely kept it on my radar since its inception back in 2016, but haven’t had the chance to take a really deep dive and see what the technology is about and get a feel for the ecosystem surrounding it.</p>
<p>Well, that time has come and just last week I decided to go in and see what’s up and how everything works. Suffice it to say, I am impressed. The promise of being able to write .NET code everywhere has been pretty much fulfilled.</p>
<p>In terms of features, .NET Core is still missing some stuff but the feature set is mature enough that it can handle most use cases. Me being first and foremost a web developer though, what has me most excited is <a href="https://docs.microsoft.com/en-us/aspnet/core/?view=aspnetcore-2.1">ASP.NET Core</a>, .NET Core’s own web development framework inspired by <a href="https://www.asp.net/">classic ASP.NET</a>.</p>
<p>ASP.NET Core is not just a port of ASP.NET, it is a complete rewrite with a new architecture, new features and ways to develop apps.</p>
<p>At the core of ASP.NET Core (no pun intended) there’s the notion that an HTTP request gets processed by a pipeline composed of several components named middleware, which also produce a response. When the server receives a request, each of the middleware components in the pipeline take their turn in processing the request, are responsible for calling the next middleware in the pipeline and, finally, generating a response to send back to the client.</p>
<p>ASP.NET Core is not the only modern web framework with this pipeline based request processing design. <a href="https://docs.zendframework.com/zend-expressive/">Zend Expressive</a>, for example, also uses this architecture.</p>
<p>Overall, I think this approach is a good solution to the problem of request processing which results in a good <a href="https://en.wikipedia.org/wiki/Mental_model">mental model</a> for developers to have about how the application is working under the hood.</p>
<p>All of this is more concerned with application startup and initial configuration though. When it comes to actually writing business logic, ASP.NET Core offers two alternatives for web apps with GUIs: <a href="https://docs.microsoft.com/en-us/aspnet/core/mvc/overview?view=aspnetcore-2.1">MVC</a> and, most recently, <a href="https://docs.microsoft.com/en-us/aspnet/core/razor-pages/?view=aspnetcore-2.1&tabs=visual-studio">Razor Pages</a>.</p>
<p>From the point of view of the developer’s experience, MVC is very similar to classic ASP.NET MVC: there are Controllers which contain Action Methods which return Views which are processed by a templating engine (i.e. <a href="https://docs.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-2.1">Razor</a>) to generate a GUI that a user can see and interact with. Razor Pages though, is a new approach that is exclusive to ASP.NET Core and makes things slightly different. The developers at Microsoft say that this new approach is even preferred to MVC for building web UIs. That is, non-<a href="https://en.wikipedia.org/wiki/Web_API">API</a> web applications.</p>
<p>With Razor Pages, Controllers, Actions and Views are gone and in their place we’ve got Pages and so-called Page Models. So, while in MVC we have the router match a URL to a Controller and Action combo that handles the request and returns a processed View to the client; in Razor Pages we have the router match a URL to a Page and an optional Page Model file (via their location in the project directory) that handle the request and finally present some GUI to the client. Another way to look at it is that, while MVC focuses on Controllers and Actions to handle a request, Razor Pages focuses on actual files.</p>
<p>To better illustrate their differences, here’s what the directory structure of each looks like:</p>
<p><img src="whats-the-deal-with-asp-net-core-razor-pages/files-comparison.png" alt="MVC vs Razor pages files comparison"></p>
<p>As you can see, the main difference is that the Controllers, Models and Views folders are gone and instead we have a Pages folder. The Pages folder contains all of the .cshtml files (which are the Views or templates) together with accompanying .cshtml.cs files (which are the presentation logic behind the views, or Page Models). These Page Model files contain the logic to respond to requests as well as any properties containing values used for presentation in the template. These properties are a substitute for having separate View Model <a href="https://www.martinfowler.com/eaaCatalog/dataTransferObject.html">DTO</a> objects that the Actions pass to the Views.</p>
<p>Looking at this directory structure, it’s very easy to see that this model, on the surface at least, looks an awful lot like a Page/Code Behind model that hasn’t been used since the first iteration of ASP.NET: <a href="https://docs.microsoft.com/en-us/aspnet/web-forms/what-is-web-forms">Web Forms</a>. This, together with the fact that MVC’s clean cut separation of Controllers and Views seems to have been merged into some <a href="https://en.wikipedia.org/wiki/Single_responsibility_principle">SRP</a>-violating Page Model file, has raised some red flags in the eyes of some people in the community.</p>
<p>Perhaps unsurprisingly, Razor Pages has sparked quite the controversy. There’s a big discussion over on the <a href="https://github.com/aspnet/Docs/issues/6146">project’s GitHub page</a>, as I’m sure there is elsewhere in places like <a href="https://stackoverflow.com/questions/46777404/why-is-razor-pages-the-recommended-approach-to-create-a-web-ui-in-asp-net-core-2">StackOverflow</a>.</p>
<p>And, I gotta be honest, I myself, being very familiar with Web Forms and its pitfalls, had a similar initial gut reaction as well. And I think everything is also compounded by the fact that now, seemingly out of nowhere, the ASP.NET Core team are touting this new, inferior (to the eyes of some) method as the preferred method for building new applications moving forward. So, people fear that their preferred style will stop being supported and go away for good.</p>
<p>So, what is the truth? Well, I don’t think it’s as bad as some people are making it out to be. Of course, the technology is fairly new and still needs to be battle tested in large scale real world projects to make sure it’s got what it takes. But as of now, I don’t think the design is fundamentally flawed. In fact, I think it offers a competent way to develop web apps, and I can definitely see why the ASP.NET Core team sees this as the preferred method over MVC.</p>
<p>I’m going to offer some arguments as to why I see Razor Pages as a perfectly fine technology choice. To do that, I’ll address some of the critiques that I’ve seen have been levied against it.</p>
<h3 id="1-its-a-worse-design-than-mvc">1. It’s a worse design than MVC</h3>
<p>Some people are saying that MVC is just better on the grounds that it has better <a href="https://en.wikipedia.org/wiki/Separation_of_concerns">Separation of Concerns</a>. This may look like fact at first because of how the files are structured. Like we saw before, in MVC, presentation logic is distributed among three types of files: Controllers (which contain Action Methods), Views and, generally View Models. In Razor Pages we only have a template and an accompanying file which merges Controller, Action and View Model all together. However, we have to ask ourselves: in MVC, are Controllers, Actions and View Models truly separated? And should they even be?</p>
<p>In the context of MVC, View Models are nothing more than DTOs that serve as containers for data that the Actions in the Controller send to the View for rendering. As such, these are most of the time tightly and statically coupled with the View that uses them and the Action that generates them. Rarely you see a View Model being reused across multiple Actions and Views, or injected as an abstract dependency to the Controller.</p>
<p>So, this means that, with a design like the one Razor Pages uses, we’re not actually losing much in the way of Separation of Concerns. Actually, we gain something in the convenience of having all related presentation logic consolidated and close together instead of dispersed across the directory tree.</p>
<p>Finally, I think one frame of mind that helps further see the value of what Razor Pages offers is thinking of the Page Model as a View Model. And I mean View Model in the sense of <a href="https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93viewmodel">MVVM</a> as opposed to its MVC counterpart. As we’ve already discussed, in MVC, a View Model is nothing more than a DTO that serves to carry data from the Controller to the View. In MVVM, a View Model is much more, as it serves not only as a data store but also as a mechanism to respond to user interactions and execution of presentation logic. This is the pattern used by many modern front end development frameworks like <a href="https://reactjs.org/">React</a>, <a href="https://angularjs.org/">Angular</a> and <a href="https://vuejs.org/">Vue.js</a>. The only difference is that, while the MVVM JavaScript frameworks bridge the gap between the <a href="https://www.w3schools.com/js/js_htmldom.asp">DOM</a> and our app’s code; Razor Pages is bridging the gap between an HTTP client and a server. As a consequence, the vocabulary of the messages is completely different. In the front end, we have the View Model responding to click, and hover and mouseEnter events that come from the DOM. In Razor Pages, we have our “View Model” (i.e. the Page Model) responding to events that come from the client via HTTP; so we get things like GET or POST or PUT, with the associated request payloads.</p>
<h3 id="2-its-just-web-forms-all-over-again">2. It’s just Web Forms all over again</h3>
<p>This one is simply not true as both technologies are very different. It is true that, from a file organization point of view, you can definitely see some similarities. After all, both in Razor Pages and Web Forms we have the focus on pages with some code behind them. However, the way that they go about implementing actual websites is completely different.</p>
<p>Just to be clear, I don’t subscribe to the idea that Web Forms is a bad technology or that it was badly designed. For its time, Web Forms was a huge step forward in terms of rapid application development. Also, its model tried to abstract away all the nuance of HTTP and offer a development experience that was very similar to what was seen before in Windows Forms. Now, most modern web development frameworks embrace HTTP’s idiosyncrasies rather than trying to take them out of the way of developers. However, at the time it was an interesting proposition that helped to bridge the gap that developers had to cross in order to move from building desktop apps on Windows to building web apps on a Windows based server.</p>
<p>Anyway, Razor Pages is very different from ASP.NET Web Forms. First of all, in Razor Pages, there’s no abstraction of HTTP whatsoever. Actually, what you see in those Page Model files is not an HTML page’s “Code Behind“, what you see are HTTP-<a href="https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html">Verb</a>-specific Action Methods of sorts that the framework calls for handling requests. You’re going to see methods like OnGet and OnPost, instead of Button3_OnClick or DropDownList1_OnSelect. Much like MVC, you also have access to the complete Request, Response and HttpContext objects that help out in processing requests. Also, the main pain points in WebFroms: Postbacks and the ViewState, are nowhere to be found. Instead, everything is stateless and you’re 100% in control of the messaging that’s happening between server and client.</p>
<p>Here’s what an empty Page Model file looks like:</p>
<p><img src="whats-the-deal-with-asp-net-core-razor-pages/pagemodel.png" alt="Empty Razor Pages Page Model class"></p>
<p>In reality, these Page Model files look quite a bit like traditional MVC Controllers in that they contain, essentially, Action Methods. The difference is that, in MVC, we had many Action Methods in one Controller where each Action Method corresponded to a specific route. Through routing, each Action Method was executed depending on the incoming request URL. Then the Action Method was responsible of handling the request depending on the HTTP Verb (i.e. GET, POST, etc) and other incoming data. Finally it would produce a useful response.</p>
<p>Meanwhile, in Razor Pages we have these “Action Methods” of sorts that are geared towards handling the various HTTP Verbs for requests that come to the front facing page that they are associated to. This means that a given Page Model and Page combo will always only serve a particular, specific route (i.e. URL). Where a traditional MVC Controller would potentially server multiple routes, depending on how many Action Methods were defined within it.</p>
<p>So, while it is true that, with Razor Pages, we “went back” to having pairs of files representing one web app functional unit, like we did in ASP.NET Web Forms; the way they go about it and the developer experience that they offer is completely different. Also, Razor Pages is not that different than MVC after all. All the building blocks are still there, they are just slightly rearranged.</p>
<h3 id="3-its-going-to-replace-mvc-completely">3. It’s going to replace MVC completely</h3>
<p>I understand the fear that, when the new thing comes out, the old thing is going to be left to the wayside. This is the same feeling that many had (me included!) when ASP.NET MVC for the <a href="https://www.urbandictionary.com/define.php?term=OG">OG</a> .NET Framework first came out way back in 2009. Web Forms was doomed. Well, guess what? Web Forms is still around. While it’s true that the last major update happened years ago, the technology is still a first class citizen in the .NET ecosystem and <a href="https://docs.microsoft.com/en-us/aspnet/web-forms/what-is-web-forms#advantages-of-a-web-forms-based-web-application">perfectly usable</a>. And I attribute the lack of recent big updates with new features to the fact that it is a mature framework that is “pretty complete”, rather than Microsoft pulling the plug. After all, it’s still supported in the newest version of Visual Studio and the .NET Framework and <a href="https://www.youtube.com/watch?v=KFeuCplwhaQ">new features and improvements do get developed</a>, albeit small.</p>
<p>Here’s what Visual Studio shows when asked to create a new web application project:</p>
<p><img src="whats-the-deal-with-asp-net-core-razor-pages/webformslives.png" alt="Create New ASP.NET Web Application dialog"></p>
<p>Yup, still alive and kicking.</p>
<p>So yeah, I feel like this is pretty natural human behavior but it’s ultimately baseless, given the history of this ecosystem. Much like Web Forms is still alive after years and years, I think MVC in ASP.NET Core is going to be just fine.</p>
<p>And that concludes my long winded opinion piece about the whole MVC vs Razor Pages controversy and my recent discovery of how cool the .NET Core framework is. As far as I see it, the technology is ready for prime time and I’m looking forward to using it in my projects.</p>
<p>(Reposted from <a href="https://superlativelabs.com/2018/08/14/whats-the-deal-with-asp-net-core-razor-pages/">Kevin’s blog</a>.)</p>
Regular Expression Inconsistencies With Unicodehttps://www.endpointdev.com/blog/2018/01/regular-expression-inconsistencies-with-unicode/2018-01-23T00:00:00+00:00Phineas Jensen
<p><img src="/blog/2018/01/regular-expression-inconsistencies-with-unicode/mud-run.jpg" alt="A mud run"><br/>
<small>A casual stroll through the world of Unicode and regular expressions—<a href="https://www.flickr.com/photos/presidioofmonterey/7025086135">Photo</a> by Presidio of Monterey</small></p>
<p>Character classes in regular expressions are an extremely useful and widespread feature, but there are some relatively recent changes that you might not know of.</p>
<p>The issue stems from how different programming languages, locales, and character encodings treat predefined character classes. Take, for example, the expression <code>\w</code> which was introduced in Perl around the year 1990 (along with <code>\d</code> and <code>\s</code> and their inverted sets <code>\W</code>, <code>\D</code>, and <code>\S</code>).</p>
<p>The <code>\w</code> shorthand is a character class that matches “word characters” as the C language understands them: <code>[a-zA-Z0-9_]</code>. At least when ASCII was the main player in the character encoding scene that simple fact was true. With the standardization of Unicode and UTF-8, the meaning of <code>\w</code> has become a more foggy.</p>
<h4 id="perl">Perl</h4>
<p>Take this example in a recent Perl version:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-perl" data-lang="perl"><span style="color:#080;font-weight:bold">use</span> <span style="color:#00d;font-weight:bold">5.012</span>; <span style="color:#888"># use 5.012 or higher includes Unicode support</span>
<span style="color:#080;font-weight:bold">use</span> <span style="color:#b06;font-weight:bold">utf8</span>; <span style="color:#888"># necessary for Unicode string literals</span>
<span style="color:#080;font-weight:bold">print</span> <span style="color:#d20;background-color:#fff0f0">"username"</span> =~<span style="color:#080;background-color:#fff0ff"> /^\w+$/</span>; <span style="color:#888"># 1</span>
<span style="color:#080;font-weight:bold">print</span> <span style="color:#d20;background-color:#fff0f0">"userاسم"</span> =~<span style="color:#080;background-color:#fff0ff"> /^\w+$/</span>; <span style="color:#888"># 1</span>
</code></pre></div><p>Perl is treating <code>\w</code> differently here because the characters “اسم” (“ism” meaning “name” in Arabic) definitely don’t fall within <code>[a-zA-Z0-9_]</code>!</p>
<p>Beginning with Perl 5.12 from the year 2010, character classes are handled differently. Documentation on the topic is found in <a href="https://perldoc.perl.org/perlrecharclass.html#Backslash-sequences">perlrecharclass</a>. The rules aren’t as simple as with some languages, but can be generalized as such:</p>
<p><code>\w</code> will match Unicode characters with the “Word” property (equivalent to <code>\p{Word}</code>), unless the <code>/a</code> (ASCII) flag is enabled, in which case it will be equivalent to the original <code>[a-zA-Z0-9_]</code>.</p>
<p>Let’s see the <code>/a</code> flag in action.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-perl" data-lang="perl"><span style="color:#080;font-weight:bold">use</span> <span style="color:#00d;font-weight:bold">5.012</span>;
<span style="color:#080;font-weight:bold">use</span> <span style="color:#b06;font-weight:bold">utf8</span>;
<span style="color:#080;font-weight:bold">print</span> <span style="color:#d20;background-color:#fff0f0">"username"</span> =~<span style="color:#080;background-color:#fff0ff"> /^\w+$/</span>a; <span style="color:#888"># 1</span>
<span style="color:#080;font-weight:bold">print</span> <span style="color:#d20;background-color:#fff0f0">"userاسم"</span> =~<span style="color:#080;background-color:#fff0ff"> /^\w+$/</span>a; <span style="color:#888"># 0</span>
</code></pre></div><p>However, you should know that for code points below 256, these rules can change depending on whether Unicode or locale rules are on, so if you’re unsure, consult the <a href="https://perldoc.perl.org/perlre.html">perlre</a> and <a href="https://perldoc.perl.org/perlrecharclass.html">perlrecharclass</a>.</p>
<p>Keep in mind that these same questions of what the character classes include can apply to every predefined character class in whatever language you’re using, so remember to check language-specific implementations for other character class shorthands, such as <code>\s</code> and <code>\d</code>, not just <code>\w</code>.</p>
<p>Every language seems to do regular expressions a little bit differently, so here’s a short, incomplete guide for several other languages we use frequently.</p>
<h4 id="python">Python</h4>
<p>Take this example in Python 3.6.2:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python">>>> re.match(<span style="color:#d20;background-color:#fff0f0">r</span><span style="color:#d20;background-color:#fff0f0">'^\w+$'</span>, <span style="color:#d20;background-color:#fff0f0">'username'</span>)
<_sre.SRE_Match <span style="color:#038">object</span>; span=(<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">8</span>), match=<span style="color:#d20;background-color:#fff0f0">'username'</span>>
>>> re.match(<span style="color:#d20;background-color:#fff0f0">r</span><span style="color:#d20;background-color:#fff0f0">'^\w+$'</span>, <span style="color:#d20;background-color:#fff0f0">'userاسم'</span>)
<_sre.SRE_Match <span style="color:#038">object</span>; span=(<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">7</span>), match=<span style="color:#d20;background-color:#fff0f0">'userاسم'</span>>
</code></pre></div><p>Python is also treating <code>\w</code> differently here. Let’s take a look at <a href="https://docs.python.org/3/library/re.html#regular-expression-syntax">the Python docs</a>:</p>
<blockquote>
<p><code>\w</code></p>
<p>For Unicode (str) patterns:</p>
<pre><code>Matches Unicode word characters; this includes most characters that can be part of a word in any language, as well as numbers and the underscore. If the ASCII flag is used, only [a-zA-Z0-9_] is matched (but the flag affects the entire regular expression, so in such cases using an explicit [a-zA-Z0-9_] may be a better choice).
</code></pre>
<p>For 8-bit (bytes) patterns:</p>
<pre><code>Matches characters considered alphanumeric in the ASCII character set; this is equivalent to [a-zA-Z0-9_]. If the LOCALE flag is used, matches characters considered alphanumeric in the current locale and the underscore.
</code></pre>
</blockquote>
<p>So <code>\w</code> includes “most characters that can be part of a word in any language, as well as numbers and the underscore”. A list of the characters that includes is difficult to pin down, so it would be best to use the <code>re.ASCII</code> flag as suggested when you’re unsure if you want letters from other languages matched:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-python" data-lang="python">>>> re.match(<span style="color:#d20;background-color:#fff0f0">r</span><span style="color:#d20;background-color:#fff0f0">'^\w+$'</span>, <span style="color:#d20;background-color:#fff0f0">'userاسم'</span>, flags=re.ASCII)
>>> re.match(<span style="color:#d20;background-color:#fff0f0">r</span><span style="color:#d20;background-color:#fff0f0">'^\w+$'</span>, <span style="color:#d20;background-color:#fff0f0">'username'</span>, flags=re.ASCII)
<_sre.SRE_Match <span style="color:#038">object</span>; span=(<span style="color:#00d;font-weight:bold">0</span>, <span style="color:#00d;font-weight:bold">8</span>), match=<span style="color:#d20;background-color:#fff0f0">'username'</span>>
</code></pre></div><h4 id="ruby">Ruby</h4>
<p>Ruby’s <a href="https://ruby-doc.org/core-2.5.0/Regexp.html#class-Regexp-label-Character+Classes">Regexp class</a> documentation gives a simple and useful explanation: backslash character classes (e.g. <code>\w</code>, <code>\s</code>, <code>\d</code>) are ASCII-only, while POSIX-style bracket expressions (e.g. <code>[[:alnum:]]</code>) include other Unicode characters.</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-ruby" data-lang="ruby">irb(main):<span style="color:#00d;font-weight:bold">001</span>:<span style="color:#00d;font-weight:bold">0</span>> <span style="color:#080;background-color:#fff0ff">/^\w+$/</span> =~ <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>
=> <span style="color:#080">nil</span>
irb(main):<span style="color:#00d;font-weight:bold">002</span>:<span style="color:#00d;font-weight:bold">0</span>> <span style="color:#080;background-color:#fff0ff">/^[[:word:]]+$/</span> =~ <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>
=> <span style="color:#00d;font-weight:bold">0</span>
</code></pre></div><h4 id="javascript">JavaScript</h4>
<p>JavaScript doesn’t support POSIX-style bracket expressions, and its backslash character classes are simple, straightforward lists of ASCII characters. The <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions#Using_special_characters">MDN</a> has simple explanations for each one.</p>
<p>JavaScript regular expressions do accept a <code>/u</code> flag, but it does not affect shorthand character classes. Consider these examples in Node.js:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">> <span style="color:#080;background-color:#fff0ff">/^\w+$/</span>.test(<span style="color:#d20;background-color:#fff0f0">"username"</span>);
<span style="color:#080;font-weight:bold">true</span>
> <span style="color:#080;background-color:#fff0ff">/^\w+$/</span>.test(<span style="color:#d20;background-color:#fff0f0">"userﺎﺴﻣ"</span>);
<span style="color:#080;font-weight:bold">false</span>
> <span style="color:#080;background-color:#fff0ff">/^\w+$/u</span>.test(<span style="color:#d20;background-color:#fff0f0">"username"</span>);
<span style="color:#080;font-weight:bold">true</span>
> <span style="color:#080;background-color:#fff0ff">/^\w+$/u</span>.test(<span style="color:#d20;background-color:#fff0f0">"userﺎﺴﻣ"</span>);
<span style="color:#080;font-weight:bold">false</span>
</code></pre></div><p>We can see that the <code>/u</code> flag has no effect on what <code>\w</code> matches. Now let’s look at Unicode character lengths in JavaScript:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">> <span style="color:#d20;background-color:#fff0f0">'❤'</span>.length
<span style="color:#00d;font-weight:bold">1</span>
> <span style="color:#d20;background-color:#fff0f0">'👩'</span>.length
<span style="color:#00d;font-weight:bold">2</span>
> <span style="color:#d20;background-color:#fff0f0">'🀄️'</span>.length
<span style="color:#00d;font-weight:bold">3</span>
</code></pre></div><p>Because of the way Unicode is implemented in JavaScript, strings with Unicode characters outside the BMP (Basic Multilingual Plane) will appear to be longer than they are.</p>
<p>This can be accounted for in regular expressions with the <code>/u</code> flag, which only corrects character parsing, and does not affect shorthand character classes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-javascript" data-lang="javascript">> <span style="color:#080;font-weight:bold">let</span> mystr = <span style="color:#d20;background-color:#fff0f0">"hi👩there"</span>;
<span style="color:#080;font-weight:bold">undefined</span>
> mystr.length
<span style="color:#00d;font-weight:bold">9</span>
> <span style="color:#080;background-color:#fff0ff">/hi.there/</span>.test(mystr);
<span style="color:#080;font-weight:bold">false</span>
> <span style="color:#080;background-color:#fff0ff">/hi..there/</span>.test(mystr);
<span style="color:#080;font-weight:bold">true</span>
> <span style="color:#080;background-color:#fff0ff">/hi.there/u</span>.test(mystr); <span style="color:#a61717;background-color:#e3d2d2">#</span> note the /u from here on
<span style="color:#080;font-weight:bold">true</span>
> <span style="color:#080;background-color:#fff0ff">/hi..there/u</span>.test(mystr);
<span style="color:#080;font-weight:bold">false</span>
> <span style="color:#080;background-color:#fff0ff">/hi..there/u</span>.test(<span style="color:#d20;background-color:#fff0f0">"hi👩👩there"</span>);
<span style="color:#080;font-weight:bold">true</span>
</code></pre></div><p>The excellent article <a href="http://blog.jonnew.com/posts/poo-dot-length-equals-two">“💩”.length === 2</a> by Jonathan New goes into detail about the why this is, and explores various solutions. It also addresses some legacy inconsistencies, like how the old HEAVY BLACK HEART character and other older Unicode symbols might be represented differently.</p>
<h4 id="php">PHP</h4>
<p>PHP’s documentation explains that <code>\w</code> matches letters, digits, and the underscore as defined by your locale. It’s not totally clear about how Unicode is treated, but it uses the PCRE (Perl Compatible Regular Expressions) library which supports a <code>/u</code> flag that can be used to enable Unicode matching in character classes:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-php" data-lang="php"><?php
<span style="color:#080;font-weight:bold">echo</span> preg_match(<span style="color:#d20;background-color:#fff0f0">"/^</span><span style="color:#04d;background-color:#fff0f0">\\</span><span style="color:#d20;background-color:#fff0f0">w+$/"</span>, <span style="color:#d20;background-color:#fff0f0">"username"</span>), <span style="color:#d20;background-color:#fff0f0">"</span><span style="color:#04d;background-color:#fff0f0">\n</span><span style="color:#d20;background-color:#fff0f0">"</span>; <span style="color:#888"># 1
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">echo</span> preg_match(<span style="color:#d20;background-color:#fff0f0">"/^</span><span style="color:#04d;background-color:#fff0f0">\\</span><span style="color:#d20;background-color:#fff0f0">w+$/"</span>, <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>), <span style="color:#d20;background-color:#fff0f0">"</span><span style="color:#04d;background-color:#fff0f0">\n</span><span style="color:#d20;background-color:#fff0f0">"</span>; <span style="color:#888"># 0
</span><span style="color:#888"></span>
<span style="color:#080;font-weight:bold">echo</span> preg_match(<span style="color:#d20;background-color:#fff0f0">"/^</span><span style="color:#04d;background-color:#fff0f0">\\</span><span style="color:#d20;background-color:#fff0f0">w+$/u"</span>, <span style="color:#d20;background-color:#fff0f0">"username"</span>), <span style="color:#d20;background-color:#fff0f0">"</span><span style="color:#04d;background-color:#fff0f0">\n</span><span style="color:#d20;background-color:#fff0f0">"</span>; <span style="color:#888"># 1
</span><span style="color:#888"></span><span style="color:#080;font-weight:bold">echo</span> preg_match(<span style="color:#d20;background-color:#fff0f0">"/^</span><span style="color:#04d;background-color:#fff0f0">\\</span><span style="color:#d20;background-color:#fff0f0">w+$/u"</span>, <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>), <span style="color:#d20;background-color:#fff0f0">"</span><span style="color:#04d;background-color:#fff0f0">\n</span><span style="color:#d20;background-color:#fff0f0">"</span>; <span style="color:#888"># 1
</span></code></pre></div><h4 id="net">.NET</h4>
<p>The <a href="https://docs.microsoft.com/en-us/dotnet/standard/base-types/character-classes-in-regular-expressions">.NET Quick Reference</a> has a comprehensive guide to character classes. For word characters, it defines a specific group of Unicode categories including letters, modifiers, and connectors from many languages, but also points out that setting the <a href="https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-options#ECMAScript">ECMAScript Matching Behavior</a> option will limit <code>\w</code> to <code>[a-zA-Z_0-9]</code>, among other things. Microsoft’s documentation is clear and comprehensive with great examples, so I recommend referring to it frequently.</p>
<h4 id="go">Go</h4>
<p>Go follows the regular expression syntax used by <a href="https://github.com/google/re2/wiki/Syntax">Google’s RE2 engine</a>, which has easy syntax for specifying whether you want Unicode characters to be captured or not:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#080;font-weight:bold">package</span> main
<span style="color:#080;font-weight:bold">import</span> (
<span style="color:#d20;background-color:#fff0f0">"fmt"</span>
<span style="color:#d20;background-color:#fff0f0">"regexp"</span>
)
<span style="color:#080;font-weight:bold">func</span> <span style="color:#06b;font-weight:bold">main</span>() {
<span style="color:#888">// Perl-style
</span><span style="color:#888"></span> fmt.<span style="color:#06b;font-weight:bold">Println</span>(regexp.<span style="color:#06b;font-weight:bold">MatchString</span>(<span style="color:#d20;background-color:#fff0f0">`^\w+$`</span>, <span style="color:#d20;background-color:#fff0f0">"username"</span>)) <span style="color:#888">// true
</span><span style="color:#888"></span> fmt.<span style="color:#06b;font-weight:bold">Println</span>(regexp.<span style="color:#06b;font-weight:bold">MatchString</span>(<span style="color:#d20;background-color:#fff0f0">`^\w+$`</span>, <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>)) <span style="color:#888">// false
</span><span style="color:#888"></span>
<span style="color:#888">// POSIX-style
</span><span style="color:#888"></span> fmt.<span style="color:#06b;font-weight:bold">Println</span>(regexp.<span style="color:#06b;font-weight:bold">MatchString</span>(<span style="color:#d20;background-color:#fff0f0">`^[[:word:]]+$`</span>, <span style="color:#d20;background-color:#fff0f0">"username"</span>)) <span style="color:#888">// true
</span><span style="color:#888"></span> fmt.<span style="color:#06b;font-weight:bold">Println</span>(regexp.<span style="color:#06b;font-weight:bold">MatchString</span>(<span style="color:#d20;background-color:#fff0f0">`^[[:word:]]+$`</span>, <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>)) <span style="color:#888">// false
</span><span style="color:#888"></span>
<span style="color:#888">// Unicode character class
</span><span style="color:#888"></span> fmt.<span style="color:#06b;font-weight:bold">Println</span>(regexp.<span style="color:#06b;font-weight:bold">MatchString</span>(<span style="color:#d20;background-color:#fff0f0">`^\pL+$`</span>, <span style="color:#d20;background-color:#fff0f0">"username"</span>)) <span style="color:#888">// true
</span><span style="color:#888"></span> fmt.<span style="color:#06b;font-weight:bold">Println</span>(regexp.<span style="color:#06b;font-weight:bold">MatchString</span>(<span style="color:#d20;background-color:#fff0f0">`^\pL+$`</span>, <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>)) <span style="color:#888">// true
</span><span style="color:#888"></span>}
</code></pre></div><p>You can see this code in action <a href="https://play.golang.org/p/Y0HEhWXgXYa">here</a>.</p>
<h4 id="grep">grep</h4>
<p>Implementations of grep vary widely across platforms and versions. On my personal computer with GNU grep 3.1, <code>\w</code> doesn’t work at all with default settings, matches only ASCII characters with the <code>-P</code> (PCRE) option, and matches Unicode characters with <code>-E</code>:</p>
<div class="highlight"><pre tabindex="0" style="background-color:#fff;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">[phin@caballero ~]$ grep <span style="color:#d20;background-color:#fff0f0">"^\w+</span>$<span style="color:#d20;background-color:#fff0f0">"</span> <(<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"username"</span>) <span style="color:#888"># no match</span>
[phin@caballero ~]$ grep -P <span style="color:#d20;background-color:#fff0f0">"^\w+</span>$<span style="color:#d20;background-color:#fff0f0">"</span> <(<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"username"</span>)
username
[phin@caballero ~]$ grep -P <span style="color:#d20;background-color:#fff0f0">"^\w+</span>$<span style="color:#d20;background-color:#fff0f0">"</span> <(<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>) <span style="color:#888"># no match</span>
[phin@caballero ~]$ grep -E <span style="color:#d20;background-color:#fff0f0">"^\w+</span>$<span style="color:#d20;background-color:#fff0f0">"</span> <(<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"username"</span>)
username
[phin@caballero ~]$ grep -E <span style="color:#d20;background-color:#fff0f0">"^\w+</span>$<span style="color:#d20;background-color:#fff0f0">"</span> <(<span style="color:#038">echo</span> <span style="color:#d20;background-color:#fff0f0">"userاسم"</span>)
userاسم
</code></pre></div><p>Again, implementations vary a lot, so double check on your system before doing anything important.</p>
<h3 id="other-links">Other links</h3>
<p>As great as Unicode and regular expressions are, their implementations vary widely across various languages and tools, and that introduces far more unexpected behavior than I can write about in this post. Whenever you’re going to use something with Unicode and regular expressions, make sure to check language specifications to make sure everything will work as expected.</p>
<p>Of course, this topic has already been discussed and written about at great length. Here are some links worth checking out:</p>
<ul>
<li><a href="https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/">The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)</a> - This is an oft-referenced article by Joel Spolsky. It was written in 2003 but the wealth of valuable information within is still very relevant and it helps greatly in going from Unicode noob to having a comfortable, useful knowledge of many common issues.</li>
<li><a href="https://mathiasbynens.be/notes/es-regexp-proposals">ECMAScript regular expressions are getting better!</a> - This article by a V8 developer at Google shows some nice JavaScript regular expression improvements planned for ES2018, including Unicode property escapes.</li>
<li><a href="https://github.com/LuminosoInsight/python-ftfy">ftfy for Python</a> - ftfy is a Python library that takes corrupt Unicode text and attempts to fix it as best it can. I haven’t yet had a chance to use it, but the examples are compelling and it’s definitely worth knowing about.</li>
</ul>
Series Digital joins End Point!https://www.endpointdev.com/blog/2017/06/series-digital-joins-end-point/2017-06-01T00:00:00+00:00Jon Jensen
<p>End Point has the pleasure to announce some very big news!</p>
<p>After an amicable wooing period, End Point has purchased the software consulting company <a href="http://www.seriesdigital.com/">Series Digital</a>, a NYC-based firm that designs and builds custom software solutions. Over the past decade, Series Digital has automated business processes, brought new ideas to market, and built large-scale dynamic infrastructure.</p>
<p><a href="http://www.seriesdigital.com/"><img align="right" alt="Series Digital website snapshot" src="/blog/2017/06/series-digital-joins-end-point/image-0.png" style="width: 393px; height: 409px; margin-left: 1em"/></a></p>
<p>Series Digital launched in 2006 in New York City. From the start, Series Digital managed large database installations for financial services clients such as Goldman Sachs, Merrill Lynch, and Citigroup. They also worked with startups including Drop.io, Byte, Mode Analytics, Domino, and Brewster.</p>
<p>These growth-focused, data-intensive businesses benefited from Series Digital’s expertise in scalable infrastructure, project management, and information security. Today, Series Digital supports clients across many major industry sectors and has focused its development efforts on the Microsoft .NET ecosystem. They have strong design and user experience expertise. Their client list is global.</p>
<p>The Series Digital team began working at End Point on April 3rd, 2017.</p>
<p>The CEO of Series Digital is Jonathan Blessing. He joins End Point’s leadership team as Director of Client Engagements. End Point has had a relationship with Jonathan since 2010, and looks forward with great anticipation to the role he will play expanding End Point’s consulting business.</p>
<p>To help support End Point’s expansion into .NET solutions, End Point has hired <a href="/team/dan-briones/">Dan Briones</a>, a 25-year veteran of IT infrastructure engineering, to serve as Project and Team Manager for the Series Digital group. Dan started working with End Point at the end of March.</p>
<p>The End Point leadership team is very excited by the addition of Dan, Jonathan, and the rest of the talented Series Digital team: Jon Allen, Ed Huott, Dylan Wooters, Vasile Laur, Liz Flyntz, Andrew Grosser, William Yeack, and Ian Neilsen.</p>
<p>End Point’s reputation has been built upon its excellence in e-commerce, managed infrastructure, and database support. We are excited by the addition of Series Digital, which both deepens those abilities, and allows us to offer new services.</p>
<p><a href="/contact/">Talk to us</a> to hear about the new ways we can help you!</p>