End Point Dev blog

Introducing EP Audit

2026-05-11T00:00:00+00:00

Photo by Bimal Gharti Magar, 2026.

In the complex landscape of cloud security, staying compliant and prepared for audits is important. Enter EP Audit, a compliance-focused audit process, AI-powered but led by senior engineers. It’s designed for Azure, AWS, and Google Cloud setups. EP Audit emerged from necessity: When a financial services client required an Azure security audit, we at End Point built a solution that not only met their immediate needs, but also evolved into an internal tool we continue to refine. Today, it serves as a key part of our multi-cloud security audit framework and is tested against our own cloud accounts before being used in client engagements.

The Problem

Cloud environments are dynamic: New projects, personnel changes, and vendor updates can significantly alter configurations over time. As environments evolve, the security posture originally reviewed and approved by your engineers may no longer fully match what is running in production. Unused resources, overly permissive access, and orphaned infrastructure can also accumulate over time, increasing both risk and monthly costs. This often goes unnoticed until an audit, customer review, or security incident forces a closer look.

Modern compliance frameworks—SOC 2, ISO 27001, HIPAA, PCI DSS—require regular, documented security reviews. Yet many organizations still struggle to provide consistent documentation with timestamps, severity tracking, and proof of remediation. EP Audit helps address this gap by generating a structured audit trail throughout the review process.

What EP Audit Looks Like

An EP Audit engagement is structured as a scheduled review cycle with two phases: a Base scan to establish a security snapshot, and a Final scan performed after remediation work to verify fixes. Both phases use a command-line runner with a read-only audit identity, ensuring no changes are made during the audit itself. Each Base scan produces nine deliverables designed to provide both high-level summaries and detailed technical findings:

Executive Summary: Severity-ranked audit findings.
Client Report: Full narrative with risk scorecard and roadmap.
Structured Findings Report: Engineer-friendly findings by category.
Pre-filled Hardening Checklist: CIS-aligned checklist with auto-populated FAIL statuses.
Fillable Hardening Checklist: A remediation tool for your team.
Network Diagram: Auto-generated deployment topology.
Findings Workbook: Excel file with color-coded severity tabs.
Remediation Plan: Phased plan with affected resources.
Partner Remediation Identity Setup: Guide for temporary write scope during remediation.

These deliverables are standardized across Azure, AWS, and GCP, helping simplify multi-cloud audit reviews and recurring engagements.

Sample summary of findings by severity and category on EP Audit.

Every finding in EP Audit is mapped to controls from recognized security frameworks. For Azure, we reference the CIS Microsoft Azure Foundations Benchmark and the Microsoft Cloud Security Benchmark. AWS findings align with the CIS AWS Foundations Benchmark, AWS Foundational Security Best Practices, and related guidance. GCP findings follow the CIS GCP Foundation Benchmark and the Google Cloud Architecture Framework Security Pillar. This helps keep audit findings grounded in established industry standards rather than proprietary scoring systems or arbitrary recommendations.

A Service, Not Just a Tool

There are already plenty of cloud configuration scanners available. What matters is how the findings are reviewed, prioritized, and applied in real environments.

Senior Engineering Review: Our engineers provide context and actionable recommendations for each finding, bridging the gap between automated results and practical solutions.
End Point-Specific Enhancements: We include checks beyond published frameworks, such as billing data analysis for orphaned resources and third-party security tool integration.
Internal Validation: Every framework change is tested on End Point’s own cloud accounts before client deployment.

A single audit provides a snapshot in time, but recurring audits make it easier to identify trends, validate remediation progress, and detect configuration drift over time. EP Audit includes year-over-year comparison reporting to help highlight both improvements and regressions. In practice, this helps support compliance reviews, procurement questionnaires, and ongoing security review processes while providing better visibility into how environments evolve over time.

Phased Remediation

EP Audit generates remediation scripts and recommendations organized by severity:

P1 (0–7 days): Critical issues like exposed management ports.
P2 (30 days): High-priority issues such as encryption gaps.
P3 (90 days): Medium-priority defense-in-depth weaknesses.
P4 (Maintenance): Low-priority configuration hygiene.

Scripts default to dry-run mode, allowing proposed changes to be reviewed before execution. Our team can assist with remediation directly, or clients can implement the recommendations independently using the provided guidance.

Over multiple engagements, organizations begin building a clearer picture of how their cloud environments evolve over time. Recurring audits make it easier to track remediation progress, identify recurring issues, and detect configuration drift before it becomes a larger problem. The resulting documentation can also help support compliance reviews, customer security questionnaires, cyber insurance requirements, internal security tracking, and reduce monthly costs.

Working with End Point

If your organization operates workloads in Azure, AWS, or Google Cloud and needs a structured, standards-aligned cloud security review process, feel free to contact us. If you are working with us already, you can talk with your representative to find out more.

Designing software architecture with Domain-Driven Design

2026-05-05T00:00:00+00:00

Photo by Juan Pablo Ventoso, 2023.

This is part 3 of a series of blog posts on Domain-Driven Design:

High level system analysis and design with Domain-Driven Design

Implementing business logic with Domain-Driven Design

Designing software architecture with Domain-Driven Design

Domain-Driven Design is an approach to software development that focuses on, as Eric Evans puts it, “tackling the complexity in the heart of software”. And what is in the heart of software? The business domain in which it operates. Or more specifically: a model of it, made of code. That is, the code that implements the business logic that comes into play when solving problems within the realm of a particular business activity.

DDD is not just about writing code though. It’s a whole methodology that touches on business needs, requirements gathering, organizational dynamics, high level architectural design, and lower level patterns for implementing software intensive systems.

As a result, DDD offers a treasure trove of concepts, patterns and tools that can be applied to any software project, regardless of the size and complexity.

In this series of blog posts we’re going to explore many aspects of DDD. We will be following the structure laid out by Vlad Khononov’s excellent book on the topic “Learning Domain-Driven Design: Aligning Software Architecture and Business Strategy”. So you can think of this series as a summary of that book. An abridged version that can serve as a review for anybody who has read it; but also as an entry point for people who are new to DDD.

Designing software architecture with Domain-Driven Design

Section 8: Architectural patterns

Now that we’ve seen various patterns for implementing business logic, i.e. “the heart of software”. We turn our attention to architecture.

Indeed, the business logic is the raison d’être for a software application. But applications have other responsibilities that are also important. Like interacting with users, receiving requests and returning results, storing data, interfacing with external services. In order to balance all these concerns and make sure the code base does not devolve into an unmaintainable big ball of mud, we need to be intentional in how we organize it. We need to design its architecture.

That is, the rules and principles that we follow to organize the various aspects of the code base and create clear boundaries between them. In essence, software architecture is about defining a system’s big logical components, their dependencies and interactions.

In this section we will see three common architectural patterns: layered architecture, ports and adapters, and command query responsibility segregation.

Layered architecture

The layered architecture is one of the most common architectural patterns out there. It has been present, in one form or another, for a long time. The main idea of the pattern is to separate applications into three layers: the presentation layer, the business logic layer, and the data access layer.

Layered architecture.

The presentation layer (or user interface layer) is meant to implement the mechanisms through which consumers interact with the application. This means the app’s graphical user interface (GUI), command line interface (CLI) or application programming interface (API).

The business logic layer (or domain layer) implements the business rules, validation and invariants. This is where we’d implement the patterns that we’ve seen so far like transaction script, active record, or a domain model.

Sometimes, an additional “service layer” or “application layer” emerges between the presentation and business logic layers. When the domain logic needs some level of orchestration, such as it is the case with active records and domain models, it is often useful to further separate the presentation and business logic layers by exposing a sort of “public interface” to the domain. That is, a series of procedures (i.e. transaction scripts) that serve as a facade that maps presentation layer actions (e.g. user interactions) to business domain transactions.

The service layer sits between the presentation and business logic layers. It implements a series of actions which map closely to the operations exposed to users by the presentation layer.

Here’s an example of refactoring a “fat” controller by introducing a service layer:

// This is an MVC controller that implements a REST API endpoint for adding
// items to a shopping cart. It lives in the application's presentation layer
// and leverages business logic layer components to run its logic.

namespace MvcEcommerce.WebApi.Controllers;

[Route("api/[controller]")]
[ApiController]
public class QuoteItemsController : ControllerBase
{
    // ...

    // This action takes care of processing user input, fetching HTTP context
    // values like cookies, responding with proper HTTP status codes and
    // orchestrating the business logic.
    [HttpPost]
    public async Task<ActionResult> Post([FromBody] QuoteItemPost payload)
    {
        var quoteId = _quoteCookieManager.GetQuoteIdFromCookie(Request);
        var quote = await _quoteRepository.FindOpenByIdAsync(quoteId);
        if (quote == null) return NotFound("Quote not found");

        var product = await _productRepository.FindByIdAsync(payload.ProductId);
        if (product == null) return NotFound("Product not found");

        var matchingQuoteItem = quote.GetItemBy(productId: payload.ProductId);
        if (matchingQuoteItem != null) return BadRequest("Item already exists");

        var quoteItem = new QuoteItem()
        {
            Product = product,
            Quantity = payload.Quantity,
        };
        quote.Items.Add(quoteItem);

        await _quoteRepository.UpdateAsync(quote);

        return Ok(quoteItem);
    }
}

This controller does too much. We can move a lot of its logic into a service layer component:

namespace MvcEcommerce.WebApi.Controllers;

[Route("api/[controller]")]
[ApiController]
public class QuoteItemsController : ControllerBase
{
    // ...

    // After refactoring, this action is now much simpler and only concerned
    // with presentation layer issues. That is, handling user input and the HTTP
    // specific aspects of request processing. It relies on the service layer to
    // orchestrate the business logic.
    [HttpPost]
    public async Task<ActionResult> Post([FromBody] QuoteItemPost quoteItem)
    {
        try
        {
            var quoteId = _quoteCookieManager.GetQuoteIdFromCookie(Request);

            var result = await _quoteItemCreator.Run(new() {
                QuoteId = quoteId.Value,
                ProductId = quoteItem.ProductId,
                Quantity = quoteItem.Quantity
            });

            return Ok(result);
        }
        catch (EntityNotFoundException ex)
        {
            return NotFound(new ErrorMessage(ex));
        }
        catch (DomainException ex)
        {
            return BadRequest(new ErrorMessage(ex));
        }
    }
}

// This service object implements all the domain logic orchestration that
// used to live in the controller. Notice how this code is not concerned with
// presentation layer responsibilities like handling HTTP specific logic, for
// example.

namespace MvcEcommerce.ServiceLayer.Services;

public class QuoteItemCreator
{
    // ...

    public async Task<QuoteItem> Run(InputPayload payload)
    {
        var product =
            await _productRepository.FindByIdAsync(payload.ProductId) ??
                throw new EntityNotFoundException("Product not found");

        var quote =
            await _quoteRepository.FindOpenByIdAsync(payload.QuoteId) ??
                throw new EntityNotFoundException("Quote not found");

        var matchingQuoteItem = quote.GetItemBy(productId: payload.ProductId);
        if (matchingQuoteItem != null)
            throw new DomainException("Item already exists");

        var quoteItem = new QuoteItem()
        {
            Product = product,
            Quantity = payload.Quantity,
        };
        quote.Items.Add(quoteItem);

        await _quoteRepository.UpdateAsync(quote);

        return quoteItem;
    }
}

Finally, the data access layer is meant to provide the means of interacting with persistent storage like databases, search indexes, file systems, etc. In modern systems, this layer has evolved into more of a “infrastructure” layer, and taken on the responsibility of interacting with external APIs and other kinds of web services. So, not strictly limited to pure “data storage”.

The communication between these layers is one-way, from top to bottom. Meaning that the presentation layer holds a reference to, depends on, and calls to the business logic layer. Same with the business logic layer to the data access layer.

This communication pattern is excellent for active record and transaction script based systems. For domain models, it begins to fall a bit short. This is because the business logic depending on the data access logic contradicts one of the core principles of domain models: the fact that they are supposed to be plain old objects, with no dependencies on frameworks or infrastructure.

Ports and adapters

The ports and adapters architecture leverages the dependency inversion principle to address the shortcomings of the traditional layered architecture and make it ideal for implementing domain models. Its main advantage is that it decouples the business logic layer from the infrastructure.

The dependency inversion principle dictates that, instead of higher level components directly depending on and referencing lower level ones; it is the lower level components that should depend on the higher level ones. This is done by the higher level components defining contracts for the lower level components to implement, and through those, be integrated into the higher level components’ workflows. The higher level components only ever interact with the lower level ones through the contracts that they themselves define.

Case in point: instead of the business logic depending on data access logic, like in the layered architecture; the business logic layer takes center stage and defines the contracts that the data access layer (and really, all infrastructure) must abide to in order to be usable to the business logic.

And that’s precisely where the ports and adapters name comes from. The business logic defines contracts/interfaces, AKA “ports”; and the infrastructure provides concrete implementations for these interfaces which can talk to external components: the “adapters”. Then, application bootstrapping logic, or dependency injection, takes care of supplying the concrete objects (or adapters) to the abstract interfaces (or ports) that the business logic specifies. This is exactly what a domain model calls for.

Here’s an example of a minimal, thin vertical slice of an application designed using the ports and adapters pattern:

The business logic implements some procedure which necessitates interacting with the database, an email delivery service, and a payment processor:

namespace Ecommerce.BusinessLogicLayer.Services;

public class OrderCreator
{
    // This service does not directly depend on concrete classes. Instead, it
    // references abstract interfaces. The concrete implementations are given
    // via dependency injection through the constructor.
    private readonly IOrderRepository _orderRepository;
    private readonly IOrderConfirmationMailer _orderConfirmationMailer;
    private readonly IPaymentGateway _paymentGateway;

    public OrderCreator(
        IOrderRepository orderRepository,
        IOrderConfirmationMailer orderConfirmationMailer,
        IPaymentGateway paymentGateway
    ) {
        _orderRepository = orderRepository;
        _orderConfirmationMailer = orderConfirmationMailer;
        _paymentGateway = paymentGateway;
    }

    public async Task<Order> Run(InputPayload payload)
    {
        // ...

        var order = new Order(payload);

        var result = _paymentGateway.SubmitPayment(order);
        if (!result.IsSuccess)
            throw new DomainException("Error processing payment");

        await _orderRepository.Save(order);

        await _orderConfirmationMailer.Send(order);

        return order;
    }
}

The business logic layer also defines its ports. That is, interfaces that the infrastructure layer will have to implement:

namespace Ecommerce.BusinessLogicLayer.Interfaces;

public interface IPaymentGateway
{
    PaymentTransactionResult SubmitPayment(Order order);
}

public interface IOrderRepository
{
    Task<Order> Save(Order Order);
}

public interface IOrderConfirmationMailer
{
    Task Send(Order order);
}

The infrastructure layer provides implementations for these interfaces:

namespace Ecommerce.InfrastructureLayer.Payments;

public class AuthorizeNetPaymentGateway : IPaymentGateway
{
    public AuthorizeNetPaymentGateway(/* ... */) { /* ... */ }

    public PaymentTransactionResult SubmitPayment(Order order) { /* ... */ }
}

namespace Ecommerce.InfrastructureLayer.Repositories;

public class OrderRepository : IOrderRepository
{
    public OrderRepository(/* ... */) { /* ... */ }

    public Task<Order> Save(Order Order) { /* ... */ }
}

namespace Ecommerce.InfrastructureLayer.Mailers;

public class OrderConfirmationAwsSesMailer : IOrderConfirmationMailer
{
    public OrderConfirmationAwsSesMailer(/* ... */) { /* ... */ }

    public Task Send(Order order) { /* ... */ }
}

Like mentioned before, everything can be wired together via dependency injection or some other form of bootstrapping. Most frameworks have their own way of resolving dependencies and instantiating service objects like these; in order to execute them as a result of requests from consumers (e.g. a CLI command, a web request, the click of a button). In essence, it’s something like this:

var repository = new OrderRepository(dbContext);
var mailer = new OrderConfirmationAwsSesMailer(mailerConfig);
var paymentGateway = new AuthorizeNetPaymentGateway(paymentConfig);

var orderCreator = new OrderCreator(repository, mailer, paymentGateway);

orderCreator.Run(payload);

So, the ports and adapters architectural pattern has a business logic layer which has no dependencies on any other components outside of itself. It defines a set of interfaces for all external components that want to interact with it. It also has an infrastructure layer which implements concrete classes for the domain layer’s interfaces. Data access, interaction with external services, user interface, and presentation logic all live here. And finally, it has a service/application layer which, similar to its counterpart from the layered architecture, can emerge between the business logic and infrastructure layers when needed to expose a set of procedures that closely map to user interface actions. It exposes all the business operations that the system supports and orchestrates the business logic to carry them out.

When organizing the code following the the ports and adapters architectural pattern, the business logic layer defines interfaces/ports, which the infrastructure layer implements concrete objects for. These objects take care of interacting with the external world.

Clean architecture, onion architecture and hexagonal architecture are all different names for the same core concepts and principles espoused by ports and adapters; sometimes with slight variations depending on the particular flavor and tech stack.

Command query responsibility segregation

The command query responsibility segregation pattern (CQRS) builds on the principles from ports and adapters and adds support for multiple different representations of the system’s data. That is, having multiple persistence models for the same data set. One of the most common examples is a system that stores and operates on day to day business transactions using an OLTP (online transaction processing) representation, but also needs to provide an OLAP (online analytical processing) data warehouse for high level business analysis. One ground-truth source of data (the OLTP) is used to produce additional representations with a different schema (the OLAP). CQRS enables this.

As such, CQRS is ideal for event sourced domain models, because it allows persisting the many projections of an aggregate’s data into their own databases. (Remember that in the context of event sourcing, a projection is a representation of the state of a business entity which is constructed based on its stored domain events). It allows of course, querying of these projections with much more flexibility than what a pure event store allows on its own.

At the core of CQRS there are two types of models: a command execution model (the C in CQRS) and one or many read, or query, models (the Q in CQRS). In database terms, this is similar to a primary-replica type of situation, where the command execution model represents the primary, and the read models represent the replicas.

CQRS exposes two types of models, one for executing commands and many others for reading. In the backend, a projection engine keeps the read models up to date with the latest changes from the command execution model.

The command execution model is the system’s source of truth, whose data is strongly consistent. It’s the one used to execute and record business operations, and enforce business rules and invariants. All operations that result in changes to the system state are handled here.

The query, or read, models take care of exposing different projections of the system state. These are read-only models that are meant for presenting the system’s data to its consumers. They are generated based on the main data source, which is the data maintained in the command execution model. In fact, read models should be capable of being easily destroyed and recreated from the main data.

Read models are generated using components that we call projection engines, which can work synchronously or asynchronously when fetching data from the command execution model to generate their projections.

Synchronously, projection engines generate read models using a catch-up subscription design. It works like this:

The projection engine determines the last query checkpoint. Basically, when was the last time it read the main database.
The projection engine queries the main database and identifies newly added and updated records since the last query checkpoint.
The projection engine uses the new data to regenerate or update the read model.
The projection engine updates the latest query checkpoint, to be used during the next execution of the engine, starting again at step 1.

Synchronous projection engine.

This checkpoint can be implemented in various ways. The main idea is to offer a mechanism for the projection engine to be able to tell which records are new or have changed since its last run. One way to do it is using something akin to SQL Server’s rowversion feature. This is a database-wide, auto-incrementing numeric value that increases after every INSERT and UPDATE operation. The newly incremented value is assigned to the rows that were added or updated. With this, the projection engine can easily query all records whose rowversion is greater than the last checkpoint. Similar functionality can be implemented using database triggers too. Or database tables can also include timestamp fields that indicate when records were last touched.

Asynchronous projection engines on the other hand, rely on the command execution model publishing all changes to a message bus. The projection engine can subscribe to these messages and update its read model as they come. This method, while scalable, comes with drawbacks inherent to distributed computing like handling duplicates and out of order messages. It’s also more difficult to destroy and regenerate read models that rely on asynchronous messaging only, since the messages are gone after they are processed. Often the better solution is to use a synchronous design and then, only if needed, augment it with the asynchrony.

Naturally, CQRS is ideal when the system needs to support different types of databases. Imagine an online store, for example, which has an inventory management component that works directly with the command execution model backed by a relational database. But for displaying the product catalog to users, it uses a read model backed by a search index, optimized for full text search. And for event sourced domain models, CQRS is practically mandatory.

Scope

The patterns that we’ve seen in this section are not exclusively meant as system-wide nor even bounded-context-wide organizational principles. Arbitrarily enforcing a single pattern everywhere often leads to accidental complexity. Instead we should follow DDD’s core principles and deploy these strategies according to what the business domain necessitates. Indeed, within a bounded context, especially one that deals with many subdomains, there should be logical separation between these subdomains. Each resulting module could use a different architectural pattern. In other words, these patterns divide the code base into horizontal slices; and the subdomains can be used to define the vertical slices. This way, a monolithic bounded context can be modularized. This leaves the code base in a good position for future refactoring and further physical separation into distinct bounded contexts in the future (i.e. into separate applications, services and/or processes).

When needed, different architectural patterns can be deployed to different subdomains within the same bounded context.

Section 9: Communication patterns

In the last few sections we’ve discussed how to implement business logic and how to leverage architectural patterns to organize the code within a bounded context. In this section, we will take a higher level view and go beyond the scope of a single bounded context. We will learn about patterns of communication across bounded contexts. In other words, how to integrate them.

Model translation

Back when we talked about integrating bounded contexts, we discussed how they can use different communication strategies depending on the disposition of the teams that own them. When the teams have strong communication, cooperation patterns like partnership and shared kernel can be used for integration. Here, bounded contexts can communicate without friction. The protocols for doing so are defined by the teams in an ad-hoc manner and development moves forward without too many issues.

However, when the teams are not closely aligned, and cooperation patterns are impossible, a customer-supplier relationship emerges. This can be addressed by implementing an anticorruption layer in the downstream consumer, to adapt the upstream supplier’s model to the consumer’s needs. Another option is for the upstream supplier to implement an open-host service and expose an integration-specific published language, which isolates consumers from the details of its internal model. Both approaches are meant to protect a bounded context’s model from the influence of external models.

In these non-cooperation cases, we have to be more intentional in how we design the communication between bounded contexts, and the need for a translation of their models/languages arises. This logic that translates the models can either be stateless or stateful.

Stateless model translation

Stateless model translation can happen on the fly via the proxy design pattern. The idea is to put an intermediary component in place, which receives messages in one language, translates them, and forwards them to their destination.

If the proxy is implemented by the consumer or downstream component, functioning as an anticorruption layer, then it intercepts outgoing requests and translates them to a language that the upstream supplier can understand. It does the same with the incoming responses from upstream: it translates them to a language that the consumer can understand.

If the proxy is implemented by the supplier or upstream component, it functions as an open-host service. It takes the incoming requests that arrive using the published language and translates them to the supplier’s internal one. Then, outgoing responses get translated to the published language.

In essence, a proxy is nothing more than a component that sits between two other components and translates the messages passing between them.

These proxies can be synchronous or asynchronous.

Synchronous translation is straightforward. Messages are translated just as they are being received or sent, depending on whether the translation happens upstream or downstream. Normally, the translation logic is implemented directly in the bounded context that needs it.

In some cases though, it makes sense to separate it into its own independent component, implementing an API gateway pattern. Sometimes, off the shelf software or cloud services like KrakenD or AWS API Gateway can be used to implement them.

Having a separate API gateway has some advantages, other than the obvious decoupling of model translation logic from actual business logic. It can make it easier to expose multiple versions of the API. That is, of the published language. It can also be consumed by multiple downstream components, making it essentially an integration-specific bounded context. We call these, interchange contexts.

When extracted into its own component, a proxy becomes an API gateway. This is useful for further separation of concerns, offering multiple versions of the API’s published language, and serving multiple consumers.

Asynchronous translation on the other hand, is not direct. It relies on an event driven design to implement a message proxy; which subscribes to events published by one bounded context, translates them, and forwards them to their destination.

The proxy can also apply filtering to the messages, deciding which ones to forward and which ones to ignore. This is useful, for example, to keep the published language free of domain events that are meant to be internal to the bounded context that produces them.

An asynchronous proxy listens to events in one language and translates them to another language.

Stateful model translation

Stateful model translation comes into play when more complex translation logic that requires persistent storage is needed. This is useful for two common scenarios: when the requests need to be aggregated, and when data from multiple sources needs to be unified. In both cases the component doing the translation needs to keep track of the incoming requests and messages. It stores them in a database in order to reconstruct them eventually.

Data aggregation may be needed for performance reasons when, for example, multiple requests need to be collected together and dispatched as one batch. Another case is when multiple disparate messages need to be combined into one single bigger message that conveys more complete information. This can work both synchronously and asynchronously. That is, the messages can be direct requests or published events.

An API gateway on its own is not appropriate for this type of model translation, as it doesn’t provide persistent storage or more complex processing logic beyond mapping and rerouting. Instead, it can be implemented from scratch as a bespoke solution, or leveraging off the shelf stream processing platforms like Apache Kafka or AWS Kinesis.

A proxy can aggregate and/or combine multiple separate messages into a single one for performance and consolidation purposes.

The unification of multiple data sources is also a common use case for stateful model translation. This is necessary for example when a component needs to process data from many different upstream suppliers and apply complex business logic to it. This is commonly seen in the backend for frontend pattern. In this pattern, an API is defined to meet all of a frontend’s needs, while serving as a facade for a number of backend services. This allows the frontend to call a single backend to interact with all the various services it may need.

Integrating aggregates

Before, we talked about the rules that we follow to limit the scope of aggregates and ensure they encapsulate a coherent set of business logic. We strive to keep them focused, with tight boundaries. They represent strong data consistency and transactional boundaries. That is, they should only contain strongly consistent data and the system’s transactions should be limited to a single aggregate instance. It is also true however, that there are business processes that involve multiple aggregates. The following patterns allow us to implement such business processes without breaking the aggregates’ isolation rules.

Outbox

As we know, along with commands, domain events are part of an aggregate’s public interface. By publishing domain events to a message bus, aggregates communicate with the outside world. Or more specifically, to their subscribers, who listen to these events in order to trigger their own logic. The outbox pattern allows aggregates to reliably publish domain events.

The outbox pattern works by storing domain events in the same database as the aggregate’s data. Leveraging database transactions to ensure both the state changes and the events are saved atomically. The process is this:

Aggregate state changes and new events are committed in the same database transaction.
The message relay picks up the new events from the database.
The message relay publishes the new events to the message bus.
After successful publishing, the message relay marks the event as such, or deletes it.

The outbox pattern is about storing an aggregate’s domain events in a database, making sure to commit them in the same transaction as state changes.

Saga

A saga leverages the reliable publishing of domain events afforded to us by the outbox pattern to implement business processes that span multiple aggregates. They do this by listening to events and responding to them by issuing commands to relevant components. A saga represents a long-running business process. One that spans multiple transactions.

Sagas can vary in complexity. Sometimes it is enough to respond to events by directly issuing commands from the saga itself.

public class OrderProcessingSaga
{
    private readonly IOrderRepository _orderRepository;
    private readonly IPaymentService _paymentService;
    private readonly IInventoryService _inventoryService;
    private readonly IShippingService _shippingService;

    //...

    // These event handler methods all follow a similar pattern:
    // Receive the event, then call the corresponding command.
    public void Process(OrderCaptured @event)
    {
        var order = _orderRepository.Fetch(@event.OrderId);

        _paymentService.ProcessPayment(order);
    }

    public void Process(PaymentAccepted @event)
    {
        var order = _orderRepository.Fetch(@event.OrderId);

        _inventoryService.PreparePackage(order);

        order.UpdateStatus(@event.NewOrderStatus);
        _orderRepository.Save(order);
    }

    public void Process(InventoryCleared @event)
    {
        var order = _orderRepository.Fetch(@event.OrderId);

        _shippingService.ScheduleDelivery(order);

        order.UpdateStatus(@event.NewOrderStatus);
        _orderRepository.Save(order);
    }

    public void Process(ShipmentDispatched @event)
    {
        var order = _orderRepository.Fetch(@event.OrderId);

        order.UpdateStatus(@event.NewOrderStatus);
        _orderRepository.Save(order);
    }
}

Other times, a saga needs to keep its own state. For example, when the saga needs to track received events; to react to certain failure conditions; issue retries, etc. In these cases, the saga can be implemented as an event sourced aggregate, keep a record of all received events, and publish events of its own that contain the commands that it wants to execute. Then, a separate message relay component should be listening to the saga’s events and run the commands contained within.

public class OrderProcessingSaga
{
    // Like any other event sourced aggregate, this saga stores its list of
    // events.
    private readonly IList<DomainEvent> _events = [];

    // ...

    // These event handlers all also follow a similar pattern. Instead of
    // directly calling the commands though, they register an event with the
    // necessary information for the message relay to call the commands.
    public void Process(OrderCaptured @event)
    {
        // ...

        _events.Append(@event);
        _events.Append(new CommandIssuedEvent(
            target: Target.PaymentService,
            command: new ProcessPaymentCommand(@event.OrderId)
        ));
    }

    public void Process(PaymentAccepted @event)
    {
        // ...

        _events.Append(@event);
        _events.Append(new CommandIssuedEvent(
            target: Target.InventoryService,
            command: new PreparePackageCommand(@event.OrderId)
        ));
    }

    public void Process(InventoryCleared @event)
    {
        // ...

        _events.Append(@event);
        _events.Append(new CommandIssuedEvent(
            target: Target.ShippingService,
            command: new ScheduleDeliveryCommand(@event.OrderId)
        ));
    }

    public void Process(ShipmentDispatched @event)
    {
        // ...

        _events.Append(@event);
    }
}

Sagas represent a long running business process. One that spans many transactions. They receive events and invoke commands as a result.

Always remember though, sagas make it possible to implement business processes that span multiple aggregates, but the data across aggregates is only eventually consistent. The core aggregate design rules still apply: only the data within an aggregate is strongly consistent.

Process manager

The process manager pattern is essentially a more complex saga. While sagas mostly deal with linear flows, and direct mapping between events and commands; a process manager implements more complex business logic that keeps track of the sequence of events, has its own state, and determines how to react. It does not only have a simple mapping of events to commands. Instead it has complex logic that decides how to respond to incoming events. It is a central processing unit, listening to events from and issuing commands to multiple sources, and managing an overall business process.

Another difference between them is that sagas are instantiated implicitly, while process managers are instantiated explicitly. Meaning that sagas live and die within the context of the events that they listen to. They “get activated” when an event that they are interested in is published. Process managers on the other hand get activated when the business process they manage gets initiated. They stay alive through the duration of the process’s many steps.

Implementation wise, they are essentially aggregates with a strong focus on responding to events. They have their own persistent state, explicit identity, lifecycle, and related objects; as well as a series of event handlers, just like a saga would have.

A process manager orchestrates a long running business process with complex logic that involves many components. The application layer instantiates them explicitly, as it would any other aggregate.

public class OrderProcessManager
{
    private readonly IList<DomainEvent> _events = [];

    // Process managers have their own explicit identity. They are very similar
    // to aggregates or entities.
    private OrderId _id;

    // They can include other properties to track state as needed.

    // ...

    // Process managers usually include an initialization procedure that kicks
    // off the process, and provides the necessary parameters.
    public void Initialize(/* ... */)
    {
        // ...

        _events.Append(orderProcessingInitiated);
        _events.Append(new CommandIssuedEvent(
            target: Target.PaymentService,
            command: new ProcessPaymentCommand(@event.OrderId)
        ));
    }

    // The event handlers are very similar to those of the event sourced
    // stateful saga. The difference is that here they run more complex logic to
    // determine how to continue the process. As opposed to the saga's simpler
    // mapping of incoming events to commands.
    public void Process(PaymentAccepted @event)
    {
        // ...

        _events.Append(@event);
        _events.Append(new CommandIssuedEvent(
            target: Target.InventoryService,
            command: new PreparePackageCommand(@event.OrderId)
        ));
    }

    public void Process(InventoryCleared @event)
    {
        // ...

        _events.Append(@event);
        _events.Append(new CommandIssuedEvent(
            target: Target.ShippingService,
            command: new ScheduleDeliveryCommand(@event.OrderId)
        ));
    }

    public void Process(ShipmentDispatched @event)
    {
        // ...

        _events.Append(@event);
    }
}

Architectural patterns to use within bounded contexts

These are the main concepts around the architectural patterns that we’ve discussed.

Patterns for communicating across bounded contexts

These are the main concepts around the communication patterns that we’ve discussed.

Implementing business logic with Domain-Driven Design

2026-04-21T00:00:00+00:00

Photo by Bimal Gharti Magar, 2026.

This is part 2 of a series of blog posts on Domain-Driven Design:

High level system analysis and design with Domain-Driven Design

Implementing business logic with Domain-Driven Design

Designing software architecture with Domain-Driven Design

As a result, DDD offers a treasure trove of concepts, patterns and tools that can be applied to any software project, regardless of the size and complexity.

Implementing business logic with Domain-Driven Design

Section 5: Implementing simple business logic

Now that we’ve explored DDD’s higher level system design concepts, it’s time to zoom in and start looking at how to implement business logic: the most important part of software. We will start by discussing two patterns that are ideal for implementing simple business logic: transaction script and active record.

Transaction script

According to Martin Fowler, the transaction script pattern “organizes business logic by procedures where each procedure handles a single request from the presentation.”

For most programmers that have experience with procedural languages, the transaction script is easy to grasp. This pattern is about conceptualizing a system as a collection of transactions. And organizing these transactions as independent, transactional procedural scripts. Think one transaction script per use case, exposed for the users to invoke when they need to.

The core idea of transaction script is about putting business transactions front and center. Closely related scripts can be implemented as methods in a service class. Or you could also have a separate “service object” class for each script.

Indeed, on its own, the transaction script pattern is a simple way of organizing relatively simple domain logic. However, as we’ll see throughout the next few sections, it is also a foundational pattern that is present when implementing the more advanced ones. Also, even though the concept is simple, implementation of transaction script in the real world has to be done carefully, in order to avoid falling into common pitfalls, often related to ensuring atomicity.

class AddItemToQuote
{
    // ...

    // This method adds a new item to a shopping cart and records the operation
    // into a log table. Both operations are done separately.
    public void Run(int quoteId, int productId, int quantity)
    {
        _db.ExecuteSql($"""
            INSERT INTO quote_items(quote_id, product_id, quantity)
            VALUES ({quoteId}, {productId}, {quantity});
        """);

        // If some error happens after the previous command and before the one
        // below, the system will be left in an inconsistent state. The new item
        // would have been added to the shopping cart, but the audit log would
        // not have a corresponding event.

        _db.ExecuteSql($"""
            INSERT INTO quote_audit_log(
                quote_id, event_description, date_created
            )
            VALUES ({quoteId}, 'Item added to quote', CURRENT_DATE);
        """);
    }
}

That is, a transaction script needs to actually be transactional. It needs to be atomic. Of course, each domain’s requirements will dictate how true this is, but in general, we want transaction scripts to operate as a unit, and make sure that they don’t leave the system in an inconsistent state in situations when the script fails midway through its execution.

If all the script does is interact with a relational database, it is easy to address this issue. The solution is to perform the operations within a database transaction:

class AddItemToQuoteWithTransaction
{
    // ...

    public void Run(int quoteId, int productId, int quantity)
    {
        try
        {
            // We can use a database transaction to fix the issue.
            _db.BeginTransaction();

            _db.ExecuteSql($"""
                INSERT INTO quote_items(quote_id, product_id, quantity)
                VALUES ({quoteId}, {productId}, {quantity});
            """);

            _db.ExecuteSql($"""
                INSERT INTO quote_audit_log(
                    quote_id, event_description, date_created
                )
                VALUES ({quoteId}, 'Item added to quote', CURRENT_DATE);
            """);

            _db.CommitTransaction();
        }
        catch
        {
            _db.RollbackTransaction();
            throw;
        }
    }
}

The problem gets more complicated when the script performs a distributed transaction. That is, when it interacts with other systems which are outside of the scope of a relational database transaction. For example, interacting with the file system, or with an external web service.

class AddItemToQuoteWithDistributedTransaction
{
    // ...

    public void Run(int quoteId, int productId, int quantity)
    {
        _db.ExecuteSql($"""
            INSERT INTO quote_items(quote_id, product_id, quantity)
            VALUES ({quoteId}, {productId}, {quantity});
        """);

        // We're back to our initial problem. Any failure at this point in the
        // script will have updated the shopping cart without having sent the
        // notification to the external system.

        _api.NotifyQuoteItemAdded(quoteId, productId, quantity);
    }
}

These are hard to deal with and often require very particular code to handle properly. Further in this series we’ll discuss this topic again when we talk about CQRS and the outbox pattern.

Sometimes though, the distributed transaction is not as obvious. Consider a web application that exposes an endpoint to increment the quantity of particular shopping cart items by one:

class IncrementQuoteItemQuantity
{
    // ...

    public void Run(int quoteItemId)
    {
        _db.ExecuteSql($"""
            UPDATE quote_items
            SET quantity = quantity + 1
            WHERE id = {quoteItemId};
        """);
    }
}

Even though that’s a single database operation, there is inter-system communication between the user’s browser and the application server; and between the application server and the database. A network outage, for example, that happens in the middle of this operation can leave the system in an inconsistent state. If it fails after it’s already submitted the UPDATE command to the database, and responds with a failure message to the client, the client might attempt to try and increase the quantity again. This would result in the increment being done two times, when actually it should have been only one.

In the context of client-server applications, some transactions are distributed even though at first glance they might not seem that way. In this simple operation, there are three systems interacting over the network: The client, the application server and the database.

There are two possible ways of handling this type of situation: making the operation idempotent, or using optimistic concurrency control.

One way to make the operation idempotent, that is, to make sure that it always produces the same results, no matter how many times it’s done, is to have the caller specify the quantity to update the item to. That is, changing the operation from “add one quantity”, to “set the quantity to this”. Like so:

class IncrementQuoteItemQuantityIdempotent
{
    // ...

    public void Run(int quoteItemId, int quantity)
    {
        _db.ExecuteSql($"""
            UPDATE quote_items
            SET quantity = {quantity}
            WHERE id = {quoteItemId};
        """);
    }
}

On the other hand, implementing optimistic concurrency control in this scenario could be done by effectuating the record update while using the current state of the record when checking for a match. This boils down to including the expected values of the different fields in the WHERE clause. This way, the record will be updated only if it is in the state that the caller expected it to be:

class IncrementQuoteItemQuantityOptimisticConcurrency
{
    // ...

    public void Run(int quoteItemId, int quantity)
    {
        _db.ExecuteSql($"""
            UPDATE quote_items
            SET quantity = quantity + 1
            WHERE id = {quoteItemId} AND quantity = {quantity};
        """);
    }
}

In both cases, the caller would have to obtain the current state of the record before attempting the update operation.

Active record

Martin Fowler describes the active record pattern as “an object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data.”

The main advantage of the active record pattern is that it greatly simplifies database access. Especially when leveraging ORM frameworks. With active record, you usually end up with a set of classes that closely mirror your database structure. You have one class per table, where the fields of those classes represent table columns and the instances of those classes represent individual records in those tables.

So, when the underlying data model is complex, business logic organized in transaction scripts can be augmented with active record to reduce a great deal of the complexity involved in database interactions. This is because the active record pattern takes care of mapping between database records and in-memory objects, as well as all the data retrieval and manipulation commands (I.e. the CRUD). This allows the transaction scripts to focus more on domain logic, and less on manipulating the underlying data.

class AddItemToQuoteActiveRecord
{
    // ...

    public void Run(int quoteId, int productId, int quantity)
    {
        // Active record allows us to work with objects instead of directly
        // issuing database commands.
        var item = new QuoteItem
        {
            QuoteId = quoteId,
            ProductId = productId,
            Quantity = quantity
        };

        item.Save();
    }
}

Of course, active records can (and should) also include domain logic. Aspects like the relationships between the domain entities, validation, complex database queries, and even the business procedures that involve them are all part of an active record object.

# This class is an Order active record. It establishes relationships with other
# entities, defines basic validation rules, and has custom queries.
class Order <  ApplicationRecord
    self.table_name = "orders"

    enum :status, [ :pending, :paid, :shipped, :cancelled ]
    enum :payment_method, [ :cash, :credit_card, :transfer ]

    has_many :order_items
    has_one :shipping_address
    has_one :billing_address

    validates :email, presence: true, email: {mode: :strict}
    validates :phone, length: {minimum: 10, allow_blank: true}

    scope :sorted, -> { order(created_at: :desc) }
    scope :new_ones, -> { where("created_at >= ?", 1.day.ago) }
    scope :have_phone, -> { where.not(phone: nil) }
    scope :where_email_is, ->(email) { where("lower(email) = ?", email.downcase) }
end

# The framework augments it with data access logic, like querying:
order = Order.find(123)
expensive_items = order.order_items.where("price >= ?", 200)
orders_from_us = Order.where(country_code: "US")
new_orders = Order.new_ones.sorted

# ...and making changes:
new_order = Order.new(email: "test@email.com")
new_order.save

order.update(
    shipping_number: "1234567890",
    status: :shipped
)

The Active Record framework from Ruby on Rails is a great implementation of the pattern that puts business logic in a centralized location while magically extending the objects with all the data access logic they need.

This is a big step up from raw transaction scripts, but still it has its own disadvantages. First, the active records can end up having too much responsibility, and grow to unmanageable sizes, especially for core business entities. Second, the active record pattern by itself does not enforce access control on its properties. Meaning that external processes can freely modify their state, potentially ignoring any business rules that could bind them.

Something else to be aware of is that when active records define little to no behavior, that is, when they don’t implement business logic, they become an anemic domain model. Active record objects that focus only on database access tasks, and higher level “service objects” that manipulate them and implement all the domain logic themselves are the hallmark of anemic domain models.

An anemic domain model may look like a full fledged domain model, with all the objects representing domain concepts and their relationships, but they are hollow. Their most important part is missing. In the next section we will see what a real domain model looks like, DDD’s definitive pattern for implementing business logic in complex subdomains.

When to use them

In some circles, the transaction script and active record patterns are considered anti patterns. But really, they are just tools for the job. When misapplied, they become detrimental, but when used to solve the problems they are good at, they shine. In fact, they can give you a lot of bang for your design buck. But when the domain logic you’re implementing is very complex, they can begin to fall short, as their relatively low level of abstraction becomes insufficient and prone to code repetition, and inconsistencies when this repeated code goes out of sync. Which is a big problem when dealing with complex subdomains.

Section 6: Tackling complex business logic

When implementing complex business logic, the patterns that we’ve seen up to this point can only get you so far. They start to leave a lot to be desired due to their relatively low level of abstraction. When the situation calls for a higher level of abstraction, in order to produce a more supple design, DDD calls for the domain model pattern.

Domain model

Taking from Martin Fowler’s definition, we learn that the domain model is a full object-oriented model of the domain, that incorporates both behavior and data. Domain models take the form of a large web of interconnected objects, where each one represents a meaningful concept in the business domain.

In his book, Eric Evans expanded greatly on this definition, introducing a set of patterns and tools for implementing domain models. Indeed, to build a domain model, we incorporate other building block patterns: value objects, entities, aggregates, domain events, domain services.

Also, there are two main rules that a domain model needs to follow:

There should be no dependency on particular frameworks or infrastructure. Objects in the domain model should be plain old objects, focused only on domain logic.
The domain model should speak the ubiquitous language of the bounded context in which it operates. That means that all identifiers should call back to business concepts and it should represent the mental model of the domain experts.

Value object

A value object is an object whose identity is given by its properties. That is, its value. It does not have an explicit identifier, like an Id field. Consider for example a Point object:

class Point
{
    double _x;
    double _y;
}

The X and Y values completely define a point. Two instances of this class with the same X and Y coordinates represent the same point. Changing the value of either coordinate produces a new point.

They are useful for representing properties of other objects and are usually implemented as immutable objects. One of the great strengths of value objects is that they allow the model to speak the ubiquitous language by replacing primitives with bespoke small objects that make the code clearer and encapsulate related business logic.

As an example, consider this Order class.

class Order
{
    private int _id;
    private string _status;
    private string _email;
    private string _phone;
    private double _shippingWeight;
    private string _countryCode;

    public Order(/*...*/) {/*...*/}
}

// This class can be instantiated like this:
var order = new Order(
    id: 12345,
    status: "Processing",
    email: "test@email.com",
    phone: "123-456-7890",
    shippingWeight: 10,
    countryCode: "US"
);

Here, all the values are assigned by convention. An email, a phone number, a status… They are all just strings with no special behavior or meaning. We must know what they look like beforehand in order to assign them correctly.

If we use value objects, this class could be implemented like this instead:

class Order
{
    private OrderId _id;
    private OrderStatus _status;
    private PersonName _name;
    private EmailAddress _email;
    private PhoneNumber _phone;
    private Weight _shippingWeight;
    private CountryCode _country;

    public Order(/*...*/) {/*...*/}
}

// Now, instances can be created like this:
var order = new Order(
    id: new OrderId(12345),
    status: OrderStatus.Processing,
    name: new PersonName("Kevin", "Campusano"),
    email: EmailAddress.Parse("test@email.com"),
    phone: PhoneNumber.Parse("1234567890"),
    shippingWeight: Weight.FromLbs(10.25),
    country: CountryCode.Parse("US")
);

This approach has many advantages:

First of all the Order class does not have to validate its fields. Validation can happen in the value objects. This is good because it allows other domain objects to have fields of the same type without having to duplicate the validation logic themselves. Imagine you also have a contact object somewhere in your model that includes a phone number field, for example. Both it and Order can reuse the phone number value object, and the logic it carries.

Secondly, value objects can capture the business logic that’s closely related to them. A phone number field, for example can implement methods to obtain further information about it like its area code or the country it belongs to. A “weight” value object can implement logic for converting from one measuring system to another.

// Here's a weight value object that encapsulates logic like converting from one
// unit to another and comparing the magnitude of different values.
class Weight : IComparable<Weight>, IEquatable<Weight>
{
    const double LbsPerKg = 2.20462;

    double _lbs;

    private Weight(double lbs) { _lbs = lbs; }

    public static Weight FromLbs(double lbs) => new Weight(lbs);
    public static Weight FromKgs(double kgs) => new Weight(kgs * LbsPerKg);

    public double ToLbs() => _lbs;
    public double ToKgs() => _lbs / LbsPerKg;

    public int CompareTo(Weight? other) => ToLbs().CompareTo(other?.ToLbs());
    public bool Equals(Weight? other) => ToLbs().Equals(other?.ToLbs());
}

Finally, using value objects lets the model speak the ubiquitous language, and thus makes it clearer. Strongly typing properties in this way, beyond just using language primitives, very clearly captures the intent of the property.

Entities

Entities are objects that represent the concepts in the domain that have a lifecycle and explicit identification. A person, an order, a lead, a transaction. These are all examples of entities. Entities, as opposed to value objects, are not immutable, and are expected to change throughout their life in the system. Value objects, like we saw before, are ideal for representing properties of entities.

Entities are a core building block for a domain model. However, they are not used independently. They are used as part of an aggregate.

Aggregates

An aggregate is a hierarchy of entities and value objects that are bound together by closely related business logic. The aggregate forms a boundary that protects the consistency of the objects that compose it. It achieves this by preventing external objects from directly modifying its state and defining a public interface through which other parts of the system can interact with it.

The rest of the system cannot directly mutate the state of the entities within an aggregate. They can only do so via the methods exposed by the aggregate’s public interface. These methods, so called commands, encapsulate the aggregate’s business logic and protect them from corruption, by enforcing the necessary validations and invariants. Indeed, all the business logic that’s closely related to the aggregate lives in one place: the aggregate itself.

// Here we have an aggregate that represents a shopping cart and its items.
public class Quote
{
    // It does not expose its list of items publicly. Instead, it implements
    // public methods for manipulating them.
    IList<QuoteItem> _items = [];

    QuoteItem? GetItemBy(ProductId productId) =>
        _items.FirstOrDefault(i => i.ProductId == productId);

    public void AddItem(ProductId productId, int quantity)
    {
        var item = GetItemBy(productId);

        if (item != null)
            throw new InvalidOperationException("Item already exists.");

        _items.Add(new QuoteItem
        {
            ProductId = productId,
            Quantity = quantity
        });
    }

    // Even though this method modifies an item, and not the quote itself, it's
    // still implemented here, because this is the aggregate root.
    public void SetItemQuantity(ProductId productId, int quantity)
    {
        var item = GetItemBy(productId) ??
            throw new InvalidOperationException("Item does not exist.");

        item.Quantity = quantity;
    }

    // ...
}

These commands, which are the public interface of an aggregate, should all be defined in a single entity within the aggregate. We call this entity the aggregate root. If the aggregate is a hierarchy of objects, and we can picture it as a tree, then the root is the object that exists at the root of the tree, where all branches come from.

The aggregate is a hierarchy of objects. The aggregate root is the sole object in this hierarchy with which other components interact.

This focus on commands makes the implementation of the application layer components more straightforward. I.e. they become transaction scripts. By “application layer components”, I mean those components that orchestrate calls to the domain model in order to fulfill use cases in response to, say, user requests. They follow a general pattern of:

Load the aggregate. Typically from persistent storage.
Invoke the desired command.
Persist the new state of the aggregate.

For example:

// Here's an application service leveraging the quote aggregate to add an item
// to a shopping cart.
class AddItemToQuote
{
    // ...

    public Result Run(QuoteId quoteId, ProductId productId, int quantity)
    {
        try
        {
            var quote = _quoteRepository.FindById(quoteId);
            if (quote == null) return Result.Error("Quote not found.");

            quote.AddItem(productId, quantity);

            _quoteRepository.Save(quote);

            return Result.Success();
        }
        // Notice how the application layer logic takes care of database related
        // errors. Remember that the aggregate is a plain old object which has
        // no knowledge about infrastructure or frameworks. The data access
        // logic, in this case encapsulated in the _quoteRepository, is the one
        // that will produce such errors when trying to commit changes to the
        // underlying storage.
        catch (ConcurrencyException ex)
        {
            return Result.Error(ex);
        }
    }
}

There are other rules that aggregates need to follow. One of them is that the aggregate acts as a transaction boundary for aggregate operations. That is, all changes to an aggregate should be transactional, atomic. Also, no system operation should involve a transaction that includes different aggregates. We should have one aggregate per database transaction.

This reveals another aspect of aggregates: An aggregate should expect strong consistency only on its own objects. For objects that are outside of the aggregate, eventual consistency should suffice. Or, looking at it from a different angle, this means that when designing an aggregate, data consistency is a guiding principle. The data that needs to be strongly consistent in order to fulfill the business requirements, should be included in the aggregate. The data that can be eventually consistent and still meet the requirements, probably belongs in a different aggregate.

At first glance, all these rules for aggregates may seem overly limiting. But the main idea is to keep their scope as constrained as possible, to prevent them from growing too much and taking on too many responsibilities. We should strive to keep aggregates small, highly cohesive, and decoupled from other system components. That unlocks the ability for them to be reorganized and reused in many ways. This helps avoid code duplication when fulfilling the requirements of today while also reducing the cost of evolving to meet the requirements of tomorrow.

Domain event

Through their commands, the outside world can send messages to aggregates. Domain events are the mechanism through which aggregates can themselves send messages to the outside world. As their name suggests, domain events are messages that describe important events that have happened in the business domain, related to an aggregate. Think “order placed”, “user registered” or “product out of stock”. The events should provide all necessary data that allows consumers to understand what has happened.

// This JSON data describes the event of an item being added to a shopping cart.
// It includes all the details that subscribers need to know about the event.
{
    "quote-id": "f3774200-9e57-4ad2-9d93-ffb9e92b8364",
    "event-id": 123,
    "event-type": "item-added-to-quote",
    "event-time": 1628970815,
    "product-id": "7e7aee52-9aa1-4e0e-810e-666cedab5a7a",
    "quantity": 1
}

public class Quote
{
    // ...

    IList<DomainEvent> _domainEvents = [];

    public void AddItem(ProductId productId, int quantity)
    {
        // ...

        // This command creates the event and adds it to the aggregate's list.
        _domainEvents.Add(new ItemAddedToQuote(
            quoteId: _id,
            productId: productId,
            quantity: quantity
        ));
    }
}

Domain events are also part of an aggregate’s public interface. Just like its commands. Other parts of the system can subscribe to these events, and when they happen, react accordingly. We will learn more about domain events, and see how to push them to subscribers, later in the series.

Domain service

Sometimes there’s business logic that doesn’t belong to a particular aggregate or value object, or that involves multiple aggregates. Domain services can be implemented in these cases. Domain services are simple stateless objects that implement some business logic. Somewhat like an aggregate’s command, but defined outside of an aggregate.

// Domain services are a good solution for implementing logic that orchestrates
// calls to different system components.
class PlaceOrder
{
    // ...

    public Result Run(QuoteId quoteId)
    {
        try
        {
            var quote = _quoteRepository.FindById(quoteId);
            if (quote == null) return Result.Error("Quote not found.");

            quote.Close();
            _quoteRepository.Save(quote);

            var order = new Order(quote);
            order.PlaceOrder();
            _orderRepository.Save(order);

            // We can imagine the above code producing an "order placed" event,
            // and a separate payment processing component picking it up and
            // getting to work. Then, inventory and shipping components could
            // continue processing the order after the payment is successful.

            return Result.Success();
        }
        catch (ConcurrencyException ex)
        {
            return Result.Error(ex);
        }
    }
}

Of course, the rule of “modify only one aggregate per transaction” still applies. Even for domain services that orchestrate business operations that involve multiple aggregates. Remember, if strong consistency is needed across separate aggregates, and thus they have operations that need to be executed within the same transaction, then maybe these objects should be part of the same aggregate in the first place.

It’s also important to consider domain services as a sort of last resort. A final domain modeling tool to use only when the other tools like aggregates, value objects, commands and domain events, fall short and truly can’t meet the requirements on their own.

Data access concerns

As we’ve stated at the beginning, the domain model should be made of plain old objects. That means no dependency on framework components or particular infrastructure, just pure domain logic. Data access logic is one of those things that it should be oblivious about.

In practical terms, your choice of technology stack and software development framework will usually dictate the mechanisms you use for interacting with persistent data storage. But that doesn’t matter to the domain model, with the correct abstraction, it should be compatible with any data access mechanism. Here are a few patterns worth mentioning:

We’ve already seen active record being used as a data access strategy. Particularly useful when paired with an ORM framework. Due to its nature of tieing infrastructure concerns (i.e. data access logic) with the business logic, you have to jump through some hoops to make it work with a domain model. But it can be done. The repository pattern is also a good fit for solving the data access problem in the context of DDD. It offers a clear, intention-revealing interface for data retrieval and modification. Unit of work is also a pattern worth looking into, for coordinating numerous database operations.

But the main takeaway is this: the domain model does not concern itself with data access, or frameworks, or infrastructure. So make sure to keep it plain and use abstractions to keep it unconcerned.

Section 7: Modeling the dimension of time

The event sourced domain model is a further evolution of the domain model which incorporates the dimension of time. By leveraging domain events as the source of truth for system data, it allows for a model that can provide deeper insight into the data, rich audit logging, and visibility into the state of the aggregates and entities at any previous point in their lifecycle.

Data storage and retrieval

The main characteristic that differentiates event sourcing from a traditional domain modeling is how the data that represents the aggregates is persisted. Instead of persisting the aggregate’s current state, event sourced domain models persist the domain events produced by the aggregates. These domain events are generated as a result of any operation that changes the state of the aggregate. Then, to obtain the current state of the aggregates, all the events are retrieved from storage and used to reconstruct the full object in memory.

In an event sourced domain model, aggregates produce events and commit them to the event store. To instantiate aggregates then, the same events are fetched from the event store and used to rehydrate the in-memory objects.

For example, in a database that backs an order processing system, an orders table might look like this:

id	status	email	phone	shipping_weight	country_code	created_at	updated_at
1	pending	john.doe@example.com	555-0101	2.5	US	2025-01-10	2025-01-10
2	paid	sarah.smith@example.com	555-0102	5.3	CA	2025-01-09	2025-01-11
3	shipped	mike.johnson@example.com	555-0103	1.8	GB	2025-01-08	2025-01-12
4	pending	emma.wilson@example.com	555-0104	3.2	AU	2025-01-11	2025-01-11
5	cancelled	david.brown@example.com	555-0105	4.7	US	2025-01-07	2025-01-13

Here, each row represents an order and their current state. The database schema resembles the order domain entity.

With event sourcing, what we persist into our database is a series of events that capture the full history of each order, as they journey through the system:

{
    "order_id": 123,
    "event_id": 0,
    "event_type": "order_created",
    "timestamp": "2025-01-10T10:00:00Z",
    "data": {
        "email": "john.doe@example.com",
        "phone": "555-0101",
        "country_code": "US",
        "status": "pending",
        // ...
    }
},
{
    "order_id": 123,
    "event_id": 1,
    "event_type": "payment_failed",
    "timestamp": "2025-01-11T12:00:00Z",
    "data": {
        "status": "payment_failed",
        "reason": "insufficient_funds"
    }
},
{
    "order_id": 123,
    "event_id": 2,
    "event_type": "payment_details_updated",
    "timestamp": "2025-01-11T12:05:00Z",
    "data": {
        "status": "pending",
        "payment_method_nonce": "new-nonce-xyz",
    }
},
{
    "order_id": 123,
    "event_id": 3,
    "event_type": "payment_succeeded",
    "timestamp": "2025-01-11T12:05:00Z",
    "data": {
        "status": "paid"
    }
},
{
    "order_id": 123,
    "event_id": 4,
    "event_type": "inventory_stock_updated",
    "timestamp": "2025-01-11T15:00:00Z",
    "data": {
        "status": "processing",
        "items": [
            {"product_id": "A1", "quantity": 2},
            {"product_id": "B2", "quantity": 1}
        ]
    }
},
{
    "order_id": 123,
    "event_id": 5,
    "event_type": "shipping_scheduled",
    "timestamp": "2025-01-12T09:00:00Z",
    "data": {
        "status": "awaiting_shipment",
        "shipping_number": "TRACK1234567890"
    }
},
{
    "order_id": 123,
    "event_id": 6,
    "event_type": "order_shipped",
    "timestamp": "2025-01-13T14:00:00Z",
    "data": {
        "status": "shipped"
    }
}

We call this database, the event store. This is an append-only storage mechanism that needs to support two features: fetching events that belong to a particular business entity and adding new ones.

// An event store only needs to support two features: fetching and appending.
interface IEventStore
{
    IEnumerable<DomainEvent> Fetch(Guid Id);
    void Append(Guid Id, IEnumerable<DomainEvent> events, int expectedVersion);
}

In code, when we want to retrieve an order from storage, what we do is fetch all its events, iterate over them and apply the changes they represent to an in-memory object, until it is fully reconstructed, or “rehydrated”. In the context of event sourcing, we call these representations of collections of events, projections. In an event sourced domain model, aggregates leverage these projections to figure out their current state.

// This is a hypothetical order aggregate in an event sourced domain model.
// Its constructor expects a collection of domain events which it iterates over
// to apply them to its internal state representaton object.
class Order
{
    private List<DomainEvent> _events = new();
    private OrderStateProjection _state = new();

    public Order(IEnumerable<DomainEvent> events)
    {
        _state = new OrderStateProjection();
        foreach (var e in events)
        {
            _events.Add(e);
            _state.Apply((dynamic)e);
        }
    }

    // Properties can be exposed to the outside world by leveraging the state
    // projection.
    public OrderId Id => _state.Id;
    public int Version => _state.Version;

    // ...
}

// This is a general use event sourced projection of an order, used to capture
// the current state of an order entity or aggregate.
class OrderStateProjection
{
    // This projection is meant to expose the full state of the order aggregate
    // so it includes all the properties that belong to the order.
    public OrderId Id { get; private set; }

    // With event sourcing, we need to keep track of the version of the entity.
    // Every event that happens increments the version number.
    public int Version { get; private set; }

    public OrderStatus Status { get; private set; }
    public PersonName Name { get; private set; }
    public EmailAddress Email { get; private set; }
    public PhoneNumber Phone { get; private set; }
    public Weight ShippingWeight { get; private set; }
    public CountryCode Country { get; private set; }

    public string PaymentMethodNonce { get; private set; }
    public bool InventoryResolved { get; private set; }
    public string TrackingNumber { get; private set; }

    // It defines a series of "Apply" method overloads, one for each supported
    // domain event. The Order aggregate from above passes all the domain events
    // to the various overloads, calling them one by one, in order, until the
    // full state is rehydrated into the projection. Each "Apply" overload takes
    // the data captured in its event and assigns it to the correct property in
    // the projection.
    public void Apply(OrderCreatedEvent e)
    {
        Id = e.Id;
        Status = e.Status;
        Name = e.Name;
        Email = e.Email;
        Phone = e.Phone;
        ShippingWeight = e.ShippingWeight;
        Country = e.Country;
        Version = 1;
    }

    public void Apply(PaymentFailedEvent e)
    {
        Status = OrderStatus.PaymentFailed;
        Version++;
    }

    public void Apply(PaymentDetailsUpdatedEvent e)
    {
        Status = OrderStatus.Pending;
        PaymentMethodNonce = e.PaymentMethodNonce;
        Version++;
    }

    public void Apply(PaymentSucceededEvent e)
    {
        Status = OrderStatus.Processing;
        Version++;
    }

    public void Apply(InventoryStockUpdatedEvent e)
    {
        Status = OrderStatus.Processing;
        InventoryResolved = true;
        Version++;
    }

    public void Apply(ShippingScheduledEvent e)
    {
        Status = OrderStatus.AwaitingShipment;
        TrackingNumber = e.TrackingNumber;
        Version++;
    }

    public void Apply(OrderShippedEvent e)
    {
        Status = OrderStatus.Shipped;
        Version++;
    }
}

Notice the Version field in the projection, which indicates the number of changes that the entity has gone through.

On the other hand, when it comes to appending new events for an aggregate, the event store needs to be aware of potential concurrency problems and handle them. After all, the ordering of the events matter. That’s why the event store usually implements optimistic concurrency control, leveraging the version field we touched on earlier. Essentially, when trying to append new events, the version being worked with is specified. If it doesn’t match the current version of the aggregate in the event store (maybe because some other process appended a new event), then the operation has to fail, or otherwise adapt to make sure the data remains consistent.

Advantages and disadvantages

One advantage of this design is that we can produce many distinct projections of the same underlying event sourced data. In the previous example, we saw a projection that represents the full current state of the order. But we might need other projections which are optimized for different use cases.

For example here’s a projection that captures all the statuses that an order went through, which might help different business analysis use cases:

// This projection only cares about the changes in status of the order, so
// instead of attempting to capture its full state, in keeps track of all the
// statuses in a collection.
class OrderStatusHistoryProjection
{
    public OrderId Id { get; private set; }
    public int Version { get; private set; }

    public List<OrderStatus> Statuses = new();

    // All the "Apply" overloads capture the status change that each event
    // caused.
    public void Apply(OrderCreatedEvent e)
    {
        Statuses.Add(OrderStatus.Pending);
        Version = 1;
    }

    public void Apply(PaymentFailedEvent e)
    {
        Statuses.Add(OrderStatus.PaymentFailed);
        Version++;
    }

    public void Apply(PaymentDetailsUpdatedEvent e)
    {
        Statuses.Add(OrderStatus.Pending);
        Version++;
    }

    public void Apply(PaymentSucceededEvent e)
    {
        Statuses.Add(OrderStatus.Processing);
        Version++;
    }

    public void Apply(InventoryStockUpdatedEvent e)
    {
        Statuses.Add(OrderStatus.Processing);
        Version++;
    }

    public void Apply(ShippingScheduledEvent e)
    {
        Statuses.Add(OrderStatus.AwaitingShipment);
        Version++;
    }

    public void Apply(OrderShippedEvent e)
    {
        Statuses.Add(OrderStatus.Shipped);
        Version++;
    }
}

We can even store these projections into separate databases if we need to. We’ll learn more about that aspect later when we discuss CQRS.

Another feature that we can implement easily thanks to event sourcing is time travel. We can easily apply the stored events up to a specific point in time or up to a specific version number and obtain the state of the entity as it was at that time. This can, for example, help troubleshooting, and unlock more avenues for business analysis.

An obvious disadvantage of event sourcing is the complexity that it introduces. This complexity, compounded by the potential learning curve on teams that aren’t used to this kind of design, can be very detrimental when utilized in projects that don’t need it. As usual, follow DDD’s core principle of tying the system’s design to the domain’s needs, make sure to use the right tool for the job, and only deploy an event sourced domain model when the situation really calls for it. That is, whenever requirements dictate that the features enabled by such a design are necessary.

The event sourced domain model

So, in summary, an event sourced domain model is a domain model that uses the event sourcing pattern to represent and operate on its aggregates. In a traditional, non-event-sourced domain model, the current state of the aggregates is persisted, commands modify this state and domain events are emitted for certain important operations, when needed. In the event sourced variant, domain events are used much more frequently, as they are the only source of truth. Everything and anything that changes the state of the aggregates produces an event. No changes are done directly, only through events. This is necessary because it is the events that are persisted, and it is from these events that the aggregates’ current state is derived.

In general, operations involving event sourced aggregates go through the following steps:

Load the aggregate’s domain events from the event store.
Reconstruct the aggregate’s state using these events. You can use the particular projection needed for the task at hand.
Run the necessary aggregate commands. Which in turn produce new domain events.
Append the new events into the event store. Making sure to handle any concurrency errors.

public class RescheduleOrderForPayment
{
    private IOrderRepository _orderRepository;

    // ...

    // This is the general pattern that application services often follow when
    // interacted with event sourced domain model aggregates: load events,
    // construct the aggregate with the events, run commands and save the newly
    // created events.
    public Result Run(OrderId orderId, string paymentMethodNonce)
    {
        try
        {
            var events = _orderRepository.LoadEvents(orderId);
            var order = new Order(events);
            var originalVersion = order.Version;

            // See below for what commands like these generally look like.
            order.UpdatePaymentDetails(paymentMethodNonce);
            order.ScheduleForPaymentProcessing();

            _orderRepository.Save(order, expectedVersion: originalVersion);

            return Result.Success();
        }
        // The repository uses the given version parameter to implement
        // optimistic concurrency control.
        catch (ConcurrencyException ex)
        {
            return Result.Error(ex.Message);
        }
    }
}

// ...

class Order
{
    // ...

    // Commands of event sourced aggregates don't modify state directly.
    // Instead, they create the appropriate events and append them.
    public void UpdatePaymentDetails(string paymentMethodNonce)
    {
        var e = new PaymentDetailsUpdatedEvent(_state.Id, paymentMethodNonce);

        _events.Add(e);
        _state.Apply(e);
    }
}

DDD tools for implementing business logic

These are the main concepts that we’ve explored, and how they relate to each other.

Observing End Point Dev's Approach to AI

2026-04-16T00:00:00+00:00

Photo by Garrett Skinner, 2022.

When I joined End Point Dev in October of 2025, there was one clear directive among a wide-ranging set of responsibilities: AI is causing drastic changes in our industry, and we need to tackle it head-on.

As a non-engineer working in a software development consultancy, there is a double edged sword in focusing my work on AI. The downside, of course, is my lack of expertise in anything involving code. I’ve sold software for a majority of my career, but I haven’t been the one building it. A reasonable person could wonder: how would I have a useful perspective on the developments around AI?

To that there are a few answers. For starters, I don’t get bogged down in the “how” AI is working as much as I am interested in the results of its work. I’ve certainly learned context windows, token usage, rate limits, and other immediately useful information around utilizing AI.

I also get to be a guinea pig for End Point’s suite of AI services. Our team might be primarily technical users, but that does not mean our clients necessarily have that same background. I get to approach AI tools as an interested user, not a development wizard.

With that said, these past six months have been an incredible learning experience within the space. I’ve experienced how AI can help with projects that would otherwise take ten times the amount of work, seen how development times can be improved with properly utilized generative AI, and observed healthy debate internally about where AI ought to be deployed.

Time Sheet Analysis and AI Analysis

One of the most impressive aspects of the End Point culture is an emphasis on dutiful and detailed timesheet management. As I first stepped into the role, I gained access to the historical timesheet entries across the entire organization, which was a little overwhelming at first, if I’m being completely honest! For billable and non-billable work alike, whether from directors or part-time developers, there was a detailed record of work in 15-minute increments.

The application itself was novel—a lightweight yet powerful tool that organized this work by type and client and allowed multiple types of reporting for someone like me to get a quick lay of the land. Within a few hours of my first day on the job, I was able to get an overview of our top clients, what work was done with them, and which developers were involved.

This was directionally useful, but it was also a bit of context overload—a good problem to have.

So, I decided to keep my findings high-level. I got my overview and moved on, using these reports to pick a lane in the company and start swimming (Visionport). At the risk of mixing metaphors, I didn’t want to boil the ocean. A few months later, though, the allure of that timesheet data reentered my mind. There was so much valuable data there—beyond what I had looked at from that bird’s-eye view. AI is a great way to aggregate data, so I wondered: What else could be explored with the use of AI?

With a little help from my OpenClaw instance, JJ, a plan was devised. I downloaded a year’s worth of timesheet data for virtually every developer on our team, down to the nitty-gritty of each daily entry. I then fed this into my OpenAI pro account and gave it a fairly detailed prompt, asking it to locate patterns of work and opportunities to expand within our current clients. For good measure, I also asked it to organize the data to my specifications, creating a spreadsheet with the tech stack, percentage of hours worked by client for each employee, and other quality-of-life tracking.

As we all know, AI sometimes aims to please a bit too much, so before I acted on any of this data, I took it to the team. While there were certainly some clean upsell opportunities given by ChatGPT, it became clear that the real value was a common understanding for each of our clients between myself and the client rep—a conversation starter, if you will. The project led to an expedited process for me to know our clients, our common workloads, and a better relationship between me and my fellow End Pointers. A solid case for AI enabling real, human work.

Traditional Software Engineering and Spec-Driven Development

There has been a healthy debate that I’ve witnessed from both my timesheet analysis and internal conversations here at End Point, and that has been the function of AI in a team of expert coders.

It’s no secret that AI has made tremendous progress in its ability to write large quantities of code, especially with the recent leaps in aptitude from the likes of GPT-5.3-Codex and Opus 4.6. Those most bullish on AI might claim that software engineers will be the first industry gobbled up by the ever-looming march of our sycophantic AI overlords. A step below that is an emphatic push from Nvidia’s CEO for their head developers to spend around two hundred and fifty thousand dollars on tokens, lest there be a problem with their productivity.

On the other side of the spectrum, there are those who believe that, while AI can sure as heck write code quickly, it can also cause more harm than good. As much as “vibe coders” have taken over, the myriad bugs and suboptimizations in their projects have given birth to a novel job title: the “Vibe Code Cleanup Specialist.”

With these bullish opinions seemingly everywhere, I had to search for a counterargument within End Point itself. As the resident AI cynic at End Point, Senior Developer Gered King takes a very wary stance towards the involvement of AI in software development:

“While vibe-coding can certainly feel like magic when you first give it a try, as the saying goes, ’there is no such thing as a free lunch.’ Under a very strict definition of ‘vibe-coding’ as it was originally coined by Andrej Karpathy, the vibe-coder is not even looking at the code at all. I think most today would agree without too much arm-twisting that this is likely a bad idea for anything that you eventually want to release into production. Languages like English can be incredibly ambiguous, and combined with the fact that LLMs are not deterministic, you could be really taking quite the gamble that you are crafting sufficiently detailed and unambiguous prompts to get an LLM to spit out quality results without looking at the code to see for yourself.

Even if we decide to take a less strict approach and agree to actually look at the code but still predominantly use AI assistance to write the majority of it for us, this can still lead to problems down the road. Anthropic’s own research is clear that offloading parts of your coding to an AI leads to a statistically significant decrease in skills development and understanding. Struggling with problems is a big part of how we learn. If you constantly take the easy path or are otherwise taking shortcuts, you probably aren’t learning a lot along the way!

Certainly, some of this AI-generated code is going to be of dubious quality, which further impairs our understanding of it. CircleCI recently did an analysis and found that their data across 28 million workflows from customers using their CI tools showed that in the last couple of years, where AI tools are seeing more and more adoption, only 5% of teams are measurably improving their output. The other 95% are barely seeing any difference at all, and in many cases, teams have even gotten slightly worse at shipping new software releases.”

What Gered’s viewpoint represents, beyond the skepticism towards an industry enthralled with powerful tools, is a stalwart protection of human knowledge. I interpret his words as noting that coding is an art as much as a science, and that offloading too much of our problem-solving to AI comes with the risk of losing the ability to develop creative or optimal solutions to the sticky problems within software development.

As our company pushes toward a future where AI is not only prevalent but ever-improving, this skepticism is welcome and, in my humble opinion, critical.

Spec-Driven Development with AI

While it’s foolish to proclaim that any sort of best practice around AI truly exists, a few truisms have emerged since the launch of ChatGPT in 2023. Chief among them is that AI produces dramatically better results when given clear, detailed specifications, and that unstructured “vibe coding” tends to produce unstructured results.

End Point has leaned into this reality. Over the past several months, our team has developed and begun using a framework for AI-assisted, spec-driven development, and the results have been nothing short of a revelation. Rather than treating AI as a replacement for engineering discipline, the approach doubles down on it: detailed project specifications, strict security standards baked into the framework’s core governance, rigorous testing requirements, and structured issue-driven execution with AI agents working within those guardrails rather than outside them.

A tremendous bonus we’ve found is the quality of documentation that AI agents produce alongside the code itself. When working within well-defined specifications, the agents generate thorough, consistent documentation as a natural byproduct—something that has historically been one of the easier things to let slip in traditional development.

CEO Ben Goldstein has been hands-on in developing and refining the framework in consultation with End Point team members and multiple AI systems, and End Point developers are actively using it on real projects. The shared experience has reinforced a central insight: the better the spec, the better the output.

Final Thoughts

Depending on where you look, generative AI is being touted as either the most important technological development in our lifetimes or a bubble that’s just waiting to burst. The beauty of working with the brilliant developers at End Point is that, regardless of how things shake out, preparations are being made.

Buried in the hype of AI’s progress is a question begging to be answered: Does every solution need to utilize AI? One of our leaders in AI, Edgar Mlowe, recently wrote about a niche use case for LLMs, drilling into a specific need with a specific solution. When a clear solution isn’t present, though, can it be helped to avoid bringing in LLMs for a solution that simple automation is more appropriate to solve? In other words, is our industry becoming too AI-reliant, or is the emphasis on bringing the “Elephant Gun” drawn from a need to better familiarize us with working alongside LLMs?

This is why a healthy balance must be struck in how these models are leveraged, and why I find it so valuable and important that End Point is home to a range of opinions on AI—from the overt enthusiast to the hardened critic. My role requires a vantage point of all sides and helping craft solutions that leverage the full power of AI, without sacrificing that which makes End Point a special place: the human touch.

Getting the Most from your Claude Subscription

2026-04-14T00:00:00+00:00

Photo by Josh Ausborne, 2006.

Every prompt you send to Claude Code costs tokens. If you understand where those tokens go and how to control them, you can stretch your subscription dramatically further. This guide covers the practical steps I have taken to keep my usage lean without sacrificing capability.

What are Tokens?

A token is the smallest unit of text Claude processes. A token is roughly three-quarters of a word. The sentence “Hello, how are you today?” is about seven tokens. Every interaction, yours and Claude’s, is measured in tokens drawn from a fixed context window.

Think of the context window as a whiteboard. Everything Claude needs to know must fit on it: your instructions, the conversation so far, any files it reads, and its own responses. When the whiteboard fills up, older content must be erased to make room.

What Loads with every Prompt

Most people assume they are only paying for the text they type. In reality, Claude loads a stack of context before it even reads your message.

Breakdown of what gets loaded:

Source	Tokens	When
System prompt	~4,200	Every message
Environment info	~280	Every message
CLAUDE.md hierarchy	Varies	Session start
MEMORY.md index	~200–680	Session start
Conversation history	Grows	Every message
File reads and tool output	~2,500 per file	On demand

The conversation history is the big one. It compounds with every exchange. A ten-turn conversation where Claude reads a few files each turn can consume most of the context window before you realise it.

The CLAUDE.md Hierarchy

CLAUDE.md files are persistent instructions that load at session start. They exist at multiple levels and all of them stack:

Each file you add here is loaded in full every session. Keep them concise. Aim for under 200 lines per file. Long CLAUDE.md files reduce instruction adherence and burn tokens on every single message.

Memory

Claude Code maintains an auto-memory system at ~/.claude/projects/<project>/memory/. The MEMORY.md index file (first 200 lines) loads every session. Individual topic files only load on demand when Claude decides they are relevant.

This is generally lightweight, but if your MEMORY.md grows large, it silently eats into your budget on every prompt.

The Three Models and How They Use Tokens

Claude Code gives you access to three models, each with different token economics. The model you use makes a significant difference in how fast you burn through your allocation.

Model	Strength	Token Cost	Best For
Haiku	Fast, cheap	Lowest	Simple tasks, quick lookups
Sonnet	Balanced	Medium	Most coding work
Opus	Deep reasoning	Highest	Complex architecture, debugging

How Your Prompt Selects the Model

Your subscription plan and settings determine the default model. You can override it several ways:

During a session: /model sonnet or /model opus
At startup: claude --model haiku
Permanently: Add "model": "sonnet" to your settings file

The key insight is that you should match the model to the task. Do not use Opus to rename a variable. Do not use Haiku to design a database schema. The difference in token consumption between Opus and Haiku for the same task can be several times over.

Effort Levels

Each model also has an effort setting that controls how much “thinking” it does:

Level	Behaviour	Use When
`low`	Minimal reasoning	Straightforward, mechanical tasks
`medium`	Balanced (default)	General development
`high`	Deep analysis	Tricky bugs, architecture decisions
`max`	No token limit on thinking	Last resort, Opus only

Change it with /effort low before a simple task, then /effort high when you need deep analysis. This alone can save significant tokens.

Monitoring your Token Usage

You cannot manage what you cannot see. Setting up a statusline is the single most impactful thing I did for token awareness.

Setting up the Statusline

The statusline sits at the bottom of your Claude Code terminal and shows real-time token consumption. Configure it in ~/.claude/settings.json:

{
  "statusLine": {
    "type": "command",
    "command": "bash -c 'data=$(cat); m=$(echo \"$data\" | jq -r \".model.display_name // \\\"Unknown\\\"\"); p=$(echo \"$data\" | jq -r \".context_window.used_percentage // 0\"); ti=$(echo \"$data\" | jq -r \".context_window.total_input_tokens // 0\"); to=$(echo \"$data\" | jq -r \".context_window.total_output_tokens // 0\"); tt=$((ti+to)); f=$(awk \"BEGIN{printf \\\"%d\\\",int($p/10+0.5)}\"); pi=$(awk \"BEGIN{printf \\\"%d\\\",$p+0.5}\"); bar=\"\"; for((i=0;i<f;i++)); do bar+=\"\\xe2\\x96\\x88\"; done; for((i=f;i<10;i++)); do bar+=\"\\xe2\\x96\\x91\"; done; if((pi<50)); then pc=\"\\033[32m\"; elif((pi<80)); then pc=\"\\033[33m\"; else pc=\"\\033[31m\"; fi; if((tt>=1000000)); then td=$(awk \"BEGIN{printf \\\"%.1fM\\\",$tt/1000000}\"); elif((tt>=1000)); then td=$(awk \"BEGIN{printf \\\"%.1fk\\\",$tt/1000}\"); else td=\"$tt\"; fi; printf \"\\033[1;36m%s\\033[0m  [${pc}%b\\033[90m%b\\033[0m] ${pc}%s%%\\033[0m  \\033[37m%s tok\\033[0m\\n\" \"$m\" \"$(printf \"%b\" \"$(for((i=0;i<f;i++)); do printf \"\\\\xe2\\\\x96\\\\x88\"; done)\")\" \"$(printf \"%b\" \"$(for((i=f;i<10;i++)); do printf \"\\\\xe2\\\\x96\\\\x91\"; done)\")\" \"$pi\" \"$td\"'"
  }
}

This reads the session JSON from stdin via jq, then renders:

Model name in cyan
A 10-segment progress bar using block characters, colour coded: green under 50%, yellow under 80%, red above 80%
Context percentage as a number
Total tokens (input + output) in a compact format (e.g. 142.3k tok)

Drop this into ~/.claude/settings.json and it appears at the bottom of every Claude Code session. When the bar turns yellow, compact. When it turns red, compact immediately or /clear.

Other Monitoring Commands

Command	What It Shows
`/context`	Visual grid of context usage with tips
`/status`	Current model, account, version
`/cost`	Token usage stats (API users)

Compaction: Your Most Important Habit

Compaction summarizes older conversation history to free up context space. Claude Code does this automatically when the context fills up, but waiting for autocompact is wasteful.

Why Compact at 60%

When you wait until the context is nearly full:

Autocompact fires but can only free a small amount
The next file read or tool output fills it right back up
Autocompact fires again. Then again. This is thrashing.
Eventually Claude gives up and you lose your session

When you compact at 60%:

There is enough history to summarize meaningfully
The resulting summary is concise
You get a large chunk of free space back
You can continue working without interruption

Using /compact Effectively

You can guide what gets preserved:

/compact                          -- general compaction
/compact focus on the API changes -- preserve specific context
/compact keep only the plan       -- aggressive, targeted

Always tell Claude what matters. A focused compaction produces a better summary and frees more space.

Time of Day and Rate Limits

Claude Code has rate limits measured in tokens per minute (TPM) and requests per minute (RPM). These limits are shared across your team.

The practical effect: during peak hours when your entire team is active, you hit rate limits faster. Early morning or late evening sessions often feel smoother because fewer concurrent users are competing for the same allocation.

Team Size	TPM per User	RPM per User
1–5	200k–300k	5–7
5–20	100k–150k	2.5–3.5
20–50	50k–75k	1.25–1.75

Anthropic’s infrastructure serves a global user base, so peak load follows the sun. The busiest window is when North American and European working hours overlap. Here is a rough guide for three major time zones:

City	Peak Hours (Local)	Low Traffic (Local)
London (GMT/BST)	09:00 - 17:00	21:00 - 06:00
New York (EST/EDT)	09:00 - 18:00	22:00 - 06:00
Los Angeles (PST/PDT)	08:00 - 17:00	21:00 - 06:00

The global peak, when the most users are active simultaneously, is roughly 13:00 - 17:00 UTC. That window is mid-afternoon in London, late morning in New York, and early morning in Los Angeles. If you have flexibility in when you do heavy Claude work, shifting to early morning or evening local time can noticeably reduce rate limit friction.

This means your workflow matters. If you are sending rapid-fire prompts during a team standup where everyone is also using Claude, you will hit limits. Batch your work thoughtfully.

Practical Habits That Save Tokens

Keep Tasks Small and Focused

Vague prompts like “improve the codebase” force Claude to scan broadly, reading files it does not need. Specific prompts like “add input validation to the login endpoint in src/auth.ts” let Claude work surgically.

Group Work by Skill

If you need to write tests, write all the tests in one session. If you need to update API endpoints, do them together. Switching between unrelated tasks within a single session wastes context because Claude has to hold multiple unrelated threads in memory.

Use /clear Between Unrelated Tasks

/clear wipes the conversation history and frees all context. Before clearing, name your session with /rename so you can find it later with /resume.

A good workflow:

Start a focused task
Work until done or until context hits 60%
/compact if continuing, /clear if switching topics
/rename "descriptive name" before clearing

Use Plan Mode for Complex Work

Press Shift+Tab twice to enter plan mode. Claude reads and proposes an approach before writing code. This prevents expensive rework when the initial direction is wrong. Review the plan, iterate in conversation, then execute.

Delegate Verbose Work to Subagents

Test runs, log parsing, and documentation fetching consume enormous context. Subagents run in their own context window and return only a summary. The verbose output stays out of your main conversation.

Understanding Your Subscription Quota

Claude subscriptions use a rolling 5-hour window for usage limits, plus a separate weekly quota. Understanding these mechanics unlocks a simple but powerful trick.

The 5-Hour Rolling Window

The timer does not reset at midnight. It starts when you send your first prompt and rolls forward from that moment. Messages you sent at 9 AM stop counting against your quota by 2 PM. After 5 hours, your allocation gradually replenishes.

This creates a practical opportunity: send a short prompt early to start the timer, even if you are not ready to work. A quick “hello” or a /status check is enough. The 5-hour clock begins ticking immediately. By the time you sit down for focused work 30 or 60 minutes later, you are already that much closer to your window expiring and resetting. Over a full workday this can mean the difference between one reset and two.

The Weekly Quota

Separately from the 5-hour window, there is a 7-day rolling cap. If you exhaust the weekly quota, waiting 5 hours will not help. You must wait for the 7-day window to roll past your heaviest usage period. This is why daily token discipline matters. Burning through tokens recklessly on Monday can leave you throttled by Thursday.

Extra Usage: Pay-as-You-Go Overflow

If you have a payment method on file, you can enable extra usage so Claude Code continues working when you hit your plan limits instead of blocking you. Overflow tokens are billed at standard API rates.

To enable it, go to Settings > Usage on claude.ai and click Adjust limit to set a monthly spending cap. You can also configure it from the terminal with /extra-usage. Once enabled, hitting a rate limit is seamless. Claude keeps working and the overflow cost appears on your next bill.

This is worth enabling even if you rarely hit limits. It acts as a safety net for those sessions where you are deep in a complex problem and cannot afford to stop.

Putting It All Together

The short version:

Monitor your token usage with a statusline
Match the model and effort to the task
Compact at 60%, not at the limit
Clear between unrelated tasks
Be specific in every prompt
Group related work together
Delegate verbose operations to subagents
Time heavy work outside peak team hours

Token efficiency is not about being stingy. It is about being intentional. Every token saved is a token available for actual work. Set up your monitoring, build the habits, and your subscription will go much further.

High Level System Analysis and Design with Domain-Driven Design

2026-04-06T00:00:00+00:00

Photo by Zed Jensen, 2022.

This is part 1 of a series of blog posts on Domain-Driven Design:

High level system analysis and design with Domain-Driven Design

Implementing business logic with Domain-Driven Design

Designing software architecture with Domain-Driven Design

As a result, DDD offers a treasure trove of concepts, patterns and tools that can be applied to any software project, regardless of the size and complexity.

High level system analysis and design with Domain-Driven Design

Section 1. Analyzing business domains

It is a well understood notion that before writing code, we need to understand the problem we’re trying to solve. DDD is consistent with this notion and argues that developers need to, first and foremost, gain an understanding of the business that the software is being built for. To this end, DDD relies on three concepts: domains, subdomains and domain experts.

Domains and subdomains

In simple terms, the domain of a business is what it does, its area of activity. For example, Starbucks is in the business of making coffee, Ford is in the business of making automobiles, AMC is in the business of movie theaters.

Of course, analyzing a business as a single integrated whole can be unmanageable. That’s where subdomains come in. Subdomains are the different divisions within a domain. Starbuck’s domain for example, is making coffee. But there are many smaller parts that make up that business and allow it to be successful. There’s of course, the subdomain of coffee preparation. But there’s also real estate management to find and secure good locations, there’s inventory management and logistics, there’s marketing, there’s human resources, etc. All these are the subdomains that make up the overall business of Starbucks.

Depending on the business and on the project, these will vary greatly in granularity. And you can also decompose subdomains further and discover new fine-grained subdomains nested within more coarse-grained ones. The sizes and nesting levels can be very different from business to business, so it can be difficult to clearly delineate where a subdomain ends and another one starts, which activities belong to one or the other. One good rule of thumb to keep in mind is that generally, subdomains encapsulate a set of coherent, closely related use cases. That is, use cases that involve the same set of closely related actors, business entities and/or data.

And finally, we have domain experts. As the name suggests, these are the people within the organization who have intimate knowledge of the business, or certain areas of it. They are the subject matter experts. Usually they are the ones who identify the problems and come up with requirements. Developers need to rely on domain experts to gain the necessary understanding to be able to produce useful software solutions.

So, when approaching software projects, DDD suggests that developers work closely with domain experts in order to learn from them about the business domain and subdomains. After all, it is their mental models and understanding that will be modeled and implemented in code.

Types of subdomains

With the help of domain experts, developers can identify subdomains, understand their business value and how they fit within the overall business strategy. This is very important because it helps making some initial architecturally significant decisions. Namely, the general approach to solving the problems in the subdomains, how much to invest, how to organize development teams, etc. The main objective in this analysis stage is to identify the subdomains and whether they fall into one of three types: core, generic, and supporting.

Core subdomains are the activities of the business with highest value. The ones that confer differentiation in the market and a competitive advantage. They are the business’ raison d’être. For example, for Google, their search engine is a core subdomain. For Ford, their automotive engineering area would be a core subdomain. Core subdomains are generally the most complex parts of the business. They are constantly evolving and improving, and the company is compelled to invest heavily in them.

Generic subdomains are also very complex. However, they are not business differentiators. Instead, these are the areas of the business that all organizations handle in the same way. Think accounting, a ticketing system, an online storefront. There’s no pressure to innovate in these areas, so the solutions are very stable and evolution is slow.

Supporting subdomains on the other hand, do not provide any competitive advantage, nor are they very complex. They are however, necessary because they support the core business activities, and are fairly unique. The solutions to problems in these areas usually take the form of CRUD or ETL oriented activities. Imagine for example populating a data warehouse, translating transactional business data into a format appropriate for analytics and business intelligence in a manufacturing corporation. Or maybe the digitization and storage of a registry of court documents for a law firm. These are behind the scenes activities that support the organizations’ main businesses, but their business logic complexity is not high, and they don’t really represent big selling points for the company.

It’s worth noting as well, that there may be subdomains where software solutions are not appropriate, even if they are highly complex core subdomains. They are still part of the business so it’s worth identifying and considering for high level architectural design decisions. If anything, to know what parts of the business the planned software system should and should not focus on. You could have a restaurant, for example, who prides itself in having the best desserts in the city. For their business, the recipe development activities constitute a core subdomain. This is dependent on the art and craftsmanship of the chefs. Not an area in which software solutions could help a whole lot.

It’s also worth noting that, just as organizations’ business strategies are dynamic, so too can be their subdomain distribution. Today’s generic subdomain can be tomorrow’s core subdomain, and so on. For example, imagine a big retail store chain that, up until now, managed its inventory in an industry standard way. But it has grown so much that the standard way of doing things has become a bottleneck for them. So they design a new procedure for highly efficient inventory management, and that gives them an edge against competitors. Inventory management started as a generic subdomain for them, but due to an ever evolving business strategy, it became a core subdomain.

The knowledge of subdomain types can also help the analysis when dividing the business and discovering subdomains. If you find yourself recursively finding more and more fine-grained subdomains from already identified ones, a good time to stop recursing is when you find that a generic or supporting subdomain, when broken down, reveals only new finer-grained generic or supporting subdomains. The reasoning for this is simple: this analysis activity is most valuable when trying to identify core subdomains. When there are no more core subdomains to be found within a particular area of the business, then it’s probably not worth it to keep digging deeper.

Here we have an example of a hypothetical help desk system subdomain, part of a customer service subdomain, being decomposed and revealing that it only contains generic subdomains.

In summary, these are the main characteristics of the three types of subdomains:

Core subdomains have high complexity, provide competitive advantage, and change and evolve frequently.
Generic subdomains have high complexity, do not provide competitive advantage and change overtime, although at a slower pace than core subdomains.
Support subdomains have low complexity, do not provide competitive advantage and are the slowest to change.

This is where each type of subdomain fall when considering their business logic complexity and business differentiation.

Using subdomains to make strategic decisions

Like I’ve already alluded to, armed with this knowledge, DDD practitioners are ready to start making some higher level architectural decisions. Depending on the type of subdomain that a problem belongs to, DDD has specific prescriptions on how to handle the implementation of software solutions for them.

When working on core subdomains, that’s where we want to make the biggest investments. We deploy the most advanced engineering tools, patterns and practices. This is to make sure that the software is efficient and high quality, and also easy to maintain and evolve. This is necessary because core subdomains have to evolve rapidly by nature, if the business is to maintain competitive advantage. Software solutions that operate within the context of core subdomains have to be implemented by high skill and high trust teams. Either in-house, or via trusted development partners, working hand in hand with domain experts.

Problems in generic subdomains, by nature of their business logic being very complex but also very common, are likely to have already been solved. For these types of problems, DDD recommends against implementing custom software, and instead buying and/or adopting tried and true, industry standard, off-the-shelf solutions. Their implementation and integration can be outsourced or handled by less specialized or skilled teams. The change management of these solutions is simple, as they get delivered generally via patches and updates.

For support subdomains, whose business logic is generally simple but uncommon, it is less likely that off-the-shelf solutions would be available. So software addressing problems in these subdomains will most likely have to be implemented as custom solutions. Due to their low complexity though, they can be easily outsourced, or handled by more junior team members. They can also be handled with RAD, low-to-no-code technologies, since often times they are little more than ETL and pure CRUD applications.

Here’s a table to summarizes the differences between the types of subdomains:

Type of subdomain	Core	Generic	Supporting
Competitive advantage	Yes	No	No
Complexity	High	High	Low
Rate of change	High	Medium	Low
Implementation	Custom development	Buy/adopt off-the-shelf	Custom development
Team composition	In-house/partners	Can outsource	Can outsource
Skill level	High	High/regular	Low
Investment	High	Medium	Low
Problems	Interesting	Solved	Simple

Section 2. Discovering domain knowledge

After identifying the business domain and categorizing the various subdomains that compose it, we have a bird’s-eye view of the business. This is good enough to get started and make high level architectural decisions of potential software solutions to problems within these domains. To actually build the software though, we need much more than that. In order to gain a thorough understanding of the business logic, and be able to eventually model and implement it in code, DDD proposes the ubiquitous language as a tool.

The ubiquitous language

The ubiquitous language is DDD’s tool for knowledge sharing, effective communication and software modeling. In simple terms, the ubiquitous language is the language of the business. It’s the language that domain experts use on a day to day basis to talk and reason about the business.

For a software project to be successful, it’s essential that engineers and stakeholders understand each other. They have to be aligned when it comes to the meanings of the core concepts of the business domain. That’s why it’s so important that everybody uses the ubiquitous language for all project related communications, requirements, documentation, face to face discussions, and even code itself. It all needs to share the same language.

Engineers will need to interact with domain experts in order to learn about the business domain and its rules. They need to acquire the necessary knowledge that allows them to implement these rules in the software. During these interactions, they converse using the ubiquitous language. However, the knowledge transfer is not always strictly unidirectional. Yes, engineers have to learn from the domain experts. But also, through conversations and questioning, engineers can help domain experts deepen and flesh out their own understanding about their domains.

A classic example of this is when domain experts focus too much on the “happy paths” of given business processes. Then, through discussions with developers, on account of the inherent precision that software demands, they are forced to consider more edge cases, better specify ambiguous terms and fill out gaps in their understanding.

This deepening of knowledge feeds back into the ubiquitous language and improves it. Making it more insightful and more precise. This means that, throughout the life cycle of a project, the ubiquitous language should keep evolving, expanding and refining. This allows it to continuously improve as an effective model of the business domain, for the problem that it’s trying to solve. Indeed, just like code is a model of the business domain, so too is the ubiquitous language.

A classic pitfall in software projects is a manner of communication and knowledge sharing where developers are various steps removed from the domain experts. Consistent use of the ubiquitous language addresses this.

The ubiquitous language as a model of the domain

Through consistent use of the ubiquitous language, developers are able to obtain a deeper understanding of the domain, the problems we’re trying to solve, the reasoning behind the requirements and the mental model of the domain experts. This allows the construction of effective software solutions, with deep business insight, that go beyond simply translating requirements into code. If the implementation models the business domain effectively, there is potential for it to evolve with the business and better adapt as requirements change; and we reduce the risk of missing edge cases that may not be obvious to domain experts.

Under DDD, developers and domain experts develop the ubiquitous language, which then informs the implementation.

Effectively, when we build a ubiquitous language, we’re building a model of the domain. A model that reflects the relevant business entities, their behavior, and relationships. A model that will be eventually implemented into code, but also a model that needs to be understood by all stakeholders, regardless of their technical level. So it needs to be precise and rigorous, but also understandable. As such, in order to be useful, the ubiquitous language needs to respect certain restrictions:

It must not include technical jargon like programming languages, constructs or frameworks; nor mention specific system structures like database tables, servers or programs. It is the language of the business and it needs to be understandable by non technical folks.
It must avoid using the same term for different concepts. In conversation with humans, a lot of the meaning is extracted from the context. In software, not so much. Just like you can’t have two classes with the same name within the same namespace, the ubiquitous language cannot allow such ambiguities.
It should also avoid using different words for the same concept. The classic example for this is words like users, accounts, customers, visitors. They all refer to closely related concepts which might actually have differences. The problem with these overlapping terms is that those differences are obscured by the notion of them “being the same” and being used interchangeably. For usage within the ubiquitous language, it’s best to be precise and clearly delineate terms and definitions. Eliminate what’s redundant, be strict with definitions.
It should avoid including extraneous details. Just like code needs to include only what’s needed to solve the problem at hand and nothing else (in order to avoid accidental complexity); the ubiquitous language must not be polluted with details from outside of the area of business activity that it represents. That can create confusion and unnecessary cognitive load.

The ubiquitous language represents a unified model of the domain. This is in contrast to a more traditional process where domain knowledge gets “translated” multiple times before turning into code.

Tools for capturing the ubiquitous language

The Agile Manifesto declares: “Individuals and interactions over processes and tools”. And of course, when it comes to the ubiquitous language, direct interaction and conversations between engineers, domain experts and other stakeholders reigns supreme. That’s when it can be most useful. It can also be useful, however, to capture the ubiquitous language in documentation and even in runnable form.

A glossary of terms is a great asset for keeping a ubiquitous language. A wiki is a good place to put this. Definitions for key concepts in the business like entities, processes and rules can be captured here. The only caveat is that documentation is static by nature, while the ubiquitous language is continually evolving. So, great care needs to be taken to keep the wiki updated at all times to reflect the latest and most complete understanding of the domain. This should not be relegated to or gate-keeped by only certain people; it should be a team effort where everyone contributes.

An automated acceptance tests suite, written using Behavior Driven Development frameworks, like Cucumber, is also a great way of capturing the ubiquitous language. The advantage of these tests is that they are written in plain human-readable language, not code. And while it may be far fetched to think that non technical domain experts would be capable of writing and maintaining such tests, they certainly can read and understand them, which is a great boon. These tests speak the language they understand: the language of the business.

This is what Cucumber tests look like.

By nature of being executable and tied closely to the implementation code, there is less chance that they become out of date. This can happen more easily with static documentation written in a wiki. The disadvantage is that they require much more effort. But for the right kind of project, one where business logic is very complex or the scope is very wide, they can be very well worth the cost.

Section 3. Managing domain complexity

We can attempt to model an entire business domain with one big ubiquitous language, but sometimes that’s impossible. Especially so for businesses of a certain size, you will inevitably find inconsistencies and conflicts between the mental models of different domain experts. The simplest example of this scenario is when different experts from different areas of the organization have the same word to describe different concepts. Or when they look at the same business entity with different levels of detail.

When the different mental models are valid, and these inconsistencies are legitimate and cannot be reconciled, the solution is to follow the divide and conquer principle and decompose the language into separate ones, each one working within its own bounded context.

Simply put, a bounded context is the context within which a ubiquitous language, and the model it represents, operate, have meaning and are useful. Sure enough, just like a domain can be divided into subdomains, a ubiquitous language can be decomposed into smaller languages, each with its own context, to model different parts of the business. While we can attempt to capture an entire business domain with a single model, this is not advisable in the case of complex systems. It’s better to split up the model into smaller, more precise ones, tailored to work on specific problem domains.

Reasons for creating bounded contexts

Like mentioned before, different domain experts having conflicting mental models is a clear indication that a division in the model needs to happen. However, that only tells us half of the story. There are other indicators that point to when and where further division should happen:

First of all, there is of course, size. Size by itself is not a deciding factor, but it is something to consider and balance. Fewer, bigger models can help keep the overall environment simpler, but if they become too big they can become unmanageable and prone to corruption. A higher number of smaller models keep cognitive load low, but you run the risk of exploding integration and management complexity. The better principles to keep in mind here are coupling and cohesion. Beware of separating closely tied use cases, that deal with similar actors, entities and data.

System-level nonfunctional requirements also play a role in dividing a model into separate bounded contexts. For example, you might need to decouple the life cycle of several software components. Have them be developed, evolved, versioned and deployed independently. You might also need them to scale separately. You might even need to use completely different technology stacks, whatever is appropriate for the task at hand. This means that bounded contexts generally get implemented as individual services and/or applications. That is, as individual runtime components.

The organization’s composition also plays a role when designing bounded contexts. The general rule is that a given ubiquitous language, the model it represents and the bounded context is lives in, must be owned by a single team. In software, high ownership, cohesion and consistency are desirable traits. Having a single team own and maintain a particular component foments these. Single ownership also helps reduce communication overhead, and prevents people from stepping on one another’s toes. Of course, a single team can own multiple bounded contexts, what cannot happen is one bounded context being owned by multiple teams.

Indeed, in bounded contexts, we have the tools we need to make strategic decisions related to the decomposition of software systems into architecturally significant components or modules. With bounded contexts, we are able to specify the physical and ownership boundaries of these components.

Bounded contexts vs subdomains

Bounded contexts and subdomains are closely related concepts, but there’s one key distinction: subdomains are discovered, while bounded contexts are designed. Subdomains are useful because they help us understand the business strategy. Splitting the business domain into smaller problem domains is useful because it can break down a complex whole into smaller and more manageable parts.

Depending on the situation, it is certainly possible to end up with a set of bounded contexts that align one-to-one with the business subdomains. However, this is not mandatory. We can develop a solution with a single bounded context that spans multiple subdomains; the same way that we can decompose the problem into many bounded contexts, some of which operate within the same particular subdomain.

Bounded contexts are closely related to subdomains, but aren’t tied to them. They are flexible and can be organized in many ways.

Each bounded context becomes a separate major architectural component. That is, a (micro) service, a project, an application. When we have a component that spans multiple subdomains, programming language organizational structures like namespaces or modules can be used to logically separate the subdomains within.

Section 4. Integrating bounded contexts

For a system to function, its components need to interact with each other. So, once we have decomposed the problem domain into separate bounded contexts, we need to decide their relationship and integration strategies. This need for interaction between them implies that there are touch points between bounded contexts. We call them contracts.

These contracts are necessary because each bounded context contains its own version of the world. That is, its own model and ubiquitous language. In order to integrate, some level of translation needs to happen. They need to be adapted to one another. The contracts define these adaptations.

Domain-Driven Design offers various patterns that are useful for defining contracts between bounded contexts. The decision to use one pattern vs the other depends greatly on the nature of the teams tackling the project. Depending on the teams’ relationship, we can put the patterns in one of three categories: cooperation, customer-supplier, and separate ways.

Integration patterns are determined mainly by the level of cooperation between the teams that own the interacting bounded contexts.

Cooperation patterns

When the components that need to interact are owned by teams which are in close communication, work well together, and whose goals are aligned, cooperation patterns can be applied.

If the teams meet these criteria, a partnership model can be implemented. This is where the integration is managed in an ad-hoc manner. Both teams work together to define the API through which their components interact and that’s that. Whenever changes happen on either side, the other team learns about it quickly and adjusts their code right away. Continuous integration is a great tool here, as breakages are immediately apparent, closing down the feedback loop.

Sometimes, when different bounded contexts need to implement the same functionality, it makes more sense to develop this functionality once and package it as a reusable library, or a sub-module within a shared repository. This is what DDD calls a shared kernel. A shared kernel is effectively a bounded context of its own, one that is statically linked to other components and implements the logic that others depend on.

A shared kernel is its own bounded context, but also “belongs” to multiple other bounded contexts.

The shared kernel integration pattern needs to be used carefully though, as it creates tight coupling between the components that use it. If the involved bounded contexts are owned by different teams, then this shared kernel that emerges violates the DDD principle of bounded contexts having single-team ownership. This is something to watch out for, as bad team synergy can produce problems during development and maintenance.

When using a shared kernel, the quality of the communication and the coordination between the teams has to strike the right balance: when they aren’t strong enough to support a partnership model, but not so weak that the shared kernel would become more trouble than it’s worth.

The alternative to a shared kernel is the duplication of the logic across multiple components. When deciding whether to apply this pattern, the costs of duplication vs coordination need to be considered. That is, the cost of implementing the same logic within multiple components and its subsequent changes, vs the cost of changes in the shared kernel propagating into dependent components and coordinating with the owners of each. When it is cheaper to just duplicate the logic, shared kernel should not be applied.

This means that complicated models that change frequently, like those of core subdomains, are good candidates for shared kernel. Shared kernels allow complexity to be encapsulated and exposed through stable contracts. This means that frequent changes are easier to contain. Frequent changes in logic that’s highly complex and also repeated in multiple places, quickly becomes much more expensive to maintain.

Another good scenario for applying a shared kernel is when refactoring a legacy monolithic system into more separated modules. The legacy system can become the shared kernel during the time that modules are being extracted from it but not yet fully decoupled.

Customer-supplier patterns

Customer-supplier patterns establish a relationship between components where one (the provider, who is “upstream”), provides a service to another (the consumer, who is “downstream”). These types of patterns emerge when the teams who own the involved bounded contexts are not in close collaboration and have their own independent goals.

One form of customer-supplier integration is the conformist pattern. This happens when the supplier defines the integration contract/API, using its own language, concepts and model, and the consumer accepts it. This can happen when the upstream service has a well established or industry-standard model. Or maybe the model is “good enough” for the consumer to interact with directly. Organizational politics may also be a reason for this type of integration, where there is an imbalance of power favoring the supplier’s team, and they get to impose their model or can’t be bothered to adapt it.

Sometimes, the consumer won’t accept the supplier’s model. In that case, an anticorruption layer can be created. An anticorruption layer is implemented by the consumer and translates the supplier’s model into its own. This allows the consumer to use the supplier’s service without polluting its model with extraneous concepts.

Anticorruption layers can be good solutions for the following scenarios:

When the consumer represents a core subdomain. The consumer is solving hard and interesting problems, so it’s best to protect its model with the ACL.
When the supplier’s model is messy and inconvenient. The ACL can protect the consumer’s model from having to contend with a mess of extraneous concepts.
When the supplier’s contract changes frequently. The ACL can protect the consumer from those changes, encapsulating them, allowing it to be less volatile.

A third pattern represents an anticorruption layer of sorts, only built on the supplier’s side. DDD calls this the open-host service pattern. Here, the upstream service comes up with a new language, separate from its own, tailored to its consumers’ convenience, and exposes that as its contract. This public protocol is called a “published language”.

An open-host service provides a protocol for consumers to interact with. It can also support multiple versions of this protocol simultaneously.

An open-host service applies to similar scenarios as an anticorruption layer. Decoupling a service’s internal model from its integration model frees it up for continuous evolution without fear of breaking its consumers. Another advantage of this is that multiple versions of the published language can be exposed, affording clients options on what to support and gradually migrate if they so choose.

Separate ways

Sometimes, the right decision is to not integrate at all. I alluded to this outcome back when discussing the pitfalls of the shared kernel. Sometimes duplication, in spite of how bad it smells, may be the most cost effective solution to a given situation. So the teams go their separate ways.

This can happen when the involved teams cannot collaborate effectively for whatever reason. It could be due to geographical, timezone, or organizational issues.

This can also be a good solution when the repeated logic belongs in a generic subdomain, and it’s easy to integrate. For example consider a logging framework. Exposing that kind of functionality in a service for others to consume, in most cases would be much more trouble than just including some third party library or package.

It may also be that the models being integrated are just so different that they are fundamentally incompatible. It may be cost-prohibitive for collaboration or customer-supplier patterns to be applied; and duplication again, is cheaper.

Going separate ways can be dangerous when we’re talking about core subdomains though. So we have to tread carefully in those scenarios. Remember that models that represent core subdomain should be implemented in the most effective and efficient ways, with few shortcuts and minimized technical debt.

The context map

The context map can be a useful tool for high level design as it plots all the major bounded contexts (i.e. modules, components, subsystems) that we’ve designed and their interaction patterns.

Here’s a context map which captures the various bounded contexts that compose a big system and their interactions. The arrows point to the upstream component in the relationship. “ACL” denotes an anticorruption layer and “OHS” represents an open-host service.

Of course, they can also offer valuable insight into organizational dynamics, as team composition and relationships with others are intrinsic parts of the discussion when talking about bounded contexts. For example, it can show teams that prefer to collaborate closely or at a healthy distance. I can also show problematic components, which are surrounded by anticorruption layers or have had their ties completely cut via a separate ways approach.

As with any document, they run the risk of becoming stale as the system evolves. So it should be a team-wide responsibility to keep it up to date. Each team taking care of their own components and their integration points.

The DDD high level design concept map

These are the main concepts that we’ve explored so far, and how they relate to each other.

Containerizing Claude Code with Podman

2026-03-31T00:00:00+00:00

Photo by Seth Jensen, 2026.

I’ve been experimenting with many different AI tools, and my favorite is Claude Code. It provides the impressive performance of the Opus models without forcing me to use Visual Studio Code (or a fork of it).

IDEs and fancy editors like IntelliJ and VS Code are great, but I often prefer the simplicity and low memory footprint of working directly in the terminal. Claude Code works well with my tmux-centered work environment.

However, I don’t like giving AI agents access to all of my files, and I really don’t like letting them run arbitrary commands in my shell. I’m already pretty cautious about running unvetted code on my machine (I’m looking at you, install.sh files I’m supposed to blindly curl | bash), and the nondeterministic nature of LLMs takes this to the next level. It’s not just data-sniffing closed-source code or malware you need to worry about, it’s the product itself running commands and editing files in ways that, by design, are unique and untested.

Claude Code has a sandbox mode which is supposed to limit filesystem access to the folder it’s run from, but since it’s closed-source (in theory), you can’t verify this isolation. Even in this mode, it can run commands outside the sandboxed folder (in my experience, it asks first, but I wouldn’t count on this).

So, I wrote a little wrapper script to run Claude Code in a Podman container with access only to Claude’s config directory and the working directory you pass to the script. I also let it run with the --dangerously-skip-permissions flag, which I would never do on my host machine (it didn’t take long to find a horror story of a typo in an rm -rf command deleting half of the programs in /Applications on one Redditor’s Mac).

Important security notes

Claude has full access to the folder you pass to this script. That means it can run rm -rf as much as it wants, and you won’t be prompted before it does so, especially when using the --dangerously-skip-permissions flag.

You should only run this script if you don’t mind everything in that folder getting deleted — for example, in a Git repository where everything has been pushed to a remote, or in a sandbox folder — it’s easy to create a copy of your working directory for Claude to play in!

There is no constraint on outward network traffic, so Claude can send data anywhere it wants. If you’re working with sensitive data, you should set up a firewall so it only has access to the necessary Anthropic servers (and perhaps an allowlist you provide). Think carefully about what data you give to powerful AI agents.

Using my claude-container script

Make sure you have Podman and Podman Compose installed (this would also work with Docker & Docker Compose, just replace podman compose with docker compose in the claude-container script)
Clone the repo from my GitHub.
From the cloned folder, run ./claude-container /path/to/working/directory
Optionally, add the script to your PATH. This is what I ran on macOS:
```
ln -s /Users/myuser/repos/claude-container/claude-container /Users/myuser/bin/claude-container
```
You can then run the script from anywhere: claude-container /my/favorite/sandbox

That’s all you need to run it! Keep reading if you’re interested in the technical details.

What the script is doing

The claude-container script is a lightweight wrapper around podman compose, setting a couple of environment variables so the process is as simple as running Claude in the first place.

The compose setup uses bind mounts to give Claude access to the folder you pass, along with your Claude config directory. This means you don’t have to reauthenticate every time, and that session history is shared between containers/your local Claude install.

Podman compose will automatically build the image if it doesn’t exist. If you want to trigger a rebuild, you can run it with the -b flag: claude-container -b /path/to/working/directory.

The first time you run the script, it will ask you if you’re okay bypassing permissions. Because we’re only mounting two directories, this should be much safer than it normally would be (provided you’re smart about which folders you pass).

The Dockerfile

The Dockerfile is based on the one from Claude’s devcontainer docs. I’ve stripped out some development helpers since I’m running Claude directly in the CMD, not using the shell.

I previously copied the working directory into the image, but have since moved to using bind mounts only. If I want to make sure that Claude doesn’t touch any files on my machine, I just create a copy of the working directory to bind mount to the container.

Another important thing I removed from Claude’s devcontainer Dockerfile is the firewall setup with init-firewall.sh. Limiting outgoing traffic is a good idea, but the repos I was working with weren’t sensitive enough for me to spend the time getting this working (especially since it seems you need to run the container with the NET_ADMIN capability, which I don’t understand well enough to be using).

Running this with no firewall is dangerous, but less so than running the agent outside of a container.

No container-ception, unfortunately

At some point, I wanted to give Claude access to a backend dev server (it was running a frontend dev server in the container). But the backend dev server runs in a container itself, so I couldn’t just have Claude spin up the backend dev server in its container.

After trying a few things, my workaround was to connect Claude’s container to the backend container’s Podman network. Once all your containers have been started, find the relevant container network with:

podman network ls

If you’re not sure which is the right network, you can check which containers are connected to a network using podman network inspect <network_name>. To connect Claude to the desired network, run:

podman network connect <relevant container's network> <claude container>

And of course, if this container is connected to something important (like a database), this will allow Claude access to it. Always be careful before letting an LLM have access to data. In my case, this was an empty database for development, so in terms of exposing data to Claude, it was similar to running the backend dev server within the container.

A few notes

Claude won’t have access to some important developer tools in the container (e.g., Playwright). If you want to give it more tools, you could modify the Dockerfile to install them at build time. I tried installing Playwright at first, but found I actually prefer manually testing outside of the container & copying output back to Claude. I appreciate the extra time to slow down, check Claude’s decisions, and catch bugs.

This setup intentionally doesn’t give your container SSH access — I don’t ever want an LLM to have access to my SSH key, and I would suggest thinking very hard before giving it access to any servers. That’s one of the big reasons I don’t want to run Claude on my own machine, especially when using an SSH agent.

I’ve enjoyed developing using Claude Code from within the confines of a single folder. I get the solid code analysis and generation without giving away an excessive amount of data.

Why I Am Focusing on Intelligent Document Processing

2026-03-27T00:00:00+00:00

AI moves fast. A new model drops, a new framework launches, a new thing comes out, and suddenly what you learned last month feels old. LLMs, agents, fine-tuning, RAG, computer vision, multimodal models, prompt engineering, AI coding tools — the list keeps growing and it is hard to know where to focus.

Recently I decided to stop trying to follow all of it and focus on one area. In this post, I will explain what that area is, how I found it, and why I think it is worth paying attention to.

The problem with being an AI generalist

When AI started becoming a practical tool for software engineers (not just researchers), I jumped in. I wrote about deploying LLMs with Mixture of Experts, built an LLM-powered blog search, and worked on AI extraction pipelines for document processing. At work, our team was building document processing systems with LLMs. On the side, I was reading about everything else.

I was learning a lot, but I could not clearly say what I specialize in. If someone asked, “What is your thing in AI?” my answer was too broad to be useful.

Looking at the pattern in the projects

The turning point was stepping back and looking at the projects I had been involved in. Not the articles I had been reading, but the real systems I had been working on with the team.

At End Point, one of the main projects our team has been working on is an AI order processing system. The problem was simple: a client receives purchase orders from dealers and resellers via email as PDFs, attachments, and sometimes just plain body text. Before our solution, someone had to open each email, read what was being ordered, and manually type the data into the ERP system. Invoicing addresses, delivery addresses, line items, quantities, all entered by hand.

The solution is a pipeline that takes those raw email exports and PDF attachments, extracts structured data using LLMs with a defined schema, validates the output, handles exceptions through a human-in-the-loop review step, and produces a clean CSV that imports directly into the ERP. No more manual data entry.

Looking at these projects, I started seeing the same pattern repeat across different industries, different source documents, and different target systems. But they shared a problematic pattern:

Unstructured data in → AI extraction and validation → Structured data out → Business system integration.

That pattern has a name

I thought this was just a type of project that kept showing up. But when I researched it, I found out it is an established market with a formal name: Intelligent Document Processing (IDP).

IDP is about using AI (OCR, NLP, LLMs, computer vision) to automatically extract, classify, and transform data from unstructured documents like PDFs, emails, scanned forms, images, and legacy files into structured formats that plug into business systems.

This is not something small. Major research firms track it:

Gartner publishes a Market Guide for Intelligent Document Processing Solutions
IDC rated 22 IDP vendors in one of their MarketScape assessments

The vendor landscape includes names you already know: Google (Document AI), Microsoft (Azure Document Intelligence), Amazon (Textract), ABBYY, UiPath, Automation Anywhere, and many more.

The market is real and growing fast

The numbers vary by research firm because they define the market scope differently, but they all point in the same direction:

Source	2024–2026 Estimate	Projected By 2034	Annual Growth
Precedence Research	$4.3B (2026)	$43.9B	~34% per year
Fortune Business Insights	$14.2B (2026)	$91.0B	~26% per year

Growing at 26–34% per year makes IDP one of the fastest-growing areas in enterprise AI.

This matters because market growth creates demand for people who can actually build these systems. Not just people who buy vendor platforms, but people who can design extraction schemas, debug LLM outputs, build validation pipelines, and connect everything to real business systems.

What skills matter in IDP

Knowing the field has a name also made it clear what skills matter most. IDP work sits at the intersection of a few areas:

LLM-based data extraction: schema design, prompt engineering for structured output, few-shot examples for edge cases
Document parsing: handling PDFs, emails (.msg, .eml), scanned documents, HTML exports
Data validation and QA: confidence scoring, exception routing, human-in-the-loop review workflows
Pipeline architecture: from document intake to extraction, normalization, validation, and output
Business system integration: understanding ERP and CRM import formats, APIs, and data models

Why this matters

When a client has purchase orders stuck in email inboxes that aren’t processed fast enough, what they need is a reliable pipeline that extracts the right fields, handles the edge cases (like character encoding issues in .msg files), validates the output, and delivers a clean import file.

That is what IDP is about: turning data that businesses already have but cannot use into data that their existing systems can act on. Structured, validated output that goes into the ERP, the CRM, or the database, and saves real time and money.

Final thoughts

AI is broad enough that no one person can master all of it. But looking at the projects I was actually working on, instead of trying to follow every new trend, made it much easier to find direction. In my case, that field turned out to be Intelligent Document Processing.

If you need help parsing through legacy or unstructured data, we can help! Don’t hesitate to send us a message.

Why Your AI Extractor Fails on .msg Emails (and How to Fix Decoding)

2026-03-13T00:00:00+00:00

I want to share a debugging lesson that saved me from tuning the wrong layer in an AI extraction pipeline.

It started with a familiar symptom: extraction output looked inconsistent. Some rows were fine, but some had extra characters, especially accents. My first instinct was the same one most of us have: maybe the model needs prompt tuning.

It turned out not to be a model problem. The root cause was upstream data integrity: decoding .msg email HTML with the wrong charset.

The pattern that gives it away

If you see this mix, think decoding first:

output is mostly correct, but certain names and addresses look garbled
problems appear only for some senders or date ranges
.eml looks stable, but .msg is inconsistent

A classic sign looks like this:

expected: Müller
corrupted: MÃ¼ller

By the time your extractor sees that text, the meaning is already damaged.

Why `.msg` bites harder than `.eml`

Quick definitions:

.eml is the standard MIME email format and usually includes charset metadata per part.
.msg is an Outlook container format (MAPI), where body bytes and encoding hints can be stored separately.

That difference matters.

If your code assumes UTF-8 for .msg HTML bytes, non-UTF messages can decode into garbage. Then downstream steps (HTML-to-PDF, OCR, LLM extraction, post-processing) just preserve and propagate bad text.

The fix: strict, explicit, controlled

You do not need a big rewrite. A small decode policy change can remove a whole class of silent failures. For .msg HTML bytes:

Read the encoding hint from message metadata.
Map that hint to a decoder codec.
Decode in strict mode.
If needed, use one controlled strict fallback.
If decode still fails, fail loud.

Minimal example in Python:

from extract_msg.encoding import lookupCodePage

PR_INTERNET_CODEPAGE = "3FDE0003"

def decode_msg_html_bytes(html_bytes: bytes, message) -> str:
    codepage_id = message.getPropertyVal(PR_INTERNET_CODEPAGE)
    codec = (
        lookupCodePage(codepage_id)
        if isinstance(codepage_id, int) and codepage_id > 0
        else "utf-8"
    )
    try:
        return html_bytes.decode(codec, errors="strict")
    except (LookupError, UnicodeDecodeError):
        return html_bytes.decode("utf-8", errors="strict")

Why I prefer fail-loud here

errors="replace" keeps jobs moving, but it can hide real data corruption.

For low-stakes preview features, that may be acceptable. For transactional extraction (orders, invoices, legal, shipping), silent corruption is usually worse than an explicit failure.

Use this decision rule:

Use case	Policy
Preview/search UX	Best-effort can be acceptable with clear flags
Transactional extraction	Strict decode + fail loud
Mixed systems	Strict on extraction path, best-effort on preview path

How to roll this out safely

Keep blast radius low:

Change only the failing decode path first.
Validate on a representative dataset, not one sample file.
Leave unrelated paths untouched until evidence says otherwise.
Expand strict policy incrementally.

This gives reliability without destabilizing the rest of the ingestion stack.

Observability that makes this easier next time

Log these fields per message:

Source file
Content source used (HTML or plain text)
Whether encoding hint was found
Selected codec
Whether fallback was used
Result of decoding (success, fallback, manual review, fail)

With this, “random extraction quality” turns into a clear ingestion signal.

How to Mount a BitLocker Drive on Linux at Login

2026-02-12T00:00:00+00:00

I recently reinstalled Linux on my desktop machine alongside Windows on a separate SSD. I also have a 3TB hard drive for backups and slower storage, which I formatted on Windows. I didn’t manually enable any encryption, but I got the following error when trying to mount it directly:

mount: /mnt/<mydrive>: unknown filesystem type 'BitLocker'

Looks encrypted to me! After some googling, I found a common solution is to use dislocker to decrypt the drive.

I was confused because my drive wasn’t encrypted, but the filesystem type was still “BitLocker”. My guess is that Windows encrypted the drive by default, though as you’ll see soon, it uses a “clear key,” meaning it’s technically encrypted but the key is stored unencrypted, meaning anyone can decrypt the drive. I suppose they want me to set up encryption somehow, but since this is a home computer, I don’t mind it being unencrypted for now.

If your drive is encrypted, there are a few extra steps. There’s an excellent guide on std.rocks¹ for this. I’ll also annotate the fstab-specific section so you know where to put the recovery key there.

Installing dislocker & FUSE

From this point on, all of the commands need to be run as superuser. Add sudo before the command or run them from the root user. And, of course, proceed with caution and read your man pages before blindly trusting me :D

You can find dislocker packages in the default repos for most distros, or build it from source. I’m running Ubuntu right now, so I just had to run:

apt-get install dislocker ntfs-3g

I was able to mount the filesystem as read-only with the default ntfs driver, but to write I had to install ntfs-3g.

Disabling Fast Startup in Windows

If Fast Startup is enabled in Windows, the drive can get corrupted or be forced into read-only mode in Linux. To prevent this, disable Fast Startup by entering the following in an administrator PowerShell:

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Power\" -Name "HiberbootEnabled" -Value "0"

You can also disable it from the control panel — just search “Fast Startup”.

Mounting on login with `fstab`

It’s well and good to mount a drive one time, but since this is a persistent volume, I want it to be mounted consistently on boot. This is where fstab comes in.

To find a device, run lsblk, find the partition you’re looking for, and note the device name. You should see something like:

sda      8:0    0 232.9G  0 disk 
├─sda1   8:1    0     1G  0 part /boot/efi
└─sda2   8:2    0 231.8G  0 part /
sdb      8:16   0   2.7T  0 disk 
├─sdb1   8:17   0    16M  0 part 
└─sdb2   8:18   0   2.7T  0 part

In my case, I want to mount /dev/sdb2. First, though, we’ll need the device’s PARTUUID. Device names like /dev/sdb2 can change when you add or remove disks, so referencing a UUID is recommended in fstab’s man page to make sure you’re mounting the same disk every time. To get the PARTUUID, run the following:

blkid | sort

Here’s my output:

/dev/sda1: UUID="A1A1-A1A1" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="1a1a1a1a-1a1a-1a1a-1a1a-1a1a1a1a1a1a"
/dev/sda2: UUID="1a1a1a1a-1a1a-1a1a-1a1a-1a1a1a1a1a1a" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="1a1a1a1a-1a1a-1a1a-1a1a-1a1a1a1a1a1a"
/dev/sdb1: PARTLABEL="Microsoft reserved partition" PARTUUID="1a1a1a1a-1a1a-1a1a-1a1a-1a1a1a1a1a1a"
/dev/sdb2: TYPE="BitLocker" PARTLABEL="Basic data partition" PARTUUID="<mydrive-part-uuid>"

You should also be able to see the UUID after running lsblk -f, but for some reason, that command didn’t show me any disk information for /dev/sdb. It also didn’t show me the disk’s UUID, just the PARTUUID for each partition. That worked well enough for me, but please leave a comment if you know why that’s happening.

Open /etc/fstab in your favorite editor and add the following at the end:

/dev/disk/by-partuuid/<mydrive-part-uuid> /mnt/bitlocker  fuse.dislocker  nofail,clear-key                              0   0
/mnt/bitlocker/dislocker-file             /mnt/<mydrive>  ntfs            nofail,x-gvfs-show,x-gvfs-name=bitlocker-raw  0   0

I specified the clear-key option which is assumed in the dislocker command — the fuse.dislocker filesystem type takes dislocker’s long options. If your drive is encrypted, you’ll want to replace this with recovery-password=000000-000000-000000-000000-000000-000000-000000-000000 (recovery password is required if TPM is enabled) or user-password=123456 (a PIN code here works if TPM is disabled, according to std.rocks¹).

Now, I recommend trying to mount the drive before rebooting to avoid any startup issues if there are syntax errors or similar. After saving the file, run the following to get your fstab changes into systemd:

systemctl daemon-reload

Now you can mount the drive, then mount the dislocker-file just by naming the mount points:

mount /mnt/bitlocker
mount /mnt/mydrive

If you see files in /mnt/mydrive, hooray, it worked! Your drive should be mounted on reboot now. If not, there may be an error in your fstab file. I’d recommend trying the manual mounting method in the next section and noting at what step the failure happens.

Troubleshooting: manual mounting

I found it easier to diagnose and fix issues when I mounted manually first:

mkdir /mnt/bitlocker
dislocker -V /dev/sdb2 -- /mnt/bitlocker

If your drive is encrypted, dislocker should prompt you for a user or recovery password. Since mine isn’t, I instead used the default “clear key” stored on the volume. The guide on std.rocks¹ has more info for this.

If you check in the /mnt/bitlocker folder, you can see that there’s now a file called dislocker-file there. This file is what we mount:

mount -o loop -t ntfs /mnt/bitlocker/dislocker-file /mnt/<mydrive>

-o loop says that dislocker-file is a normal file you’re mounting via the loop interface
-t ntfs specifies that this drive is an NTFS filesystem, the default on modern Windows (post-1993, that is 😉)

After running this, you should see your files in /mnt/<mydrive>.

References

https://github.com/Aorimn/dislocker
man fstab

https://std.rocks/gnulinux_bitlocker.html ↩︎ ↩︎ ↩︎

Running Ruby in Podman (When rbenv install Fails on Apple Silicon)

2026-01-12T00:00:00+00:00

I recently worked on a legacy Ruby backend app which hadn’t been changed for years. A Ruby development environment used to be provided by DevCamps for this site, but over the years we stopped using Ruby other than this small backend, so we also stopped using camps.

The frontend development now uses its own local dev server, so I decided to do the same for the backend by running the Unicorn server locally. I’m on a MacBook M2, so I should just be able to install rbenv using Homebrew. Easy, right?

Unfortunately not. I had a great deal of trouble getting any version of Ruby to run natively on my MacBook M2, so I eventually resorted to using Podman. You can skip to the end for my solution.

First try: native rbenv

The rbenv installation went just fine:

brew install rbenv ruby-build

And after adding eval "$(rbenv init -)" to my ~/.zshrc file, I tried to install the old version of Ruby:

rbenv install 2.4.10

However, this returned a BUILD FAILED message. Strangely, it also failed with a new version of Ruby (3.3.10). I tried debugging for a while before giving up — I figured this project isn’t worth multiple hours of getting a local Ruby installation to work (despite my obstinate attempts to do so).

Second try: mise

Several people on forums & blogs recommended using mise, a language-agnostic version manager. After installing it with Homebrew, I ran:

mise use --global ruby@2.4.10

This failed too! There was a good explanation, though: older Ruby versions don’t run on ARM processors. Of course, that makes sense. So I just needed to install a newer version from after Apple silicon was supported:

mise use --global ruby@3.3.10

However, this failed as well, just like rbenv:

mise ERROR Failed to install core:ruby@3.3.10:
   0: ~/Library/Caches/mise/ruby/ruby-build/bin/ruby-build exited with non-zero status: exit code 1

I’m sure there is a way to get Ruby running through a version manager on Apple silicon; I found several tutorials claiming you can just run these commands as normal and it’ll work. However, after spending longer than I’d like to admit, I was completely unable to do so despite trying many fixes found online.

This may be a unique problem on my machine, or I may be missing something. But the fact was, I’d waited through about a dozen failed Ruby installs (which take a long time!) and I just needed the app to work now. So I moved to a third option: run Ruby in a container.

Third try: Podman

Podman is a fully open-source containerization system which can run Dockerfiles and docker-compose.yml files. The app is a lightweight Sinatra backend with Unicorn as a server, and I just needed to add CloudFlare Turnstile to reduce bot traffic.

The app uses two main files: unicorn.rb (which holds the configuration for the Unicorn server) and config.ru (which defines and runs the Sinatra app). The command to start the app is pretty simple: bundle exec unicorn -c unicorn.rb config.ru. I also set up two files for the container: Containerfile and container-compose.yml.

Note: Podman does find files named Dockerfile and docker-compose.yml, but here I’m using Podman’s convention of Containerfile and container-compose.yml, which takes precedence if both are present in the folder.

To run this, I just needed to set up the environment in my Containerfile.

`Containerfile`

FROM ruby:2.4.10

WORKDIR /usr/src/app

COPY . .

RUN mkdir -p /var/log && \
 gem i bundler && \
 bundle install

EXPOSE 8080

CMD ["bundle", "exec", "unicorn", "-c", "unicorn.rb", "config.ru"]

It’s a pretty simple setup:

set the working directory
copy files from the source directory into the container
create the log directory defined in unicorn.rb
install bundler
install gems from Gemfile.lock
expose the port defined in unicorn.rb
define the server start command using a CMD instruction

`container-compose.yml`

services:
  backend:
    build:
        context: .
        dockerfile: Containerfile
    volumes:
        - .:/usr/src/app
    ports:
        - "8080:8080"

The volumes section means that changes on the host machine will be propogated to the container (and vice versa). This means you don’t need to rebuild the container when you change the app, instead you can use a change-watching Gem like rerun.

Note: You can run this without a compose file, the compose file just stores the volume & port information so you can run it more easily.
podman run -d --name mybackend -v .:/usr/src/app -p 8080:8080 <image_id>

Minimal `unicorn.rb` configuration file

listen "0.0.0.0:8080"
working_directory "/usr/src/app"

This is simple:

Use 0.0.0.0 to connect to the container’s network interface
Define our listening port
Set the working directory to the same value as in our Containerfile

Note: When we forward port 8080 to our Podman container, that traffic arrives on the container’s network interface, not on its internal loopback (localhost). If your app only listens on localhost, it’s essentially saying “I only accept connections from myself,” so the forwarded traffic gets rejected. By binding to 0.0.0.0, your app listens on all interfaces, which includes the one Podman uses to route external traffic into the container.

Simple `config.ru`

require 'sinatra'
get '/' do
  "Hello from Sinatra in Podman!\n"
end
run Sinatra::Application

This is also very simple:

After importing sinatra, define a GET route for the / location (so this will be accessed from localhost:8080/ on the host machine)

$ podman build .
$ podman run -p 8080:8080 <image-id>

This returned a successful response on my host machine:

$ curl localhost:8080
Hello from Sinatra in Podman!

When the app failed to run, it kept trying to restart until I moved a .env file into place and it worked. That shows that the volume is working correctly.

I’m not a Ruby expert, but with this setup I was able to convert the JavaScript code for Turnstile to Ruby and get it working!

Quick and Dirty Admin Pages with Rails scaffold_controller

2025-12-29T00:00:00+00:00

You’re building a Rails application, and you’ve reached that point where you or your team needs a simple way to manage data. You need an admin interface. You’ve heard of heavy-weight solutions like ActiveAdmin or Rails Admin, but they feel like overkill for adding a few new categories or moderating user posts. You don’t want to spend an afternoon configuring a gem. You need something now.

Luckily Rails has a built-in lightning-fast way to generate a fully functional admin CRUD interface for any existing model. Meet the often-overlooked scaffold_controller generator. It’s the secret weapon for building “quick and dirty” internal tools.

What Is scaffold_controller and Why Use It?

When you run rails generate scaffold Post, it creates everything: the model, migration, controller, and views. The scaffold_controller generator does only half of that, creating only the controller and views (as long as the model already exists).

This is perfect for creating a separate admin namespace because:

It’s Non-Destructive: Your existing, user-facing PostsController remains untouched
It’s Fast: One command gives you a complete set of CRUD actions and ERB views
It’s Simple: No new gems, dependencies, or complex configurations to learn

Let’s Build an Admin Interface for a Product Model

Imagine you have a Product model in your application. You have a ProductsController for your main site, but now you need an admin area to create, edit, and delete products.

Step 1: Create the Admin Namespace and Route

First, we’ll set up a routing namespace to keep our admin logic separate. This will put our URLs in this format: /admin/products

Open your config/routes.rb file and add:

# config/routes.rb
Rails.application.routes.draw do
  # ... existing routes ...

  namespace :admin do
    resources :products
  end
end

Running rails routes will now show new routes like admin_products_path, edit_admin_product_path, etc.

Step 2: Generate the Scaffold Controller

Now for the magic. In your terminal, run rails generate scaffold_controller Admin::Product

Important notes:

The namespace (Admin::) must match the one you used in your routes
The model name (Product) must be singular, just like with a regular scaffold

This one command generates several files:

Controller: app/controllers/admin/products_controller.rb
Views: A full set of ERB templates in app/views/admin/products/ (index.html.erb, show.html.erb, new.html.erb, edit.html.erb, _form.html.erb)

Step 3: Update the Generated Controller

The generated controller is a great start but it needs one crucial tweak. It doesn’t know about our namespace, so it will look for the Product model in the wrong place. The key change is ensuring every reference to the model is to Product, not Admin::Product.

Open products_controller.rb and change the class definition and any model references. Here’s a simplified example of what it should look like:

# app/controllers/admin/products_controller.rb
module Admin
  class ProductsController < ApplicationController
    before_action :set_product, only: [:show, :edit, :update, :destroy]

    # GET /admin/products
    def index
      @products = Product.all
    end

    # ... other actions ...

    private
      def set_product
        @product = Product.find(params[:id])
      end

      def product_params
        params.require(:product).permit(:name, :description, :price, :stock)
      end
  end
end

Step 4: Add a Link and Access Control

Your admin interface is now live at http://localhost:3000/admin/products. Before you deploy you must add some form of access control. This can be as simple as a basic HTTP authentication check in your Admin::ProductsController.

# app/controllers/admin/products_controller.rb
module Admin
  class ProductsController < ApplicationController
    http_basic_authenticate_with name: ENV['ADMIN_USER'], password: ENV['ADMIN_PASSWORD']

    # ... rest of the controller code ...
  end
end

It would also be good to add a link in your navigation for easy access:

<%= link_to "Admin Products", admin_products_path if Rails.env.development? %>

When Should You Use This Method?

Ideal for:

Internal tools, administrative, and support functions
Rapid prototyping
Simple CRUD needs where a full-blown admin gem is overkill

Not ideal for:

Customer-facing admin panels that need complex filtering, dashboards, or custom forms
Applications where you need fine-grained, role-based permissions

Conclusion

The scaffold_controller generator is a fantastic piece of Rails tooling that is often overlooked. It embodies the Rails philosophy of providing quick solutions for common problems. In just a few minutes you can have a functional, separate admin area for any model without the overhead of a new gem or the risk of breaking your public-facing code. So next time you need a simple data management interface, skip the gem research and let scaffold_controller do the heavy lifting for you.

End Point Is a Top Cloud Consulting Company on Techreviewer

2025-12-08T00:00:00+00:00

We recently received some good news: Techreviewer.co included us in their list of top cloud migration consulting companies for 2025! While we don’t usually dwell on rankings, it’s always rewarding when thoughtful, steady work over many years gets noticed.

Cloud adoption has been a major part of our engineering efforts since it became “a thing”: For many of our clients, moving to the cloud isn’t just a technology upgrade, but a strategic shift that affects performance, costs, security, and long-term maintainability. Helping organizations navigate that transition in a careful and grounded way is something we’ve been doing since long before “cloud migration” became a buzzword.

Our approach to cloud migration

At End Point Dev, we’ve always taken a pragmatic path: understand the real constraints, design strong systems that will age well, and keep things secure and predictable. Cloud work touches many layers, and affects DevOps, databases, applications, and ongoing operations. Our team of software engineering and infrastructure experts supports our clients through that entire lifecycle.

Our day-to-day cloud work includes, but is not limited to:

Planning and executing migrations to AWS, Azure, Google Cloud, and others
Automating deployments and CI/CD pipelines for safer, faster releases
Scaling and optimizing PostgreSQL and other databases
Designing secure and resilient infrastructure that is ready to keep growing
Building custom software that fits cleanly into modern cloud ecosystems
Supporting and operating hosted environments around the clock

Using containerization and orchestration tools like Docker and Kubernetes to improve reliability is key, when the project and client needs requires it.

Why this matters

This acknowledgment isn’t something we chased, but it does capture what we care deeply about: Having a staff that brings a good mix of application knowledge and infrastructure engineering, and prioritizing client trust and long-term support.

Our cloud migration frameworks meet the highest standards of quality and reliability
Our DevOps, infrastructure, and database expertise place us among industry leaders
Our client-centered approach sets us apart in a rapidly evolving landscape
Our long-term support model delivers value far beyond the initial project

A lot of what we do happens behind the scenes, especially in maintenance, monitoring, and incident response. So it’s nice for us to see that steady reliability reflected in external evaluations.

Looking ahead

The landscape of cloud computing moves quickly, and we like keeping pace. In the coming year, we will be focused on strengthening our multi-cloud capabilities, expanding our tooling around automation and observability, deepening AI integration within our daily processes, and exploring new AI-driven approaches to optimization and scaling.

We extend our sincere gratitude to our clients, partners, and team members. Every project, migration, deployment, and late-night incident response has helped shape End Point Dev’s reputation for reliability and excellence. This recognition is a tribute to your trust and collaboration. We look forward to continuing to support organizations worldwide with solutions that are robust, secure, and thoughtfully engineered.

Using ActiveSupport::ExecutionContext to improve Rails logging

2025-12-05T00:00:00+00:00

If you’ve ever tried to debug a complex user workflow in a modern Rails application, you know how difficult it can be. A single web request can spawn multiple background jobs, Turbo Stream updates, and a flurry of database queries. Answering the question “What was this user doing?” should be simple, but tracing the flow of events through a web of log files can be difficult and time consuming.

You can add custom log tags but that gets tedious and messy fast. Fortunately, Rails has a powerful, under-documented feature designed specifically to address this problem: ActiveSupport::ExecutionContext.

In this post, we’ll walk through what ExecutionContext is and how you can use it to add meaningful structure to your logs, making debugging a much more straightforward task.

The Problem: A Tangled Web of Logs

Imagine a user places an order on your site. The OrdersController#create action fires, which then enqueues a ReceiptJob and an InventoryUpdateJob. The controller also renders a Turbo Stream to update the UI. You have at least four separate units of work: the HTTP request and three background tasks.

Now, if the InventoryUpdateJob fails, your log might show an exception, but it won’t immediately tell you which user’s order triggered it. You’re left grepping for job IDs or tracing timestamps. ExecutionContext solves this problem by providing a shared context that is automatically shared across these different units of work.

How ExecutionContext Works

Think of ActiveSupport::ExecutionContext as a container for data that automatically gets passed along. When you store something in it during a web request, that data is bundled up and made available in any background jobs you start from that request without you having to manually send it.

It’s important to note that while ActiveSupport::ExecutionContext values are available within the same process (e.g., during the web request), they don’t automatically serialize and propagate to background jobs. For the context to be available in jobs, you’ll need to explicitly pass relevant values as job arguments. In our example, since we’re already passing the order object to both jobs, we can access order.user_id and order.id directly in the job. For contexts without such natural carriers consider extracting the relevant values and passing them explicitly as additional job arguments.

This is the magic that allows you to trace a chain of events.

A Practical Example: Tracing an Order

Let’s implement a solution for the ordering scenario. Our goal is to tag every log line related to this specific order with the user’s ID and the order ID.

First, we’ll set the context in an around_action in our ApplicationController:

# app/controllers/application_controller.rb
class ApplicationController < ActionController::Base
  around_action :set_execution_context

  private

  def set_execution_context
    # We only set the context if a user is logged in.
    if current_user
      ActiveSupport::ExecutionContext[:user_id] = current_user.id
      # We can also trace requests
      ActiveSupport::ExecutionContext[:request_id] = request.request_id
    end

    yield
  ensure
    # Ensure the context is cleared after the request is done.
    ActiveSupport::ExecutionContext.clear
  end
end

# app/controllers/orders_controller.rb
class OrdersController < ApplicationController
  def create
    @order = current_user.orders.create!(order_params)

    # Update the context with the real order_id
    ActiveSupport::ExecutionContext[:order_id] = @order.id

    ReceiptJob.perform_later(@order)
    InventoryUpdateJob.perform_later(@order)

    respond_to do |format|
      format.turbo_stream
    end
  end
end

Because we used perform_later, Active Job will automatically serialize our ExecutionContext (containing user_id and order_id) and make it available when the job runs.

Let’s look at the InventoryUpdateJob:

# app/jobs/inventory_update_job.rb
class InventoryUpdateJob < ApplicationJob
  def perform(order)
    logger.info "Updating inventory for order"

    # ... your business logic ...
  end
end

Making the Context Visible in Logs

Setting the context is only half the battle; we also need to see it in our logs. We can do this by customizing Rails’s log formatter.

Here’s a simple formatter that appends the execution context to every log line:

# config/initializers/log_formatting.rb
class ContextAwareFormatter < Logger::Formatter
  def call(severity, time, progname, msg)
    context = ActiveSupport::ExecutionContext.to_h

    tags = context.map { |key, value| "#{key}=#{value}" }.join(" ")
    tags = "[#{tags}] " unless tags.empty?

    "#{time.utc.iso8601(3)} #{severity} ##{Process.pid} #{tags}#{msg2str(msg)}\n"
  end
end

# Apply it to the Rails logger
Rails.logger.formatter = ContextAwareFormatter.new

With this in place, a log line from our InventoryUpdateJob might now look like this:

2025-12-01T15:33:01.123 INFO #123 [user_id=1001 order_id=998842] Updating inventory for order

If that job fails, the stack trace will be prefixed with the same [user_id=1001 order_id=998842] context. You can instantly see which user and order were affected.

Going Beyond: Tagging Database Queries

One of the most powerful applications is tagging database queries. This is incredibly useful for identifying expensive queries related to a specific user in a production environment.

You can subscribe to the SQL event and include the context in the log:

# config/initializers/log_formatting.rb
ActiveSupport::Notifications.subscribe("sql.active_record") do |event|
  context = ActiveSupport::ExecutionContext.to_h
  next if context.empty?

  payload = event.payload
  sql_with_context = "/* #{context.map { |k, v| "#{k}:#{v}" }.join(', ')} */ #{payload[:sql]}"

  Rails.logger.debug("SQL: #{sql_with_context}")
end

Here is what that log line would look like:

DEBUG -- : SQL: /* user_id:1001, order_id:998842 */ SELECT "orders".* FROM "orders" WHERE "orders"."user_id" = $1 LIMIT $2  [["user_id", 1001], ["LIMIT", 1]]

The SQL notification subscriber will fire on every database query which in production could introduce noticeable overhead. Consider wrapping this functionality in a Rails.env.development? check or using feature flags to control its activation. Be mindful of this trade-off when adding context to high-volume SQL logging.

A Stitch in Time Saves Nine

ActiveSupport::ExecutionContext is a robust solution to a common problem in modern, event-driven Rails applications. It provides a clean, built-in mechanism for propagating context, which leads to more debuggable and observable systems.

The next time you find yourself lost in a sea of log files, remember this tool. By adding a few lines of code to set the context and customizing your log formatter, you can transform a chaotic log file into a well-organized story of what your application is doing for each and every user.

Build a Smarter Telegram Bot: Integrating a RAG Pipeline for FAQ Answering

2025-12-01T00:00:00+00:00

In this post, we will show you how to build an intelligent FAQ bot using Python and the Telegram Bot API. We’ll go beyond simple commands by integrating a Retrieval-Augmented Generation (RAG) pipeline with LangChain.

This RAG pipeline lets our bot pull information from a custom knowledge base (in our case, a simple faqs.json file) and use a local Large Language Model (LLM) through Ollama to generate accurate answers. The best part? This approach (which works great with interfaces like Open WebUI) gives you full control over your models and data with zero API costs.

What is Telegram?

You’ve probably heard of Telegram—it’s a popular, cloud-based instant messaging app. It’s fast, works everywhere (mobile, web, and desktop), and has powerful features like huge group chats and easy file sharing.

One of its most powerful features for developers is the Telegram Bot API, an open platform that allows anyone to build and integrate automated applications (like ours!) directly into the chat interface.

A Warning on Privacy and Encryption

Before we build our bot, it is critical to understand how Telegram handles encryption, as it directly impacts user privacy.

Cloud Chats (The Default): All standard chats, group chats, and all bot interactions are “Cloud Chats.” These use server-client encryption. This means your messages are encrypted between your device and Telegram’s servers, and then stored (encrypted) on their servers. This is what allows you to access your chat history from any device. However, Telegram itself holds the encryption keys and can access this data.
Secret Chats (Manual): Telegram also offers “Secret Chats,” which are end-to-end encrypted (E2EE). In this mode, only you and the recipient can read the messages. Telegram has no access. However, bots cannot operate in Secret Chats.

This means that any message a user sends to our bot is a “Cloud Chat” and is not end-to-end encrypted. The data is accessible to Telegram and will be processed in plain text by our bot.py script on our server.

For this reason, you should never build a bot that asks for or encourages users to send sensitive private data such as passwords, financial information, or social security numbers. Always treat bot conversations as non-private.

What is Retrieval-Augmented Generation (RAG)?

Retrieval-Augmented Generation (RAG) is a technique that makes Large Language Models (LLMs) smarter by connecting them to external, private knowledge.

The Problem: An LLM like llama3 only knows the information it was trained on. It has no access to your company’s internal FAQs, new documents, or any private data.
The Solution (RAG): RAG solves this in two steps:
- Retrieve: When you ask a question, the system first retrieves relevant information from your own knowledge base (for us, our faqs.json file).
- Augment: It then augments the LLM’s prompt by pasting that retrieved information in as context, along with your original question.

In short, instead of just asking the bot “What’s the shipping policy?”, we’re effectively asking, “Based on this specific text: ‘…We offer standard shipping…’ — what is the shipping policy?” This forces the LLM to base its answer on our facts, not its own general knowledge, making the response accurate and reliable.

What you’ll build

Telegram bot
faqs.json knowledge base
RAG pipeline with local embeddings (FAISS) + LLM (Open WebUI)

Prerequisites

You’ll need:

Python 3.12+
a Telegram bot token (from BotFather)
access to an LLM via a locally hosted Open WebUI instance (OpenAI-compatible API)

Setting up the Project for Telegram Bot

uv is a high-performance Python package manager, so we’ll use it to set up our project. If you don’t have it installed, you can get it with or visit the site for installation steps:

pip install uv

Create a new project directory and navigate into it:

mkdir telegram-rag-bot
cd telegram-rag-bot

Initialize a new Python project.

uv init --bare

This command creates a minimal pyproject.toml file. This file will track our project’s metadata and, most importantly, its dependencies.

Create a virtual environment using uv:

uv venv

This will create a .venv directory. Activate it with the following:

source .venv/bin/activate

# On Windows, use
#.venv\Scripts\activate

Install the necessary Python packages using uv:

uv add python-telegram-bot python-dotenv langchain langchain-openai langchain-community faiss-cpu jq sentence-transformers

The key libraries are:

python-telegram-bot: For handling all Telegram communication
langchain: The primary framework for building the RAG pipeline
langchain-openai: Connector to Open WebUI’s OpenAI-compatible API
faiss-cpu: An efficient library for similarity search, used as a local vector store to quickly find relevant chunks of your FAQ data

Environment and configuration

The bot reads the Telegram token from the environment variable BOT_TOKEN. We can store it in a .env file as BOT_TOKEN=your-token-here.

# .env (Open WebUI)
# Open WebUI_URL must end with /v1 (e.g., http://localhost:3000/v1).
BOT_TOKEN=123456:abcdefg
Open WebUI_URL=http://localhost:3000/v1
Open WebUI_API_KEY=your_key_here

Inline mode requires enabling inline for the bot via BotFather.

Create a new file named bot.py and add the following code to set up and add message handlers for the Telegram bot.

import logging
import os
from uuid import uuid4
from telegram import Update, InlineQueryResultArticle, InputTextMessageContent
from telegram.ext import filters, MessageHandler, ApplicationBuilder, CommandHandler, ContextTypes, InlineQueryHandler
from dotenv import load_dotenv

# load .env variables
load_dotenv()
bot_token = os.getenv("BOT_TOKEN", "")

# Setup logging
logging.basicConfig(
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    level=logging.INFO
)
logger = logging.getLogger(__name__)

async def start(update: Update, context: ContextTypes.DEFAULT_TYPE):
    await context.bot.send_message(chat_id=update.effective_chat.id, text="I'm a bot, please talk to me!")

async def echo(update: Update, context: ContextTypes.DEFAULT_TYPE):
    await context.bot.send_message(chat_id=update.effective_chat.id, text=update.message.text)

async def caps(update: Update, context: ContextTypes.DEFAULT_TYPE):
    text_caps = ' '.join(context.args).upper()
    await context.bot.send_message(chat_id=update.effective_chat.id, text=text_caps)

async def inline_caps(update: Update, context: ContextTypes.DEFAULT_TYPE):
    query = update.inline_query.query
    if not query:
        return
    results = []
    results.append(
        InlineQueryResultArticle(
            id=str(uuid4()),
            title='Caps',
            input_message_content=InputTextMessageContent(query.upper())
        )
    )
    await context.bot.answer_inline_query(update.inline_query.id, results)

async def unknown(update: Update, context: ContextTypes.DEFAULT_TYPE):
    await context.bot.send_message(chat_id=update.effective_chat.id, text="Sorry, I didn't understand that command.")

async def document(update: Update, context: ContextTypes.DEFAULT_TYPE):
    if update.message.document:
        file = await update.message.document.get_file()
        file_name = update.message.document.file_name
        await file.download_to_drive(file_name)
    elif update.message.photo:
        # Get the largest photo size
        file = await update.message.photo[-1].get_file()
        file_name = f"photo_{file.file_unique_id}.jpg" # Create a unique name for photos
        await file.download_to_drive(file_name)
    elif update.message.video:
        file = await update.message.video.get_file()
        file_name = update.message.video.file_name
        await file.download_to_drive(file_name)
    else:
        await update.message.reply_text("Please send a document, photo, or video.")
        return

def main() -> None:
    start_handler = CommandHandler('start', start)
    echo_handler = MessageHandler(filters.TEXT & (~filters.COMMAND), echo)
    caps_handler = CommandHandler('caps', caps)
    inline_caps_handler = InlineQueryHandler(inline_caps)
    document_handler = MessageHandler(filters.PHOTO | filters.Document.PDF | filters.VIDEO, document)
    unknown_handler = MessageHandler(filters.COMMAND, unknown)

    application = ApplicationBuilder().token(bot_token).build()

    application.add_handler(start_handler)
    application.add_handler(echo_handler)
    application.add_handler(caps_handler)
    application.add_handler(inline_caps_handler)
    application.add_handler(document_handler)
    application.add_handler(unknown_handler)

    # Run the bot
    logger.info("Starting bot polling...")
    application.run_polling()

if __name__ == '__main__':
    main()

Try it out

To run the application, simply run:

uv run bot.py

Search for your bot name in Telegram and send the bot a message or command like /start or /caps.

What this bot does

Responds to /start with a greeting
Echoes back any plain text message (that isn’t a command)
Converts text to uppercase via /caps or inline mode
Downloads files users send (photos, PDFs, and videos) to local storage
Politely handles unknown commands

Core structure

start(): Sends a simple welcome message when the user runs /start
echo(): Replies with the exact same text the user sent (only for non-commands)
caps(): Turns the arguments after /caps into uppercase and sends them back
inline_caps(): Provides an inline result that uppercases whatever users type after @YourBotName in any chat
document(): Saves received media to disk:
- Photos: Downloads the largest size, naming it photo_<unique_id>.jpg
- PDFs: Downloads using the document’s file name. Note: The filter only accepts PDFs as documents
- Videos: Downloads using the video’s file name
- If none of these are present, it prompts the user to send a supported file
unknown(): Catches any unrecognized commands and replies with a friendly error

Handlers and filters

CommandHandler('start', start) and CommandHandler('caps', caps) handle commands
MessageHandler(filters.TEXT & (~filters.COMMAND), echo) ensures normal text (not commands) is echoed
InlineQueryHandler(inline_caps) answers inline queries
MessageHandler(filters.PHOTO | filters.Document.PDF | filters.VIDEO, document) restricts downloads to photos, PDFs, and videos
MessageHandler(filters.COMMAND, unknown) is added last to catch all other commands

Running the bot

main() wires up the handlers, builds the Application with the token, logs a startup message, and starts long polling via application.run_polling().

This script is a clean, async-first Telegram bot scaffold that demonstrates commands, inline mode, message filtering, and media downloads—ready to extend for more sophisticated behaviors.

Now that we have our bot ready, we will extend the code to add a RAG pipeline to the bot.

Setting up the knowledge base

Let’s set up a knowledge base by creating a file named faqs.json to hold our data. The RAG pipeline will load and search this content. An example structure is shown below.

[
 {
    "category": "General",
    "question": "What are your operating hours?",
    "answer": "Monday to Friday, 9:00 AM–5:00 PM (local time)."
  },
  {
    "category": "Accounts",
    "question": "How do I reset my password?",
    "answer": "Go to our website, click Login, then Forgot Password. Check your email for the reset link."
  }
]

Setting up the RAG Pipeline

The RAG pipeline is the engine that converts our static JSON file into a searchable brain for our bot. This part initializes once and creates a vector database. In simple steps,

Load faqs.json
Create embeddings
Store them in FAISS
When a user asks a question, find similar answers and ask the LLM to write a reply based only on those

Data Ingestion (Indexing the FAQs) handled by `setup_rag_chain()` method

This part happens once when the bot starts. We load the faqs.json file, create vector embeddings, and store them in a searchable database (FAISS).

Load Data: Read the faqs.json file
Embeddings: Use an embedding model (like HuggingFace (Sentence-Transformers) all‑MiniLM‑L6‑v2.) to convert the text into numerical vectors
Vector Store: Store these vectors in a FAISS index for fast retrieval

The RAG Retrieval Logic handled by `handle_message()` method

When a user asks a question:

Embed Query: The user’s question is converted into an embedding vector
Retrieve Context: The query vector is used to perform a similarity search against the FAISS index. This returns the top K most relevant FAQs (question and answer pairs)
Construct Prompt: A final prompt is built, containing the user’s question and the retrieved relevant context

Example Prompt Template:

You are an expert FAQ assistant. Use the following context to answer
the user's question. If the context does not contain the answer,
state that you cannot help with this specific question. Context:
[Retrieved FAQs] Question: [User's message]

Generate Response: The complete prompt is sent to our Open WebUI model via the OpenAI-compatible API, e.g. gpt-5 or another model exposed by Open WebUI, which generates a coherent, context-grounded final answer
Send to Telegram: The bot sends the LLM’s final response back to the user

New capabilities added to `bot.py`

RAG pipeline: Loads FAQs from a local JSON file, embeds them with HuggingFace, retrieves the most relevant entries via FAISS, and drafts answers with an LLM served by Open WebUI.
Inline UX polish: Sends a “typing…” chat action while the model thinks.
Persisted chain: The RAG chain is built once at startup and stored in bot_data for reuse across messages.

The core logic of our bot will revolve around an update to the standard message handler. When a user sends a question, the bot no longer looks for a simple command; instead, it passes the question to the RAG pipeline.

Try it out

To run the application, simply run

uv run bot.py

Search for your bot name in telegram and send the bot a message like What are your operating hours?.

Changes to the RAG pipeline setup are available here. The source code is available here.

By integrating a RAG pipeline, we’ve leveled up our Telegram bot from a simple command processor to a knowledge-aware assistant. This approach ensures our bot’s answers are accurate, grounded in our provided faqs.json data, and remain consistent, dramatically reducing the chance of “hallucinations” from the underlying LLM.

This architecture is powerful and scalable. To expand its capabilities, we only need to update the faqs.json file and re-run the indexing step—no need to retrain or modify the core LLM!

Migrating from Legacy Networking to systemd-networkd on Ubuntu 24.04

2025-11-25T00:00:00+00:00

During a recent Ubuntu server upgrade, I migrated a production system from the legacy ifupdown networking stack to the modern systemd-networkd service. This migration was part of preparing several remote production servers for a smooth upgrade to Ubuntu 24.04, which relies heavily on systemd-managed components.

In this post, I’ll walk through the full migration process, how we automated it safely across multiple remote servers, and the lessons learned from implementing rollback and watchdog mechanisms for zero-downtime transitions.

Why migrate to systemd-networkd?

Ubuntu 24.04 uses systemd-networkd as the default backend for network management. The traditional ifupdown scripts (/etc/network/interfaces) are no longer the recommended way to configure networking. Some of the modern features enabled by systemd-networkd include:

Predictable network interface naming (e.g., ens3, eth0, etc.)
Built-in support for bridges, VLANs, and bonds
Faster boot times with asynchronous network handling
Unified configuration under /etc/systemd/network

Migrating now also avoids future compatibility issues and takes advantage of systemd’s integrated logging and management.

Step 1: Check the current setup

Before the migration, I confirmed the system was still using ifupdown:

ls /etc/network/interfaces
cat /etc/network/interfaces

Output showed legacy configuration similar to:

auto eth0
iface eth0 inet static
    address 10.42.41.1
    netmask 255.255.0.0

Step 2: Install and enable systemd-networkd

Since systemd-networkd wasn’t already active, I installed and enabled it:

sudo apt install systemd-networkd systemd-resolved -y
sudo systemctl enable systemd-networkd
sudo systemctl enable systemd-resolved

Then, I stopped ifupdown to prevent conflicts.

Step 3: Create a network configuration file

Each interface now has its own .network file under /etc/systemd/network/.

DHCP-based interface:

# /etc/systemd/network/10-eth0.network
[Match]
Name=eth0

[Network]
DHCP=yes

Static interface:

# /etc/systemd/network/20-eth1.network
[Match]
Name=eth1

[Network]
Address=10.42.41.1/16
Gateway=10.42.0.1
DNS=8.8.8.8
DNS=1.1.1.1

Step 4: Configure systemd-resolved

This step ensures your system’s DNS resolver works correctly with systemd-networkd.

systemd-resolved listens locally on 127.0.0.53 and handles DNS resolution, caching, and per-interface settings.

Link your system’s /etc/resolv.conf to use it:

sudo ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

Then to verify:

resolvectl status

If we skip this, the system might lose DNS resolution after reboot. Applications like apt, curl, and ping may fail to resolve hostnames even though interfaces are up.

Restart and verify

Restart both services:

sudo systemctl restart systemd-networkd
sudo systemctl restart systemd-resolved

Check active links:

networkctl list

Detailed view for one interface:

networkctl status eth0

Sample output:

● 2: eth0
       Type: ether
      State: routable (configured)
     Address: 10.42.41.1/16
     Gateway: 10.42.0.1
     DNS: 8.8.8.8

Test:

ping -c 3 8.8.8.8
ping -c 3 google.com

Step 6: Reboot and confirm

Reboot once to ensure everything persists. After the reboot,

systemctl status systemd-networkd
systemctl status systemd-resolved

and confirm interface and DNS are still functional.

Handling remote migration and downtime risks

Performing this migration remotely on production systems without physical console access required special care. I prepared several layers of protection to prevent accidental lockouts.

Console or IPMI verification: Ensured each node had out-of-band access in case SSH connectivity was lost.

Timed rollback safety script: A background process automatically reverted to legacy networking if the new configuration didn’t come up within a few minutes.

#!/bin/bash
sleep 300
if ! ping -c1 8.8.8.8 &>/dev/null; then
    systemctl disable systemd-networkd
    systemctl enable networking
    reboot
fi

Automating the migration and rollback process

To streamline upgrades across multiple servers, I created automation scripts that handle migration, rollback, and ongoing monitoring.

migrate_to_systemd_networkd.sh

This script performs a complete, logged migration with built-in validation and rollback. It checks for valid .network files, disables legacy services, enables systemd-networkd, tests connectivity, and if all retries fail, restores the old configuration automatically.

Key features

Validates configuration syntax using systemd-analyze verify
Masks conflicting services (ifup@eth*, NetworkManager)
Performs up to three connectivity tests
Restarts SSH and dependent services
Schedules a controlled reboot or user-confirmed one
Rolls back cleanly if connectivity fails

upgrade_watchdog.sh

During do-release-upgrade, networking may restart multiple times. This watchdog script runs in the background to ensure systemd-networkd remains active and services like SSH come back automatically.

Executes every 10 minutes
Verifies /etc/systemd/network/*.network exists and services are running as expected if it fails, this script restarts the service
Restarts systemd-networkd, ssh, and dependent services

Together, both scripts allowed us to perform fully remote OS upgrades with zero manual downtime and consistent recovery paths.

Conclusion

Migrating to systemd-networkd modernizes Ubuntu servers and simplifies long-term maintenance. With the automation scripts and safety mechanisms in place, we successfully migrated multiple remote servers with no downtime and automatic rollback protection.

This upgrade not only prepares infrastructure for Ubuntu 24.04 but also integrates cleanly with our automation and monitoring systems, ensuring reliable networking for years ahead.

Adventures in Vibe-Coding with Replit

2025-10-29T00:00:00+00:00

A few weeks back, I tried out Replit Agent to see how viable it is as a development tool. I built two apps from scratch, which took 10–15 hours total, with a decent amount of human input along the way to clarify the agent’s questions.

Project 1: Multi-LLM code review API

The idea for the first project was to have a multi-LLM code review app, where you would pass a repository on GitHub or locally to an API, and it would run several full-repo code reviews, ranking and deduplicating them before returning a response. This setup should allow me to have a web frontend or CLI without much added complexity.

The prompt

Please build an app based on this app specification: app_spec.yaml (460 lines)

The results

After much back and forth (and about 40 agent-generated Git commits), I got this message:

🎉 SUCCESS! The Multi-LLM Code Review Assistant is now fully working!

Well, I actually got this message several times after sending just about any command to the agent. It’s very excited about its own work, even when it doesn’t work! In this case, the API connections were working, and testing with curl gave output, but I couldn’t get it to give more than one code review tip for a large codebase — not a very effective code-review app.

I tried asking the agent to diagnose and fix this:

So, this seems to not be returning very many code review suggestions. If I go
look at the repos I'm putting in, there are plenty of code issues that could be
returned, especially with documentation. How can we make this system find more
issues?

In response, Replit’s agent created a full React frontend for the API! Neat, but far from what I asked — the app still doesn’t return substantial code reviews.

So after ~$100 of agent credits, I found that Replit was not the tool for this app — I’m now trying to build a CrewAI app to accomplish the same thing (stay tuned for a post about that).

Project 2: End Point Ecommerce + Hugo site

A common theme in Replit apps we’ve experimented with is that it prefers to make everything a web app with a React frontend. I was curious how it’d do with a different web stack: The Hugo static site generator for the frontend and our recently launched End Point Ecommerce for the backend.

This is an interesting problem for an LLM, since Hugo is widely used (including on this site), but still small compared to React, Vue, and similar frameworks, and since End Point Ecommerce is well-documented but too new for any online discussion. However, it’s a modern .NET ecommerce framework and the code is readable and straightforward.

The prompt

Build me a hugo website frontend to interact with end point ecommerce:
https://ecommerce.endpointdev.com/

For the UI, make a halloween-themed storefront with orange text, black and grey
backgrounds, and halloween-themed test products. Use the Ballast font for
headers, and charter for body text for now.

After providing this initial prompt, I followed up with these:

Here is the github link for end point ecommerce, which this site will connect to:
https://github.com/EndPointCorp/end-point-ecommerce

Please make the app as simple as possible, while still working

Oh, and please use this demo API to connect to for now, so you don't have to
build any backend at all, just interface with this:
https://demo.ecommerce.endpointdev.com/swagger/index.html

The results (version 1)

It generated a nice, simple ecommerce storefront, using Hugo and End Point Ecommerce. Impressive! However, there’s one red flag: if you look at the products in the demo API we’re using, they are simple groceries — Apple, Banana, etc. So where did “Pumpkin spice treats,” “Spider Web Candy,” and “Ghost Marshmallows” come from?

It turns out, these are “fallback products” Replit had made when it failed to connect to the End Point Ecommerce demo API. Not great for an ecommerce site! Your users would see bogus but real-looking products when they’re unable to see or buy real products (I didn’t test this far, but the site may have even accepted payment info for fake products).

So I followed up:

Please remove the fallback products

The results (version 2)

This looks more promising! We get an error message written by the frontend. In production we might want to display more specific messages based on the response, but for this demo, I’m happy with this way of displaying errors.

In true “vibe-coding” fashion, I asked the Replit agent to diagnose the network error:

There is a network error while loading products, what is the issue?

It responded with a couple options, the main one being assume it’s a CORS issue.

Possible Solutions
Option 1: Server-Side Proxy (Recommended)
- Create a simple server endpoint that fetches the products
- Your frontend calls your server, your server calls the external API
- No CORS issues since server-to-server requests aren't restricted

I was suspicious of it being a CORS issue since everything was hosted in the Replit container, so after poking around myself, I wrote this prompt:

Please use the web api to sign up for an account by POSTing to the /api/User
end point and getting an API key

However, after reviewing the project for this post, I found that I misunderstood the issue in this prompt — no authentication is needed to GET /api/products on the demo API of End Point Ecommerce, the site was just disallowing this cross-origin request. Replit’s agent went ahead and implemented the CORS change anyway, though it didn’t correct my prompt.

After a few more followup prompts…

Please use the "basePrice", "discountAmount", and "discountedPrice" keys that
are returned from the API to show the real prices of the items

Now, please fetch the images from the API as well

…we have a decent working Hugo site connected with the End Point Ecommerce demo API!

Feature addition: cart

I wrote this prompt next, without having done my research on the backend (which made for an interesting unintentional experiment):

Please add a lightweight cart management system which uses localStorage, with
options to add to, remove from, or update quantity of items in the cart. Add a
simple checkout form with contact information, shipping address, and billing
address. For now, instead of payment information, just have a button which adds
a commonly used dummy credit card number.

There’s an issue here: I specified using localStorage, which is a fine way to set up a frontend cart system, but it ignores the existing system End Point Ecommerce already supplies. I hadn’t checked the documentation before making this prompt, and neither did Replit; it happily obliged and created a cart management system on the frontend. It’s nice that the agent did what I asked, but if I were trying to work with the agent, without cross-referencing documentation myself, I would be disappointed.

That’s the interesting part about using a non-deterministic system like AI agents: responding to one prompt, it might ignore my specific (and misguided) requests, but in another it might blindly execute the tasks it’s given. By the nature of LLMs, you can’t predict what you’ll get.

Bonus: a few extra notes

A couple final notes:

No .gitignore was generated, so the Hugo-generated public/ folder is tracked in Git
The agent used plain CSS, without SCSS or a framework — I’d recommend specifying the stack further in the prompt if you want an app that’ll be nice to maintain. You don’t want to assume the agent will make smart coding decisions on its own.

When I’d use Replit (and similar do-it-all AI agents)

I would use Replit for very quick-and-dirty web development, when sites which shouldn’t last a long time — although I’ve seen too many projects originally intended to be short-lived become important for many years…

Replit handles its containerized deployments well; if I needed to share a working demo site within a few hours, Replit would be a good pick. Especially if you like React frontends, even when you might not have asked for one! But if I was dealing with customer data, orders, or my own data I cared about, I would want to be heavily involved throughout the development process.

For projects where performance and accuracy matter (read: most projects), I will be taking a more directed approach. I’ve had some good results using Continue or GitHub CoPilot as a coding assistant for small tasks in existing repos, rather than letting AI taking the wheel completely.

A Preact Web App Without npm Build

2025-10-15T00:00:00+00:00

Modern JavaScript frameworks such as React and Preact usually require a full “build pipeline” with Node.js, npm, and bundlers like Webpack or Vite. That’s fine for large web apps, but sometimes you just need a small, self-contained browser interface inside a project that isn’t mainly about the web — say, a Java program that processes transit data and simply needs a page to display results.

This post explores how to use Preact and its companion library @preact/signals without any npm build process at all. By taking advantage of browser-native import maps and writing a bit of code directly in JavaScript instead of JSX, it shows how to build a lightweight, modern UI that runs straight from static HTML, CSS, and JS files — no build tools required.

The Scenario

This might look like a strange problem to solve but sometimes the web UI part just isn’t a main character in your project.

For instance, I have a GTFS data processor whose main goal is to preprocess public transport stop timetables. It’s written in Java and I want a simple browser UI to check its output. So I want the web UI part of the project to be just static HTML, CSS, and JavaScript files.

I also don’t want to run npm or yarn inside what would otherwise be a pure Maven-managed project. But at the same time, I would like to use a modern JS UI framework such as Preact.

The Main Issues

To pull off this trick, there are two main challenges:

JSX
Imports

JSX

JSX is basically JavaScript templates. When you write something like:

function Greeting() {
  return (
    <div className="greeting">
      Hello world
    </div>
  )
}

function App() {
  return (<Greeting/>)
}

During project build Babel will transform it into:

function Greeting() {
  // Particular function for creating elements may vary.
  return h('div', {className: "greeting"}, 'Hello world')
}

function App() {
 return h(Greeting, null);
}

This is a pretty straightforward transformation, and no one is stopping you from importing the element creation function (h in the example above) and composing what would be Babel output yourself.

Imports

Now for the more interesting issue: imports.

Normally, with a build process, you could just write:

import { h } from 'preact'
import { signal } from '@preact/signals'

What’s more, some packages will try to resolve their own dependencies by importing them by global name. For instance, @preact/signals will try to import @preact/signals-core, and there isn’t a ready-made UMD (Universal Module Definition) version of @preact/signals-core available.

The Solution: Browser Import Maps

I would like to be able to just map library names to URLs of ES modules.

There is a relatively new browser feature that does exactly this: an import map. It’s a special type of <script> block containing a JSON object with your mapping.

<script type="importmap">
  {
    "imports": {
      "preact": "https://unpkg.com/preact@latest/dist/preact.mjs",
      "preact/hooks": "https://unpkg.com/preact@latest/hooks/dist/hooks.mjs",
      "preact/compat": "https://unpkg.com/preact@latest/compat/dist/compat.mjs",
      "@preact/signals-core": "https://unpkg.com/@preact/signals-core@latest/dist/signals-core.mjs",
      "@preact/signals": "https://unpkg.com/@preact/signals@latest/dist/signals.mjs"
    }
  }
</script>

And that’s it!

With the import map in place, your browser can resolve all those imports directly from a CDN like unpkg.com. You can then use Preact, hooks, and signals exactly as if you were running a fully bundled app.

Conclusion

This approach is ideal for small, embedded, or hybrid projects where a full JavaScript toolchain would be overkill. It lets you:

Build modern UI components with Preact
Avoid npm, yarn, Babel, and bundlers entirely
Keep your project clean and language-native (e.g., Maven for Java, without extra layers)
Run everything as plain static assets, right from your browser

Everything works, and nothing extra gets in the way.

Creating Agentic AI Applications

2025-10-14T00:00:00+00:00

Photo by Steve Johnson on Pexels

In the rapidly evolving world of AI, agentic AI is emerging as a game-changer. These systems go beyond simple chatbots or predictive models: they’re designed to act autonomously, make decisions, and interact with the real world to accomplish goals. In this blog post, we’ll dive into what agents are, explore agentic applications and orchestration, and walk through how to build your own agentic applications with open-source examples.

What Are Agents?

AI agents are autonomous entities that can perceive their environment, reason about it, and take actions to achieve specific objectives. Think of an AI agent as more than just a chatbot, it’s like a digital teammate that can look around, make decisions, and take action on its own. Unlike traditional AI models that respond passively to inputs, agents are proactive: they can break down goals into sub-tasks, use tools (like APIs or databases), maintain memory across interactions, and adapt based on feedback. This makes AI agents ideal for applications requiring independence, such as automation, research, or problem-solving.

At their core, agents typically integrate large language models (LLMs) like GPT-4 or Llama for reasoning, combined with mechanisms for tool invocation and state management. They can operate in loops, iteratively refining their approach until the goal is met.

Open-Source Examples of AI Agents

If you want to get hands-on with agents, there are already plenty of open-source projects you can experiment with and build on. Here are a couple of the most notable:

AutoGPT: One of the earliest and best-known autonomous agents. AutoGPT takes a high-level goal, breaks it down into steps, and tries to complete them—whether that means researching a topic, generating code, or managing a workflow. It comes with a simple frontend for building agents, plus a library of ready-made ones (including some quirky ones, like creating viral videos from Reddit trends).
AgentGPT: A browser-based platform that lets you spin up and deploy agents without extra setup. Just type in a goal and watch the agent go—browsing the web, running code, or connecting with external services to move toward the objective.

For a longer list of projects, check out this open-source AI agents directory on Hugging Face.

What Are Agentic Applications and Agent Orchestration?

An agentic application is simply an app that uses one or more AI agents to take on complex tasks by itself. Instead of coding every step by hand, you can plug agents together like modules to tackle real-world problems, whether that’s analyzing data, generating content, or building software. The word “agentic” really just highlights one thing: these systems can act on their own with little human input, following the prompts they are given.

When you bring more than one agent into the mix, you need a way to coordinate them. This is where orchestration comes in—getting multiple agents to work together smoothly. That might mean giving them specific roles (say, a researcher and a writer), setting up how they communicate, keeping track of what they’ve learned, and making sure they stay on task.

To make this manageable, orchestration frameworks provide ready-made structures, like graphs, crews, or conversation flows, that organize the collaboration. With these in place, it becomes much easier to grow from a single helpful agent to an entire team working together.

Open-Source Examples of Agentic Applications and Orchestration

If you want to start building agentic applications yourself, there are already a number of open-source frameworks that can help. Here are some of the most popular:

CrewAI: A lightweight Python framework for orchestrating agents that work together in teams. You can assign roles, define tasks, and let the agents collaborate—without the bulk of larger frameworks like LangChain. It’s simple, fast, and flexible enough for everything from small experiments to enterprise automations.
AutoGen: Created by Microsoft, AutoGen is all about multi-agent conversations. Agents can chat with each other, work independently, or take direction from a human. It also supports tool use like code execution and web browsing, making it handy for more specialized domains like math or chemistry.
LangGraph: Part of the LangChain ecosystem, LangGraph lets you design agents as graphs, which makes them resilient and stateful. It’s well-suited for long-running workflows, offering built-in memory, human-in-the-loop controls, and debugging support through LangSmith—perfect for production-grade projects.

Other frameworks worth exploring include OpenAI’s Agents SDK (built and evolved upon Swarm) for multi-agent workflows, and MetaGPT, which simulates role-based teams of agents.

Writing Agentic Applications

To build an agentic application, the first step is choosing the right framework for your needs, for instance, CrewAI if you want simplicity and lightweight orchestration, or AutoGen if you need extensibility and layered APIs.

Once you’ve selected a framework, you’ll need to:

Define agent roles, e.g., researcher, writer, reviewer. Each agent is an actor to complete one or more tasks.
Equip agents with tools, like search APIs, code interpreters, or custom functions. As you can see, theoretically there is no limit on what you can do with agents.
Assign tasks and orchestration rules, deciding how agents will interact, share state, and collaborate.
Add safeguards, error handling, memory management, and human oversight to ensure reliability and trustworthiness.

With those elements in place, you can start assembling workflows. Below, we’ll look at how to get started with one of the most widely used frameworks (CrewAI).

Building with CrewAI

First, install CrewAI and ddgs (duckduckgo-search) which is a free search tool that I used for this example.

pip install crewai 'crewai[tools]' ddgs

CrewAI uses decorators for agents and tasks. Below is a simple example of a crew for researching and reporting on a given topic. It defines two agents (a researcher and a reporting analyst) and one task for each agent (research task and reporting task). This code creates a sequential workflow:

The researcher gathers data
Then, the analyst compiles a report

You can extend it with more agents with more tasks or parallel processes.

To run this script, you need to set your LLM API key. See how to do that in the crewAI docs.

from typing import Type, Optional
import json

from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from crewai.tools import BaseTool
from pydantic import BaseModel, Field

from ddgs import DDGS

class DDGSearchInput(BaseModel):
    query: str = Field(..., description="Search query")

class DDGSearchTool(BaseTool):
    name: str = "DuckDuckGo Search"
    description: str = "Web search via DuckDuckGo. Returns a JSON list of results."
    args_schema: Type[BaseModel] = DDGSearchInput

    # declare this as pydantic field
    max_results: int = Field(8, description="Maximum number of results to return")

    def _run(self, query: Optional[str] = None, **kwargs) -> str:
        if query is None:
            query = kwargs.get("query")

        if not query:
            return json.dumps([], ensure_ascii=False)

        results = []
        with DDGS() as ddgs:
            for r in ddgs.text(query, max_results=self.max_results):
                results.append({
                    "title": r.get("title"),
                    "url": r.get("href"),
                    "snippet": r.get("body"),
                })
        return json.dumps(results, ensure_ascii=False)

@CrewBase
class ResearchCrew():
    """
    Crew of agents for researching and reporting on given topic.
    """

    def __init__(self, topic):
        self.topic = topic

    @agent
    def researcher(self) -> Agent:
        return Agent(
            role='Senior Data Researcher',
            goal=f'Uncover cutting-edge developments in {self.topic}',
            backstory=f'Seasoned researcher skilled in finding relevant information about {self.topic}.',
            verbose=True,
            tools=[DDGSearchTool(max_results=10)]
        )

    @agent
    def reporting_analyst(self) -> Agent:
        return Agent(
            role='Reporting Analyst',
            goal=f'Create detailed reports from {self.topic} research findings',
            backstory=f'Meticulous analyst who turns {self.topic} data into clear reports.',
            verbose=True
        )

    @task
    def research_task(self) -> Task:
        return Task(
            description=f'Conduct thorough research on {self.topic}.',
            expected_output='A list of 10 bullet points with key findings.',
            agent=self.researcher()
        )

    @task
    def reporting_task(self) -> Task:
        return Task(
            description=f'Expand the research on {self.topic} into a full report.',
            expected_output='A markdown-formatted report with detailed sections.',
            agent=self.reporting_analyst(),
            output_file=f'ai_report-{self.topic}.md'
        )

    @crew
    def crew(self) -> Crew:
        """Assembles the crew."""
        return Crew(
            agents=[self.researcher(), self.reporting_analyst()],
            tasks=[self.research_task(), self.reporting_task()],
            process=Process.sequential,
            verbose=True
        )

## run the crew of agents
if __name__ == '__main__':
    topic = input('Topic: ')
    crew = ResearchCrew(topic).crew()
    result = crew.kickoff(inputs={'topic': topic})
    print(result)

Building A Dynamic Agent

We can follow a dynamic approach to create an agentic AI application as well. In this approach, without other 3rd party agent orchestration frameworks we can dynamically create task and agentic processes. Here I present a simple dynamic agent which can work on any given goal until the goal is met. I used OpenAI as the LLM in this example. Using the dynamic approach, the agent crew extracts sub-goals from the given main goal. Then it works on tasks and decides if the goals are met.

import os
import json
from openai import OpenAI

class DynamicAgent:
    def __init__(self, goal: str, model: str = "gpt-4o"):
        self.goal = goal
        self.model = model
        self.llm = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
        self.memory = []  # store context and past decisions
        self.max_iterations = 5  # prevent infinite loops

    def get_llm_response(self, prompt: str) -> str:
        resp = self.llm.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": "You are an autonomous agent that creates plans and conditions to achieve goals."},
                {"role": "user", "content": prompt}
            ],
            temperature=0.2,
        )
        result = None
        try:
            result = resp.choices[0].message.content
        except Exception:
            # fallback
            result = str(resp)
        print(f"llm result: {result}")
        return result

    def generate_plan(self) -> list:
        prompt = (
            f"Given the goal '{self.goal}', generate a pure JSON list of actionable steps with conditions for success."
            "Don't use any formatting like markdown outside of the JSON content.\n"
            "Each step should include: {'step': 'description', 'condition': 'success criteria'}.\n"
            "Do not rely on predefined rules; infer the steps and conditions from the goal."
        )
        plan = json.loads(self.get_llm_response(prompt))
        self.memory.append({"action": "planning", "output": plan})
        return plan

    def execute_step(self, step: dict) -> str:
        prompt = (
            f"Execute this step: {step['step']}.\n"
            f"Success condition: {step['condition']}.\n"
            "Provide the result and whether the condition was met."
        )
        result = self.get_llm_response(prompt)
        self.memory.append({"step": step['step'], "result": result})
        return result

    def run(self):
        print(f"Starting agent with goal: {self.goal}")
        plan = self.generate_plan()
        results = []
        for i, step in enumerate(plan[:self.max_iterations]):
            print(f"Executing step {i+1}: {step['step']}")
            result = self.execute_step(step)
            results.append(result)
            print(f"Result: {result}")
            # check if goal met
            if "Goal achieved" in result:
                break
        self.save_results_to_markdown(results)

    def save_results_to_markdown(self, results: list):
        filename = f"dynamic_agent_report-{self.goal}.md"
        with open(filename, "w", encoding="utf-8") as f:
            f.write("\n".join(results))
        print(f"Results saved to {filename}")

if __name__ == "__main__":
    goal = input("Goal: ")
    agent = DynamicAgent(goal=goal)
    agent.run()

Use Cases of Agentic AI Applications

Agentic AI applications are already finding their way into practical scenarios. By combining autonomy with orchestration, these systems go beyond demos and research projects to deliver measurable impact.

Customer Support Automation: Instead of static chatbots, agentic systems can act as full-service assistants—resolving customer queries, escalating complex cases, and even initiating refunds or ticket creation. They maintain context across long interactions, providing a more human-like experience.
Research and Analysis: Enterprises and individuals use research agents to autonomously scan news sources, academic databases, or financial reports. For example, a financial analyst could deploy agents to track company earnings, summarize investor calls, and produce actionable insights daily.
Content Creation Pipelines: Marketing teams benefit from multi-agent setups where one agent researches a topic, another drafts a blog post, and another optimizes it for SEO. The result is faster, higher-quality content production with less manual effort.
Software Development Assistants: Multi-agent workflows can handle bug triaging, code generation, testing, and documentation. For instance, one agent detects issues, another suggests fixes, while another runs automated tests and updates docs.
Healthcare and Biomedicine: Specialized agents can search the latest medical literature, cross-reference patient data, and generate preliminary diagnostic reports, supporting clinicians in decision-making while reducing information overload.

At End Point, we use agents to speed up development, using human review to keep our company’s long-standing commitment to high-quality products. We also build agentic AI solutions for clients, speeding up and modernizing legacy workflows.

Challenges Ahead

While agentic AI holds promise, several challenges remain before widespread adoption becomes seamless:

Reliability and Hallucination: LLM-based agents can still generate incorrect or fabricated outputs. Ensuring trustworthiness through validation, feedback loops, and human-in-the-loop oversight is crucial. But, personally I am not against hallucination — We humans hallucinate too.
Security and Safety: Autonomous agents that can browse the web, execute code, or control external systems pose risks if misconfigured or exploited. Sandboxing, permissions, and guardrails are essential to prevent harmful actions.
Evaluation and Benchmarking: Unlike traditional ML models, measuring the success of agentic systems is difficult. Metrics must capture not only accuracy but also task completion, collaboration quality, and user satisfaction.
Ethics and Alignment: As agents gain autonomy, aligning them with human values, legal frameworks, and ethical principles becomes critical. Misaligned objectives could cause unintended consequences.
Integration with Legacy Systems: Many organizations still rely on older infrastructure. Seamlessly embedding agentic AI into these environments can be a non-trivial engineering challenge.

Wrapping up

Agentic AI applications represent the future of intelligent software, blending autonomy with collaboration. By leveraging open-source tools like CrewAI and LangGraph or specialized custom agents, you can create powerful solutions tailored to your needs. As AI evolves, we might expect even more sophisticated orchestration capabilities.

.NET 10 is Coming: What's New, and Why It Matters

2025-09-29T00:00:00+00:00

Microsoft has everything almost ready for the upcoming release of .NET 10 in November this year, and it promises to be a major milestone for developers. Currently in beta, the previews released so far have shown improvements in performance, security, compatibility, and more. Let’s take a look and what’s coming, and why it matters.

Performance and runtime improvements

One of the core aspects of .NET 10 will be the improved efficiency in different areas. To mention a few of them:

Optimized compression streams such as GZipStream and async implementations for ZipArchive methods
Devirtualization improvements for interface methods, especially when arrays are involved, resulting in faster code execution
Faster stack allocation for objects through the expanded use of scape analysis

All of these features (and others) contribute to lower latency and more efficient resource usage. There is a comprehensive blog post by Stephen Toub, .NET team developer, where he runs several benchmarks proving how these improvements result in faster responses and smaller allocations.

Security and increased cryptographic support

.NET 10 also introduces enhancements to encryption and authentication, increasing application security with support for new encryption policies and ways of authentication.

Enhanced certificate thumbprint support adds the ability to search certificates by thumbprint using more secure hash algorithms (not just SHA-1)
WebAuthn support enables passwordless logins with secure authentication methods like biometrics or physical security keys
More cryptographic APIs: Better support for post-quantum cryptography APIs

API & web enhancements

For Web APIs, services, and Blazor apps, .NET 10 has several useful features aiming to increase productivity, security, and cloud capabilities.

Minimal APIs now support built-in validation through Data Annotations
OpenAPI 3.1 support: ASP.NET Core in .NET 10 will generate OpenAPI 3.1 documents with the latest JSON Schema specifications
Several improvements to Blazor assets delivery and environment configuration. One key change is that Blazor scripts will be served as a static web asset

AI, cloud, and DevOps

AI developers will also benefit from the new .NET, as well as applications residing in the cloud and serverless environments.

The ML.NET Framework will keep evolving with expanded capabilities
Cloud Native Development: .NET 10 is designed to be fundamental to cloud native deployments, thanks to smaller base images and the automatic trimming of unused bits
The HybridCache library unifies in-memory and distributed caching, with tagging for smarter invalidation

C# 14

With .NET 10, Microsoft is also rolling out a new version of the C# language, including several improvements that will result in cleaner syntax and better performance. Let’s mention some key features:

User-defined compound assignment operators: We will be able to define our own operators for types like +=, -=, etc.
Using nameof(List<>) without specifying type arguments if we only need the name, avoiding boilerplate in generic programming and logging
Partial constructors and partial events, expanding features like partial methods and properties from the previous language versions

You can find more details about all these upcoming changes on the official .NET website. Based on what we’ve seen above, .NET 10 promises to be one of the most significant releases to the .NET ecosystem, consolidating many incremental improvements that will result in measurable gains in performance, security, and productivity.

I encourage you to explore the .NET 10 Release Candidate 1 to try things yourself and get an early look into what’s about to come out in the .NET world!

Making Blog Search Smarter with LLMs and Open WebUI

2025-09-29T00:00:00+00:00

We recently released LLM Expanded Search for our blog’s vector search. It builds on what we covered in our earlier posts about AI-powered search and vector search basics. Here’s how we built it with our internal AI setup (Open WebUI running an OpenAI-compatible API), why it makes search better, and what’s coming next.

What “LLM Expanded Search” actually does

Here’s the basic idea: when you search for something, we first ask an LLM to come up with related terms and phrases. Then we search for all of those terms, not just your original query.

Your search gets expanded by an open-source LLM through our AI portal (Open WebUI with an OpenAI-compatible API)
Those extra terms give our vector index more ways to find posts that match what you’re looking for
We combine the results, remove duplicates, and sort by relevance before showing the best matches with snippets and links

This really helps with short or vague searches where regular vector search might miss the relevant context — for example, “S3” refers to Amazon S3, which is a cloud object storage system, so whereas “S3” doesn’t provide enough context for a useful vector search. An LLM can expand this short search and include context about cloud object storage in general, as well as give enough context to return results about S3.

How it works

The frontend is pretty straightforward: our search bar has two options, “Search” (just hit Enter) and “LLM Expanded Search” (Shift/Ctrl/Command+Enter).

When you use expanded search, here’s what happens:

We call our Open WebUI endpoint with a prompt that asks for 8–15 related terms
We turn both your original query and the expanded terms into embeddings
We search our vector store with all these terms and combine the results
Caching and rate limiting keep things fast and cheap

Here’s a simple example of how we expand queries:

from openai import OpenAI
import os

client = OpenAI(
    base_url=os.getenv("OPENAI_BASE_URL"),   # e.g., http://openwebui.local/api/v1
    api_key=os.getenv("OPENAI_API_KEY")      # token managed in your environment
)

def expand_query(raw_query: str) -> list[str]:
    messages = [
        {
            "role": "system",
            "content": (
                "You expand a short search query into a concise, comma-separated list of "
                "synonyms and closely related phrases (8–15 items). No explanations."
            )
        },
        {"role": "user", "content": raw_query}
    ]
    res = client.chat.completions.create(
        model=os.getenv("OPENAI_MODEL", "local-llm"),
        messages=messages,
        temperature=0.2,
        max_tokens=200,
    )
    text = res.choices[0].message.content
    return [t.strip() for t in text.split(",") if t.strip()]

After that, we embed the original query and the expanded terms, search the vector index, then sort by score and drop duplicates so each post appears once. Finally, we render concise snippets.

For example, after a similarity search you can rank and de-duplicate like this:

# given: results = [(doc, score), ...]
valid = [(d, float(s)) for d, s in results if float(s) > 0.05]
valid.sort(key=lambda x: x[1], reverse=True)  # highest score first

seen = set()
unique = []
for doc, score in valid:
    src = doc.metadata.get("source", "")
    if src not in seen:
        unique.append((doc, score))
        seen.add(src)

# unique now holds top ranked, de‑duplicated posts

Why we chose Open WebUI

A few reasons made Open WebUI the right choice:

It’s open source and works great self-hosted
The OpenAI-compatible API means we can drop it into existing code
We can use whatever models and inference backends we want
It’s easy to experiment with different prompts and workflows

What’s next: Moving more into Open WebUI

We’re looking into moving more of the search pipeline directly into Open WebUI workflows:

Query expansion (LLM)
Vector retrieval (custom tool that hits our index)

This would give us tighter integration, fewer network calls, and simpler deployment, and make it easier to try new approaches.

What you’ll notice when using it

Short searches work way better, you get more relevant results and fewer dead ends
It’s still experimental, so sometimes results might drift into related topics. Stick with regular “Search” if you want more exact matches
We cache common terms to keep things smooth

Give it a try at our blog. Just use the search bar in our header: press Enter for regular search, or Shift/Ctrl/Command+Enter for LLM Expanded Search.

Want to know more about why we built this? Check out the announcement and vector search posts linked above.

If you’re interested in setting up LLM-expanded vector search or running something similar self-hosted with Open WebUI, we’d love to help out.

How to Download All Videos from a YouTube Channel with yt-dlp

2025-09-11T00:00:00+00:00

Recently, one of our clients temporarily lost access to their YouTube account to a potentially bad actor. As we met with them to regain access, we wanted to back up all their videos as quickly as possible to guarantee they retained access to over a decade of videos. Their channel had over 1000 videos, many of which were over an hour long.

The Google-approved solution here is Google Takeout, but this requires access to the Google account, meaning it wasn’t an option. What’s more, it only downloads in up to 720p quality (or as low as 360p, “depending on the video size”)! This was unacceptable for proper archiving. It also told me I need to improve my personal Google backup process, which currently relies on Takeout 🙃.

I have used yt-dlp for years, so I knew it would be a good option. It’s an open-source command-line tool which is perfect for this situation: you provide the URL of a YouTube video, playlist, or channel and it downloads the corresponding video(s). It’s easy to automate and repeat, and is important for archival purposes, personal backups, and situations like ours. The final command we used was:

yt-dlp <channel URL>

By default, yt-dlp will download the best video and audio options available — the actual best quality available, not capped at 720p — but you can list the formats with -F/--list-formats and manually select formats with -f/--format. We first tried downloading only in mp4 format, but since it’s a less storage-efficient format than webm, YouTube compresses mp4 more heavily, resulting in a lower-quality video. So we let yt-dlp choose the best option, and usually this meant webm files.

You can provide command line options via a config file in ~/.config/yt-dlp/config, which makes it easy to automate or to run multiple times with consistent results. Here’s what we set up for our download:

--output "%(upload_date)s - %(title)s.%(ext)s"
--download-archive archive.txt
--embed-metadata
--embed-thumbnail
--embed-subs
--all-subs

We included a timestamp in the output filenames, making sorting by time easy
--download-archive writes each completed file to the specified archive file (we chose archive.txt in whatever folder yt-dlp is run from) so that if you have to rerun the command it efficiently skips these videos
--embed-metadata embeds metadata in the file, which is good for archiving
--embed-thumbnail and --embed-subs will change the output muxer to something that supports embedding, like mkv. --all-subs makes sure you get all languages and implies the --write-subs flag

You can also save comments using the --write-comments flag. They will be saved to a JSON info file for each video. You can limit the maximum number of comments to fetch with the max_comments argument (passed using the --extractor-args flag).

We were running low on disk space on the server we used, so we periodically copied all of the video files to Google Drive using rclone with the move subcommand.

rclone move --exclude='*.part' --exclude='*.txt' --progress channel-download/ gdrive:channel-drive-folder/

If you need to back up your YouTube videos, especially if you don’t have access to your account anymore, consider using yt-dlp!

Using esbuild and pnpm to Set Up Frontend Asset Bundling in an ASP.NET Core Web App

2025-08-25T00:00:00+00:00

Recently we had to build an admin dashboard on ASP.NET Core. We decided to hit the ground running by looking for a template online. Luckily, we found a very cool one from Start Bootstrap that met our requirements. It was built using Bootstrap, had nice styling, and included examples of graphs, tables, and basic common pages like login and registration.

However, integrating the template into an ASP.NET Core web app was a bit involved. The template uses tools like Pug and Sass/SCSS, which are not supported out of the box by ASP.NET. So some work had to be done to properly integrate it.

We ended up with a nice approach for handling frontend asset bundling using pnpm and esbuild, with support for JavaScript and Sass.

You can find a codebase that uses the setup we’ll be discussing here on Github. It’s an empty ASP.NET Core Razor Pages project that implements Start Bootrap’s admin template. Feel free to review it alongside this article and/or use it for your own projects.

We’ve also uploaded to NuGet a template for an empty ASP.NET Core Razor Pages web app that has implemented the esbuild-based frontend asset bundling.

Installing pnpm and the necessary packages

First of all we need to have the .NET framework installed. We also need to install Node.js and the pnpm package manager.

$ dotnet --version
9.0.302
$ node --version
v22.18.0
$ pnpm --version
10.14.0

Once we have those, we need a Razor Pages project to apply the changes to. For our purposes, I’m going to assume we’re starting off with a fresh project, created using something like dotnet new webapp.

With that out of the way, we can create a package.json file in the root of our project:

$ pnpm init
Wrote to /path/to/code/package.json

{
  "name": "demo",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "packageManager": "pnpm@10.14.0"
}

We can adjust the resulting package.json to better reflect our project. Maybe remove the main and scripts.test settings, which we don’t need. Maybe also set a name and description.

In any case, now that we have this, we can install the Node.js packages that we need. I mentioned that we were going to use esbuild and SCSS so let’s install those as dev dependencies:

$ pnpm add esbuild esbuild-sass-plugin sass@1.78.0 --save-dev
Packages: +41
+++++++++++++++++++++++++++++++++++++++++
Progress: resolved 103, reused 50, downloaded 0, added 41, done

devDependencies:
+ esbuild 0.25.9
+ esbuild-sass-plugin 3.3.1
+ sass 1.78.0 (1.90.0 is available)

╭ Warning ───────────────────────────────────────────────────────────────────────────────────╮
│                                                                                            │
│   Ignored build scripts: esbuild.                                                          │
│   Run "pnpm approve-builds" to pick which dependencies should be allowed to run scripts.   │
│                                                                                            │
╰────────────────────────────────────────────────────────────────────────────────────────────╯

Done in 2.5s using pnpm v10.14.0

I’ve pinned the version of the sass package to 1.78.0 in order to work around some compatibility warnings with Bootstrap’s stylesheets. You may or may not need to do this in your own projects.

We have to address the warning message and allow esbuild to run scripts. Running pnpm approve-builds and selecting esbuild (by pressing either the “Space” or “a” keys) takes care of that:

$ pnpm approve-builds
✔ Choose which packages to build (Press <space> to select, <a> to toggle all, <i> to invert selection) · esbuild
✔ The next packages will now be built: esbuild.
Do you approve? (y/N) · true
node_modules/.pnpm/esbuild@0.25.9/node_modules/esbuild: Running postinstall script, done in 40ms

With that, the package.json file should be looking like this:

// ./package.json

{
  "name": "demo",
  "version": "1.0.0",
  "description": "",
  "scripts": {},
  "keywords": [],
  "author": "",
  "license": "ISC",
  "packageManager": "pnpm@10.14.0",
  "devDependencies": {
    "esbuild": "^0.25.9",
    "esbuild-sass-plugin": "^3.3.1",
    "sass": "1.78.0"
  }
}

There should also be a new pnpm-workspace.yaml file that contains the setting for the approval we gave for esbuild:

# ./pnpm-workspace.yaml

onlyBuiltDependencies:
  - esbuild

In this particular example, we’re adapting Start Bootrap’s admin template, so we’re going to install the packages that it needs:

$ pnpm add @fortawesome/fontawesome-free bootstrap chart.js jquery simple-datatables
Packages: +9
+++++++++
Progress: resolved 112, reused 59, downloaded 0, added 9, done

dependencies:
+ @fortawesome/fontawesome-free 7.0.0
+ bootstrap 5.3.7
+ chart.js 4.5.0
+ jquery 3.7.1
+ simple-datatables 10.0.0

Done in 2.9s using pnpm v10.14.0

These are the ones we need for this project but of course, in your own projects you can install whatever you want.

Authoring JavaScript and SCSS source files

Now that we have the Node.js part of the project set up, we need a place where we can put our JavaScript and SCSS files. For that, we created two new directories: JavaScript and Stylesheets.

Since we’re adapting Start Bootrap’s admin template, we copied all its SCSS files into the Stylesheets directory and put its JavaScript into the JavaScript directory. It all ended up looking like this:

 .
+├── JavaScript
+│   └── site.js
 ├── Pages
+├── Stylesheets
+│   ├── layout
+│   ├── navigation
+│   ├── plugins
+│   ├── variables
+│   ├── _global.scss
+│   ├── site.scss
+│   └── _variables.scss
 ├── wwwroot
 │   ├── css
 │   ├── js
 │   └── favicon.ico
 ├── appsettings.Development.json
 ├── appsettings.json
+├── package.json
+├── pnpm-lock.yaml
+├── pnpm-workspace.yaml
 ├── Program.cs
 ├── RazorStartBootstrapAdmin.csproj
 └── README.md

Of course, the specific file contents will be different in your own projects, but the take home message is that these two new directories are meant for the source files of your frontend assets.

Bundling frontend assets with esbuild

With that, we have the NodeJS package management ready, the tools we need, and a project directory structure that supports writing JavaScript and SCSS.

Now we need to put together an esbuild.config.mjs file that processes these files and produces the bundles. Here it is:

// ./esbuild.config.mjs

import * as esbuild from 'esbuild';
import { sassPlugin } from 'esbuild-sass-plugin';

// SCSS
// This is a list of all the SCSS files that we want to bundle and their
// respective output files. In this project's case, we have only one.
// We put the resulting file under ./wwwroot/css. "wwwroot" is the default
// location used to expose files to the public in ASP.NET Core Razor Pages
// projects.
const scssFiles = [
  { entry: './Stylesheets/site.scss', outfile: './wwwroot/css/site.css' },
];

// Here we tell esbuild to take the entry SCSS files as input and produce CSS
// files ready to be interpreted by a browser. Since the source files are SCSS,
// we use the esbuild-sass-plugin plugin to translate them into CSS.
scssFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    // Some libraries, like Bootstrap, include icons as woff and woff2 font
    // files. To handle this, we specify this loader setting. Instructing
    // esbuild to use the "file" loader, when it encounters such files. The
    // "file" loader takes care of copying the files into the output
    // location.
    // More info: https://esbuild.github.io/content-types/#file
    loader: { '.woff': 'file', '.woff2': 'file' },
    outfile: file.outfile,
    plugins: [sassPlugin()]
  });
});

// JavaScript
// Similar to the stylesheet files, for JavaScript we also declare a list of the
// files that we want to bundle.
const jsFiles = [
  { entry: './JavaScript/site.js', outfile: './wwwroot/js/site.js' },
];

// Bundling here is simpler than SCSS because we don't need custom loaders or
// plugins. Just specify entry points and output files and esbuild knows what to
// do.
jsFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    outfile: file.outfile,
  })
});

One important thing to note here is that with esbuild, we can safely use JavaScript modules and import statements for both JavaScript and SCSS. This means that our frontend logic and styling rules can be broken up into any number of files, forming a tree of dependencies through import statements. We need only to specify the root files, or entry points, and esbuild takes care of building a complete and self-contained bundle. Not really a revolutionary idea in the grand scheme of things, but not always a given when working with JavaScript on the browser.

With this, we can trigger the bundling process: node esbuild.config.mjs. However, it’d be nice to have it better integrated with a typical .NET development flow. For that, we can add the following under scripts in package.json:

// ./package.json

{
  // ...
  "scripts": {
    "build": "node esbuild.config.mjs"
  },
  // ...
}

This allows us to run pnpm run build instead.

Finally, we can have this command run automatically as part of dotnet build if we add the following to our .csproj file:

<!-- ./RazorStartBootstrapAdmin.csproj -->

<Project Sdk="Microsoft.NET.Sdk.Web">

  <!-- ... -->

  <Target Name="BundleFrontendAssets" BeforeTargets="Build">
    <Exec Command="pnpm install" />
    <Exec Command="pnpm run build" />
  </Target>

</Project>

All this will result in our build process producing ./wwwroot/css/site.css, and ./wwwroot/js/site.js bundles. Which we can include in our .cshtml files like we would any other .js or .css file:

<!-- ./Pages/Shared/_Layout.cshtml -->

<!-- ... -->
<link rel="stylesheet" href="~/css/site.css" asp-append-version="true" />
<!-- ... -->
<script src="~/js/site.js" asp-append-version="true"></script>
<!-- ... -->

Importing libraries as modules

One interesting aspect to note is that, with esbuild, we are using JavaScript modules with import statements. This is particularly important for libraries, since some of them won’t necessarily support being included as JavaScript modules by default or in an obvious manner.

Our site.js demonstrates this. For example, it loads Bootstrap and the Font Awesome icons using the following statements:

// ./JavaScript/site.js

import 'bootstrap';
import '@fortawesome/fontawesome-free/js/all';

// ...

This makes it so any page that includes site.js will also have Bootstrap and the Font Awesome icons available.

To get the same for something like JQuery, for example, the strategy is a little different:

// ./JavaScript/site.js

import jQuery from 'jquery';
window.$ = window.jQuery = jQuery;

// ...

A similar scenario happens with the stylesheets. For SCSS, we have to use the @import statement. Our site.scss contains examples of this. Here’s how we load Bootstrap’s CSS, for instance:

// ./Stylesheets/site.scss

// ...
@import "bootstrap/scss/bootstrap.scss";
// ...

In your projects, you’ll encounter libraries with varying levels of support for JavaScript modules, and you will have to adapt accordingly, depending on the library itself and how you want to use it.

Importing libraries via HTML tags

There are also some libraries that only work via traditional <script> tags. For these, you might want to have esbuild copy files directly from the node_modules directory into wwwroot, without any preprocessing. You could use something like this in your esbuild.config.mjs file for that:

// ./esbuild.config.mjs

// Raw files
const rawFiles = [
  { entry: './node_modules/path/to/package.js', outfile: './wwwroot/lib/package.js' },
  { entry: './node_modules/path/to/package.css', outfile: './wwwroot/lib/package.css' },
];

rawFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: false,
    sourcemap: false,
    loader: { ".*": 'copy' },
    outfile: file.outfile
  });
});

Then you can include the files from wwwroot using the appropriate <script> or <link rel="stylesheet"> tags.

Summary

So, in the end, using esbuild and pnpm to set up frontend asset bundling in an ASP.NET Core web app is perfectly doable and can be summarized like this:

Install NodeJS and pnpm
Create a package.json file with pnpm init
Install esbuild as a dev dependency with pnpm add esbuild --save-dev

Also make sure to run pnpm approve-builds to allow esbuild to run scripts
Install any additional preprocessor that you might need like sass with esbuild-sass-plugin, etc.
Create JavaScript and Stylesheets directories and put your source files in them
Add an esbuild.config.mjs file that can bundle your assets
Include the asset bundling step as part of the dotnet build command

Your package.json should end up looking something like this:

{
  "name": "",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    // Register this so that esbuild can be called with "pnpm run build"
    "build": "node esbuild.config.mjs"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    // Add your dependencies here.
  },
  "devDependencies": {
    "esbuild": "0.25.9",
    // Add other preprocessors and plugins here, like:
    "esbuild-sass-plugin": "3.3.1",
    "sass": "1.78.0"
  }
}

Here’s an example of an esbuild.config.mjs that covers the common scenarios:

import * as esbuild from 'esbuild';
import { sassPlugin } from 'esbuild-sass-plugin';

// SCSS
const scssFiles = [
  { entry: './Stylesheets/site.scss', outfile: './wwwroot/css/site.css' },
  // Add other stylesheet files here.
];

scssFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    loader: { '.woff': 'file', '.woff2': 'file' },
    outfile: file.outfile,
    plugins: [sassPlugin()]
  });
});


// JavaScript
const jsFiles = [
  { entry: './JavaScript/site.js', outfile: './wwwroot/js/site.js' },
  // Add other JavaScript files here.
];

jsFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: true,
    sourcemap: true,
    outfile: file.outfile,
  })
});

// Raw files
const rawFiles = [
  { entry: './node_modules/path/to/package.js', outfile: './wwwroot/lib/package.js' },
  { entry: './node_modules/path/to/package.css', outfile: './wwwroot/lib/package.css' },
  // Add other library files here.
];

rawFiles.forEach(async file => {
  await esbuild.build({
    entryPoints: [file.entry],
    bundle: false,
    sourcemap: false,
    loader: { ".*": 'copy' },
    outfile: file.outfile
  });
});

This is the addition that the .csproj file needs in order to bundle the frontend assets during dotnet build:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <!-- ... -->

  <Target Name="BundleFrontendAssets" BeforeTargets="Build">
    <Exec Command="pnpm install" />
    <Exec Command="pnpm run build" />
  </Target>

</Project>

With that, you’re free to add the bundles and raw files in your pages using traditional methods like:

<link rel="stylesheet" href="~/css/site.css" asp-append-version="true" />
<link rel="stylesheet" href="~/lib/package.css" asp-append-version="true" />

and…

<script src="~/js/site.js" asp-append-version="true"></script>
<script src="~/lib/package.js" asp-append-version="true"></script>

As you create more entrypoint JavaScript and SCSS files, you have to remember to add them to their respective arrays in esbuild.config.mjs.

You can add more packages using pnpm add <package_name>. Just remember that they have to be imported by your JavaScript files as modules. If you need to add raw files, especially from libraries that don’t support being imported as JavaScript modules, the esbuild.config.mjs file also supports that thanks to the “raw files” section. These raw files can be loaded normally via <link> and <script> HTML tags.

All right, that’s it for now. We’ve seen how to integrate esbuild with ASP.NET Core in order to setup a basic frontend asset bundling process. We did that through adapting Smart Bootstrap’s free admin template into an ASP.NET Core Razor Pages web app project — a project that’s up on GitHub. We saw how to organize source files in our repo, a basic esbuild config script that handles the most common scenarios, and how to handle a few different cases when it comes to including NodeJS packages into our apps.

Announcing End Point Ecommerce, Our New Open Source Project

2025-08-21T00:00:00+00:00

Today we’re pleased to announce a new open source project: End Point Ecommerce.

End Point Ecommerce is a minimalist ecommerce system that’s quick to set up and easy to understand. It is meant for developers to own, adapt, customize, and extend.

You can learn more about it on our landing page. You can also fork it today on GitHub. There are running demos here:

Are you interested in using End Point Ecommerce for your ecommerce site? Feel free to contact us and we can work together to help you deploy, customize, and maintain it.

Key features

Built with ASP.NET, PostgreSQL, and Authorize.net
Multiple deployment strategies, including Docker Compose
Code base designed to follow Clean Architecture and Domain Driven Design concepts, without being dogmatic about it — simplicity trumps all
Good test coverage with xUnit
Includes a backend admin portal for managing the store, a REST API for building user-facing store frontends, and a simple web frontend
Limited functionality — implements the bare minimum of ecommerce use cases
Product catalog, shopping cart, order submission, email delivery

How we got here

We recently had the opportunity to help one of our clients migrate their ecommerce site. They were using an established ecommerce engine and were having various problems with it. Their requirements were simple, and they didn’t need all the bells and whistles of big ecommerce solutions. They didn’t want to deal with the overhead, bloat, and difficulty of finding maintainers.

During initial investigation and planning, we didn’t find any options that were small and easy to set up, understand, and customize, so we decided to build them a new site from scratch.

We realize that there may be others out there in the same boat that we were. You want to set up a custom ecommerce site but you don’t want to deal with the complexity of bigger off-the-shelf solutions. You want something which you can thoroughly understand quickly, and modify to your heart’s content, as if it were your own code base that you developed from scratch.

Who is this for?

This project is meant for .NET developers who want a solid and simple foundation to build and customize ecommerce sites. To that end, the feature set is very basic, the code base is very straightforward, and the documentation is very developer-oriented. The idea being that developers can quickly get up to speed with how everything works, develop ownership over the code base, and implement their bespoke use cases.

This project is not meant for non-technical folks. There is no installation wizard, no templating engine, no plugin framework, no no-code customization capabilities.

If you want to set up a simple online store, or develop a highly customized one from scratch, think of this as a way of skipping the first few steps of building the foundational architecture and basic functionality.

Please check it out! And do whatever you want with it!

Risks of Paper Checks for Secure Transactions

2025-08-18T00:00:00+00:00

Fraud is on the rise, and paper checks are one of the easiest targets. In the not-so-distant past, a grandparent could tuck a crisp $5 bill into an envelope and trust it would arrive. Today, even checks sent to pay your bills are being stolen, altered, and cashed by criminals before they reach their destination.

So, the practice of sending a paper check in the mail to reach a creditor should be stopped. According to the US Financial Crimes Enforcement Network (FinCEN), there were over 660,000 check fraud reports in 2024. This number has stayed in the 600,000s every year since 2022, when it nearly doubled from the previous year.¹ The U.S. Postal Service has seen similar increases in mail theft, with criminals targeting collection boxes and even robbing postal carriers for paper checks.

Fraudsters have many ways to manipulate checks. Intercepting checks that are not in a secured location by simply paying attention to where mail is lying around is at the top of the list. Once they get a check in hand, they can “wash” original amounts and payees and replace them with higher numbers and their own names.

A recent incident at our company showed how risky paper checks can be. A check mailed to End Point by one of our long time customers was intercepted and fraudulently cashed by someone else. We contacted the customer multiple times to collect on the open invoice. They were sure they had paid the invoice. When they sent us a photo of the front and back of the cancelled check, we could see that an unknown person had endorsed the check and cashed it! In some cases, the sender is left holding the financial loss with no way to recover the funds.

At End Point, we don’t cash customer checks in person — all payments are deposited directly into our business accounts. While this adds some protection, it’s useless if the check never reaches us.

The safest way to pay today is electronically. We strongly encourage customers to switch to secure electronic methods such as ACH transfers or direct deposits. If a check is unavoidable, send it via Certified Mail with tracking. Protecting your payment protects both your business and ours.

Data aggregated from https://www.fincen.gov/reports/sar-stats ↩︎

End Point Dev blog

Introducing EP Audit

The Problem

What EP Audit Looks Like

A Service, Not Just a Tool

Phased Remediation

Working with End Point

Designing software architecture with Domain-Driven Design

Table of contents

Section 8: Architectural patterns

Layered architecture

Ports and adapters

Command query responsibility segregation

Scope

Section 9: Communication patterns

Model translation

Stateless model translation

Stateful model translation

Integrating aggregates

Outbox

Saga

Process manager

Architectural patterns to use within bounded contexts

Patterns for communicating across bounded contexts

Implementing business logic with Domain-Driven Design

Table of contents

Section 5: Implementing simple business logic

Transaction script

Active record

When to use them

Section 6: Tackling complex business logic

Domain model

Value object

Entities

Aggregates

Domain event

Domain service

Data access concerns

Section 7: Modeling the dimension of time

Data storage and retrieval

Advantages and disadvantages

The event sourced domain model

DDD tools for implementing business logic

Observing End Point Dev's Approach to AI

Time Sheet Analysis and AI Analysis

Traditional Software Engineering and Spec-Driven Development

Spec-Driven Development with AI

Final Thoughts

Getting the Most from your Claude Subscription

What are Tokens?

What Loads with every Prompt

The CLAUDE.md Hierarchy

Memory

The Three Models and How They Use Tokens

How Your Prompt Selects the Model

Effort Levels

Monitoring your Token Usage

Setting up the Statusline

Other Monitoring Commands

Compaction: Your Most Important Habit

Why Compact at 60%

Using /compact Effectively

Time of Day and Rate Limits

Practical Habits That Save Tokens

Keep Tasks Small and Focused

Group Work by Skill

Use /clear Between Unrelated Tasks

Use Plan Mode for Complex Work

Delegate Verbose Work to Subagents

Understanding Your Subscription Quota

The 5-Hour Rolling Window

The Weekly Quota

Extra Usage: Pay-as-You-Go Overflow

Putting It All Together

High Level System Analysis and Design with Domain-Driven Design

Table of contents

Section 1. Analyzing business domains

Domains and subdomains

Types of subdomains

Using subdomains to make strategic decisions

Why `.msg` bites harder than `.eml`

Mounting on login with `fstab`

`Containerfile`

`container-compose.yml`

Minimal `unicorn.rb` configuration file

Simple `config.ru`

Using ActiveSupport::ExecutionContext to improve Rails logging