High Level System Analysis and Design with Domain-Driven Design

2026-04-06T00:00:00+00:00

Photo by Zed Jensen, 2022.

Domain-Driven Design is an approach to software development that focuses on, as Eric Evans puts it, “tackling the complexity in the heart of software”. And what is in the heart of software? The business domain in which it operates. Or more specifically: a model of it, made of code. That is, the code that implements the business logic that comes into play when solving problems within the realm of a particular business activity.

DDD is not just about writing code though. It’s a whole methodology that touches on business needs, requirements gathering, organizational dynamics, high level architectural design, and lower level patterns for implementing software intensive systems.

As a result, DDD offers a treasure trove of concepts, patterns and tools that can be applied to any software project, regardless of the size and complexity.

In this series of blog posts we’re going to explore many aspects of DDD. We will be following the structure laid out by Vlad Khononov’s excellent book on the topic “Learning Domain-Driven Design: Aligning Software Architecture and Business Strategy”. So you can think of this series as a summary of that book. An abridged version that can serve as a review for anybody who has read it; but also as an entry point for people who are new to DDD.

High level system analysis and design with Domain-Driven Design

Section 1. Analyzing business domains

It is a well understood notion that before writing code, we need to understand the problem we’re trying to solve. DDD is consistent with this notion and argues that developers need to, first and foremost, gain an understanding of the business that the software is being built for. To this end, DDD relies on three concepts: domains, subdomains and domain experts.

Domains and subdomains

In simple terms, the domain of a business is what it does, its area of activity. For example, Starbucks is in the business of making coffee, Ford is in the business of making automobiles, AMC is in the business of movie theaters.

Of course, analyzing a business as a single integrated whole can be unmanageable. That’s where subdomains come in. Subdomains are the different divisions within a domain. Starbuck’s domain for example, is making coffee. But there are many smaller parts that make up that business and allow it to be successful. There’s of course, the subdomain of coffee preparation. But there’s also real estate management to find and secure good locations, there’s inventory management and logistics, there’s marketing, there’s human resources, etc. All these are the subdomains that make up the overall business of Starbucks.

Depending on the business and on the project, these will vary greatly in granularity. And you can also decompose subdomains further and discover new fine-grained subdomains nested within more coarse-grained ones. The sizes and nesting levels can be very different from business to business, so it can be difficult to clearly delineate where a subdomain ends and another one starts, which activities belong to one or the other. One good rule of thumb to keep in mind is that generally, subdomains encapsulate a set of coherent, closely related use cases. That is, use cases that involve the same set of closely related actors, business entities and/or data.

And finally, we have domain experts. As the name suggests, these are the people within the organization who have intimate knowledge of the business, or certain areas of it. They are the subject matter experts. Usually they are the ones who identify the problems and come up with requirements. Developers need to rely on domain experts to gain the necessary understanding to be able to produce useful software solutions.

So, when approaching software projects, DDD suggests that developers work closely with domain experts in order to learn from them about the business domain and subdomains. After all, it is their mental models and understanding that will be modeled and implemented in code.

Types of subdomains

With the help of domain experts, developers can identify subdomains, understand their business value and how they fit within the overall business strategy. This is very important because it helps making some initial architecturally significant decisions. Namely, the general approach to solving the problems in the subdomains, how much to invest, how to organize development teams, etc. The main objective in this analysis stage is to identify the subdomains and whether they fall into one of three types: core, generic, and supporting.

Core subdomains are the activities of the business with highest value. The ones that confer differentiation in the market and a competitive advantage. They are the business’ raison d’être. For example, for Google, their search engine is a core subdomain. For Ford, their automotive engineering area would be a core subdomain. Core subdomains are generally the most complex parts of the business. They are constantly evolving and improving, and the company is compelled to invest heavily in them.

Generic subdomains are also very complex. However, they are not business differentiators. Instead, these are the areas of the business that all organizations handle in the same way. Think accounting, a ticketing system, an online storefront. There’s no pressure to innovate in these areas, so the solutions are very stable and evolution is slow.

Supporting subdomains on the other hand, do not provide any competitive advantage, nor are they very complex. They are however, necessary because they support the core business activities, and are fairly unique. The solutions to problems in these areas usually take the form of CRUD or ETL oriented activities. Imagine for example populating a data warehouse, translating transactional business data into a format appropriate for analytics and business intelligence in a manufacturing corporation. Or maybe the digitization and storage of a registry of court documents for a law firm. These are behind the scenes activities that support the organizations’ main businesses, but their business logic complexity is not high, and they don’t really represent big selling points for the company.

It’s worth noting as well, that there may be subdomains where software solutions are not appropriate, even if they are highly complex core subdomains. They are still part of the business so it’s worth identifying and considering for high level architectural design decisions. If anything, to know what parts of the business the planned software system should and should not focus on. You could have a restaurant, for example, who prides itself in having the best desserts in the city. For their business, the recipe development activities constitute a core subdomain. This is dependent on the art and craftsmanship of the chefs. Not an area in which software solutions could help a whole lot.

It’s also worth noting that, just as organizations’ business strategies are dynamic, so too can be their subdomain distribution. Today’s generic subdomain can be tomorrow’s core subdomain, and so on. For example, imagine a big retail store chain that, up until now, managed its inventory in an industry standard way. But it has grown so much that the standard way of doing things has become a bottleneck for them. So they design a new procedure for highly efficient inventory management, and that gives them an edge against competitors. Inventory management started as a generic subdomain for them, but due to an ever evolving business strategy, it became a core subdomain.

The knowledge of subdomain types can also help the analysis when dividing the business and discovering subdomains. If you find yourself recursively finding more and more fine-grained subdomains from already identified ones, a good time to stop recursing is when you find that a generic or supporting subdomain, when broken down, reveals only new finer-grained generic or supporting subdomains. The reasoning for this is simple: this analysis activity is most valuable when trying to identify core subdomains. When there are no more core subdomains to be found within a particular area of the business, then it’s probably not worth it to keep digging deeper.

Here we have an example of a hypothetical help desk system subdomain, part of a customer service subdomain, being decomposed and revealing that it only contains generic subdomains.

In summary, these are the main characteristics of the three types of subdomains:

Core subdomains have high complexity, provide competitive advantage, and change and evolve frequently.
Generic subdomains have high complexity, do not provide competitive advantage and change overtime, although at a slower pace than core subdomains.
Support subdomains have low complexity, do not provide competitive advantage and are the slowest to change.

This is where each type of subdomain fall when considering their business logic complexity and business differentiation.

Using subdomains to make strategic decisions

Like I’ve already alluded to, armed with this knowledge, DDD practitioners are ready to start making some higher level architectural decisions. Depending on the type of subdomain that a problem belongs to, DDD has specific prescriptions on how to handle the implementation of software solutions for them.

When working on core subdomains, that’s where we want to make the biggest investments. We deploy the most advanced engineering tools, patterns and practices. This is to make sure that the software is efficient and high quality, and also easy to maintain and evolve. This is necessary because core subdomains have to evolve rapidly by nature, if the business is to maintain competitive advantage. Software solutions that operate within the context of core subdomains have to be implemented by high skill and high trust teams. Either in-house, or via trusted development partners, working hand in hand with domain experts.

Problems in generic subdomains, by nature of their business logic being very complex but also very common, are likely to have already been solved. For these types of problems, DDD recommends against implementing custom software, and instead buying and/or adopting tried and true, industry standard, off-the-shelf solutions. Their implementation and integration can be outsourced or handled by less specialized or skilled teams. The change management of these solutions is simple, as they get delivered generally via patches and updates.

For support subdomains, whose business logic is generally simple but uncommon, it is less likely that off-the-shelf solutions would be available. So software addressing problems in these subdomains will most likely have to be implemented as custom solutions. Due to their low complexity though, they can be easily outsourced, or handled by more junior team members. They can also be handled with RAD, low-to-no-code technologies, since often times they are little more than ETL and pure CRUD applications.

Here’s a table to summarizes the differences between the types of subdomains:

Type of subdomain	Core	Generic	Supporting
Competitive advantage	Yes	No	No
Complexity	High	High	Low
Rate of change	High	Medium	Low
Implementation	Custom development	Buy/adopt off-the-shelf	Custom development
Team composition	In-house/partners	Can outsource	Can outsource
Skill level	High	High/regular	Low
Investment	High	Medium	Low
Problems	Interesting	Solved	Simple

Section 2. Discovering domain knowledge

After identifying the business domain and categorizing the various subdomains that compose it, we have a bird’s-eye view of the business. This is good enough to get started and make high level architectural decisions of potential software solutions to problems within these domains. To actually build the software though, we need much more than that. In order to gain a thorough understanding of the business logic, and be able to eventually model and implement it in code, DDD proposes the ubiquitous language as a tool.

The ubiquitous language

The ubiquitous language is DDD’s tool for knowledge sharing, effective communication and software modeling. In simple terms, the ubiquitous language is the language of the business. It’s the language that domain experts use on a day to day basis to talk and reason about the business.

For a software project to be successful, it’s essential that engineers and stakeholders understand each other. They have to be aligned when it comes to the meanings of the core concepts of the business domain. That’s why it’s so important that everybody uses the ubiquitous language for all project related communications, requirements, documentation, face to face discussions, and even code itself. It all needs to share the same language.

Engineers will need to interact with domain experts in order to learn about the business domain and its rules. They need to acquire the necessary knowledge that allows them to implement these rules in the software. During these interactions, they converse using the ubiquitous language. However, the knowledge transfer is not always strictly unidirectional. Yes, engineers have to learn from the domain experts. But also, through conversations and questioning, engineers can help domain experts deepen and flesh out their own understanding about their domains.

A classic example of this is when domain experts focus too much on the “happy paths” of given business processes. Then, through discussions with developers, on account of the inherent precision that software demands, they are forced to consider more edge cases, better specify ambiguous terms and fill out gaps in their understanding.

This deepening of knowledge feeds back into the ubiquitous language and improves it. Making it more insightful and more precise. This means that, throughout the life cycle of a project, the ubiquitous language should keep evolving, expanding and refining. This allows it to continuously improve as an effective model of the business domain, for the problem that it’s trying to solve. Indeed, just like code is a model of the business domain, so too is the ubiquitous language.

A classic pitfall in software projects is a manner of communication and knowledge sharing where developers are various steps removed from the domain experts. Consistent use of the ubiquitous language addresses this.

The ubiquitous language as a model of the domain

Through consistent use of the ubiquitous language, developers are able to obtain a deeper understanding of the domain, the problems we’re trying to solve, the reasoning behind the requirements and the mental model of the domain experts. This allows the construction of effective software solutions, with deep business insight, that go beyond simply translating requirements into code. If the implementation models the business domain effectively, there is potential for it to evolve with the business and better adapt as requirements change; and we reduce the risk of missing edge cases that may not be obvious to domain experts.

Under DDD, developers and domain experts develop the ubiquitous language, which then informs the implementation.

Effectively, when we build a ubiquitous language, we’re building a model of the domain. A model that reflects the relevant business entities, their behavior, and relationships. A model that will be eventually implemented into code, but also a model that needs to be understood by all stakeholders, regardless of their technical level. So it needs to be precise and rigorous, but also understandable. As such, in order to be useful, the ubiquitous language needs to respect certain restrictions:

It must not include technical jargon like programming languages, constructs or frameworks; nor mention specific system structures like database tables, servers or programs. It is the language of the business and it needs to be understandable by non technical folks.
It must avoid using the same term for different concepts. In conversation with humans, a lot of the meaning is extracted from the context. In software, not so much. Just like you can’t have two classes with the same name within the same namespace, the ubiquitous language cannot allow such ambiguities.
It should also avoid using different words for the same concept. The classic example for this is words like users, accounts, customers, visitors. They all refer to closely related concepts which might actually have differences. The problem with these overlapping terms is that those differences are obscured by the notion of them “being the same” and being used interchangeably. For usage within the ubiquitous language, it’s best to be precise and clearly delineate terms and definitions. Eliminate what’s redundant, be strict with definitions.
It should avoid including extraneous details. Just like code needs to include only what’s needed to solve the problem at hand and nothing else (in order to avoid accidental complexity); the ubiquitous language must not be polluted with details from outside of the area of business activity that it represents. That can create confusion and unnecessary cognitive load.

The ubiquitous language represents a unified model of the domain. This is in contrast to a more traditional process where domain knowledge gets “translated” multiple times before turning into code.

Tools for capturing the ubiquitous language

The Agile Manifesto declares: “Individuals and interactions over processes and tools”. And of course, when it comes to the ubiquitous language, direct interaction and conversations between engineers, domain experts and other stakeholders reigns supreme. That’s when it can be most useful. It can also be useful, however, to capture the ubiquitous language in documentation and even in runnable form.

A glossary of terms is a great asset for keeping a ubiquitous language. A wiki is a good place to put this. Definitions for key concepts in the business like entities, processes and rules can be captured here. The only caveat is that documentation is static by nature, while the ubiquitous language is continually evolving. So, great care needs to be taken to keep the wiki updated at all times to reflect the latest and most complete understanding of the domain. This should not be relegated to or gate-keeped by only certain people; it should be a team effort where everyone contributes.

An automated acceptance tests suite, written using Behavior Driven Development frameworks, like Cucumber, is also a great way of capturing the ubiquitous language. The advantage of these tests is that they are written in plain human-readable language, not code. And while it may be far fetched to think that non technical domain experts would be capable of writing and maintaining such tests, they certainly can read and understand them, which is a great boon. These tests speak the language they understand: the language of the business.

This is what Cucumber tests look like.

By nature of being executable and tied closely to the implementation code, there is less chance that they become out of date. This can happen more easily with static documentation written in a wiki. The disadvantage is that they require much more effort. But for the right kind of project, one where business logic is very complex or the scope is very wide, they can be very well worth the cost.

Section 3. Managing domain complexity

We can attempt to model an entire business domain with one big ubiquitous language, but sometimes that’s impossible. Especially so for businesses of a certain size, you will inevitably find inconsistencies and conflicts between the mental models of different domain experts. The simplest example of this scenario is when different experts from different areas of the organization have the same word to describe different concepts. Or when they look at the same business entity with different levels of detail.

When the different mental models are valid, and these inconsistencies are legitimate and cannot be reconciled, the solution is to follow the divide and conquer principle and decompose the language into separate ones, each one working within its own bounded context.

Simply put, a bounded context is the context within which a ubiquitous language, and the model it represents, operate, have meaning and are useful. Sure enough, just like a domain can be divided into subdomains, a ubiquitous language can be decomposed into smaller languages, each with its own context, to model different parts of the business. While we can attempt to capture an entire business domain with a single model, this is not advisable in the case of complex systems. It’s better to split up the model into smaller, more precise ones, tailored to work on specific problem domains.

Reasons for creating bounded contexts

Like mentioned before, different domain experts having conflicting mental models is a clear indication that a division in the model needs to happen. However, that only tells us half of the story. There are other indicators that point to when and where further division should happen:

First of all, there is of course, size. Size by itself is not a deciding factor, but it is something to consider and balance. Fewer, bigger models can help keep the overall environment simpler, but if they become too big they can become unmanageable and prone to corruption. A higher number of smaller models keep cognitive load low, but you run the risk of exploding integration and management complexity. The better principles to keep in mind here are coupling and cohesion. Beware of separating closely tied use cases, that deal with similar actors, entities and data.

System-level nonfunctional requirements also play a role in dividing a model into separate bounded contexts. For example, you might need to decouple the life cycle of several software components. Have them be developed, evolved, versioned and deployed independently. You might also need them to scale separately. You might even need to use completely different technology stacks, whatever is appropriate for the task at hand. This means that bounded contexts generally get implemented as individual services and/or applications. That is, as individual runtime components.

The organization’s composition also plays a role when designing bounded contexts. The general rule is that a given ubiquitous language, the model it represents and the bounded context is lives in, must be owned by a single team. In software, high ownership, cohesion and consistency are desirable traits. Having a single team own and maintain a particular component foments these. Single ownership also helps reduce communication overhead, and prevents people from stepping on one another’s toes. Of course, a single team can own multiple bounded contexts, what cannot happen is one bounded context being owned by multiple teams.

Indeed, in bounded contexts, we have the tools we need to make strategic decisions related to the decomposition of software systems into architecturally significant components or modules. With bounded contexts, we are able to specify the physical and ownership boundaries of these components.

Bounded contexts vs subdomains

Bounded contexts and subdomains are closely related concepts, but there’s one key distinction: subdomains are discovered, while bounded contexts are designed. Subdomains are useful because they help us understand the business strategy. Splitting the business domain into smaller problem domains is useful because it can break down a complex whole into smaller and more manageable parts.

Depending on the situation, it is certainly possible to end up with a set of bounded contexts that align one-to-one with the business subdomains. However, this is not mandatory. We can develop a solution with a single bounded context that spans multiple subdomains; the same way that we can decompose the problem into many bounded contexts, some of which operate within the same particular subdomain.

Bounded contexts are closely related to subdomains, but aren’t tied to them. They are flexible and can be organized in many ways.

Each bounded context becomes a separate major architectural component. That is, a (micro) service, a project, an application. When we have a component that spans multiple subdomains, programming language organizational structures like namespaces or modules can be used to logically separate the subdomains within.

Section 4. Integrating bounded contexts

For a system to function, its components need to interact with each other. So, once we have decomposed the problem domain into separate bounded contexts, we need to decide their relationship and integration strategies. This need for interaction between them implies that there are touch points between bounded contexts. We call them contracts.

These contracts are necessary because each bounded context contains its own version of the world. That is, its own model and ubiquitous language. In order to integrate, some level of translation needs to happen. They need to be adapted to one another. The contracts define these adaptations.

Domain-Driven Design offers various patterns that are useful for defining contracts between bounded contexts. The decision to use one pattern vs the other depends greatly on the nature of the teams tackling the project. Depending on the teams’ relationship, we can put the patterns in one of three categories: cooperation, customer-supplier, and separate ways.

Integration patterns are determined mainly by the level of cooperation between the teams that own the interacting bounded contexts.

Cooperation patterns

When the components that need to interact are owned by teams which are in close communication, work well together, and whose goals are aligned, cooperation patterns can be applied.

If the teams meet these criteria, a partnership model can be implemented. This is where the integration is managed in an ad-hoc manner. Both teams work together to define the API through which their components interact and that’s that. Whenever changes happen on either side, the other team learns about it quickly and adjusts their code right away. Continuous integration is a great tool here, as breakages are immediately apparent, closing down the feedback loop.

Sometimes, when different bounded contexts need to implement the same functionality, it makes more sense to develop this functionality once and package it as a reusable library, or a sub-module within a shared repository. This is what DDD calls a shared kernel. A shared kernel is effectively a bounded context of its own, one that is statically linked to other components and implements the logic that others depend on.

A shared kernel is its own bounded context, but also “belongs” to multiple other bounded contexts.

The shared kernel integration pattern needs to be used carefully though, as it creates tight coupling between the components that use it. If the involved bounded contexts are owned by different teams, then this shared kernel that emerges violates the DDD principle of bounded contexts having single-team ownership. This is something to watch out for, as bad team synergy can produce problems during development and maintenance.

When using a shared kernel, the quality of the communication and the coordination between the teams has to strike the right balance: when they aren’t strong enough to support a partnership model, but not so weak that the shared kernel would become more trouble than it’s worth.

The alternative to a shared kernel is the duplication of the logic across multiple components. When deciding whether to apply this pattern, the costs of duplication vs coordination need to be considered. That is, the cost of implementing the same logic within multiple components and its subsequent changes, vs the cost of changes in the shared kernel propagating into dependent components and coordinating with the owners of each. When it is cheaper to just duplicate the logic, shared kernel should not be applied.

This means that complicated models that change frequently, like those of core subdomains, are good candidates for shared kernel. Shared kernels allow complexity to be encapsulated and exposed through stable contracts. This means that frequent changes are easier to contain. Frequent changes in logic that’s highly complex and also repeated in multiple places, quickly becomes much more expensive to maintain.

Another good scenario for applying a shared kernel is when refactoring a legacy monolithic system into more separated modules. The legacy system can become the shared kernel during the time that modules are being extracted from it but not yet fully decoupled.

Customer-supplier patterns

Customer-supplier patterns establish a relationship between components where one (the provider, who is “upstream”), provides a service to another (the consumer, who is “downstream”). These types of patterns emerge when the teams who own the involved bounded contexts are not in close collaboration and have their own independent goals.

One form of customer-supplier integration is the conformist pattern. This happens when the supplier defines the integration contract/API, using its own language, concepts and model, and the consumer accepts it. This can happen when the upstream service has a well established or industry-standard model. Or maybe the model is “good enough” for the consumer to interact with directly. Organizational politics may also be a reason for this type of integration, where there is an imbalance of power favoring the supplier’s team, and they get to impose their model or can’t be bothered to adapt it.

Sometimes, the consumer won’t accept the supplier’s model. In that case, an anticorruption layer can be created. An anticorruption layer is implemented by the consumer and translates the supplier’s model into its own. This allows the consumer to use the supplier’s service without polluting its model with extraneous concepts.

Anticorruption layers can be good solutions for the following scenarios:

When the consumer represents a core subdomain. The consumer is solving hard and interesting problems, so it’s best to protect its model with the ACL.
When the supplier’s model is messy and inconvenient. The ACL can protect the consumer’s model from having to contend with a mess of extraneous concepts.
When the supplier’s contract changes frequently. The ACL can protect the consumer from those changes, encapsulating them, allowing it to be less volatile.

A third pattern represents an anticorruption layer of sorts, only built on the supplier’s side. DDD calls this the open-host service pattern. Here, the upstream service comes up with a new language, separate from its own, tailored to its consumers’ convenience, and exposes that as its contract. This public protocol is called a “published language”.

An open-host service provides a protocol for consumers to interact with. It can also support multiple versions of this protocol simultaneously.

An open-host service applies to similar scenarios as an anticorruption layer. Decoupling a service’s internal model from its integration model frees it up for continuous evolution without fear of breaking its consumers. Another advantage of this is that multiple versions of the published language can be exposed, affording clients options on what to support and gradually migrate if they so choose.

Separate ways

Sometimes, the right decision is to not integrate at all. I alluded to this outcome back when discussing the pitfalls of the shared kernel. Sometimes duplication, in spite of how bad it smells, may be the most cost effective solution to a given situation. So the teams go their separate ways.

This can happen when the involved teams cannot collaborate effectively for whatever reason. It could be due to geographical, timezone, or organizational issues.

This can also be a good solution when the repeated logic belongs in a generic subdomain, and it’s easy to integrate. For example consider a logging framework. Exposing that kind of functionality in a service for others to consume, in most cases would be much more trouble than just including some third party library or package.

It may also be that the models being integrated are just so different that they are fundamentally incompatible. It may be cost-prohibitive for collaboration or customer-supplier patterns to be applied; and duplication again, is cheaper.

Going separate ways can be dangerous when we’re talking about core subdomains though. So we have to tread carefully in those scenarios. Remember that models that represent core subdomain should be implemented in the most effective and efficient ways, with few shortcuts and minimized technical debt.

The context map

The context map can be a useful tool for high level design as it plots all the major bounded contexts (i.e. modules, components, subsystems) that we’ve designed and their interaction patterns.

Here’s a context map which captures the various bounded contexts that compose a big system and their interactions. The arrows point to the upstream component in the relationship. “ACL” denotes an anticorruption layer and “OHS” represents an open-host service.

Of course, they can also offer valuable insight into organizational dynamics, as team composition and relationships with others are intrinsic parts of the discussion when talking about bounded contexts. For example, it can show teams that prefer to collaborate closely or at a healthy distance. I can also show problematic components, which are surrounded by anticorruption layers or have had their ties completely cut via a separate ways approach.

As with any document, they run the risk of becoming stale as the system evolves. So it should be a team-wide responsibility to keep it up to date. Each team taking care of their own components and their integration points.

The DDD high level design concept map

These are the main concepts that we’ve explored so far, and how they relate to each other.

Studying “The Mythical Man-Month” in Collaboration with a Client

2024-12-23T00:00:00+00:00

Here at End Point we did an enlightening and fun eleven-week study group in collaboration with one of our clients. We worked through Frederick Brooks Jr.’s book The Mythical Man-Month: Essays on Software Engineering, which was originally published in 1975, with an expanded edition published in 1995. In certain ways this book is still applicable to our time, and it was a delight to discuss it in a structured study group with coworkers and peers. This particular study group even piqued the curiosity of the owners of both our companies who participated throughout the course!

As part of our effort to continually improve our abilities and awareness, and thus build a strong technical company culture, End Point has done many study groups over the years, such as the Ruby Fight Club and the You Don’t Know JS study group. Participation in these groups is voluntary, and they have covered a range of other topics including PostgreSQL, terminal fluency skills, and regular expressions. The opportunity to step away from regular work duties for an hour during the week, and to have meaningful discourse between coworkers has been invaluable for building relationships across teams.

For this group, we collaborated with our client Cybergenetics, who works in forensic DNA analysis. We have been working with Cybergenetics since 2003 on PostgreSQL, websites, system deployment automation, and security. A few Cybergenetics staff members joined our PostgreSQL study group some years ago.

Dr. Mark Perlin, the Chief Scientific and Executive Officer of Cybergenetics, approached us a few months earlier with the desire for his team to better understand the history and controversies of software engineering, and to relate relevant topics to software user or developer experiences. He thought working together through this book would be a good way to achieve both those goals, and we agreed.

Fred Brooks’s book on software engineering and project management has a central theme that adding more people to a software project that is behind schedule usually delays it even longer, a concept known as “Brooks’s Law.” Each chapter covers various reasons why this is the case, such as increased communication complexity and overhead, conceptual integrity, and design cohesion degradation. He posits that properly planning projects, including using realistic estimations and considering communication needs, is essential to avoid delays.

Though not all of us found significant modern application for Brooks’s book, it was an interesting historical read, with some relevant points. One such point is his idea that looking forward to some “silver bullet” solution wastes effort and inhibits forward progress. Today, when some are counting on AI technology to magically fix their code, remove the need to properly document that code, or erase the discipline of software development altogether, it seems wise to apply Brooks’s advice and avoid treating AI and similar advances as a “silver bullet.”

At End Point we love encouraging thought and discussion on relevant topics, and we appreciate having a historical awareness of what we’re doing in the context of decades. We also love an opportunity to collaborate with a client on something that is mutually beneficial.

Key Takeaways from Practical Object-Oriented Design by Sandi Metz

2024-02-27T00:00:00+00:00

Practical Object-Oriented Design: An Agile Primer Using Ruby by Sandi Metz is one of those books that everybody who writes or aspires to write object-oriented code should read. Indeed, reading through this book will be valuable for seasoned practitioners and inexperienced novices alike. Whether it be to discover new concepts, remember why they are important, or articulate the reasons behind that warm fuzzy feeling you get in your stomach when you read well designed code, POODR can help.

I personally really like this book, and here at End Point it does have quite a following as well; it was the subject of a study group that we ran back in 2016. I like to dust it off every now and then to give it a re-read. To help with that, I’ve come up with a list of key takeaways from the book, with some of my own interpretations sprinkled in. For me, it serves as a sort of summary to help commit the book’s contents to memory. I thought I’d share it with you today.

While this book uses the Ruby language for its examples and discussions, the concepts it describes are equally applicable to any classical object-oriented language. Dynamically typed language users do get a little more mileage than users of statically typed ones. But still, something like 90% of the content is relevant to both. For this summary, I’ve taken care not to tie the discussion to Ruby too much and be as language agnostic as possible. I’ve chosen to write the code examples in C#.

Images taken from the book. Code taken from the book’s examples in Ruby and translated to C#.

Chapter 1: Object-Oriented Design

About design

Following the principles and patterns of object-oriented design produces code that is both cost-effective and a joy to work with. “Cost effective” means that it doesn’t take more effort than necessary to evolve, to change.
The reason design is important is because software changes. A good design is one that makes software easy to change.
Object-oriented design sees the world as a collection of parts that interact with each other. The parts we call “objects”, and the interactions we call “messages”.
In order to send messages to each other, objects need to have some knowledge about one another. This knowledge produces “dependencies” between them.
Managing these dependencies is a big part of object-oriented design. OOD aims to create dependencies in such a way that they don’t make the process of changing objects too difficult.
Design is important no matter the size of the application. Because both small and large applications change. And change is what design allows. A badly designed small application will eventually become a badly designed large one.
Design is the way in which code is arranged. Good design is an arrangement of code that makes change easy.
Our job as developers is to take our application’s requirements and our knowledge of design principles and use them to produce code that is cost effective today and tomorrow: code that is easy to change.
The idea is not to predict future changes. Instead, we have to accept that change will come and prepare for it. We prepare for it by writing code that leaves our options open for the future.
“The purpose of design is to allow you to design later, and its primary goal is to reduce the cost of change.”

Failures of design

Design fails when you don’t do it. Code bases that lack in design will eventually evolve into unmaintainable messes, where change becomes increasingly hard even for the most minor new requirements. “Throwing everything away and beginning from scratch” becomes a viable alternative.
Design fails when you overdesign. Armed with basic design skills, it’s easy to fall into the trap of developing wrong abstractions, applying principles incorrectly, seeing the wrong pattern in the wrong context. Applications that suffer from overdesign become gigantic castles of code full of indirection that become hard to change.
Design fails when you separate it from programming. Design is best executed as an iterative and incremental process with a quick feedback loop. The design needs to be adjusted frequently, as understanding of the domain changes. Design that is dictated to programmers from on high as part of a “Big Up Front Design” is doomed to failure. Such arrangements are not agile enough to be resilient to requirement changes.

Agile and design

Agile development says: “Avoid Big Up Front Design because you can’t know what the application will end up being because requirements always change and they do so often.”
Agile also says: “Good design is essential. Because change comes often, your code needs to be ready to accommodate those changes in an efficient and cost effective way.”
Design is not a “one time at the beginning of the project” type of task. It is an ongoing, iterative and incremental task that happens as requirements change, and as our domain knowledge and skills increase.

Metrics

Bad object-oriented design metrics are an indicator of a bad design. But good metrics are not an indicator of good design. A code base with bad metrics will most likely be hard to change.
The ideal software metric would be “cost per feature per time interval”. But this is nigh on impossible to calculate because “cost”, “feature”, and “time interval” are in most cases nebulous concepts.
Still, “cost per feature per time interval” tells us that our goal should always be to write code with a low cost per feature. Making sure that it is also low cost to evolve in the future.

Technical debt

Technical debt is borrowing time from the future. That is, putting out a bad design today in order to release a feature quickly. Tomorrow, when change comes, the bad design will prevent cost effective change. So time will have to be spent refactoring, turning the bad design into a good one. If refactoring doesn’t happen, the debt increases, making future changes more and more expensive.
How much design you do depends on your skill and your time frame. You can’t do so much design that it prevents you from delivering on time. Design is an investment, and for an investment to be good, it needs to return some profit. Good design’s returns are quick and plentiful.
“The trick to getting the most bang for your design buck is to acquire an understanding of the theories of design and to apply them appropriately, at the right time, and in the right amounts.”

Chapter 2: Designing Classes with a Single Responsibility

About classes

Classes are the most basic organizational structure of object-oriented systems written with class-based object-oriented languages. As such, that’s the first thing that we focus on when designing such systems.
Your first instinct should be to keep things simple. Use classes to model only the features that your application needs today and make sure that they are easy to change tomorrow.
“Design is more the art of preserving changeability than it is the act of achieving perfection.”

Write code that is TRUE

In order to be easy to change, code should be TRUE:
Code should be “Transparent”. It should be easy to tell what will be the consequences of changing the code.
Code should be “Reasonable”. The cost of a change should be proportional to the size of the requirement that provoked the change.
Code should be “Usable”. The code should be easy to reuse in situations other than the one it’s currently being used in.
Code should be “Exemplary”. The code should exhibit qualities that guide and/or encourage those who change it to keep and replicate those qualities.
The Single Responsibility Principle is a prerequisite to creating code that is TRUE.

Single Responsibility

A class should have one single responsibility. That means that it needs to do one thing and do it well. It needs to be as small as possible and fulfill a single purpose. It needs to have only one reason to change.
A good way to identify what classes your system needs is to read through your user stories or problem descriptions and identify nouns. Those nouns are good candidates for classes.
If there are verbs and properties associated with those nouns, then it is most likely a class. Because classes represent bundles of both data (attributes) and behavior (methods).
Classes that have many responsibilities are hard to reuse. And because they are not “Usable”, they are not easy to change.
A class that has many responsibilities usually has unrelated code tangled together within it. This makes the class hard to reason about, and hard to reuse.
In order to reuse the behavior defined within a class with multiple tangled responsibilities, you either need to duplicate code or use the entire class, even the parts that you don’t need.
If you use a class that has many reasons to change, when something unrelated to your use case changes, you will still have to change your code. This is unexpected and surprising. This is not “Transparent”.
Depending on classes that have many responsibilities increases the chances of your application breaking.
A good way to determine whether a class has a single responsibility is to try to enunciate what it does in a single sentence. That sentence should be simple, because classes should be simple. Look out for the words “and” and “or” in this sentence, as they point to it having more than one responsibility.
Another good way to determine whether a class has a single responsibility is thinking about every one of its methods as a question that the class would be able to respond to if it were a sentient being. For example: “Array, what is your length?” If the question doesn’t make sense, then maybe the method does not belong in the class.
Cohesion is the measure of how related to their core purpose the elements of a component are. Classes that have a single responsibility are highly cohesive because all of their parts work towards a single goal.

When to make design decisions

Only design as much as you need to for right now. Don’t make design decisions prematurely. Sometimes waiting for more information before committing to a design is the way to go. More information in the future will often reveal a better design.
Consider postponing design decisions when the information available to you today does not point clearly to a good design. Also, when the cost of not doing it now does not considerably make tomorrow’s change more expensive.
On the other hand, consider that a class that’s left in a “bad state” may be reused by others, making their code depend on badly designed code. The application will suffer as a result.
The structure of a class is supposed to reveal your design’s intention to other developers. When it doesn’t, when it is not “Exemplary”, then future maintenance costs increase.
The tension between “improve it now” and “improve it later” is always present. Our job as designers is to strike a good balance “between the needs of the present and the possibilities of the future” so that the costs of change are kept low.

Techniques for writing code that’s easy to change

Depend on behavior, not data: hide instance variables. Instead of accessing instance variables directly, use accessor methods. Even from within the class that owns them. “Always send messages to access data.”
That way you consolidate the knowledge of what the data represents in a single place. That’s the Single Responsibility Principle in action. Changing it becomes easy as a result because instead of changing references to a variable in potentially multiple places; you only need to change the definition of a method, in a single location.
Depend on behavior, not data: hide data structures. When code depends on or receives as input a complex data structure like a big array or hash, encapsulate the data structure in a class with a clear interface, instead of accessing and manipulating the structure directly.
Directly manipulating complex data structures is a style of coding that tends to propagate. If the incoming data structure changes, even slightly, then a lot of your code also needs to change because it depends on its very particular structure.
Handling complex raw data structures through classes and messages with clear, intention-revealing names demystifies the structure and gives it meaning within the context of the application domain.

For example, instead of this…

class ObscuringReferences
{
    int[][] data;

    public ObscuringReferences(int[][] data)
    {
        this.data = data;
    }

    public IEnumerable<int> GetDiameters()
    {
        return data.Select(cell => cell[0] + (cell[1] * 2));
    }
}

…do this:

class RevealingReferences
{
    IEnumerable<Wheel> Wheels { get; set; }

    public RevealingReferences(int[][] data)
    {
        Wheels = Wheelify(data);
    }

    // This method is now more readable.
    public IEnumerable<int> GetDiameters()
    {
        return Wheels.Select(wheel => wheel.Rim + (wheel.Tire * 2));
    }

    // Now everyone can send rim/tire to wheel.
    record Wheel(int Rim, int Tire);

    private IEnumerable<Wheel> Wheelify(int[][] data)
    {
        return data.Select(cell => new Wheel(cell[0], cell[1]));
    }
}

Enforce single responsibility everywhere: extract extra responsibilities from methods. Methods should also have a single responsibility. Just like classes, that makes them easy to reuse, change, and understand.
To determine if a method has a single responsibility, the same techniques that work for classes apply. Try to enunciate their purpose in a single sentence, and ask them what they do.
Methods that iterate on items and act upon them too are a common case of multiple responsibilities. Separating iteration and action into two methods is a common refactoring to correct it.
Complex calculations embedded within methods are also good candidates to be separated into other methods. It gives them a name and makes them reusable.

For example, instead of this:

IEnumerable<int> GetDiameters()
{
    return Wheels.Select(wheel => wheel.Rim + (wheel.Tire * 2));
}

…do this:

// First - iterate over the collection.
IEnumerable<int> _GetDiameters()
{
    return Wheels.Select(wheel => GetDiameter(wheel));
}

// Second - calculate diameter of ONE wheel.
int GetDiameter(Wheel wheel)
{
    return wheel.Rim + (wheel.Tire * 2);
}

These refactorings are useful and necessary even when you don’t know what the final design will look like. In fact, these refactorings (and good practices like these) will often reveal the final design.
Small methods with a single responsibility have many benefits. They make a class’s details, features, and components more obvious, they are self-documenting, encourage reuse, establish a style of programming that is self-perpetuating, become easy to move to another class when and if that time comes.
Enforce single responsibility everywhere: isolate extra responsibilities in classes. Move the responsibilities that are extraneous to a class into another class. If you can afford it, create a separate class to hold them. If not, at least encapsulate them in an embedded class so that they are contained and don’t leak.
Embedded classes say: This class only has meaning and value when thought about within the context of this other class.
Embedded classes can be easily promoted to independent classes later if the need arises.
Once all methods inside a class have a single responsibility and are each doing the smallest useful thing, it becomes easier to identify the features that may belong to another class.
The purpose of design is not to produce “perfect” code but instead to produce code that’s “good enough”.
The Single Responsibility Principle “allows change without consequence and reuse without duplication”.

Chapter 3: Managing Dependencies

About dependencies

Objects should have a single responsibility and do the smallest useful thing. This means that, in order to fulfill complex application requirements, objects need to collaborate with one another.
Objects collaborate by knowing things about and sending messages to each other. That creates dependencies. A good design manages these dependencies, making sure they don’t couple objects so much that change becomes painful.
Coupling measures how much interdependency exists between software components. High coupling makes things hard to change.
An object has a dependency when it knows the name of another class.
An object has a dependency when it knows the name of a method from another class.
An object has a dependency when it knows the arguments of that method.
An object has a dependency when it knows the order of those arguments.
Dependencies create a situation where if an object changes, then other objects that depend on it may be forced to change as well.
Dependencies couple objects to each other. Highly coupled objects behave as if they were a unit. It becomes hard to reuse one without the other. If not managed properly, dependencies will disseminate and entangle the entire code base.
Some dependencies are unavoidable. So the purpose of design is to reduce and contain dependencies. Ensure objects only have the minimum number of dependencies that they need to do their jobs. Keep the essential ones, remove the rest.

Techniques for writing loosely coupled code

Dependency injection. It can help eliminate dependencies on concrete classes. Instead of instantiating new objects of another class inside your object, let the object receive it as a constructor or method parameter. The object’s users can do the instantiation.
An object that knows less can often times do more. The knowledge of how to instantiate a particular object limits its potential. Object creation and utilization don’t need to be contained within the same unit of code.
When an object hard-codes the reference to a class, it advertises that it can’t work with any other type. Replace the reference to the concrete class with an abstraction (e.g. an interface or duck type). That gives it the flexibility to be able to collaborate with other objects that respond to the same messages.

With dependency injection you can turn this:

class Gear
{
    float Ratio { get; set; }
    int Rim { get; set; }
    int Tire { get; set; }

    public Gear(float ratio, int rim, int tire)
    {
        Ratio = ratio;
        Rim = rim;
        Tire = tire;
    }

    public float GetGearInches()
    {
        return Ratio * new Wheel(Rim, Tire).GetDiameter();
    }
}

Into this:

class Gear
{
    float Ratio { get; set; }
    Wheel Wheel { get; set; }

    // Instead of rim and tire, Gear's constructor now receives a wheel.
    public Gear(float ratio, Wheel wheel)
    {
        Ratio = ratio;
        Wheel = wheel;
    }

    public float GetGearInches()
    {
        // Instead of creating a new wheel, it uses the one that's been injected.
        return Ratio * Wheel.GetDiameter();
    }
}

Sometimes it is not possible to remove a dependency from an object. In those cases, your best bet is to isolate it.
Isolate dependencies: isolate instance creation. Sometimes it is not possible to apply dependency injection. In those cases, the next best thing is to encapsulate instantiation of external classes into their own methods.
An instantiation encapsulated in a method allows the code to clearly and explicitly communicate that there’s an extraneous dependency here. One that we’d like to remove but for now cannot. We also protect the class from that dependency propagating through it.
Isolate dependencies: isolate vulnerable external messages. The same technique can be applied to invocations of methods on external classes. Encapsulate such calls into their own methods in order to contain the dependency.

Here’s what isolating some dependencies could look like. From this…

float GetGearInches()
{
    return Ratio * new Wheel(Rim, Tire).GetDiameter();
}

…to this:

float GetGearInches()
{
    return Ratio * GetWheelDiameter();
}

// The dependency on the Wheel class is isolated in this method.
int GetWheelDiameter()
{
    return new Wheel(Rim, Tire).GetDiameter();
}

Reduce argument dependencies: use keyword arguments. If your language supports them, keyword arguments are a great way of reducing the dependency incurred via method invocations. They allow the parameters to be sent in any order, which means that changes to the method’s definition and signature are less likely to cause the callers to have to also change. Also, they have a self-documenting property on both caller and callee. (See named arguments in C# and keyword arguments in Ruby).
Reduce argument dependencies: explicitly define defaults. If your language has the feature, parameter defaults are a great way of reducing dependencies to method parameters, as establishing a default for a parameter makes it optional.
Reduce argument dependencies: isolate multiparameter initialization. If you don’t own the code for a complex object initialization or method invocation, you can always encapsulate it in an object or method of your own. It can be done with a thin wrapper that exposes an interface that follows the techniques we’ve discussed and that uses the language of your application.
Classes should depend on things that change less often than they do.
There are three truths about code related to change and dependencies:
1. “Some classes are more likely to change than others”.
2. “Concrete classes are more likely to change than abstract classes”.
3. “Changing a class that has many dependents will have many consequences”.
It’s OK to depend on classes with a low likelihood of change. For example, classes in the built-in library of your language generally will change less than your own custom classes. Framework classes commonly change less often too, but that may not always be the case if you framework is new or rapidly evolving.
Abstractions are generally much more stable than concretions. Because of this, it is safer to depend on abstract classes (e.g. interfaces and duck types) than on concrete ones.
Avoid classes that have lots of dependencies. Any changes done to them produce a ripple effect of changes throughout the code base. Also, because of that, people will go through great lengths to try to avoid changing them.
When thinking about dependents and likelihood of change, the following are harmless:
1. Classes with few dependents and low likelihood of change (e.g. specialized infrastructure interaction classes).
2. Classes with few dependents and high likelihood of change (e.g. concrete classes that implement your app’s domain logic).
3. Classes with many dependents and low likelihood of change (e.g. interfaces and other abstractions).
When thinking about dependents and likelihood of change, the following is harmful: Classes with many dependents and high likelihood of change (e.g. concrete classes that are used throughout the code base).

Chapter 4: Creating Flexible Interfaces

About interfaces

The messages that objects pass between each other are a big concern for design. In addition to what objects know (responsibilities), and who they know (dependencies), design cares about how they talk to one another. Messages pass between objects through their interfaces.
In a well designed application, the messages that pass between objects follow a pattern that’s closer to the diagram on the right than that of the diagram on the left:

Objects that expose too much of themselves and know too much about others are hard to reuse and make systems hard to change.
Objects that minimize what they expose of themselves and know little about others are easily reusable and changeable.
An object’s public interface is the set of messages that it responds to. That is, the set of methods that other objects are welcome to invoke. Good design calls for objects with clear and concise public interfaces.
Classes should have one responsibility, but they will likely have many methods. Some methods are more general and expose the main features of a class; they are the services that the class offers its callers. These should make up the class’s public interface. Other methods are more specific, serve to support those features, and contain implementation details internal to the class and uninteresting to callers. These make up the class’s private interface.
The methods in a class’s public interface:
1. “Reveal its primary responsibility”
2. “Are expected to be invoked by others”
3. “Are not likely to change”
4. “Are safe for others to depend on”
5. “Are directly covered by tests”
The methods in a class’s private interface:
1. “Handle implementation details”
2. “Are not expected to be invoked by other objects”
3. “Are likely to change”
4. “Are unsafe for others to depend on”
5. “May not even be referenced in the tests”
Public methods list the specific features of a class that allow it to fulfill its responsibility. They advertise to the world what the purpose of the class they belong to is.
Public methods are stable and are expected to not change often, so others are welcome to depend on them.
Private methods are not stable at all. They are hidden from others so nobody should depend on them.

Discovering less obvious objects and messages

When analyzing a problem domain, the nouns in the narrative usually become your first classes. These are called “domain objects”.
Don’t let domain objects be your only classes, though. Shift your focus toward messages and away from classes to discover new, less obvious objects that will implement key functionality. Not doing so will risk overloading your domain objects with more responsibility than what they can handle.
UML sequence diagrams are an excellent way to explore design alternatives. You can use them to draft objects and their interactions, i.e. the messages they send each other. They are quick, low cost, and allow easy communication of ideas between team members. Here’s an online tool that generates them based on plain text: sequencediagram.org.
The transition from class-based design to message-based design is one that yields more flexible applications. It means asking “I need to send this message, who should respond to it?” instead of “I know I need this class, what should it do?”.

Focusing on “what” instead of “how”

Public methods should focus on “what” instead of “how”. That is, they should express what the caller wants instead of how the callee must behave.
If you’re in a situation where an object calls many methods on another in succession, try to refactor so that all the logic is executed as a result of a single message. This consolidation reduces the size of the second object’s public interface, which reduces what callers need to know about it, which reduces dependency, which makes change easier. The caller only knows “what” it needs, not “how” to make the callee deliver.

When focusing on “what” instead of “how”, code like this:

class Trip
{
    IEnumerable<Bicycle> Bicycles { get; set; }
    Mechanic Mechanic { get; set; }

    public void Prepare()
    {
        foreach (var bike in Bicycles)
        {
            Mechanic.CleanBicyble(bike);
            Mechanic.PumpTires(bike);
            Mechanic.LubeChain(bike);
            Mechanic.CheckBrakes(bike);
        }
    }
}

class Mechanic
{
    public void CheckBrakes(Bicycle bike) { }
    public void CleanBicyble(Bicycle bike) { }
    public void LubeChain(Bicycle bike) { }
    public void PumpTires(Bicycle bike) { }
}

Can be transformed into code like this:

class Trip
{
    IEnumerable<Bicycle> Bicycles { get; set; }
    Mechanic Mechanic { get; set; }

    // This method was greatly simplified and its dependencies were reduced.
    public void Prepare()
    {
        foreach (var bike in Bicycles)
        {
            Mechanic.PrepareBicycle(bike);
        }
    }
}

class Mechanic
{
    public void PrepareBicycle(Bicycle bike)
    {
        CleanBicyble(bike);
        PumpTires(bike);
        LubeChain(bike);
        CheckBrakes(bike);
    }

    // Notice how Mechanic's public interface was reduced considerably.
    // All of these methods are private now.
    private void CheckBrakes(Bicycle bike) { }
    private void CleanBicyble(Bicycle bike) { }
    private void LubeChain(Bicycle bike) { }
    private void PumpTires(Bicycle bike) { }
}

Liberate objects from their context

An object’s context is made up of the things that an object knows about others. What other objects it calls and how. What it needs in order to be able to work.
A good design tries to allow objects to work with minimal context. “Objects that have a simple context are easy to use and easy to test; they expect few things from their surroundings. Objects that have a complicated context are hard to use and hard to test; they require complicated setup before they can do anything”.
In order to reduce context, we need to reduce the knowledge that callers have about the other components they call. A simple public interface helps with that. In practical terms that means fewer and less verbose methods, fewer parameters, fewer dependencies.
Objects need to trust their collaborators to do their part. Focusing on what the caller wants instead of what the callee does is a way of keeping context in check. This allows objects to collaborate with others regardless of what type they are and what they do.
Dependency injection is a mechanism through which objects can collaborate with others when they don’t know their type. A dependency injected via parameter (declared as an interface or duck type) hides its identity from its user.
Naming the methods of this injected dependency from the perspective of the caller reveals a generic interface that offers the features that the caller wants in the vocabulary that it understands. The caller does not need to know what the injected dependency does, only what it needs it to do.
Highly coupled objects with verbose public interfaces say to their collaborators: “I know what I want, and I know how you do it”.
More decoupled objects with concise public interfaces say to their collaborators: “I know what I want, and I know what you do”.
Highly decoupled objects with concise public interface and minimal required context say to their collaborators: “I know what I want, and I trust you to do your part”.

When objects trust their collaborators, and focus on what they want instead of what others do, the previous Trip/Mechanic example can become like this:

class Trip
{
    public IEnumerable<Bicycle> Bicycles { get; set; }
    Mechanic Mechanic { get; set; }

    public void Prepare()
    {
        // Trip now fully trusts Mechanic, and doesn't even know what it does.
        Mechanic.PrepareTrip(this);
    }
}

class Mechanic
{
    // The knowledge of "how a mechanic prepares a trip" now completely lives
    // within Mechanic.
    public void PrepareTrip(Trip trip)
    {
        foreach (var bike in trip.Bicycles)
        {
            PrepareBicycle(bike);
        }
    }

    private void PrepareBicycle(Bicycle bike)
    {
        CleanBicyble(bike);
        PumpTires(bike);
        LubeChain(bike);
        CheckBrakes(bike);
    }

    private void CheckBrakes(Bicycle bike) { }
    private void CleanBicyble(Bicycle bike) { }
    private void LubeChain(Bicycle bike) { }
    private void PumpTires(Bicycle bike) { }
}

Rules of thumb for writing code with good interfaces

Create explicit interfaces. Be intentional and obvious when defining public and private methods. Use your language access modifiers (i.e. public, private, protected, etc). Your public methods should:
1. Be explicitly identified as such
2. Be more about what than how
3. Have names that are unlikely to change
4. Use keyword arguments
Respect the public interface of others. Invoke only public methods on the classes that you use. Avoid circumventing their public interfaces and directly calling private members. Depending on the private interface of framework and library classes is like a ticking time bomb. The reason they are private is because they are expected to change or disappear entirely.
Minimize context. Focus on “what” instead of “how” when designing public interfaces. Favor public methods that allow callers to access your classes’ functionality without having to know how they do it. Use interface and duck types to name methods from the perspective of and with the vocabulary of the callers.
If you have to interact with a class that does not have a clean interface, and you don’t own it or otherwise can’t refactor it so that it does, isolate it. Use the same techniques for dependency isolation mentioned in Chapter 3. Wrap the invocation in a new class or method to contain it and give that wrapper a clean public interface.
Follow the law of Demeter. The law of Demeter states that you shouldn’t chain multiple method calls that navigate across many different types of objects. In other words “talk only to your close neighbors” or “use only one dot”.

Violations ot the law of Demeter look like this:
customer.GetBicycle().Wheel.Rotate();

Violations of the law of Demeter make for code that’s not TRUE. Changes in the object at the end of the chain ripple through the entire chain. This is unexpected and laborious, making the code neither Transparent nor Reasonable. The class that uses the chain depends on all the objects in the chain, making it non-Usable. These chains are easy to replicate and harm changeability, making the code not Exemplary.
Always evaluate the cost of violating the law of Demeter versus the cost of abiding by it. Method chains that ultimately read an attribute are generally less harmful. Also, method chains on really stable classes like those of your language library or framework have low impact.
Delegation can appear to be a solution to law of Demeter violations. Unfortunately, all it does is remove the evidence that it’s there.
In reality, a violation to the law of Demeter indicates that there’s an object missing or that the public interface of an object is lacking. Find the object, and its interface, by thinking more about messages and less about classes. Define the interface by thinking about what the object needs of its collaborators, and not about how it can get it by itself.

Chapter 5: Reducing Costs with Duck Typing

About duck types

Duck types are public interfaces that manifest themselves across classes. When multiple classes accept the same messages (i.e. define methods with the same signature), they are said to be of the same duck type.
“Duck type” is a dynamic language concept. Static languages offer equivalent features via “interfaces”.
The difference is that, in statically typed languages, interfaces need to be explicitly defined in code and concrete classes need to explicitly implement them. On the other hand, duck types in dynamically typed languages need only be acknowledged in an abstract sense. They don’t have to be explicitly defined in code.
Duck types and interfaces establish a contract. A set of messages that a given object is expected to be able to receive. A protocol that users of these objects can be confident that they will adhere to.
Just as a class can implement multiple interfaces, an object can be of many duck types. That’s because interfaces and duck types materialize depending on the situation in which objects that implement them are used. The same object can be used in different places for different things. Each user will see the object from their own perspective, and each can expect it conform to a different interface. See the interface segregation principle.
Moreover, code that depends on an interface or duck type is flexible enough to collaborate with any concrete class that implements that interface or duck type. See the Liskov substitution principle.
As we’ve established, it’s better to depend on abstractions than to depend on concretions. So, users of an object should not care what its type is, only what it does. Behavior over identity. In other words, they should focus on messages and public interfaces, instead of specific classes. Duck types and interfaces bring to the forefront what an object does and abstracts away what it is.
Applying duck types to a situation makes the code more abstract and less concrete. Concrete code is easier to understand but harder to extend. Abstract code is harder to understand at the beginning but easier to change. In the long run, abstract code tends to reduce maintenance costs.
When an object invokes methods on multiple objects in order to achieve a single purpose, that’s a situation where a duck type may be helpful. Think from the perspective of the caller and what it needs to come up with a message that makes sense for it to send its many collaborators. That method should be part of the public interface of each of its collaborators. To the caller’s eyes, each of its collaborators belong in the same duck type.

Writing code that leverages duck types

Whenever you see conditional logic that sends a different message depending on the concrete class of a given object, that’s the code telling you that it needs a duck type or interface.
Come up with a message that makes sense from the perspective of the caller and add it to each of the classes expected by your code in the conditional. These classes now share the same public interface, they are of the same duck type. Then remove the conditional and just call that method. Replace conditional logic with polymorphism.
In general, try to avoid code that explicitly mentions concrete classes or checks for the existence of particular methods in order to determine further behavior. Instead, come up with a shared interface and send those messages to the collaborators regardless of their actual type.

Higly conditional and concrete logic like this…

class Trip
{
    public IEnumerable<Bicycle> Bicycles { get; set; }
    public IEnumerable<Customer> Customers { get; set; }
    public Vehicle Vehicle { get; set; }

    public void Prepare(IEnumerable preparers)
    {
        foreach (var preparer in preparers)
        {
            if (preparer is Mechanic)
            {
                ((Mechanic)preparer).PrepareBicycles(Bicycles);
            }
            else if (preparer is TripCoordinator)
            {
                ((TripCoordinator)preparer).BuyFood(Customers);
            }
            else if (preparer is Driver)
            {
                ((Driver)preparer).GasUp(Vehicle);
                ((Driver)preparer).FillWaterTank(Vehicle);
            }
        }
    }
}

class Mechanic
{
    public void PrepareBicycles(IEnumerable<Bicycle> bicycles) { }
}

class TripCoordinator
{
    public void BuyFood(IEnumerable<Customer> customers) { }
}

class Driver
{
    public void FillWaterTank(Vehicle vehicle) { }
    public void GasUp(Vehicle vehicle) { }
}

…can be rewritten to be more abstract if we leverage polymorphism. Like this:

class Trip
{
    public IEnumerable<Bicycle> Bicycles { get; set; }
    public IEnumerable<Customer> Customers { get; set; }
    public Vehicle Vehicle { get; set; }

    // This method can be much simpler now, and abstract.
    // Ready to collaborate with any "TripPreparer".
    // The conditional logic has been replaced with polymorphism.
    public void Prepare(IEnumerable<ITripPreparer> preparers)
    {
        foreach (var preparer in preparers)
        {
            preparer.PrepareTrip(this);
        }
    }
}

// New interface that all preparers implement. It has a single method.
// It greatly simplifies the communication between Trip and its many
// possible preparers.
interface ITripPreparer
{
    void PrepareTrip(Trip trip);
}

class Mechanic : ITripPreparer
{
    public void PrepareTrip(Trip trip)
    {
        PrepareBicycles(trip.Bicycles);
    }

    private void PrepareBicycles(IEnumerable<Bicycle> bicycles) { }
}

class TripCoordinator : ITripPreparer
{
    public void PrepareTrip(Trip trip)
    {
        BuyFood(trip.Customers);
    }

    private void BuyFood(IEnumerable<Customer> customers) { }
}

class Driver : ITripPreparer
{
    public void PrepareTrip(Trip trip)
    {
        FillWaterTank(trip.Vehicle);
        GasUp(trip.Vehicle);
    }

    private void FillWaterTank(Vehicle vehicle) { }
    private void GasUp(Vehicle vehicle) { }
}

The one exception to this rule is when dealing with exceptionally stable classes. Like those in your language libraries, where the introduction of duck types would mean modifying core language libraries. Explicit type checks against these classes are usually low cost.
Duck types are abstract and less obvious in the code. That’s why they need to be well documented and tested.

Chapter 6: Acquiring Behavior through Inheritance

About inheritance

Inheritance establishes a relationship between two classes where it is said that a subclass B “is a” superclass A. Subclass and superclass are also referred to as derived class and base class, respectively.
Inheritance can be more technically defined as an “automatic message forwarding mechanism”. That is, when a subclass receives a message that it cannot respond to directly (i.e. a method that it does not implement), the language runtime takes care of automatically sending the message to the superclass.
That is, inheritance establishes a hierarchy where superclasses share their code with their subclasses.
Subclasses are meant to be specialized versions of their superclass. Subclasses should be “the same, plus more” when compared to their superclass. They should offer the same public interface.
Inheritance is ideal to solve the problem where you need a series of slightly different classes that share a lot of behavior.
A telltale sign that inheritance needs to be applied is when code contains an “if” statement checking an attribute that contains the “category” of the object, and based on that determines what code to execute. Watch out for this anti-pattern and for variables with names like “type”, “category”, “style” that control branches in behavior.
This is an anti-pattern that reveals that the object knows too much and needs to be broken down into smaller pieces. It increases the costs of change.

Here’s what the anti-pattern looks like:

class Bicycle
{
    string Style { get; set; }
    string TapeColor { get; set; }
    string FrontShock { get; set; }

    public Bicycle(Dictionary<string, string> options)
    {
        Style = options["style"];
        TapeColor  = options["tape_color"];
        FrontShock = options["front_shock"];
    }

    // Checking 'style' starts down a slippery slope.
    public Dictionary<string, string> GetSpares()
    {
        if (Style == "road")
        {
            return new() {
                ["chain"] = "11-speed",
                ["tire_size"] = "23",
                ["tape_color"] = TapeColor
            };
        }
        else if (Style == "mountain")
        {
            return new() {
                ["chain"] = "11-speed",
                ["tire_size"] = "2.1",
                ["front_shock"] = FrontShock
            };
        }
    }
}

Applying inheritance

Be on the lookout for existing classes that may lead you down the wrong path when it comes to inheritance. More often than not, classes that already exist in the code base are not good candidates for extension via inheritance. They are not good candidates to be superclasses.
The path forward is likely to be a new class to serve as the base, and then update your existing class to be a subclass. Then add other subclasses as peers of it.
For inheritance to work, the objects being modeled need to truly share a generalization-specialization relationship.
The superclass needs to define the common behavior that is shared among subclasses. The subclasses define the specializations.
In many cases, the superclass should be abstract, meaning that they are not supposed to be instantiated. They represent an incomplete object which only becomes whole when looked at in the context of each of its subclasses.
It almost never makes sense to create a superclass with only a single subclass. In fact, creating an inheritance hierarchy with two subclasses is often risky. You risk coming up with the wrong abstraction.
Three subclasses is the sweet spot to commit to inheritance. Put off the decision to implement inheritance until that point, if you can. That’s the point when there’s enough information available to confidently determine an abstraction that will be useful and cost effective.
If you put off implementing an inheritance hierarchy, then you won’t be able to share code between the highly related classes. That will likely lead to code repetition, which is also costly. Consider what costs more: having the repetition for the time being; or doing nothing and waiting for more information to avoid making the wrong decision.
“Every decision you make includes two costs: one to implement it and another to change it when you discover that you were wrong”.
When refactoring towards an inheritance hierarchy, consider that “promoting” code from the concrete subclass up into the abstract superclass is often easier and less error prone.
Errors in promoting are easy to identify and fix. All that can happen if you miss a promotion is that a subclass that was meant to inherit some behavior won’t have it.
Going in the opposite direction and missing a “demotion” will produce design errors that are harder to spot and have dire consequences if left alone.
That would mean that concrete behavior, which does not apply to all subclasses, stays incorrectly in the abstract base class. That’ll throw a wrench in the works and the whole inheritance hierarchy will be on shaky grounds. The abstract base class won’t be generic enough and subclasses will be tempted to circumvent it.
“The general rule for refactoring into a new inheritance hierarchy is to arrange code so that you can promote abstractions rather than demote concretions”.
The template method pattern provides a clean way of defining a common basic algorithm in the superclass and allowing subclasses to supply specializations for it. Superclasses can define an algorithm, and call certain methods in key points within it. These are extension points. These methods can then be implemented by subclasses, letting them control part of the overall logic.
When promoting concrete code to the more abstract superclass, consider using the template method when methods cannot be promoted wholesale, and have to be broken up instead. The parts that can be promoted become the template method; the parts that can’t be become the specialization methods that each concrete subclass implements.
To avoid future bugs, superclasses should provide default implementations for the specialization methods that it expects its subclasses to implement. These can be no-ops or, even better, raise errors. These errors let developers know that extending this inheritance hierarchy requires these methods to be implemented.
Hook methods are a slightly lighter iteration of the template method concept that helps alleviate superclass-subclass coupling. Same principle: let the superclass send a message that the subclass implements in order to provide a specialization.
A hook method need not be a part of a common abstract algorithm, hence the slight distinction from the full fledged template method design pattern.
Beware of subclasses explicitly invoking functionality on their superclasses. Languages often offer keywords such as “base” or “super” that allow easily sending messages to superclasses. These are dangerous because they couple subclasses with their superclasses. They reveal that subclasses know the general algorithm.
The template method pattern and hook methods invert this dependency, allowing the superclass to call the subclass. Allowing the subclass to provide specializations without knowing too much about the superclass.

Here’s an example of a properly factored inheritance hierarchy:

// The main purpose of the base class is to establish a basic functionality
// that can be easily extended by subclasses.
// This class cannot be instantiated, as given by the "abstract" modifier.
// This means that our application has no need of holding onto pure Bicycle
// instances, only the concrete subclasses are to be used.
// This abstract class is just a vehicle for sharing code between other classes.
abstract class Bicycle
{
    public string Size { get; set; }
    public string Chain { get; set; }
    public string TireSize { get; set; }

    // Notice how this constructor implements the template method pattern
    // in order to allow subclasses to make changes to the overall algorithm.
    // Specifically, it allows them to provide default values for the Chain and
    // TireSize attributes as well as run any additional logic after
    // initialization.
    public Bicycle(Dictionary<string, string> options)
    {
        Size = options.GetValueOrDefault("size");
        Chain  = options.GetValueOrDefault("chain") ?? GetDefaultChain();
        TireSize = options.GetValueOrDefault("tire_size") ?? GetDefaultTireSize();

        AfterInitialize(options);
    }

    // This method establishes the basic spares information and allows
    // subclasses to supply more by calling the GetLocalSpares hook method.
    public Dictionary<string, string> GetSpares()
    {
        var defaultSpares = new Dictionary<string, string> {
            ["tire_size"] = "23",
            ["chain"] = "11-speed"
        };

        return new(defaultSpares.Concat(GetLocalSpares()));
    }

    // This method, which is marked as abstract, has to be implemented by
    // subclasses. It allows subclasses to contribute specializations.
    protected abstract string GetDefaultTireSize();

    // These methods, marked as virtual, all have default implementations of
    // varying complexity but the important aspect is that they can be
    // overridden by subclasses. They also allow subclasses to contribute
    // specializations.
    protected virtual void AfterInitialize(Dictionary<string, string> options) { }
    protected virtual Dictionary<string, string> GetLocalSpares() => new();
    protected virtual string GetDefaultChain() => "11-speed";
}

// Notice how the following concrete subclasses are very simple.
// Extending this kind of code is easy, because creating new subclasses is easy.
// They leverage their abstract base class to implement most of their features.
// Their job is to supply specializations on the core logic defined in the base
// class. To do so, they override the hook methods defined by the base class.
class RoadBike : Bicycle
{
    public string TapeColor { get; set; }

    public RoadBike(Dictionary<string, string> options) : base(options) { }

    protected override void AfterInitialize(Dictionary<string, string> options) =>
        TapeColor = options.GetValueOrDefault("tape_color");

    protected override Dictionary<string, string> GetLocalSpares() =>
        new() { ["tape_color"] = TapeColor };

    protected override string GetDefaultTireSize() => "23";
}

class MountainBike : Bicycle
{
    public string FrontShock { get; set; }
    public string RearShock { get; set; }

    public MountainBike(Dictionary<string, string> options) : base(options) { }

    protected override void AfterInitialize(Dictionary<string, string> options)
    {
        FrontShock = options.GetValueOrDefault("front_shock");
        RearShock = options.GetValueOrDefault("rear_shock");
    }

    protected override Dictionary<string, string> GetLocalSpares() =>
        new() { ["front_shock"] = FrontShock };

    protected override string GetDefaultTireSize() => "2.1";
}

class RecumbentBike : Bicycle
{
    public string Flag { get; set; }

    public RecumbentBike(Dictionary<string, string> options) : base(options) { }

    protected override void AfterInitialize(Dictionary<string, string> options) =>
        Flag = options.GetValueOrDefault("flag");

    protected override Dictionary<string, string> GetLocalSpares() =>
        new() { ["flag"] = Flag };

    protected override string GetDefaultChain() => "10-speed";
    protected override string GetDefaultTireSize() => "28";
}

About roles

Using classical inheritance is always optional. Every situation that calls for inheritance could be solved using another technique that allows for sharing code between otherwise unrelated classes.
The concept of “roles” is an alternative to classical inheritance. This concept emerges from the need of multiple classes to be used in a particular context to do the same thing. So they themselves need to share behavior.
Think of a role as an augmented form of a duck type. A group of classes are said to play the same role when they belong in the same duck type or implement the same interface. In addition to exposing the same public interface, roles allow them to share internal behavior.
Roles are ideal for storing responsibilities that are orthogonal to classes. It allows classes that are otherwise unrelated to share behavior. They establish a “behaves like” type of relationship between objects, as opposed to the “is a” relationship that is established with inheritance.
When you include a module into an existing class, all the methods defined in the module become available to the class. This is the same thing that happens when a subclass inherits from a superclass.
Many languages offer native features to allow for this kind of relationship between objects. In Ruby, we use Modules. Other languages use Mixins, Traits, Default Methods. Simply put, all these are ways of bundling together a group of methods that can be easily plugged into existing classes.

Writing inheritable code

The same principles, techniques, and anti patterns that apply to the design of duck types and inheritance hierarchies also apply to roles.
Beware of objects that use a variable with a name like “type”, “category” or “kind” to determine what code to execute. The object is likely concealing (and acting like) two or more types. These would be subclasses or module includers.
When code is checking the type of an object to decide which message to send it, that’s a signal that there’s a missing abstraction. There’s a duck type or role in there that needs to be explicitly brought to light. A public interface needs to be defined for it. If there is a need to share behavior, put it in a module, mixin, etc.
“All of the code in the abstract superclass should apply to every class that inherits it”.
Beware of subclasses or module includers that override a superclass/module method just to raise a “not implemented” error. Chances are that if only part of the superclass applies to it, then it doesn’t belong in the hierarchy. Or maybe the whole hierarchy needs a redesign.
Subclasses agree to the contract specified by their superclasses. They must respond to every message in the superclass’s public interface, accept the same types of inputs and return the same types outputs.
Put in other words, “subtypes must be substitutable for their supertype”. That is, all subclasses should act like their superclass. This means that any code that expects a superclass should be able to work, without change, with all its subclasses as well. That’s the Liskov substitution principle.
Through the concept of variance, subclasses have a slightly higher degree of freedom. Subclasses can accept inputs of more abstract types and can return outputs of more concrete types than those specified in the superclass’s public interface while still being substitutable for their superclass.
For example, given a class “Object” that is a superclass of a class “String”; if a method in a superclass “MyClass” accepts a String parameter, a subclass of MyClass could override that same method and accept Object and still be compliant. Likewise, if a method in a superclass returns an Object result, the subclass could return a String for that same method, and still be compliant. Still be substitutable.
In other words, some code that sends a message with a String parameter can send the same message to a receiver that accepts Object. Likewise, some code that expects an Object as a result of sending a message can send the same message to a receiver that returns a String. Because a String “is an” Object, any code that works with an Object can work with a String, it’ll just treat it like an Object and only access Object’s public interface, which String fully supports.

This is what variance allows:
class Superclass
{
    public virtual Object DoSomething(String parameter) => new Object();
}

class Subclass : Superclass
{
    public override String DoSomething(Object parameter) => "the result";
}
Be aware that C# currently does not support variance in parameters, only in return types. So this code would not actually work as it is, since the parameter types are different.

The template method pattern is the ideal technique for creating superclasses that are easy to inherit from. It allows clean separation of an abstract algorithm from the concrete specializations.
Be on the lookout for code that directly invokes superclass behavior with keywords like “super” or “base” etc. Instead, use hook methods to allow subclasses to contribute to parts of the common algorithm.
Favor shallow hierarchies instead of deep ones. Narrow is also preferable to wide. Wide ones are easier to live with as long as they are shallow. But deep and wide ones are a maintenance nightmare. That is, keep the vertical levels of inheritance as low as you can. Hierarchies with many layers of superclasses are hard to understand and maintain.

Chapter 8: Combining Objects with Composition

About composition

Composition establishes a “has a” relationship between objects. It is a technique where you combine simple, independent objects to turn them into larger, more complex ones. Composition combines distinct parts so that the resulting entity is more than the sum of its parts.
In composition, the container object (or composed object) “has a” contained component (or part). Mechanically, this usually means that the container’s class has an attribute which holds a reference to the part object. It’s a dependency that’s usually injected via constructor parameter.
The container communicates with the part via its public interface. The part plays a role, and the container collaborates with it. This means that containers generally interact with their parts through interfaces or duck types.
Composition is generally a cheaper alternative to code sharing when compared to inheritance.
When refactoring from inheritance into composition, the parts sometimes need to share some code, in addition to sharing the same public interface. This is because all possible parts play the same role for the container. In these cases, an inheritance hierarchy of parts may be what’s needed. Or, they could share behavior via modules or mixins.
In technical terms, the technique of composition takes two forms: aggregation and, well, composition. Under composition, parts don’t have any use or value outside of their container objects. Under aggregation, on the other hand, parts can live on their own.

Factories

When creating certain objects becomes complex, encapsulate that complexity into a factory. That way the knowledge is stored in a single place in the code base. In principle, factories are simple: They are objects that create other objects.
An advantage of factories is that they make the process of turning complex data structures into objects easy. You can have a “specification”, stored as pure data in a file or database. The factory can then implement the logic that understands the meaning of the data structure and how to create living objects based on it.

Deciding between inheritance and composition

Inheritance and composition are fundamentally code arrangement techniques where logic is distributed among various objects. With inheritance, objects are organized into a strict hierarchical structure and get automatic message delegation. With composition, objects are independent from each other but messages need to be manually delegated.
In general, “favor composition over inheritance”. Composition is lower cost (i.e. fewer dependencies) than inheritance in most circumstances where both could be used to solve a problem.

Pros and cons of inheritance

When properly applied, inheritance is excellent at producing code that is Reasonable, Usable and Exemplary.
Inheritable code is Reasonable because small changes in code can produce great changes in behavior. This is because code near the top of the hierarchy is defined once but used by all subclasses. Changing such code allows to alter the behavior of many subclasses.
Inheritable code is Usable because superclasses are literally designed to be easy to reuse. Inheritance hierarchies adhere to the open-closed principle. It’s easy to extend the hierarchy by creating new subclasses without having to touch existing code, thus, reusing it.
Inheritable code is Exemplary because it intrinsically provides a clear path to extend it. Even novice programmers won’t have a hard time creating new subclasses when the superclasses implement template methods and hooks which guide any extension efforts.
One important disadvantage of inheritance is that the cost of mistakes is considerably high. Incorrect applications of inheritance are costly, whether it be that the wrong abstraction was created or that inheritance was just the wrong tool for the job altogether.
Another disadvantage is that, for other contexts outside of the hierarchy, the ways in which it’s possible to interact with the hierarchy are limited. If the behavior defined within the hierarchy needs to be used, a subclass needs to be created. Other use cases may be incapable of tolerating that dependency.
If you’re writing a framework or a library, this aspect becomes even more important. You can’t know all the scenarios in which your library will be used, and forcing users to create subclasses in order to reuse the logic defined in the hierarchy may be more than what they can afford.

Pros and cons of composition

“When using composition, the natural tendency is to create many small objects that contain straightforward responsibilities that are accessible through clearly defined interfaces”. “Composition results in applications built of simple, pluggable objects that are easy to extend and have a high tolerance for change”.
When properly applied, composition is excellent at producing code that is Transparent, Reasonable and Usable.
Composable code is Transparent because small objects are easy to understand and changes have obvious effects. These objects also don’t necessarily form part of hierarchies, which means they are not susceptible to changes in superclasses.
Composable code is Reasonable because its behavior is easy to extend by just implementing new objects that play the role of parts. All new parts need to do is implement the public interface that the container object already expects.
Composable code is Usable because it’s made up of small independent objects. They don’t have any structural dependencies preventing them from being reused in different contexts, completely unrelated to the container or even the part role.
One disadvantage of composition is the fact that it may be more difficult to understand how the whole application works. While the individual objects may be small and simple, how they come together to solve problems may not be.
Another disadvantage is that objects need to delegate messages to each other manually, as opposed to inheritance where this happens automatically. This means that there is the explicit dependency of the container object having to know which messages to call on its parts.
Composition excels at separating containers from parts and assembling such objects. However, it doesn’t have an answer for the scenario where it is necessary to handle a collection of types of parts that are very similar to each other. That’s where inheritance comes in.
Use inheritance for “is a” relationships. Use duck types, interfaces, and behavior sharing via modules or mixins for “behaves like a” relationships. Use composition for “has a” relationships.

Chapter 9: Designing Cost-Effective Tests

The benefits of testing

The goal of design is to write code that is easy to change. In order to do that, we need good object-oriented design skills, good refactoring skills and good testing skills.
Good object-oriented design skills are needed because badly designed code, by definition, is hard to change.
Good refactoring skills are needed because design needs to be evolving constantly. As new requirements and new information about the domain become available, the code needs to adapt.
Good testing skills are needed because high value, solid tests enable continuous refactoring without fear of breaking the code.
Just like design, the true purpose of testing is to reduce costs. Tests do help reduce bugs, provide documentation, and improve the design; but those are only the means through which tests achieve their ultimate goal of reducing the cost of change.
Tests help by finding bugs: Test help expose bugs early in the process. Bugs are easier to find and cheaper to correct the closer in time we are to their introduction. Testing at the same time a feature is being implemented catches bugs before they go out the door. Bugs caught early are not given the chance to cause problems or have code depend on them.
Tests help by supplying documentation: Tests represent the best and most reliable documentation of the design. When static documents and memories get outdated and/or disappear, tests remain. Write tests that tell the story of how the code works.
Tests help by deferring design decisions: As a code base evolves, there will be spots in the code that are less than stellar. Spots which we’re not confident enough in to commit to a particular design. So we postpone making a decision and hack together a solution that works today. However, we know that it will need to be refactored to handle the requirements of tomorrow. These spots represent a lack of knowledge. Knowledge that the design is waiting for in order to improve. While waiting for that knowledge to come, we put the hack behind a clear, stable public interface and write our tests against that. In doing so, we’ll be free to refactor into a good design when the missing information arrives. Knowing that the tests have our backs.
Tests help by supporting abstractions: “Good design naturally progresses toward small independent objects that rely on abstractions”. One disadvantage to this is that, while small independent objects are easy to understand by themselves, the behavior of the whole application, with all the little objects working together, becomes obscured. Tests solve that problem by highlighting the abstractions, their interfaces, how to work with them, and how they work together.
Tests help by exposing design flaws: If tests are hard to write, that’s the perfect indicator that the code under test is hard to reuse. Which indicates high coupling, which indicates increased costs of change. If testing an object requires a lot of setup and a lot of other objects, then it requires too much context and has too many dependencies.

What, when and how to test

“One simple way to get better value from tests is to write fewer of them. The safest way to accomplish this is to test everything just once and in the proper place”.
Tests should focus only on the incoming messages that are defined in each object’s public interface (“A” in the picture above). That is, the public methods. Public interfaces expose the services that a class offers its users. They are also stable, so depending on them is safe. Tests that cover public methods are resilient to refactorings and add value because they exercise objects in the same way that their users in application code do.
Never test private methods (“B” in the picture above). They are meant to be used by the object internally. They change often and other objects should not depend on them because they will have to change with them. The same applies to tests. Tests that exercise private code break on every refactoring.
An object’s test suite should never assert the return value of outgoing messages that it sends to other objects (“C” in the picture above). These messages are part of the public interface of the receiver object, so the receiver’s test suite should be the one testing it.
On the other hand, an object’s test suite should always assert that necessary outgoing messages are sent, and with the correct parameters (“C” in the picture above).
A “test of state” is a test that asserts the return value of a method.
A “test of behavior” is a test that asserts whether the object under test calls a particular method on another object, how many times, and with what parameters.
A “query” is a method call with no side effects that is only made to get some value back.
A “command” is a method call that has side effects that are important to the overall application.
All public methods should be covered by tests of state. A test suite should only cover the public methods of its own object under test.
Queries should not be tested for state or behavior.
Commands should be tested for behavior.
Write tests first when it makes sense to do so. Writing tests before implementing the feature establishes an intention for the design from the beginning. Since tests are reused, they make sure the code to be written will have the bare minimum of reusability.
Testing first is no silver bullet, though. Even though it helps steer the design in the right direction, by itself it won’t produce a well-designed application. Fundamental object-oriented design techniques still need to be applied.
Well-designed applications are easy to change. Well-designed tests may never have to change. This is because they depend on abstractions and public interfaces, which are stable.
Two major styles of testing exist: Behavior Driven Development and Test Driven Development.
“BDD takes an outside-in approach, creating objects at the boundary of an application and working its way inward, mocking as necessary to supply as-yet-unwritten objects”.
“TDD takes an inside-out approach, usually starting with tests of domain objects and then reusing these newly created domain objects in the tests of adjacent layers of code”.
Both styles can be followed to produce valuable tests.

Writing valuable tests

Incoming messages (i.e. public methods) need to be covered by tests of state. That is, the values that they return must be validated.
Be on the lookout for public methods that don’t receive any calls. These are good candidates for deletion. If no other object calls it, then it needs to be private.
Objects should be tested in isolation. When that’s not possible, there’s a problem with the design. It means that the object is too coupled to its collaborators and can’t be reused on its own. It needs to be refactored.
It’s important that objects are tested in isolation because a test suite should care only about the one object that its testing. The other objects will be tested by their own test suites. Changes in an object should affect its own test suite, not that of other objects.
When the object under test depends on another object in order to work properly, refactor into dependency injection and have the test provide the dependency that way.
When using dependency injection, don’t pass in a concrete, “real” object in your tests. Instead, inject a test double or mock into the object under test.
Dependency injection also hints at the emergence of a role, interface, or duck type. Instead of depending on a concrete class, refactor the object under test to depend on a role. The double or mock in the test suite will be one of the objects that play the collaborator role that the object under test expects.
Private methods and query method calls need not be tested at all.
Testing private methods is redundant because they are internal to the object under test and invoked by public methods, which already have tests.
Private methods are also unstable. Not meant to be depended on because they change often. That will make for brittle tests.
Private methods covered by tests may mislead others in thinking that the method is stable and, thus, dependable.
Query method call results are not relevant to the overall application, only to the object under test. They are hidden within it. Also, the receiver of the query already includes tests for it, given that it is part of its public interface. So, they should be ignored by the test suite.
Command method calls do need to be covered by tests in order to prove that they were called with the correct parameters. In other words, it is the responsibility of the object under test to send that message, so its test suite needs to prove it.
Mocks are the way to test that command messages get sent. They help with tests of behavior. Using mocks injected into the object under test, a test case can validate that the method is called properly.
Commands should be tested for behavior, not state. So, mocks should not be used for tests of state. That is, tests should not be concerned with asserting what they return. However, they can be configured to return some value, if the operation requires it.
In dynamically typed languages that don’t have a compiler to do type checks, it is important to test objects for conformance with the public interfaces of their roles, and for proper integration with their inheritance hierarchies.
“Tests should document the existence of roles, prove that each of their players behave correctly, and show that dependents interact with them appropriately”.
Pragmatically, this means writing test cases that assert whether objects implement the specific messages defined in their roles’ public interface and those that their superclasses expect from them.
Multiple subclasses will undoubtedly share many of the same tests. In those cases, try to encapsulate the test cases into reusable components that every subclass’s test suite can invoke. Just like in application code, it’s better to avoid repetition.
Testing abstract base classes can sometimes be challenging because they are not supposed to be instantiated. In some languages, it is downright impossible to instantiate classes marked as abstract. For these scenarios, a test-specific subclass can be created with minimal implementation that allows tests to exercise the base class.
To avoid test brittleness, test doubles that play certain roles and inherit from superclasses should also be tested for conformance.

The Pragmatic Programmer book, 20th anniversary edition

2020-10-16T00:00:00+00:00

The Pragmatic Programmer is a now-classic book about software development, first published in 1999. It is old enough that it predates the famous Agile Manifesto of 2001. The authors of the book, Andy and Dave, were two of the 14 authors of that manifesto.

For its 20th anniversary in 2019, Dave and Andy created a new edition. They updated things that had become dated, such as mentions of programming languages, tools, operating systems, websites, etc. That was, I imagine, the easy part. They went on to extensively revise the entire text and incorporate new lessons learned in the past two decades.

A classic

This book is part of our company’s “work philosophy canon” that I ask every software developer at End Point to read, so for that reason and others I wanted to be familiar with the new edition and make sure it is still something I want to recommend so broadly.

The book is also required reading for university courses at Cornell (CS 3110: Data Structures and Functional Programming) and the University of Washington (CSE 331: Software Design and Implementation), probably among others!

I first read this book 19 years ago and really enjoyed it. I found that it clearly expressed many useful concepts I had stumbled upon in my own programming experience. More importantly, it gave me other helpful guidance and warned of pitfalls. Having that all collected in book form with concise, memorable chapter names and tips gave me more shared vocabulary to use with others in our work.

I also enjoyed reading the new 20th anniversary edition. It did not have the same personal impact, which is to be expected since I am at a very different point in my programming career, technical reading, and life experience. But I think the new edition has as great or even increased power for new readers.

Overview of changes

The authors estimate that around 75% of the text has changed, and that seems about right. A few areas that stood out to me were:

The now widely-known adage “don’t repeat yourself” originally meant primarily not to duplicate knowledge, that each piece of data should appear in only one canonical location. Over time, “don’t repeat yourself” has been more often used to preach against copying and pasting code. While that is also a good general rule, it has been oversimplified and turned into a blanket prohibition. Dave & Andy mention cases where duplicating code is the right thing to do, because the problem space is not the same and factoring out all such code duplication will make later changes far more complicated as the code has to behave differently for different callers.
Computer security was not nearly as big of a concern when they were writing the first edition in the late 1990s, and now that almost every computer is networked and attackers are motivated to reach every system from anywhere in the world, they gave the topic more attention.
Concurrency gets much more discussion, as is sensible in a time where scaling up typically means using many more CPU cores and processors.
Unit testing was far less common in the late ’90s, and there were few good test frameworks, so they recommended you write your own. Now there are many good test frameworks for every language, so they of course recommend you choose one of those and use it.
Dave writes “A Confession” on page 223: He tells us that writing tests for 30+ years made testing so much a part of the way he thinks and programs, that he now writes testable interfaces even when he doesn’t write tests. He isn’t saying that testing is no longer important, but rather that it’s not a religion, that it can and should be examined by experienced programmers, and it has important effects on the programmer aside from the tests themselves.
The old edition had checklists scattered throughout, and collected in the pull-out card at the end. The new edition doesn’t have checklists anymore, except the Debugging Checklist on page 97. I think that’s ok. They weren’t something I ever referred back to.

Tips

The old edition had 70 tips that appear throughout the chapters. The new edition has 100 tips. In both cases they are extracted and collected on a pull-out card at the end of the book. What changed?

Removed or heavily reworked tips

Write Code That Writes Code
Use Exceptions for Exceptional Problems
Design Using Services
Separate Views from Models
Don’t Use Wizard Code You Don’t Understand
Abstractions Live Longer than Details
Costly Tools Don’t Produce Better Designs

Tips that changed slightly

“Use a Single Editor Well” is now “Achieve Editor Fluency”
“Don’t Panic When Debugging” is now simply “Don’t Panic”
“Minimize Coupling Between Modules” is now “Decoupled Code Is Easier to Change”
“Configure, Don’t Integrate” is now “Parameterize Your App Using External Configuration”
“Don’t Gather Requirements—Dig for Them” is now “Requirements Are Learned in a Feedback Loop”
“Organize Teams Around Functionality” is now “Organize Fully Functional Teams”
“Gently Exceed Your Users’ Expectations” is now “Delight Users, Don’t Just Deliver Code”
“Put Abstractions in Code, Details in Metadata” is now “Policy Is Metadata”
“Always Design for Concurrency” is now “Random Failures Are Often Concurrency Issues”
“Listen to Nagging Doubts—Start When You’re Ready” is now “Listen to Your Inner Lizard”
“Don’t Be a Slave to Formal Methods” is now “Do What Works, Not What’s Fashionable”
“Some Things Are Better Done than Described” transformed to “Agile Is Not a Noun; Agile Is How You Do Things”

New tips

You Have Agency
Good Design Is Easier to Change Than Bad Design
Forgo Following Fads
Failing Test Before Fixing Code
Read the Damn Error Message
Act Locally
Take Small Steps—Always
Avoid Fortune-Telling
Tell, Don’t Ask
Don’t Chain Method Calls
Avoid Global Data
If It’s Important Enough To Be Global, Wrap It in an API
Programming Is About Code, But Programs Are About Data
Don’t Hoard State; Pass It Around
Don’t Pay Inheritance Tax
Prefer Interfaces to Express Polymorphism
Delegate to Services: Has-A Trumps Is-A
Use Mixins to Share Functionality
Shared State Is Incorrect State
Use Actors For Concurrency Without Shared State
Testing Is Not About Finding Bugs
A Test Is the First User of Your Code
Build End-To-End, Not Top-Down or Bottom Up
Use Property-Based Tests to Validate Your Assumptions
Keep It Simple and Minimize Attack Surfaces
Apply Security Patches Quickly
Name Well; Rename When Needed
No One Knows Exactly What They Want
Programmers Help People Understand What They Want
Don’t Go into the Code Alone
Maintain Small Stable Teams
Schedule It to Make It Happen
Deliver When Users Need It
Use Version Control to Drive Builds, Tests, and Releases
First, Do No Harm
Don’t Enable Scumbags
It’s Your Life. Share it. Celebrate it. Build it. And have fun!

The new edition’s PDF version does not include the pull-out card with tips that comes with the printed book. That’s too bad, because the tips were what I most often referred to in the old edition after I finished reading it. At least Andy and Dave have published on their website a list of the 100 updated tips. It would be even nicer to have a PDF of the book’s pull-out card as they did for the old edition.

Upgrade?

As you can see, there are many new tips. The text has been updated. It is still of excellent quality. So should you get the new edition?

I mentioned earlier two university courses that use the book. In both courses the professors allow students to use either the original 1999 version or the 2019 version. That seems like the right thing to do to me as well.

If you already have the old edition of the book, I don’t think it’s necessary to rush out and buy the new edition. The old one is still quite good and relevant, and you’ll be learning a bit of software development history by reading the older one.

If you do not yet have the book, or if you will be going over it with others who are new to it, it is probably best for you to get the new edition as well so you can be familiar with the new tips and the ways Dave & Andy describe things now.

Summing up

I am glad that Dave and Andy updated their book. Their updates make sense and make the book more relevant to present-day readers.

Relatedly, it would be good to also see an update of the also excellent book Practices of an Agile Developer by Venkat Subramaniam and Andy. That is coming up on 15 years old itself. Perhaps some of its topics have been incorporated in the new edition of The Pragmatic Programmer, but as I review the table of contents, it looks like it is still mostly additive. I have cited its important maxim “Different Makes a Difference” so often that I thought it was from The Pragmatic Programmer instead!

I’ll close with this good observation and advice from Andy in their Changelog podcast interview:

Was it Dijkstra who had the Turing Award lecture about the very humble programmer? That is a critical piece of early literature… And you talk about things that haven’t changed. This was 1972? … And he makes the very important point that complexity will overwhelm us if we don’t take a very humble, very measured approach. And it’s been 30–40 years and everyone—present company included—has ignored this wonderful advice.

Humility is difficult in our environment, in our culture, and it is probably, of all the human factorsy things that you need to be a good developer, I would submit that being humble, realizing you don’t know all the answers, that you need to find out, that you need to experiment, get feedback, try it.

This part of our headlong rush into the shiny new thing is this kind of faith that “Well, that’s gonna be better. I can do it better. I’m better than this/that.” Yeah, maybe… But you should validate that. You should try it. You should go back and read these things. You should try these other experiments.

Thanks, Andy & Dave!

Reference

All tips from each edition

Interviews

Functional Geekery Episode 126 – Andy Hunt and Dave Thomas from July 2, 2019 by Steven Proctor
The Changelog – Episode #352: The Pragmatic Programmers with Andy Hunt & Dave Thomas by Adam Stacoviak & Jerod Santo
CodeNewbie Podcast season 9 episode 8 from August 26, 2019: Why you should read the new edition of the Pragmatic Programmer with Andy Hunt, Dave Thomas by Saron Yitbarek (author of the new edition’s foreword)

Other comparisons of the two editions

Sections added, changed significantly, and removed by Kevin Hooke
The Pragmatic Programmer Book 2nd Edition Differences by Miranda Limonczenko (very detailed, but covers only the first 3 chapters)

Book pages

The Pragmatic Programmer, 20th Anniversary Edition book home page, errata, source code downloads, and links to buy
Practices of an Agile Developer book home page, samples, pull-out card, and review by Ethan Rowe

A tribute to Kyle Simpson’s JavaScript book series

2019-07-24T00:00:00+00:00

Photo by othree, used under CC BY 2.0

Inspired by the Ruby Fight Club, a group of us have been reading Kyle Simpson’s You Don’t Know JS series (1st edition; in 2020 Kyle started publishing drafts of the 2nd edition of the books). These books are a great source of inspiration and available for free. I meet weekly with our small group to discuss chapters from these books. Each time we have a presenter who walks us through the chapter that we all read beforehand.

During these sessions we have learned a lot about JavaScript, but also about preparing presentations. The increasing quality level of the meetings was noticable each week. I think we all owe a large thanks to Kyle Simpson. In this article I will focus on the book “You Don’t Know JS: ES6 and Beyond”.

Past, present and future

ECMAScript (ES for short) was versioned with a small number up until now, like 5. ES1 and ES2 were not widely known or implemented. ES3 was used by Internet Exporer 6–8 and Android 2.x. ES4 never came out. ES5 came out in 2009. ES5.1 came out in 2011 and was widely used by Firefox, Chrome, Opera, Safari, etc.

Now, version names will be in the format ES, but it might change to a per-feature basis.

In the past JavaScript versions were based on major releases of ES. However, due to the importance of the language, it is too much of a hassle to wait till 20 features are ready and release them together. It is much better to make finished features available as soon as they are ready. As a result, we know what functionalities we’re gaining.

It’s good to reflect on just how important the language is. JavaScript is the lingua franca for web browsers. Web developers need to be aware of JavaScript to its slightest details. While the markup of a webpage is HTML and its design is CSS, the client-side programming is done via JavaScript, more or less in a standard manner.

Since JavaScript is almost completely standard for browsers, programmers would have a much easier life if they were writing JavaScript when coding on server-side as well. There is a mental leap when one works both on the client-side and server-side of a feature and has to write Java/C++/PHP/Ruby or whatever server-side code and then, in the next moment, they have to change their way of thinking and switch to the client-side and write JavaScript code. This is not difficult for a seasoned programmer who’s used to doing this. But when such a switch happens 50–60 times a day, it gets tiresome, and the programmer might not even realize why the day was tiring, since all they did was implement some simple features or fix some simple bugs.

It is perfectly logical that JavaScript found its way into server-side programming. Node.js is a standalone JavaScript runtime and whoever uses it for server-side web programming automatically gains the benefit of being able to work with the same language both on server side and client side. Of course, one still has to work with HTML, CSS, and a database as a web programmer, so multiple languages will be needed at some point, but there is a level of comfort given to web programmers using Node.js. Of course, the event loop used by JavaScript along with its single-threaded approach makes it less effective in doing some CPU-intensive stuff.

However, for server-side calculations, I would not be surprised at all if sooner or later full support is added to Node.js for multithreaded work. A great possible benefit of using JavaScript both on server-side and client-side, especially if there is a WebSocket connection involved, is that the server and the client could use the very same object, which opens the possibility to create a new paradigm. I know it’s science fiction, but imagine how cool it would be to implement a JavaScript class/prototype and while doing so being able to define what should be available for the client-side as well. Object state change could happen on both server-side and client-side and synchronization could be triggered in such a duplex channel. Internet connection problems could be handled as well. Let’s imagine the case when there is a grid to be shown for the user. The user wants to define filters, sort, maybe layout, the server has to find the data, possibly store the settings, also, generate the structure upon object creation. Of course, before this can be realized a lot of open questions need to be answered. However, as a utopian view, it looks great.

Compiling and Transpiling

JavaScript is a hybrid interpreted and compiled language, since most modern runtimes compile the code, but it can be interpreted as well at runtime. Let’s see this code:

var foo = [1,2,3];
var obj = {
 foo  // means `foo: foo`
};
obj.foo; // [1,2,3]

The code above natively assumes that the value of the foo attribute of obj is foo, because when an attribute is not defined, a value of the same name is the default, in this case, the value will be our foo variable. However, in old versions of JavaScript, this was not yet supported, so if we really need to use this code in older browsers, then we will need to convert our source-code into its oldschool equivalent, using a Transpiler (the action of transpiling is to convert a source-code to another source-code, possibly of different language, not modifying its essence).

JavaScript transpilers are a big topic, but for the sake of readability, we won’t delve too much into the details. I have the habit of using Babel whenever I’m in doubt that the result of compiling is the one that I expect, or I suspect an error in the result, or I am interested for any other reason about the actual result.

Polyfilling

“A polyfill, or polyfiller, is a piece of code (or plugin) that provides the technology that you, the developer, expect the browser to provide natively. Flattening the API landscape if you will.” —Remy Sharp

If we expect a functionality or value to be in the web browser or, more widely, in JavaScript, but it is not existing yet, or is not sure to exist everywhere we intend to use it, then we define it. In general, if we expect something defined by the name of myCoolStuff, then we can do polyfilling like this:

if (!myCoolStuff) { //Here we assume that some variable called myCoolStuff exists and we check whether it's falsy
    //Define myCoolStuff
}

The code above is unsafe though, because if myCoolStuff is not defined at all, then an error will be thrown. A better, more reliable approach is to compare its type against undefined.

if (typeof myCoolStuff === "undefined") {
    //Define myCoolStuff
}

So far, so good. However, we might be tired of writing that kind of condition over and over again. An alternative is to abstract the approach, like:

var toPolyfill = [
    {
        context: window,
        myCoolStuff: function () {/*...*/},
        someOtherStuff: 42,
        yetAnother: function (someParameter) {/*...*/},
        //...
    },
    {
        context: Object,
        foo: "bar"
    }
];

and then do something like this:

for (var obj of toPolyfill) {
    for (var key in obj) {
        if ((key !== "context") && (typeof obj.context[key] === "undefined")) {
            obj.context[key] = obj[key];
        }
    }
}

As we can see, we have an array called toPolyfill whose items are all objects. Each object has a context attribute, which specifies where should we check the other attributes for existence and use the default if not found.

Let’s take a look at the polyfilling of Object.is, which is a function to determine whether two variables hold the same values:

if (Object.is === undefined) {
    Object.is = function(v1, v2) {
        // test for `-0`
        if (v1 === 0 && v2 === 0) {
            return 1 / v1 === 1 / v2;
        }
        // test for `NaN`
        if (v1 !== v1) {
            return v2 !== v2;
        }
        // everything else
        return v1 === v2;
    };
}

Object.is("abc", "abc") && Object.is(1/0, Infinity) && Object.is(NaN, NaN); //true

Well, that was nice, wasn’t it? But this is always called as Object.is(). Why not make it instance level?

Object.prototype.is = function(value) {
    return Object.is(this, value);
};

It’s looking great and it should work, right? Let’s test it:

"abc".is("abc"); //false

What? Don’t worry, we just missed using strict, which ensures that we are in strict mode, introduced in ES5:

"use strict";
if (!Object.is) {
    Object.is = function(v1, v2) {
        // test for `-0`
        if (v1 === 0 && v2 === 0) {
            return 1 / v1 === 1 / v2;
        }
        // test for `NaN`
        if (v1 !== v1) {
            return v2 !== v2;
        }
        // everything else
        return v1 === v2;
    };
}

Object.prototype.is = function(value) {
    return Object.is(this, value);
}

"abc".is("abc"); //true

Block scope

Virtually anyone with some experience in JavaScript has met a situation when variables did not behave as expected, especially when asynchronous stuff was involved. One reason is that variables created using the var keyword are function-scoped, not block-scoped. Proof:

function foo() {
    if (1) {
        var something = 5;
    }
    something; //5
}

foo();
something; //error

The example above illustrates that the variable created in the inner scope of the function can be used without a problem outside the inner scope of the function, as long as it’s inside the function and that it cannot be reached outside the function; in short, we can see that our variable is function-scoped.

So let’s see an example where this is a problem. Let’s calculate the sum of natural numbers from 1 to 100, we expect, of course, 5050 as a result:

var sum = 0;
for (var index = 1; index <= 100; index++) setTimeout(function() {sum += index}, 100);
setTimeout(function() {
    alert(sum); //10100
}, 200);

Wait, what? We expected 5050 as a result, yet, it was 10100, exactly twice as much. What happened? Let’s study this carefully:

we initialize sum with 0
we iterate a variable called index from 1 to 100 and add functions to the event loop to be executed 100 milliseconds later than the moment we reached to them at the iteration
when index reaches 100, the last iteration is executed and index is incremented
index is 101 at the end of the cycle
we add a new function to the event loop to be executed 200 milliseconds later, alerting the result
100 milliseconds later the function which increases sum by index (101) will be executed 100 times
as a result, sum will be 100 * 101 = 10100
100 more milliseconds later, the function which alerts sum (10100) is executed

Okay, we understand this. But why was index 101 and why not its current value of the iteration? Well, the answer is simple: the cycle ran before the functions which it added to the event loop. So we clearly have a problem with the function-scoped variable in this case. Luckily, we are able to use block scope as well, using the let keyword:

var sum = 0;
for (let index = 1; index <= 100; index++) setTimeout(function() {sum += index}, 100);
setTimeout(function() {
    alert(sum); //5050
}, 200);

Wow, that was neat. Our cycle creates a block-scoped variable on each iteration and as a result the function given to setTimeout inside the same block will use the correct variable each time.

Defaults

JavaScript has a neat way of setting defaults for parameters, like:

function theMeaningOfLife(result = 42) {
    setTimeout(function() {
        console.log("The meaning of life is " + result);
    }, 10000000*365.25*24*60*60*1000);
}

This is nice, but there might be cases when the parameters are actually attributes of an object and that’s passed as parameter:

function defaultize(params, expectedParams) {
    if (typeof params !== "object") {
        //params is not an object, do something
    }
    for (var key in expectedParams) {
        if (!(key in params)) params[key] = expectedParams[key];
    }
}

function bar(params) {
    defaultize(params, {
        foo: "fooDefault",
        bar: "barDefault",
        lorem: "loremDefault",
        ipsum: "ipsumDefault"
    });
    console.log(params);
}

bar({
    bar: "beer bar",
    ipsum: "dolor"
});

We can even use a function call as default:

function defaultize(params, expectedParams) {
    if (typeof params !== "object") {
        //params is not an object, do something
    }
    for (var key in expectedParams) {
        if (!(key in params)) params[key] = expectedParams[key];
    }
}

function bar(params, defaults =
    defaultize(params, {
        foo: "fooDefault",
        bar: "barDefault",
        lorem: "loremDefault",
        ipsum: "ipsumDefault"
    })) {
    console.log(params);
}

bar({
    bar: "beer bar",
    ipsum: "dolor"
});

In the example above, if defaults is not passed to bar, then defaultize will be called, passing params and the defaults. Since params is an object, defaultize changes its attributes when needed. A neat example of default parameters can be seen in the next chunk of code, where a variable actually changes when a value is not passed to the function:

var value = 0;

function something(x = value++) {
    console.log(x);
}

something(); //x has a value of 0, value is incremented to 1
something(); //x has a value of 1, value is incremented to 2
something(); //x has a value of 2, value is incremented to 3
something(1); //x has a value of 1, value remains 3
something(); //x has a value of 3, value is incremented to 4

Applying defaults

The book (ES6 and beyond) gives us an example of applying defaults:

var defaults = {
    options: {
        remove: true,
        enable: false,
        instance: {}
    },
    log: {
        warn: true,
        error: true
    }
};

var config = {
    options: {
        remove: false,
        instance: null
    }
};

config = Object.assign( {}, defaults, config );

With the code above we see that nested values in the default are not transferred. Losing data is not a desired situation. The problem is that Object.assign is shallow. The book gives this solution:

var defaults = {
    options: {
        remove: true,
        enable: false,
        instance: {}
    },
    log: {
        warn: true,
        error: true
    }
};

var config = {
    options: {
        remove: false,
        instance: null
    }
};

config.options = config.options || {};
config.log = config.log || {};

({
    options: {
        remove: config.options.remove = defaults.options.remove,
        enable: config.options.enable = defaults.options.enable,
        instance: config.options.instance = defaults.options.instance
    } = {},
    log: {
        warn: config.log.warn = defaults.log.warn,
        error: config.log.error = defaults.log.error
    } = {}
} = config);

This solves our problem. enable is successfully set to false, but I did not like this solution very much, because it is a very particular solution for this data and is not reusable for other cases, where config and options have different attributes, so I came up with this reusable code:

var defaults = {
    options: {
        remove: true,
        enable: false,
        instance: {}
    },
    log: {
        warn: true,
        error: true
    }
};

var config = {
    options: {
        remove: false,
        instance: null
    }
};

function applyDefaults(d, t) {
    if ((typeof d !== "object") && (typeof d !== "array")) {
        return d;
    }
    if (t === undefined) {
        if (typeof d === "object") {
            t = {};
        } else if (typeof d === "array") {
            t = [];
        }
    }
    for (var key in d) {
        if (t[key] === undefined) {
            t[key] = d[key];
        } else {
            applyDefaults(d[key], t[key]);
        }
    }
    return t;
}

applyDefaults(defaults, config);
console.log(config);

We can see that the recursive applyDefaults function does not rely on the attribute names and due to its agnosticity, we can reuse it for any defaulting.

ES5 Getter/Setter

A very nice example in the book about ES5 getters and setters is this one:

var o = {
    __id: 10,
    get id() { return this.__id++; },
    set id(v) {this.__id = v;}
}

console.log(o.id);                        // 10
console.log(o.id);                        // 11
console.log(o.id = 20);
console.log(o.id);                        // 20
// and:
console.log(o.__id);                        // 21
console.log(o.__id);                        // 21 – still!

The getter always returns the current value of id and increments it.

Computed property names

var myObject = {
    a: 1,
    b: 2,
    c: 3
};

What if we want to define some operations for it:

var myObject = {
    a: 1,
    b: 2,
    c: 3,
    &&: function() {/*...*/},
    ||: function() {/*...*/},
    ===: function() {/*...*/},
    !==: function() {/*...*/},
    ==: function() {/*...*/},
    !=: function() {/*...*/},
};

The code will of course break, because data members of an object have the same syntax constraint in their naming as variables, see more here. You can also try out a name validator. We can solve the problem like this:

var myObject = {
    a: 1,
    b: 2,
    c: 3
};
myObject["&&"] = function(){/*...*/};
myObject["||"] = function(){/*...*/};
myObject["==="] = function(){/*...*/};
myObject["!=="] = function(){/*...*/};
myObject["=="] = function(){/*...*/};
myObject["!="] = function(){/*...*/};

However, this doesn’t look as neat as the code which failed. After some research and experimenting I came up with this solution:

var myObject = {
    a: 1,
    b: 2,
    c: 3,
    ["&&"]: function() {/*...*/},
    ["||"]: function() {/*...*/},
    ["==="]: function() {/*...*/},
    ["!=="]: function() {/*...*/},
    ["=="]: function() {/*...*/},
    ["!="]: function() {/*...*/},
};

Kyle Simpson makes sure that in the future none of the JavaScript programmers who had the blessing of reading his books will not have to suffer like me. He comes up with some very illustrative examples:

We can add computed properties to an object with the assignment of foo["b" + "ar"] = "baz", like:

var prefix = "user_";

var o = {
    baz: function(){  }
};

o[ prefix + "foo" ] = function(){ console.log("foo"); };
o[ prefix + "bar" ] = function(){ console.log("bar"); };

o[prefix + "foo"]();
o[prefix + "bar"]();

However, we can define these properties at initialization with the syntax of:

var prefix = "user_";

var o = {
    baz: function(){  },
    [ prefix + "foo" ] : function(){ console.log("foo"); },
    [ prefix + "bar" ] : function(){ console.log("bar"); }
};

o[prefix + "foo"]();
o[prefix + "bar"]();

Symbol properties:

var o = {
    [Symbol.toStringTag]: "really cool thing",
};

console.log(o.toString());

var o2 = {};

console.log(o2.toString());

o2[Symbol.toStringTag] = "really cool thing";

console.log(o2.toString());

Computed properties can be names of concise methods or concise generators:

var o = {
    ["f" + "oo"]() {/*...*/},        // computed concise method
    *["b" + "ar"]() {/*...*/}        // computed concise generator
};

We can even set the prototype:

var foo = 5;

var o1 = {
    foo
};

console.log(o1.foo); //5
foo++;

var o2 = {
    __proto__: o1,
};

console.log(o2.foo++); //5

var o3 = {
    __proto__: o2
}

console.log(o3.foo); //6

Object super

We can set the prototype of an object and then calling super inside a function can make sense. Consider this example:

var o1 = {
    foo() {
        console.log( "o1:foo" );
    }
};

var o2 = {
    foo() {
        super.foo();
        console.log( "o2:foo" );
    }
};

Object.setPrototypeOf( o2, o1 );

o2.foo();  // o1:foo
           // o2:foo

Template literals

JavaScript provides us with a neat way to manage long texts in the form of template literals. Let’s see some examples from the book:

var name = "Kyle";
var greeting = `Hello ${name}!`;

console.log( greeting );          // "Hello Kyle!"
console.log( typeof greeting );   // "string"

Notice that we have a template literal assigned to greeting and how neatly we can put into it dynamic values. In this case, the value of the name variable.

Another example is:

var text =
`Now is the time for all good men
to come to the aid of their
country!`;

console.log( text );
// Now is the time for all good men
// to come to the aid of their
// country!

We can conveniently put HTML into template literals like this:

var myHTML = `
<!DOCTYPE html>
<html>
<body>

<h2>An Unordered HTML List</h2>

<ul>
  <li>Coffee</li>
  <li>Tea</li>
  <li>Milk</li>
</ul>  

<h2>An Ordered HTML List</h2>

<ol>
  <li>Coffee</li>
  <li>Tea</li>
  <li>Milk</li>
</ol> 

</body>
</html>
`;

It is as if we were looking at only HTML code. Imagine how great it is to read code like this instead of concatenated strings with lots of addition signs and quotes. Let’s implement a function which generates the options of a select tag:

function generateOptions(input) { //format of [{key, value}, …] is expected
    var output = "";
    for (let item of input) output += `<option value="${item.key}">${item.value}</option>`;
    return output;
}

var options = generateOptions([
    {key: 1, value: "Coffee"},
    {key: 2, value: "Tea"},
    {key: 3, value: "Milk"},
]);

I think this is very elegant. Let’s make a select:

var select = `<select class="my-class">${options}</select>`;
//<select class="my-class"><option value="1">Coffee</option><option value="2">Tea</option><option value="3">Milk</option></select>

That’s not very pretty to read, is it? Let’s make it nicer:

function generateOptions(input) { //format of [{key, value}, …] is expected
    var output = "";
    for (let item of input) output += `\n    <option value="${item.key}">${item.value}</option>`;
    return output;
}

var options = generateOptions([
    {key: 1, value: "Coffee"},
    {key: 2, value: "Tea"},
    {key: 3, value: "Milk"},
]);

var select = `<select class="my-class">${options}\n</select>`;
/*
<select class="my-class">
    <option value="1">Coffee</option>
    <option value="2">Tea</option>
    <option value="3">Milk</option>
</select>
*/

But wait, we do not even need the \n (we can keep it though if we want, it’s a matter of preference):

function generateOptions(input) { //format of [{key, value}, …] is expected
    var output = "";
    for (let item of input) output += 
`
    <option value="${item.key}">${item.value}</option>`;
    return output;
}

var select = 
`<select class="my-class">${options}
</select>`;

Let’s add a button:

function generateOptions(input) { //format of [{key, value}, …] is expected
    var output = "";
    for (let item of input) output += `
    <option value="${item.key}">${item.value}</option>`;
    return output;
}

var myChunk = `<select class="my-class">${options}
</select>
<input type="button" value="GO">`;

I’m literally (no pun intended) in awe to see that in JavaScript we can template HTML so nicely.

Even some more complex problems are not problems when one uses template literals:

function upper(s) {
    return s.toUpperCase();
}

var who = "reader";

var text =
`A very ${upper( "warm" )} welcome
to all of you ${upper( `${who}s` )}!`;

console.log( text );
// A very WARM welcome
// to all of you READERS!

for…of loops

The syntax looks like this:

for (var item of collection) {
    //Do something with item
}

for…of loops can only be executed for iterables. It is as if we were doing:

for (var key in collection) {
   //Do something with collection[key]
}

However, there are technical differences for the loop of:

for (var item of collection) {
    //Do something with item
}

Babel generates:

"use strict";

for (var _iterator = collection, _isArray = Array.isArray(_iterator), _i = 0, _iterator = _isArray ? _iterator : _iterator[Symbol.iterator]();;) {//Do something with item

  var _ref;

  if (_isArray) {
    if (_i >= _iterator.length) break;
    _ref = _iterator[_i++];
  } else {
    _i = _iterator.next();
    if (_i.done) break;
    _ref = _i.value;
  }

  var item = _ref;
}

Let’s see some examples:

var a = ["a","b","c","d","e"];

for (var idx in a) {
    console.log( idx );
}
// 0 1 2 3 4

for (var valA of a) {
    console.log( valA );
}
// "a" "b" "c" "d" "e"

var b = {
    key1: "value1",
    key2: "value2",
    key3: "value3",
    key4: "value4",
    key5: "value5",
    key6: "value6",
}
//”key1” “key2” “key3” “key4” “key5” “key6”
for (var idy in b) console.log(idy);

for (var valB of b) console.log(valB);
//Uncaught TypeError: b is not iterable

A pre-ES6 alternative is to get the keys of the object and iterate the set of keys:

var a = ["a","b","c","d","e"],
        k = Object.keys( a );

for (var val, i = 0; i < k.length; i++) {
    val = a[ k[i] ];
    console.log( val );
}
// "a" "b" "c" "d" "e"

An alternative using the iterator Symbol:

var a = ["a","b","c","d","e"];

for (var val, ret, it = a[Symbol.iterator]();
    (ret = it.next()) && !ret.done;
) {
    val = ret.value;
    console.log( val );
}
// "a" "b" "c" "d" "e"

Supported built-in values in JavaScript that are by default iterable:

Arrays
Strings
Generators
Collections

An example with Strings:

for (var c of "hello") {
        console.log( c );
}
// "h" "e" "l" "l" "o"

for (var c of new String("hello")) {
    console.log( c );
}
// "h" "e" "l" "l" "o"

And another one with arrays and destructuring:

var o = {};

for (o.a of [1,2,3]);
console.log(o);

for ({x: o.a} of [ {x: 1}, {x: 2}, {x: 3} ]);
console.log(o);

And another one with generators:

function *myGenerator() {
    var y = 1;
    while (y < 10) yield y++;
}

var values = myGenerator();

for (v of values) console.log(v);

We have an upper boundary for y. If there was no such boundary, the for…of would go on forever.

However, generators have a duplex channel of communication between the function and the caller. We can pass some values to a generator when we iterate it. I was thinking whether there is a way to pass values somehow to a generator while iterating it with a for…of loop. Consider this example:

function *myGenerator(input) {
    console.log(input); //undefined
    while(input = yield) {
        console.log(input); //never gets here
    };
}

var gen = myGenerator(1);
for (var input of gen);

We do not have difficulties in doing that with a while cycle:

function *myGenerator(input) {
    console.log(input);
    while(input = yield) {
        console.log(input);
    };
}

var gen = myGenerator(1);
//for (var input of gen);
var counter = 10;
while (!gen.next(--counter).done);
console.log("Finished");

If we intend to parameterize our generator using a for..of loop, then we are entering a rabbit hole and a lot of suffering awaits us. So, from our for…of loop that we would like to parameterize:

function *myGenerator(input) {
    console.log(input); //undefined
    while(input = yield) {
        console.log(input); //never gets here
    };
}

var gen = myGenerator(1);
for (var input of gen);

Babel generates this hairy monster:

"use strict";

var _marked = /*#__PURE__*/regeneratorRuntime.mark(myGenerator);

function myGenerator(input) {
    return regeneratorRuntime.wrap(function myGenerator$(_context) {
        while (1) {
            switch (_context.prev = _context.next) {
                case 0:
                    console.log(input);
                case 1:
                    _context.next = 3;
                    return;
                case 3:
                    if (!(input = _context.sent)) {
                        _context.next = 7;
                        break;
                    }
                    console.log(input);
                    _context.next = 1;
                    break;
                case 7:
                    ;
                case 8:
                case "end":
                    return _context.stop();
            }
        }
    }, _marked, this);
}

var gen = myGenerator();
for (var _iterator = gen, _isArray = Array.isArray(_iterator), _i = 0, _iterator = _isArray ? _iterator : _iterator[Symbol.iterator]();;) {
    var _ref;
    if (_isArray) {
        if (_i >= _iterator.length) break;
        _ref = _iterator[_i++];
    } else {
        _i = _iterator.next();
        if (_i.done) break;
        _ref = _i.value;
    }
    var input = _ref;
}

This is not enough, since this strangely throws an error because regeneratorRuntime does not exist, yet we try to call it. Let’s wrap a function around this code and download Babel’s polyfill:

"use strict";
function mainFunction() {

var _marked = /*#__PURE__*/regeneratorRuntime.mark(myGenerator);

function myGenerator(input) {
    return regeneratorRuntime.wrap(function myGenerator$(_context) {
        while (1) {
            switch (_context.prev = _context.next) {
                case 0:
                    console.log(input);
                case 1:
                    _context.next = 3;
                    return;
                case 3:
                    if (!(input = _context.sent)) {
                        _context.next = 7;
                        break;
                    }

                    console.log(input);
                    _context.next = 1;
                    break;
                case 7:
                    ;
                case 8:
                case "end":
                    return _context.stop();
            }
        }
    }, _marked, this);
}


var gen = myGenerator(1);

for (var _iterator = gen, _isArray = Array.isArray(_iterator), _i = 0, _iterator = _isArray ? _iterator : _iterator[Symbol.iterator]();;) {
    var _ref;

    if (_isArray) {
        if (_i >= _iterator.length) break;
        _ref = _iterator[_i++];
    } else {
        _i = _iterator.next();
        if (_i.done) break;
            _ref = _i.value;
        }
        var input = _ref;
    }
}

var script = document.createElement('script');
script.onload = function () {
    mainFunction();
};
script.src = "https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/7.2.5/polyfill.min.js";

document.head.appendChild(script);

We can pass values to the iterator, like this:

"use strict";

function mainFunction() {
    var _marked = /*#__PURE__*/regeneratorRuntime.mark(myGenerator);
    function myGenerator(input) {
        return regeneratorRuntime.wrap(function myGenerator$(_context) {
            while (1) {
                switch (_context.prev = _context.next) {
                    case 0:
                        console.log(input);
                    case 1:
                        _context.next = 3;
                        return;
                    case 3:
                        if (!(input = _context.sent)) {
                            _context.next = 7;
                            break;
                        }
                        console.log(input);
                        _context.next = 1;
                        break;
                    case 7:
                        ;
                    case 8:
                    case "end":
                        return _context.stop();
                }
            }
        }, _marked, this);
    }
    var gen = myGenerator(1);
    var index = 0;
    for (var _iterator = gen, _isArray = Array.isArray(_iterator), _i = 0, _iterator = _isArray ? _iterator : _iterator[Symbol.iterator]();;) {
        var _ref;
        if (_isArray) {
            if (_i >= _iterator.length) break;
            _ref = _iterator[_i++];
        } else {
            _i = _iterator.next(index = (index + 1) % 10); //passing parameters
            if (_i.done) break;
            _ref = _i.value;
        }
        var input = _ref;
    }
}


var script = document.createElement('script');
script.onload = function () {
    mainFunction();
};

script.src = "https://cdnjs.cloudflare.com/ajax/libs/babel-polyfill/7.2.5/polyfill.min.js";
document.head.appendChild(script);
//1 2 3 4 5 6 7 8 9

We can work with this, but we burden ourselves too much with:

having to care about the Polyfills
having to understand a bunch of unintuitive functions
having to write and maintain unreadable code

If we are not that masochistic, we will reach the conclusion that we need an alternative, a plan B here:

function *myGenerator(input) {
    console.log("start " + input);
    while((input = yield (input * 2)) < 10) {
        console.log("inner " + input);
    };
    return false;
}

var index = 1;
var gen = myGenerator(index = 1);
gen.next();
var tmp;
while(!(tmp = gen.next(++index)).done) console.log("tmp " + tmp.value);

Final notes

I could write a lot more about Kyle’s books, notably promises, generators and their combinations are very interesting. But frankly, I think the book about asynchronous work does an excellent job explaining those. I recommend Kyle Simpson’s books to everyone. I think we owe a big thank you to him and his effort to educate us. You can find his website at me.getify.com.

Work philosophy canon

2018-05-07T00:00:00+00:00

Shared culture

Having some shared culture is important for working together well, and we can build that culture on familiar terminology, understanding, experiences, stories, and ideas.

To help give all of us at End Point some common reference points, we have collected a set of valuable articles and books that we encourage everyone to read.

Some of these have been standard reading at End Point for more than a decade, while others have been added over the years since we began doing this. Some are short and simple, others more in-depth. Our list is intentionally general, mostly avoiding specific technologies that only a subset of us use.

No one article or book can be entirely authoritative for all situations, but each gives us more of the wisdom out there to consider and judiciously apply where it fits.

Slow and steady

When new employees start at End Point, we ask them to read the articles during their first week or two, and the relevant books within roughly their first year.

Reading only a little at each sitting and spreading the reading out over time allows the ideas to sink in gradually and be incorporated into our work, rather than overwhelming with new information that cannot all be absorbed at once.

For everyone

Because we work in the software development industry, it is important that not only technical people such as developers, database experts, and system administrators be part of the shared culture. Everyone else at End Point including designers, project managers, sales, marketing, etc. should also be familiar with these articles and the terms and concepts they discuss:

Speed matters: Why working quickly is more important than it seems by James Somers
Please don’t just say hello in chat originally by Brandon High at nohello.com
Learn to Read the Source, Luke by Jeff Atwood—practical reasons free software and open source matter, and why actually using the source code matters
What is free software? by the Free Software Foundation and Richard Stallman
The Open Source Definition by the Open Source Initiative
Josh’s Rules (of Database Contracting) by Josh Berkus
On excellence by Ethan Rowe—about ownership, taking responsibility for the problem at hand
Conventions to know by Jon Jensen—little but important things that ease communication in our work
Selling Without Being Salesy by Jason Parks
Five Whys by Eric Ries (covered in more depth in his book The Lean Startup)
Software Is a Mess and I Feel Fine by Alan Stevens
Choose Boring Technology by Dan McKinley
Getting Real book by 37signals (now Basecamp)

For developers

These articles and books are targeted primarily at developers but enlightening for everyone:

A summary of the international standard date and time notation by Markus Kuhn
The Admin Zen by Michael Prokop—in true DevOps fashion, this system administration advice applies to development too
Things You Should Never Do, Part I by Joel Spolsky—don’t rewrite things from scratch (almost never, anyway)
On Git Commit Messages by Michael Jones; to get a feel for the long history of others sermonizing on the same worthwhile topic, see also:
- How to Write a Git Commit Message by Chris Beams
- On commit messages by Peter Hutterer
- A Note About Git Commit Messages by Tim Pope
Every line of code is always documented by Mislav Marohnić; see also The git pickaxe by Philip Potter
How to Read Other People’s Code—and Why by Christopher Schanck
You don’t know JavaScript by Michael Woloszynowicz—the lessons apply to most technologies
The Pragmatic Programmer book by Andy Hunt and Dave Thomas (see the write-up on the new 20th anniversary edition by Jon Jensen)
Practices of an Agile Developer book by Venkat Subramaniam and Andy Hunt—kind of a sequel to The Pragmatic Programmer (see the review by Ethan Rowe)

Any others?

We are always on the lookout for other readings that are helpful, so feel free to leave a comment with your favorites!

(Updated in early 2021 with articles on Git commit messages and history spelunking, and Choose Boring Technology.)

Ruby Fight Club

2016-08-31T00:00:00+00:00

Photo by Peter Gordon

First Rule: Do Not Talk About Ruby Fight Club

This post may get me kicked out for talking about the club, but…I was asked to share a few thoughts about something we tried out earlier this year at End Point. We ran an internal meetup dubbed Ruby Fight Club to read and discuss Practical Object-Oriented Design in Ruby (POODR) by Sandi Metz.

In the past we’ve met together as a company and in smaller team or project focused groups but this was first time we used a book as the guiding, recurring topic with a smaller, committed group who made time for it. Attendance was optional and we did have some folks who opted out even though they loved the subject matter and the idea, because they knew they couldn’t make it most of the time. I think this made the group tighter-knit.

Our group of six remote engineers met weekly for one hour to discuss one chapter from the book. We worked through each chapter together which often led to Q&A sessions and deeper discussions about Ruby and computer science theory in general. Each week, one member of the team would lead the discussion. It required some preparation in advance and gave us all a chance to work on our presenting and moderating skills. We used Google Hangouts to meet and the presenter would share their screen to show slides and example code.

Honing Our Craft

As consultants we have to balance technical perfection with other considerations—most often budget and timelines/deadlines. This reality does not lend itself well to developing and learning to apply new technical concepts. It’s also challenging to apply these new concepts in large, established projects where our role is to maintain and extend existing code. The conventions and architecture are well established and it’s hard to push against projects with inertia like this. Setting some time aside each week not tied to client projects gave us the opportunity to focus on our skills and develop them together. Although several in the group had read the POODR book before, they reported that working through each chapter and discussing it in a group setting greatly increased their understanding of the material.

Face Time

Our company has a large number of full-time remote engineers. At time of this writing close to 90% of us work in a location other than our head office in NYC. Remote work allows us the freedom to live where we want (I’m writing this from my home office in a small town in the interior of British Columbia, Canada) but requires extra effort for us to build team camaraderie and social connections. Ruby Fight Club was great for this. Although Hangouts, Skype and other video-conferencing apps have their challenges, everyone let us know they appreciated being able to spend time with the group each week. We’d often chat with each other for a few minutes before and after our book discussions.

Discussing the book (somewhat) face-to-face over video also increased the fidelity and efficiency of our discussions. When you can see people’s faces, expressions and gestures it’s much easier to glean information and understand them. It feels silly and obvious to say this after the fact but it’s novel when the majority of your workday communication is done over IRC/Slack, email and phone/audio.

Got Meetups?

During one of our meetings I asked the group if anyone went to meetups local to them. I was surprised to learn that none in the group did so. For every person the response was similar: “I would but there aren’t any groups meeting nearby”. I had assumed that I was the odd one out in this way because I live in a small town. The leader of our group lives in Chicago and I thought for sure there would be a meetup for him to attend but there wasn’t. The closest group to him required a long trip across town on public transit. Occasionally I do travel 45-60 minutes (each way!) from home to attend meetups in larger, nearby towns. People are often surprised I travel so far but it’s worth it to me.

What Worked Well

We stuck to the time limit (mostly) which was a good thing. We found we usually had enough time to socialize a bit, discuss the book, have a Q&A time and then get back to our client work. Sharing the presenting/discussion leading role was also nice. Leading a session took a little time to prepare but because we all took turns doing this we shared the load.

I’d like to personally thank Phunk (our fearless Ruby Fight Club leader) for having the idea and executing so well on it. We all really appreciate it and look forward to running more meetups like it in future.

What We Might Do Differently Next Time

I think it would be good to allow a little more time for socializing and chat. This happened naturally in Ruby Fight Club but it would be cool to devote a block of time specifically to it.

It would also be nice to allow time for people to do a show and tell/lightning talk about something they are interested in. That said, having a book to work through was really great. Every week there is something to discuss and it was often a starting point to further discussions.

What are you doing at your companies to stay sharp and connected? Let us know, we’d love to hear from you!

Book review: “Two Scoops of Django: Best Practices for Django 1.8”

2016-07-13T00:00:00+00:00

Two Scoops of Django: Best Practices for Django 1.8 is, shockingly, a book about best practices. It’s not a Django library reference, or a book about Django fundamentals, or tips and tricks. It’s a book designed to help web developers, novices and experts alike, avoid common pitfalls in every stage of the web development process, specifically the process of developing with Django.

The book can be used as a reference of best practices and a cover-to-cover guide to best practices. I’ve done both and found it to be enjoyable, accessible, and educational when read cover-to-cover and a valuable reference when setting up a new Django project or doing general Django development. It covers a huge range of material, answering questions like:

Where should I store secret keys?
Should I use virtualenv?
How should I structure my Django project?
When should I use ‘blank’ and ‘null’ in model fields?
Should I use Class-Based Views or Function-Based Views?
How should I structure my URLConfs?
When should I use Forms?
Where should templates be stored?
Should I write custom template tags and filters?
What package should I use to create a REST API?
What core components should I replace?
How can I modify or replace the User and authentication system?
How should I test my app?
How can I improve performance?
How can I keep my app secure?
How do I properly use the logging library?
How do I deploy to a Platform as a Service or my own server(s)?
What can I do to improve the debugging process?
What are some good third-party packages?
Where can I find good documentation and tutorials?
Where should I go to ask more questions?

The question, then, is whether or not this book delivers this information well. For the most part, it does. It’s important to recognize that the book doesn’t cover any of these subjects in great detail, but it does do a great job explaining the “why” behind some of the simple rules that are established and referencing online resources that can be used to go much more in-depth with the subject. It does a great job showing clearly marked bad examples, making it very easy to see whether or not you are or were planning on doing something badly. The writing style is very accessible and straightforward; I read large portions of the book during breakfast or lunch.

Two sections stood out to me as being very helpful for my own projects. First is Chapter 4, Fundamentals of Django App Design, which explained better than any resource I’ve found yet exactly what a Django “app” (as in ./manage.py startapp polls) should be used for. It explains what an app should or shouldn’t have in it, how much an app should do, when you should break into separate apps, and more.

The next section that really helped me was Chapter 6, Model Best Practices, which explained things like what code should and shouldn’t be in a model, how to use migrations and managers, and ModelFields that should be avoided. Perhaps the most useful part of that chapter is a table in the section “When to use Null and Blank,” which makes for an easy and quick reference to which fields go well with the null and blank parameters and when you should use both or neither.

The only real problem I had with Two Scoops of Django was that the illustrations rarely felt necessary or helpful. The majority of them are ice cream-themed jokes that aren’t particularly funny. Overall, I really enjoyed this book and I definitely recommend it for anybody who is or is interested in doing serious Django development.

Usability: Don’t Make Me Think and a Bookmarklet

2015-08-13T00:00:00+00:00

Hi! Steph here, former long-time End Point employee now blogging from afar as a software developer for Pinhole Press. While I’m no longer an employee of End Point, I’m happy to blog and share here.

I’m only about a quarter of the way into Don’t Make Me Think (Revised) by Steve Krug, but I can already tell it’s a winner. It’s a great (and quick) book about web usability, with both high level concepts and nitty gritty examples. I highly recommend it! Even if you aren’t interested in web usability but are a web developer, it’s still a quick read and would be invaluable to whomever you are coding for these days.

A Bookmarklet

The book inspired me to write a quick bookmarklet to demonstrate some high level concepts related to usability. Here’s the bookmarklet:

javascript:(function() {
  var possible = " ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
  $('*:not(iframe)').contents().filter(function() {
     return this.nodeType == Node.TEXT_NODE && this.nodeValue.trim() != '';
  }).each(function() {
     var new_content = '';
     for(var i = 0; i<this.nodeValue.trim().length; i++) {
       new_content += possible.charAt(Math.floor(Math.random() * possible.length));
     }
     this.nodeValue = new_content;
  });
})();

To add the bookmarklet to your browser, simply copy the code in as the location for a new bookmark (and name it anything you want). Note that this particular bookmarklet assumes jQuery is installed, so it may not work on all websites. Gist available here

What does it do?

In short, the bookmarklet converts readable text on the page to jibberish (random characters of the same length). Pictures are worth a thousand words here. Here are some example pages with the bookmarklet in action:

End Point home page.

End Point client list page.

Stance popup, with item in cart.

Backcountry.com product listing page.

CityPASS home page.

Why does this matter?

The bookmarklet provokes thought related to high level usability concepts, such as:

Is it clear which buttons are clickable?
Is the visual hierarchy clear?
What conventions does the user interface follow?
Users browsing behavior is often to hyperfocus and click on what they are looking for while ignoring other content entirely. Does the user interface aid or hinder that behavior?
How and what do images communicate on the page?

All of these ideas are great things to talk through when implementing user interface changes or rolling out an entirely new website. And if you are interested in learning more, visit Steve Krug’s website.

A review of The Rails 4 Way

2013-11-27T00:00:00+00:00

With a brand new project on the horizon (and a good candidate for Ruby on Rails 4 at that), I thought it would be a productive use of my time to bone up on some of the major differences introduced with this new version. My goal was to trade a little research for as many precluded that’s-not-how-we-do-it-anymore-500-errors as I could when fleshing out my first scaffold.

With this goal in mind, I purchased a copy of The Rails 4 Way by Obie Fernandez, Kevin Faustino, and Vitaly Kushner. Considering the free-nature of everything else Rails related, I will admit to a slight aversion to paying money for a Rails book. For those of you out there with similar proclivities, I felt compelled to share my experience.

The Rails 4 Way presents itself not as a tutorial of Ruby on Rails, but as “a day-to-day reference for the full-time Rails developer.” The tone of the book and the depth of information presented hints that the authors don’t want to teach you how to write Ruby on Rails applications. They want to share insight that will help you write better Ruby on Rails applications. Much of this is accomplished by fleshing out—utilizing a plentiful source of examples and snippets—the methods and features the framework has to offer.

Early on the authors concede the point, “There are definitely sections of the text that experienced Rails developer will gloss over. However, I believe that there is new knowledge and inspiration in every chapter, for all skill levels.” I would classify myself as an experienced Rails developer and I found this statement to be completely true. When settling down to read the book for the first time, I scrolled through the pages until a specific topic caught my eye. As a result of this, I started my read part-way through chapter 4. Upon completion of the last section, I wrapped back around to the beginning and dutifully read everything including the foreword. I did this specifically because each of the chapters, if not sections, had contained at least one little nugget of information that I found valuable. I felt compelled to go back and look for the extra tidbits that I’d passed over with the initial, casual flicks of my mouse.

I mentioned that the authors focus more on helping you produce better code, and to this end, they don’t dance around any issues. My favorite quote from the book illustrates this, “Frankly, there’s not much of a reason why that should be a problem unless you’ve made some pretty bad data-modeling decisions.” By all means, call it like it is.

The book is still a work-in-progress. Somewhere around 10% of the sections contain “TBD” rather than a body. I gather that the book is based on its Rails 3 counterpart and some of the sections have yet to be enhanced with the Rails-4-specific content. A majority share of the Rails 3 content is still applicable, so this shortcoming did not mar my experience at all. Especially since I will be given access to the new content once it is added. At the time of my download, the last change was authored 12 days ago, so for that reason also, I am unconcerned.

In conclusion, I took enough new information away, and enjoyed the process enough that I feel confident in recommending The Rails 4 Way to friends and coworkers. The bulk of this book reads like an API document, and in those sections, there’s nothing technical that you can’t find on the Rails API page itself. What the book does offer is back-story into what these methods were designed to do and what they pair well with. I would compare the Rails API and this book to a map in your hand and a local who knows the area. Most of the time you can figure out how to get where you need to go by carefully reading your unfolded map. But sometimes it’s faster to ask a local who can tell you, “See that red sign off in the distance? Head over to it and take a right.”

Book Recommendation: Ghost in the Wires

2011-11-29T00:00:00+00:00

I recently listened to Ghost in the Wires by Kevin Mitnick as an audiobook during my long Thanksgiving vacation drives. This non-fiction book is a first-person account about Kevin Mitnick’s phone and computer break-in (or what he claims to be ethical hacking) adventures in the late eighties and early nineties, and it touches on the following legal proceedings from 1995 on. A couple of interesting things stood out to me:

Kevin’s tactics revolve around social engineering, or techniques that capitalize on flaws in “human hardware” to gain information. The book was an eye opener in terms of how easily Kevin gained access to systems, as there are countless examples of Kevin’s ability to gain credibility, pretext, introduce diversions, etc.
Another highlight of the book for me was learning details of how bug reports were exploited to gain sensitive information. Kevin gained access to bug reports on proprietary software to exploit the software and gain access to the systems running the software. I don’t think of my own clients’ bug reports as an extremely valuable source of information for exploiting vulnerabilities to gain user information, but there have been a few instances in the past where bugs could have been used maliciously.

Follow-up Comments

One thing that strikes me is how the internet and technology has changed since Kevin’s infringements, specifically in the development of open source software. End Point works with open source operating systems, packages, monolithic ecommerce applications, and modular open source elements (e.g. Rails gems, CPAN modules). Bug reports on open source applications are easily accessible. For example, here is an article on the security vulnerabilities in recent versions of Rails.

The responsibility of keeping up with security updates shifts to the website owner leveraging these open source solutions (or the hosting provider and/or developer in some cases). I spoke with a few developers a couple of years ago about how public WordPress security vulnerabilities enable unethical hackers to easily gain access to sites running WordPress without the security updates. With the increased popularity of open source and visibility of security vulnerabilities, it’s important to keep up with security updates, especially those which might make sensitive user information available.

With the advancement in technology, security processes should become a normal part of development. For example, End Point has standard security processes in place such as use of ssh keys, firewalls for server access, and PGP encryption. Our clients also follow PCI compliance regulations regarding storing credit card numbers and security numbers in encrypted form only, or in some cases not at all if a third party payment processor is used. It’s nice to use a third party service for storing credit card data since the responsibility of storing sensitive cardholder data shifts to the third party (however, the interaction between your site and the third party must be protected).

This is an interesting read (or listen) that I recommend to anyone working with sensitive information in the tech field. Learning about the social engineering techniques was fascinating in itself and technical bits are scattered throughout the book which make it suitable for tech-savvy and non-tech-savvy readers.

Pentaho Reporting 3.5 for Java Developers

2009-10-28T00:00:00+00:00

I was recently asked to review a copy of Will Gorman’s Pentaho Reporting 3.5 for Java Developers, and came to a few important realizations. Principal among these is that my idea of what “reporting” means includes far too little. Reporting includes much more than just creating a document with some graphs on it—any document or presentation including information from a “data source” comprises a “report”. This includes the typical sheaf of boardroom presentation slides, but also includes dashboards within an application, newsletters, or even form letters. I recently discovered that a local church group uses a simple reporting application to print its membership directory. In short, it’s not just the analysts and managers that can use reporting.

The book gives the reader a tour of Pentaho’s newest Pentaho Reporting system, which consists of a desktop application where users define reports, and a library by which developers can integrate those reports into their own applications. So as an example, not only can Pentaho Reporting publish weekly sales printouts, but it can also produce real-time inventory information in a J2EE-based web application or even a Swing application running on a user’s desktop. Gorman describes, step-by-step, the process of building a report, integrating advanced data sources, working with graphics and visualizations, and building custom reporting components such as user-defined functions and custom report controls. Although this step-by-step description combined with Java’s native verbosity occasionally make it difficult to read cover to cover, it provides a valuable reference and starting point for users wanting to implement advanced reporting features.

Shortly after having grasped the idea that “reporting”, as a concept, was broader than I’d originally assumed, I got the idea that I’d like to know how to use Pentaho to replace a reporting system I worked on a while ago, which used Jython to gather data from a JMX server. Pentaho Reporting allows users to create data sources in languages other than Java, via the Apache Bean Scripting Framework, suggesting that my old Jython code might work mostly unchanged via Bean Scripting. Disappointingly, the book doesn’t give an example of this technique, but since the Bean Scripting integration is still considered experimental (along with, apparently, several other data source types, contributed by the Pentaho community). Perhaps I’ll figure this out one day, and make it the subject of a future blog post.

This fairly slight omission notwithstanding, I enjoyed the book and the ideas it suggested about places I might apply more interesting reporting. Thanks to PacktPub for the opportunity to review it.

David Mamet on software development

2009-02-07T00:00:00+00:00

I recently read the book True and False: Heresy and Common Sense for the Actor (1997) by David Mamet. (A gift from Uncle Steve. Thanks!) Mamet’s film The Spanish Prisoner is one of my favorites, and I liked Oleanna, Things Change, and Glengarry Glen Ross a lot too. Wait, looking over his writing credits at IMDb, I see I’d forgotten The Winslow Boy, State and Main, and Wag the Dog, which were good too.

Enough of the fanboyism. This little book of his is packed with what sounds like good advice for actors. Not having been an actor in anything more than a school play or two, I can only imagine, but I was surprised to find myself nodding in agreement during many passages because they applied so well to my field, software development.

I’ve selected a few quotations that seemed especially apt:

I was once at a marriage ceremony where the parties swore “to try to be faithful, to try to be considerate …” That marriage was, of course, doomed. Any worthwhile goal is difficult to accomplish. To say of it “I’ll try” is to excuse oneself in advance. Those who respond to our requests with “I’ll try” intend to deny us, and call on us to join in the hypocrisy—as if there were some merit in intending anything other than accomplishment. (p. 34)

This reminded me of Richard Stallman’s take on the same subject:

Yoda’s philosophy (“There is no ‘try’”) sounds neat, but it doesn’t work for me. I have done most of my work while anxious about whether I could do the job, and unsure that it would be enough to achieve the goal if I did. But I tried anyway, because there was no one but me between the enemy and my city. Surprising myself, I have sometimes succeeded. (from the essay, The GNU Project)

Though they appear to be contradictory, I think both approaches can apply depending on circumstances and I’ve employed each at various times. Whoa, there … nuance.

I don’t have comments about the rest of these excerpts:

The simple performance of the great deed, onstage or off, is called “heroism.” The person who will not be swayed, who perseveres no matter what—that hero has the capacity to inspire us, to suggest that we reexamine our self-imposed limitations and try again. (p. 13)

The middle-class work ethic: “But I did my preparation. It is not my fault if the truth of the moment does not conform.” That ethic is not going to avail. Nobody cares how hard you worked. Nor should they.

Acting, which takes places for an audience, is not as the academic model would have us believe. It is not a test. It is an art, and it requires not tidiness, not paint-by-numbers intellectuality, but immediacy and courage. (p. 32)

Let us learn acceptance. This is one of the greatest tools an actor can have. The capacity to accept: to wish things to happen as they do. It is the root of all happiness in life, and it is the root of wisdom for an actor. Acceptance. Because the capacity to accept derives from the will and the will is the source of character. Applying our intention to use only one meaning for words, character is the same onstage and off. It is habitual action. …

The habit of cheerful acceptance is an aide in the greater life in the theatre, too, because it induces truthful consideration: “The world is as it is, what can I do about it?” … (pp. 70–71)

Carve the big tasks up into small tasks and perform these small tasks. (p. 76)

The more you are concerned with yourself, the less you are worthy of note.

The more a person’s concentration is outward, the more naturally interesting that person becomes. …

The person with attention directed outward becomes various and provocative. The person endeavoring to become various and provocative is stolid and unmoving. (p. 95)

“Luck,” in one’s business dealings, and “talent,” its equivalent onstage, seem to reward those with an active and practicable philosophy. … [H]ard work and perseverence will be rewarded. (p. 99)

In the theatre, as in other endeavors, correctness in the small is the key to correctness in the large. (p. 101)

Be generous to others. … There is certainly something you can correct or improve in yourself today—over which you have control. That habit will make you strong. Yearning to correct or amend something in someone else will make you petty. …

Cultivate the habit of mutuality. Create with your peers, and you are building a true theatre. When you desire and strive to rise from the ranks rather than with the ranks, you are creating divisiveness and loneliness in yourself, in the theatre, and in the world. (pp. 102–103)

It is not a sign of ignorance not to know the answers. But there is great merit in facing the questions. (p. 127)

That’s enough for now, but there’s much more in the book that I found useful and applicable to other areas of life as well. It’s a quick read and I really enjoyed it.

The Orange Code

2009-01-15T00:00:00+00:00

I’ve been reading the new book The Orange Code, the story of ING Direct by Arkadi Kuhlmann and Bruce Philp. Here are a few passages I liked from what I read today:

The commitment to constantly learn is the only fair way to bring everyone in the company under the same umbrella. It is a leveler. (p. 213)

… [W]e’ve got to earn it each day, and we need to feel that we have new challenges that can make us or break us every day. … Each day’s work will last only as long as it’s relevant. … [W]e did okay in each of the last seven years, but we are only ever as good as our last year, our last day, our last transaction. We still have a lot to do, since our competition is not resting. (pp. 208–209)

Trust and faith not only are built over time, but they actually need the passage of time to validate them. (p. 197)

Contributing is a privilege earned, not a right. And there are, indeed, bad ideas, most of which are answers to questions the contributors didn’t really understand in the first place. There is a reason why some of the world’s finest jazz musicians were classically trained: You have to understand the rules before you can intelligently improvise on them. (p. 195)

Review of The Book of IMAP

2008-08-26T00:00:00+00:00

Former End Point employee Ryan Masters had his book review of the No Starch Press published over at OSNews.com. Sounds like it was a decent book!

New edition of The Book of JavaScript reviewed

2007-02-21T00:00:00+00:00

The Book of JavaScript (2nd edition) is a new and comprehensive introduction to the JavaScript language presented in an entertaining, practical format. I have significant practical experience with JavaScript, so I do not consider myself in the target audience for this book; however, I still found much of it useful so it will remain as a valuable reference on my bookshelf.

My full review of the book was just published at OS News.

Review: Practices of an Agile Developer

2006-06-20T00:00:00+00:00

Practices of an Agile Developer by Venkat Subramaniam and Andy Hunt (Pragmatic Bookshelf, 2006)

Practices of an Agile Developer is a fast, enjoyable, often-amusing read. It’s also full of wisdom and excellent advice for engineers of all levels, technical managers, and just about anyone with an interest in the software development process. The book makes for a great companion to The Pragmatic Programmer (also co-authored by Andy Hunt), but is more concise and focuses on bigger-picture development practices than does the often-implementation-oriented latter. The authors clearly outline a number of agile methodologies, with attention given to tasks ranging from managing teams, managing time, and keeping estimates realistic, to integration automation, release management, and test-oriented development.

Of particular interest are topics that relate to developer/client interaction, which could apply to a variety of professional relationships (consultant and client; development team and sales team; etc.). The practices outlined present a path to greater success for all parties, with a greater sense of ownership for the client and a greater sense of satisfaction for the developer. Sounds pretty good, doesn’t it?

A common scenario in developer/client requirements gathering and project planning involves the production of elaborate specifications of enormous scope; the client is supposed to approve the requirements, effectively saying “yes, this represents everything I want, the way I want it”, while the developer must then both estimate the time for completion and, assuming the estimate is accepted, move forward with implementation. In a consulting situation, the client may require a fixed price for the work, rather than just an estimate. This scenario seems perfectly rational, as the requirements represent a point of consensus around which work will be done, bills paid, and so on.

Or so it may seem, until one has been down that road.

Software development is really the business of change; the authors take great pains to illustrate this repeatedly, and they make their point effectively. While requirements and project scope are established, neither the client nor the developer possesses perfect information or a complete understanding of the possibilities. Usability studies may reveal a better way to approach certain problems; market research may indicate more profitable ways of selling goods; any such discovery may necessitate a requirements change on the client’s part. Furthermore, during project development, all parties may observe new possibilities implicit in the growing system – possibilities that would never occur to developer or client without the benefit of actual working examples. Given tightly-specified requirements and strict budgetary constraints, these new discoveries and possibilities cannot be pursued or easily integrated into the project. Software, and its theoretical ability to accomodate real-world change, is stifled by this development approach.

The Agile Developer authors instead favor a model that could be described as a “feedback loop”; the development team works in short, manageable iterations lasting between a few days to perhaps a couple of weeks, and at the end of each iteration provides tangible deliverables to the client. This provides the client with the opportunity to review the results and play a part in steering where the next iteration will go. While a master set of goals or desirables may guide the entire process, the individual iteration is the largest chunk of development work ever pursued at any one time; estimates can be based on each iteration, and each successful iteration brings the software closer to the final goals. Most critically, the process allows all parties to account for change as new information and experience inevitably brings new desires and ideas.

Such an approach fits nicely into other development methodologies advocated by the authors, guided by axioms like “keep it releasable” (indicating that a developer should never go home for the day leaving code in a broken state), “integrate early; integrate often” (effectively requiring developers to work in small chunks that continuously get merged back into the main project code base), and a variety of suggestions regarding test-driven development practices (develop test cases before implementing actual systems as a means of establishing implementation/interface requirements, automate execution of test cases as part of the continuous build process, treat failed tests as bugs, etc.).

Ultimately, Practices of an Agile Developer illustrates just how subject to change all aspects of the software development process can be, and offers compelling ways to build the management of change into your systems and methodologies. Such methods become all the more valuable as companies and individuals increasingly look to the web as a means of delivering a service, and services increasingly appear as unique web-based applications which need to organically grow with their user community, rather than in discrete, versioned releases to be consumed and re-consumed by users over time. Check out this book if you haven’t already; it highlights a great approach to which anyone involved in the software development cycle can aspire.