The idea of this book is to provide a practical guide on how to implement DDD, CQRS and Event Sourcing. I chose to use JavaScript and Node.js as alternative to Java or C#, which are often used in comparable literature. After reading it, one should understand the fundamental concepts, how to apply them and how they can work in code.
You can access a free sample of the book on the webpage in different formats.
I worked on the book for four years as a side project. In 2016, I had worked in three different projects that applied either some or all of the concepts. The last one was the main motivating factor for the book. It was a startup that was settled on DDD, CQRS and Event Sourcing, even though the developers had limited knowledge.
Together, we learned, experimented and applied as much as we could. We read "Domain-Driven Design" by Eric Evans, "Implementing Domain-Driven Design" by Vaughn Vernon and whatever we could find from Greg Young. With the knowledge we picked up and the experimentation we did, we felt in a good position. Still, there were many unanswered questions on how to put everything into code.
In 2016, before leaving the startup, I first thought about writing a book. Effectively, I wanted to create the guide the startup would have needed in the beginning. When I started, I was overly confident about my knowledge. However, throughout the years, I learned many details. Today, I know that I could not have written it in 2016 at once.
The final book is different from what I envisioned it to be at first. My initial goal was to write a short book with approximately 150 pages. The published version has now over 450 pages. However, I consider this a good thing. The book provides enough context to be read without any prior knowledge in DDD, CQRS or Event Sourcing.
I'd be happy to discuss your thoughts on and experiences with DDD, CQRS and Event Sourcing. Regardless whether it is about their combination, their sometimes counterproductive conflation or one of the concepts. Finally, if you have any input or feedback for me, it would be highly appreciated!
My experience is that most supposed implementations of DDD are problematic because the focus is on the infrastructure, plumbing and implementation, rather than on the real challenge which is modeling a complex problem domain.
This is the exact opposite of Evan’s intention as stated in his book.
Because there is a general inability to model a complex system into an object model as is the basis of his book, focus falls onto programming constructs and things like “ubiquitous language” which pre-dates Evans and is self evident to most.
About 80% of CQRS/event sourcing projects I review or consult on are unsuccessful. In most cases this is primarily due to them being applied where they shouldn’t be, by (inexperienced) teams viewing these techniques as “silver bullets” to success.
I should note that I am mainly involved with line-of-business type systems, rather that projects focusing on computing infrastructure and the computing and data sciences, and therefore speak to the former.
It is important to note that my book is not an alternative to "Domain-Driven Design" by Eric Evans. Rather, it is an introduction to DDD, an explanation of CQRS and Event Sourcing and an implementation guide. Also, it is not explaining how to approach and model complex domain problems. I wouldn't consider myself experienced in that. Everything I worked with so far was rather trivial.
I can very much relate to your experience with projects that applied CQRS and Event Sourcing. In fact, the longest project I worked on was a negative example. The patterns were used as part of the system architecture. Their use was never justified with any reason specific to the domain or the business. In the end, some parts of the system would have been better off with a completely different architecture.
I already have convinced project teams to not use Event Sourcing when they thought they needed it. Also, I often hear it being conflated with Event-Driven Architecture.
I love this comment. I’ve definitely witnessed DDD terminology used as a lofty sounding justification for sexy tech, rather than a genuine attempt to model a complex domain.
I think that software developers too often underinvest in modeling a domain in favor of propping up an imagined silver bullet.
I wonder if there’s an easy way to hire for good behavior?
This is interesting to me as someone who loves DDD but finds the tactical side hard to implement in Node.
A few questions stand out:
AFAICT implementing tactical DDD involves a lot of boilerplate code. That seems to be consistent with what you've written in your article on writing the book, mentioning "...we even built a Node.js framework as byproduct." If tackling DDD in Node requires so much work on something that isn't the businesses' core domain, and there's a lot of surface area for it to go wrong, why do it in Node?
As someone who writes predominantly TypeScript, static types feel like a much lower hanging fruit for alleviating some of the pain in tackling rich domains. As you've written the book in JavaScript, I'm curious whether you used plain JavaScript on the professional projects?
Also curious to hear from someone who has done tactical DDD in Java or C#: do you find a good supporting framework essential?
Could you explain which aspects specifically require a lot of boilerplate code? Judging from my experience, I think the tactical DDD patterns are straight forward to implement. I didn't mention this in my post, but the Node.js framework was only concerned with CQRS & Event Sourcing.
I fully agree with you that (static) types are inevitable for tackling rich/complex domains adequately. Personally, I would often recommend the use of TypeScript over plain JavaScript.
As for the book, when I started working on it, I was using plain JavaScript. Over the course of the past four years, there were times, when I considered to rewrite the DDD parts with TypeScript. My current plan is to add an appendix or integrate additional content about (static) typing.
While types are an important aspect for the Domain layer, I think the rest of the book works fine without TypeScript. Its absence might even help to keep the code example more concise.
> I'm curious whether you used plain JavaScript on the professional projects?
Yes, in both relevant projects we were using plain JavaScript (or CoffeeScript). We had many runtime type checks and a high test coverage to overcome the lack of static types. TypeScript would have definitely been the better choice.
> Could you explain which aspects specifically require a lot of boilerplate code?
Probably the most frustrating part for me is tackling domain events and the implementation of the aggregate root. How have you tackled broadcasting domain events when an aggregate changes in a way that might be of interest to other bounded contexts?
Without Event Sourcing, I would say there is almost no boilerplate code for aggregates themselves or their root Entities. After all, an Entity can be implemented as plain old class or object. For event-sourced aggregates, it depends on the implementation style. With OOP, there might the typical AggregateRoot base class. With a more functional style, there can even be less of boilerplate code.
About the Domain Event publishing, I'm afraid I don't understand the question. What challenges did you face? Is it about event-sourced aggregates?
If you have an aggregate root, it is responsible for knowing when changes to the aggregate have occurred, and thus making those visible.
What is the mechanism for it to do so? Does the aggregate root retain the events and decide when to dispatch them (likely after successful persistence to a datastore)?
The Aggregate Root must track all Domain Events that occur upon executing an action and make them accessible somehow. Only after the successful persistence of the associated state change (and/or the events), they can be published via an Event Bus. The Aggregate itself only expresses that Domain Events occurred, but does not deal with publishing.
The article from Udi Dahan shows Domain Event handlers that "will be run on the same thread within the same transaction". This implementation is not suitable for handlers that affect other transactions. Udi explains that "you should avoid performing any blocking activities, like using SMTP or web services. Instead, prefer using one-way messaging to communicate to something else which does those blocking activities." What he is referring to as "one-way messaging" is the actual event publishing in my opinion and must guarantee event delivery.
In my book, the example implementations store all occurred Domain Events together with the Aggregate state. This is because it resembles the later use of Event Sourcing. There is a separate component that watches for changes in Aggregate data and publishes new Domain Events. After publishing, the events are marked accordingly.
Regardless of where newly occurred Domain Events are retained, it should be persistent. Many times, the events are stored in a separate table inside the same store as the affected Aggregate. Later, they are retrieved, published and either marked or deleted. This is called a Transactional Outbox: https://microservices.io/patterns/data/transactional-outbox.... The approach ensures that both the Aggregate change and the request to publish an event happen within the same transaction. The actual publishing happens in a separate one. This way, you get guaranteed event delivery or more specifically "at least once" delivery.
I only heard of Nest.js a few weeks ago and haven't had the chance to look it. Therefore, take my reply with a grain of salt.
Libraries and frameworks can help you with different things, but they can also get in your way. If the CQRS module of Nest.js seems useful for a specific scenario, I would give it a try.
CQRS can be implemented in many different ways. Even more, a lot of software is applying this pattern inherently to some extent.
If you separate a software (or a part of it) into a write side and a read side, the two become independent. They both only interact with the synchronization mechanism. Therefore, they are free to use their own set of patterns, libraries and even technologies. Many times, the two parts are operated as individual programs/services and can scale autonomously.
Personally, I never encountered a situation with implementing CQRS where I really would have needed a framework. In fact, I contributed to a Node.js framework for implementing CQRS and Event Sourcing. In hindsight, I think the framework was a mistake and should not even be used.
My book explains how to implement all the concepts, patterns and mechanisms without any third-party code. The goal is not to use the custom code in production, but to understand how everything works.
Sorry, there is no printed version at this point. I wanted to focus on the e-book variant first.
While this is not necessarily an argument against a printed version, note that this book works best when reading on a computer. It comes with the full source code bundle and a small utility that allows to execute all code examples interactively.
Also, it is likely that there will be updates to the book in the near future. I'm already considering to write an appendix. By exclusively providing an e-book, iterative improvements are easier to do and the readers get all updates for free.
Nevertheless, I may consider to offer a printed version later on.
This is an interesting thought. The interactivity and the implementation are actually as important or possibly more important than the book text.
This could lead to a dramatic shift in how technical books are created.
Imagine a new website that didn't offer videos or text or tutorials, but actually was integrated with all of those things together?
So in your case, the book can be read or listened to, as you work on the example implementations. Using a little ML, you could probably even offer suggestions to roadblocks.
Just got sidetracked on this thought.
I'm a DDD guy, but have never personally implemented CQRS and event streams. I love the idea of separate read and write streams, but what is the best mesh of tech to do it well? (rhetorical question)
It's a good idea to work on the e-book first, iterate and fix most of the issues. However, a printed version would be great (for example, I'm unable to focus while reading on the computer). +1 for the printed version!
I'm in the midst of an event driven microservice project. On our first try, we've created a distributed monolith.
That's why we decided to rewrite it in a more DDD driven way.
I've started reading the blue book, the red book and Scott Millett's "Patterns and Principles of DDD".
But the most helpful book I've found so far was "Domain-Driven Design in PHP", which I find a lot easier to grasp thanks to its hands on approach.
That said, as a Go/Javascript person I really appreciate another book on the topic. I appreciate the effort and I am going to buy it.
This is really the big problem with DDD microservices. Whilst I'm a huge fan of the two, my go to is to always start out with a monolith and enforce boundraries within the application that could potentially become microservices at somepoint in the future. Even go as far as to use an in-memory event-driven system in the monolith. Then and only then when the need arises do we pull it out into it's own microservice.
This has various benefits but the biggest one being you get to play with the design of the 'microservices' before actually committing to making them microservices allowing you to make better informed decisions. The harder part is that it does require stronger discipline and if not done correctly, a harder transition to microservices (e.g. pulling out the components)
I think this is a great recommendation for any new project that is considering to use Microservices. I like the approach of an in-memory event-driven system. This helps to enforce a message-only relationship between different conceptual parts from the start.
In fact, I did the same in one project. Also, while I am not covering Microservices in my book, the Sample Application uses a similar approach. The different contexts integrate via event communication. At first, this is an in-memory implementation and everything is running in a single process. Later on, it is replaced with a filesystem-based variant and the individual contexts are operated as individual programs.
How do you determine when "the need arises?" We've typically used bottlenecks, e.g. one piece of the app could benefit from additional scale while others wouldn't, and gone from there, but I feel like there's a better or more systematic approach that we're missing.
Do you use c# ? I let only application layers talk to each other for now. Also, i have some issues finding a decent saga to use that stores it's state ( eg. Something like the library Mass transit)
This is great approach. Implementing well-designed horizontally-scalable monolith app usually leads to keeping it monolithic for years, saving enormous amount of time, money and effort, keeping devs team compact and agile.
Good question. It's an IOT project with nodes that need to work in an offline situation. The decision was made before I joined the team and there was nothing I could do about it.
In the end, it doesn't really matter. Both approaches are legit, if done right. The real problem is that we produced a distributed monolith due to our inexperience. That's what we are trying to fix now.
In one project, where we could have done otherwise, we went consciously full Event Sourcing. In another project, the Domain Model was inherently event-based. In my book, the Sample Application first persists domain objects and is later refactored towards Event Sourcing.
While I read about the possibility of doing both at the same time, I never encountered this practice in a project. How I see it, there is always only one source of truth. If you apply Event Sourcing, it is the event log. Any other representation should be seen as derivation/projection.
The one long-term project where we applied Event Sourcing showed quite some challenges you can face with this pattern. For me, the most challenging aspect is the versioning of an event-sourced system. When the Domain Model evolves, it might necessitate changes to events or the introduction of new ones. Greg Young even wrote a book on this topic: https://leanpub.com/esversioning (I haven't read it though).
Please note that I have limited experience with domains where Event Sourcing is without a doubt the weapon of choice. Hope my input still helps. Maybe there are other people that can give more qualified input on that. In the end, I am not trying to advocate for the pattern.
The project was about building a SaaS for structured meetings, both for on-site and remote use. It involved topical areas such as collaboration, moderation and presentation. The model defined that every meeting consists of six steps, where each is facilitated with one specific method (out of multiple). There were also additional functionalities such as a collaborative whiteboard and audio/video communication.
I briefly mentioned this in another comment: With regard to software architecture and pattern use, the project is a negative example. CQRS and Event Sourcing was prescribed on a system level. Regardless of conceptual/technological boundaries, every part made use of it. This decision was by no means reinforced with domain- or business-specific arguments. However, for us as developers, it was a great learning opportunity.
In terms of significant benefits for specific contexts, I can only think of the whiteboard part. However, even for that, it is about a feature we never got to implement. At some point, we wanted to add a "Undo" functionality. With Event Sourcing, you can simply append events that represent the reversal of previously persisted state transitions.
Independent of domain-specific aspects, I experienced three universal benefits. I hate to say it, but one was in fact reporting. Another benefit is the increased possibility to solve concurrency conflicts in a domain-specific way. Third, an event-sourced service is an ideal foundation for sending notifications, such as when publishing Domain Events. In the project, it helped us to build a real-time reactive User Interface. Of course, I'm not saying you necessarily need Event Sourcing for that.
Just browsed the table of contents and was both happy and surprised to see an entire chapter dedicated to the UI.
How big is the sample application in the book? I've read a lot about DDD but things just don't seem to click for me, partly because example are trivial. For example, I read "Architecture Patterns with Python" (https://www.cosmicpython.com/book/preface.html) but just didn't find it practical. Some random thoughts I had while reading, in no particular order:
* what if domain logic can be optimized in the database?
* how do joins across multiple tables work?
* in Django I've come across plenty of examples where you have property method on a model that goes and performs additional queries. For instance, something along the lines of `my_post.get_latest_comment()`. Should this instead be somewhere in the service layer?
Thanks for the feedback. Happy to hear that you appreciate the UI chapter. For me, it was important to explain how to approach the User Interface. I often had the feeling that this part was a bit neglected when reading about CQRS & Event Sourcing.
The book contains numerous standalone examples throughout the chapters, which are all executable. At the end of each chapter, the explained concepts are applied to the Sample Application context.
The Sample Application is a greatly simplified task board. Users can register, login, create projects, manage team members and work with a task board. The task board is essentially a collection of items, grouped into three columns. Individual tasks contain details such as title, description, status, assignee, etc.
While there are some constraints/invariants in the Domain Model, I would consider the Sample Application fairly trivial. It does not represent a complex domain problem. One reason for this is that I personally did not encounter any highly complex domain so far. It would have been weird if I tried to explain something I am not really experienced in.
> what if domain logic can be optimized in the database?
IMHO, the question is a bit too generic. The generic recommendation would be to not put domain logic into the database. However, one would need to look at what the domain logic in question is. Also, what does "in the database" mean? Are we talking about code that is being executed in the database environment, such as PL/SQL?
> examples where you have property method on a model that goes and performs additional queries
This sounds a bit like the Active Record pattern or the use of an ORM, where one "model" is somehow linked to another. Again, a generic answer would be that domain objects should stay free of infrastructural concerns. If you have posts and comments and need to get the latest comment on a post, this represents a use case. Use cases are executed by an Application Service. This part is allowed to query any number of domain objects / data sources. If you would be using CQRS, you would likely have a denormalized Read Model that contains comments per post.
I have to say that my recommendations on JavaScript education resources might be pretty outdated. The book that helped me a lot with the JavaScript language (in 2010) was https://www.goodreads.com/book/show/2998152-javascript. However, this was before the ECMAScript standard and the Web APIs developed as fast as they are doing today. Node.js I learned exclusively from reading the documentation and experimenting with it.
Similar to what zeroc8 said, I heard that people liked "You don't know JS". Also, I agree with joshxyz that you can learn a lot from the MDN and the Node.js API docs. Especially MDN is a great resource for learning specific concept. As example, look at their Promises tutorial: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Furthermore, I also think the publications from Axel Rauschmayer are worth checking out (exploringjs.com).
The section in the Preface of my book might sound a bit daunting. Even if you don't know JavaScript, but have experience with at least one other programming language, that might be fine. I think the most challenging aspect is the asynchronous programming part, which is being used extensively due to Node.js.
For Javascript, I learned a ton from Frontend Masters. In particular for a deeper understanding of the language, Kyle Simpson's Deep Javascript Foundations where he connects the behavior of Javascript to the Standard. Overall, they have a number of high quality Javascript related courses.
I wouldn't recommend that book if the only reason you learn JavaScript is to be able to follow along this book (DDD, CQRS...).
"You don't know JS" is a great book, and it's essential for someone who actually writes and works with JS all day. You'll pass most JS questions if you interview for a new job. At the same time, in my opinion, it's just a book full of gotchas that shouldn't even be present in good codebases (because most developers don't know JS :)).
Another architecture that seems to be becoming more common is having your CRUD functionality in a traditional monolith or a set of microservices, then writing events out or using change data capture to move that data into a data warehouse, where it is transformed for reporting purposes. While this is not the same as CQRS and event sourcing, it's a strategy that can be "bolted on" to your existing application and potentially capture some of the same benefits.
Yup, that's how it has worked at my past couple of jobs. One question that always comes up is what should be in a UI vs. what should be in a reporting dashboard owned by the data team?
That is a good question and, ideally, I should be able to answer it. Unfortunately, I don't know. The book started as a passion project and I never analysed the market in any useful way. However, I think that there is a certain potential for extending the existing market by making people interested in the topics.
This comment breaks both the HN guidelines and the Show HN guidelines. Please don't do that. We're trying to have a culture in which people who share their work get substantive feedback and discussion. Cheap dismissals undermine that badly—and since they're the internet default, we need to be conscious about this.
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
In fact, the book was called "Implementing DDD, CQRS and Event Sourcing in Node.js" up until the release of its complete version. I thought about this for a longer time and decided to remove the "in Node.js" part from the title. Instead, I made it clear in the book description. In my opinion, the book can be valuable for anyone who is interested in the concepts, regardless of technology choices.
I think that it is fair to say that my book is partially also about Node.js, but not primarily. I wouldn't say though it is about JS. In fact, many of the code examples would look similar in other languages. Furthermore, the use of Node.js helps to provide concise code examples and executable code without much ceremony.
When reading "Implementing Domain-Driven Design" by Vaughn Vernon, all the code examples are in Java. The book "Enterprise Integration Patterns" has a lot of code examples and all of them are also in Java. The most recent edition of "Refactoring" by Martin Fowler uses JavaScript. Still, I wouldn't say that these books are about a particular programming language.
I don't know where that came from but please don't make things up that look like quotes when they're not, and especially please don't use that as a snark trope.
(If someone said that and you're actually quoting them and they edited it out, that's of course different.)
Absolutely true that languages are built to fulfill specific roles and have associated trade-offs. I think you answered your own question - JavaScript's strengths are being very widespread in current software development and having a minimal syntax/type system. IMO it can be a good language for MVPs/POCs and serve as a higher form of pseudocode. Perhaps the author's choice of JS as a way of reaching the widest possible audience and focusing on the concepts more than the implementation details.
A relatively recent personal example: in working through some DSL and parser combinator examples in Rust, sometimes I get too distracted with lifetimes, annotations to deal with deeply recursive functions, etc. and just search for examples in a different language that allow me to focus on the concepts I'm trying to learn.
Since I am asked from time to time about the technology choices for my book, I decided to write a blog post to explain my reasoning: https://www.alex-lawrence.com/posts/why-my-book-uses-nodejs-... Some of my arguments pretty much match with your statements.
You can access a free sample of the book on the webpage in different formats.
I worked on the book for four years as a side project. In 2016, I had worked in three different projects that applied either some or all of the concepts. The last one was the main motivating factor for the book. It was a startup that was settled on DDD, CQRS and Event Sourcing, even though the developers had limited knowledge.
Together, we learned, experimented and applied as much as we could. We read "Domain-Driven Design" by Eric Evans, "Implementing Domain-Driven Design" by Vaughn Vernon and whatever we could find from Greg Young. With the knowledge we picked up and the experimentation we did, we felt in a good position. Still, there were many unanswered questions on how to put everything into code.
In 2016, before leaving the startup, I first thought about writing a book. Effectively, I wanted to create the guide the startup would have needed in the beginning. When I started, I was overly confident about my knowledge. However, throughout the years, I learned many details. Today, I know that I could not have written it in 2016 at once.
The final book is different from what I envisioned it to be at first. My initial goal was to write a short book with approximately 150 pages. The published version has now over 450 pages. However, I consider this a good thing. The book provides enough context to be read without any prior knowledge in DDD, CQRS or Event Sourcing.
I've written a blog post that covers my motivation on writing the book in even more detail: https://www.alex-lawrence.com/posts/why-i-wrote-a-book-on-dd...
I'd be happy to discuss your thoughts on and experiences with DDD, CQRS and Event Sourcing. Regardless whether it is about their combination, their sometimes counterproductive conflation or one of the concepts. Finally, if you have any input or feedback for me, it would be highly appreciated!