Introduction

After doing some research on various CQRS and Event Sourcing libraries / frameworks for golang, I decided to write my own event store which is capable of being the central place for communicating and storing events.  As long as a process has access to the underlying  store (e.g kafka) then it can listen for events and access past events.

To assist in building aggregate roots that follow this event stream, the library will support snapshotting as well – which can be tuned on a per system basis so that rebuilding the aggregate root does not always have to go back to the beginning of time.

Terminology

Stream – A common stream of events stored in the service. Commonly used one a 1 to 1 basis by an aggregate root in CQRS.

Local Event – An event being persisted by the same process

Remote Event – An event persisted by an external process

The Design

I have chosen to use the ‘commanded’ (See github repo) library as a reference design for the event store.  I have worked with this in the past and does everything that I need it to.

Subscriptions

The subscriptions API allows the calling code to receive notifications when events come in.  They can choose all events, events of a particular type, local events, remote events etc.. etc..

In general, it is assumed that you will always want both remote and local events and they will appear the same outside of this interface.  In a CQRS system where this could be used, the aggregate roots can follow the stream using this interface and they would stay up to date whether the change was made locally or remotely.  This also gives them a chance of detecting collisions (version numbers can also help with this)

Publishing

The publishing API allows the calling code to publish events which will get stored and all subscribers will be notified of the event(s)

Snapshot

The snapshot API allows the calling code to store a snapshot of the object that the stream represents – for example an aggregate root in a CQRS system

Catchup

The catchup API allows the caller to play catch up with the event store.  This allows for remote event stores to follow this one, but if they go offline then they can catchup at any time

Concurrent and Parallel Activity

When the store is asked to persist an event, from the point where it was asked to where the event is actually persisted, another process or goroutine within this process may well have added new events.

It will be up to the calling code how to deal with this, this library can only report it.  As this library does not cache anything then it has nothing to worry about as the next time the caller asks for the events for a stream, they will get the correct events with both sets added.  However, in a CQRS system, an aggregate root might have had an event applied to it to change something and then it would need to deal with this other set of events as well.  In the event that both events modify the same thing in the aggregate root, it would be up to the caller what it does – the chances are it would replay the events from the beginning to ensure everything is in the correct order – as the events could have been communicated out of order via the publish / subscribe mechanism

The Development Setup

After doing a bit of research on the popular ways to setup my project, I decided that I completely disliked the central GOPATH – i.e. in ~/go for example as I could not get my head around how versions would be managed.  After looking round for dependency managers it appears I was right – the versions were not managed.

So, I decided firstly on ‘dep’ (https://github.com/golang/dep) which appears to be the norm – and even if its not, coming from ruby, it seems to be similar to bundler.  It is also being developed by the go team themselves I believe so more likely to become the standard in the future.  My second option was govendor which seems very good as well.

Next, I am a TDD / BDD developer so I’m not going anywhere without a test framework.  I have used what is built into go itself, but I didn’t like it much.  Judging by the number of testing packages out there, neither does everyone else.

I have read many good articles highly recommending Ginkgo and Gomega so first things first, lets try and use the dependency management system ‘dep’ to install these and make sure they end up where I expect (in the vendor folder of my project)

I had a bit of trouble as ‘dep’ was trying to be too clever and trying to make sure I was actually using the dependencies – but it had no go code to check and gave this error

no dirs contained any Go code

So, just go get a stage further, I created a main.go file and added :-

package main

Then ran this command

dep ensure -add github.com/onsi/ginkgo/ginkgo

and I got this in return

Fetching sources...

"github.com/onsi/ginkgo/ginkgo" is not imported by your project, and has been temporarily added to Gopkg.lock and vendor/.
If you run "dep ensure" again before actually importing it, it will disappear from Gopkg.lock and vendor/.

So, it looks like I’m doing things in the wrong order – trying to sort my dependencies before I write the code that depends on them, but Ive started so Ill finish

Next, lets try the same thing with omega

dep ensure -add github.com/onsi/gomega

and things got slightly worse – this is what I got

Warning: the following project(s) have [[constraint]] stanzas in Gopkg.toml:

✗ github.com/onsi/ginkgo

However, these projects are not direct dependencies of the current project:
they are not imported in any .go files, nor are they in the 'required' list in
Gopkg.toml. Dep only applies [[constraint]] rules to direct dependencies, so
these rules will have no effect.

Either import/require packages from these projects so that they become direct
dependencies, or convert each [[constraint]] to an [[override]] to enforce rules
on these projects, if they happen to be transitive dependencies,

Warning: Gopkg.lock is out of sync with Gopkg.toml or the project's imports.
Fetching sources...

"github.com/onsi/gomega" is not imported by your project, and has been temporarily added to Gopkg.lock and vendor/.
If you run "dep ensure" again before actually importing it, it will disappear from Gopkg.lock and vendor/.

so, I think I am getting the message – Im putting the cart before the horse ?

But, no harm done – lets carry on in part 2.