MessageBus – Automated Documentation of our Messages

Written by Martin Führlinger, Software Engineer Backend

ALL ABOUT MESSAGE BUS

Welcome to the sixth publish on this message bus associated weblog publish sequence. I extremely advocate studying the others too, to get the complete context. These are: 

• Using RabbitMQ for decoupling our providers
• How we outlined our message content material. 
• How we preserve our receivers quick and resilient. 
• How useless letter dealing with with a useless letter change works
• How we’re dealing with our useless letters

Now, I need to present extra insights into our documentation round messages and the way we automated this.

Message Documentation

Documentation is difficult. No matter who you ask, documentation is all the time exhausting to create and most necessary preserve updated. As all the time, we began with documenting our messages in our firm wiki’s information area. Whenever new messages had been added or present ones modified, somebody had so as to add or replace them. The message subjects had been listed in a matrix, containing the producer, the subjects, the customers, and a few further notes. It appeared like this:

This course of was gradual and error-prone. So we wished to enhance this. 

How to Automate Documentation 

During considered one of our Day of New Ideas (DONIs), some colleagues and myself began to work on a small service to routinely doc the present messages in our system. We got here up with a fairly easy service, named MessageDoc, simply listening to all attainable messages, so to all routing keys (discover particulars about routing right here), by listening to `#.#`.

Whenever we obtain a message, we parse it and retailer its data. This means we retailer a doc in a small MongoDB containing the messages meta information, just like the producer, the message itself and a few extra attributes. This is completed per subject. The circulation of receiving and storing will be seen within the following diagram.

To see how this seems to be like in our database, right here is an instance: 

{
  "_id" : "sample.updated.altered",
  "producer" : "samples",
  "description" : "",
  "consumers" : [
    
  ],
  "entities" : [ {
    "type" : "run_session",
    "updated_at" : ISODate("2020-11-11T14:56:02.304Z"),
    "example" : ,
  } ],
  "version" : 4124,
  "created_at" : ISODate("2019-11-28T11: 15: 26.105Z"),
  "updated_at" : ISODate("2020-11-12T06: 00: 12.950Z")
}

As we don’t need to retailer each single message, we solely replace the message’s content material as soon as a day. We generally have polymorphic messages. This means we’re sending several types of entities inside the identical message sort. For occasion, a “sample_message” will be of sort “run_session”, “sleep_session” or others. We retailer just one instance per entity sort.

We know who the producer of a message is, as it’s a part of the message payload (see how we outlined our message content material). Whenever a doc is written to mongodb, we additionally replace the customers asynchronously. We achieved this by querying the Rabbitmq API configuration through a HTTP request (utilizing the all-configuration path), parsing it and updating all customers of all our saved messages. Details about “consumers” will be discovered within the first weblog publish on this sequence.

Building a easy Web-Interface

To be capable to entry the saved data with out querying the database straight, we constructed a easy internet interface on high of this. Basically, the overview seems to be much like the earlier matrix within the wiki. It exhibits all subjects grouped collectively by the producer.

When accessing a subject, the subject particulars are proven, with all of the customers once more for this particular subject. This additionally means the customers of a subject will be completely different within the particulars view, as e.g.”samples.created.accomplished” messages are consumed by different providers than “samples.updated.altered”. This subject element view additionally permits so as to add some description to the subject.

As talked about above, we retailer an instance message for each subject and each entity sort as soon as a day. These will be proven too by increasing the corresponding entity hyperlink under.

Making Manual Message Documentation Obsolete 

With this service in place, we had been in a position to deprecate the message documentation within the wiki. Even if we delete information saved within the message doc service, it’s routinely recreating all the knowledge. The solely factor which might be misplaced are the extra descriptions per subject which had been added manually. This easy service, with lower than 1000 traces of code general, made guide documentation of messages and its content material out of date, and our life simpler.