ruedigergad

In this post, I write a bit about some things I did for fun in the area of Message-oriented Middleware (MoM). I am not sure how much others relate to this but I hope that some may find it a bit entertaining as well.

With “Polyglot Messaging”, I refer to jointly using multiple MoM protocols. With “Polyglot Integration”, I refer to integrating implementations written in different languages (C, Clojure, JavaScript, and Python) via the Polyglot Messaging.

Overview

To cut a long story short, the image below shows the visible outcome. It shows three applications that display data that was generated programmatically and sent via a Message-oriented Middleware (MoM). As a bit of a teaser, there are also animations that show the data in motion a bit further down in this post.

The generated data sent via the MoM are dots in a 3D space that resemble a rotating rainbow-ish colored Yin Yang symbol. The applications shown in the image are:

• Bottom Left: An application implemented in C that 3D-renders to a terminal. I called it parTTYcles. It is based on TermGL, MQTT-C, and json-parser.
• Bottom Right: A Python application that renders the data via matplotlib.
• Background Right: A web page that renders the data via A-Frame.

Setup

The image below shows an overview of the setup and how the different elements interact.

bowerick is the central part that provides the Message-oriented Middleware (MoM) and message generation. bowerick and its message generation are implemented in Clojure, indicated using Blue color. bowerick wraps various MoM-related libraries such as ActiveMQ etc., which provide, e.g., the embedded broker and the transports for the supported protocols (OpenWire, MQTT, STOMP, and STOMP via WebSockets).

Most of the libraries used in bowrick are implemented in Java, indicated using Green color. I feel very satisfied with using Clojure for implementing the surrounding bowerick code and its automated tests (not shown here).

I started development of bowerick in 2014. Since then, I regularly evolved bowerick and kept all dependencies up-to-date. I am very happy with the entire ecosystem and found that Clojure is a great basis for this work.

Integration via Polyglot Messaging

The message generator produces a list of “3D dots” with additional scale and color information. This data is serialized to JSON in UTF-8 strings and the UTF-8 strings are transferred in byte-array-messages.

The embedded ActiveMQ broker automatically bridges between the different protocols. Thanks to the standardized serialization (JSON -> UTF-8 -> byte-array-messages), the bridging between protocols is fully transparent such that each protocol can be equivalently used.

For the example shown in this post, the most distinguishing factor regarding the selection of one protocol over another was the availability and ease of use of a corresponding library that supports the protocol for the corresponding programming language. I basically ended up with the protocol choice as it is because it was the easiest way for me to implement the different parts in this way. Of course someone else may have other choices but in general, I very much enjoy this freedom of choice offered by the polyglot messaging part.

Animations

The animated image below shows the running data display that represents the rotating Yin Yang sign. The message generation was set to about 10 messages per second. So, the “animation frame rate” also corresponds to roughly 10 frames per second.

Running

Below, I provide instructions on running the applications as shown above.

To start the broker and message generation, download bowerick and run:

java -jar dist/bowerick-2.8.0-standalone.jar -G yin-yang -I 100 -u '[tcp://127.0.0.1:1031 mqtt://127.0.0.1:1701 ws://127.0.0.1:1864 stomp://127.0.0.1:2000]'

To run parTTYcles, clone the repository and follow the instructions on building it and run it. Below a brief summary is shown:

git clone https://github.com/ruedigergad/parTTYcles.git
cd parTTYcles
git submodule init
git submodule update
make
./parTTYcles

For the Python example, clone the bowerick repository and run the Python example located in examples/python_particles_example as follows:

virtualenv -p python3.8 env
source env/bin/activate
pip3 install -r requirements.txt
./python_particles_example.py

For the A-Frame example, clone the bowerick repository, open the corresponding web page in a browser, and follow the instructions.

Alternative Message Generation

Thanks to the loose coupling via the Message-oriented Middleware (MoM), alternative ways for generating messages can be easily implemented. In principle, any MoM will do. For my example, I will continue to use bowerick. The most important point for the alternative message generation is to start bowerick without the message generator, which can be done as follows:

java -jar dist/bowerick-2.8.0-standalone.jar -u '[tcp://127.0.0.1:1031 mqtt://127.0.0.1:1701 ws://127.0.0.1:1864 stomp://127.0.0.1:2000]'

In this example, I use curl as alternative tool for sending messages into the MoM. With this change, the setup looks as follow:

A single message can be sent as follows:

curl -d '[{"x": 0.0, "y": 0.0, "z": 0.0, "scale_x": 2.0, "scale_y": 2.0, "scale_z": 2.0, "color_r": 0.0, "color_g": 1.0, "color_b": 0.0}]' mqtt://localhost:1701/bowerick/message/generator

This will display a single green dot. Below an example is shown:

To illustrate generation of an animation, or a stream of messages that produce an animation, I use the example below:

while true ;
do
    for i in $(seq -1 0.1 1);
    do
        curl -d '[{"x": '${i}', "y": 0.0, "z": 0.0, "scale_x": 2.0, "scale_y": 2.0, "scale_z": 2.0, "color_r": 0.0, "color_g": 1.0, "color_b": 0.0}]' mqtt://localhost:1701/bowerick/message/generator
        sleep 0.1s ;
    done ;
done

End

Thanks a lot for reading. I hope you enjoyed the post. In an earlier post, I described some more details of the used messaging protocol.

Thanks a lot also to everybody who contributed to the libraries, tools, documentation etc. that were used and to their surrounding communities.

If you have comments or feedback, please let me know.