When playing in a group, you must unify and harmonize with the group and with all of their concentrated efforts in order to project that group in a most favorable light. - Horace Silver, The Art of Small Combo JazzFifteen developers, eight different languages, and plenty of pizza - that was the recipe for the first Microservices Hackathon in London last month. I had organised it but wasn’t sure what the result would be - in fact we managed to learn a whole bunch of new technologies, co-operate successfully to build a variety of components in several languages, and construct a simple MUD using combo-style microservices. To learn how, read on!
After a couple of experiments writing microservices systems on my own, and a well-attended talk at SkillsMatter, I’d decided to try out combo-style microservices with a larger group at a Saturday hackathon. My goal was to get everyone writing code, learning new things, and trying out real polyglot programming using the independence afforded by the microservices style.
I started by setting up the Meetup group and was surprised and pleased at the vigorous response (with no marketing at all). Once I knew people would really come spend a Saturday with me hacking microservices, I got going on creating the code we’d use to communicate. In accordance with my ideas about the combo style, this consisted of a RabbitMQ instance for pubsub and a PostgreSQL installation for the fact database, together with an archivist component to record published facts in the database. I also built a sample application and included a deployment mechanism (which we turned out not to need). I installed all of these on Amazon Web Services and opened all the firewalls to the world (fun to ignore security for once).
Everyone settled in to a quiet afternoon of coding. Other than helping some of the teams to make decisions about what facts to publish and when, I didn’t have much to do besides order the pizza and eat some of it. I was pleased to see that we appeared to have achieved component independence, one of the main goals of a microservices architecture - that is to say, once the co-ordination language was agreed, the individual component teams were free to get on and implement without worrying about orchestration or API design.
Toward the end of the day, I was pleasantly surprised to see that all the teams were close to finished. Because all of them had already been publishing and subscribing throughout the day, integration just consisted of cleaning up the fact database and using each component in turn - entering some rooms, doors, and descriptions using the room manager, issuing move commands using the command-line UI, and observing the results - either successful movement to a new room, or failed movement when no door was present, thanks to the movement component. Everything worked smoothly including a nice dynamic update to the room map. It was really impressive to see that we’d managed to create so much in just a few hours, even though most of us didn’t know the others’ languages!
Some interesting observations along the way:
- I had expected everyone to want to deploy code onto servers, and was ready to spin up more including Windows instances if needed. As it turned out, with good wifi and fast laptops, it was easiest just to have each team run its own code locally - in essence we had a five-server cluster (the one I set up at AWS running the archivist and RabbitMQ, plus one machine per team in the room). This means we didn’t get to practise using any of the clever tools for microservices deployments, but the gain in efficiency was definitely worth it.
- The big monitor in the meeting space was really nice - we particularly found it helpful to run a real-time view of the fact database, so everyone could see what was being published and picked up (or not picked up) by their components.
- Clearly the most painful bit was connecting to RabbitMQ and PostgreSQL. I was disappointed as I thought these were mature technologies with high-quality libraries in most languages - perhaps we just didn’t know enough to pick the right ones. Next time I plan to provide an HTTP shim for both with verbs like “publish” and “query”, so no one has to struggle for very long with a recalcitrant library. (Well, COBOL folks might have to hunt around for a good HTTP library, but I’ll take that risk.)
- The hackathon format didn’t lend itself to stress-testing the design. Not only did we not have time to create a well-documented, thoroughly tested product, we didn’t get to some of the communication and evolution challenges that microservices critics have noted - we didn’t see any emergent behaviour, didn’t try to evolve our language or even our components very much, and we didn’t worry at all about latency. I think this is OK since we weren’t really trying to do such a stress test - we just wanted to learn about this new way to write polyglot, robust code.
Nearly everyone said the day was well spent and expressed interest in attending another hackathon, so I’m planning to hold one in early September. Join the Meetup group to find out when that is!
(Thanks to my employer Osper for sponsoring the event. If you’re interested, you can read about some more cool stuff we’re trying at Osper, and of course we’re hiring.)