There are a lot of articles talking about the benefit of using microservices. So I wanna do something opposite here. Please don’t get me wrong, it is a good architecture to distribute your system to smaller parts, and help people to identify the problem quickly. Besides it also allows us to upgrade individual services independently, or even share the services to other systems much easier. It does help us to divide a big monster system into smaller parts, allocate tasks to different teams, and reduce some development dependency.
However, sometimes we may too focus on those benefits. It may overlook some other things as well. Here, I would like to put out some sour scenarios that happen to me while I am using the microservices. The scenario I put up may not happen to you. It is also not related to technical as well, I just hope it can bring up awareness or concern while planning on microservices.
Step up
Let’s imagine you are managing a multinational e-commerce website, which mainly sells various goods and products just like amazon. Currently, this website is in the middle of the migration from a legacy monster backend service to a microservice. This microservice is only handling the customer website, only 50 percent of the total customer is assigned to the microservice system. The rest of the customers will continue using legacy backend services.
On those microservices, the first service is the legacy posting service, where it manages the seller posting advertisements to the website. When a newly posted item is added, it will fire up an event to advertisement feed service. This new service will broadcast out the advertisement information to other systems. In order for customers to find the product, a search service will subscribe to the advertisement feed service. After the search service received it is indexed and stored on the service database.
Scenario
Since this is a multinational website it also can mean each of the services is controlled by a different team manager and software engineer located at different locations and time zones.
Now an unexpected issue happens, a seller has run out of stock. So as an experienced user, they do a quick action by putting a negative number into the admin service and it does successfully update the stock number as he wanted. However, this action doesn’t prevent customers from keeping ordering the item on the customer website. So the seller makes a complaint to your customer service deck because it is hurting his reputation on the website.
As a customer service, you tried to escalate the situation to your developer via your intranet system and after an hour, the developer told you it was out of his scope and you should report it to the advertisement feed service team who are located in another country.
Since they are from another country you know they don’t have the access to your current intranet system. The only thing you can contact them will be via email. After some investigation, the advertisement feed service team informs you the message they broadcast is correct and you should process it to the search service team.
Then finally found out the root cost was on the search service team. Where they do not accept a negative number from the feed service, because logically the deduction should be on the customer side not from the feed service. So after some meetings and discussions, the team agrees to make some changes to the system and update the stock accordingly.
Since it happens a while the seller wishes you can help them to erase the bad review that happened on these few days due to these incidents. You can imagine you may need to go on and on another round of meetings to verify and remove the review accordingly.
Afterthought
As a developer, we wish we can have a utopian land that does things that we think are cool, simple, and easy to manage on our end. However, sometimes it may affect other parties that are trying to support you. So we need to think a bit wider, like aftercare, support team, and culture of your current organization as well. Even if you have a great system and the brightest developer, it will become broken when we are unable to support it.
Personal Perspective
The thing I try to point out here is microservice will be powerful from the perspective of engineers. It may look simple at the beginning, but when you scale up the system it will be a different story. Things may get complicated in managing resources and expectations when your engineering teams are getting bigger and distributed to different regions.
Team communication can be out of sync if they are too focused on their own task, things might look logical for them but still better to verify with another team documentation or the legacy system code as well.
To ease out the situation:
- Mapping out the whole microservices flow will also help the new joiner to refer to it easily.
- Logging and monitoring on the API side and consumer side.
- General knowledge sharing or catch up between the teams biweekly.
- List down some common request actions like update stock, changing product name, expired a product, and anything else. So when a similar problem happens.
It also can shorten the service team to find the correct person instead of wasting time waiting for a reply and redirecting to others. With this, we can have a proper expectation time to inform the end-users as well.
Ending Note
To conclude, if we try to build a better service, just remember to share the knowledge as well. At the end of the day, outsiders will only view us as a team, not individual microservices.
