System Design

What is the problem with the following architecture and how would you fix it?

Difficulty: unrated

Source: bregman-arie/devops-exercises by Arie Bregman

Answer

The load on the producers or consumers may be high which will then cause them to hang or crash.

Instead of working in "push mode", the consumers can pull tasks only when they are ready to handle them. It can be fixed by using a streaming platform like Kafka, Kinesis, etc. This platform will make sure to handle the high load/traffic and pass tasks/messages to consumers only when the ready to get them.