I see a fair amount of Kafka, while most other platforms have diminished. I think that is because people treat Kafka like a database/system of record. A queue is not a system of record.
A lot of the difficulty in modeling a complex system has to do with deciding what is durable state vs what is transient state.
Almost all state should be durable and/but durability is more expensive upfront. So people make tradeoffs to model a transition as transient and put in a queue. One or two or three years in, that is almost always a regretted decision.
Message queues that are not databases/systems of record wind up glossing this durable/transient state problem, and then when you have also this unique piece of infrastructure to support, this is a now you have two problems moment.
A lot of the difficulty in modeling a complex system has to do with deciding what is durable state vs what is transient state.
Almost all state should be durable and/but durability is more expensive upfront. So people make tradeoffs to model a transition as transient and put in a queue. One or two or three years in, that is almost always a regretted decision.
Message queues that are not databases/systems of record wind up glossing this durable/transient state problem, and then when you have also this unique piece of infrastructure to support, this is a now you have two problems moment.