← All Concepts
Async Processing
Message Queues
Enables asynchronous communication between services by buffering messages for later processing.
**Message queues** decouple producers from consumers, enabling asynchronous, reliable communication.
**Types:**
- **Point-to-point** (SQS): Each message consumed by one consumer. Good for task distribution.
- **Pub/Sub** (Kafka, SNS): Messages broadcast to all subscribers. Good for events.
- **Streaming** (Kafka): Ordered, persistent log. Consumers track their position.
**Key concepts:**
- **Producers** send messages to a queue/topic
- **Consumers** read and process messages
- **Acknowledgment**: Consumer confirms processing before message is removed
- **Dead letter queue (DLQ)**: Failed messages sent here for inspection
- **Ordering**: FIFO within a partition/queue
- **At-least-once vs exactly-once delivery**
**Common tools:** Apache Kafka, RabbitMQ, AWS SQS/SNS, Redis Streams.
Common Use Cases
- Decoupling microservices (order service → payment service)
- Background job processing (email sending, image resizing)
- Event-driven architectures
- Rate leveling (absorb traffic spikes)
Advantages
- +Decouples services (independent scaling and deployment)
- +Handles traffic spikes by buffering
- +Enables retry logic and fault tolerance
- +Natural fit for event-driven architectures
Disadvantages
- -Adds latency (async processing)
- -Message ordering can be tricky
- -Exactly-once delivery is hard to guarantee
- -Operational complexity (managing queues, monitoring lag)