Messaging Patterns: Pub/Sub, Request/Reply, and Choreography

• Architecture • By TryzTech Team
MessagingArchitecturePubSubMicroservicesBackend

As backend systems evolve from monolithic architectures into distributed microservices, engineering teams quickly face a pivotal question: how should independent services communicate without introducing tight coupling and cascading outages?

Many teams naturally start by chaining synchronous HTTP REST calls between services. While intuitive at first, this approach collapses under scale, leading to cumulative latency bottlenecks, distributed timeouts, and deployment dependencies where one down service breaks the entire system.

This article explores three core messaging patterns in modern distributed systems—Request/Reply, Publish/Subscribe (Pub/Sub), and Event Choreography—using an e-commerce Order & Payment Lifecycle case study, accompanied by Python worker implementations and message broker trade-offs.


Case Study: The E-Commerce Checkout Lifecycle

To clearly evaluate these patterns, consider an e-commerce application where a customer clicks “Place Order”. Completing this transaction involves:

  1. Validating payment and charging the customer (Payment Service).
  2. Reserving physical inventory in the warehouse (Inventory Service).
  3. Generating a shipment label (Shipping Service).
  4. Dispatching order confirmation emails and notifications (Notification Service).

Let’s dissect how each messaging pattern approaches this workflow.


1. Request/Reply: Synchronous Point-to-Point Communication

The Request/Reply pattern is a direct, point-to-point communication style where a sender (Requester) delivers a payload to an explicit recipient (Replier) and blocks execution until a response is returned.

sequenceDiagram
    autonumber
    actor Client
    participant OrderService as Order Service
    participant PaymentService as Payment Service
    participant InventoryService as Inventory Service

    Client->>OrderService: POST /checkout
    activate OrderService
    OrderService->>PaymentService: POST /charges (Wait for Confirmation)
    activate PaymentService
    PaymentService-->>OrderService: 200 OK (Payment Success)
    deactivate PaymentService
    OrderService->>InventoryService: POST /reserve-stock (Wait for Confirmation)
    activate InventoryService
    InventoryService-->>OrderService: 200 OK (Stock Reserved)
    deactivate InventoryService
    OrderService-->>Client: 201 Created (Order Complete)
    deactivate OrderService

When to Use It

  • Operations requiring immediate certainty before the user can proceed (e.g., credit card pre-authorization, session authentication checks, or real-time wallet debit).
  • Read-heavy queries that mandate strict, deterministic consistency.

Trade-offs

  • Pros: Intuitive mental model, straightforward linear debugging, and immediate consistency (ACID-like certainty).
  • Cons: Cumulative latency ($T_{total} = T_{payment} + T_{inventory} + \dots$). If the Inventory Service experiences downtime, the entire checkout pipeline fails. To build resilience in this pattern, services should incorporate mechanisms like the Circuit Breaker Pattern.

2. Publish/Subscribe (Pub/Sub): 1-to-Many Event Broadcasting

In the Publish/Subscribe pattern, the sender (Publisher) has no awareness of downstream consumers. Instead of directing commands to specific endpoints, it broadcasts domain events to a centralized topic or channel. Interested consumers (Subscribers) bind to that topic and process events asynchronously.

flowchart LR
    OS[Order Service] -- "Publish: OrderPaid" --> Topic((Topic: order.events))
    Topic --> NS[Notification Service]
    Topic --> AS[Analytics Service]
    Topic --> FS[Fraud Detection Service]

Python Implementation (Asynchronous Event Worker)

Here is an example demonstrating a publisher emitting events and a subscriber worker handling side effects:

# order_publisher.py
import json
import uuid
from datetime import datetime

def publish_order_paid(channel, order_id: str, customer_id: str, total_amount: float):
    event_payload = {
        "event_id": str(uuid.uuid4()),
        "event_type": "OrderPaid",
        "timestamp": datetime.utcnow().isoformat(),
        "data": {
            "order_id": order_id,
            "customer_id": customer_id,
            "amount": total_amount,
        }
    }
    # Publish domain event to the exchange
    channel.basic_publish(
        exchange="order_exchange",
        routing_key="order.paid",
        body=json.dumps(event_payload),
    )
    print(f"[x] Emitted OrderPaid event for order: {order_id}")

And the consumer handling email dispatch without blocking the main checkout transaction:

# notification_worker.py
import json

def handle_order_paid_event(ch, method, properties, body):
    payload = json.loads(body)
    data = payload["data"]
    
    # Process email notification asynchronously
    send_email_receipt(
        customer_id=data["customer_id"],
        order_id=data["order_id"],
        amount=data["amount"]
    )
    # Acknowledge successful message processing
    ch.basic_ack(delivery_tag=method.delivery_tag)

def send_email_receipt(customer_id: str, order_id: str, amount: float):
    print(f"[Email] Receipt sent for Order {order_id}: ${amount:,.2f}")

When to Use It

  • Asynchronous side effects: transactional emails, audit logs, real-time analytics, and search index updates.
  • Scenarios where new consumers need to be added in the future without modifying or redeploying the originating service.

3. Event Choreography: Decentralized Workflow Coordination

In Choreography, services coordinate multi-step workflows without a central orchestrator. Each service reacts to incoming domain events, completes its localized transaction, and emits a subsequent event to trigger the next phase of the workflow.

sequenceDiagram
    autonumber
    participant Order as Order Service
    participant Payment as Payment Service
    participant Inventory as Inventory Service
    participant Shipping as Shipping Service

    Order->>Payment: Event: OrderCreated
    Note over Payment: Process card charge...
    Payment->>Inventory: Event: PaymentCaptured
    Note over Inventory: Reserve physical stock...
    Inventory->>Shipping: Event: StockReserved
    Note over Shipping: Print shipping label...

Trade-offs of Choreography

  • Pros:
    • Loose Coupling: Individual microservices remain decoupled, communicating strictly through contract-driven events.
    • Independent Scalability: Heavy components (like label printing or payment validation) scale horizontally according to their isolated queue depths.
  • Cons:
    • Low Operational Visibility: Without distributed tracing (e.g., OpenTelemetry / correlation IDs), diagnosing where a distributed transaction stalled becomes challenging.
    • Complex Compensating Transactions: If inventory turns out to be out of stock after a successful payment capture, the system must emit an InventoryReservationFailed event to orchestrate a refund.

Comparison Matrix: The 3 Core Patterns

DimensionRequest / ReplyPublish / SubscribeEvent Choreography
CouplingHigh (Target endpoint awareness)Very Low (Topic awareness only)Low (Domain contract awareness)
Execution ModelSynchronous (Blocking wait)Asynchronous (Fire-and-forget)Asynchronous (Multi-step flow)
Data ConsistencyImmediate Consistency (ACID)Eventual ConsistencyEventual Consistency
Debugging ComplexityLow initially, high at scaleModerateHigh (Demands distributed tracing)
Primary Use CasesPayment capture, auth validationNotifications, analytics, auditCross-service order fulfillment

Choosing a Message Broker: RabbitMQ vs. Apache Kafka vs. Redis

A messaging pattern requires the right underlying transport infrastructure. Here is how the most common brokers compare:

flowchart TD
    Q{What is your primary system constraint?}
    Q -- "Flexible routing, complex task queues" --> RMQ[RabbitMQ]
    Q -- "Millions of events/sec, event replay" --> KFK[Apache Kafka]
    Q -- "Sub-millisecond latency, in-memory" --> RDS[Redis Pub/Sub / Streams]

1. RabbitMQ (AMQP)

  • Architecture: Smart Broker, Dumb Consumer. The broker tracks message acknowledgements, routing keys (topic, direct, fanout), and Dead-Letter Exchanges (DLX).
  • Latency: Extremely low (~1-5 ms).
  • Throughput: ~20,000 – 100,000 messages/sec per node.
  • When to Choose: Complex routing topologies, background task processing, and choreographed workflows requiring fine-grained routing logic.

2. Apache Kafka

  • Architecture: Dumb Broker, Smart Consumer. Messages persist as an append-only distributed log, with consumer positions tracked via partition offsets.
  • Latency: Moderate (~5-20 ms, optimized for batched sequential I/O).
  • Throughput: Extremely high (hundreds of thousands to millions of events/sec).
  • Key Capability: Event Replay—new services can replay historical event streams from offset zero.
  • When to Choose: Event-sourced architectures, audit pipelines, high-volume real-time telemetry, and log ingestion.

3. Redis (Pub/Sub & Streams)

  • Architecture: In-Memory Data Store. For comprehensive architectural patterns, see our guide on Redis for Backend: Cache, Session, and Pub/Sub.
  • Latency: Sub-millisecond (in-memory execution).
  • Throughput: High memory operations (~100,000+ ops/sec).
  • When to Choose: Ephemeral WebSocket fanouts, real-time UI synchronization, and lightweight worker queues when infrastructure footprint must remain minimal.

Engineering Decision Framework

  1. Reserve Request/Reply for synchronous boundaries where callers strictly cannot progress without an immediate response (e.g., auth tokens and payment gateway authorizations).
  2. Adopt Pub/Sub to decouple downstream secondary concerns (email receipts, analytics events, cache invalidation) from critical user paths.
  3. Use Choreography for distributed multi-service workflows, provided your platform incorporates centralized logging, correlation IDs, and idempotent consumer designs.
  4. Avoid adopting distributed log brokers like Kafka prematurely if your workload is below thousands of events per second; simpler tools like RabbitMQ or Redis Streams consistently yield lower operational overhead for most engineering teams.

Keep reading within the same topic.

Don't Miss Out

Get the latest tech articles, tips, and insights delivered to your inbox.