GraphQL Query Batching | Cosmo by WunderGraph - WunderGraph

Batched GraphQL Operations Through One HTTP Request

Preserve response order across multiple operations, even when execution finishes at different times, without rewriting existing GraphQL clients.

Support legacy batch clients without client-side rewrites. Batching is off by default and can be enabled when needed.

The problem

Support legacy batch clients without overwhelming the router

Teams need compatibility without giving up observability or control over resource usage.

Migration hits the clients first

A gateway that doesn't speak batched requests forces every client team to ship a new release before you can move traffic. That's quarters, not weeks.

Batched requests can overwhelm a router

A client sending one request with 200 operations can consume significant router resources. Without controls, a single batch can saturate the router.

Batch executions are opaque

Standard tracing shows one HTTP request, but not how many operations it contained or which one was slow.

Our solution

Keep legacy batch clients working without overwhelming the router

Cosmo Router accepts multiple GraphQL operations in a single HTTP request, executes them concurrently with configurable concurrency limits, and returns responses in the original request order, regardless of completion time.

This lets teams support existing batch-based clients without requiring client-side rewrites while still controlling resource usage at the router level.

OpenAPI Initiative

batching:
  enabled: true
  max_concurrency: 10
  max_entries_per_batch: 100
  omit_extensions: false

Before & after

Migration and operations

Before Cosmo	With Cosmo
Batch-dependent clients block the gateway migration	Existing batch clients work without code changes
Oversized batches can exhaust router resources	`max_entries_per_batch` and `max_concurrency` limit resource usage
Batched requests hide inside single-request traces	Per-operation tracing attributes expose batch contents
Batch execution lacks configurable concurrency limits	Configurable concurrency limits control parallel execution

Tracing attributes

Per-operation visibility

wg.operation.batching.is_batched
wg.operation.batching.operations_count
wg.operation.batching.operation_index

See how individual operations behave inside a batched request.

Use cases

Patterns teams enable first

Support legacy migrations, control resource usage, and improve observability from a single configuration.

Legacy client migration

Compatibility

If your app already packs several GraphQL operations into one HTTP request, turn on batching in the router config and set limits for batch size and concurrency. Those requests keep working when you move to Cosmo Router—no client rewrite for the cutover.

Resource-controlled batch processing

Limits

Configure max_entries_per_batch: 50 and max_concurrency: 5 to cap batch size and parallel execution. This keeps resource usage predictable and protects the router from oversized requests.

Batch performance analysis

Observability

Use OpenTelemetry spans from the router to analyse batch behaviour in your observability platform alongside the rest of your traces.

How query batching works

Opt in via config. Off by default.

Accept the batched POST

Clients send a JSON array of operations. Router recognizes the array shape when batching is enabled.

Reject oversized batches early.

Validate and enforce limits

Router checks the array against max_entries_per_batch. Batches larger than the limit are rejected before any planning happens.

Concurrency limited by max_concurrency.

Plan and execute concurrently

Each operation is planned and executed independently, with concurrency controlled by max_concurrency.

Failures are isolated per operation.

Return in order

Responses are collected and returned as an array in the same order as the request. Errors are returned at the matching position in the response array. One bad operation does not fail the batch.

Turn on batching for existing batch-based clients

Set concurrency limits, cap batch size, and trace individual operations.

FAQ

GraphQL query batching

Should you enable GraphQL query batching in a federation router?

Does GraphQL query batching reduce HTTP overhead compared to separate requests?

Is GraphQL query batching compatible with Apollo Client's BatchHttpLink?

How do you set a maximum batch size for GraphQL requests?

How does GraphQL query batching interact with rate limiting?

What happens if one operation in a batched GraphQL request fails?

Does each operation in a batched GraphQL request get its own query plan?

Can you include GraphQL subscriptions in a batched HTTP request?

How do you trace individual operations inside a batched GraphQL request?

Can you suppress batch operation error codes from GraphQL response extensions?

Full reference in the query batching documentation.