# Cosmo Observability

## See every span, metric, and log in your federated graph

OpenTelemetry-native traces, Prometheus metrics, structured access logs, and pprof, built into the Cosmo Router. Every signal carries GraphQL operation, subgraph, and client context out of the box.

## Overview

### What Cosmo Observability is

Cosmo Observability is the telemetry layer of the Cosmo Router: distributed traces, Prometheus metrics, structured access logs, and Go pprof endpoints. Every signal is GraphQL-aware: spans, metrics, and log fields carry the operation name, operation type, subgraph identity, and client information that matter for federated APIs.

### Why GraphQL-aware observability matters

Generic HTTP telemetry treats every GraphQL request the same. In a federated graph, the labels and spans that matter (operation, subgraph, client, fetch type) get lost. The result: long incident response times and metric dashboards that cannot answer simple questions.

#### Federated requests are invisible across services.

One query can touch dozens of subgraphs. Without correlated trace context, teams spend hours per incident scanning logs across services to find the failing one.

#### Generic HTTP metrics miss GraphQL.

Operation name, operation type, subgraph, and client are the labels that matter. Stock router metrics rarely carry any of them.

#### DIY instrumentation rots.

Building custom OTEL spans, label schemes, and exporters in every service is its own maintenance burden, and the labels drift over time as teams change.

#### High-cardinality metrics explode.

Without exclusion patterns or a cardinality limit, GraphQL operation labels can overwhelm a monitoring backend. Then teams strip the dimensions that made the metrics useful in the first place.

Cosmo Router handles all of this natively. Traces, metrics, logs, and profiles in one binary, one config.

## Cosmo Observability capabilities

01 Telemetry foundation  
02 Tracing  
03 Metrics & dashboards  
04 Logs & profiling

### Which GraphQL observability capability do you need?

| If you are… | Start here |
| --- | --- |
| Standing up GraphQL observability from scratch | [OpenTelemetry](/content/cosmo/observability/opentelemetry/index.html) |
| Debugging a slow or failing federated query in production | [Distributed Tracing](/content/cosmo/observability/distributed-tracing/index.html) |
| Understanding the exact execution plan for a specific query | [Advanced Request Tracing](/content/cosmo/observability/advanced-request-tracing/index.html) |
| Monitoring p99 latency, error rate, and request volume | [Prometheus Metrics](/content/cosmo/observability/prometheus-metrics/index.html) |
| Want production dashboards working in under 30 minutes | [Grafana Integration](/content/cosmo/observability/grafana-integration/index.html) |
| Routing traces and metrics to multiple backends from one place | [OTEL Collector Integration](/content/cosmo/observability/otel-collector-integration/index.html) |
| Capturing per-request logs with GraphQL operation context | [Access Logs](/content/cosmo/observability/access-logs/index.html) |
| Diagnosing CPU hotspots, memory leaks, or goroutine blocks | [Profiling (pprof)](/content/cosmo/observability/profiling/index.html) |

## How Cosmo Observability compares

|  | Cosmo Observability | Apollo Router | DIY instrumentation |
| --- | --- | --- | --- |
| OpenTelemetry native | Yes | Partial | Manual |
| GraphQL-aware metric dimensions | Native | Limited | Manual |
| Multi-exporter | Yes | Limited | Custom |
| Built-in cardinality controls | Yes (2000 / metric default) | Manual | Manual |
| Pre-built Grafana dashboards | [Cache, Go runtime](/content/cosmo/observability/grafana-integration/index.html) | N/A | Self-built |
| Go pprof exposed | Yes (on-demand) | N/A (Rust) | Varies |

## GraphQL observability use cases

### Incident response

#### Checkout starts returning errors at peak traffic

##### Scenario
A critical checkout API returns intermittent errors during peak load. The team needs to find the failing subgraph fast.

##### How Cosmo handles it
Filter traces in Cosmo Studio by error status. The span tree shows the inventory subgraph timing out, and the span details carry the error message, extension codes, and stack trace. The auto-refreshing trace view updates every 10 seconds while the incident is live.

##### Outcome
Root cause identified in five minutes instead of two hours: database connection exhaustion in the inventory subgraph.

### SLO monitoring

#### Track p99 latency for a critical operation against an SLO

##### Scenario
The platform team needs an alert when p99 latency on the most important GraphQL operation drifts above its SLO budget.

##### How Cosmo handles it
Query `router_http_request_duration_milliseconds` with `histogram_quantile()`, filtered by the `wg_operation_name` label. Wire the result into Alertmanager.

##### Outcome
Automated alerting fires the moment p99 latency exceeds the SLO threshold. No custom instrumentation, no schema changes.

### Multi-backend

#### Send telemetry to Cosmo Cloud and an existing observability stack

##### Scenario
An organization uses Cosmo Cloud for GraphQL analytics and Datadog for company-wide dashboards. They don't want a duplicate exporter in every service.

##### How Cosmo handles it
Configure two exporters in the router, or run an OpenTelemetry Collector as a single intermediary with two pipelines: one to Cosmo Cloud, one to Datadog. Either path needs zero application code changes.

##### Outcome
One router configuration. Data flows to both platforms automatically. Credentials and protocol translation stay centralized.

### Performance

#### Diagnose memory growth in a long-running router

##### Scenario
Router instances grow heap usage over several days and need periodic restarts. Metrics show the symptom; the team needs the cause.

##### How Cosmo handles it
Enable pprof with `PPROF_ADDR=:6060`, capture heap profiles at intervals, and compare them with `go tool pprof` in diff mode to see which allocations grow.

##### Outcome
A subscription handler not releasing resources is identified directly from the heap profile. The fix eliminates the memory growth pattern.

## Why teams run Cosmo Observability

- **OpenTelemetry-native, no vendor lock-in.** Native OTEL SDK, W3C Trace Context propagation, export over HTTP or gRPC to any OTEL-compatible backend.
- **GraphQL-aware dimensions on every signal.** Operation name, operation type, subgraph, and client labels on metrics. Span attributes carry the same context. Access logs include operation hash and per-stage timing.
- **One config exports to many backends.** Configure multiple exporters in the router, or run one OTEL Collector pipeline that fans out to Cosmo Cloud, Jaeger, Prometheus, Datadog, and beyond.

## Observability FAQ

Common questions about traces, metrics, logs, and profiling on the Cosmo Router.

- What does observability mean in Cosmo Router?
- Is OpenTelemetry supported natively?
- Can I send telemetry to multiple backends?
- What Prometheus metrics does the router expose?
- How do I debug a slow federated query?
- Is observability available on the free plan?
- How do I diagnose CPU hotspots, memory leaks, or deadlocks?

## Get started

### Run federated GraphQL with full observability on the Cosmo Router.