Tracing

A user clicks a button. Your API receives the request, queries a database, calls a cache, fans out to two downstream services, and returns a response. Distributed tracing captures that full journey as a tree of spans so you can see exactly where time is spent and where failures originate.

JuhJuh collects traces from your services, auto-discovers the services involved, maps their dependencies, and links traces to logs and metrics for cross-signal investigation.

Core concepts¶

Traces¶

A trace represents a single end-to-end operation, like an HTTP request or a background job execution. Every trace has a unique trace_id that ties all its parts together across services.

Spans¶

A span is one unit of work within a trace. A trace is a tree of spans. The root span is the entry point (your API receiving a request), and child spans represent downstream operations (database queries, cache lookups, HTTP calls to other services).

Each span records:

• Operation name: what happened (GET /api/users, SELECT users, redis.get)
• Service name: which service performed the work
• Duration: how long it took
• Status: whether it succeeded or failed
• Attributes: additional context (HTTP method, URL, status code, database statement)

Service map¶

JuhJuh automatically builds a dependency map from your trace data. When service A calls service B, and that relationship appears in span parent-child links, JuhJuh records the dependency and displays it as a directed graph in the service map view.

graph LR
    GW[API Gateway] --> US[User Service]
    GW --> PS[Payment Service]
    US --> DB[(Database)]
    US --> Cache[(Cache)]
    PS --> DB
    PS --> Ext[External Payment API]

Sending traces¶

Send traces to either endpoint:

POST /api/ingest/traces/
POST /api/ingest/v1/traces/
Authorization: Bearer jjk_YOUR_API_KEY
Content-Type: application/json

Both endpoints accept the same payloads. The /v1/traces/ path follows the OTLP standard convention.

The endpoint auto-detects the format. If the top-level key is resourceSpans, it parses OTLP. If the key is spans, it parses simple JSON.

Simple JSON format¶

{
  "spans": [
    {
      "trace_id": "abc123def456789",
      "span_id": "span_001",
      "parent_span_id": "",
      "operation_name": "GET /api/users",
      "service_name": "api-gateway",
      "span_kind": "server",
      "status_code": "ok",
      "duration_ms": 142,
      "started_at": "2026-04-09T14:30:00Z",
      "http_method": "GET",
      "http_url": "/api/users",
      "http_status_code": 200,
      "attributes": {
        "user.id": "usr_12345"
      },
      "events": []
    },
    {
      "trace_id": "abc123def456789",
      "span_id": "span_002",
      "parent_span_id": "span_001",
      "operation_name": "SELECT users",
      "service_name": "user-service",
      "span_kind": "client",
      "status_code": "ok",
      "duration_ms": 23,
      "started_at": "2026-04-09T14:30:00.050Z",
      "attributes": {
        "db.system": "postgresql",
        "db.statement": "SELECT * FROM users WHERE id = $1"
      }
    }
  ]
}

OTLP format¶

OpenTelemetry SDKs send traces in OTLP format automatically:

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          { "key": "service.name", "value": { "stringValue": "api-gateway" } }
        ]
      },
      "scopeSpans": [
        {
          "spans": [
            {
              "traceId": "abc123def456789",
              "spanId": "span_001",
              "parentSpanId": "",
              "name": "GET /api/users",
              "kind": 2,
              "startTimeUnixNano": "1712672400000000000",
              "endTimeUnixNano": "1712672400142000000",
              "status": { "code": 1 },
              "attributes": [
                { "key": "http.method", "value": { "stringValue": "GET" } },
                { "key": "http.url", "value": { "stringValue": "/api/users" } },
                { "key": "http.status_code", "value": { "intValue": "200" } }
              ]
            }
          ]
        }
      ]
    }
  ]
}

JuhJuh extracts standard HTTP and database attributes automatically:

Span kinds¶

Span kind categorizes what role the span plays in the trace:

Server and client spans are the most common. They form the cross-service links that build your service dependency map.

Service auto-discovery¶

JuhJuh automatically registers every unique service_name it encounters in your trace data. For each discovered service, JuhJuh tracks:

No manual registration needed. Send traces and your services appear in the service list.

Service dependency map¶

When a span from service A is the parent of a span from service B, JuhJuh records that dependency. Over time, this builds a complete map of how your services communicate.

The dependency map shows:

• Directed edges between services (who calls whom)
• Request volume on each edge
• Error rates on each connection
• Latency percentiles per dependency

Use the map to identify:

• Single points of failure: services that many others depend on
• Error propagation: a failing downstream service causing errors upstream
• Latency bottlenecks: a slow dependency dragging down overall response times

Viewing traces¶

Trace list¶

The trace list shows recent traces with filtering options:

Trace detail¶

Click a trace to see the full span tree. The waterfall view shows:

• Each span as a horizontal bar, positioned by start time and sized by duration
• Parent-child relationships as nesting
• Service name, operation, and duration for each span
• Status indicators (green for ok, red for error)
• Click any span to see its attributes, events, and linked logs

Trace-to-log correlation¶

If your logs include a trace_id attribute (see Logging), the trace detail view shows all matching log entries alongside the span tree. This gives you both the timing view (trace) and the narrative view (logs) for the same request.

Batching¶

Send multiple spans in a single request:

{
  "spans": [
    { "trace_id": "abc123", "span_id": "s1", "operation_name": "GET /", "service_name": "web" },
    { "trace_id": "abc123", "span_id": "s2", "parent_span_id": "s1", "operation_name": "SELECT", "service_name": "db" },
    { "trace_id": "def456", "span_id": "s3", "operation_name": "POST /pay", "service_name": "payments" }
  ]
}

Spans from different traces can be mixed in the same request. JuhJuh groups them by trace_id during processing.

Frontend-to-backend tracing¶

Real User Monitoring connects browser-side errors to server-side traces using W3C Trace Context headers. When the RUM SDK detects a same-origin request, it injects a traceparent header. If your backend propagates that trace ID, JuhJuh links the frontend session to the backend trace.

Start from a user's session, find a slow page load, and follow it through to the exact database query that caused the delay.

Data retention¶

Traces and spans are retained for 14 days. After the retention period, trace data is permanently deleted in daily cleanup cycles.

Example: manual span creation in Python¶

Create spans manually without auto-instrumentation:

from opentelemetry import trace

tracer = trace.get_tracer("my-service")

def handle_request(request):
    with tracer.start_as_current_span("handle_request", kind=trace.SpanKind.SERVER) as span:
        span.set_attribute("http.method", request.method)
        span.set_attribute("http.url", request.path)

        with tracer.start_as_current_span("query_database", kind=trace.SpanKind.CLIENT) as db_span:
            db_span.set_attribute("db.system", "postgresql")
            db_span.set_attribute("db.statement", "SELECT * FROM users")
            result = database.query("SELECT * FROM users")

        span.set_attribute("http.status_code", 200)
        return result

The OpenTelemetry SDK handles trace ID generation, parent-child linking, and export to JuhJuh automatically. See Instrumentation for full SDK setup.

Related¶

• Observability for an overview of all observability signals
• Instrumentation for SDK setup and connecting your application
• Logging for trace-to-log correlation with trace IDs
• Metrics for metric-to-trace exemplar linking
• Alerts for notifications on error rate spikes in traces
• Real User Monitoring for frontend-to-backend trace correlation