Overview

Puzzlet builds on top of OpenTelemetry for collecting telemetry data from your prompts. This helps you monitor, debug, and optimize your LLM applications in production.

​

What We Track

Puzzlet automatically collects:

• Inference Spans: Full lifecycle of prompt execution

  • Token usage
  • Response times
  • Model information
  • Completion status
  • Cost
  • Response status

• Tool Calls: When your prompts use tools

  • Tool name and parameters
  • Execution duration
  • Success/failure status
  • Return values

• Streaming Metrics: For streaming responses

  • Time to first token
  • Tokens per second
  • Total streaming duration

​

Basic Usage

Enable telemetry in your Puzzlet client:

const prompt = puzzletClient.fetchPrompt('my-prompt.prompt.mdx');
const props = { num: 1 };
const result = runInference(prompt, props, {
  telemetry: {
    enabled: true,
    metadata: {
      userId: '1234567890'
    }
  }
});

​

Learn More

For detailed information about spans, metrics, and custom configuration, see:

• Spans & Traces
• Metrics