Puzzlet allows you to test your inference directly in your application by setting up a webhook endpoint. This endpoint receives test requests from Puzzlet and returns the inference results back to the platform. All inference results must be returned in Agentmark format.

Setting Up Your Webhook

1. Get Your Webhook Credentials

  1. Navigate to the Settings page in your Puzzlet dashboard
  2. Under “Webhook Url”, you’ll find:
    • A webhook URL input field
    • A webhook secret input field. This must match the secret in your test endpoint.

Keep your webhook secret secure! This is used to verify that requests are coming from Puzzlet.

2. Create Your Webhook Endpoint

Here’s an example of setting up a webhook endpoint using Next.js:

NextJs
import { runInference, ModelPluginRegistry, getFrontMatter } from "@puzzlet/agentmark";
import { verifySignature } from "@puzzlet/shared-utils";
import { NextRequest, NextResponse } from "next/server";
import { Puzzlet, trace } from "@puzzlet/sdk";
import AllModels from "@puzzlet/all-models";
import { createTemplateRunner } from "@puzzlet/agentmark";

export const dynamic = 'force-dynamic'

// Register all available models
ModelPluginRegistry.registerAll(AllModels);

// Initialize Puzzlet client for dataset runs
const puzzlet = new Puzzlet(
  {
    apiKey: process.env.PUZZLET_API_KEY!,
    appId: process.env.PUZZLET_APP_ID!,
    baseUrl: process.env.PUZZLET_BASE_URL,
  },
  (ast) => createTemplateRunner(ast)
);

// Start tracing for dataset runs
// Disable batching to ensure that the traces are sent immediately
puzzlet.initTracing({disableBatch: true});

export async function POST(request: NextRequest) {
  const payload = await request.json();
  const headers = request.headers;

  // Get the Puzzlet signature from headers
  const xPuzzletSign = headers.get("x-puzzlet-signature-256");

  // Verify the signature exists
  if (!xPuzzletSign) {
    return NextResponse.json(
      { message: "Missing x-puzzlet-signature-256 header" },
      { status: 400 }
    );
  }

  try {
    // Verify the request is from Puzzlet
    if (
      !(await verifySignature(
        process.env.PUZZLET_TEST_API_SECRET!,
        xPuzzletSign,
        JSON.stringify(payload)
      ))
    ) {
      return NextResponse.json(
        { message: "Invalid signature" },
        { status: 401 }
      );
    }

    const { event } = payload;

    if (event.type === "prompt-run") {
      // This event is triggered when a prompt is tested/run via puzzlet. 
      // The results returned are used to display in the Prompt "Output" section within the dashboard
      const frontmatter = getFrontMatter(event.data.prompt) as any;
      const props = frontmatter.test_settings?.props || {};

      // Run the inference
      const output = await runInference(event.data.prompt, props, {
        telemetry: { isEnabled: false },
      });
      return NextResponse.json(output);
    }

    if (event.type === "dataset-run") {
      // This event is triggered when a dataset is run via Puzzlet. 
      // Specific id's are generated and passed in which create the proper associations for traces/logs/etc.
      const { promptPath, dataset, runId, runName, datasetPath } = event.data;
      const prompt = await puzzlet.fetchPrompt(promptPath);

      // Run each dataset item
      await Promise.all(
        dataset.items.map((item) =>
          trace(`ds-run-${runName}-${item.name}`, () =>
            prompt.run(item.input, {
              telemetry: {
                isEnabled: true,
                metadata: {
                  dataset_run_id: runId,
                  dataset_path: datasetPath,
                  dataset_run_name: runName,
                  dataset_item_name: item.name,
                },
              },
            })
          )
        )
      );

      return NextResponse.json({ message: "ok" });
    }

    return NextResponse.json(
      { message: "Unknown event type" },
      { status: 400 }
    );
  } catch (error) {
    return NextResponse.json(
      { message: "Internal server error" },
      { status: 500 }
    );
  }
}

Security Considerations

  1. Signature Verification: Always verify the x-puzzlet-signature-256 header using your webhook secret
  2. Environment Variables: Store your webhook secret as an environment variable
  3. Error Handling: Implement proper error handling and logging

Webhook Request Format

Puzzlet sends POST requests to your webhook URL with the following:

Headers

{
  "x-puzzlet-signature-256": "signature-hash",
  "content-type": "application/json"
}

Body

Prompt Run

{
  "event": {
    "data": {
      "prompt": {
        ...A prompt in Agentmark format
      },
    },
    "type": "prompt-run",
  },
}

Dataset Run

{
  "event": {
    "data": {
      "promptPath": "path/to/prompt",
      "dataset": {
        ... The dataset object to test against
      },
      "datasetPath": "path/to/dataset",
      "runId": "run-id",
      "runName": "run-name",
    }
  }
}

Testing Your Webhook

  1. Set up your webhook endpoint
  2. Add your webhook URL and secret in Puzzlet settings
  3. Create a prompt in Puzzlet
  4. Click “Run” in the prompt to send a request to your webhook
  5. View the results in Puzzlet’s dashboard

Make sure your webhook endpoint is publicly accessible and can handle POST requests.

Troubleshooting

  • Verify your webhook URL is correct and accessible
  • Check that your webhook secret matches the one in Puzzlet
  • Ensure proper error handling in your endpoint
  • Make sure your returning a JSON response that matches whats provided by Agentmark