Puzzlet allows you to test your inference directly in your application by setting up a webhook endpoint. This endpoint receives test requests from Puzzlet and returns the inference results back to the platform. All inference results must be returned in Agentmark format.

​
Setting Up Your Webhook

​
1. Get Your Webhook Credentials

  1. Navigate to the Settings page in your Puzzlet dashboard
  2. Under “Webhook Url”, you’ll find:
    • A webhook URL input field
    • A webhook secret input field. This must match the secret in your test endpoint.

Keep your webhook secret secure! This is used to verify that requests are coming from Puzzlet.

​
2. Create Your Webhook Endpoint

Here’s an example of setting up a webhook endpoint using Next.js:

NextJs
import { runInference, ModelPluginRegistry, getFrontMatter } from "@puzzlet/agentmark";
import { verifySignature } from "@puzzlet/shared-utils";
import { NextRequest, NextResponse } from "next/server";
import { Puzzlet, trace } from "@puzzlet/sdk";
import AllModels from "@puzzlet/all-models";
import { createTemplateRunner } from "@puzzlet/agentmark";

export const dynamic = 'force-dynamic'

// Register all available models
ModelPluginRegistry.registerAll(AllModels);

// Initialize Puzzlet client for dataset runs
const puzzlet = new Puzzlet(
  {
    apiKey: process.env.PUZZLET_API_KEY!,
    appId: process.env.PUZZLET_APP_ID!,
    baseUrl: process.env.PUZZLET_BASE_URL,
  },
  (ast) => createTemplateRunner(ast)
);

// Start tracing for dataset runs
// Disable batching to ensure that the traces are sent immediately
puzzlet.initTracing({disableBatch: true});

export async function POST(request: NextRequest) {
  const payload = await request.json();
  const headers = request.headers;

  // Get the Puzzlet signature from headers
  const xPuzzletSign = headers.get("x-puzzlet-signature-256");

  // Verify the signature exists
  if (!xPuzzletSign) {
    return NextResponse.json(
      { message: "Missing x-puzzlet-signature-256 header" },
      { status: 400 }
    );
  }

  try {
    // Verify the request is from Puzzlet
    if (
      !(await verifySignature(
        process.env.PUZZLET_TEST_API_SECRET!,
        xPuzzletSign,
        JSON.stringify(payload)
      ))
    ) {
      return NextResponse.json(
        { message: "Invalid signature" },
        { status: 401 }
      );
    }

    const { event } = payload;

    if (event.type === "prompt-run") {
      // This event is triggered when a prompt is tested/run via puzzlet. 
      // The results returned are used to display in the Prompt "Output" section within the dashboard
      const frontmatter = getFrontMatter(event.data.prompt) as any;
      const props = frontmatter.test_settings?.props || {};

      // Run the inference
      const output = await runInference(event.data.prompt, props, {
        telemetry: { isEnabled: false },
      });
      return NextResponse.json(output);
    }

    if (event.type === "dataset-run") {
      // This event is triggered when a dataset is run via Puzzlet. 
      // Specific id's are generated and passed in which create the proper associations for traces/logs/etc.
      const { promptPath, dataset, runId, runName, datasetPath } = event.data;
      const prompt = await puzzlet.fetchPrompt(promptPath);

      // Run each dataset item
      await Promise.all(
        dataset.items.map((item) =>
          trace(`ds-run-${runName}-${item.name}`, () =>
            prompt.run(item.input, {
              telemetry: {
                isEnabled: true,
                metadata: {
                  dataset_run_id: runId,
                  dataset_path: datasetPath,
                  dataset_run_name: runName,
                  dataset_item_name: item.name,
                },
              },
            })
          )
        )
      );

      return NextResponse.json({ message: "ok" });
    }

    return NextResponse.json(
      { message: "Unknown event type" },
      { status: 400 }
    );
  } catch (error) {
    return NextResponse.json(
      { message: "Internal server error" },
      { status: 500 }
    );
  }
}

​
Security Considerations

  1. Signature Verification: Always verify the x-puzzlet-signature-256 header using your webhook secret
  2. Environment Variables: Store your webhook secret as an environment variable
  3. Error Handling: Implement proper error handling and logging

​
Webhook Request Format

Puzzlet sends POST requests to your webhook URL with the following:

​
Headers

{
  "x-puzzlet-signature-256": "signature-hash",
  "content-type": "application/json"
}

​
Body

​
Prompt Run

{
  "event": {
    "data": {
      "prompt": {
        ...A prompt in Agentmark format
      },
    },
    "type": "prompt-run",
  },
}

​
Dataset Run

{
  "event": {
    "data": {
      "promptPath": "path/to/prompt",
      "dataset": {
        ... The dataset object to test against
      },
      "datasetPath": "path/to/dataset",
      "runId": "run-id",
      "runName": "run-name",
    }
  }
}

​
Testing Your Webhook

  1. Set up your webhook endpoint
  2. Add your webhook URL and secret in Puzzlet settings
  3. Create a prompt in Puzzlet
  4. Click “Run” in the prompt to send a request to your webhook
  5. View the results in Puzzlet’s dashboard

Make sure your webhook endpoint is publicly accessible and can handle POST requests.

​
Troubleshooting

  • Verify your webhook URL is correct and accessible
  • Check that your webhook secret matches the one in Puzzlet
  • Ensure proper error handling in your endpoint
  • Make sure your returning a JSON response that matches whats provided by Agentmark

Was this page helpful?

Model SchemasOverview