Model Schemas

Model schemas gives developers control over a few key aspects of Puzzlet:

1. Model Availability: Define which models (both built-in and custom) are available in your Puzzlet dashboard

2. User Interface Control: Precisely specify which parameters platform users can modify through the UI, while keeping sensitive or advanced configurations under developer control

3. Cost Customization: Customize the costs for your custom models. These allow you to see logs/metrics/traces for your custom models in Puzzlet.

​

Providers CLI

Puzzlet doesn’t provide any models out of the box, but we do expose some managed providers through a CLI.

The providers CLI command is used to easily list, pull, and configure managed providers in your platform locally with their model schemas.

​

Listing Providers

npx @puzzlet/cli@latest providers list

Example output:

openai
anthropic
...

​

Getting Provider Info

npx @puzzlet/cli@latest providers info openai

Example output:

Provider: openai
Models: gpt-4, gpt-3.5-turbo, ...

​

Pulling Schemas

npx @puzzlet/cli@latest providers pull openai,anthropic

Example output:

Pulling schemas for openai...
Pulling schemas for anthropic...

​

Model Schema Configuration

Each model schema allows you to:

• Set a user-friendly display label
• Configure available settings and their UI controls
• Define parameter constraints and defaults
• Control the order and presentation of settings

Here’s an example of a model schema:

{
  ...,
  "modelSchemas": {
    "gpt-4": {
      "label": "GPT-4",
      "settings": {
        "temperature": {
          "minimum": 0,
          "maximum": 1,
          "default": 0.7,
          "multipleOf": 0.01,
          "label": "Temperature",
          "order": 2,
          "type": "number",
          "ui": "slider"
        },
        "max_tokens": {
          "minimum": 16,
          "default": 4096,
          "maximum": 4096,
          "multipleOf": 1,
          "label": "Max Tokens",
          "order": 3,
          "type": "number",
          "ui": "slider"
        }
      }
    }
  }
}

​

Multiple Models

You can define multiple models in your schema, each with its own configuration:

{
  "modelSchemas": {
    "gpt-4": {
      "label": "GPT-4",
      "settings": { ... }
    },
    "gpt-3.5-turbo": {
      "label": "GPT-3.5 Turbo",
      "settings": { ... }
    },
    "custom-model": {
      "label": "My Custom Model",
      "settings": { ... }
    }
  }
}

​

Setting Properties

Each setting in a model schema can define:

• minimum/maximum: The allowed range for numeric values
• default: The initial value
• multipleOf: Step size for numeric inputs
• label: Display name in the UI
• order: Control the display order in forms
• type: The type of the setting (e.g. “number”, “string”, “boolean”)
• ui: The UI control to use for the setting (e.g. “slider”…more to come)

​

Custom Model Schemas

Puzzlet is fully extensible, so you can define your own model schemas.

{
  "modelSchemas": {
    "my-custom-model": {
      "label": "GPT-4",
      "settings": { ... }
    },
  }
}

​

Customizing Costs

You can also customize the costs for your custom models. These allow you to see logs/metrics/traces for your custom models in Puzzlet.

{
  "modelSchemas": {
    "my-custom-model": {
      "label": "My Custom Model",
      "cost": {
        "inputCost": 0.01, // $0.01 per 1000 tokens
        "outputCost": 0.02, // $0.02 per 1000 tokens
        "unitScale": 1000
      }
    }
  }
}