Databricks - Portkey Docs

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Databricks Model Serving endpoints. With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

Provider Slug: databricks

Quick Start

Get Databricks working in 3 steps:

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @databricks provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @databricks provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @databricks provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @databricks provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

# 1. Add @databricks provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'

Tip: You can also set provider="@databricks" in Portkey() and use just model="databricks-meta-llama-3-1-70b-instruct" in the request.

Add Provider in Model Catalog

Go to Model Catalog → Add Provider
Select Databricks
Choose existing credentials or create new by entering your Databricks personal access token and workspace name
Name your provider (e.g., databricks-prod)

Configuration Parameters

Parameter	Description	Required
`apiKey`	Databricks personal access token	Yes
`databricksWorkspace`	Databricks workspace name (used to construct the URL: `https://<workspace>.cloud.databricks.com`)	Yes

When using headers directly, pass the workspace name via the x-portkey-databricks-workspace header.

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

Supported Endpoints

Endpoint	Support
`/chat/completions`	Supported
`/completions`	Supported
`/embeddings`	Supported
`/responses`	Supported
`/messages`	Supported

Supported Features

Feature	Support
Thinking/Reasoning	Supported via `thinking` and `reasoning_effort` parameters
Streaming	Supported

Embeddings

Generate embeddings using Databricks-hosted embedding models:

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.embeddings.create(
    model="@databricks/databricks-bge-large-en",
    input="The quick brown fox jumps over the lazy dog"
)

print(response.data[0].embedding[:5])

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.embeddings.create({
    model: "@databricks/databricks-bge-large-en",
    input: "The quick brown fox jumps over the lazy dog"
})

console.log(response.data[0].embedding.slice(0, 5))

curl https://api.portkey.ai/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-bge-large-en",
    "input": "The quick brown fox jumps over the lazy dog"
  }'

Responses API

Use the OpenAI Responses API format with Databricks models:

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.responses.create(
    model="@databricks/databricks-meta-llama-3-1-70b-instruct",
    input="Tell me a three sentence bedtime story about a unicorn."
)

print(response)

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.responses.create({
    model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
    input: "Tell me a three sentence bedtime story about a unicorn."
})

console.log(response)

curl https://api.portkey.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
    "input": "Tell me a three sentence bedtime story about a unicorn."
  }'

Messages API

Use the Anthropic Messages API format with Databricks models:

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.with_options(
    headers={"x-portkey-anthropic-version": "2023-06-01"}
).post(
    "/v1/messages",
    body={
        "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
        "max_tokens": 250,
        "messages": [{"role": "user", "content": "Say this is a test"}]
    },
    cast_to=object
)

print(response)

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.post("/v1/messages", {
    body: {
        model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
        max_tokens: 250,
        messages: [{ role: "user", content: "Say this is a test" }]
    },
    headers: { "x-portkey-anthropic-version": "2023-06-01" }
})

console.log(response)

curl https://api.portkey.ai/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
    "max_tokens": 250,
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'

Supported Models

Databricks Model Serving supports a variety of foundation models and custom endpoints:

Meta Llama Models: databricks-meta-llama-3-1-70b-instruct, databricks-meta-llama-3-1-405b-instruct
DBRX: databricks-dbrx-instruct
Embedding Models: databricks-bge-large-en, databricks-gte-large-en
Custom Endpoints: Any model deployed as a Databricks serving endpoint

For the complete list of available models, refer to the Databricks Foundation Model APIs documentation.

Next Steps

Add Metadata

Add metadata to your Databricks requests

Gateway Configs

Add gateway configs to your Databricks requests

Tracing

Trace your Databricks requests

Fallbacks

Setup fallback strategies with Databricks

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation

​Quick Start

​Add Provider in Model Catalog

​Configuration Parameters

Complete Setup Guide →

​Supported Endpoints

​Supported Features

​Embeddings

​Responses API

​Messages API

​Supported Models

​Next Steps

Add Metadata

Gateway Configs

Tracing

Fallbacks

SDK Reference

Quick Start

Add Provider in Model Catalog

Configuration Parameters

Supported Endpoints

Supported Features

Embeddings

Responses API

Messages API

Supported Models

Next Steps