> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# OpenAI
> Trace OpenAI SDK calls, evaluate GPT models, and route them through the Braintrust gateway
If you are a coding agent, prefer the Braintrust [`bt` CLI](/docs/reference/cli/quickstart) for repeatable, scriptable work: running evals, instrumenting code, querying logs, syncing data, managing functions, and configuring coding agents. Use the MCP server for reasoning over Braintrust data in conversation, such as ad-hoc lookups and exploration from your IDE.
Braintrust integrates with [OpenAI](https://openai.com) so you can call GPT models from the Braintrust playground, API, and SDKs. Braintrust also traces OpenAI SDK calls from your application, including streaming, structured outputs, and function calling.
## Add OpenAI as an AI provider
To use OpenAI models in the Braintrust playground, API, and gateway, connect OpenAI as a provider in your organization or project AI providers.
1. Go to ** Settings** > [** AI providers**](https://www.braintrust.dev/app/~/configuration/org/secrets).
2. Click **Organization provider** or **Project provider**, depending on whether you want the provider to be available across every project in the organization or just the current project.
3. Under **Model providers**, click **OpenAI**.
4. Choose your authentication method:
* **API key**: Visit [OpenAI's API platform](https://platform.openai.com/api-keys), create a new API key, and paste it into the **Secret** field.
API keys are stored as one-way cryptographic hashes, never in plaintext.
* **Workload identity federation**: Exchange a Braintrust-signed OIDC token for an OpenAI access token, instead of storing a long-lived OpenAI API key in Braintrust.
Workload identity federation is available only for organization-level providers on Braintrust-hosted organizations with the Braintrust gateway enabled. Project-level providers and self-hosted deployments must use **API key** authentication.
5. If you chose **Workload identity federation**, use the setup values shown in Braintrust to configure OpenAI:
1. [Create a workload identity provider](https://developers.openai.com/api/docs/guides/workload-identity-federation) in OpenAI. Enter a descriptive **Name**, use the **OIDC issuer URL** and **Audience** shown in Braintrust, and leave uploaded JWKS and attribute transformations disabled.
2. From the workload identity provider details page, create a mapping. Use `sub` as the **Key** and the subject pattern shown in Braintrust as the **Value**. Add a mapping attribute for each additional claim shown in Braintrust. Choose the OpenAI **Project**, **Service account**, and **Permissions** Braintrust should use.
3. Paste the OpenAI IDs back into Braintrust:
* **Identity provider ID**: The workload identity provider ID configured for Braintrust.
* **Service account ID**: The OpenAI service account ID Braintrust should use.
* **Subject suffix**: A stable suffix for this OpenAI connection. It must match the final part of the subject pattern used in OpenAI.
For general OpenAI concepts and dashboard details, see [OpenAI's workload identity federation docs](https://developers.openai.com/api/docs/guides/workload-identity-federation).
6. Click **Save**.
For the GPT-5 family, whether the `temperature` parameter is configurable depends on the reasoning effort. GPT-5.1 and later accept `temperature` only when reasoning effort is set to `none`. At any higher reasoning effort, and for older GPT-5 models (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) and GPT-5 Pro, `temperature` isn't configurable and is disabled in the Braintrust UI.
Tracing
Braintrust traces OpenAI calls automatically with the `braintrust/hook.mjs` import hook, or manually with `wrapOpenAI`. Either path produces the same spans. Auto-instrumentation is the recommended path for most users.
Using the OpenAI Agents SDK? See the [OpenAI Agents SDK](/docs/integrations/agent-frameworks/openai-agents-sdk) framework docs.
Setup
Install the Braintrust SDK alongside the OpenAI SDK, then configure your API keys.
```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add braintrust openai
```
```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install braintrust openai
```
Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).
```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=
BRAINTRUST_API_KEY=
# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=
```
Auto-instrumentation
To trace OpenAI calls without modifying your application code, initialize Braintrust, create a normal OpenAI client, then run your app with Braintrust's import hook to patch the OpenAI SDK at startup.
```javascript title="trace-openai-auto.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import OpenAI from "openai";
import { initLogger } from "braintrust";
initLogger({
projectName: "My Project", // Replace with your project name
apiKey: process.env.BRAINTRUST_API_KEY,
});
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const result = await client.chat.completions.create({
model: "gpt-5-mini",
messages: [{ role: "user", content: "What is machine learning?" }],
});
```
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
node --import braintrust/hook.mjs trace-openai-auto.js
```
The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects — compile your TypeScript to JavaScript, then run the compiled file with the import hook.
If you're using a bundler, see [Trace LLM calls](/docs/instrument/trace-llm-calls#auto-instrumentation) for plugin and loader setup.
Manual instrumentation
To trace OpenAI calls manually, wrap your client with `wrapOpenAI` yourself. Once wrapped, every `chat.completions.create` call (including streaming) emits a span.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import OpenAI from "openai";
// Initialize the Braintrust logger
const logger = initLogger({
projectName: "My Project", // Your project name
apiKey: process.env.BRAINTRUST_API_KEY,
});
// Wrap the OpenAI client with wrapOpenAI
const client = wrapOpenAI(
new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
}),
);
// All API calls are automatically logged
const result = await client.chat.completions.create({
model: "gpt-5-mini",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "What is machine learning?" },
],
});
```
For more control over tracing, learn how to [customize traces](/docs/instrument/advanced-tracing).
Streaming
`wrapOpenAI` can automatically log metrics like `prompt_tokens`, `completion_tokens`, and `tokens` for streaming LLM calls if the LLM API returns them. Set `include_usage` to `true` in the `stream_options` parameter to receive these metrics from OpenAI.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
model: "gpt-5-mini",
messages: [{ role: "user", content: "Count to 10" }],
stream: true,
stream_options: {
include_usage: true, // Required for token metrics
},
});
for await (const chunk of result) {
process.stdout.write(chunk.choices[0]?.delta?.content || "");
}
```
Structured outputs
OpenAI's structured outputs are supported with the wrapper functions.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { z } from "zod";
// Define a Zod schema for the response
const ResponseSchema = z.object({
name: z.string(),
age: z.number(),
});
const completion = await client.beta.chat.completions.parse({
model: "gpt-5-mini",
messages: [
{ role: "system", content: "Extract the person's name and age." },
{ role: "user", content: "My name is John and I'm 30 years old." },
],
response_format: {
type: "json_schema",
json_schema: {
name: "person",
// The Zod schema for the response
schema: ResponseSchema,
},
},
});
```
Function calling and tools
Braintrust supports OpenAI function calling for building AI agents with tools.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const tools = [
{
type: "function" as const,
function: {
name: "get_weather",
description: "Get current weather for a location",
parameters: {
type: "object",
properties: {
location: { type: "string" },
},
required: ["location"],
},
},
},
];
const response = await client.chat.completions.create({
model: "gpt-5-mini",
messages: [{ role: "user", content: "What's the weather in San Francisco?" }],
tools,
});
```
To trace multimodal content, attachments, errors, and masking sensitive data, see the [customize traces](/docs/instrument/advanced-tracing) guide.
Gateway
To call OpenAI through the [Braintrust gateway](/docs/deploy/gateway), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](/docs/integrations/ai-providers) to call OpenAI models.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const client = new OpenAI({
baseURL: "https://gateway.braintrust.dev/v1",
apiKey: process.env.BRAINTRUST_API_KEY,
});
const response = await client.chat.completions.create({
model: "gpt-5-mini",
messages: [{ role: "user", content: "What is a proxy?" }],
seed: 1, // A seed activates the proxy's cache
});
```
The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](/docs/deploy/gateway#generate-embeddings) for an example.
What Braintrust traces
Braintrust instruments the OpenAI client and emits an LLM span per call. Chat and responses spans capture the request messages or input, the response output, and the request parameters as metadata. Embeddings and moderation spans capture their inputs and results.
**Spans**
| Span | Coverage |
| -------------------------- | ----------------------------------------------------------------------------- |
| `Chat Completion` | `chat.completions.create`, `.parse`, and `.stream` (including streaming) |
| `Embedding` | `embeddings.create` (output records the embedding length, not the raw vector) |
| `Moderation` | `moderations.create` |
| `openai.responses.create` | `responses.create` and the `responses.stream` helper (including streaming) |
| `openai.responses.parse` | `responses.parse` (structured outputs) |
| `openai.responses.compact` | `responses.compact` (OpenAI SDK v6.10.0 or later) |
**Metrics**
| Metric | Description |
| ----------------------------- | --------------------------------------------------------- |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `completion_reasoning_tokens` | Reasoning tokens |
| `time_to_first_token` | First-token latency (chat and responses spans) |
| `cached` | Whether the response was served from the Braintrust cache |
Tracing resources
* [Braintrust JavaScript SDK](https://github.com/braintrustdata/braintrust-sdk-javascript)
* [OpenAI Node SDK](https://github.com/openai/openai-node)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)
Evals
Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. Braintrust `Eval` is a simple function composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](/docs/evaluate/run-evaluations) guide.
Basic eval setup
Evaluate the outputs of OpenAI models with Braintrust.
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { Eval } from "braintrust";
import { OpenAI } from "openai";
const client = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
Eval("OpenAI Evaluation", {
// An array of user inputs and expected outputs
data: () => [
{ input: "What is 2+2?", expected: "4" },
{ input: "What is the capital of France?", expected: "Paris" },
],
task: async (input) => {
// Your OpenAI LLM call
const response = await client.chat.completions.create({
model: "gpt-5-mini",
messages: [{ role: "user", content: input }],
});
return response.choices[0].message.content;
},
scores: [
{
name: "accuracy",
// A simple scorer that returns 1 if the output matches the expected output, 0 otherwise
scorer: (args) => (args.output === args.expected ? 1 : 0),
},
],
});
```
Learn more about eval [data](/docs/annotate/datasets) and [scorers](/docs/evaluate/write-scorers).
Use OpenAI as an LLM judge
You can use OpenAI models to score the outputs of other AI systems. This example uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system.
Install the [`autoevals`](/docs/evaluate/autoevals) package to use the `LLMClassifierFromSpec` scorer.
```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add autoevals
```
```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install autoevals
```
Create a scorer that uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system. You can then include `relevanceScorer` as a scorer in your `Eval` function (see above).
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { LLMClassifierFromSpec } from "autoevals";
const relevanceScorer = LLMClassifierFromSpec("Relevance", {
choice_scores: { Relevant: 1, Irrelevant: 0 },
model: "gpt-5-mini",
use_cot: true,
});
```
Tracing
Braintrust traces OpenAI calls automatically with `auto_instrument()`, or manually with `wrap_openai`. Either path produces the same spans. Auto-instrumentation is the recommended path for most users.
Using the OpenAI Agents SDK? See the [OpenAI Agents SDK](/docs/integrations/agent-frameworks/openai-agents-sdk) framework docs.
Setup
Install the Braintrust SDK alongside the OpenAI SDK, then configure your API keys.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install braintrust openai
```
Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).
```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=
BRAINTRUST_API_KEY=
# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=
```
Auto-instrumentation
To trace OpenAI calls without modifying your client construction, call `auto_instrument()` before importing the OpenAI SDK. Braintrust patches the SDK so every call emits a span.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
import braintrust
braintrust.auto_instrument()
braintrust.init_logger(project="My Project")
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
result = client.chat.completions.create(
model="gpt-5-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is machine learning?"},
],
)
```
Manual instrumentation
To trace OpenAI calls manually, wrap your client with `wrap_openai` yourself. Once wrapped, every `chat.completions.create` call (including streaming) emits a span.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
from braintrust import init_logger, wrap_openai
from openai import OpenAI
logger = init_logger(project="My Project")
client = wrap_openai(OpenAI(api_key=os.environ["OPENAI_API_KEY"]))
# All API calls are automatically logged
result = client.chat.completions.create(
model="gpt-5-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is machine learning?"},
],
)
```
For more control over tracing, learn how to [customize traces](/docs/instrument/advanced-tracing).
Streaming
`wrap_openai` can automatically log metrics like `prompt_tokens`, `completion_tokens`, and `tokens` for streaming LLM calls if the LLM API returns them. Set `include_usage` to `true` in the `stream_options` parameter to receive these metrics from OpenAI.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
model="gpt-5-mini",
messages=[{"role": "user", "content": "Count to 10"}],
stream=True,
stream_options={"include_usage": True}, # Required for token metrics
)
for chunk in result:
print(chunk.choices[0].delta.content or "", end="")
```
Structured outputs
OpenAI's structured outputs are supported with the wrapper functions.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
from pydantic import BaseModel
class Person(BaseModel):
name: str
age: int
completion = client.beta.chat.completions.parse(
model="gpt-5-mini",
messages=[
{"role": "system", "content": "Extract the person's name and age."},
{"role": "user", "content": "My name is John and I'm 30 years old."},
],
response_format=Person,
)
```
Function calling and tools
Braintrust supports OpenAI function calling for building AI agents with tools.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"},
},
"required": ["location"],
},
},
}
]
response = client.chat.completions.create(
model="gpt-5-mini",
messages=[{"role": "user", "content": "What's the weather in San Francisco?"}],
tools=tools,
)
```
Streaming audio transcriptions
Braintrust traces streaming audio transcription calls for sync and async OpenAI clients. Each span captures the audio file as an attachment and the final transcript as the span output.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
with open("audio.m4a", "rb") as f:
stream = client.audio.transcriptions.create(
model="gpt-4o-transcribe",
file=f,
stream=True,
)
for event in stream:
if event.type == "transcript.text.delta":
print(event.delta, end="", flush=True)
```
To trace multimodal content, attachments, errors, and masking sensitive data, see the [customize traces](/docs/instrument/advanced-tracing) guide.
Gateway
To call OpenAI through the [Braintrust gateway](/docs/deploy/gateway), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](/docs/integrations/ai-providers) to call OpenAI models.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.braintrust.dev/v1",
api_key=os.environ["BRAINTRUST_API_KEY"],
)
response = client.chat.completions.create(
model="gpt-5-mini",
messages=[{"role": "user", "content": "What is a proxy?"}],
seed=1, # A seed activates the proxy's cache
)
```
The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](/docs/deploy/gateway#generate-embeddings) for an example.
What Braintrust traces
Braintrust patches the OpenAI SDK and emits an LLM span per call across the chat, responses, embeddings, moderation, audio, and image APIs. Chat and responses spans capture the request input, the response output, and the request parameters as metadata. Audio and image spans capture files as attachments. Every span records `provider: "openai"`.
**Spans**
| Span | Coverage |
| --------------------------------------------------- | ----------------------------------------------------------------------------- |
| `Chat Completion` | `chat.completions.create` and `.parse` (including streaming) |
| `openai.responses.create`, `openai.responses.parse` | Responses API, with `tool`-typed child spans for tool calls |
| `Embedding` | `embeddings.create` (output records the embedding length, not the raw vector) |
| `Moderation` | `moderations.create` |
| `Transcription`, `Translation`, `Speech` | Audio APIs (audio captured as an attachment) |
| `Image Generation`, `Image Edit`, `Image Variation` | Image APIs (images captured as attachments) |
**Metrics**
| Metric | Description |
| ----------------------------- | --------------------------------------------------------- |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `completion_reasoning_tokens` | Reasoning tokens |
| `time_to_first_token` | First-token latency (streaming) |
| `cached` | Whether the response was served from the Braintrust cache |
Tracing resources
* [Braintrust Python SDK](https://github.com/braintrustdata/braintrust-sdk-python)
* [OpenAI Python SDK](https://github.com/openai/openai-python)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)
Evals
Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. Braintrust `Eval` is a simple function composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](/docs/evaluate/run-evaluations) guide.
Basic eval setup
Evaluate the outputs of OpenAI models with Braintrust.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
from braintrust import Eval
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
def task(input):
response = client.chat.completions.create(
model="gpt-5-mini",
messages=[{"role": "user", "content": input}],
)
return response.choices[0].message.content
def accuracy_scorer(output, expected, **kwargs):
return 1 if output == expected else 0
Eval(
"OpenAI Evaluation",
data=[
{"input": "What is 2+2?", "expected": "4"},
{"input": "What is the capital of France?", "expected": "Paris"},
],
task=task,
scores=[accuracy_scorer],
)
```
Learn more about eval [data](/docs/annotate/datasets) and [scorers](/docs/evaluate/write-scorers).
Use OpenAI as an LLM judge
You can use OpenAI models to score the outputs of other AI systems. This example uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system.
Install the [`autoevals`](/docs/evaluate/autoevals) package to use the `LLMClassifierFromSpec` scorer.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install autoevals
```
Create a scorer that uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system. You can then include `relevance_scorer` as a scorer in your `Eval` function (see above).
```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
from autoevals import LLMClassifierFromSpec
relevance_scorer = LLMClassifierFromSpec(
"Relevance",
choice_scores={"Relevant": 1, "Irrelevant": 0},
model="gpt-5-mini",
use_cot=True,
)
```
Tracing
Braintrust traces OpenAI calls automatically when you load `braintrust/setup`, or manually with `Braintrust.instrument!`. Either path produces the same spans. Auto-instrumentation is the recommended path for most users.
Setup
Install the Braintrust gem alongside the OpenAI gem, then configure your API keys.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
gem install braintrust openai
```
Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).
```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=
BRAINTRUST_API_KEY=
BRAINTRUST_DEFAULT_PROJECT= # Project that spans are logged to
# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=
```
Auto-instrumentation
To trace OpenAI calls without modifying your client construction, load `braintrust/setup` early in your application. Braintrust intercepts `require "openai"` and patches the gem so every chat completion, response, and moderation call emits a span.
```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'braintrust/setup'
require 'openai'
client = OpenAI::Client.new(api_key: ENV.fetch('OPENAI_API_KEY', nil))
client.chat.completions.create(
model: 'gpt-5-mini',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is machine learning?' }
]
)
```
In a Rails app, add `gem "braintrust", require: "braintrust/setup"` to your Gemfile to enable auto-instrumentation without an explicit `require` line.
Manual instrumentation
To trace OpenAI calls manually, instrument your client with `Braintrust.instrument!` yourself. Once instrumented, every `chat.completions.create` call (including streaming) emits a span.
```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'braintrust'
require 'openai'
# Initialize Braintrust
Braintrust.init(default_project: 'My Project')
# Create OpenAI client
client = OpenAI::Client.new(api_key: ENV.fetch('OPENAI_API_KEY', nil))
# Instrument the client with Braintrust tracing
Braintrust.instrument!(:openai, target: client)
# All API calls are automatically logged
client.chat.completions.create(
model: 'gpt-5-mini',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is machine learning?' }
]
)
```
For more control over tracing, learn how to [customize traces](/docs/instrument/advanced-tracing).
Gateway
To call OpenAI through the [Braintrust gateway](/docs/deploy/gateway), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](/docs/integrations/ai-providers) to call OpenAI models.
```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'openai'
client = OpenAI::Client.new(
base_url: 'https://gateway.braintrust.dev/v1',
api_key: ENV.fetch('BRAINTRUST_API_KEY', nil)
)
client.chat.completions.create(
model: 'gpt-5-mini',
messages: [{ role: 'user', content: 'What is a proxy?' }],
seed: 1 # A seed activates the proxy's cache
)
```
The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](/docs/deploy/gateway#generate-embeddings) for an example.
What Braintrust traces
Braintrust instruments the OpenAI gem's chat, responses, and moderation APIs and emits an LLM span per call. Chat and responses spans capture the request input, the response output, and the request parameters as metadata.
**Spans**
| Span | Coverage |
| --------------------------- | ------------------------------------------------------------ |
| `Chat Completion` | `chat.completions.create` and `stream` (including streaming) |
| `openai.responses.create` | Responses API (`create` and `stream`) |
| `openai.moderations.create` | Moderations API (no token metrics) |
**Metrics**
| Metric | Description |
| ---------------------- | --------------------------------- |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `time_to_first_token` | First-token latency (streaming) |
Tracing resources
* [Braintrust Ruby SDK](https://github.com/braintrustdata/braintrust-sdk-ruby)
* [OpenAI Ruby SDK](https://github.com/openai/openai-ruby)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)
Evals
Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. The Braintrust `Eval` API is composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](/docs/evaluate/run-evaluations) guide.
Basic eval setup
Evaluate the outputs of OpenAI models with Braintrust.
```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'braintrust'
require 'openai'
Braintrust.init
client = OpenAI::Client.new(api_key: ENV.fetch('OPENAI_API_KEY', nil))
Braintrust::Eval.run(
project: 'OpenAI Evaluation',
experiment: 'basic-eval',
# An array of user inputs and expected outputs
cases: [
{ input: 'What is 2+2?', expected: '4' },
{ input: 'What is the capital of France?', expected: 'Paris' }
],
# Your OpenAI LLM call
task: lambda do |input|
response = client.chat.completions.create(
model: 'gpt-5-mini',
messages: [{ role: 'user', content: input }]
)
response.choices[0].message.content
end,
# A simple scorer that returns 1 if the output matches the expected output, 0 otherwise
scorers: [
Braintrust::Eval.scorer('accuracy') do |_input, expected, output|
output == expected ? 1.0 : 0.0
end
]
)
```
Learn more about eval [data](/docs/annotate/datasets) and [scorers](/docs/evaluate/write-scorers).
Tracing
The Braintrust Go SDK ships an OpenAI middleware that you can attach manually, or apply automatically at compile time with [Orchestrion](https://github.com/DataDog/orchestrion). Either path produces the same traces. Auto-instrumentation is the recommended path for most users.
Setup
Install the Braintrust Go SDK alongside the OpenAI Go SDK, then configure your API keys.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
go get github.com/braintrustdata/braintrust-sdk-go
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai
go get github.com/openai/openai-go
```
Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).
```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=
BRAINTRUST_API_KEY=
# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=
```
Auto-instrumentation
To trace OpenAI calls without modifying your application code, build your app with [Orchestrion](https://github.com/DataDog/orchestrion), a compile-time tool that rewrites every `openai.NewClient` call to attach Braintrust's middleware automatically.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
go install github.com/DataDog/orchestrion@latest
```
```go title="orchestrion.tool.go" #skip-compile #skip-format theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
//go:build tools
package main
import (
_ "github.com/DataDog/orchestrion"
_ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"
)
```
Orchestrion instruments every `openai.NewClient` call at build time, so no middleware wiring is needed.
```go Go #skip-compile #skip-format theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main
import (
"context"
"log"
"os"
"github.com/openai/openai-go"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/sdk/trace"
"github.com/braintrustdata/braintrust-sdk-go"
)
func main() {
ctx := context.Background()
tp := trace.NewTracerProvider()
defer tp.Shutdown(ctx)
otel.SetTracerProvider(tp)
_, err := braintrust.New(tp,
braintrust.WithProject("My Project"),
braintrust.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
)
if err != nil {
log.Fatal(err)
}
client := openai.NewClient()
result, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
Messages: []openai.ChatCompletionMessageParamUnion{
openai.SystemMessage("You are a helpful assistant."),
openai.UserMessage("What is machine learning?"),
},
Model: openai.ChatModelGPT5Mini,
})
if err != nil {
log.Fatal(err)
}
_ = result
}
```
Build with Orchestrion to enable auto-instrumentation:
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
go mod tidy
orchestrion go build -o myapp
./myapp
```
Instead of running `orchestrion go build`, you can set a `GOFLAGS` environment variable to enable Orchestrion for normal `go build` commands:
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
export GOFLAGS="-toolexec='orchestrion toolexec'"
go build ./...
```
Manual instrumentation
To trace OpenAI calls manually, attach Braintrust's tracing middleware yourself by passing `traceopenai.NewMiddleware()` as an option on `openai.NewClient`. Once attached, every `Chat.Completions.New` call (including streaming) emits a span.
```go Go theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main
import (
"context"
"log"
"os"
"github.com/openai/openai-go"
"github.com/openai/openai-go/option"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/sdk/trace"
"github.com/braintrustdata/braintrust-sdk-go"
traceopenai "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"
)
func main() {
// Set up OpenTelemetry TracerProvider
tp := trace.NewTracerProvider()
defer tp.Shutdown(context.Background())
otel.SetTracerProvider(tp)
// Initialize Braintrust client
_, err := braintrust.New(tp,
braintrust.WithProject("My Project"),
braintrust.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
)
if err != nil {
log.Fatal(err)
}
// Create OpenAI client with tracing middleware
client := openai.NewClient(
option.WithMiddleware(traceopenai.NewMiddleware()),
)
// All API calls are automatically logged
result, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
Messages: []openai.ChatCompletionMessageParamUnion{
openai.SystemMessage("You are a helpful assistant."),
openai.UserMessage("What is machine learning?"),
},
Model: openai.ChatModelGPT5Mini,
})
if err != nil {
log.Fatal(err)
}
_ = result
}
```
For more control over tracing, learn how to [customize traces](/docs/instrument/advanced-tracing).
Gateway
To call OpenAI through the [Braintrust gateway](/docs/deploy/gateway), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](/docs/integrations/ai-providers) to call OpenAI models.
```go Go theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main
import (
"context"
"log"
"os"
"github.com/openai/openai-go"
"github.com/openai/openai-go/option"
)
func main() {
ctx := context.Background()
client := openai.NewClient(
option.WithBaseURL("https://gateway.braintrust.dev/v1"),
option.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
)
response, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
Model: openai.ChatModelGPT5Mini,
Messages: []openai.ChatCompletionMessageParamUnion{
openai.UserMessage("What is a proxy?"),
},
Seed: openai.Int(1), // A seed activates the proxy's cache
})
if err != nil {
log.Fatal(err)
}
_ = response
}
```
The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](/docs/deploy/gateway#generate-embeddings) for an example.
What Braintrust traces
Braintrust's OpenAI middleware emits an LLM span per call and routes by request path, so OpenAI-compatible providers are traced the same way. Chat and responses spans capture the request input, the response output, and the request parameters as metadata.
**Spans**
| Span | Coverage |
| -------------------------- | ------------------------------------------------------ |
| `Chat Completion` | `/v1/chat/completions` (including streaming) |
| `openai.responses.create` | `/v1/responses` (including streaming) |
| `openai.embeddings.create` | `/v1/embeddings` (output records the embedding length) |
**Metrics**
| Metric | Description |
| ---------------------- | ---------------------------------------------- |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `time_to_first_token` | First-token latency (chat and responses spans) |
Tracing resources
* [Braintrust Go SDK](https://github.com/braintrustdata/braintrust-sdk-go)
* [OpenAI Go SDK](https://github.com/openai/openai-go)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)
Evals
Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. The Braintrust `Evaluator` API is composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](/docs/evaluate/run-evaluations) guide.
Basic eval setup
Evaluate the outputs of OpenAI models with Braintrust.
```go Go theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main
import (
"context"
"log"
"os"
"github.com/openai/openai-go"
"github.com/openai/openai-go/option"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/sdk/trace"
"github.com/braintrustdata/braintrust-sdk-go"
"github.com/braintrustdata/braintrust-sdk-go/eval"
traceopenai "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"
)
func main() {
ctx := context.Background()
// Set up OpenTelemetry TracerProvider
tp := trace.NewTracerProvider()
defer tp.Shutdown(ctx)
otel.SetTracerProvider(tp)
// Initialize Braintrust
bt, err := braintrust.New(tp,
braintrust.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
)
if err != nil {
log.Fatal(err)
}
// Create OpenAI client with tracing
client := openai.NewClient(
option.WithMiddleware(traceopenai.NewMiddleware()),
)
// Create evaluator
evaluator := braintrust.NewEvaluator[string, string](bt)
// Run evaluation
_, err = evaluator.Run(ctx, eval.Opts[string, string]{
Experiment: "OpenAI Evaluation",
// Dataset of user inputs and expected outputs
Dataset: eval.NewDataset([]eval.Case[string, string]{
{Input: "What is 2+2?", Expected: "4"},
{Input: "What is the capital of France?", Expected: "Paris"},
}),
// Task function with OpenAI LLM call
Task: eval.T(func(ctx context.Context, input string) (string, error) {
response, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
Model: openai.ChatModelGPT5Mini,
Messages: []openai.ChatCompletionMessageParamUnion{
openai.UserMessage(input),
},
})
if err != nil {
return "", err
}
return response.Choices[0].Message.Content, nil
}),
// Simple scorer that returns 1 if output matches expected, 0 otherwise
Scorers: []eval.Scorer[string, string]{
eval.NewScorer("accuracy", func(ctx context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
score := 0.0
if r.Output == r.Expected {
score = 1.0
}
return eval.S(score), nil
}),
},
})
if err != nil {
log.Fatal(err)
}
}
```
Learn more about eval [data](/docs/annotate/datasets) and [scorers](/docs/evaluate/write-scorers).
Tracing
The Braintrust Java SDK ships an OpenAI interceptor that you can attach manually with `BraintrustOpenAI.wrapOpenAI()`, or have applied automatically by the [Braintrust Java agent](/docs/instrument/trace-llm-calls#auto-instrumentation). Both paths produce the same spans. Auto-instrumentation is the recommended path for most users.
Setup
Install the Braintrust Java SDK alongside the OpenAI Java SDK, then configure your API keys.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# add to build.gradle dependencies{} block
implementation 'dev.braintrust:braintrust-sdk-java:'
implementation 'com.openai:openai-java:'
```
Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).
```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=
BRAINTRUST_API_KEY=
# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=
```
Auto-instrumentation
To trace OpenAI calls without modifying your application code, attach the [`braintrust-java-agent`](/docs/instrument/trace-llm-calls#auto-instrumentation) at JVM startup. The agent intercepts every OpenAI client build and applies the Braintrust interceptor automatically.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# build.gradle
configurations {
btAgent
}
dependencies {
btAgent 'dev.braintrust:braintrust-java-agent:'
}
tasks.withType(JavaExec).configureEach {
jvmArgs "-javaagent:${configurations.btAgent.singleFile.absolutePath}"
}
```
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
./gradlew run
```
OpenAI client builds in your application code are now intercepted automatically. No call to BraintrustOpenAI.wrapOpenAI() is required.
The agent instruments both synchronous and asynchronous client builds, so `OpenAIOkHttpClient` and `OpenAIOkHttpClientAsync` clients are traced automatically.
Manual instrumentation
To trace OpenAI calls manually, wrap your client with `BraintrustOpenAI.wrapOpenAI` yourself. Once wrapped, every `chat().completions().create()` call (including streaming) emits a span.
```java Java theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import dev.braintrust.Braintrust;
import dev.braintrust.instrumentation.openai.BraintrustOpenAI;
class OpenAITracing {
public static void main(String[] args) {
var braintrust = Braintrust.get();
var openTelemetry = braintrust.openTelemetryCreate();
// Wrap the OpenAI client with Braintrust instrumentation.
// To trace an async client, build it with OpenAIOkHttpClientAsync and wrap it the same way,
// importing BraintrustOpenAI from dev.braintrust.instrumentation.openai.v2_15_0.
OpenAIClient client = BraintrustOpenAI.wrapOpenAI(openTelemetry, OpenAIOkHttpClient.fromEnv());
// All API calls are automatically logged
var request = ChatCompletionCreateParams.builder()
.model(ChatModel.GPT_5_MINI)
.addSystemMessage("You are a helpful assistant.")
.addUserMessage("What is machine learning?")
.build();
var result = client.chat().completions().create(request);
}
}
```
For more control over tracing, learn how to [customize traces](/docs/instrument/advanced-tracing).
Gateway
To call OpenAI through the [Braintrust gateway](/docs/deploy/gateway), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](/docs/integrations/ai-providers) to call OpenAI models.
```java Java theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
class OpenAIProxy {
public static void main(String[] args) {
OpenAIClient client = OpenAIOkHttpClient.builder()
.apiKey(System.getenv("BRAINTRUST_API_KEY"))
.baseUrl("https://gateway.braintrust.dev/v1")
.build();
var response = client.chat().completions().create(
ChatCompletionCreateParams.builder()
.model(ChatModel.GPT_5_MINI)
.addUserMessage("What is a proxy?")
.seed(1L) // A seed activates the proxy's cache
.build());
}
}
```
The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](/docs/deploy/gateway#generate-embeddings) for an example.
What Braintrust traces
Braintrust instruments the OpenAI client at the HTTP layer and emits an LLM span per request, naming the span by endpoint. Chat and responses spans capture the request input, the response output, request metadata (provider, request path and method, and model), and token metrics.
**Spans**
| Span | Coverage |
| ----------------- | ----------------------------------------------------- |
| `Chat Completion` | `chat().completions().create()` (including streaming) |
| `responses` | Responses API |
| `Embeddings` | Embeddings API (token metrics only) |
**Metrics**
| Metric | Description |
| ----------------------------- | ------------------------------------ |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `completion_reasoning_tokens` | Reasoning tokens (responses spans) |
| `time_to_first_token` | First-token latency (streaming only) |
Tracing resources
* [Braintrust Java SDK](https://github.com/braintrustdata/braintrust-sdk-java)
* [OpenAI Java SDK](https://github.com/openai/openai-java)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)
Evals
Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. To learn more about evaluations, see the [Experiments](/docs/evaluate/run-evaluations) guide.
Basic eval setup
Evaluate the outputs of OpenAI models with Braintrust.
```java Java theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import dev.braintrust.Braintrust;
import dev.braintrust.eval.DatasetCase;
import dev.braintrust.eval.Scorer;
import dev.braintrust.instrumentation.openai.BraintrustOpenAI;
import java.util.function.Function;
class OpenAIEvaluation {
public static void main(String[] args) {
var braintrust = Braintrust.get();
var openTelemetry = braintrust.openTelemetryCreate();
OpenAIClient client = BraintrustOpenAI.wrapOpenAI(openTelemetry, OpenAIOkHttpClient.fromEnv());
Function taskFunction = (String input) -> {
var request = ChatCompletionCreateParams.builder()
.model(ChatModel.GPT_5_MINI)
.addUserMessage(input)
.build();
var response = client.chat().completions().create(request);
return response.choices().get(0).message().content().orElse("");
};
var eval = braintrust.evalBuilder()
.name("OpenAI Evaluation")
.cases(
DatasetCase.of("What is 2+2?", "4"),
DatasetCase.of("What is the capital of France?", "Paris"))
.taskFunction(taskFunction)
.scorers(
Scorer.of("contains_answer", (evalCase, output) ->
output.contains("4") || output.contains("Paris") ? 1.0 : 0.0))
.build();
var result = eval.run();
System.out.println(result.createReportString());
}
}
```
Learn more about eval [data](/docs/annotate/datasets) and [scorers](/docs/evaluate/write-scorers).
Tracing
Trace your OpenAI LLM calls for observability and monitoring. Install the SDK, wrap your client, and every call emits a span.
Setup
Install the Braintrust .NET SDK alongside the OpenAI .NET SDK, then configure your API keys.
```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# add to .csproj file
dotnet add package Braintrust.Sdk
dotnet add package OpenAI
```
Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).
```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=
BRAINTRUST_API_KEY=
# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=
```
Manual instrumentation
To trace OpenAI calls, wrap your client with `BraintrustOpenAI.WrapOpenAI`. Once wrapped, every `CompleteChat` and `CompleteChatAsync` call emits a span.
```csharp C# theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
using System;
using System.Threading.Tasks;
using Braintrust.Sdk;
using Braintrust.Sdk.OpenAI;
using OpenAI;
using OpenAI.Chat;
class OpenAITracing
{
static async Task Main(string[] args)
{
var braintrust = Braintrust.Sdk.Braintrust.Get();
var activitySource = braintrust.GetActivitySource();
var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY");
if (string.IsNullOrEmpty(apiKey))
{
Console.WriteLine("Error: OPENAI_API_KEY environment variable is not set.");
return;
}
// Wrap the OpenAI client with Braintrust instrumentation
var client = BraintrustOpenAI.WrapOpenAI(
activitySource,
apiKey
);
// All API calls are automatically logged
var chatClient = client.GetChatClient("gpt-5-mini");
var messages = new ChatMessage[]
{
new SystemChatMessage("You are a helpful assistant."),
new UserChatMessage("What is machine learning?")
};
var result = await chatClient.CompleteChatAsync(messages);
}
}
```
For more control over tracing, learn how to [customize traces](/docs/instrument/advanced-tracing).
Gateway
To call OpenAI through the [Braintrust gateway](/docs/deploy/gateway), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](/docs/integrations/ai-providers) to call OpenAI models.
```csharp C# theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
using System;
using System.Threading.Tasks;
using OpenAI;
using OpenAI.Chat;
class OpenAIProxy
{
static async Task Main(string[] args)
{
var apiKey = Environment.GetEnvironmentVariable("BRAINTRUST_API_KEY");
if (string.IsNullOrEmpty(apiKey))
{
Console.WriteLine("Error: BRAINTRUST_API_KEY environment variable is not set.");
return;
}
var client = new OpenAIClient(
new System.ClientModel.ApiKeyCredential(apiKey),
new OpenAIClientOptions
{
Endpoint = new Uri("https://gateway.braintrust.dev/v1")
}
);
var chatClient = client.GetChatClient("gpt-5-mini");
var messages = new ChatMessage[]
{
new UserChatMessage("What is a proxy?")
};
var response = await chatClient.CompleteChatAsync(messages);
}
}
```
The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](/docs/deploy/gateway#generate-embeddings) for an example.
What Braintrust traces
Braintrust instruments the OpenAI chat completions API and emits an LLM span per call, capturing the request messages, the response choices, the model, and the request parameters as metadata.
**Spans**
| Span | Coverage |
| ----------------- | -------------------------------------- |
| `Chat Completion` | `CompleteChat` and `CompleteChatAsync` |
**Metrics**
| Metric | Description |
| --------------------- | ------------- |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `time_to_first_token` | Call latency |
Tracing resources
* [Braintrust .NET SDK](https://github.com/braintrustdata/braintrust-sdk-dotnet)
* [OpenAI .NET SDK](https://github.com/openai/openai-dotnet)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)
Evals
Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. To learn more about evaluations, see the [Experiments](/docs/evaluate/run-evaluations) guide.
Basic eval setup
Evaluate the outputs of OpenAI models with Braintrust.
```csharp C# theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
using System;
using System.Threading.Tasks;
using Braintrust.Sdk;
using Braintrust.Sdk.Eval;
using Braintrust.Sdk.OpenAI;
using OpenAI;
using OpenAI.Chat;
class OpenAIEvaluation
{
static async Task Main(string[] args)
{
var braintrust = Braintrust.Sdk.Braintrust.Get();
var activitySource = braintrust.GetActivitySource();
var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY");
if (string.IsNullOrEmpty(apiKey))
{
Console.WriteLine("Error: OPENAI_API_KEY environment variable is not set.");
return;
}
var client = BraintrustOpenAI.WrapOpenAI(
activitySource,
apiKey
);
// Define the task function that uses OpenAI
string TaskFunction(string input)
{
var chatClient = client.GetChatClient("gpt-5-mini");
var messages = new ChatMessage[]
{
new UserChatMessage(input)
};
var response = chatClient.CompleteChat(messages);
return response.Value.Content[0].Text;
}
// Create and run the evaluation
var eval = await braintrust
.EvalBuilder()
.Name("OpenAI Evaluation")
.Cases(
new DatasetCase("What is 2+2?", "4"),
new DatasetCase("What is the capital of France?", "Paris")
)
.TaskFunction(TaskFunction)
.Scorers(
new FunctionScorer("accuracy", (expected, actual) =>
actual.Contains(expected) ? 1.0 : 0.0)
)
.BuildAsync();
var result = await eval.RunAsync();
Console.WriteLine(result.CreateReportString());
}
}
```
Learn more about eval [data](/docs/annotate/datasets) and [scorers](/docs/evaluate/write-scorers).
## Cookbooks
* [Evaluating audio with the OpenAI Realtime API](/docs/cookbook/recipes/Realtime)
* [Using Python functions to extract text from images](/docs/cookbook/recipes/ToolOCR)
* [Using functions to build a RAG agent](/docs/cookbook/recipes/ToolRAG)