Types

Open Responses data model

Classes

Class Description
ItemType Item type discriminator.
ItemStatus Item processing status.
MessageRole Message role discriminator.
ContentType Content type discriminator.
ImageDetail Image detail level.
FinishReason Constants for generation stop reasons.
InputText Input text content (user/system/developer messages).
InputImage Input image content.
InputAudio Input audio content.
InputVideo Input video content.
InputFile Input file content.
OutputText Output text content (assistant responses).
Refusal Refusal content (model declined to respond).
Text Generic text content.
ReasoningText Reasoning text content (chain-of-thought).
SummaryText Summary text content (reasoning summary).
UnknownContent Unknown content type (forward compatibility).
CodeBlock Detected code block with position metadata.
Item Base class for items.
MessageItem Message item (user, assistant, system, developer).
FunctionCallItem Function/tool call item.
FunctionCallOutputItem Function/tool call output item.
ReasoningItem Reasoning item (chain-of-thought).
ItemReferenceItem Item reference (pointer to another item).
UnknownItem Unknown item type (forward compatibility).
SessionRecord Portable session metadata for storage backends.

class ItemType

Item type discriminator.

class ItemStatus

Item processing status.

class MessageRole

Message role discriminator.

class ContentType

Content type discriminator.

class ImageDetail

Image detail level.

class FinishReason

Constants for generation stop reasons.

Attributes
EOS_TOKEN

End-of-sequence token generated.

LENGTH

Maximum token limit reached.

STOP_SEQUENCE

User-defined stop sequence matched.

TOOL_CALLS

Model requested tool execution.

CANCELLED

Request was cancelled (client disconnect, stop flag set).

class InputText

InputText(self, text: str = '')

Input text content (user/system/developer messages).

Quick Reference

Methods

Method Description
to_dict() Convert to dictionary format for Zig bridge ser...

Methods

def to_dict(self) → dict[str, Any]

Convert to dictionary format for Zig bridge serialization.

class InputImage

InputImage(
    self,
    image_url: str = '',
    detail: ImageDetail = 0
)

Input image content.

Stores an image as a data URI (base64-encoded) or external URL.

Use factory methods for convenient creation
Examples

From file (MIME auto-detected):

>>> img = InputImage.from_file("photo.jpg")
>>> img = InputImage.from_file("scan.png", detail="high")

From bytes with explicit MIME:

>>> img = InputImage.from_bytes(png_data, mime_type="image/png")

From URL (passed through as-is):

>>> img = InputImage.from_url("https://example.com/image.png")

With detail control for vision models:

>>> img = InputImage.from_file("document.png", detail="high")

Attributes ---------- image_url : str Base64 data URI (data:<mime>;base64,<data>) or external URL. detail : ImageDetail Vision model detail level (AUTO, LOW, HIGH).

Quick Reference

Methods

Method Description
from_bytes() Create InputImage from raw bytes.
from_file() Create InputImage from a file.
from_url() Create InputImage from a URL.
to_dict() Convert to dictionary format for Zig bridge ser...

Methods

def from_bytes(
    cls,
    data: bytes,
    mime_type: str | None = None,
    detail: Literal['auto', 'low', 'high'] | ImageDetail | None = None
) → InputImage

Create InputImage from raw bytes.

Parameters
data : bytes

Raw image bytes.

mime_type : str, optional

MIME type (e.g., 'image/jpeg'). Defaults to 'image/png'.

detail : {'auto', 'low', 'high'} or ImageDetail, optional

Vision model detail level.

Returns

InputImage Image content with base64-encoded data URI.

Examples
>>> img = InputImage.from_bytes(png_data)
>>> img = InputImage.from_bytes(jpeg_data, mime_type="image/jpeg")

def from_file(
    cls,
    path: str | Path,
    detail: Literal['auto', 'low', 'high'] | ImageDetail | None = None
) → InputImage

Create InputImage from a file.

MIME type is auto-detected from the file extension.

Parameters
path : str or Path

Path to the image file.

detail : {'auto', 'low', 'high'} or ImageDetail, optional
Vision model detail level
  • 'low': Faster, fewer tokens, lower resolution
  • 'high': Slower, more tokens, full resolution
  • 'auto': Let the model decide (default)
Returns

InputImage Image content with base64-encoded data URI.

Raises
FileNotFoundError

If the file does not exist.

Examples
>>> img = InputImage.from_file("photo.jpg")
>>> img = InputImage.from_file("document.png", detail="high")

def from_url(
    cls,
    url: str,
    detail: Literal['auto', 'low', 'high'] | ImageDetail | None = None
) → InputImage

Create InputImage from a URL.

The URL is stored as-is and will be fetched by the model.

Parameters
url : str

URL to the image (http/https).

detail : {'auto', 'low', 'high'} or ImageDetail, optional

Vision model detail level.

Returns

InputImage Image content with external URL.

Examples
>>> img = InputImage.from_url("https://example.com/photo.jpg")

def to_dict(self) → dict[str, Any]

Convert to dictionary format for Zig bridge serialization.

Returns

dict Dictionary with 'type', 'image_url', and 'detail' keys. The 'detail' is converted to string ('auto', 'low', 'high').

Examples
>>> img = InputImage.from_file("photo.jpg")
>>> img.to_dict()
{'type': 'input_image', 'image_url': 'data:image/jpeg;base64,...', 'detail': 'auto'}

class InputAudio

InputAudio(self, audio_data: str = '')

Input audio content.

Stores audio as a data URI (base64-encoded) or external URL.

Use factory methods for convenient creation
Examples

From file (MIME auto-detected):

>>> audio = InputAudio.from_file("voice.wav")
>>> audio = InputAudio.from_file("podcast.mp3")

From bytes with explicit MIME:

>>> audio = InputAudio.from_bytes(wav_data, mime_type="audio/wav")

From URL:

>>> audio = InputAudio.from_url("https://example.com/audio.mp3")

Attributes ---------- audio_data : str Base64 data URI (data:<mime>;base64,<data>) or external URL.

Quick Reference

Methods

Method Description
from_bytes() Create InputAudio from raw bytes.
from_file() Create InputAudio from a file.
from_url() Create InputAudio from a URL.
to_dict() Convert to dictionary format for Zig bridge ser...

Methods

def from_bytes(
    cls,
    data: bytes,
    mime_type: str | None = None
) → InputAudio

Create InputAudio from raw bytes.

Parameters
data : bytes

Raw audio bytes.

mime_type : str, optional

MIME type (e.g., 'audio/wav'). Defaults to 'audio/wav'.

Returns

InputAudio Audio content with base64-encoded data URI.

Examples
>>> audio = InputAudio.from_bytes(wav_data)
>>> audio = InputAudio.from_bytes(mp3_data, mime_type="audio/mpeg")

def from_file(cls, path: str | Path) → InputAudio

Create InputAudio from a file.

MIME type is auto-detected from the file extension.

Parameters
path : str or Path

Path to the audio file.

Returns

InputAudio Audio content with base64-encoded data URI.

Raises
FileNotFoundError

If the file does not exist.

Examples
>>> audio = InputAudio.from_file("voice.wav")
>>> audio = InputAudio.from_file("podcast.mp3")

def from_url(cls, url: str) → InputAudio

Create InputAudio from a URL.

The URL is stored as-is and will be fetched by the model.

Parameters
url : str

URL to the audio file (http/https).

Returns

InputAudio Audio content with external URL.

Examples
>>> audio = InputAudio.from_url("https://example.com/voice.wav")

def to_dict(self) → dict[str, Any]

Convert to dictionary format for Zig bridge serialization.

Returns

dict Dictionary with 'type' and 'audio_data' keys.

Examples
>>> audio = InputAudio.from_file("voice.wav")
>>> audio.to_dict()
{'type': 'input_audio', 'audio_data': 'data:audio/wav;base64,...'}

class InputVideo

InputVideo(self, video_url: str = '')

Input video content.

Stores video as a data URI (base64-encoded) or external URL.

Use factory methods for convenient creation
Examples

From file (MIME auto-detected):

>>> video = InputVideo.from_file("clip.mp4")
>>> video = InputVideo.from_file("recording.webm")

From bytes with explicit MIME:

>>> video = InputVideo.from_bytes(mp4_data, mime_type="video/mp4")

From URL:

>>> video = InputVideo.from_url("https://example.com/video.mp4")

Attributes ---------- video_url : str Base64 data URI (data:<mime>;base64,<data>) or external URL.

Quick Reference

Methods

Method Description
from_bytes() Create InputVideo from raw bytes.
from_file() Create InputVideo from a file.
from_url() Create InputVideo from a URL.
to_dict() Convert to dictionary format for Zig bridge ser...

Methods

def from_bytes(
    cls,
    data: bytes,
    mime_type: str | None = None
) → InputVideo

Create InputVideo from raw bytes.

Parameters
data : bytes

Raw video bytes.

mime_type : str, optional

MIME type (e.g., 'video/mp4'). Defaults to 'video/mp4'.

Returns

InputVideo Video content with base64-encoded data URI.

Examples
>>> video = InputVideo.from_bytes(mp4_data)
>>> video = InputVideo.from_bytes(webm_data, mime_type="video/webm")

def from_file(cls, path: str | Path) → InputVideo

Create InputVideo from a file.

MIME type is auto-detected from the file extension.

Parameters
path : str or Path

Path to the video file.

Returns

InputVideo Video content with base64-encoded data URI.

Raises
FileNotFoundError

If the file does not exist.

Examples
>>> video = InputVideo.from_file("clip.mp4")
>>> video = InputVideo.from_file("recording.webm")

def from_url(cls, url: str) → InputVideo

Create InputVideo from a URL.

The URL is stored as-is and will be fetched by the model.

Parameters
url : str

URL to the video file (http/https).

Returns

InputVideo Video content with external URL.

Examples
>>> video = InputVideo.from_url("https://example.com/clip.mp4")

def to_dict(self) → dict[str, Any]

Convert to dictionary format for Zig bridge serialization.

Returns

dict Dictionary with 'type' and 'video_url' keys.

Examples
>>> video = InputVideo.from_file("clip.mp4")
>>> video.to_dict()
{'type': 'input_video', 'video_url': 'data:video/mp4;base64,...'}

class InputFile

InputFile(
    self,
    filename: str | None = None,
    file_data: str | None = None,
    file_url: str | None = None
)

Input file content.

Quick Reference

Methods

Method Description
to_dict() Convert to dictionary format for Zig bridge ser...

Methods

def to_dict(self) → dict[str, Any]

Convert to dictionary format for Zig bridge serialization.

class OutputText

OutputText(
    self,
    text: str = '',
    logprobs: list[dict[str, Any]] | None = None,
    annotations: list[dict[str, Any]] | None = None,
    code_blocks: list[CodeBlock] | None = None
)

Output text content (assistant responses).

Attributes
text

The generated text content.

logprobs

Token log probabilities (if requested).

annotations

Content annotations (citations, etc.).

code_blocks

Detected code blocks with position metadata.

class Refusal

Refusal(self, refusal: str = '')

Refusal content (model declined to respond).

class Text

Text(self, text: str = '')

Generic text content.

class ReasoningText

ReasoningText(self, text: str = '')

Reasoning text content (chain-of-thought).

class SummaryText

SummaryText(self, text: str = '')

Summary text content (reasoning summary).

class UnknownContent

UnknownContent(
    self,
    raw_type: str = '',
    raw_data: str = ''
)

Unknown content type (forward compatibility).

class CodeBlock

CodeBlock(
    self,
    index: int,
    fence_start: int,
    fence_end: int,
    language_start: int,
    language_end: int,
    content_start: int,
    content_end: int,
    complete: bool
)

Detected code block with position metadata.

Positions are byte offsets into the source text. All ranges are half-open: [start, end).

Attributes
index

Sequential index of this block (0-based).

fence_start

Byte offset where opening fence begins.

fence_end

Byte offset after closing fence (or current position if incomplete).

language_start

Byte offset where language identifier begins.

language_end

Byte offset after language identifier.

content_start

Byte offset where code content begins.

content_end

Byte offset where code content ends.

complete

True if closing fence was found.

Example
>>> # Given text: "```python\\nprint('hi')\\n```"
>>> block = output.code_blocks[0]
>>> lang = block.get_language(output.text)  # "python"
>>> code = block.get_content(output.text)   # "print('hi')"

Quick Reference

Methods

Method Description
from_dict() Create a CodeBlock from a dictionary.
get_content() Extract the code content from the source text.
get_language() Extract the language string from the source text.

Methods

def from_dict(cls, data: dict[str, Any]) → CodeBlock

Create a CodeBlock from a dictionary.

Parameters
data

Dictionary with code block fields.

Returns

CodeBlock instance.

def get_content(self, source: str) → str

Extract the code content from the source text.

Parameters
source

The full text containing this code block.

Returns

The code content (without fences or language identifier).

def get_language(self, source: str) → str

Extract the language string from the source text.

Parameters
source

The full text containing this code block.

Returns

Language identifier (e.g., "python", "rust"), or empty string if none.

class Item

Item(
    self,
    id: int,
    type: ItemType,
    status: ItemStatus,
    created_at_ms: int
)

Base class for items.

class MessageItem

MessageItem(
    self,
    id: int,
    status: ItemStatus,
    created_at_ms: int,
    role: MessageRole = 1,
    content: tuple[ContentPart, ...] = (),
    raw_role: str | None = None,
    generation: dict[str, Any] | None = None
)

Message item (user, assistant, system, developer).

Quick Reference

Properties

Name Type
text str

Methods

Method Description
create() Create a MessageItem with auto-generated metadata.
to_message_dict() Convert to OpenAI-compatible message dict.

Properties

text: str

Get first text content (convenience for simple messages).

Methods

def create(
    cls,
    role: MessageRole | str,
    content: str | tuple[ContentPart, ...]
) → MessageItem

Create a MessageItem with auto-generated metadata.

This is the ergonomic way to construct a MessageItem for use with chat.append(). The id, status, and created_at_ms fields are auto-populated.

Parameters
role

Message role. Can be a MessageRole enum or string ("system", "user", "assistant", "developer").

content

Message content. Can be a string (converted to InputText) or a tuple of content parts.

Returns

A new MessageItem ready for use with chat.append().

Raises
ValueError

If role string is not one of the valid roles.

Example
>>> item = MessageItem.create("user", "Hello!")
>>> chat.append(item)
>>>
>>> # With enum and explicit content
>>> item = MessageItem.create(
...     MessageRole.ASSISTANT,
...     (OutputText(text="Hi there!"),)
... )

def to_message_dict(self) → dict[str, str]

Convert to OpenAI-compatible message dict.

Returns the standard {"role": "...", "content": "..."} format used by OpenAI, Anthropic, HuggingFace, and most LLM APIs.

This is useful for
  • Debugging ("What does the LLM see?")
  • Exporting conversations to JSON
  • Sending to other APIs
  • Interoperability with other libraries
Returns

Dict with 'role' and 'content' keys.

Example
>>> item = MessageItem(role=MessageRole.USER, content=(InputText(text="Hello"),))
>>> item.to_message_dict()
{'role': 'user', 'content': 'Hello'}

class FunctionCallItem

FunctionCallItem(
    self,
    id: int,
    status: ItemStatus,
    created_at_ms: int,
    name: str = '',
    call_id: str = '',
    arguments: str = ''
)

Function/tool call item.

class FunctionCallOutputItem

FunctionCallOutputItem(
    self,
    id: int,
    status: ItemStatus,
    created_at_ms: int,
    call_id: str = '',
    output_text: str | None = None,
    output_parts: tuple[ContentPart, ...] | None = None
)

Function/tool call output item.

Quick Reference

Properties

Name Type
output str

Properties

output: str

Get output as text.

class ReasoningItem

ReasoningItem(
    self,
    id: int,
    status: ItemStatus,
    created_at_ms: int,
    content: tuple[ContentPart, ...] = (),
    summary: tuple[ContentPart, ...] = (),
    encrypted_content: str | None = None
)

Reasoning item (chain-of-thought).

Quick Reference

Properties

Name Type
summary_text str
text str

Properties

summary_text: str

Get summary text.

text: str

Get reasoning text (from content parts).

class ItemReferenceItem

ItemReferenceItem(
    self,
    id: int,
    status: ItemStatus,
    created_at_ms: int,
    ref_id: str = ''
)

Item reference (pointer to another item).

class UnknownItem

UnknownItem(
    self,
    id: int,
    status: ItemStatus,
    created_at_ms: int,
    raw_type: str = '',
    payload: str = ''
)

Unknown item type (forward compatibility).

class SessionRecord

Portable session metadata for storage backends.

This is the format used in PutSession storage events. It contains session-level metadata that should be stored separately from individual items.

Zig is the single source of truth for session metadata. This ensures that GenerationConfig (temperature, max_tokens, etc.) is portable across Python, Rust, and any future bindings.

Fields

session_id: Session identifier (user-provided or generated) model: Model identifier (optional) title: Human-readable title (optional) system_prompt: System prompt text (optional) config: GenerationConfig as dict (temperature, max_tokens, etc.) marker: Session marker (optional: "pinned", "archived", "deleted"; empty = normal) parent_session_id: Parent session identifier (optional) group_id: Group identifier for multi-tenant listing (optional) head_item_id: Latest item_id in the session (0 when no items yet) ttl_ts: Expiration timestamp (Unix ms). 0 = no expiry metadata: Session metadata dict (optional) search_snippet: Search result snippet text (optional, present in search results) source_doc_id: Document ID that spawned this session (optional, for lineage) created_at_ms: Unix timestamp in milliseconds when session was created updated_at_ms: Unix timestamp in milliseconds when session was last updated

Example
>>> record: SessionRecord = {
...     "session_id": "user_123",
...     "title": "Weather Chat",
...     "system_prompt": "You are a helpful weather assistant.",
...     "config": {"temperature": 0.7, "max_tokens": 1024},
...     "created_at_ms": 1705123456000,
...     "updated_at_ms": 1705123456789,
... }

ContentPart

ContentPart = InputText | InputImage | InputAudio | InputVideo | InputFile | OutputText | Refusal | Text | ReasoningText | SummaryText | UnknownContent

ConversationItem

ConversationItem = MessageItem | FunctionCallItem | FunctionCallOutputItem | ReasoningItem | ItemReferenceItem | UnknownItem