Command Line

Overview

This page focuses on the core CLI workflow for everyday use, with reference details at the end.

Quickstart

talu get downloads a model from HuggingFace and stores it locally. The weights stay in their original safetensors format, alongside the tokenizer and chat template — nothing is converted at this stage. Once cached, the model is available offline.

talu ask sends a prompt to a model and streams the response to the terminal. The model's chat template is detected and applied automatically, so the prompt is wrapped in the correct format for that architecture.

The -S flag sets a system prompt that controls the model's behavior and tone. When omitted, the default system prompt is "You are a helpful assistant."

Working with models

talu ls shows all cached models in a table with columns for SIZE, DATE, TYPE (architecture), QUANT (quantization format), and MODEL (the HuggingFace identifier). Passing an organization name filters the list to that org; passing a full model name shows the individual weight files.

When talu get is called without a model name, it opens an interactive browser. This lets you search HuggingFace by keyword, see parameter counts and architecture details, and then select a model to download — useful when you're exploring what's available rather than fetching a specific model.

talu rm removes a model from the local cache and frees the associated disk space.

Profiles

A profile stores a default model, conversation history, and other settings in ~/.talu/, persisted across terminal sessions. By default, talu uses a profile named default. The active profile can be changed with the TALU_PROFILE environment variable, which makes it straightforward to maintain separate configurations for different projects or use cases.

talu set assigns a default model to the current profile, which means -m no longer needs to be passed on every command.

Once a default is set, every command that needs a model uses it automatically.

Piping also works with a default model. Text from stdin is treated as the prompt for ask.

The -m flag overrides the default for a single command without changing the profile. talu set show displays the current profile configuration, and talu set without arguments opens an interactive picker that lists all cached models.

Conversations

By default, each call to talu ask creates a new conversation. Conversations are persisted automatically — the full transcript is stored, so it can be reviewed, continued with follow-up prompts, or resumed at a later time.

talu ask without a prompt lists all stored conversations, showing their session IDs and first messages.

Passing a session ID with --session displays the full transcript of that conversation.

Adding a prompt along with --session continues the conversation. The model receives the full history as context, so it remembers earlier exchanges.

The --delete flag removes a conversation and its transcript permanently.

--no-bucket creates an ephemeral conversation that is not persisted. This is useful for one-off questions where saving the transcript is unnecessary.

Quantization

Models from HuggingFace are typically in full-precision (F16) safetensors format. Quantization compresses the weights to a smaller representation, reducing memory usage and increasing inference speed with minimal quality loss.

talu convert quantizes a model using Grouped Affine (GAF) quantization. The model is positional (talu convert <model>). When --scheme is not specified, the default scheme is gaf4_64.

A different scheme can be selected with --scheme.

Smaller group sizes produce higher accuracy at the cost of slightly larger files. talu convert --scheme help prints this table in the terminal.

Converted models are saved under $TALU_HOME/models/ with the scheme as a suffix (e.g. Qwen/Qwen3-0.6B-GAF4). The --output flag writes to a different directory instead. The original model is never modified — both versions remain available and talu set can switch between them.

Tokenization

talu tokenize shows how a model splits text into tokens. Each token is displayed alongside its numeric ID, which is useful for understanding context window usage or debugging prompt formatting.

Scripting and CI

Most commands work in scripts and CI pipelines. The CLI separates human-readable formatting from machine-parseable output: a set of flags control what goes to stdout, while logs always go to stderr. Interactive modes still require a TTY.

Stdout contracts

Model URI flows

The --model-uri flag changes stdout to print only the model URI, making it easy to capture in a shell variable and pass to subsequent commands. This is the typical pattern for chaining download, convert, and set operations in a script.

The same flag works with talu convert, outputting the URI of the newly quantized model.

And with talu set, it confirms the URI that was actually assigned to the profile.

Session flows

The --session-id flag prints only the session ID to stdout, allowing scripts to capture it and reference the conversation in later steps.

That session ID can then be passed back with --session to continue the same conversation, building up context across multiple script steps.

Output controls

Response text only.

OpenResponses JSON.

Write to file.

Silent (exit code only).

Reference

Environment variables

Environment variables provide an alternative to flags, useful in CI or when setting parameters for an entire shell session. Flags take precedence when both are specified.

Global flags

These apply to all commands. Logs are always written to stderr.

Common commands

This table focuses on everyday CLI workflows, plus a few advanced commands for completeness.