> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tinfoil.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Rust

> Rust SDK for Tinfoil's secure AI inference API

> [card] [View on GitHub](https://github.com/tinfoilsh/tinfoil-rs)
>
> tinfoilsh/tinfoil-rs

## Overview

The Tinfoil Rust SDK is a thin wrapper around the [`async-openai`](https://github.com/64bit/async-openai) crate that provides secure communication with Tinfoil enclaves. It uses the same types and request/response shapes as `async-openai` and adds automatic verification that the endpoint is running in a secure Tinfoil enclave, TLS certificate pinning, and attestation validation.

## Installation

> [accordion] New to Rust? Start here - Project Setup
>
> If you don't have a Cargo project yet, create one:
>
> ```bash theme={"dark"}
> cargo new my-tinfoil-app
> cd my-tinfoil-app
> ```
>
> The minimum supported Rust version is **1.88**.

Add the SDK and a runtime to `Cargo.toml`:

```bash theme={"dark"}
cargo add tinfoil --git https://github.com/tinfoilsh/tinfoil-rs
cargo add tokio --features full
cargo add serde --features derive
cargo add serde_json futures-util base64
cargo add reqwest@0.13.3 --no-default-features --features json,rustls-no-provider,stream,multipart
```

## Migration from `async-openai`

`tinfoil::Client` exposes the same chat, audio, and embeddings handlers as `async-openai`. The example below shows how to change the client setup:

```diff theme={"dark"}
// Before (async-openai)
- use async_openai::{Client, config::OpenAIConfig};
-
- let client = Client::with_config(
-     OpenAIConfig::new().with_api_key("OPENAI_API_KEY")
- );

// After (Tinfoil)
+ use tinfoil::Client;
+
+ let client = Client::new_default().await?;
```

`tinfoil::Client` derefs to `async_openai::Client`, so all OpenAI methods (`client.chat()`, `client.embeddings()`, `client.audio()`, ...) work unchanged.

## Usage

```rust theme={"dark"}
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Create a client
    let client = Client::new_default().await?;

    // 2. Use client as you would async_openai::Client
    // see https://docs.rs/async-openai for API documentation
    Ok(())
}
```

The types in `tinfoil::chat`, `tinfoil::audio`, and `tinfoil::embeddings` are re-exported from the corresponding `async_openai::types::*` modules.

## Model Examples

Below are specific examples for current Tinfoil model categories.

### Chat Models

> [accordion] Kimi K3.1 Pro
>
> ```rust theme={"dark"}
> use tinfoil::chat::{
>     ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
>     ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
> };
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let request = CreateChatCompletionRequestArgs::default()
>         .model("kimi-k3-1-pro")
>         .messages(vec![ChatCompletionRequestMessage::User(
>             ChatCompletionRequestUserMessage {
>                 content: ChatCompletionRequestUserMessageContent::Text(
>                     "Review this codebase and suggest architectural improvements for \
>                      better modularity and testability.".to_string(),
>                 ),
>                 name: None,
>             },
>         )])
>         .build()?;
>
>     let response = client.chat().create(request).await?;
>     println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
>     Ok(())
> }
> ```

> [accordion] GLM-5.3
>
> ```rust theme={"dark"}
> use tinfoil::chat::{
>     ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
>     ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
> };
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let request = CreateChatCompletionRequestArgs::default()
>         .model("glm-5-3")
>         .messages(vec![ChatCompletionRequestMessage::User(
>             ChatCompletionRequestUserMessage {
>                 content: ChatCompletionRequestUserMessageContent::Text(
>                     "Solve this step by step: If a train travels 120 miles in 2 hours, \
>                      and then increases its speed by 25% for the next 3 hours, how far \
>                      does it travel in total?".to_string(),
>                 ),
>                 name: None,
>             },
>         )])
>         .build()?;
>
>     let response = client.chat().create(request).await?;
>     println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
>     Ok(())
> }
> ```

> [accordion] Gemma 4 31B
>
> ```rust theme={"dark"}
> use tinfoil::chat::{
>     ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
>     ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
> };
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let request = CreateChatCompletionRequestArgs::default()
>         .model("gemma4-31b")
>         .messages(vec![ChatCompletionRequestMessage::User(
>             ChatCompletionRequestUserMessage {
>                 content: ChatCompletionRequestUserMessageContent::Text(
>                     "Explain the trade-offs between accuracy and latency in a production AI system.".to_string(),
>                 ),
>                 name: None,
>             },
>         )])
>         .build()?;
>
>     let response = client.chat().create(request).await?;
>     println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
>     Ok(())
> }
> ```

> [accordion] Llama 3.3 70B
>
> ```rust theme={"dark"}
> use tinfoil::chat::{
>     ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
>     ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
> };
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let request = CreateChatCompletionRequestArgs::default()
>         .model("llama3-3-70b")
>         .messages(vec![ChatCompletionRequestMessage::User(
>             ChatCompletionRequestUserMessage {
>                 content: ChatCompletionRequestUserMessageContent::Text(
>                     "What are the key differences between renewable and non-renewable \
>                      energy sources?".to_string(),
>                 ),
>                 name: None,
>             },
>         )])
>         .build()?;
>
>     let response = client.chat().create(request).await?;
>     println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
>     Ok(())
> }
> ```

> [accordion] GPT-OSS 120B
>
> ```rust theme={"dark"}
> use tinfoil::chat::{
>     ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
>     ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
> };
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let request = CreateChatCompletionRequestArgs::default()
>         .model("gpt-oss-120b")
>         .messages(vec![ChatCompletionRequestMessage::User(
>             ChatCompletionRequestUserMessage {
>                 content: ChatCompletionRequestUserMessageContent::Text(
>                     "Analyze the trade-offs between different database architectures for \
>                      a high-traffic e-commerce platform.".to_string(),
>                 ),
>                 name: None,
>             },
>         )])
>         .build()?;
>
>     let response = client.chat().create(request).await?;
>     println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
>     Ok(())
> }
> ```

### Audio Models

> [accordion] Voxtral Small 24B
>
> #### Transcription
>
> The `transcribe` shortcut on `Client` forwards to `client.audio().transcription().create(request)`.
>
> ```rust theme={"dark"}
> use std::path::PathBuf;
> use tinfoil::async_openai::types::InputSource;
> use tinfoil::audio::{AudioInput, CreateTranscriptionRequestArgs};
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let request = CreateTranscriptionRequestArgs::default()
>         .file(AudioInput {
>             source: InputSource::Path { path: PathBuf::from("meeting_recording.mp3") },
>         })
>         .model("voxtral-small-24b")
>         .language("en")
>         .prompt("This is a business meeting discussing quarterly results")
>         .build()?;
>
>     let transcription = client.transcribe(request).await?;
>     println!("Transcription: {}", transcription.text);
>     Ok(())
> }
> ```
>
> #### Audio Q\&A
>
> ```rust theme={"dark"}
> use tinfoil::chat::{
>     ChatCompletionRequestMessage, ChatCompletionRequestMessageContentPartAudio,
>     ChatCompletionRequestUserMessage, ChatCompletionRequestUserMessageContent,
>     ChatCompletionRequestUserMessageContentPart, CreateChatCompletionRequestArgs,
>     InputAudio,
> };
> use tinfoil::multimodal::InputAudioExt;
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let audio = InputAudio::from_path("audio.mp3")?;
>
>     let request = CreateChatCompletionRequestArgs::default()
>         .model("voxtral-small-24b")
>         .messages(vec![ChatCompletionRequestMessage::User(
>             ChatCompletionRequestUserMessage {
>                 content: ChatCompletionRequestUserMessageContent::Array(vec![
>                     ChatCompletionRequestUserMessageContentPart::Text(
>                         "What is the speaker talking about in this audio?".into(),
>                     ),
>                     ChatCompletionRequestUserMessageContentPart::InputAudio(
>                         ChatCompletionRequestMessageContentPartAudio { input_audio: audio },
>                     ),
>                 ]),
>                 name: None,
>             },
>         )])
>         .build()?;
>
>     let response = client.chat().create(request).await?;
>     println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
>     Ok(())
> }
> ```

### Embedding Models

> [accordion] Nomic Embed Text
>
> The `embed_batch` shortcut takes a list of strings, preserves input order, and returns `Vec<Vec<f32>>`.
>
> ```rust theme={"dark"}
> use tinfoil::Client;
>
> #[tokio::main]
> async fn main() -> Result<(), Box<dyn std::error::Error>> {
>     let client = Client::new_default().await?;
>
>     let documents = [
>         "Artificial intelligence is transforming modern technology.",
>         "Machine learning enables computers to learn from data.",
>         "The weather today is sunny and warm.",
>         "Deep learning uses neural networks with multiple layers.",
>     ];
>
>     let embeddings = client.embed_batch("nomic-embed-text", documents.iter().copied()).await?;
>
>     let similarity = cosine_similarity(&embeddings[0], &embeddings[1]);
>     println!("Similarity between first two AI-related documents: {:.3}", similarity);
>     println!("Embedding dimension: {}", embeddings[0].len());
>     Ok(())
> }
>
> fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
>     let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
>     let norm_a: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
>     let norm_b: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
>     if norm_a == 0.0 || norm_b == 0.0 { 0.0 } else { dot / (norm_a * norm_b) }
> }
> ```

## Vendor Extensions

Tinfoil exposes a few request/response fields the upstream OpenAI schema does not (for example `web_search_options`, `pii_check_options`, structured-output regex / choice constraints, and custom `finish_reason` values). To use them without `async-openai` rejecting the response, send the request through `client.chat_relaxed()`:

```rust theme={"dark"}
use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("gpt-oss-120b")
        .messages([json!({ "role": "user", "content": "What's the latest on AI safety?" })])
        .web_search_with_options(json!({ "max_results": 5 }));

    let response = client.chat_relaxed().create(body).await?;
    if let Some(content) = response.content() {
        println!("{}", content);
    }
    Ok(())
}
```

Streaming uses the same builder and surfaces vendor-specific events (such as `web_search_call`) verbatim:

```rust theme={"dark"}
use futures_util::StreamExt;
use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("gpt-oss-120b")
        .messages([json!({ "role": "user", "content": "Explain quantum entanglement" })]);

    let mut stream = client.chat_relaxed().create_stream(body).await?;

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        if let Some(delta) = chunk.delta_content() {
            print!("{}", delta);
        } else if chunk.is_vendor_event() {
            println!("[event] {:?}", chunk.event_type());
        }
    }
    Ok(())
}
```

## Advanced Functionality

For advanced use cases requiring manual verification or direct HTTP access, use `SecureClient`:

```rust theme={"dark"}
use tinfoil::SecureClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Construct a secure client for manual verification and HTTP access
    let mut secure = SecureClient::new(
        "inference.tinfoil.sh",
        "tinfoilsh/confidential-model-router",
        std::env::var("TINFOIL_API_KEY")?,
    );

    // Verify the enclave attestation
    let ground_truth = secure.verify().await?;
    let enclave_fp = ground_truth.enclave_fingerprint.clone();
    let code_fp = ground_truth.code_fingerprint.clone();

    println!("Enclave: {}", secure.host());
    println!("Enclave fingerprint: {}", enclave_fp);
    println!("Code fingerprint: {}", code_fp);

    // Get the verified HTTP client for custom requests (multipart uploads,
    // unofficial endpoints like /v1/convert/file, etc.)
    let _http = secure.http_client()?;
    Ok(())
}
```

## API Documentation

This library is a thin wrapper around the [`async-openai`](https://github.com/64bit/async-openai) crate that can be used with Tinfoil. All methods and types are identical. See the [`async-openai` documentation](https://docs.rs/async-openai) for complete API usage.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.