llms-sdk: A Unified Rust Interface for OpenAI and Anthropic APIs
A new Rust-based SDK, llms-sdk, provides a unified request and response model for interacting with OpenAI-compatible and Anthropic LLM APIs.
llms-sdk provides a standardized Rust interface to bridge the gap between competing LLM provider APIs.
- llms-sdk is a Rust crate designed to provide a single interface for OpenAI and Anthropic Large Language Model (LLM) APIs.
- The library supports multimodal inputs including images, audio (OpenAI), and PDF documents (Anthropic).
- Advanced features include structured JSON output via JSON Schema, tool calling, and streaming with reasoning deltas.
Overview of llms-sdk
The development of Large Language Models (LLMs) has led to a variety of proprietary APIs, each with unique request structures and response formats. To address this fragmentation, developer Clelia Astra Bertelli has released llms-sdk, a Rust-based software development kit that provides a unified interface for calling OpenAI-compatible chat completions and the Anthropic Messages API. The project aims to simplify the integration of multiple AI providers into Rust applications by utilizing a single request and response model.
As of version 0.1.0, the crate is available on Crates.io and provides a standardized way to handle complex LLM interactions, such as multimodal inputs and tool calling, which often differ significantly between providers. By abstracting these differences, the SDK allows developers to switch between backends or support multiple models with minimal changes to their codebase.
Unified Request and Response Model
The core of the library is the LLMRequest structure. This struct encapsulates all necessary parameters for an inference call, including the API type (OpenAI or Anthropic), the base URL, API keys, and model identifiers. According to the technical documentation, the SDK utilizes a builder pattern to construct these requests fluently. This design choice enables developers to specify parameters like temperature, max tokens, and sampling methods in a consistent manner regardless of the target provider.
The library also manages the complexities of provider-specific serialization. For example, while OpenAI and Anthropic use different formats for their message histories, llms-sdk uses a unified Message and MessagePart system. This abstraction ensures that a user message containing text and images can be sent to either provider without the developer manually restructuring the payload for each API's requirements.
Multimodal Support and Provider Nuances
One of the distinctive features of llms-sdk is its native support for multimodal inputs. The library includes specific enums and conversion methods for various media types. For instance, the ImagePart allows for the inclusion of visual data, while the AudioPart and DocumentPart handle sound and file-based inputs, respectively.
However, the SDK remains transparent about the limitations of the underlying providers. The documentation notes that AudioPart support is currently limited to OpenAI-compatible endpoints, while DocumentPart support, specifically for PDF files, is designated for the Anthropic Messages API. This allows developers to leverage the specific strengths of each model while maintaining a consistent coding style. Files can be loaded directly from the filesystem using convenience methods like try_from_file and try_from_pdf_file, which automate the encoding process required for API transmission.
Structured Output and Tool Calling
For applications requiring machine-readable data, llms-sdk integrates with the schemars and serde libraries to provide structured outputs. By defining a Rust struct and deriving the JsonSchema trait, developers can enforce a specific output format. The SDK then handles the inclusion of the JSON Schema in the API request, ensuring that the LLM returns data that adheres to the defined structure. This is particularly useful for extracting data or building agents that require precise output control.
Tool and function calling are also supported through provider-specific serialization logic. Developers can define tools using the Tool::new method, providing a name, description, and an argument schema. The SDK then manages the interaction between the model's desire to call a tool and the application's execution of that tool. This functionality includes support for automatic tool choice and parallel tool calls, where the model may suggest multiple actions in a single response.
Streaming and Reliability
To support real-time user interfaces, the crate provides a stream_response method. This feature is built on top of the futures_util crate and allows applications to consume tokens as they are generated. The streaming model is comprehensive, capturing not only text deltas but also tool call deltas and reasoning deltas. The inclusion of reasoning deltas is particularly relevant for modern "reasoning" models that provide a chain-of-thought process before the final answer.
Reliability is addressed through a configurable transient retry policy. Using the RetryPolicy struct, developers can define how the SDK should behave when encountering network errors or rate limits. This built-in resiliency reduces the amount of boilerplate code needed for production-grade AI integrations.
Additional Features and Roadmap
Beyond the library itself, llms-sdk includes an optional command-line interface (CLI) binary, which can be enabled via the cli feature flag. The creator indicates on LinkedIn that the CLI is primarily intended for testing and rapid experimentation. Furthermore, there are plans to expand the project's reach, with Python and TypeScript bindings stated as coming soon.
The project is licensed under the MIT License, encouraging open-source contribution and commercial use. Developers interested in contributing or viewing the source code can find the repository on GitHub, where instructions for unit and integration testing are also provided.
Enjoyed this?
Get more posts like this delivered to your inbox.
