​
Introduction

​
What is Solo?

Solo is an open-source platform for Hardware Aware Inference.

Using Solo, you can easily set up your environment, download and serve models, benchmark performance, fine-tune, ensemble, distill, and deploy across a variety of inference engines and platforms.

Quickstart

Get your solo server set up locally for easy model inference.

Tutorials

Learn different ways you can use solo server to manage and optimize model inference.

Examples

Checkout examples in our cookbook repo to bootstrap

​
Why Use Solo?

Solo Server is designed to meet the evolving needs of effective AI deoloyment by focusing on several key differentiators:

Hardware Aware

  • Solo Server dynamically adjusts its operations and configs based on the underlying hardware

Domain Specific

Designed to integrate seamlessly with domain-specific workflows, it accelerates deployment speed.

Performance Tuned

Engineered for speed, Solo Server handles demanding workloads with optimized performance settings.

Private and Pragmatic

Solo Server emphasizes privacy, allowing you to keep sensitive data local without compromising on performance or functionality.

​
Prerequisites

Before installing Solo Server, ensure your system meets the following prerequisites:

​
Software Requirements

  • Docker:

    Solo Server is designed to run within a containerized environment. Make sure Docker is installed on your system.

​
Hardware Requirements

  • Processor: At least a 1 GHz CPU core (more cores are recommended for production workloads).

  • Memory: Minimum of 2GB RAM (4GB or more is recommended for smoother performance, especially under load).

  • Storage: Sufficient disk space for the Docker images, logs, and any persistent data (typically 10GB or more depending on your usage).

​
Operating System Requirements

  • Linux: Ubuntu 18.04 or later, CentOS 7+, or equivalent distributions.

  • macOS: Version 10.13 or later.

  • Windows: Windows 10 or later (Docker Desktop required).

These prerequisites ensure that Solo Server runs efficiently in your environment while leveraging containerization for consistent deployment.