Running AI at scale
Behind-the-scenes software for serving AI to many people, storing AI data and monitoring how it performs.
82 projects, highest rated first.
Apache Airflow
Schedules and monitors chains of automated tasks, such as data processing and AI model training jobs.
Running AI at scaleLiteLLM
Lets programs talk to over 100 AI model providers in one common way, with cost tracking built in.
Running AI at scaleOmniRoute
One doorway to hundreds of AI providers, which can make the most of their free allowances.
Running AI at scalevLLM
Serves AI models to many users at once, quickly and at lower cost.
Running AI at scaleSGLang
Software that serves large AI models quickly to many users, on a wide range of hardware.
Running AI at scaleRay
Spreads heavy AI and Python work across many computers, from a laptop up to a large cluster.
Running AI at scaleMLflow
A platform for building, testing, monitoring and managing AI apps and machine learning models.
Running AI at scaleKong Gateway
A central gateway that routes, secures and monitors traffic to an organisation's APIs and AI models.
Running AI at scaleMilvus
A database built to store and search huge amounts of AI data, such as text and images, by meaning.
Running AI at scaleNew API
A self-hosted gateway that puts many AI model services behind one address, with usage and cost tracking.
Running AI at scaleONNX Runtime
Makes trained AI models run faster on many kinds of computers and chips.
Running AI at scaleModular Platform (MAX and Mojo)
Open-source parts of Modular's platform: the Mojo programming language and the MAX engine for running AI models.
Running AI at scaleApache Doris
A fast database for analysing large amounts of data, with search features for AI apps and agents.
Running AI at scaleDapr
Behind-the-scenes software that keeps multi-step apps and AI agents running reliably, even when servers crash.
Running AI at scaleLangfuse
Helps teams track, test and improve their AI apps by recording what the AI did and how well it did.
Running AI at scaleQdrant
A database for AI 'embeddings' that lets apps search by meaning, find similar images or make recommendations.
Running AI at scaleChroma
A database that lets AI apps store information and search it by meaning, not just exact words.
Running AI at scale1Panel
A web control panel for managing a Linux server, its websites and apps, and the AI agents on it.
Running AI at scaleKubeflow
A collection of open-source tools for building, training and running AI at scale on cloud servers.
Running AI at scaleOpenVINO
A toolkit that makes AI models run faster and lighter on ordinary processors and Intel chips.
Running AI at scaleWeaviate
A database that finds information by meaning, not just keywords, used to power AI search and chatbots.
Running AI at scaleOpik
Track, test and monitor AI apps and agents so teams can see what they're doing and catch mistakes.
Running AI at scalevLLM-Omni
Serves AI models that handle text, images, audio and video quickly to many users.
Running AI at scaleLanceDB
A database for storing and searching the data AI apps use, including text, images and video.
Running AI at scaleSkyPilot
One tool for running AI training and serving jobs across many clouds and your own GPU clusters.
Running AI at scaleWasmEdge
A small, fast engine for running sandboxed programs, which can also run open AI models on your devices.
Running AI at scaleVolcano
A Kubernetes scheduler for running large AI training, big data and computing jobs efficiently.
Running AI at scaleDatabend
A data warehouse for large-scale analysis that also supports AI search and running AI agents on company data.
Running AI at scaleLance
A data storage format built for AI, holding text, images, video and search data in one place.
Running AI at scaleMooncake
The serving system behind the Kimi chatbot, built to handle many AI requests by sharing cached work.
Running AI at scaleKServe
A platform for serving AI models to many users on Kubernetes.
Running AI at scaleTriton Inference Server
NVIDIA's server software for running trained AI models quickly for many users at once.
Running AI at scaleBifrost
A gateway that puts many AI providers behind one connection, with backups if one fails.
Running AI at scaleOpenSandbox
Gives AI agents safe, walled-off computers to run code, use files and browse, on your own servers.
Running AI at scaleCube Sandbox
Gives AI agents fast, sealed-off mini computers where they can safely run code.
Running AI at scaleArize Phoenix
A dashboard for watching, testing and debugging AI apps so teams can see what their AI is doing.
Running AI at scaleTyk Gateway
A gateway that controls and protects traffic to APIs and AI services, including AI agent tool connections.
Running AI at scalevLLM Semantic Router
Automatically picks which AI model should handle each request, based on rules you set for quality, speed and cost.
Running AI at scaleVespa
A platform for large-scale AI search and recommendations over text, vectors and structured data.
Running AI at scaleParadeDB
Adds fast keyword search, meaning-based search and analytics to the Postgres database.
Running AI at scaleTensorFlow Serving
Software for running trained AI models on servers so apps can send them requests.
Running AI at scaleAgentgateway
A traffic controller that adds security, monitoring and rules to how AI agents talk to models, tools and each other.
Running AI at scaleOne API
One gateway for reaching many AI providers, such as OpenAI, Claude and Gemini, in a single standard way.
Running AI at scaleLMCache
Helps servers that run AI chat models answer faster and more cheaply by reusing work they've already done.
Running AI at scaleFeast
A feature store that keeps the data machine-learning models use consistent between training and live use.
Running AI at scaleDaft
A fast data engine for processing images, audio, video and tables for AI, from a laptop up to big clusters.
Running AI at scaleOpenLLMetry
Lets developers see what their AI app is doing behind the scenes, using standard monitoring tools.
Running AI at scaleCog
Packages AI models into ready-to-run containers so they're easier to put online.
Running AI at scaleMetaflow
A framework for building, running and deploying AI and machine learning systems.
Running AI at scaleFlyte
Runs machine-learning pipelines, AI models and agents reliably across many computers, written in Python.
Running AI at scaleGorse
A recommendation engine that learns from user activity to suggest items each person may like.
Running AI at scaleAxonHub
An AI gateway that lets apps switch between AI model providers without changing their code.
Running AI at scaleNCCL
NVIDIA's library that lets many graphics cards share data quickly, used when training large AI models.
Running AI at scaleRagas
Measures how well an AI app is answering, using clear scores instead of guesswork.
Running AI at scaleEvidently
Tests and monitors AI and machine-learning systems to check they keep working well.
Running AI at scaleSynapseML
A Microsoft library for building machine-learning pipelines that run across large computer clusters.
Running AI at scaleClearML
A suite of tools for tracking AI experiments, managing data, and running and serving models.
Running AI at scaleBentoML
A Python library for turning AI models into online services that apps can call.
Running AI at scaleHelicone
Tracks every request an app sends to AI models, showing cost, speed and quality, and routes between providers.
Running AI at scaleTensorRT
NVIDIA's toolkit for making trained AI models run faster on NVIDIA graphics cards.
Running AI at scaleMicrosandbox
Runs AI agents and untrusted code inside tiny, sealed-off virtual computers on your machine.
Running AI at scaleAngel
A system for training machine learning models on very large data across many computers.
Running AI at scaleNasiko
Finds the AI coding agents a team runs, shows what each costs, and routes them to chosen models.
Running AI at scaleXorbits Inference (Xinference)
Server software that runs open AI models for text, speech and images behind one common doorway.
Running AI at scaleManifest LLM Gateway
Routes your apps' AI requests through one doorway that picks a model, tracks cost and handles failures.
Running AI at scalesmolvm
Lightweight virtual machines for running AI agents and their code safely, separate from your main system.
Running AI at scaleGPUStack
Manages groups of graphics cards to run AI models for many users and hand out GPU machines on demand.
Running AI at scaleLMDeploy
A toolkit for compressing AI language models and serving them quickly on your own GPUs.
Running AI at scaleZenML
A platform for running and tracking machine learning and AI agent workflows reliably in companies.
Running AI at scalePlano
Behind-the-scenes software that routes, watches and protects traffic between AI agents and models.
Running AI at scaleText Embeddings Inference
Serves text embedding models quickly, so apps can search and compare text by meaning.
Running AI at scaleiFixAi
Runs a quick audit to check whether an AI agent is doing the job it is supposed to do, safely.
Running AI at scaleAI Gateway
A single doorway that lets apps reach many AI models, with automatic retries, backups and safety checks.
Running AI at scaleCoze Loop
A platform for developing, testing and monitoring AI agents and their prompts.
Running AI at scaleHelixDB
An open-source database that stores graphs, vectors and text search together.
Running AI at scaleNVIDIA DALI
Speeds up AI training by loading and preparing images, video and audio on the graphics card.
Running AI at scaleOpenLLM
Run open AI models on your own servers as a ready-to-use service that works like OpenAI's.
Running AI at scaleAim
An open-source tool for tracking and comparing machine learning training runs.
Running AI at scaleGPTCache
Saves AI answers so repeated or similar questions don't need a new, paid request.
Running AI at scaleNano-vLLM
A small, easy-to-read version of the vLLM engine that runs AI language models quickly.
Running AI at scaleRagaAI Catalyst
Track, test and debug AI apps and agents, and see each step they took.
Running AI at scaleAgentOps
Tracks what AI agents do, step by step, and how much they cost, so developers can debug them.