Your GPUs.
Your models.
One control plane.
Loadcore is a distributed system that spins up, runs and governs open LLMs across your own GPU servers. Drop an agent on each box, deploy models from one console, and serve them to your whole team over an OpenAI-compatible API.
Three components. One distributed system.
The control plane decides what should run where. Agents make it so on your hardware. Your tools talk to the agents directly.
Your team manages the fleet. The API keeps state and validates keys.
Sends heartbeats and status up. Runs llama.cpp instances locally.
Requests go straight to the agent's /v1 endpoint, never through Loadcore.
Loadcore Console
A web app to see every server, pull models, start instances, manage people and chat with deployed models.
- +servers, models, instances
- +members, departments, invites
- +personal API keys
- +in-browser chat
Loadcore API
Multi-tenant SaaS that holds fleet state, access rules and keys. It orchestrates, it never proxies inference.
- +organizations and roles
- +server registry via heartbeats
- +instance and download state
- +key validation per model
Loadcore Agent
A single Go binary. Installs llama.cpp, downloads weights, supervises processes and serves /v1 directly.
- +auto-installs llama.cpp
- +GGUF downloads with progress
- +15s instance health checks
- +OpenAI-compatible :9090/v1
From bare GPU box to serving tokens.
Everything you used to do by SSH-ing into servers one at a time, done from one place and kept in sync across the fleet.
- 01
Register
Add a server in the console and get a one-time server key.
servers / new - 02
Connect
Start the agent. It installs llama.cpp and starts sending heartbeats.
./loadcore-agent - 03
Pull
Search HuggingFace, pick a quantization, download to the target box.
Q4_K_M 18.6 GB - 04
Serve
Start an instance with your port, context size and GPU layers.
:8081 ctx 32k ngl 99 - 05
Scale
Run the same model on more servers for throughput, or mix models freely.
3 replicas
Bigger than one GPU.
A distributed instance splits one model across several GPU servers using llama.cpp RPC. Run a 70B+ model on several 24 GB cards you already own.
Holds llama-70b-Q8_0.gguf (75 GB) and 24 GB of VRAM.
24 GB of VRAM each, no model file needed.
One instance, one endpoint, on your local network.
Start an instance on a main server and choose "Distribute across servers" to add rpc servers. The console shows combined GPU memory against the model size.
Only the main server needs the model file. The rpc servers contribute their GPU memory.
One main server and up to 7 rpc servers per distributed instance.
Servers must share a fast local network. Each rpc server accepts connections from the main server only.
This is for fitting bigger models, not for speed. To spread load, run separate instances and choose between them. Sizes in the diagram are an example.
Decide who gets which model.
Put members into departments and grant model access per department. Every member gets personal API keys, and the agent enforces the rules on every request.
| Department | qwen3-coder-next | gpt-oss-120b | gemma-4-31b | gemma-4-26b-a4b |
|---|---|---|---|---|
| Engineering | ||||
| Research | ||||
| Support | ||||
| Org admins |
Engineering, Research, Support, whatever matches your org. A person can sit in several.
Access is granted per model, so an large reasoning model can stay with the team that needs it.
Shown once, revocable any time. A key carries its owner's permissions, nothing more.
Invite by email, or let colleagues request to join and approve them in the console.
Your prompts never touch our servers.
Loadcore separates control traffic from inference traffic. The control plane knows which servers and models exist and who may use them. It never sees a prompt, a completion or a model weight.
- Prompts and completions
- Streaming responses
- Stays inside your network
- Heartbeats and system info
- Download progress and instance status
- API key checks, cached 5 minutes
Model weights download from HuggingFace straight to your server.
One command per server.
Start the agent with the key you got when registering the server. Then point any OpenAI client at it with your personal key.
# on your GPU server
$ ./loadcore-agent \
--api-key="lc_your-server-key" \
--backend-url="https://api.loadcore.io"
Loadcore Agent starting on port 9090
llama.cpp not found, installing latest version...
llama.cpp installed
Agent listening on :9090
OpenAI-compatible API at http://localhost:9090/v1curl http://gpu-ams-01:9090/v1/chat/completions \
-H "Authorization: Bearer $LOADCORE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3-coder-next.gguf",
"messages": [{"role": "user", "content": "Hello"}]
}'Own the throughput.
A rented H200 running Llama 4 Maverick under Loadcore, compared with paying per token for hosted models. Blended price per 1M tokens.
Pricing as of April 2026. Blended rate assumes a 50/50 input/output split. Self-hosted throughput depends on quantization and context length, and hosted APIs offer different model capabilities.
What does Loadcore run on my servers?
One Go binary, the Loadcore Agent. It installs llama.cpp, downloads GGUF models into ~/.loadcore/models, and starts and supervises llama.cpp processes on the ports you choose.
Do my prompts go through Loadcore?
No. Your tools call the agent on your server directly. The control plane only receives heartbeats, download progress, instance status and API key checks.
What hardware do I need?
A Linux server with a GPU that llama.cpp supports. Your own racks, a colo box or a rented cloud GPU all work the same way.
Which models can I run?
Any GGUF model on HuggingFace, in any quantization that fits your VRAM. Run different models on different servers, or the same model on several for throughput.
How is access controlled?
Members belong to departments, and departments are granted specific models. Organization admins can use every model. Each request carries a personal sk-lc- key that the agent checks.
Is there a self-hosted control plane?
No. The Loadcore API is a managed multi-tenant service. Only the agent runs on your infrastructure, and that is where all inference happens.
Put your GPUs to work.
Register your first server and serve a model to your team today.