How to Tame the Neural Network Zoo with TokenHub
When a team starts integrating neural networks into their services, key management almost always descends into chaos. Developers use OpenAI, analysts experiment with DeepSeek and Qwen, someone spins up a local Llama via Ollama, and designers want access to image generation. At the end of the month, accounting sees the total bill and tries to figure out which department spent the budget.
Sound familiar? If you've ever tried to centralize access to dozens of different LLMs, you know how much duct tape and baling wire goes into it: proxies, homegrown routers, log aggregators. The author of the popular Beego Go framework (astaxie) released a project called TokenHub, which takes on this headache.
What's Under the Hood and Why You Need It
TokenHub is a private AI gateway written in Go. You deploy it within your infrastructure, add API keys from external providers or addresses for local models, and expose a single unified interface outward.
The core idea is that application developers shouldn't need to rewrite code for every new API. TokenHub supports standard OpenAI endpoints (/v1/chat/completions, /v1/embeddings, /v1/responses) and the Anthropic Messages specification (/v1/messages). If your service needs to switch from GPT-4o to Claude 3.5 Sonnet or a local vLLM, you won't need to change the client code—just update the model name or configure a routing rule on the gateway.
Core Gateway Features
The repository focuses on enterprise scenarios where cost control and clear separation of responsibilities are critical.
Clear Role Separation
The interface and access rights in TokenHub are divided into three levels:
- Regular User. Sees the available model catalog, creates their own project API keys, views personal statistics and documentation.
- Team Leader. Manages project workspaces, team members, shared quotas, and sees detailed cost reports for a specific project.
- Administrator. Configures external providers, connects corporate SSO via OAuth/OIDC, and configures failover and audit rules.
This setup is convenient: team leads don't need to bother DevOps every time a new intern needs a test key.
Smart Routing and Failover
If an external provider returns a 500 error or hits a rate limit, TokenHub can automatically switch requests to backup channels (failover). You can set priorities, weights for distributing load across multiple accounts, and check route health via built-in diagnostics.
Support for 150+ Provider Templates
Out of the box, the gateway works with native adapters for Azure OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen, as well as local inference servers like vLLM and Ollama. Any third-party service with an OpenAI-compatible API connects via ready-made templates in a couple of clicks.
It's worth highlighting the Codex subscription integration: TokenHub can proxy calls from local CLI utilities (e.g., Gemini CLI) through isolated profiles.
Image Generation and Editing
Beyond text and embeddings, the system handles image generation requests via endpoints /v1/images/generations and /v1/images/edits. The gateway supports async tasks and server-side storage of generated images, removing the need for client applications to immediately download files to their own S3.
Deployment and Architecture
The project is written in Go and consumes minimal resources. SQLite is used for data storage by default, so you can spin up the service for a small team in literally minutes without spinning up heavy databases.
For testing on Linux via systemd:
curl -fsSL https://raw.githubusercontent.com/astaxie/TokenHub/main/deploy/native/install.sh \
-o /tmp/tokenhub-install.sh
sudo bash /tmp/tokenhub-install.sh install
Or the classic Docker Compose launch:
cp deploy/.env.example deploy/.env
# Задайте надежные пароли в deploy/.env
./deploy/install.sh
After startup, the web management console is available on port 3000, and the API itself accepts requests on port 8080.
If you're planning high load with multiple backend replicas, SQLite is easily replaced with an external PostgreSQL. In this case, state is distributed across instances, and version updates can be rolled out centrally.
Who This Project Is For Right Now
TokenHub is a great fit if you:
- Are developing internal AI tools in your company and want to centrally track spending by projects and departments.
- Are building a resilient setup with multiple LLM providers, where one API going down shouldn't take down your client service.
- Want to safely expose access to locally deployed models (Ollama / vLLM) to your entire team without granting direct access to inference servers.
- Are tired of manually managing dozens of API keys across different dashboards.
The project is still fresh, but it's backed by an experienced maintainer, and the Go codebase looks clean and understandable. If you're choosing between heavyweight commercial solutions and homegrown proxies, spinning up TokenHub in a test environment is definitely worth a try.
Related projects