How many users can one AI server handle?
Last updated
One AI server can handle as many users as it can serve at the same moment at a comfortable speed. In Olares' own test, Olares One served 8 concurrent requests at about 81 tokens per second each with vLLM on Qwen3-30B-A3B.
Olares One → AI server for small business → Accounts · documents · remote access → A private AI for the whole team
How many users can one AI server handle at peak times?
Count requests that arrive at the same moment, because headcount misleads. A person sends a question, reads the answer, then thinks or types, and most of that time their seat sends nothing. A team gets busy together, on Monday morning or before a deadline. Size the server for those peaks.
What did Olares measure on Olares One?
Olares measured one model, Qwen3-30B-A3B, with two serving programs. With vLLM it reached 157 tokens per second for one request and about 81 per request with 8 at once. With Ollama it reached 107 for one request and 32 each with 8. All four are Olares' own figures.
| Serving software | Requests at once | Speed per request |
|---|---|---|
| vLLM | 1 | 157 tokens/s |
| vLLM | 8 | about 81 tokens/s |
| Ollama | 1 | 107 tokens/s |
| Ollama | 8 | 32 tokens/s |
vLLM groups requests into batches, so speed holds up as load grows. Ollama is simpler and fine for a few people. At about 81 tokens per second, each of the 8 people still gets text faster than they can read it.
What limits how many people it can serve?
Memory is the first limit. Each active request needs memory for its context, so a model that fills most of the 24 GB leaves little room for parallel requests. Larger dense models also run slower per request, and questions about long documents take longer before the first word appears.
Qwen3-30B-A3B is a mixture-of-experts model, which is why it runs fast for its size.
When should you add a second box?
Add a second box only after a software fix has failed. Run a pilot with real staff for a week and watch wait times at busy hours. If waits grow, move from Ollama to vLLM or to a smaller model first. Both cost nothing, and a second unit costs another 4,999 USD at list price.
According to Olares' FAQ, two or more Olares devices can form a cluster. That raises total throughput and allows larger models through vLLM pipeline parallelism. A single request does not get faster, and setup is command line today.
For buying advice, see the 10-person office guide. For memory, see what a small business AI server needs. For the first install, see setting up an AI server in the office.
Questions & answers
How many people can share Olares One?
Olares does not publish a headcount. Its own test shows 8 concurrent requests at about 81 tokens per second each with vLLM on Qwen3-30B-A3B. Staff rarely ask at the same moment, so a team larger than 8 can share it. Test with your own users.
Why is vLLM faster than Ollama for teams?
vLLM batches many requests together. In Olares' own test on Olares One with Qwen3-30B-A3B and 8 requests at once, vLLM gave about 81 tokens per second per request and Ollama gave 32.
Does a second box make answers faster?
No. According to Olares' FAQ, clustering raises total throughput and allows larger models. Each single task runs at the same speed.
Sources for this entry
- Olares One FAQ (Olares docs) (Company-published)
- Olares blog: local AI hardware benchmarking (Oct 2025) (Company-published)
Continue this reading path
How the related websites connect
Cite this entry
Best AI Server for Small Business. “How many users can one AI server handle?.” Content reviewed 13 November 2026.
https://bestaiserverforsmallbusiness.com/ai-server-concurrent-users/