How concurrency works
Concurrency counts work that has started but hasn’t finished. If your client starts 10 fetches and all are waiting for responses, its request concurrency is 10. Each response or terminal failure frees a slot for the next job.
A program can create this overlap with asynchronous I/O, threads or processes. Concurrency doesn’t require tasks to run on separate CPU cores.
For proxy traffic, distinguish in-flight requests from open TCP connections. An idle keep-alive connection counts toward connection concurrency but not request concurrency. HTTP/2 can also carry several requests over one connection. Check whether your client or proxy server limits tasks, requests or connections.
Concurrency vs QPS
Concurrency measures simultaneous work. QPS measures request rate over time.
| Metric | What it measures | Typical unit |
|---|---|---|
| Concurrency | Work in progress | Requests or connections |
| QPS | Request rate | Requests per second |
| Bandwidth | Data transferred | Mbps or GB |
For a saturated client at steady state, completed-request throughput is approximately in-flight request concurrency divided by mean end-to-end request time. Ten concurrent requests averaging 0.5 seconds could complete about 20 requests per second.
Retries can raise total request QPS while lowering successful-request throughput because capacity is spent repeating failed work. Queueing or client overhead makes the estimate too high only when it isn’t included in the measured end-to-end time. Bandwidth can also become the bottleneck when responses are large.
Why concurrency has limits
Higher concurrency can improve throughput while requests wait on network responses. Past a certain point, it raises latency, fills connection pools and creates more timeouts.
High concurrency against one origin can also trigger rate limiting. The website, CDN or firewall may return 429 Too Many Requests. IP rotation doesn’t mean 20 simultaneous requests will use 20 unique exit IPs. Assignment depends on the proxy’s rotation and session rules.
Set a practical concurrency limit
- Start low. Use a few concurrent requests per domain.
- Increase gradually. Raise the limit in small steps while measuring latency, timeouts and status codes.
- Honor site limits. Follow robots.txt, terms and documented API quotas. Don’t use concurrency to push past access controls.
- Back off on errors. Reduce concurrency after 429 or 503 responses and use exponential backoff before retrying.
FAQ
Does more proxy concurrency mean more IP addresses?
No. Concurrency describes simultaneous work, not the number of exit IPs. Static or sticky sessions can send many concurrent requests through one address.
Is concurrency the same as threads?
No. Threads are one way to create concurrency. An asynchronous event loop can manage many in-flight requests with a small number of threads.
Why does higher concurrency cause 429 errors?
It can send requests in a tighter time window. If that rate exceeds a site’s per-IP, account or endpoint limit, the site or its front-end infrastructure may respond with 429.