C# – How to make concurrent requests with HttpClient

The HttpClient class was designed to be used concurrently. It’s thread-safe and can handle multiple requests. You can fire off multiple requests from the same thread and await all of the responses, or fire off requests from multiple threads. No matter what the scenario, HttpClient was built to handle concurrent requests.

To use HttpClient effectively for concurrent requests, there are a few guidelines:

• Use a single instance of HttpClient.
• Define the max concurrent requests per server.
• Avoid port exhaustion – Don’t use HttpClient as a request queue.
• Only use DefaultRequestHeaders for headers that don’t change.

In this article I’ll explain these guidelines and then show an example of using HttpClient while applying these guidelines.

Use a single instance of HttpClient

HttpClient was designed for concurrency. It was meant for the user to only need a single instance to make multiple requests. It reuses sockets for subsequent requests to the same URL instead of allocating a new socket each time.

HttpClient implements IDisposable, which leads developers to think it needs to be disposed after every request, and therefore use it incorrectly like this:

//Don't do this
using(HttpClient http = new HttpClient())
{
	var response = await http.GetAsync(url);
	//check status, return content
}
Code language: C# (cs)

To show the problem with this, here’s what happens when I fire off 15 requests using new instances of HttpClient for each request:

It allocated 15 sockets – one for each request. Because HttpClient was disposed, the allocated socket won’t be used again (until the system eventually closes it). This is not only a waste of resources, but can also lead to port exhaustion (more on this later).

Now here’s what happens when I fire off 15 requests using a single instance of HttpClient (with a max concurrency of four)

It received 15 concurrent requests and only allocated four sockets total. It reused the existing sockets for subsequent requests.

Define the max concurrent requests per server

To set the max concurrent requests per server:

• Create SocketsHttpHandler
• Set SocketsHttpHandler.MaxConnectionsPerServer.
• Pass in the SocketsHttpHandler in the HttpClient constructor.

Here’s an example:

var socketsHttpHandler = new SocketsHttpHandler()
{
	MaxConnectionsPerServer = 16
};
var httpClient = new HttpClient(socketsHttpHandler);
Code language: C# (cs)

Note: If you’re using .NET Framework, refer to the Setting max concurrency in .NET Framework section below.

Set the max concurrency to whatever makes sense in your situation.

The single HttpClient instance uses the connection limit to determine the max number of sockets it will use concurrently. Think of it as having a request queue. When the number of concurrent requests > max concurrency, the remaining requests wait in a queue until sockets free up.

For example, let’s say you want to fire off 15 requests concurrently (with max concurrency = four). The following diagram shows how the HttpClient will have four sockets open at once, processing a maximum of four requests concurrently. Meanwhile, the remaining 11 requests will queue up, waiting for a socket to free up.

For reference – Setting the max concurrency in .NET Framework

2023-03-10 – Added to distinguish how to set the max concurrency in .NET Core vs .NET Framework.

In .NET Core apps going forward, use SocketsHttpHandler.MaxConnectionsPerServer to set the max concurrency. In .NET Framework, you could set the max concurrency with ServicePointManager, like this:

private void SetMaxConcurrency(string url, int maxConcurrentRequests)
{
	ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLimit = maxConcurrentRequests;
}
Code language: C# (cs)

If you don’t explicitly set this, then it uses the ServicePointManager.DefaultConnectionLimit. This is 10 for ASP.NET and two for everything else.

Avoid port exhaustion – Don’t use HttpClient as a request queue

In the previous section I explained how the HttpClient has an internal request queue. In this section I’m going to explain why you don’t want to rely on HttpClient’s request queuing.

In the best case scenario, 100% of your requests get processed successfully and quickly. In the real world that never happens. We need to be realistic and deal with the possibility of things going wrong.

To illustrate the problem, I’m sending 15 concurrent requests, and they will all timeout on purpose. I have a max concurrency of four, so you would expect HttpClient to only open four sockets maximum. But here’s what really happens:

There are more than four sockets open at once, and HttpClient will keep opening new sockets as it processes requests.

In other words, when things are going right, it’ll cap the number of sockets it allocates based on the max concurrency you specified. When things are going wrong, it’ll waste sockets. If you are processing lots of requests, this can quickly snowball out of control and lead to port exhaustion. When there aren’t enough ports available to allocate sockets on, network calls start failing all over the system.

The solution is to not rely on HttpClient as a request queue. Instead, handle request queuing yourself and implement a Circuit Breaker strategy that makes sense in your situation. The following diagram shows this approach in general:

How you implement the request queuing mechanism and circuit breaker will depend on what makes sense for your situation.

Example – making concurrent requests with HttpClient

I have an endpoint at http://localhost:9000/api/getrandomnumber. This returns a randomly generated number. I’m going to use a max concurrency of four, and call this with 15 concurrent requests.

I’ve implemented this using the guidelines explained in this article:

• Use a single instance of HttpClient.
• Set the max concurrency.
• Don’t use HttpClient as a request queue.

Instead of using HttpClient as a request queue, I’m using a semaphore as a request queue. I’m using a simple circuit breaker strategy: when a problem is detected, trip the circuit, and don’t send any more requests to HttpClient. It’s not doing automatic retries, and is not automatically closing the circuit. Remember: you’ll want to use a circuit breaker strategy that makes sense in your situation.

RandomNumberService class

public class RandomNumberService
{
    private readonly HttpClient HttpClient;
    private readonly string GetRandomNumberUrl;
    private SemaphoreSlim semaphore;
    private long circuitStatus;
    private const long CLOSED = 0;
    private const long TRIPPED = 1;
    public string UNAVAILABLE = "Unavailable";

    public RandomNumberService(string url, int maxConcurrentRequests)
    {
        GetRandomNumberUrl = url;

        var socketsHttpHandler = new SocketsHttpHandler()
        {
            MaxConnectionsPerServer = maxConcurrentRequests
        };

        HttpClient = new HttpClient(socketsHttpHandler);
        
        semaphore = new SemaphoreSlim(maxConcurrentRequests);

        circuitStatus = CLOSED;
    }

    public void CloseCircuit()
    {
        if (Interlocked.CompareExchange(ref circuitStatus, CLOSED, TRIPPED) == TRIPPED)
        {
            Console.WriteLine("Closed circuit");
        }
    }
    private void TripCircuit(string reason)
    {
        if (Interlocked.CompareExchange(ref circuitStatus, TRIPPED, CLOSED) == CLOSED)
        {
            Console.WriteLine($"Tripping circuit because: {reason}");
        }
    }
    private bool IsTripped()
    {
        return Interlocked.Read(ref circuitStatus) == TRIPPED;
    }
    public async Task<string> GetRandomNumber()
    {
        try
        {
            await semaphore.WaitAsync();

            if (IsTripped())
            {
                return UNAVAILABLE;
            }

            var response = await HttpClient.GetAsync(GetRandomNumberUrl);

            if (response.StatusCode != HttpStatusCode.OK)
            {
                TripCircuit(reason: $"Status not OK. Status={response.StatusCode}");
                return UNAVAILABLE;
            }

            return await response.Content.ReadAsStringAsync();
        }
        catch (Exception ex) when (ex is OperationCanceledException || ex is TaskCanceledException)
        {
            Console.WriteLine("Timed out");
            TripCircuit(reason: $"Timed out");
            return UNAVAILABLE;
        }
        finally
        {
            semaphore.Release();
        }
    }
}
Code language: C# (cs)

Notes:

• 2021-08-31 – Updated to use the correct circuit terminology (“closed” instead of “open”).
• 2023-03-10 – Updated to set the max concurrency with SocketsHttpHandler.MaxConnectionsPerServer (the preferred way to do it in .NET Core).

Sending 15 concurrent requests

var service = new RandomNumberService(url: "http://localhost:9000", maxConcurrentRequests: 4);

for (int i = 0; i < 15; i++)
{
	Task.Run(async () =>
	{
		Console.WriteLine($"Requesting random number ");
		Console.WriteLine(await service.GetRandomNumber());
	});
}
Code language: C# (cs)

Results

Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Requesting random number
Timed out
Timed out
Timed out
Tripping circuit because: Timed out
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Unavailable
Timed out
UnavailableCode language: plaintext (plaintext)

15 requests are sent concurrently. Only four are actually sent to HttpClient at once. The remaining 11 await the semaphore.

All four that are being processed by the HttpClient time out. All four of them try to mark the circuit as tripped (only one reports that it tripped it).

One by one, the semaphore lets the next requests through. Since the circuit is tripped, they simply return “Unavailable” without even attempting to go through the HttpClient.

Only use DefaultRequestHeaders for headers that don’t change

Updated article (9/30/21) with this new section.

HttpClient.DefaultRequestHeaders isn’t thread-safe. It should only be used for headers that don’t change. You can set these when initializing the HttpClient instance.

If you have headers that change, set the header per request instead by using HttpRequestMessage and SendAsync(), like this:

using (var request = new HttpRequestMessage(HttpMethod.Get, GetRandomNumberUrl))
{
	request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", Token);
	var response = await HttpClient.SendAsync(request);

	response.EnsureSuccessStatusCode();

	return await response.Content.ReadAsStringAsync();
}
Code language: C# (cs)