C# – How to make concurrent requests with HttpClient

The HttpClient class was designed to be used concurrently. It’s thread-safe and can handle multiple requests. You can fire off multiple requests from the same thread and await all of the responses, or fire off requests from multiple threads. No matter what the scenario, HttpClient was built to handle concurrent requests.

To use HttpClient effectively for concurrent requests, there are a few guidelines:

  • Use a single instance of HttpClient.
  • Define the max concurrent requests per URL.
  • Avoid port exhaustion – Don’t use HttpClient as a request queue.

In this article I’ll explain these guidelines and then show an example of using HttpClient while applying these guidelines.

Use a single instance of HttpClient

HttpClient was designed for concurrency. It was meant for the user to only need a single instance to make multiple requests. It reuses sockets for subsequent requests to the same URL instead of allocating a new socket each time.

HttpClient implements IDisposable, which leads developers to think it needs to be disposed after every request, and therefore use it incorrectly like this:

//Don't do this
using(HttpClient http = new HttpClient())
{
	var response = await http.GetAsync(url);
	//check status, return content
}

To show the problem with this, here’s what happens when I fire off 15 requests using new instances of HttpClient for each request:

Netstat showing multiple open sockets when HttpClient is disposed after each request

It allocated 15 sockets – one for each request. Because HttpClient was disposed, the allocated socket won’t be used again (until the system eventually closes it). This is not only a waste of resources, but can also lead to port exhaustion (more on this later).

Now here’s what happens when I fire off 15 requests using a single instance of HttpClient (with a max concurrency of four)

It received 15 concurrent requests and only allocated four sockets total. It reused the existing sockets for subsequent requests.

Define the max concurrent requests per URL

Here’s how you set the max concurrency:

private void SetMaxConcurrency(string url, int maxConcurrentRequests)
{
	ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLimit = maxConcurrentRequests;
}

If you don’t explicitly set this, then it uses the ServicePointManager.DefaultConnectionLimit. This is 10 for ASP.NET and two for everything else.

Set the max concurrency to whatever makes sense in your situation.

The single HttpClient instance uses the connection limit to determine the max number of sockets it will use concurrently. Think of it as having a request queue. When the number of concurrent requests > max concurrency, the remaining requests wait in a queue until sockets free up.

For example, let’s say you want to fire off 15 requests concurrently (with max concurrency = four). The following diagram shows how the HttpClient will have four sockets open at once, processing a maximum of four requests concurrently. Meanwhile, the remaining 11 requests will queue up, waiting for a socket to free up.

HttpClient concurrent requests queueing up

Avoid port exhaustion – Don’t use HttpClient as a request queue

In the previous section I explained how the HttpClient has an internal request queue. In this section I’m going to explain why you don’t want to rely on HttpClient’s request queuing.

In the best case scenario, 100% of your requests get processed successfully and quickly. In the real world that never happens. We need to be realistic and deal with the possibility of things going wrong.

To illustrate the problem, I’m sending 15 concurrent requests, and they will all timeout on purpose. I have a max concurrency of four, so you would expect HttpClient to only open four sockets maximum. But here’s what really happens:

There are more than four sockets open at once, and HttpClient will keep opening new sockets as it processes requests.

In other words, when things are going right, it’ll cap the number of sockets it allocates based on the max concurrency you specified. When things are going wrong, it’ll waste sockets. If you are processing lots of requests, this can quickly snowball out of control and lead to port exhaustion. When there aren’t enough ports available to allocate sockets on, network calls start failing all over the system.

The solution is to not rely on HttpClient as a request queue. Instead, handle request queuing yourself and implement a Circuit Breaker strategy that makes sense in your situation. The following diagram shows this approach in general:

HttpClient with external request queue and circuit breaker

How you implement the request queuing mechanism and circuit breaker will depend on what makes sense for your situation.

Example – making concurrent requests with HttpClient

I have an endpoint at http://localhost:9000/api/getrandomnumber. This returns a randomly generated number. I’m going to use a max concurrency of four, and call this with 15 concurrent requests.

I’ve implemented this using the guidelines explained in this article:

  • Use a single instance of HttpClient.
  • Set the max concurrency.
  • Don’t use HttpClient as a request queue.

Instead of using HttpClient as a request queue, I’m using a semaphore as a request queue. I’m using a simple circuit breaker strategy: when a problem is detected, trip the circuit, and don’t send any more requests to HttpClient. It’s not doing automatic retries, and is not automatically opening the circuit. Remember: you’ll want to use a circuit breaker strategy that makes sense in your situation.

RandomNumberService class

public class RandomNumberService
{
	private readonly HttpClient HttpClient;
	private readonly string GetRandomNumberUrl;
	private SemaphoreSlim semaphore;
	private long circuitStatus;
	private const long OPEN = 0;
	private const long TRIPPED = 1;
	public string UNAVAILABLE = "Unavailable";

	public RandomNumberService(string url, int maxConcurrentRequests)
	{
		GetRandomNumberUrl = $"{url}/api/getrandomnumber";

		HttpClient = new HttpClient();
		SetMaxConcurrency(url, maxConcurrentRequests);
		semaphore = new SemaphoreSlim(maxConcurrentRequests);

		circuitStatus = OPEN;
	}

	private void SetMaxConcurrency(string url, int maxConcurrentRequests)
	{
		ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLimit = maxConcurrentRequests;
	}

	public void OpenCircuit()
	{
		if (Interlocked.CompareExchange(ref circuitStatus, OPEN, TRIPPED) == TRIPPED)
		{
			Console.WriteLine("Opened circuit");
		}
	}
	private void TripCircuit(string reason)
	{
		if (Interlocked.CompareExchange(ref circuitStatus, TRIPPED, OPEN) == OPEN)
		{
			Console.WriteLine($"Tripping circuit because: {reason}");
		}
	}
	private bool IsTripped()
	{
		return Interlocked.Read(ref circuitStatus) == TRIPPED;
	}
	public async Task<string> GetRandomNumber()
	{
		try
		{
			await semaphore.WaitAsync();

			if (IsTripped())
			{
				return UNAVAILABLE;
			}

			var response = await HttpClient.GetAsync(GetRandomNumberUrl);
			
			if(response.StatusCode != HttpStatusCode.OK)
			{
				TripCircuit(reason: $"Status not OK. Status={response.StatusCode}");
				return UNAVAILABLE;
			}

			return await response.Content.ReadAsStringAsync();
		}
		catch(Exception ex) when (ex is OperationCanceledException || ex is TaskCanceledException)
		{
			Console.WriteLine("Timed out");
			TripCircuit(reason: $"Timed out");
			return UNAVAILABLE;
		}
		finally
		{
			semaphore.Release();
		}
	}
}

Sending 15 concurrent requests

RandomNumberService randoService = new RandomNumberService(url: "http://localhost:9000", maxConcurrentRequests: 4);

for (int i = 0; i < 15; i++)
{
	Task.Run(async () =>
	{
		Console.WriteLine($"Requesting random number ");
		Console.WriteLine(await randoService.GetRandomNumber());
	});
}

Results

15 requests are sent concurrently. Only four are actually sent to HttpClient at once. The remaining 11 await the semaphore.

All four that are being processed by the HttpClient time out. All four of them try to mark the circuit as tripped (only one reports that it tripped it).

One by one, the semaphore lets the next requests through. Since the circuit is tripped, they simply return “Unavailable” without even attempting to go through the HttpClient.

Leave a Comment