How does the number of threads affect performance?

Answer

General

GMS Mail has been observed delivering over 2, 000, 000 live email messages in a single day through a dual 166 and GMS List Server has been seen delivering at a rate of hundreds of thousands of postings per hour from a P90. We have nearly 10,000 customers users world wide, many having thousands of users attached to their mail server. Both products are in use by hundreds of ISPs in a true industrial strength environment; possibly the first free ISPs in the UK (FreeNet)scaled up from a few hundred users to over 50,000 users in just a few months using GMS Mail to deliver service. Performance, reliability, efficient use of resources and scaleability are therefore proven.

We have never had a single user suggest that the number of threads normally available (255)are a constraint. In a straw poll among our support staff, very few power users use even as many 128.

How to GMS Mail Manages Threads

GMS Mail interprets the number of threads requested by the user as the expected number of connections to be served at any one time. In order to achieve the maximum performance, memory and threads are “pre-allocated”. Hence, as soon as a remote server makes a connection, the thread is released immediately to handle the transfer of mail. This dramatically reduces the “connection latency” which may occur using other methods of thread invocation.

Reasonable number of threads

The number of threads is usually not the limiting factor for a mail server which is handling a large number of email messages. In fact there are several disadvantages to using a large number of threads:

1. Context Switching. In general, processors are poor at context switching – therefore the larger the number of threads, the more time the processor(s) spend deciding which thread to execute rather than actually processing useful code.

2. Bandwidth Limitation. If a machine has a single 100Mb/s network card, there is a physical limit of about 8 MB/second (assuming Ethernet running at maximum efficiency – 80% capacity, and 10 bits per bytes after packet headers, retries, etc. have been accounted for). Lets assume that email is going both in and out of a machine, i.e. delivery by SMTP and posting by SMTP and POP3/IMAP4 collection. At 1000 threads, this represents 4K/second (since all messages in, are likely to go out at some point) per thread. Experience has shown that at these levels of saturation, retries, failed packets and the like start to cause dramatic loss of efficiency. BTW, with an average message size of 4K, this figure equates to about 86 million messages per day!

In conclusion, increasing the number of threads available might even cause throughput to deteriorate.

Keywords:threads, performance, throughput