Latency

General concept of latency in performance testing is the time it takes the request to reach the server from the client and then from the server back to the client, commonly known as the network latency. However, its not the only latency a request encounters. The flow of the request from web server to application server to DB is controlled by the use of queues and threads. So the time for which the request waits in the application server queue for a free thread to execute or in the database server for a connection pool, is also latent time for the request.

Normally under higher user loads, all the Application server threads and DB connection pools get used up and the incoming request queue up at these places, thus increasing the latency of the later requests. So it is important to have proper configuration settings for the queues and threads so that user requests do not queue up. For example, if the max thread count is set to a low value, then a high configuration application server may be able to handle the requests fast, but the response throughput will be limited by the number of threads available to process the requests.

Coming back to the network latency, this latency can be because of the following reasons:

• transmission delays (properties of the physical medium)
  
• and processing delays (such as passing through proxy servers or making network hops on the Internet)

Network tools like ping and tracert measure latency by determining the time it takes a given network packet to travel from source to destination and back. These tools can be used as a first step in finding out the network latency issues and then allowing the network team to fix the latency issue before the end users notice its effect.