Estimating utilization of the application servers may not be always possible. In many companies, while average cpu utilization of any server is changing between %20 and %30, they may become hardly respond to incoming requests at peak times. When an unexpected over-utilization occurs, the first method that comes to mind is provisioning new instances (of course your company is using a virtualization or cloud architecture) to the application server pool.
Sometimes it may not be the solution for over-utilization. As the number of app servers increases it becomes important that the requests are distributed equally across these servers. Becoming some of the servers unresponsive may have a devastating effect on the environment.
|Application Server CPU Usages|
Traditionally, load balancers are used to distribute incoming traffic across multiple app instances. Load balancing provides some benefits like scaling the application farm, supporting heavy network traffic, detecting unhealthy app instances and automatically removing them from the app pool, routing traffic to the most eligible app instance.
There are many load balancing methods but I am going to refer the most used ones here.
- Round Robin: It is the most widely used algorithm. Once a request comes, LB forward that request to the next server in the loop.
- Least-Connection: This method is smarter. The current request is forwarded to the server that has the least number of active sessions at the current time.
- Source IP Hash: LB is configured to use the hashed value of the client IPv4 or IPv6 address to select an app server.
- URL Hash: In this method, to select an app server, the LB generates a hash value of the HTTP URL present in the incoming request.
- Least Traffic: The LB monitors the network traffic from each app server, and forwards the request to the server that has the least outgoing traffic.
- Least Response Time: LB compares response times from all servers to determine the server that is responding fastest at a particular time. Then the next request is forwarded to that app server.
Choosing the right LB algorithm that fits your environment is important. Because the wrong choice leads customer dissatisfaction.