Mean server utilization is defined as the ratio of the mean arrival rate over the mean service rate

By , Last Updated January 4, 2011

Below is a Definition of Queueing Theory, a glossary, and vocabulary. Knowing the concepts below will help you better understand these articles on Queues.

• Queue: A line (or buffer or inventory) feeding a number of servers
• Server: An operation fed by a queue.
• Arrival rate (λ): Mean number of arrivals per unit time (usually per hour or day).
• Service rate (μ): Mean number of customers that can be served at 100% utilization by each individual server per unit time (usually per hour or day). At the individual workstation level, the service rate will equal capacity.
• Channels (M): The number of parallel operations connected to an individual queue. For example, if each queue has 2 operations then it will have two channels.
• Utilization (u): A measure of how “busy” the system is. It is generally defined as the ratio of throughput to capacity. Note that u = λ/(Μμ) if λ < Μμ, i.e. the utilization is less than 100%. (Also, note that while the Greek letter μ— or mu— looks a bit like u, they are in fact two different variables.)
• Phase: A queue and its connected servers, or routes to a server.
• Balking: When a person, who would otherwise have entered a line, decides not to enter it.
• Reneging: When a person, who has entered a line, later decides to leave it without being served.
• Interarrival Time: The time between when one customer arrives at a queue and when the next customer arrives.
• Service Time: The time it takes for one particular server to complete a customer’s service. The average service time will be the same as the cycle time.

And now for some equations you’ll need to know.

• CV: The coefficient of variation. This is a measure of a random variable’s variability. For a random variable x, CVx is defined as Standard Deviation (x) x mean (x) CV = .
• CVIAT: The coefficient of variation of the interarrival time. The greater the CVIAT, the “lumpier” the arrival rate.
• CVST: The coefficient of variation of the service time. The smaller the CVST, the more “consistent” a server is.
• Lq: The average number of people in a line awaiting service.
• Wq: The average length of time a customer waits before being served.

After you have become familiar with the terms above, go ahead and read the articles on Queueing Theory.

Become a Lean Six Sigma professional today!

Start your learning journey with Lean Six Sigma White Belt at NO COST

Use coupon code:

KICKSTART

Joel Spolsky of Joel on Software fame mentions queueing theory in the latest StackOverflow podcast. He mentions a rule of thumb that wait times go up quickly as server utilization exceeds 80%. The same principle applies whether you’re talking about computer servers or human servers. I hadn’t heard of that rule of thumb, though I knew that you don’t want anywhere near 100% utilization. Here’s the graph that justifies Joel’s remark.

The vertical axis is wait time. The horizontal access is utilization.

Here are the details. Basic queuing theory assumes customer arrivals are Poisson distributed with rate λ. Service times are exponentially distributed with rate μ. The ratio λ/μ is called utilization ρ. If this ratio is greater than 1, that says customers are arriving faster than they can be served, and so the line will grow without bound. If the ratio is less than 1, the line will reach some steady state on average.

The average waiting time is W = 1/(μ-λ). Now assume the service time μ is fixed and the arrival rate λ = ρ μ. Then W = 1/μ(1-ρ) and so the wait time is proportional to 1/(1-ρ). As the utilization ρ approaches 1, the wait time goes to infinity. The graph above plots 1/(1-ρ). As Joel said, the curve does go up quickly when ρ exceeds 0.8.

Related post: What happens when you add a new teller?

For daily posts on probability, follow @ProbFact on Twitter.