For each additional 100 milliseconds that it takes to load your website, your conversions drop 1%. That's what Amazon found when it ran a test in 2006.

Source: Strangeloop Networks

Other companies ran similar tests and also found a very strong relationship between page speed and conversions. For Yahoo!, a 400 millisecond increase in page load times caused a 5-9% drop in traffic. And for Google, a 500 millisecond slowdown cut user searches by 20%. (Source)

Here's a quote from Urs Hoelzle, Senior Vice President of Technical Infrastructure at Google:

‘Fast is better than slow’ has been a Google mantra since our early days, and it’s more important now than ever. The internet is the engine of growth and innovation, so we’re doing everything we can to make sure that it’s more Formula 1 than Soap Box Derby. Speed isn’t just a feature, it’s the feature.

Source: Google.com

So how can we measure and improve application response time?

A common (but naïve) performance metric is the average response time.

If app response time were normally distributed, then that would be a valid metric.

Source: Wikimedia

But it's not normally distributed at all. Here's what application response time really looks like:

Source: Marlowe

You'll notice that there are quite a few users who are experiencing very high page load times at the right tail of the distribution. As a result, the average response time is significantly higher than the median response time. Also, many users are seeing intolerable load times.

Instead of looking at the average response time, we should use metrics that make sense in the context of this skewed distribution.

One thing we can do is specify a target acceptable response time, and measure what percent of users get acceptable performance. For example, we could target having 99% of page loads under 1 second over each 5 minute window. (That is, if there is ever a 5-minute period where more than 1% of page loads take more than 1 second, it would count as an app slowdown.)

It's important to define a time window here (e.g., 5 minutes), because app server response time can vary significantly over the course of the day. It might be quite good in the middle of the night, but be quite bad at peak time. Or it might be bad when a resource-intensive cron job is running. We want to know if there are any periods where our app is not performing up to our target Service Level Objectives (SLOs).

Another common metric to use is Apdex. Here's how it works. First, determine what a satisfactory response time is. Let's say we want our pages to load in 500 milliseconds or less. So we'll set our threshold latency for user satisfaction to 500 milliseconds. We'll define a "satisfactory" request as one that takes 500 milliseconds or less. Then, a tolerable request a request that takes up to four times the threshold latency. In our example, the tolerability threshold would be 4 x 500 milliseconds = 2 seconds. In that case, a "tolerable" request is one that takes between 500 milliseconds and 2 seconds.

Finally, use this equation to calculate Apdex:

Apdex = (# satisfactory + 0.5 * (# tolerable)) / (# total requests)

An Apdex score from 0.85 to 0.93 is generally considered a good score.

However, the Apdex score metric can hide systematic outliers if it's not used carefully and in conjunction with other application monitoring tools. For example, what if you have a critical action in your application that only happens once every 50 clicks (e.g., a user checkout) but it takes 15x as long as the median action. You can still have a good Apdex score and be blissfully unaware of users dropping off due to slow payment processing at checkout.

For practical tips on how to improve your site's speed, check out Google's developer tools.