In this post by customer Kyle Rush, VP of Engineering at Casper and former Deputy CTO on the Hillary For America campaign, we can see firsthand how his team dramatically improved the site performance of casper.com. Using a new recommendation from Optimizely that encourages more flexible implementation options for customers through Optimizely Web, Rush and his team exponentially increased the customer experience by speeding up load times. An earlier version of this article was posted on Medium.
We recently deployed a change to casper.com that loaded Optimizely from our own server instead of their server. This change shaved 1.7 seconds off of the start render time:
One last benefit of self-hosting the file is that we would have more control over the edge (CDN) and client (browser) cache. We’re not able to control Optimizely’s edge cache, but we are able to control the client cache. There is a setting that allows you to configure the cache-control value, which for us was set to 2 minutes. This is an ideal setting for us when the file is hosted by Optimizely.
In the chart above, you can see a drop in the start render time from the period of time that our self-hosted, static version of the Optimizely snippet was live in production. By self-hosting, the start render time dropped substantially because we eliminated the DNS lookup, Optimizely server connection, SSL handshake, time to first byte, and enabled H2 multiplexing.
Here is a screenshot from WebPageTest measuring the performance of the new Optimizely file hosted by Casper:
And here is a side by side comparison of data collected prior to self-hosting and after via WebPageTest:
Ideally we’d be presenting 95th percentile of real user monitoring (RUM) data for these values, but we haven’t fully implemented this for casper.com. As with any performance measurement, we observed variance in the data so we examined the numbers at different percentiles to understand the distribution.
Here’s a waterfall that shows HTTP2 multiplexing at work on casper.com and the Optimizely file. Notice how the content download for the top 5 assets starts at nearly the same time for all of them.
And lastly, as mentioned earlier, self-hosting gives us more control over caching. We configured our edge servers to keep the file in the edge and browser cache for a full year. We are able to do this because the filename is unique to the contents (we add part of the file’s hash to the filename) and replace the reference to the filename when it changes. This way, if we don’t make any changes to the Optimizely snippet, the repeat visitor’s browser will not even make a request to casper.com for the file. It will instead pull the file directly from the cache on the user’s filesystem. Super fast!
Here you can see the benefits of the file being served from the browser cache:
In this chart we can see that there was a 3 hour period on Thursday the 23rd where the snippet changed about 25 times. It’s unlikely that a large number of visitors would be downloading multiple versions of the snippet at this change frequency because our average visit duration is not very long. Overall we think there are more benefits to self-hosting than drawbacks.
This project was about a month’s worth of on-and-off work from our software engineers, product managers, site reliability engineers, and data analysts. It was a great example of some performance-minded people on the Casper Tech team identifying an issue, finding an elegant solution, shipping it to production, and making a huge impact for our customers.
If you’d like to learn more about self-hosting the Optimizely snippet, check out these guides for some of the most popular CDNs:
If you’d like to learn more about the Optimizely best practices for site performance, be sure to check out our Knowledge Base to ensure you are taking advantage of our recommendations.