Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

X

Download our FREE Testing Toolkit for A/B testing ideas, planning worksheets, presentation templates, and more!

Get It Now




On the heels of our announcement on Tuesday announcing our 100,000th experiment and TechCrunch breaking the news that we’ve surpassed Omniture Test&Target, we were thrilled this week to see a direct response from Adobe, Omniture’s parent company. Anecdotally, we’ve heard from customers that Adobe has had us in their sights for some time, but not until now have they officially made a statement. As the famous Gandhi quote goes, “First they ignore you, then they laugh at you, then they fight you, then you win.”  Today, I think we’ve officially moved into fight mode.

Here is a great quote from their blog post:

Test&Target serves over 1.2 bil­lion test page opti­miza­tion requests each day. Wow. Com­pare this to one startup who has bragged about serv­ing up 1.3 bil­lion over the course of a few years.

As our blog post and the TechCrunch article make clear, we announced 1.3 billion visitors, not requests. Clearly, we serve many more requests than visitors. So not only is this statement inaccurate, it’s also interesting for two other reasons. First, they are directly attacking us without mentioning us by name. Why are they so afraid to mention this “one startup”?

The second reason why this quote is interesting is that they choose to focus on requests and not visitors, as this highlights one of the biggest differences between our two products: the implementation. Back when I was using Test&Target as part of the Obama campaign, we had to install an mbox code snippet for every part of the page we wanted to test; and every time you wanted to test something new you needed to add an mbox. What’s worse is that Omniture charges you for each mbox request. Some of the customers who have switched from Test&Target to Optimizely have referred to this as “mbox hell.” This frustration was one of the many reasons we created Optimizely. We have built the product I wish we had in 2008 to make it dramatically easier for anyone to A/B test their website. Optimizely’s implementation is a single snippet that is unique to your account. With this single line of code  you can run as many experiments on as many parts of your site as you like.

As for ease-of-use according to their blog post:

Even basic test­ing is very easy to learn and to deploy — I’ve wit­nessed opti­miza­tion man­agers learn to run basic tests within the tool within an hour’s time.

Having used Test&Target personally and having listened to the stories of customers who have made the switch from Test&Target to Optimizely, this statement comes to me as quite a surprise. But, as a data-driven marketer, I’d gladly be convinced after seeing some data that shows Test&Target is in fact easy to use. In the spirit of real world A/B testing I’d like to offer Omniture the Optimizely Challenge (akin to the Pepsi Challenge): put your product directly up against ours and let’s see which one marketers think is easier to use. Anyone reading this post is welcome to go directly to http://www.optimizely.com/ and enter any URL you’d like to test (feel free to even try creating an experiment on adobe.com). We put our product where our mouth is. Omniture, are you willing to let marketers try your product and see how it compares?

Another great quote:

So, I encour­age you to dig deeper than the rhetoric you’re hear­ing this elec­tion year and demand the facts when look­ing at the future of your test­ing pro­gram.

Finally a point we agree on. I also encourage you to dig deeper than the rhetoric your’re hearing. Try both products and see which you prefer.

Optimizely X
comments powered by Disqus