Download our FREE ebook of 43 A/B testing case studies from the world's leading companies for test ideas and inspiration. Download Now

Optimizely Blog

Tips & Tricks for Building Your Experimentation Program

Every company wants to grow faster. But not every company has the resources to make this happen. Whether it’s a lack of time, expertise, or headcount, something is missing. Without the right resources, a company could consider: growing slower, hiring internally, getting a consultant, or using the crowd. Here are 4 instances the crowd helped companies grow revenue.

SEM Aid Increases Orders, an e-commerce company, ran PPC campaigns for thousands of products, but they weren’t spending time improving ad text due to a lack of people.

“Number one, I wanted to have better returns on our PPC campaigns, and number two, what I really wanted was scalability.” said President and COO, Omar Sayyed.’s small team size meant stale ad copy, poor keyword selection, little time spent improving campaigns, and a barrier to scaling to more products. So Sayyed and his marketing department tried Trada.

Trada is a crowdsourced PPC management company. To get started, Sayyed filled out a simple checklist; then, 10 optimizers submitted ad copy and strategy for review.


Examples of optimized PPC ads from crowdsourced experts

Sayyed also met weekly with a campaign manager to identify how to improve his account and they ended up running 100 PPC ads through Trada.

They saw fantastic results:

  • CTR went from 1.1% to 2.7%, an almost 200% increase
  • CPA decreased 71%
  • Orders from search went up 400%. reached their goals of increased returns and scalability. They continue to use outside SEM help in conjunction with their internal team. “It’s all crowdsourced at this point. We have had phenomenal success with this effort and, frankly, I have restructured my Marketing department [around this advertising model].” said Omar Sayyed.

(Source: Marketing Sherpa)

Artists Achieve Perfect Redesign

CloudSponge, a SaaS company that automates management and sharing of address books, needed to refresh their website. But Founder Jay Gibb didn’t like attempts from existing designers.


Founder Jay Gibb needed a new look for his website, but was not satisfied with the work of existing designers.

Instead, he tried 99Designs. To get started, he gave a brief of needs and goals. After receiving initial redesign concepts, Gibb worked directly with designers on the revisions.

Crowdsourced designers provided:

  • 189 unique options to review
  • A new website design within 10 days on a budget of $1500.

“The price is right, the delivery timeline is predictable and the artists are all eager to please.”


CloudSponge’s new website came from the work of crowdsourced designers who were passionate about following the brief and create a great concept.


(Source: CloudSponge Case Study)

 Journalists Grow Blog Traffic

HotPads, a service that connects property owners and renters, needed to grow their digital presence. They wanted to publish engaging content about U.S. cities, but did not have enough writers on the team.

To quickly and effectively reach their goal, they crowdsourced detailed, high-quality content from Contently.


Through Contently, they found more than 20 qualified journalists that wrote engaging, geo-specific content and produced over 270 stores in a year. This resulted in a 4000% increase in blog traffic.

(Source: HotPads Case Study)

Expert Network Maximizes Conversions, a shipping and brokerage technology company, wanted to drive more leads from existing web traffic. However, due to limited internal resources, they ran few A/B tests.

“FreightPros has been challenged with no dedicated in-house design personnel, or the time and resources to continually test landing pages and optimize our visitor funnel.” said President Chris Clever.

To get more leads out of their existing traffic, they turned to Experiment Engine, which combines a software platform with a creative network of conversion rate optimizers. Experiment Engine provided FreightPros with an all-in-one solution that helped them: identify opportunity pages to improve, generate ready-to-test page variations, and recognize the best performing page variations. As a result, FreightPros saw 32X return on investment in its first two months of using Experiment Engine.


Examples of ready-to-test variations included repositioning of value propositions, improved typography, and modified form UX. These incremental changes resulted in impactful conversion lifts.

These ready-to-test variants achieved:

  • 20% conversion lift in the first month and another 10% lift in the next month
  • A total of 32% gain in two months due to compounding gains with each win
  • A 32X return on their investment in Experiment Engine

Chris Clever remarks about the wins, “Experiment Engine has been the perfect solution to the online conversion problems a business the size of ours faces. It has resulted in tangible results; more customers requesting freight quotes.”



Hitting revenue targets can be hard. Hitting revenue targets when there aren’t enough resources is even harder. Luckily, companies have a few options through the crowd.

Optimizely X