Background
New Haven, Connecticut-based startup SeeClickFix is a social technology platform for non-emergency community issues. See a pothole on your daily commute that doesn’t seem to be getting fixed, or know of a traffic light timing glitch that should be addressed? SeeClickFix partners with local governments to have issues like these monitored and fixed.
The Experiment
CEO Ben Berkowitz wanted to use data to inform internal debates about website copy and design, and chose their most important page — their homepage — as the place to start. SeeClickFix had spent time designing and developing a beautiful Google Maps integration on their homepage, which showed the issues their users were filing in real-time:
Ben suspected however, that while the map was a great showcase of their product’s usage, it might actually be reducing page engagement. Here is his original hypothesis:
I want to do something that would previously have created a debilitating debate with the team. I want to drop the map that we worked so hard to make beautiful and functional. My goal is to focus eyeballs on the search box. This is where the culture shift comes in. Optimizely is so easy and time-light that the only reasonable outcome to suggesting the dropping of the map is, ‘lets just test it.’
Here’s what they tested instead:
Results
The new design increased engagement by 8.3% — Optimizely measures engagement as the percentage of visitors who click on any part of the experiment page. It is a handy way to measure the holistic impact of a large change. You can think of engagement as the opposite of bounce rate so the higher engagement, the better.
Here is a screenshot of the Optimizely results page with some of the numbers redacted:
Conclusion
This initial test accomplished two things for SeeClickFix: it increased their product’s usage, and it introduced data to their company culture as a change agent. From Ben’s perspective even a 5% shift in engagement or conversion, positive or negative, means an experiment was worth running, because the data resolves debates, and improves the product and user experience.
If you are interested in learning more about this experiment check out Ben’s blog post on the topic.
