CRO Research Evidence:
Homework Leads To Higher Conversion Rates

Cynthia Meyer
Cynthia Meyer
Content Marketing Manager

Editor’s Note: This CRO research marketing infographic is part of our 25 part series.
Subscribe to our blog and enjoy more creative posts like this one.
Also, this post has been updated with new links and content.
Original Publication Date: December 3, 2016

You know when you have a gut feeling about something and you’re usually right?

If only it was that easy when it comes to conversion rate optimization (CRO).

Because when it comes to CRO research, there’s a very good chance that you’re being misled by that same gut feeling. Cognitive biases and assumptions can easily lead you down the wrong path.

The most effective way to improve your ad campaigns is to put on your detective hat and do your due diligence. In the online marketing world, that means split testing, which can make or break your CRO goals.

More importantly, making changes based on test results can produce results more quickly than making changes based on your gut.

So why not be smart about it? Why not run a calculated, methodical research plan so you have more accurate data on your visitors?

With the right insights you can reach the right conclusions about your visitors. And with the right tools, you can use those conclusions to pursue your CRO and revenue goals.

We’ve teamed up with Conversion Sciences to bring you the following tips for your own conversion research. It’s time to stop following your gut and start following accurate CRO research methods, so you can start growing your business even quicker.

What is Conversion Research?

We actually defined conversion research in this conversion research blog post:

“Conversion research is a strategic approach focused on identifying and interpreting relevant data to find possible points of friction in a sales funnel – and, ultimately, to increase the overall conversion rate.”

Without the right understanding of your audience, you’ll be chasing the wrong leads and following dead ends. And by that I mean you probably won’t get the conversion rates you’re looking for.

You might even be designing landing pages with friction points that are preventing your visitors from clicking your call-to-action (CTA) buttons.

CRO research can help you uncover useful insights about your target audiences. This can help you reach your visitors and convert them on your landing pages.

Quantitative CRO Research – Numbers To Crunch

Quantitative research is the number-crunching kind of research. You’ll gather and analyze data, statistics, formulas, charts, and graphs.

You can use these reports to see where exactly your problem points are, where you can improve, and where you should invest more time and money.

There are several formats that quantitative research can take on. Here are some of the reporting formats that can help you analyze your visitors’ data:

  • Mouse behaviors: Heat mapping, scroll mapping, click mapping, confetti mapping
  • Conversion stats: Click-through rates, opt-in rates, impression shares
  • Device stats: Mobile, desktop, tablet
  • Time-based stats: Load times, time spent on page, time before CTA is clicked
  • Traffic reporting: Referrer, number of people, pageviews, visited site

In a study of load times, for example, Kissmetrics found that:

“A 1-second delay in page response can result in a 7% reduction in conversions.”

What does this quantitative research mean for an eCommerce site?

“If an eCommerce site is making $100k per day, a 1-second page delay could potentially cost $2.5m in lost sales per year.”

Here’s another example from Kissmetrics on quantitative research:

cro research - funnel report

Conversion funnel stats – image source

Quantitative CRO research also details where people abandon your landing pages. You can see where your visitors are experiencing hang-ups in your conversion funnel.

Finally, you can address specific friction points on your landing pages to improve your conversion rates.

Qualitative CRO Research – Feedback To Digest

There’s more to this than just numbers. Qualitative CRO research can help you identify the humanistic stuff, like how your visitors are feeling, and what types of questions, thoughts, and opinions they have.

Feedback tools like surveys, chat boxes, questionnaires, and forms can help you gather this type of quantitative analysis.

Here’s an example of a Qualaroo exit survey, which can help you better understand your visitors’ thoughts on their way out:

CRO research qualaroo surveys

Asking them in the moment will help you capture visitor’s fresh thoughts – image source

Another example of effective qualitative reasoning is this case study with StubHub.

Using the UserTesting platform, StubHub collected qualitative data that helped them understand their user experience.

Here’s what they learned:

“By listening to and watching UserTesting panels on their site, StubHub discovered that a link labeled ‘See Details’ was causing confusion and in need of some improvement to boost conversion rates.”

With this qualitative info, they were able to implement some changes that included updating their button copy from ‘See Details’ to ‘Go.’

CRO research Go Button Highlight

The ‘Go’ button… much more clear – image source

The result? StubHub increased their rate by 2.6%.

Other methods that you can use for pulling in qualitative data are:

  • Focus groups
  • In-person interviews
  • Direct observation
  • Frequently Asked Questions


User & Usability Testing – Do Visitors Do What They Say?

Even with all the quantitative and qualitative analysis, there could be more that you’re missing. Usability testing is another way to seek the truth about how your visitors behave.

By conducting user testing, you can find out by tasking, recording, and observing real users interacting with your site.

We may have good intentions when answering usability test questions, but oftentimes what we say and what we do are completely different things.

If you’re human (which is likely), you’re likely to fall for another cognitive bias — saying what you think moderators want to hear, or what you think you’re supposed to say.

Here are five category questions to help you structure your usability tests, provided by Jakob Nielsen at Nielsen Norman Group:

  1. Learnability – how easy is it for users to do basic tasks the first time they see your site?
  2. Efficiency – how quickly are users performing the tasks?
  3. Memorability – when users return, are they still up to speed?
  4. Errors – how many and how severe are the errors, and are the users able to recover?
  5. Satisfaction – how pleasant is the experience?

You can conduct testing in three main formats: moderated in-person, moderated remote, and unmoderated remote. Whatever format you choose, start by creating an environment where the user has the freedom to act as natural as possible for real-life results.

Recruiting Users For Testing

Daunted by the recruiting process?

Don’t be.

According to Jakob Nielsen, you really only need about five users per test. Here’s why:

user testing

No need for more than six users – image source

The amount of new findings caps out after about five users. Spread your recruited test subjects over a multitude of usability tests and spend your time uncovering more insights.

Segmented Analytics = Easier Access To The Right Insights

By breaking your CRO research down into smaller pieces, you can take a deep dive into specific subjects and uncover some real truths about your visitors.

Sure, default dashboards are helpful from a quick ‘n’ dirty overview perspective. But the more you know about a specific area of your audience, the more impactful you can make your landing page adjustments.

Segmented analytics can uncover interesting findings below the surface. But first, you have to break your analytics down into subsets based on things like:

  • Geographic location: City, state, country
  • Product or service type: Visitors to different category sections of your site
  • User subsets: Repeat visitors, first-time visitors
  • Session subsets: Sessions originating from specific campaigns, which sessions turned into conversions
  • Subsets of hits: Interactions during a session (pageview, event, transaction), segment conversion levels (i.e. those that created revenue over a certain dollar amount)

Here’s an example of what conversion segment setup looks like in Google Analytics:

conversion segments

Create multiple user-defined segments – image source

Once your reporting parameters are all set up to the way you want it, you can save your reports for quick and easy access in the future.

Comparing similar reports across different time periods can be useful in checking for significant changes month-over-month or year-over-year.

The Priority List – What To Test Next

So now that you know the importance of doing your homework, research, testing and analysis… are you confused about what to test first?

Fret no more.

Start by creating a list of what you want to test, and then prioritize them so you can put your first efforts toward the things that need the most attention.

To help with this, you can choose from a few different prioritization frameworks out there.

Here’s one called the PIE framework, which stands for Potential, Importance, and Ease:

CRO research pie score

The higher the PIE score, the more urgent the topic – image source

According to WiderFunnel, some questions you can ask about PIE are:

  • Potential: How much improvement can be made on the pages?
  • Importance: How valuable is the traffic to the pages?
  • Ease: How complicated will the test be to implement on the page or template?

ConversionXL has their own framework dubbed the PXL framework, which looks like this:

pxl framework

Still prioritizing your topic urgency by score – image source

The PXL framework focuses on making certain ratings more objective and fosters a more data-informed mindset.

There are also two more frameworks called ICE. One stands for Impact, Confidence, Ease, the other for Impact, Cost, Effort.

Whatever framework you choose, prioritizing the order of your tests can help you allocate time, money, and effort effectively.

Closing Thoughts

Laying the groundwork for proper CRO research can help you to uncover important insights about your visitors.

More importantly, insights that are based on facts and figures — and not your gut — will help you make updates that will generate results.

The more accurate the picture of your audience is, the better you can strategize and tailor to each of your visitor’s needs on your landing pages. And that, in turn, leads to higher conversion rates. Science!