What 25 Years of Usability Testing Teaches You About Building Better Digital Experiences

If you've ever watched a real user completely ignore the button your whole team spent three weeks debating, you'll feel this interview in your bones.

Toby Biddle is the founder and CEO of Loop11, a remote usability testing platform used by product teams, agencies, ecommerce brands, universities, and enterprise organizations around the world. Before building Loop11, Toby spent years running a UX consulting firm, sitting clients behind one-way mirrors, watching users struggle in real time, and quietly realizing the industry needed a better way to collect evidence at scale.

He launched Loop11 in 2009. What started as a tool to make usability testing more measurable has evolved into an AI-assisted research platform helping teams move from data to insight faster than ever before.

We sat down with Toby to talk about the origin of Loop11, what businesses consistently get wrong about their websites, where AI fits into UX research, and what one thing you could do this week to understand your users better.

‍

Origin Story & Vision

Loop11 grew out of your UX consulting work. What was the specific "aha moment" that told you a self-serve testing tool needed to exist?

The aha moment came from our consulting work.

At the time, most of the UX research we were doing was qualitative. We were testing one participant at a time in a lab, usually with clients watching from behind a one-way mirror. That work was valuable, but it had limits.

We wanted to generate quantitative UX statistics and metrics, not just observational findings from a small number of lab sessions. To do that properly, we needed a way for lots of participants to complete usability tests in their own time, from wherever they were.

That led us to the idea of a survey-style usability testing platform.

Participants could be given tasks, complete those tasks remotely, and we could then collect behavioral data, success rates, time on task, comments, satisfaction scores, and other metrics at scale.

No one was doing that particularly well at the time. So we thought, “Right, looks like we might have to build it ourselves.”

‍

You founded Loop11 back in 2009. How has your original vision for the product evolved compared to what it is today?

The original vision was to make usability testing accessible to more people, but also to make it more measurable.

Traditional lab-based testing was great for qualitative insight, but it was harder to produce meaningful UX metrics from small, one-at-a-time sessions. Loop11 was designed to help teams collect both qualitative and quantitative data, including task success, time on task, user comments, click paths, satisfaction ratings, and other useful measures.

That core idea has stayed the same.

What has changed is what we can now do with the data.

In the early days, Loop11 helped customers collect usability data at scale. Today, the bigger opportunity is helping customers get to the insights faster. It’s not enough to give people a dashboard full of data and leave them to work it all out themselves.

With AI, we can help identify patterns, summarize findings, highlight friction points, and bring the key insights to the surface much quicker.

So the vision has evolved from “make usability testing accessible and measurable” to “help teams understand digital experience faster, with both data and insight.”

‍

What was the hardest part of the early days of building Loop11, and how did you push through it?

The hardest part was probably convincing people that remote, unmoderated testing was legitimate.

At the time, a lot of UX people were used to lab-based research. You sat a participant in front of a screen, watched them through a one-way mirror, took notes, and discussed what happened afterwards.

That process had value. We knew that because we were doing it ourselves. But we also knew it didn’t scale very well if you wanted quantitative data.

So part of the challenge was explaining that Loop11 wasn’t replacing qualitative research. It was adding another layer. It allowed teams to test with more participants, collect useful metrics, and still capture open-ended feedback.

We pushed through by focusing on the evidence. Once customers saw task success rates, time on task, user comments, and where people struggled, the value became obvious.

Also, a fair bit of stubbornness helped. You need some of that in SaaS, probably more than is healthy.

‍

What was the moment you realized Loop11 was going to be a real business?

It was when customers started using it without us needing to explain everything.

In the consulting days, we were always in the middle of the process. We designed the study, recruited participants, moderated sessions, analyzed results, and presented findings.

With Loop11, people we’d never met were creating studies, launching tests, and getting value from the platform on their own. That was the shift.

It meant Loop11 wasn’t just a tool we used internally. It was something other teams could use independently, in their own way, on their own schedule.

That’s when it started to feel like a real product business.

‍

About Loop11 & User Testing

Loop11 supports unmoderated testing across any digital interface. What types of businesses or teams tend to get the most value from that flexibility?

The teams that get the most value are usually the ones with a lot of digital decisions to make and not enough time to debate all of them endlessly.

That includes product teams, UX teams, agencies, ecommerce teams, universities, financial services, government departments, and SaaS companies.

The flexibility matters because the thing you need to test is not always a finished website. It might be a Figma prototype, a new checkout flow, a mobile experience, a content structure, a pricing page, or a competitor’s website.

The best teams use testing to reduce guesswork. They don’t wait until everything is perfect. They test early, learn quickly, and make better decisions.

‍

What are the most surprising or counterintuitive things customers discover when they first test their websites?

The biggest surprise is usually that users don’t see what the business thinks is obvious.

Companies spend months refining navigation, copy, page layouts, CTAs, and product explanations. Then a participant lands on the page and completely misses the thing everyone internally thought was impossible to miss.

Another common one is that users often behave very differently from how they say they behave. That’s why task-based testing is so useful. It gives you behavior, not just opinion.

And the third surprise is how small some of the problems are. A label, a button position, a missing price, an unclear next step. Tiny things can create a lot of friction.

‍

How do you decide which new features to prioritize when there are so many directions you could take the product?

You have to be disciplined, otherwise the product becomes a junk drawer. We look at three things.

First, is it solving a real customer problem? Not a theoretical one, a real one.

Second, does it fit the direction of the platform? There are lots of good ideas that still don’t belong in the product.

Third, will it help customers get to insight faster?

That last one has become more important over time. In the past, a lot of research tools focused on collecting data. That still matters, but customers are busy. They don’t just want more charts, tables, clips, comments, and spreadsheets to sift through.

They want to know what matters, where users struggled, and what they should pay attention to next.

That’s where AI is becoming useful. It can help bring the insight closer to the customer, faster.

‍

What's the one thing you see on almost every website you test that you wish companies would fix first?

Unclear information hierarchy.

Most websites try to say too much, too soon, to too many audiences. The result is that users have to work harder than they should.

The first question a user has is usually very simple: “Am I in the right place, and can this help me?”

A lot of websites don’t answer that quickly enough.

If companies fixed that first, many other things would improve.

‍

If you could only keep one metric from a usability study, what would it be and why?

Task success.

There are lots of useful metrics, including time on task, lostness, satisfaction, NPS, SUS, and qualitative comments. But if I had to keep one, I’d keep task success.

At the end of the day, could the user do the thing they came to do?

If they couldn’t, that’s the signal you need to pay attention to. Everything else helps explain why.

The useful thing about Loop11 is that you don’t have to choose between numbers and comments. You can see the quant data, then use the qual feedback to understand what was going on. And increasingly, AI helps connect those dots faster.

‍

The UX Industry & Trends

You've been in the UX space since 2001 with your consulting firm. How has the attitude of businesses toward user testing shifted over that time?

The shift has been huge.

In the early 2000s, user testing was often treated as a nice-to-have. Something you did at the end if there was budget left, which of course there often wasn’t.

Now, most serious digital teams understand that user testing is part of building better products. It’s not some mysterious UX ritual. It’s just a practical way to find out where people struggle.

That said, there’s still a gap between believing in user testing and actually doing it regularly. A lot of companies still test too late, or only when there’s a redesign.

The more mature teams treat it as a normal part of decision-making.

‍

With AI tools now generating UI and copy at speed, do you think user testing is becoming more important or less, and why?

More important.

AI can help teams create options faster, but it doesn’t magically tell you which option works for real users.

In fact, the faster we generate interfaces, pages, copy, and workflows, the more we need a reality check. Otherwise, we’re just producing more things that may or may not work.

AI will change how UX work gets done, but it won’t remove the need to observe behavior. Users still have goals, doubts, habits, distractions, and frustrations.

Where AI does help is in getting from data to insight faster. A usability study can produce a lot of information: task results, comments, paths, recordings, survey responses, ratings, and more. AI can help find patterns and surface the important findings quickly.

So AI doesn’t make user testing less important. It makes fast, regular, evidence-based testing much more practical.

‍

What's a common mistake you see companies make when they first start doing usability testing?

They try to test too much at once.

A good usability test needs focus. You don’t need to test the entire website in one go. Pick the key tasks that matter most to the business and the user.

The other mistake is leading the participant. Teams sometimes write tasks that give away the answer or nudge people toward the right path.

A good task should describe the goal, not the steps.

‍

You've run thousands of user tests. What's the one thing users consistently do that still surprises designers?

They ignore things.

They ignore banners. They ignore carefully written copy. They ignore big buttons. They ignore things the design team has spent weeks discussing.

Designers often assume that because something is visible, users will notice it. That’s not how people behave.

Users are task-focused. They scan, guess, click, backtrack, and move on. They don’t study your interface. They use it while thinking about something else.

That still catches teams out.

‍

What's something the UX industry gets completely wrong that nobody really wants to say out loud?

We sometimes make UX sound more complicated than it needs to be.

There is a place for deep research, frameworks, models, maps, and workshops. But a lot of the time, the most useful thing is much simpler: put a real task in front of a real user and watch where they struggle.

The industry can get a bit too attached to process. Clients don’t need theatre. They need evidence that helps them make better decisions.

That may upset a few Post-it note enthusiasts, but I’ll survive!

‍

The SaaS Journey

Loop11 has grown to a large customer base globally. What's been the biggest unexpected challenge of scaling a SaaS product?

The biggest challenge is that the product never sits still.

Customers change. Technology changes. Browsers change. Devices change. Privacy expectations change. The competitive market changes. And every time you think you’ve solved one thing, another thing pops up.

With a SaaS product, you are never really “finished”. You’re constantly improving, maintaining, supporting, explaining, fixing, and prioritizing.

The unexpected part is how much discipline that takes. It’s easy to add. It’s harder to keep the product clear and useful.

‍

How do you compete in a market with a lot of user testing tools out there? What's your approach to differentiation?

We focus on flexibility, depth, and helping teams get to useful insight faster.

Some tools are very polished for one narrow use case. Loop11 has always been strong when teams need to test different types of digital experiences, from live websites to prototypes, mobile experiences, IA, benchmarks, and more.

Another important part of the product is the combination of qualitative and quantitative data. You can see what happened statistically, but you can also understand why it happened through participant comments and behavioral evidence.

The next layer is insight.

We don’t want customers to feel like they’ve run a study and then been handed a pile of data to sift through on their own. AI helps us summarize findings, identify patterns, and highlight the issues that deserve attention.

Our approach is to make user testing flexible, measurable, and faster to act on.

‍

What's the biggest myth about building a SaaS business that you believed early on but have since changed your mind about?

That if you build a good product, growth will mostly take care of itself.

That’s not true.

A good product matters, of course. But distribution, positioning, timing, support, onboarding, pricing, and trust all matter too.

SaaS is not just a product game. It’s an operations game, a marketing game, a customer support game, and sometimes a patience game.

Mostly patience, with invoices.

‍

At what stage should a startup be doing user testing: day one, or only once they have something real to show?

Day one, but the type of testing should match the stage.

You don’t need a polished product to learn from users. Early on, you can test a value proposition, a landing page, a prototype, a sign-up flow, or even a competitor experience.

The mistake is thinking user testing only starts once the product is built. By then, you may have already made expensive assumptions.

Start small. Test the riskiest assumptions first.

‍

Working with KARL Mission

You've worked with KARL Mission on SEO, email onboarding, and CRO projects. From your perspective, how do those digital marketing efforts complement what Loop11 does as a user testing tool?

They complement it really well.

User testing helps you understand where people struggle, what they miss, and why they don’t convert. SEO, onboarding, and CRO help you act on those insights across acquisition and conversion.

It’s all connected.

There’s no point driving more traffic to a page if users don’t understand the offer. There’s no point improving rankings if the onboarding experience loses people. And there’s no point tweaking conversion rates without understanding the behavior behind the numbers.

The best digital marketing work is not just about getting more people in. It’s about making the whole experience work better once they arrive.

That’s also where Loop11 and KARL Mission’s work overlap nicely. Loop11 helps identify where users struggle, and good SEO, onboarding, and CRO work helps turn those findings into better pages, better journeys, and better conversion outcomes.

‍

What's one thing the KARL Mission team helped you see about Loop11's own digital experience that surprised you?

The useful thing about working with KARL was having another team look at Loop11 from the outside.

When you work on your own product for years, you know too much. You understand the history, the edge cases, the product logic, and all the reasons things are the way they are. But new visitors don’t have that context.

KARL helped us look at areas like SEO, onboarding, and conversion with fresher eyes. That’s always valuable.

It’s also a nice reminder that even a user testing company needs outsiders to point out its own blind spots. 

‍

What’s Next

What's next for Loop11? Are there any new features or directions you're excited about?

AI is the area I’m most interested in right now.

Not because AI replaces user testing. I don’t think it does. But it can help teams move faster, analyze results more efficiently, and get to insights sooner.

That’s a big shift.

Historically, usability testing tools have been very good at collecting data. The harder part for customers is often working out what all that data means, especially when they’re busy or don’t have a large research team.

We’ve been working on AI-driven features like AI Insights and AI Browser Agents, which open up some interesting possibilities. AI can help summarize findings, identify patterns, flag friction points, and make the research easier to act on.

The future is likely a mix of human insight, quantitative UX data, qualitative feedback, and AI-assisted analysis.

The trick is using AI where it helps, without pretending it can understand everything humans do.

‍

For someone who's never done user testing before, what's one thing they could do this week to start understanding their users better?

Pick one important task on your website and ask five people to try it.

Don’t explain where to click. Don’t defend the design. Don’t help them unless they’re completely stuck.

Just watch what they do and ask them to think out loud.

You’ll learn more in an hour than you’ll get from a week of internal opinions.

And you’ll probably be slightly horrified, which is usually a good sign.

‍

Conclusion

Toby's perspective is a useful gut-check for any business that thinks they have a good handle on how their website performs. Spoiler: you probably know less than you think, and your users are quietly struggling with something you'd fix in an afternoon if only you knew about it.

The through-line in everything Toby shared is that user testing doesn't have to be complicated, expensive, or reserved for big UX teams with large budgets. It just has to happen. Regularly, with focus, and with a genuine willingness to be wrong about what you thought was obvious.

If you're curious about how user experience and behavior connects to your conversion rates, that's exactly what we work on at KARL Mission. We combine UX research, CRO strategy, and data analysis to help businesses find where they're losing people and fix it.

‍

Want to Know Where Your Website is Losing Users?

We offer a free consultation to help you identify the biggest friction points in your digital experience and figure out what's worth fixing first.