Who is Luke Freiler?
Luke Freiler is the CEO and co-founder of Centercode, a platform that provides user-testing solutions to leading tech companies. With a software development background, he spearheads this SaaS platform's design that facilitates continuous audience engagement throughout product development. As a tech idealist, Luke aims to use technology to reduce friction and solve real problems.
In today's competitive market, where 49% of software projects fail, it's crucial to underscore the critical role of effective beta testing in a product's success. This alarming statistic is a wake-up call for companies to reevaluate their testing strategies.
To discuss these strategies, we called on Centercode, a platform that is revolutionizing the beta testing landscape. With its automated solution, Centercode is helping companies transition from merely market-ready to market-proven, ensuring that new products exceed customer expectations.
Our guest today is Luke Freiler, the co-founder of Centercode. A visionary who saw the gaps in traditional beta testing methods, Luke has been instrumental in shaping Centercode into an indispensable tool for successful product launches. In this interview, we delve into the intricacies of automating beta testing and the best tips for businesses who want to test their products efficiently.
Why is beta testing considered crucial for product development and marketing efforts?
Beta testing is about breaking outside the box and introducing a product to real users in their natural environments, with enough time to let them use and adopt it into their lives and workflows. These are factors that simply can't be reproduced in a closed environment and consistently introduce unidentified issues to improve quality, unbiased perspectives to improve user experience, and authentic praise to tune and
drive messaging.
While this has been a crucial aspect of product development for the last half of the century, the previous decade has upped the ante significantly as products have become more complex, connected and shifted to continuous iterative development. At the same time, audiences have become increasingly mainstream and thus less patient for products with unreasonable friction or simply don't solve their actual problems.
When it comes to recruiting beta testers, what are some creative and effective methods that companies like Centercode can use to find the right people to participate?
The formula is pretty simple - zero in on the problem your product aims to solve, then seek out people who have that problem, ideally a portion of whom are highly enthusiastic about solving that problem. If you already have a customer base, that's where to start. Building a new audience is quite similar to generating sales; only you're trying to convince people to give you their time rather than their money - with the benefit of offering them a voice in the direction your product takes, which will be highly enticing to the right people.
Beyond that, here are a few tactics that are practical in most scenarios:
- Leverage internal allies: Teams within your organization may have direct lines to the users you seek. Collaborating with support, marketing, and sales is a great way to connect with would-be testers. Chances are they already know a few names who would make great testers.
- Go where your testers are: The ideal tester might be a niche forum, engaging with a tweet, or coming as a referral from another stellar tester. Leveraging multiple channels – from social media platforms to industry forums – expands your pool of potential testers.
- Brand your recruitment: Establishing credibility and trust is essential. Whether through personalized emails, exclusive access tokens, or recognizable branding - ensure your outreach feels official, professional, and inviting.
What industries do you primarily serve, and are there any specific product types most commonly tested through your platform?
While our initial focus was traditional consumer and business technology, including both hardware and software - this category has effectively swallowed up most other markets in the last couple of decades, as virtually every category has started incorporating technology, connectivity, and continuous development.
Today, the most common products tested on our platform are traditional software, mobile apps, and connected hardware. This category has grown significant to include things that touch every aspect of our lives. In a modern context, this is uniquely a combination of hardware, software, and connected services - which all need to work in harmony for a highly diverse audience - which happens to be an incredibly tall task.
You worked with big brands such as Google, Amazon, and Intel. Could you tell us more about how Centercode helped these brands "perfect" their product quality and customer satisfaction?
Our primary goal is effectively always the same - we use our platform and services to enable these fantastic companies to work directly with their audience throughout all of the product development, making it easy for both sides to engage in building something unique.
For testers, that translates to getting access to an early product that aims to meet their needs, with the means to provide rich feedback rapidly, and for the builder, making it so that all of that feedback is prioritized, presentable, and directly shared with the right people and systems to take action.
What role does feedback play in beta testing, and how does Centercode facilitate communication between developers and beta testers?
Great feedback is the heart of every beta test - without it, there's absolutely no point. If you approach your test wrong, feedback can be non-existent or overwhelming, making the entire process miserable. Our platform aims to find the right balance between limiting friction and ensuring feedback is rich enough to be actionable. This extends beyond the users' direct feedback to include who they are and the collaborative sentiment and thoughts of other users experiencing the same product.
Security and confidentiality are paramount in beta testing. How does Centercode address these concerns to protect the tested products and the data provided by testers?
While there are numerous core mechanics like offering a highly secured technical environment and handling the legal aspects such as non-disclosure and participant agreements - there are two key things we help with that help further protect your IP:
- The first is selecting suitable testers. Tester profiling is a crucial aspect of the Centercode platform and methods. It helps ensure testers have the right background (appropriate experience and employment) and the motivation to participate.
- Second is the ongoing messaging. In reality, most test leaks aren't malicious but rather the result of confused testers who had no idea what the ground rules were. Centercode addresses this by enabling precise and consistent messaging before and throughout the entire test.
During the beta testing phase, how do you measure the success of a testing program, and what metrics do you use to evaluate the effectiveness of the feedback?
This question had no easy answer for decades - with every company, including ours, inventing siloed solutions that fit their unique needs and often evolved ambiguously with virtually every release. Frustrated by this, we decided to design a portable system that any company could use to measure the success of their programs, resulting in three straightforward KPIs. These became the foundation of a "delta test," a focused technology-driven methodology for a modern beta test.
These KPIs are:
- Health Score - A letter grade that rates how engaged testers are, built from two criteria: (1) Test Coverage - Are testers testing all the features and experiences you want to be tested?; and (2) Feedback Submitted - Do you have rich, actionable feedback to improve the product.
- Success Score - A number between 0 and 100, which rates the product's success in the eyes of the user/customer, derived from the relative impact of the issues, ideas, and praise they've submitted. The success score begins as a range, with the delta being dialed into an eventual single-digit number as the health increases. In other words, the more engagement we have, the more confident we are in the success sentiment.
- Impact Score - A percentage-based score illustrating the ROI of the test by the degree of impact it had on the product itself. This score represents the delta between the success score of what the product would have been had any corrected issues and implemented not been found and acted on versus what the actual release looks like with those changes in place.
These three values offer a consistent benchmark from project to project, each with transparent areas to target and improve. In addition to rolling up to represent the entire project, they're also available for each test phase and individual feature or experience, offering insights into all aspects of the product.
Could you discuss some of the common challenges companies face during beta testing and how Centercode assists in overcoming these challenges?
Effective planning - Designing the proper test, including what to test, who to target, how to test, and what will be done with the results, is very challenging if you lack the experience or process. For this, we offer plenty of free test plans and resources on our site while providing a prescriptive method within our platform to organize and fully execute your test plan.
Bandwidth - The most common role tasked with running a beta test is a product manager, who is already balancing countless other responsibilities. To address this, we rely on automation for virtually all aspects of testing - from tester recruitment to engagement and feedback prioritization to circulating, integration, and presenting results.
Finding suitable testers - Finding motivated testers who match your target market is often tricky. Beyond the tools in our platform to help with profiling and selection, we offer Betabound, our global tester network, as a free resource to help locate great testers for virtually any test.
Tester engagement - Far and away, the single most challenging problem in our space is getting testers to engage consistently and produce valuable feedback. While leveraging the right testers will go a long way, engaging them consistently with clear expectations and messaging is vital. Our platform automatically does this with our bot Ted - who builds a behavioral profile on each tester and leverages it to personalize a communication strategy to maximize their engagement throughout the entire test.
Feedback management - Once the feedback rolls in, organizing, prioritizing, and distributing become the next major challenges. While each piece may take the same space on a spreadsheet, the value can vary wildly, and it's critical to invest your limited resources into the most significant opportunities. This is where our platform shines - leveraging the concept of Feedback Scoring to prioritize feedback based on various factors, including frequency and severity. From there, feedback is generally integrated into systems like Jira, with generated dashboards featuring a slick presentation mode to share results easily.
After the beta testing phase, what additional support or services does Centercode offer companies regarding product launch and customer feedback?
Customers in most product categories now expect iterative releases to address outstanding issues, inject new functionality, and continue to connect to the array of other evolving products surrounding their own. Given this, successful companies are incorporating a continuous user testing process, ensuring that each new release is thoroughly tested in the wild. Our methodology, platform, and services are all designed to deliver constant value under this model:
- Dogfooding Programs - We often help companies leverage their employees to build "dogfooding" programs, which have recently taken off to increase employee engagement in remote environments by bringing them closer to the product in a highly collaborative context.
- Internal User Tests - This idea is commonly extended to leverage employees to test 3rd party software (a CRM or ERP, for example) being rolled out within an organization, ensuring it meets its needs - which can be highly valuable as so many software roll-outs simply fail.
- Competitive Tests - Our platform and services leverage the same methodology to offer competitive tests, wherein one or more competitors' products are tested to seek the same insights - what isn't working, what could be better, and what people love. This insight can be instrumental in bringing a product to market or iterating to overcome competitors.
What are your tips for businesses who want to successfully beta test a product?
Go out of your way to make your testers feel like they're part of an exclusive team. While this also helps with confidentiality, it generates enthusiasm and helps build a sense of accountability and ownership, which will significantly propel your engagement.
While some simple incentive is common in our space, it should be considered gratitude rather than payment. Additionally, a genuine thank you from the product manager goes a long way to ensuring testers come back to deliver repeatedly.
Keep up with Centercode, and check out our other exclusive interviews with industry experts!
