The Beta Group I Text Before Every RAXXO Tool Ships

Why Testing Alone Stopped Being Enough

I used to think testing meant using my own tool until I could not find anything wrong with it. That sounds reasonable until you notice the flaw sitting right inside the sentence: I already know how the tool works. I wrote every screen, every label, every error message. Finding a problem in something you built requires forgetting what you know about it, and that is not a skill I have, no matter how many times I click through a flow pretending to be new.

The first time this caught up with me was small and forgettable in isolation, a setting in an early build of one of my tools that made complete sense to me and to nobody else who touched it. I only found out because I happened to be watching over someone's shoulder, not because any internal check flagged it. That was the moment testing alone stopped being a real answer. Not because I was careless, but because carefulness cannot fix a blind spot you cannot see you have.

The instinct after something like that is to write more documentation, more tooltips, more explanation. I tried that route first, and it barely moved the outcome. People do not read the explanation before they hit the confusing part, they hit the confusing part and then decide whether the tool is worth the effort of reading anything at all. A clearer paragraph does not fix a step that should not require a paragraph in the first place.

What actually closed the gap was not writing more, it was watching someone else use the thing before I called it done. Not a survey, not a written review after the fact, an actual person opening the tool cold and me watching where their attention goes versus where I assumed it would go. Those two things line up less often than I would like to admit. A one-person studio has no product team to catch this by default, no colleague glancing over a shoulder in the normal course of a workday. If that check is going to happen at all, it has to be built in on purpose, the same way I built in the habit I wrote about in the four-pass check every RAXXO tool goes through before I call it shipped. That check catches what breaks. This one catches what confuses, which is a different failure and needed a different fix.

The two checks also happen at different points and answer different questions. The four-pass check runs against the tool itself, alone, asking whether every function does what it claims to do under normal and edge conditions. Nobody else needs to be present for that, it is a conversation between me and the thing I built. Confusion cannot be tested that way, because I am structurally incapable of being confused by a decision I made myself an hour earlier. I know exactly why every button is where it is and what every label means, which is precisely why I am the worst possible judge of whether either one is clear to someone encountering it cold. That gap does not close with more attention. It only closes by handing the keyboard to someone whose first reaction I cannot predict from the inside.

Who Is In the Group and How It Actually Works

The group I lean on now is small on purpose, fewer than ten people, mostly past customers who bought something, used it long enough to have opinions, and said yes when I asked if they would look at something before anyone else. I keep the group small because a small group answers a text the same day. A large list turns into a newsletter nobody opens on a deadline, and a newsletter is exactly the wrong tool for catching a problem before launch, not after.

Nobody in the group gets paid for this, and nobody is under any obligation to respond. What keeps it working is that I only ask when there is something genuinely worth their ten minutes, never as a formality and never on a schedule. If a build is not ready to be looked at, I do not send it half-finished just to keep the habit going. Asking too often is the fastest way to turn a favor into a chore, and a chore gets ignored.

The mechanics are deliberately low friction. A short message, a link, one or two specific questions rather than an open "what do you think." Open questions get polite non-answers. Specific questions, does this screen tell you what to do next without me explaining it first, get honest ones, sometimes uncomfortably honest, which is the entire point of asking in the first place. I would rather hear that a screen is confusing three days before launch than read it in a review three days after.

I do not treat every piece of feedback as instruction. Sometimes two people read the same screen two different ways and neither is wrong, that is a genuine design tradeoff, not a bug. The value is not in getting unanimous agreement, it is in seeing where real attention actually lands compared to where I assumed it would, which is information I cannot generate by testing my own work no matter how many passes I give it.

The Onboarding Step Nobody but Them Would Have Caught

The clearest example of why this matters happened with an early build of OhNine, the menu bar app I built for tracking Claude usage limits. I had rewritten the onboarding screen three times already, a process I wrote about separately in the onboarding screen I rewrote three times for OhNine, and by the third pass I was confident it was clear. It was clear to me. I had, after all, written all three versions.

One person in the beta group opened it and stalled on the very first step, the part where the app needed a small piece of setup before it could actually track anything. I had written that step assuming everyone opening the app already understood why the setup was necessary, because I understood why. She did not, and reasonably so, nothing on the screen explained the why before asking for the what. She got it working eventually, but the pause was long enough that I could tell, watching her share her screen, that a chunk of people would have quit right there and never opened the app again.

That single stall changed one sentence on the onboarding screen, a short line explaining why the setup mattered before asking for it. It sounds almost too small to be worth a story. But it was the exact kind of problem no amount of solo testing would have surfaced, because I could not experience the confusion of not already knowing the answer. I had tried to imagine a first-time user's confusion and guessed wrong about where it would actually happen. Watching an actual first-time user removed the guessing entirely.

What Changed About Launch Day Itself

Before this habit existed, launch day carried real uncertainty. The tool worked, the checks passed, and I still did not know how it would land with someone seeing it for the first time, because nobody had, not really, not in a way that counted. That uncertainty is not a comfortable way to ship something you have put real hours into.

Now launch day mostly confirms what I already know. By the time a tool goes live, a handful of real people have already used it cold, already hit whatever the confusing parts were, and those parts have already been fixed. The nerves that used to show up on launch day have mostly moved earlier, into the week I spend waiting on a few texts back, which is a much better place for them to live than the moment the tool actually goes live for everyone.

This connects to a broader instinct I have around anything that reaches a customer: catch the problem before it lands on them, not after. The beta group is that instinct applied to confusion specifically, one layer earlier than any launch-day safeguard can reach, because by the time a tool is live the moment to prevent a bad first impression has already passed.

The habit also changed what happens in the days right after a tool goes live, which I wrote about separately in the first week after I ship a RAXXO tool. That first week used to be where I learned most of what I now learn earlier, watching support messages arrive and trying to reconstruct, after the fact, where someone got stuck. Reconstructing confusion from a support message is slow and often wrong, because people describe the symptom, not the moment things went sideways. Watching it happen live, before launch, skips the reconstruction entirely. The first week after launch is calmer now, not because fewer people use the tool in that window, but because fewer of them are running into something a stranger already ran into first, on my screen, with me watching.

Bottom Line

None of this replaces careful building. It replaces the assumption that careful building alone is enough, which it never was, because the person doing the building is the one person least equipped to notice what a stranger would find confusing. A small, honest group of people who already use RAXXO tools, asked specific questions at the right moment, catches a category of problem that no solo pass ever will. The fix that mattered most was not a smarter checklist or a longer testing session. It was handing the tool to someone else before I was fully ready to, and paying close attention to exactly where they got stuck.

Keep reading

AI ToolsWhy Anthropic's Possible IPO Could Break Records 9 min read

BusinessThe Shared Snippet That Almost Broke Three RAXXO Tools at Once 9 min read

BusinessWhy I Reply to Every RAXXO Review, Good or Bad 9 min read