Matt Brzowski
Giving teams confidence in what to build.

The situation

Plenty of trees. No forest.

BambooHR had roughly 70 front-line designers and PMs making product decisions every day, and one researcher. Teams were trying to understand their customers and each was doing it alone. Studies stayed wherever the person who ran them left them. Whether a finding reached a decision came down to who remembered it and who happened to be in the room. Two teams would recruit the same customers in the same month without knowing it.

I never tried to serve that as a request queue, because the math is obvious: a study I ran personally would serve one team, once, and the finding would die in a Slack thread two weeks later. One researcher running a service model for 70 people is a bottleneck with a job title.

The throughput problem was the visible one. The deeper problem was that evidence was absent at the moment decisions got made. A designer deciding how a screen should work at 2pm on a Tuesday was not going to go dig through an archive first, and there was functionally no archive to dig through. Old studies sat in UserTesting and were never opened again. Recordings lived on individual Google Drives.

The social cost was the telling part. People skipped research to avoid the headache of running it, and to avoid looking like they did not already know the answer. When research is slow and asking for it signals uncertainty, the rational move for an individual is to not ask. Blaming individuals misreads it. The system produces that behavior, and better reports will not change the system.

How I framed it

In a forest the trees are the part you can see, and they are not the part doing the work. Underneath, root systems and mycorrhizal fungi move nutrients and chemical signals between trees that never touch. A tree benefits from what a tree fifty feet away already encountered.

That underground layer is what I set out to build. Not a better archive, which is just a taller tree. Something beneath the teams that moved evidence between them without anyone having to go looking for it.

In practice that meant reframing the job from producing insights to making evidence retrievable at the point of decision. A well-written report is a dead end. A searchable, connected, queryable corpus is infrastructure.

Three requirements fell out of that framing:

• Evidence has to live in one place with consistent structure, or it can't be found.
• It has to be reachable from the tools people already work in, or it won't be.
• It has to be traceable back to a source, or nothing distinguishes evidence from assertion.

The third one carried the most weight, and it is the opposite of what people assume. The risk in a fast-moving product org is false confidence more than distrust. Without traceability, a JTBD written by a PM reads as rigorous user-centered work whether or not a user ever informed it, and both land the same way in a review. Traceability is the only thing that makes the difference between a finding and a hunch visible at all.

What I built

The repository. Launched the company research repository with a consistent structure for studies, findings, and raw evidence. Reached 70% active use across 1,600 employees and became one of the most widely adopted AI tools in Product and Marketing.

That 70% is monthly active users from the repository's own analytics, measured against all 1,600 employees, a wider denominator than the product org. CX, Sales, Marketing, PMM, and IT were all referencing customer evidence in it. It also undercounts, because it only captured people opening the tool directly and missed API and MCP traffic, which for Product and Design had become the normal route in my last three to six months. The heaviest users were the least likely to show up in the number.

The pipeline. Shipped a v0 connection from Dovetail to Databricks, moving call transcripts into the company's data environment so customer voice sat alongside product and usage data instead of in a separate silo. Transcripts only, manually searchable, in progress when the research function was cut.

The connections. Set up MCP and API access so Claude, ChatGPT, Figma, and Maze could pull customer evidence inline. This is the part that mattered most.

The moment I point to: a designer working in Figma pulled from interviews their PM had run days earlier and made live design decisions off them. They had not been in the room for those conversations and did not ask anyone for a summary. The evidence was simply there, in the tool, at the moment they needed it.

That is the thesis in a single interaction. Nobody filed a request, nobody waited two weeks, no researcher was in the loop, and the decision was still grounded in what customers had actually said.

Key decisions and tradeoffs

Chose adoption over rigor at the entry point. Making it easy to contribute meant accepting variation in quality. Quality control went downstream, into standards and review. Gatekeeping upstream was the other option, and it makes the researcher the bottleneck again. An unused system is worth nothing regardless of how good it is, so engagement came first every time.

Dovetail’s tagging and highlighting are how you get clean, queryable data, and almost nobody used them. Tagging felt manual and overwhelming, so people skipped contributing instead of posting something untagged. I told them to stop tagging. Get the recording in, add a short bullet list of what you did with it, let AI draft the list, move on. Contribution cost dropped to nothing and the corpus started growing.

The bet was that cleanup is solvable later through the API and MCP layer, and an empty repository is not solvable at all. The cleanup layer never materialized, because the hiring never came and then the function was cut. The sequencing was still right. A messy corpus people use can be cleaned. A pristine one nobody contributes to is an empty database with good manners.

Chose connections over a destination. People do not visit research portals. The connections into Figma, Maze, and the AI tools were more work and less visible, and they are the reason it got used.

Shipped v0 of the pipeline without waiting for the full version. Transcripts only, manual search. Incomplete and in use, against a complete design that existed in a doc.

What came of it

The system became the company’s voice of the customer layer, and it arrived just in time to matter structurally.

As the company moved toward AI-assisted development, including an internal platform where teams composed their own discovery-to-shipped-PR pipelines, those pipelines needed a customer evidence source. Without the repository and the transcript pipeline feeding it, the available inputs would have been sales calls and support tickets.

That substitution is a strategy problem disguised as a data problem. Sales calls tell you what a prospect says to get a discount. Support tickets tell you what broke. Neither tells you what the job was, why someone chose a workaround, or what they expected the product to do. A pipeline fed on that produces confident, well-formatted, systematically skewed output at speed, with nobody checking the source. The faster the pipeline, the worse the damage.

The thing I would point at, if I had to point at one: a pattern surfacing in support conversations could reach a designer who had never read a support ticket. Teams stayed independent and stopped being isolated. That is the whole difference between a stand of trees and a forest.

What I'd do differently

I would check for leadership demand before taking the job.

A research system competes against incentives more than against ignorance. If the people making product decisions are not expected to show where those decisions came from, every hour spent on discovery is an hour not spent shipping, and shipping is what gets recognized. When clarity and speed conflict, speed wins, and the person who moved fast on a thin basis looks like the high performer.

I built the system and I enabled people to use it. Changing what an organization rewards sits outside what any individual contributor can do. I spent real effort on internal selling, and internal selling loses to an incentive.

What I would establish now, before accepting a role, is whether leaders already ask the question. How do we know this is a problem? What is our hypothesis for why it is happening, and what did we do to test it? Where a director asks that routinely, a research system compounds. Where nobody asks it, the best system in the world produces artifacts nobody is accountable for using. That is the screen I apply to research leadership roles now.

The situation

BambooHR built its reputation on what the early 2000s rewarded: clean, guided, form-based flows that made HR software approachable. That is a real asset right up until the interaction model itself changes.

The premise was settled before the research started, and settled on evidence. AI-native competitors were going from founding to credible product in under two years. A new entrant could put an HR admin on a voice call, talk through their company and needs, and come back with a fully formed HR setup: performance cycles, pay bands drawn from public data, the whole structure. A product built on twenty-year-old assumptions about how people get work done was not going to hold.

So the question was never whether to change. It was what to change into, and in what order.

BambooHR's head of design and principal designer were asked to define that direction. They produced a large prototype, developed with Claude, showing a radically different product. Chat instead of forms. AI that takes action on its own. Prediction instead of reaction. A back-of-the-napkin sketch at full fidelity, built to get a lot of people seeing the same possible future at once.

That is a useful artifact and a dangerous one. A prototype like that functions as a claim, even when everyone in the room calls it a plan. It bundles dozens of assumptions into a single impression, and an impression either wins people over or doesn't. Tested as one thing, it teaches you nothing actionable. If leadership likes it, it ships on conviction. If they don't, it dies on conviction.

How I framed it

I broke the prototype into 28 discrete bets, each stated as a claim that could turn out to be wrong, then sorted them by what happens if one fails.

That sorting is the useful part, and it is closer to structural engineering than to a backlog. A foundation claim carries everything above it, so it has to be settled first and it has to be small. Frame claims carry load and can be rebuilt. Finish claims are expensive to get wrong and survivable.

Five claims sat in the foundation:

• AI as a coworker. Users will accept and prefer a system that acts autonomously within set limits, with trust built on transparency, sourcing, permissions parity, and undo. The master mental model every other pattern assumes.
• Forms to chat. Conversation beats forms for setup and configuration, and forms drop to the fallback position.
• Wizards to chat. Conversation beats wizards for object creation, with the system knowing the anatomy of the object and coaching the user through it.
• Shared-state canvas plus chat. A chat panel alongside a live canvas building in sync is the core interaction model for creation and configuration.
• Risk-tiered actions and confirmations. Low-stakes actions execute immediately, high-stakes actions get structured confirmation, multi-step work shows progress.

Everything above assumed those five were true. If users would not talk to the product, or would not hand it real work, the other 23 were moot. All five completed.

Fourteen claims sat in the frame, nine in the finish layer. The shape is worth noticing: the layer carrying the most weight holds the fewest claims and the most certainty, and both reverse as you go up.

The frame was where it got tactical, and the questions changed shape. Delegation stopped being a yes or no and became a question of what it requires. How do users read a record of what AI did while they were away? What happens when an automation they described but never saw misbehaves? Does an AI-triaged homepage earn enough trust to replace a toolbox? The finish layer held the longer-horizon claims: voice, mobile, user-composed tools, personality settings.

What I did

I did not show anyone the prototype first. We started with discovery interviews with admins and buyers, including CTOs and heads of HR. What problems are you dealing with right now? How is AI changing them? What other tools are you considering, and why those? The goal was to map the problem space on its own terms, before anyone had seen our picture of the future.

That order matters. If the first thing a participant sees is a compelling prototype, everything after it is a reaction to our idea. Discovery first gave us an independent account of the problems, which is what you later judge the solution against.

Then we moved into solutioning, with opposing concepts. We put several directions in front of people, usually two at a time, deliberately opposed. The question was never which one do you prefer. Preference between two things someone just met is a weak signal, and it pushes you toward picking a winner too early. I wanted to know which aspects touched a real problem and which did not. Opposing pairs make that legible. People tell you what is wrong with each one, and the shape of the thing they need shows up in the gaps between them. Then we assembled across concepts instead of picking a winner.

Rigor was assigned deliberately, bet by bet. Twenty-eight bets and one researcher shifts the real question. Testing something well matters less than working out which bets deserve how much. Treat them all the same and you either spend a year on foundational research or you concept-test your way through a fundamental product shift. So every bet was scored on two axes before any method was chosen: how much risk and impact rides on getting it wrong, and how clearly we already understand the underlying problem. Those two select the method.

How method was assigned. Every one of the 28 bets sat in one of these cells before anyone designed a study.

Risk and impact	Clarity: low	Clarity: medium	Clarity: high
Risk: high	Foundational research. Attitudinal interviews and concept testing across personas.4 bets	Comparative usability testing against the current solution.4 bets	—
Risk: medium	Quick-turn concept test. Desirability before mechanics.7 bets	Standard concept and usability testing.7 bets	Lightweight validation. A first-click test is enough.1 bet
Risk: low	Quick concept test at low rigor, or nothing yet.3 bets	Light desirability check.1 bet	—

That scoring is what kept the program honest, and it is what let one researcher cover 28 bets without faking depth or stalling the roadmap.

The tracker also recorded what was blocking what. Several bets could not be researched until an upstream definition existed. Risk-tiered confirmations was blocked on a shared taxonomy of routine versus significant actions. AI insights was blocked on an agreed definition of what an insight even is. I named those as blockers and held the studies, so we did not generate findings against questions the team had not settled.

One bet, mobile, I marked as not a research question at all. It needed a product decision about whether canvas-plus-chat was first-class on mobile before any scoring was meaningful. Saying so is part of the job.

What came of it

Phase 1 completed and the direction held, with one significant refinement: start with chat, and always give the user direct control. That became concrete design decisions. A chat panel on the right, a live canvas on the left that updates as the conversation proceeds, direct editing on the canvas at any time, suggested options when the user has to make a choice, and visible live updates instead of a loading state.

That refinement came from watching people hit the same wall. Nobody asked for it.

Any time the AI produced an artifact or changed something on the canvas and the person could not then touch it directly, the session stalled. Performance cycle timing is the clearest example. The AI would infer six month cycles from context. The admin wanted twelve. They would click the timing field. Nothing. They would click again, assuming a misfire. Nothing. Then the pause, the confusion, the assumption that it must be a prototype bug, and finally the realization that the only way to change it is to go back to chat and ask. Type out a request about the six month cycle you already specified, wait, confirm. To change one number sitting right there on screen.

Every participant reported the same rule unprompted: let the AI do the work, let me fix it however I want. Small corrections want direct hands. Big or directional changes are worth a conversation, because those are the ones where you want to say your approach is off, we run twelve month cycles, which means the questions change and admins need access, so rebuild it.

That is a design principle you cannot get from asking people whether they like chat. It only appears when you watch someone try to change one number and fail.

What I'd do differently

Not much, and I want to be specific about why, because a short answer here reads as a dodge.

The approach was right for the conditions. Discovery before stimulus, opposing concepts instead of preference tests, bets sequenced so the foundational ones resolved first. It gave the team aligned direction on what was possible and let us move at the speed the market demanded, on just enough information to commit. With competitors going from nothing to credible in under two years, a slower and more certain process would have been the wrong trade.

The cost of just enough is that you commit before the evidence is complete, and that cost is real. I would manage it by keeping phase 2 questions about failure modes funded once a direction has momentum. Those get cut first, because the direction already feels decided and nobody wants to slow it down. They are also the ones that determine whether the thing survives contact with daily use.

The situation

If the evidence infrastructure above is the first layer, this is the second. The repository made findings retrievable. It did not create new research, and one researcher cannot generate enough of it for a 100+ person product org.

Research capacity had to come from the people already there. That meant designers and PMs running their own studies, plus a guild of 6 to 8 product designers with a dotted line to me, each committing 2 to 5 hours a week.

The constraint was real and it was never going to change, so I designed around it. Waiting for it to lift was never a real option.

How I framed it

Democratization fails in two directions. Too little structure and you get bad research that produces confident wrong answers, which is worse than no research. Too much structure and you become the bottleneck you were trying to remove, reviewing every study plan while the queue grows.

So the design problem is raising the floor without lowering the ceiling, using review that does not grow one-for-one with the number of studies. Three things carry that: what people start from, where you spend review, and how fast they can reach participants.

What I built

Standards and templates for planning, execution, synthesis, and sharing. The template is the mechanism. Most quality problems in distributed research are design problems in the study, and a good template prevents them before they happen, which review cannot do.

Enablement as a standing effort. Making sure people knew how to use Dovetail and Maze, how to use them together, and how to use AI to raise data and output quality without adding overhead to their week. The tooling only helps if using it is lighter than not using it.

Review at the plan stage. I reviewed the plan, which is where a bad study gets fixed cheaply, and let execution run without me. Finished studies never came back across my desk.

The guild as the mechanism for depth. Part reactive, helping whoever needed a usability test built in Maze that week. Part proactive, running the AI in HR research program on a cadence: surveys, desk research, interviews, persona and AI sentiment updates including how inclined people were to trust and delegate to AI.

Persona work baked into tools. The updated persona cards fed into skills teams could pull into their own work, so the research traveled with the work and never settled into a deck.

A 300+ customer feedback community in Slack, where customers applied to join a channel and talked directly with designers and PMs.

We built it from the customer list of anyone who had been active in a past beta program, people who had already shown they wanted a hand in what got built. The invite went out as a branded Product Feedback Circle, and the offer was the thing that made it work: direct access to the PMs and designers deciding what gets built, inside our own Slack.

That framing is why people joined. Most feedback programs ask customers for their time and give back a thank you. This one gave back proximity to the decision, which is what an engaged customer actually wants.

Ground rules kept it from burning out. No spamming requests. Always include a link if you are asking people to talk to you. Be kind. Designers, PMs, and I could all reach in directly within those limits.

Response was often overwhelming. Basic usability test invites filled immediately. Discussion topics ran to hundreds of thread replies. Live recruiting went from one to several days over email, at a much lower response rate, to immediate for Slack responses and same-day for interviews.

That is the bottleneck that usually kills distributed research. A designer with a two-day question will not wait a week for participants, so they decide without data. Same-day recruiting is what makes the whole democratized model viable.

Key decisions and tradeoffs

Spent the review budget on plans. Cheaper, earlier, and it built judgment in people, where editing their work would have built dependence on me.

Accepted uneven output. Study volume ran roughly 3 to 5 per quarter for the most active practitioners, and near zero for others. In a voluntary model, participation tracks individual conviction, and I chose to invest in the people who wanted it and let the rest opt out. Depth with a committed group produced more usable research than breadth with a reluctant one.

What came of it

Time from research request to shared insight went from weeks to days, and the biggest single contributor was recruiting. Same-day participants change what a team is willing to ask, because a question you can answer this week is worth asking and one that takes three weeks is not.

70 designers and PMs were trained and running their own studies. The guild had mapped the full AI in HR program and a persona research plan, and was executing against both when the function was cut in August 2026.

The honest limit: 2 to 5 hours a week per person, often squeezed by other priorities, caps what a guild can produce. It created real capacity where there had been none, and it was never going to equal a staffed research team. I would make the same trade again under the same constraint, and I would be clear-eyed about the ceiling.

What I'd do differently

I would have socialized the practice far more, and earlier.

I spent my effort on the system: standards, templates, tooling, review. All of it was necessary and none of it is visible. A template sitting in a folder does not make anyone curious about research, and it does not reach the people who were never going to raise their hand in the first place.

What I would do instead is teach in public. Run workshops. Open sessions where anyone could come learn a method, see what the practice was doing, and leave with something useful that week. Give value out continuously. Waiting for requests means you only ever reach the people who were already convinced.

Enablement is a distribution problem as much as a quality one. I built for the people who showed up. I should have spent more time giving people a reason to show up.

Content first, design second

The first thing I did was refuse to look at layouts. Open a design tool or a template gallery first and you end up with a beautiful page and thin writing, because the structure starts dictating what the content has to be.

So the case studies got drafted as plain text and argued over until they were right. The visual design came after, and it came out of the content: the page has a margin column carrying sources and notes because the work itself is about attaching evidence to claims. The layout carries the argument.

Where the real work went

Drafting with a model is fast and the first draft is always subtly wrong in ways that sound right. Roughly a dozen rounds of correction went into these case studies, and every correction was factual. All of them were mine to catch.

Two examples worth naming.

The first draft said I had moved away from running studies on request. I never ran a request queue at all, because the math never worked for one researcher and 70 practitioners. The model inferred a conventional career arc and wrote it down. Plausible, wrong, and it would have changed how the whole case study reads.

The second is the one I care about. I described my bet tracker from memory and said each bet recorded our prior confidence and a recruit plan by persona. The draft wrote that up persuasively. Then I pulled the actual spreadsheet, and those columns did not exist. What the tracker really did was score each bet on risk-impact and problem clarity, and let those two axes select the method. Better than what I had described, and different.

That is the failure mode worth planning for. The model elaborates confidently on whatever you tell it, including the parts you remembered wrong. Better prompting does not fix that. Going back to the source document does.

Which is the same argument the first case study makes about research evidence, arrived at from the other direction.

A written voice spec, maintained as a file

I keep a document of how I write and how I want to be written for. No em dashes. No "not X but Y" constructions. No corporate filler. Short sentences. Take a position instead of surveying options.

Handing that to the model at the start of a session changes the output more than any prompt I could write, and it is worth maintaining as a real artifact, because re-explaining it every session is wasted effort. When a draft comes back sounding generic, the spec is usually what needs updating. The prompt is rarely the problem.

Technical decisions

One hand-written HTML file on static hosting, with zero build tooling and zero runtime dependencies. A portfolio has to render correctly on a stranger's phone three years from now. Every dependency is a future break. The only external request is a web font.

Native <details> for the expandable case studies. It works without JavaScript, it is keyboard accessible and screen-reader correct by default, and it costs nothing. A custom accordion would have been more code and worse.

Two reading speeds on purpose. Collapsed, the page is about 600 words and scans in two minutes, which is what a recruiter actually does. Expanded, it is the full argument for someone who wants it. Designing for the skim first matches how the page actually gets read.

Static hosting, no backend. There is nothing to maintain and nothing to go down.

Where the model was not useful

Judgment about what matters. The draft gave equal weight to everything I had done. Deciding that the evidence infrastructure was the lead and the internal SDLC platform work should be cut entirely was not something it could do for me, because it required knowing which roles I want.

Restraint. Left alone it adds sections, qualifiers, and framing. Most of my editing was deletion.

Knowing what is confidential. It has no idea which of these details are safe to publish. Every judgment about what stays out was mine, and that judgment cannot be delegated.