The respondent experience: the whole journey, and how to improve it

The invitation: routed by earnings per click

Most of the 1,363 people at the left edge of the map were not invited to this study. They clicked on an offer: a survey of a stated length, for a stated reward, in a panel app or on a rewards site. A router then sent them wherever the expected earnings per click were highest, on whatever it knew about them, which is usually a handful of basic socio-demographic answers, collected once from a question library that has not been reworked or improved in years. Many pass through several routers in a row, from a panel to an aggregator to an exchange to the study, and those routers do not share data. So the person answers the same age, gender and household questions again at every hop, before any survey has started. The map draws that as one supplier and one gate; the note on the first band says what it hides. From the supplier's own seat, the benchmark puts actual incidence at 45.9%: more than half of the people it screens for a study do not fit. That is what routing on earnings per click looks like from the inside.

The fix. Route on what is already known, and carry it across hops. A router that knows a person's profile can send them only where they are likely to qualify, and a chain that passes the answers down does not ask them again. Where the match is uncertain, say so in the offer rather than promising a reward the person has a coin-flip chance of earning.

Who can act. Supplier owns the routing objective and the profile data it runs on. Structure: routers belong to different companies, and none of them is paid to share what it knows with the next.

Why it rarely happens. Earnings per click is a good objective for the router and a bad one for the person: it is maximized by sending everyone everywhere and letting the screeners sort it out. It is also a volume game. Suppliers compete against each other on the same studies, and the one that sends more traffic faster fills the quota, so nobody wants to be the router that sends fewer, better-matched people and watches the study fill through someone else. And sharing profile data across hops means trusting the next company with your asset.

Pre-survey checks: judged by systems that do not agree

Before the first question, a person usually passes through more than one fraud and quality check: the supplier's, then, when the buyer hosts the questionnaire, a second set at the survey link, sometimes a third from an aggregator in between. Sometimes there is none at all: in the benchmark, 5% of suppliers and 12% of research agencies record no pre-survey controls, and just over half of the agencies rely entirely on their supplier's. Where the checks exist, they do not return the same verdict. Each runs its own rules at its own thresholds, and a device or a network that passes one fails the next. When several suppliers feed the same study, the same person can also arrive twice, through two of them, and only the survey side can see that duplicate. In this scenario 22% of the people routed at the study are rejected by a quality system before they see it: 305 of 1,363, with nothing to read and nothing to appeal. Add the supplier's pre-screen, the next section, and 36% never see the survey at all. The honest ones caught by mistake are false positives, what the Global Data Quality glossary describes as "incorrectly validated participants identified as of poor quality", and nobody in the chain measures how many there are.

The fix. One verdict, made where the signals are, by someone paid to get it right. The supplier has the most signals: it can verify identity, it holds the login history of the account and of the IP address, and it sees the person across many studies, not one session. It also has the least reason to use them. An account it removes is revenue it stops earning, and a fraudster is profitable right up to the moment they cash out. So the verdict needs a party paid for precision rather than for throughput, and it needs to travel with the person: one verdict instead of three that disagree, and a reason attached to a rejection instead of silence.

Who can act. Supplier runs the richest checks and holds the history. Buyer adds the second gate, and decides whether to trust the first.

Why it rarely happens. Trust. Buyers add their own gate because the supplier is paid per complete, its thresholds are set with that in mind, and nobody can audit them. So each side controls what it can, the checks stack, and the discrepancies land on the person. A high catch rate reads as diligence to everyone upstream, so no one is paid to ask how many of the caught were honest.

Screen-outs and over-quota: rejected on criteria nobody passed along

Answer profiling questions you have answered a hundred times, then read "unfortunately, you do not qualify" three minutes in, unpaid and unexplained. In this scenario 31% of survey starts end that way, 37% once over-quota is added. Everyone in that band had already passed a pre-screen upstream, where the supplier had turned away many more on the same criteria. The over-quota is a cousin of the screen-out: the person qualified, and the cell they fit was already full, closed mid-session, or throttled. From the inside it is the same dead end and the same zero. In our panelist survey, disqualification is the frustration named most, at 63%, ahead of surveys running long and ahead of low pay itself, and it compounds with pay: the share of panelists who feel underpaid falls from 74% among those who finish 3 or fewer surveys out of 10, to 23% among those who finish 8 or more. Same rewards. The difference is how often the journey ends in a rejection.

“I will get almost to the end of the survey, and then it will kick me out and say that I didn't match.”

Verbatim, Enlightn survey of 300 active online panelists, 2026

Repeat that enough and the person adapts. Screened out three times on the same questions, they learn what the screener wants, and the fourth time they qualify with a profile that is not theirs. In B2B it is the decision-maker answer that opens every survey. The industry files this under overclaiming and treats it as fraud. Most of it is a genuine person doing what the journey taught them, and the quality check that removes them ten minutes later is punishing a lesson the screener gave. That is the direct line from respondent experience to data quality: every screen-out teaches, and part of what the red bands remove further down the map was taught here.

Most of the band comes from the supplier not having the criteria. Either the system cannot carry them, because the structured targeting fields a router can use are a fraction of what a screener asks, or the buyer does not share them, out of fear that the supplier builds a leading pre-screener that teaches people the right answers. So the supplier knows the study's incidence and rarely knows why it is not 100%. There is no feedback loop either: it sees that its respondents were terminated, not which question terminated them.

The fix. Pass the criteria down, and pass the reason back. A supplier that knows which criterion fails can stop sending the people who fail it, and a person who is only asked what nobody knows yet gets a shorter screener and a better chance. Make the promise the most burned participants actually want: among panelists who flag disqualification as their top frustration, 35% pick "you'll likely qualify" as the single most important promise a panel could make, versus 18% of everyone else. Panelists themselves draw the line to data quality: asked what would most improve the quality of their own answers, "fewer screen-outs / better targeting" came second (50%) only to higher pay. For over-quota, pace the cells and close them before the click rather than after. Where a terminate still happens, pay something for the time. Panelists describe the fix in almost the same words a sample provider would:

“a more intelligent routing system that matches surveys with the right professionals from the start. Respecting our time by providing high quality.”

Verbatim, Enlightn survey of 300 active online panelists, 2026

Who can act. Buyer decides what criteria to share, whether a terminate reason travels back, and how quotas are designed. Supplier holds the profile data, applies the criteria it is given, and paces the invitations.

Why it rarely happens. Profiling data deep enough to spare people the screener is hard to maintain: it has to be structured, kept current, and mapped to the criteria a study actually uses, and few suppliers have invested in it. Sharing criteria means trusting the supplier not to game them, and returning terminate reasons is work nobody is paid for. Pacing costs speed, and speed is what buyers pay for.

Partials: the promised ten minutes

Told ten minutes, given twenty-five. A 40-row grid on a phone. A session that breaks at question 30 and pays nothing. In the benchmark the abandon rate is 10.5% of records for research agencies and 16.8% for suppliers: the same people, counted from two doors. And the map splits each partial band into two populations most disposition systems cannot separate: true drop-offs, which are a decision, and technical failures, which are not.

The fix. Design surveys you would enjoy taking yourself: if you would not sit through that grid on your phone, neither will they. State the real length and honor it, design for the phone the sample actually arrives on, and instrument the survey so drop-off and breakage separate: one is a questionnaire-design problem, the other an engineering bug that nobody hunts because it hides inside a metric named after respondent behavior.

Who can act. Buyer, almost entirely: the length, the grid, the device mix, and, when it hosts the survey, the instrumentation that splits the two populations.

Why it rarely happens. Questionnaire length is a negotiation between stakeholders who each add one more question, and none of them is present at minute 22 of the respondent's session. An honest length estimate costs nothing. It is simply nobody's KPI.

Quality checks: rules built for fraud, applied to humans

Three of the red bands are quality checks judging a person's answers: in the screener, mid-questionnaire, and after completion, when a review the person never sees removes the complete and sometimes reverses the reward. In the benchmark, research agencies remove 15.3% of records in-survey and 8.9% of finished completes afterwards; on B2B studies the after-the-fact figure is 59.6%. The tools were built for fraud, and fraud is real. But they are applied to humans, and humans are inattentive sometimes, contradict themselves sometimes, and misread a question that was badly written. A check that 40% of a sample fails is not proof that the sample is bad. It is a poorly designed check, or a poorly designed survey. An incoherence can reflect the complexity of real behavior as easily as a bad respondent.

The fix. Build quality checks that a genuine but imperfect human passes. Fire them early, so a problem costs a click rather than a paid session. Calibrate them on a known-good subsample before fieldwork, and read a high failure rate as a signal about the check before reading it as a signal about the people. Where a removal happens after completion, keep the reward unless there is actual fraud, and say why. Genuine panelists deserve a fair hearing, and so does the data: a rule that removes the attentive person who disagreed with the grid is removing signal.

Who can act. Buyer writes and calibrates the checks, decides whether a judgment call costs the participant their reward, and owes the supplier the exact reason behind every reconciliation. Supplier needs that feedback loop to do its part: a panelist who repeatedly terminates for quality across studies should be removed from the panel, and that is only possible if the reasons come back.

Why it rarely happens. A quality check's false positives cost the buyer nothing; the error lands entirely on the participant. Removing more reads as rigor in a deliverable, and no one measures precision because there is no gold standard to check against. Every incentive points toward judging late and harshly; the participant's side of that trade is not in anyone's ledger.

Pay, across the whole journey: the anchor has drifted

Pay is not a stage; it is the thing every stage spends. Ask 300 active panelists what a fair reward for a 10-minute survey would be, and the median answer is $1.25, about $7.50 an hour. Nearly half named a dollar or less. That is not what fair pay is; it is what the people who stayed have adapted to calling fair. 57% still feel underpaid, and they are still there, completing surveys. Underpayment doesn't show up as exit. It shows up inside the data, as attention nobody is paying for.

“Don't take the participants for granted. The better I'm treated/compensated, the better effort/higher quality my responses will be.”

Verbatim, Enlightn survey of 300 active online panelists, 2026

The fix. Price the time honestly on the studies that need engaged humans, keep rewards predictable, and pay something for terminates. Those are the improvements that reopen the pool to the people the current rate keeps out.

Who can act. Structure. The reward is what is left of the buyer's CPI after every layer's margin; no single party can raise it alone without pricing itself out.

Why it rarely happens. The CPI is set upstream by buyer budgets. A supplier that raises rewards alone prices itself out of any marketplace sorted on price. And because no buyer can verify quality at the point of purchase, quality never earns the premium that could fund the pay. So the reward settles at the lowest level the pool tolerates, which quietly selects the pool.

What shrinking the bands buys: an ecosystem the good actors stay in

Every band that is not teal makes the experience a little more adversarial, and the people it drives out are not the ones we would choose to lose. A genuine panelist who is screened out three times in a row, promised ten minutes and given twenty, or removed after finishing, draws the obvious conclusion and leaves. The bad actors do not leave. They adapt: they learn which screener answers get them through, they rush the questionnaire to maximize what they earn per hour, they run several accounts. The experience we collectively offer today is arguably hardest on the good actors and easiest on the people it was meant to keep out. In our panelist survey, fair pay and "you'll likely qualify" together account for 71% of the promises panelists most want a panel to keep; those are the two bands a person feels most directly.

Why this one pays for itself. Shrink the bands and the people who stay in the ecosystem are the ones who came for the right reasons. They do not have to be re-recruited, re-verified or re-screened, every study deepens what is known about them, and the data at the end needs fewer filters because less garbage went in. The reason it stays rare is structural: the benefit accrues to whoever keeps the participant relationship, and in a layered chain most of the parties spending a respondent's goodwill do not own it.