Synthetic Focus Groups Can Test Ideas but Not Replace Customers

You have two service concepts, a limited research budget, and a meeting next week. Asking an AI system to play several customers can produce a lively discussion in minutes. That discussion may expose unclear wording or suggest an objection you had missed. It does not tell you how many people will buy, what they will pay, or whether the imagined customers resemble your actual market.

Synthetic focus groups are best treated as a preparation tool. Use them to improve the questions you take into research, not to manufacture confidence before a decision. The distinction lets a team move quickly without confusing a convincing simulation with demand.

Know what the responses actually represent

A synthetic persona is a set of instructions and context given to a model. Its answer depends on the model, the prompt, supplied material, and generation settings. Ten fictional respondents produced by one system are not ten independently recruited people.

A 2024 psychometric study found important limitations when GPT-3.5 and GPT-4 simulated individual psychological profiles in questionnaire tasks. Those older models and specific measures do not settle every possible use of today's tools. They do show why human-like language alone is not evidence of representativeness.

Do not label a generated preference as a customer finding. Use separate headings such as simulated objections, observed interview evidence, and unresolved assumptions. That small editorial discipline prevents a speculative slide from becoming an apparent research result as it travels through the company.

Use simulation where disagreement is productive

Ask a system to identify ambiguous promises, missing information, conflicting incentives, or reasons a proposed offer might fail. These outputs can be useful even when no persona is a valid substitute for a customer.

For example, have it challenge a repair subscription from the perspective of procurement, the person operating the equipment, and the person approving emergency spending. Those roles create different questions about response times, exclusions, documentation, and purchasing authority. The exercise helps organize a research guide.

Avoid giving the model demographic labels and assuming it has captured the lives, constraints, or preferences of those groups. Ground scenarios in legitimate information you already hold, remove unnecessary personal data, and make unsupported assumptions visible.

Design a study that can contradict the simulation

Consider an illustrative regional maintenance company evaluating a priority-response subscription. The decision is whether to run a limited pilot, not whether a fictional customer panel likes the wording.

The team begins with a research brief containing the proposed service, actual coverage limits, interview questions, and known uncertainties. An analyst uses AI to produce possible objections. One plausible concern is paying a subscription while still facing an uncertain arrival time. Another is whether covered labor excludes the component that usually fails.

Those are hypotheses. A researcher recruits relevant current and prospective buyers using a documented approach, asks open questions, and examines actual call and quote records with appropriate permissions. The customer relationship system stores the real interviews and consent records; a separate workspace holds the simulated discussion.

Suppose interviews reveal that access to service records matters more than the proposed priority promise. The team updates the offer and marks the earlier simulated priority preference unconfirmed. It does not rewrite the interviews to preserve the more exciting story.

Combine methods rather than inventing a sample

Use conventional analysis for actual quote acceptance, service demand, cancellations, and recorded reasons for lost work. Use well-designed interviews to understand context and language. Where a representative estimate matters, involve someone qualified to design the sampling and analysis.

AI can help compare research notes, propose follow-up questions, and identify statements that need checking. Keep the original evidence available and verify summaries against it. Do not feed confidential interview transcripts into an unapproved service merely because the research is exploratory.

The SBA's planning guidance places market analysis within business planning. A simulated discussion can help prepare that work; it does not remove the need to understand customers, competitors, and the practical constraints of the offer.

Keep the simulation prompt, model version, assumptions, and output label with the research preparation. When notes move into a presentation, preserve those labels. A useful review question is whether another colleague could accidentally count a generated objection as an interview response. If so, correct the presentation before the decision meeting rather than relying on a verbal caveat.

Use a practical evidence ladder

Before acting on a synthetic finding, classify it by what it can support.

• A generated objection can justify asking a better question.
• A repeated interview theme can justify a more focused investigation, subject to who was interviewed.
• Observed behavior can test whether stated interest becomes action.
• A bounded pilot can reveal delivery and support problems that neither a persona nor an interview exposed.

Keep a decision log that names the assumption, the evidence supporting it, what would disprove it, and the next affordable test. Do not calculate market percentages from generated respondents.

A useful pilot measures more than initial enthusiasm. Track whether customers understand the offer, accept its limitations, use the service, and require work the proposed price cannot support. Define a stop or redesign decision before launching.

Synthetic research earns its place when it makes the real investigation more thoughtful. It becomes dangerous when speed and fluency are mistaken for access to customers who were never actually consulted.