Frequently Asked Questions About the Delphi Method

For hands-on training in designing, running, analyzing, and appraising Delphi studies, join Sean Grant for Consensus Development Using the Delphi Method, a 3-day livestream seminar, November 11–13, 2026.

The Delphi method is an iterative, anonymous, structured, group-based communication process and elicitation technique designed to help policymakers, researchers, and practitioners make decisions under conditions of uncertainty and incomplete information. It is based on the premise that asking a hand-picked group of anonymous experts the same questions several times and sharing the group’s responses will help objectively develop group consensus, which is used as a form of evidence. The Delphi method is now used extensively for expert elicitation and stakeholder engagement: thousands of articles using the Delphi method are published each year across medicine, social science, science and technology studies, and other disciplines.

In teaching Delphi seminars and consulting on Delphi studies, I am often asked similar questions about the method. This post provides answers to some of the most common ones.

LEARN MORE IN A SEMINAR

What makes a study a “Delphi” study?

Four key characteristics define the Delphi method: iteration across at least two rounds, statistical summary of group responses, controlled feedback between rounds, and anonymity among panelists.

If you’re wondering whether a study you are designing or reading is actually using the Delphi method, ask yourself these four questions. A Delphi study answers “yes” to all four:

• Did panelists get at least one chance to revise their ratings after seeing the group’s results? If yes, that is iteration.
• Was the panel’s consensus decision determined by statistics (rather than “discussion until agreement emerged”)? If yes, that is statistical summary.
• Did panelists receive a summary of the group’s responses between rounds? If yes, that is controlled feedback.
• Were panelists’ identities and individual responses hidden from one another? If so, that is anonymity.

How many rounds do I need?

In short: usually just two or three. Iteration is one of the defining features of the method. Although a single-round survey is a valid study design, it is not a Delphi study unless there are at least two rounds.

Published Delphi studies range anywhere from two to seven rounds, but two or three rounds are usually enough to reach stable results. Beyond that, each additional round risks losing panelists as well as forced consensus (where panelists who still disagree drift toward the majority because they’re tired of the process). Alternatively, you could stop when responses are stable (for example, when each item’s median and interquartile range change little, if at all, from one round to the next). Either way, pre-specify the stopping rule (e.g., number of rounds, stability measure) in your protocol.

How many panelists do I need?

Delphi study sample size is typically not planned for statistical power. The main consideration is composition: the panel should cover the range of expertise and perspectives relevant to your question. That said, sample size is still important for how you measure consensus, as individual panelists have greater influence on consensus decisions in small panels. Attrition is also common, so recruit more panelists than you need to finish with, and have a proactive plan to retain panelists across rounds. Some best practices in recruitment and retention include explaining up front that the study is iterative and how long each round will take, keeping questionnaires short and clear, sending personalized reminders, and offering incentives for completing each round.

What are consensus and stability? How do I measure them?

Consensus describes agreement among panelist ratings within a round, while stability describes changes in ratings between rounds. A panel’s ratings can stop changing while remaining split, which is stability without consensus. There are numerous ways researchers have measured consensus and stability in Delphi studies. For example, consensus typically involves some measure of central tendency (e.g., median), dispersion (e.g., interquartile range), the percentage of panelists rating within a given range (e.g., 75% rating 4 or 5 on a 5-point scale), or some combination (e.g., median and interquartile range). Choose the measure and threshold according to how stringent you want consensus to be. In other words, the more demanding the definition, the fewer items will reach it. A good default for stability is whether the consensus decision changed between rounds.

My colleagues and I showed how much this choice matters. We pooled data from eight online Delphi panels and applied 25 analysis procedures drawn from the health research literature. Depending on the procedure, the percentage of items reaching consensus ranged from 0% to 84%. This underscores the need to pre-specify your consensus definition before collecting data, including the rating criteria, the type of consensus measure, the specific threshold, and which data and participants count toward the calculation.

What do I do with items that don’t reach consensus?

Report them! Items that fail to reach consensus are a finding, just as a null result is a finding. Dropping them from the results doesn’t fully capture what the panel thought. Report each item with its final results (e.g., median, interquartile range, and distribution), and read the narrative comments to understand why panelists disagreed. Check whether disagreement runs along stakeholder lines, such as clinicians versus patients. Finally, check whether ratings were still converging between rounds. An item with ratings that stopped changing reflects genuine disagreement. An item with ratings that were still moving might have reached consensus with another round.

What should panelists see between rounds?

At minimum, panelists should see the group’s response summarized statistically (e.g., the median, interquartile range, and full rating distribution), and anonymized comments explaining why panelists rated the item as they did. Personalize the feedback for each panelist so that they also see their own previous response (rating and comment) for each item.

Do I need specialist software?

Not necessarily. Generic survey tools can be used to run a Delphi study, but they will not automatically show each panelist their own previous response beside the group summary, so you will have to assemble feedback reports by hand for every panelist in every round. Purpose-built platforms automate this and can preserve anonymity in any discussion feature. For a small panel and two rounds, manual feedback is manageable. Beyond that, I would budget for a platform if you can. Look for a platform that shows each panelist their own previous responses beside the group summary, keeps comments anonymous, and exports item-level data for analysis. While a platform facilitates running the panel, you will still need to make key decisions about study design, such as writing the items, choosing the consensus definition, and deciding when to stop.

My main advice: think through and specify your plan in advance

Whatever design features you choose—make a plan, write it down, and share it before you collect data. Decisions like the number of rounds, definitions for consensus, and panel composition should be written in a study protocol and publicly registered before the study begins.

Consensus Development Using the Delphi Method (November 11–13, 2026) works through each of these decisions, and you’ll leave with a protocol outline for your own study.

I hope to see you there!