Check survey-question provenance before summarizing results

The short answer

A spreadsheet of survey responses can look ready for analysis while hiding the information needed to interpret it. The analyst may not know the exact question shown to respondents, which choices were available, who was invited, or whether the form changed during collection. Without that lineage, a percentage can be calculated correctly and still answer the wrong question.

A research assistant should establish provenance before writing findings. The requester or research owner remains responsible for the study design and any conclusion drawn from it.

Start with the exact wording, instructions, answer choices, order, and branching logic used during collection. A planning document is not enough if the live form differed. Preserve a read-only export, version identifier, screenshots where authorized, or another stable record of the deployed questionnaire.

Wording changes matter. "Which support channel do you prefer?" asks for one preference. "Select every channel you would use" permits several answers. Results from those questions cannot be combined as though they share one denominator. Note whether an option such as "Other" allowed free text and whether blank answers were permitted.

Record display logic. If only dissatisfied customers saw a follow-up question, its responses describe that subgroup, not every respondent. Capture the rule that created the subgroup and the number of people who reached the question.

Document who could receive the survey, how invitations were sent, the collection dates, eligibility rules, duplicate controls, and any exclusions. Separate invited people, people who opened the form, eligible respondents, completed responses, and usable answers for each question.

Do not call a convenience sample representative of all customers or employees. The assistant can describe the observed group and flag missing population information. Claims about broader populations require an approved method and qualified review.

Suppose a customer survey was emailed only to people who closed a support case in September. Its satisfaction result does not describe customers who abandoned a case, used another channel, or had no recent contact. The summary should name the sampled event and period instead of shortening the result to "customers are satisfied."

Forms sometimes change while responses are arriving. A corrected typo may not affect interpretation. Adding a choice, changing a scale, or making a question required can divide the dataset into different instruments.

Build a version table with effective time, question changes, answer-option changes, branching changes, and response counts under each version. If timestamps are available, map responses to the version they encountered. If that cannot be done reliably, state the limitation and avoid a seamless combined result.

Never infer a version from the answer alone. A respondent might type a newly added category into an older free-text field. Keep source evidence separate from reconstruction.

For each reported measure, record the question, response unit, included answers, excluded answers, missing-data rule, and denominator. Multi-select questions may produce percentages that sum above 100 percent. Ranked questions need a rule for incomplete rankings. Scale questions need the full scale and direction.

An assistant should not silently drop blanks, "not applicable" responses, or invalid records to improve a result. Show how each category is treated. If two reasonable rules produce different conclusions, present the sensitivity to the research owner.

Imagine 200 people reached a question, 150 answered, and 90 selected "very useful." Ninety is 45 percent of those who reached the question and 60 percent of those who answered it. Both calculations are arithmetic facts, but they describe different things. The report must name the denominator.

Raw survey exports often need cleaning. Keep a transformation log covering duplicate removal, eligibility checks, recoding, combined categories, text coding, calculated fields, and excluded rows. Retain the original export under the organization's access and retention controls.

When coding open text, define the codebook and allow more than one code only if the method permits it. Record who coded the responses, how disagreements were handled, and whether a sample received a second review. Do not turn a handful of comments into a prevalence claim.

Protect respondent information. Remove direct identifiers from the analysis copy when they are not required. Restrict small-group reporting when combinations of attributes could reveal a person. Route privacy, employment, health, legal, or sensitive research questions to the appropriate owner.

A sound finding names the population, period, question, denominator, and observed result. It distinguishes the response from an explanation. If respondents chose speed as their top priority, the survey shows the choice under that instrument. It does not prove why they chose it or which operational change will improve satisfaction.

Link each chart or statement to a calculation record. Include the filter, field names, formula, exclusions, and source version. Another authorized reviewer should be able to reproduce the result without guessing which worksheet or cleaned file was used.

Use uncertainty plainly. Low response, unknown invitation coverage, question changes, missing answers, and self-selection are not footnotes to hide. Explain how each limitation narrows the conclusion. If the evidence cannot support the requested comparison, recommend a narrower statement or new collection rather than filling the gap with assumptions.

Survey findings are often copied into presentations months later. Store the field dates, instrument version, population, calculation, owner, and approved wording with the result. A later user can then decide whether the finding still fits the audience and decision.

Before reuse, check whether the question wording, population, product, service process, or reporting definition changed. Do not update only the chart date. The original finding remains historical evidence and should not be presented as a new measurement.

OutsourcedAssistants.com describes research assistance at /services/research-assistance. A role brief should identify approved datasets, privacy limits, analysis tools, calculation-review owner, citation practice, and conclusions reserved to qualified researchers. Use /contact-us once that lane is defined.

The U.S. Census Bureau publishes standards and guidance for survey work. The American Association for Public Opinion Research publishes disclosure standards for survey methodology. The NIST Privacy Framework addresses privacy-risk management. These sources support transparent methods and careful data handling. They do not validate a particular survey, establish representativeness, or authorize a business decision. The research owner must review the instrument, method, and claimed inference.

Build the work lane

• Write the recurring task and its finish rule
• Share an approved example and the source system
• Set response times, approval limits, and escalation rules
• Review a small sample before widening the role

Check the role before you expand it

• Can a new person complete the task from the examples?
• Who reviews the first week and records corrections?
• Which decisions and systems stay with your internal owner?
• What evidence shows the role is ready for another queue?

Common questions