ReReRe

An ensemble method for detecting careless respondents in questionnaire data, built around a permutation-based individual-coherence index (rr). Drop in a dataset; obtain a careless probability and flagged rows.

Runs entirely in your browser · no upload · no account

Your data never leaves this device — how we guarantee it →

1 · Load your data

Drop a CSV or Excel file here — or click to browse
Rows = respondents, columns = questionnaire items (Likert-type). Accepts .csv / .tsv (comma, semicolon or tab) and .xlsx / .xls (you'll be asked which sheet to use). ID columns are detected and set aside.
Advanced settings
Computing…

2 · Results

respondents
items used
rr coupled pairs
flagged careless
Distribution of the ReReRe ensemble careless probability. High P = careless. Red line = cutoff for the expected-rate slider above; bars at or above it are flagged. Move the slider to re-flag live.
Columns added: careless_prob, flagged, rr, longstring, person_total, irv, d2, n_missing

How it works

ReReRe is an ensemble. It combines three complementary careless-detection signals into a single per-respondent probability, calibrated on real data with a known careless/attentive ground truth. The distinctive ingredient — the one we built and the reason the method exists — is the rr index; the other two are established detectors that catch the failure modes rr is deliberately blind to. Together they cover careless responding far better than any one signal alone.

The rr index — the core signal

Respondents who answer attentively are internally consistent: on items that tap the same construct they give compatible answers, so those items covary within the person just as they do across the sample. Careless respondents break that internal covariance. The rr index asks: did this person answer the questionnaire's strongly-related item pairs more coherently than they would by chance?

  1. Find the coupled pairs. Correlate every pair of items across your sample; keep the top fraction with the largest |r| (default 3%) — the pairs that genuinely move together, typically items from the same scale.
  2. Score each person's coherence. For one respondent, measure how tightly their answers track across those coupled pairs (an individual-level |correlation|). Attentive people track them closely; careless ones produce near-noise.
  3. Build a personal chance baseline. Recompute the same coherence many times on random item pairs — a permutation done separately for each respondent — absorbing their own response style and scale usage.
  4. Turn it into a z-score. rr = (coupled − mean random) / sd random. A low rr means their answers to items that should agree are no more coherent than random pairs — the signature of inconsistent carelessness.

Comparing each person to their own random baseline is what makes rr self-calibrating: no assumption of multivariate normality, no external cut-off table, no per-dataset tuning. That is the crucial difference from distance-based outlier detectors such as Mahalanobis distance, whose chi-square thresholds assume a normality that Likert data violate and which, applied by the book, flag almost no one.

The two partners in the ensemble

rr targets inconsistent carelessness (random or intermittent responding). It is deliberately blind to consistent carelessness — someone who answers “3” to everything is trivially coherent. Two established detectors fill that gap:

A logistic model fitted on our labelled validation study combines the three into one probability. On that data the ensemble reaches AUC 0.985 under leave-one-out validation — matching a far heavier random forest and clearly beating rr alone (AUC 0.90). The two partners are automatically down-weighted when your data's item-mean profile is too flat to support them (a case rr handles alone), so the ensemble never does worse than rr — the results panel tells you when this happens. Two further indicators, IRV (within-person response variability) and Mahalanobis D², are computed and shown for transparency but kept out of the score: their careless direction is not stable across questionnaires (IRV) or degrades when items outnumber respondents (D²).

From probabilities to flags

The ensemble ranks every respondent by careless probability. Turning that ranking into a yes/no flag needs one number from you: the expected careless rate. The tool flags that top share — set it from a pilot, an earlier wave, or domain knowledge (5–20% is typical). Ranking quality doesn't depend on this choice; only where the line falls does.

Handling real questionnaires

Reverse-keyed items are aligned automatically from the sign of the sample correlation, so you needn't recode anything. Items on different Likert ranges are rescaled to a common proportion, so a 1–7 scale can't drown out a 1–4 one. Constant (zero-variance) respondents are handled explicitly. Every feature is standardised within your dataset before the ensemble combines them, so the fixed weights transfer across questionnaires of different length and scale. A structure diagnostic, calibrated against pure sampling noise, warns you when the data lack the cross-item structure the method needs.

When it works best

The signal grows with the number of correlated item pairs, so ReReRe is strongest on multi-construct batteries. In our validation study the ensemble is reliable (AUC ≈ 0.88–0.95) from about 45 items and 50 respondents upward — and already usable (AUC ≈ 0.80) from ~20 items — which is a wider envelope than rr alone. The applicability guide above reports where your specific dataset falls.

Privacy: your data never leaves your device — by design

Questionnaire data is often sensitive (personality, mental health, workplace surveys). This tool is built so that uploading your data is not just avoided — it is impossible: there is no server to receive it.

The static files are served by Cloudflare Pages; like any web host, Cloudflare sees the ordinary page request (your IP requesting the site) — but never your data, which is opened only after the page is already on your machine. Your institution's policies for handling data on your own computer still apply.