A medical imaging room with a glowing anatomical scan and diagnostic displays
Medical imaging AI depends on how scans, patients, and acquisition workflows are represented in a dataset.

Medical imaging AI has a credibility problem, and it is not a shortage of impressive numbers. Papers routinely report area under the curve values above 0.95 for detecting disease from X-rays, CT scans, MRI, and microscopy slides. Some of those results hold up in the clinic. Many do not, and a recurring reason is data leakage: information that should have stayed hidden from the model during development quietly reaches it, so performance looks far better than the model deserves.

Leakage is a broad term, and it helps to separate two meanings. The first is methodological leakage, where information from the test data influences training, or where the model learns a shortcut that will not exist in the real world. The second is privacy leakage, where a trained model or the data environment around it exposes the patient information it was built from. Both matter for medical AI, and both can undermine trust in a system that is supposed to protect patients.

How training and test sets become entangled

Six brain MRI examples grouped into training and evaluation above, and testing below
Example MRI images grouped as training and evaluation versus testing. A visual split does not establish patient-level independence; the dataset must be checked for overlap.

The most common form of methodological leakage is simple to describe. A model is evaluated on images it has effectively already seen. In medical imaging, this often happens because the unit of analysis is the patient, but the data are organised and split at the level of individual images or slices. If one patient contributes several images, some of those images can land in the training set while others land in the test set. The model then learns that patient's anatomy, tissue appearance, and imaging conditions, and the test score rises for reasons that have nothing to do with generalising to new people.

A 2023 study on longitudinal brain MRI shows how this works in practice. Rumala found that improper data splitting during cross-validation can make 3D convolutional networks look more accurate than they are, particularly when the same subject appears repeatedly over time. The remedy the authors emphasise is subject-wise splitting, followed by evaluation on a held-out cohort of entirely different people. The error appears across imaging modalities, which is why it is easy to miss. The code runs, the metrics look reasonable, and nothing crashes.

Our own experience at SarataniAI, a cervical cancer screening platform, shows the same risk from the data collection side. We initially organised our collection around images, treating each cervical image as an independent record. Once we looked more closely, it became clear that the unit of analysis was the woman, not the image, and that a single woman could contribute more than one image. Had we split the data by image, images from the same woman could have appeared on both sides of the split. The model would then have been partly tested on women it had already learned from, and the performance we measured would have overstated how the platform would behave with new patients. We caught the problem before any model was trained on the flawed split, but the correction was costly. We had to repeat the entire data collection, restructuring it around patient records, linking every image to a unique patient identifier, and then splitting by patient from the start. The effort was significant, but far less costly than discovering the error after a model had been reported or deployed.

The lesson is that the unit of splitting must match the unit of inference. In screening, the clinical decision concerns a woman, not a single image. Images are a means of observing her, and they should never be the basis for deciding which data go into training and which go into testing.

Preprocessing creates a second route. If normalisation, resampling, or feature extraction is fitted on the full dataset before the split, statistics from the test set shape the transformation applied to the training set. This often seems harmless, which is exactly why it persists. Data augmentation carries a similar risk. A study on oral cancer histopathology images found that the apparent gains from training under leaky protocols were methodological artefacts. Once augmented copies of a slide appear in both training and testing, the model is effectively being tested on near-duplicates of its training examples.

Feature selection can leak too. A mammography study simulated a case where features were selected based on performance on the validation set, and the same validation set was then used to judge the final model. The inflated results showed how a validation set can silently become a training signal. The lesson generalises: any step that uses outcome information, including feature choice, threshold selection, and model comparison, must be made using only the training data for that fold.

The practical safeguards follow from these examples. Assign a unique patient identifier before any analysis begins. Split by patient, not by image, at the very start. Fit all preprocessing and augmentation on the training partition only. Lock the final test set and use it once. Check for near-duplicate images across partitions, since a shared patient identifier is not the only way overlap can occur. A dataset that cannot tell you which images belong to which patients cannot support a credible claim about performance, however large it is.

Shortcut learning: the model learns the scanner, not the disease

Some of the most consequential leakage is not a bookkeeping error at all. It is shortcut learning, where a model finds an easy correlate of the label in the data and uses that instead of the clinical signal. In chest radiography, the shortcuts can be remarkably mundane: text markers burned into the image, the patient's position, the scanner model, or the hospital where the image was taken.

A 2021 study in Nature Machine Intelligence from the University of Washington examined several COVID-19 detection models built from chest X-rays. The authors found that the models relied on dataset-specific characteristics such as text markers and patient positioning rather than on the pathology a radiologist would look for. A model that learns "this scanner means COVID" will fail the moment it meets a different scanner. The authors also noted that a shortcut can seem plausible. A model might infer that an elderly patient is more likely to be positive, which is true at the population level but is not evidence a clinician would accept as a diagnosis.

Earlier work by Oakden-Rayner and colleagues described a similar pattern in pneumothorax detection, where the presence of a chest drain, a device placed after the condition has been treated, was enough to inflate performance on the images that contained one. The disease was present in both kinds of image, but the model did better when the treatment marker was visible.

Shortcut learning and leakage are closely related. In both cases the model's test performance reflects something other than the clinical question. Subject-wise splitting addresses patient overlap, but it does not catch a shortcut shared by all patients, such as a scanner used only for diseased cases at one hospital. Catching those requires external validation on data from other sites, stratified analysis by acquisition characteristics, and explicit checks for confounders.

Dataset construction and the wider literature

Leakage also enters at the level of the dataset itself. In a 2021 systematic review published in Nature Machine Intelligence, Roberts and colleagues assessed machine learning models for COVID-19 built on chest radiographs and CT scans. They concluded that none of the 62 studies they examined was of potential clinical use because of methodological flaws and biases. Among the problems they documented were training sets that mixed sources in ways that let a model distinguish sources rather than disease, and what they called Frankenstein datasets, assembled from several collections without careful reconciliation. Their recommendations covered data provenance, evaluation, reproducibility, and documentation, and they remain a useful checklist.

An earlier appraisal by Wynants and colleagues in the BMJ reached a similar conclusion about prediction models for COVID-19. Their initial review found that the studies they assessed were at high risk of bias, and that reported performance was probably optimistic. Their subsequent living review expanded the scope substantially and continued to find most models at high risk of bias.

Looking beyond medicine, Kapoor and Narayanan argued in a 2023 paper in Patterns that data leakage is a widespread cause of reproducibility failures across machine learning based science. They proposed a taxonomy of leakage types and "model info sheets," structured reports that ask authors to justify each step that could leak information. Their civil war prediction case study is a sobering example: papers claiming that complex models beat simple logistic regression did not reproduce once leakage was removed, and the simple model performed about as well. The argument transfers directly to medical imaging, where the simple baseline is often a clinician's judgement or a well-calibrated logistic model.

Privacy leakage: when the model remembers the patient

The second meaning of leakage concerns the patient data itself. Neural networks can encode information from their training sets, and in some circumstances that information can be recovered. A 2025 study introduced a data exfiltration attack for medical imaging in which someone with access to a data lake could compress images and extract them outside the protected environment, reconstructing CT and MR scans with high fidelity from the export. The authors' point was not that every model leaks patient images, but that the workflow around a model, including what is allowed to leave a secure environment, needs the same scrutiny as the model's accuracy.

Privacy and methodological leakage are linked. A model that memorises its training patients is also a model that may be evaluated poorly on new patients, and the governance question of what can leave a data environment is inseparable from the scientific question of whether a result will replicate.

Recognising when leakage is likely

Leakage is usually invisible in the final results table. A paper reports its method, metrics, and test set, and nothing on the page looks wrong. The clues tend to sit elsewhere: in the gap between internal and external performance, in how the numbers compare to simple baselines, and in details of the data pipeline that papers often omit. These clues do not prove that leakage happened, but they indicate when to be sceptical and when to ask for more evidence.

The first clue is performance that is implausibly high for the task. A model that detects a subtle condition from a single radiograph with near-perfect accuracy deserves scrutiny, especially if clinicians perform much worse on the same images. The second clue is a simple baseline that performs almost as well. If a logistic regression on a few clinical variables, or a model using only the scanner or acquisition site, scores close to the deep network, the deep network may be exploiting something other than the disease.

The third clue is a sharp fall in performance on new data. If internal validation looks excellent but external validation from another hospital, scanner, or period is much weaker, the internal test set probably shares something with the training data that the external set does not. This is one of the most reliable signs of shortcut learning, because a model that has learned the disease should not depend heavily on which hospital produced the image.

The fourth clue lies in the unit of splitting. Medical imaging datasets often contain several images per patient, several slices per scan, or repeated scans over time. If a paper does not say whether the split was made by patient, scan, or image, that omission is itself a warning. The check is straightforward: count the unique patient identifiers in each partition and confirm that none appears in more than one.

The fifth clue concerns the order of operations. Normalisation, resampling, feature selection, and augmentation should all happen after the split, and inside the cross-validation loop where one is used. If a methods section describes preprocessing on "the dataset" without saying when the split occurred, the risk is real.

The sixth clue is confounding between the label and the source. If diseased cases come mostly from one hospital and healthy controls from another, or were collected in different years with different equipment, the model has an easy way to separate the classes. Text overlays, image borders, device markers, and patient positioning are common examples. Saliency maps that highlight image corners or regions outside the anatomy are a visual warning.

Testing for leakage directly

An MRI data pipeline in which duplicate scan tiles appear on both sides of a train and test boundary, illustrating patient overlap leakage
If scans from one patient cross partition boundaries, a model can appear to generalize while recognizing patient- or scanner-specific cues.

Suspicion becomes evidence through targeted tests. The most useful is a metadata-only baseline. Train a model using only information that should be irrelevant to the diagnosis, such as scanner model, acquisition site, image dimensions, patient age, or the presence of text markers. If that model performs well above chance, the labels are predictable from non-clinical features, and the imaging model may be relying on the same signal.

A second test is to mask or remove the region of interest and see whether performance survives. A model that keeps its accuracy on images where the pathology has been hidden is not learning from the pathology. A third test is to repeat the evaluation under a stricter protocol: re-split the data by patient, move all preprocessing inside the training folds, and compare the results. A large drop indicates that the original protocol was leaking.

These tests do not require special tools. A researcher who can run the original code can usually carry them out in an afternoon, and a reader with only the paper can still look for the signals described above. Where code and data are unavailable, a reported result should be treated as provisional until an independent group reproduces it on a different cohort.

A single warning sign is not proof of leakage. Some tasks are genuinely easy, some models genuinely outperform baselines, and some datasets are genuinely well separated. The question is whether the evidence fits the claim. A result that survives external validation, a metadata-only baseline performing at chance, and a pipeline that splits by patient before any processing is far more credible than one missing those elements. When several signals appear together, the responsible conclusion is that the result cannot yet be trusted, not that the authors acted in bad faith. Most leakage is accidental, and the way to find it is to make the protocol explicit enough that someone else can test it.

Closing thought

Data leakage rarely results from carelessness or bad faith. It usually arises from ordinary steps taken in a hurry, from conventions imported from another field, or from datasets assembled without anyone noticing their structure. That is why it persists, and why the remedy is cultural as much as technical. A high score on a test set is meaningful only if the test set was genuinely unseen, the patients were genuinely new, and the signal was genuinely clinical. Before celebrating a result in medical imaging, ask how the data were split, what the model could have learned that it should not have, and who has checked.