The Missing Dataset: Failed Experiments
A failed experiment is usually excluded for a good reason. It did not produce the intended measurement, the sample was compromised, the instrument drifted, or the result did not meet the quality criteria required for the final analysis. Removing such data is often necessary when the immediate objective is a defensible scientific conclusion.
However, exclusion from a final analysis does not mean absence of information.
A failed run may show which acquisition settings produce artefacts, which sample conditions make a protocol unstable, or where the distinction between a weak positive result and technical noise becomes unreliable. A negative experiment may constrain the set of plausible mechanisms even when it does not support the original hypothesis. An image rejected by a scientist may be exactly the example an AI system needs in order to recognise when it should not make a prediction.
Most scientific datasets preserve successful measurements far more carefully than failed ones. This creates a mismatch between the data used to train AI and the environment in which the resulting model is expected to operate. The model sees a curated version of the experiment, while deployment exposes it to the complete measurement process, including all the ways in which that process can go wrong.
The missing dataset in scientific AI is therefore not simply more data. It is a structured record of failure.
Failure defines the boundary of a method
Successful examples show that a method can work under a particular set of conditions. Failed examples help determine how far those conditions extend.
This distinction is important because a scientific method is rarely useful merely because it works once. It must also be possible to state when the method is expected to work, which deviations it tolerates, and which observations should be treated as invalid. These boundaries are often learned informally by the people running the experiment. An experienced scientist may know that a particular staining pattern indicates an overexposed sample, or that a sensor becomes unreliable under a specific environmental condition, without this knowledge ever appearing in the final dataset.
For machine learning, that missing boundary produces overconfidence. If every training image passed quality control, the model has no basis for recognising the images that should have failed it. It may assign a familiar class to an unfamiliar artefact simply because classification is the only behaviour it has been taught.
Research in materials chemistry provides a clear illustration. Raccuglia and colleagues used unsuccessful hydrothermal synthesis experiments recovered from archived laboratory notebooks to train a model for predicting reaction outcomes [1]. The failed reactions were useful because they constrained the regions of experimental space in which a product did not form. They supplied information that could not be recovered from successful syntheses alone.
The relevant point is not that every failed experiment should be celebrated or retained indefinitely. It is that success and failure answer different questions. Success identifies a viable route. Failure helps define the space around it.
Curated datasets inherit a success bias
Scientific communication is naturally organised around results that can be interpreted. Papers describe methods that produced evidence, figures show measurements that passed quality control, and databases tend to collect reactions, structures or phenotypes that were sufficiently well characterised to report. Negative or inconclusive results are less likely to be published and often less carefully documented.
This selection process is understandable. A reader needs to know that the reported conclusion follows from valid evidence. The problem arises when a curated scientific record is treated as if it were an unbiased sample of experimental reality.
An AI model trained on published or cleaned data learns both the underlying phenomenon and the process by which examples were selected. If technically difficult measurements are systematically removed, the model may infer that those conditions do not exist. If negative reactions are underrepresented, a reaction-prediction model may overestimate the probability of success. If only well-focused images receive biological labels, an imaging model may interpret loss of focus as a biological change rather than an acquisition failure.
Increasing the size of such a dataset does not necessarily solve the problem. A larger success-biased dataset provides a more detailed representation of success, but it still does not reveal the missing boundary. Data-centric AI has therefore placed growing emphasis on the provenance, maintenance and coverage of datasets rather than treating model architecture as the only determinant of performance [2]. Failed experiments belong within this data lifecycle because they explain how the dataset was produced and what was removed from it.
Imaging failures contain several different signals
In microscopy and high-content screening, the term "failed image" can refer to very different situations. A field of view may be out of focus, overexposed or contaminated. Cells may be too sparse, too confluent or damaged. A segmentation may fail even though the underlying image is scientifically valid. A treatment may produce no detectable phenotype. Two experts may disagree about whether an apparent structure is real.
These cases should not be collapsed into a single negative class. A technically invalid measurement says something about the acquisition process. A true negative says something about the biological system. An ambiguous annotation says something about the limits of the available evidence. An analysis failure says something about the model or downstream pipeline.
Each can support a different learning task. Out-of-focus images can train acquisition quality control. Batch-specific artefacts can be used to evaluate robustness. Annotation disagreements can identify regions in which the label definition requires refinement. True negative phenotypes can improve the representation of biological response. Model failures can direct active learning towards examples that clarify the decision boundary.
The benefit does not come from calling all rejected data useful. It comes from recording why the data were rejected.
A reliable model must be allowed not to answer
Most classification models are trained under a closed-world assumption: every input belongs to one of the classes represented during training. Real laboratories do not satisfy this assumption. New artefacts appear, instruments are serviced, protocols change, and samples move outside the ranges represented in the original dataset.
A model that has only encountered valid examples may continue producing plausible predictions in these conditions. The output can be particularly misleading because neural-network confidence is not, by itself, evidence that an input belongs to the training distribution. The model may be highly certain about the nearest familiar class even when none of the available classes is appropriate.
Out-of-distribution detection and uncertainty estimation attempt to identify this situation [3]. Their practical value depends on exposure to examples that define the intended operating boundary. Failed experiments, rejected images and unusual runs can contribute directly to that definition. They show the model which changes are routine, which are recoverable and which should cause the system to stop.
For laboratory AI, abstention is therefore a useful output. A contaminated sample should not receive a confident biological interpretation. A plate affected by strong edge artefacts should be flagged before phenotype scores are compared. A measurement outside the validated instrument range should trigger review rather than being silently forced through the pipeline.
The ability not to answer is learned from the edge of the dataset, and failed experiments often occupy that edge.
Closed-loop laboratories need a memory of failure
The value of failure becomes more immediate when AI participates in the experimental loop. In a conventional workflow, an invalid run may be discovered during analysis, after the samples and instrument time have already been consumed. In a closed-loop workflow, detecting the same failure can change the experiment while it is still running.
If the focus begins to drift, the system can reacquire or pause. If a field of view is uninformative, it can move elsewhere. If a new artefact appears, it can request human review. If a synthesis route repeatedly produces an unwanted phase, the experimental planner can reduce the priority of similar conditions.
Autonomous materials laboratories already demonstrate this use of negative evidence. In A-Lab, failed synthesis attempts contributed observations that were used to revise subsequent experimental choices rather than being discarded as irrelevant [4]. In automated analytical workflows, anomaly-detection systems have similarly been developed to identify compromised measurements and route them for human review [5].
This does not make failure desirable. It makes the cost of failure partially recoverable. A run that does not produce the intended result can still improve the system's understanding of where to measure next or when to stop.
Laboratory memory requires context
Retaining raw data is not the same as learning from it. A directory containing thousands of rejected images has limited value if nobody recorded which images were out of focus, which contained genuine negative biology and which were produced by a known instrument fault.
A useful laboratory memory therefore needs context. At minimum, it should connect the measurement to the experimental batch, instrument, acquisition settings, protocol version, operator or automation state, quality-control decision and reason for rejection. Where a model was involved, the record should also preserve the model version, prediction, uncertainty and any subsequent expert correction.
This information makes different failure modes separable. It allows a future model to learn that one pattern indicates detector saturation while another indicates the absence of a phenotype. It also enables retrospective questions that are difficult to answer from polished datasets: when did an artefact first appear, which protocol change preceded it, and did a model's error rate increase before the problem became visible to users?
The resulting dataset does not need to contain everything. Storage without interpretation merely moves the problem. What matters is preserving the failures that clarify a boundary, reveal a recurrent mechanism or document a condition the deployed system is likely to encounter again.
Learning from failure without learning the wrong lesson
Negative data can also mislead. A reaction may have failed because the chemistry was unfavourable, or because a temperature sensor was miscalibrated. A microscopy image may show no phenotype because the treatment was ineffective, or because the cells were dead before acquisition. If these causes are not distinguished, the model can learn an incorrect relationship between experimental conditions and outcome.
Failed experiments should therefore be curated with the same care as successful ones. The record must separate a valid negative result from a technically invalid experiment and distinguish expert uncertainty from confirmed absence. In some cases the appropriate label will remain unresolved. Preserving that uncertainty is more honest than assigning a definitive category simply to make the data easier to use.
At IFLAI, we regard this as part of building data-efficient scientific AI. Data efficiency does not mean extracting the maximum possible training volume from every laboratory file. It means identifying which measurements change the model's understanding and retaining enough context for that change to be scientifically meaningful.
A successful experiment establishes what can happen. A well-documented failure establishes where that conclusion stops. Models intended for real laboratories need both.
References
- [1] Raccuglia et al., "Machine-learning-assisted materials discovery using failed experiments," Nature (2016)
- [2] Zha et al., "Data-centric Artificial Intelligence: A Survey," arXiv (2023)
- [3] Carnegie Mellon Software Engineering Institute, "Out-of-Distribution Detection: Knowing When AI Doesn't Know" (2025)
- [4] Szymanski et al., "An autonomous laboratory for the accelerated synthesis of inorganic materials," Nature (2023)
- [5] Digital Discovery, "Machine learning anomaly detection of automated HPLC experiments in the cloud laboratory" (2025)
- [6] Journal of Organic Chemistry, "Negative Data in Data Sets for Machine Learning Training" (2023)
- [7] Science Advances, "Negative chemical data boosts language models in reaction outcome prediction" (2025)
- [8] ICML, "Position: Embracing Negative Results in Machine Learning" (2024)