Serendipity and Creativity: How Accidental Discovery Works
Serendipity gets treated as a synonym for luck, which misses the point of the word entirely. Horace Walpole coined it in a 1754 letter to a friend, describing it as the faculty shown by the three princes in the Persian fairy tale "The Three Princes of Serendip" — men who, Walpole wrote, "were always making discoveries, by accidents and sagacity, of things which they were not in quest of." The key word is sagacity. Serendipity was never defined as blind luck. It was defined as the skill of recognizing the value in an accident that a less prepared observer would have ignored.
That distinction matters because it turns serendipity from something that happens to you into something you can train. This post covers what the research actually says about how accidental discovery works, the historical cases that built the concept, and what you can do to increase your own surface area for it.
Louis Pasteur's Prepared Mind
The most quoted line in the serendipity literature comes from Louis Pasteur, in an 1854 lecture at the University of Lille: "In the fields of observation, chance favors only the prepared mind." Pasteur wasn't being poetic. He'd spent years building deep expertise in crystallography and fermentation before his accidental observations about weakened cholera cultures led him toward the principle of vaccination. The accident supplied the raw event. His preparation supplied the ability to see what the event meant.
"Prepared" has a specific cognitive definition here, not a vague one. It means you've built a dense enough knowledge structure in a domain that an anomaly registers as meaningful rather than as noise. A microbiologist and a layperson can look at the same contaminated petri dish. Only one of them has the background to recognize that something worth investigating just happened.
Alexander Fleming and the Anomaly That Didn't Get Discarded
Fleming's 1928 discovery of penicillin is the canonical serendipity story, and it's worth being precise about what actually happened. Fleming returned from a two-week vacation to find a stray mold contaminating one of his staphylococcus culture plates — a common, usually worthless lab accident. What made this case different is that the bacteria immediately surrounding the mold had been killed, while colonies further away were untouched.
A less prepared researcher discards a contaminated plate without a second look; contamination is the most routine failure mode in bacteriology. Fleming's specific expertise in antibacterial substances let him recognize the ring of dead bacteria as a signal rather than as spoiled work. He didn't induce this from repeated observation — he had one unusual data point and generated the hypothesis that the mold was secreting something antibacterial. That's the abductive leap described in more detail in abductive reasoning, and it's the exact cognitive move that separates a discarded plate from a Nobel Prize.
Robert Merton's Sociology of the Accident
Sociologist Robert K. Merton spent decades studying how the concept of serendipity actually functions in scientific practice, work later compiled with Elinor Barber into The Travels and Adventures of Serendipity, written in 1958 and finally published in 2004. Merton's contribution was showing that a "serendipity pattern" in research has a specific structure: an observation that is unanticipated, anomalous, and strategic — meaning it has implications that extend beyond the immediate problem the researcher was working on.
This is a stricter definition than the popular usage. A random lucky break isn't a serendipity pattern unless the observer's existing theoretical framework lets them recognize its broader significance. Merton was explicit that serendipity is a property of the interaction between the observer and the anomaly, not a property of the anomaly alone.
Pek van Andel's Taxonomy of Unsought Findings
The most rigorous academic treatment of the concept is Pek van Andel's 1994 paper "Anatomy of the Unsought Finding," published in the British Journal for the Philosophy of Science. Van Andel collected and classified more than a thousand documented cases of serendipitous discovery across science, technology, and the arts, and found that "accidental discovery" is not one phenomenon but several distinct ones.
Some discoveries happen while searching for something specific and finding something else entirely — Fleming was studying staphylococcus, not antibiotics. Others happen during unfocused exploration with no specific target, closer to open-ended play. Van Andel's taxonomy matters practically because it shows that the conditions that produce one type of serendipitous finding (narrow expertise plus an anomaly) differ from the conditions that produce another (broad, unstructured exploration). You can't optimize for serendipity in general — you have to decide which kind of discovery process you're trying to create room for.
More Cases Worth Knowing
William Perkin and synthetic dye. In 1856, an 18-year-old chemistry student named William Perkin was trying to synthesize quinine to treat malaria. His failed experiment left a dark sludge in his flask. Instead of discarding it, he noticed it dyed silk a vivid purple that didn't fade — the first synthetic aniline dye, mauveine. Perkin's chemistry training let him recognize a byproduct as a discovery instead of simply a failed synthesis. He was 18, not a senior researcher, which is itself evidence that "prepared" means domain knowledge, not seniority.
Percy Spencer and the microwave oven. Working with radar magnetrons at Raytheon in 1945, engineer Percy Spencer noticed a chocolate bar in his pocket had melted while he stood near an active magnetron. Most engineers in that lab had stood near the same equipment. Spencer's specific curiosity — testing the effect deliberately with popcorn kernels and then an egg — converted a strange personal observation into the mechanism behind the microwave oven.
Spencer Silver and the Post-it Note. Silver's 1968 attempt to create an extremely strong adhesive at 3M instead produced one that was unusually weak and endlessly reusable — a failure by every metric his project was measuring. The discovery sat unused for six years until colleague Art Fry, frustrated with bookmarks falling out of his choir hymnal, connected Silver's "failed" adhesive to a problem it happened to solve perfectly. This case is a reminder that the prepared mind doesn't have to belong to the person who made the original observation — Fry's contribution was pattern-matching a stored anomaly to an unrelated problem years later.
Sanda Erdelez and Information Encountering
Information scientist Sanda Erdelez ran a different kind of study in the 1990s: instead of examining historical cases, she interviewed people directly about how they relate to unexpected information in their daily environment — browsing, reading, searching. She found consistent individual differences and grouped people into three categories: non-encounterers, who rarely notice information outside what they're specifically looking for; occasional encounterers, who sometimes do; and super-encounterers, who consistently notice and act on unexpected, useful information as a matter of habit.
Erdelez's framework reframes serendipity as a trainable orientation rather than a personality trait you either have or don't. Super-encounterers weren't smarter in a general sense — they'd built a habit of treating tangential information as worth a second look instead of filtering it out as irrelevant to the current task.
How to Increase Your Own Surface Area for It
Build deep expertise somewhere specific. This is the least glamorous and most important input. Pasteur's mind was prepared because he had years of narrow, specific knowledge. Breadth without depth produces curiosity without the ability to recognize significance. You need at least one domain where you know enough to notice when something doesn't fit the pattern.
Then deliberately cross domains. Once you have depth somewhere, reading across unrelated fields and exposure to associative thinking increases the odds that an anomaly in one domain will connect to something you already understand in another. Perkin's malaria research collided with dye chemistry. Darwin's finch observations collided with Malthusian economics.
Keep a running log of anomalies. Most unexpected observations get mentally discarded within seconds because they don't fit the current task. Writing down things that seemed odd, wrong, or unexpectedly interesting — even without an obvious use — creates a personal archive you can pattern-match against later, the way Art Fry pattern-matched Silver's adhesive years after the fact.
Protect unstructured time. 3M's famous policy allowing employees to spend a portion of their time on self-directed projects is frequently credited as a structural precondition for the Post-it Note's eventual development. Fully optimized schedules leave no room for the kind of unfocused exploration that Van Andel's taxonomy identifies as one of the two main paths to a serendipitous finding. This connects directly to the incubation research covered in creative block — walking away from a stuck problem isn't just rest, it's making room for an unrelated observation to reach you.
Don't discard the anomaly. The single behavior that separates every case in this article from a forgotten failure is that someone paused on something that didn't fit expectations instead of moving past it. That pause is a decision, not a talent, and it's the one part of serendipity you can practice on command starting today.
Serendipity looks like luck from the outside because the triggering event usually is arbitrary — a mold spore, a melted chocolate bar, a failed synthesis. What isn't arbitrary is the recognition. That's a skill built from expertise, cross-domain exposure, and the habit of taking a second look at things that don't fit, and it's the same underlying capacity that analogical reasoning exercises are designed to strengthen.
Ready to train your creativity? Try science-backed exercises that measure and improve your creative thinking. Start a Free Exercise