The Moving
Frontier of Randomness: Why Chance May Be the Name We Give to Unmodelled Order
A scientific and philosophical
essay on probability, pattern, ignorance and dynamic modelability
Anthony Dernellis
JUN 2026
Introduction: the useful scandal of
chance
Randomness is among the most
useful, and among the most easily misunderstood, words in the scientific
vocabulary. It is useful because it gives disciplined language to uncertainty.
Without it, modern statistics, thermodynamics, quantum theory, risk analysis,
genetics, epidemiology, cryptography, artificial intelligence, insurance,
econometrics and experimental science would be almost unthinkable. It is
misunderstood because the word often sounds more final than it ought to sound.
To say that an event is random may mean several quite different things: that no
cause exists; that a cause exists but is inaccessible; that the event belongs
to a probability distribution; that no finite description can compress its
sequence; that the system is deterministic but practically unpredictable; or,
more modestly, that no model presently available to us has succeeded in
explaining it.
This ambiguity is not a defect to
be brushed aside. It is precisely where the philosophical interest of
randomness begins. For the scientist at the bench, randomness is often an
operational term: it tells us how to calculate, test, simulate or infer. For
the philosopher of science, it is also an epistemological alarm bell: it tells
us where knowledge stops, where measurement fails, where computational
complexity overwhelms prediction, or where nature itself may resist classical
explanation. For the mathematician, randomness can be formalised with great
rigour, yet such formalisation does not, by itself, settle what randomness
means in the world.
The thesis explored in this article
is deliberately bold, but not reckless: perhaps a considerable portion of what
human beings call randomness is not the absence of order, but the presence of
order that has not yet been recognised, modelled, observed over a sufficiently
long interval, or rendered computationally tractable. In plain English, we
often call a distribution random when we have not yet found its grammar.
Yesterday’s noise may become tomorrow’s equation.
This is not a claim that all
phenomena are secretly deterministic in the simple, clockwork sense of
nineteenth-century physics. Such a claim would be scientifically incautious,
especially in the light of quantum mechanics and the experimental constraints
placed on local hidden-variable theories. Nor is it a denial of probability
theory, which remains one of the most powerful achievements of modern thought.
Rather, the proposal is subtler: randomness should often be treated not as a
static verdict, but as a moving frontier. It is the name we give to what, at a
given historical moment, lies beyond our current models, instruments, datasets
and computational reach.
The distinction matters. If
randomness is treated as an ontological finality too quickly, inquiry may stop
prematurely. If, conversely, every instance of apparent randomness is treated
as a puzzle awaiting a deterministic key, science risks falling into metaphysical
overconfidence. The wiser position is neither credulous determinism nor lazy
surrender to chance. It is a disciplined agnosticism: a readiness to use
probabilistic models with full technical seriousness, while also admitting that
scientific history has repeatedly transformed apparent disorder into
intelligible structure.
Scientific definitions and the many
faces of randomness
Contemporary science does not
define randomness in only one way. In probability theory, following the
axiomatic work of Andrey Kolmogorov, randomness is handled through probability
spaces: a sample space, events and a probability measure satisfying precise
mathematical rules. This approach is extraordinarily successful because it
avoids unnecessary metaphysics. It does not ask whether a coin toss is truly
indeterminate; it asks what formal rules probabilities must obey and how those
rules can be used.
In the frequency interpretation
associated with Richard von Mises and others, probability is connected with
long-run relative frequency. An event has probability one half, for example, if
in a sufficiently extended sequence of trials its frequency tends to one half.
Von Mises also introduced the idea of a collective: a sequence whose limiting
frequency remains stable under admissible selection rules. The intuition is
recognisable from gambling: a genuinely random sequence should not allow a
system that reliably extracts profit by selecting a favourable subsequence.
Bayesian probability, associated
historically with Thomas Bayes and later with thinkers such as Pierre-Simon
Laplace, Frank Ramsey, Bruno de Finetti, Leonard Savage and many others,
interprets probability as rational degree of belief under uncertainty. On this
view, randomness is not necessarily a property of the world alone; it is also
related to the information available to an observer. De Finetti’s famous
provocation, that probability does not exist, was not a denial of practical
probabilistic reasoning, but a rejection of probability as a freestanding
physical substance. Probability, for him, belonged to coherent expectation.
In information theory, Claude
Shannon made uncertainty measurable through entropy. A message source has high
entropy when its outputs are difficult to predict and contain much information.
This is not identical with metaphysical chance. It is a measure of
informational uncertainty. A perfectly patterned message has low entropy; a
message with many equally plausible alternatives has high entropy. Shannon’s
framework helped detach randomness from loose intuition and connect it with
communication, coding and signal processing.
Algorithmic information theory,
developed by Ray Solomonoff, Andrey Kolmogorov, Gregory Chaitin and Per
Martin-Löf, sharpened the idea further. A sequence is algorithmically random if
there is no shorter effective description of it than the sequence itself. In
that sense, randomness is incompressibility. The digits of pi may look
irregular, but they are not algorithmically random in this strict sense,
because a compact algorithm generates them. By contrast, a truly incompressible
sequence cannot be reduced to a rule simpler than itself. This concept is
severe, elegant and philosophically unsettling: it shows that apparent
irregularity and true formal randomness are not the same thing.
In dynamical systems theory,
randomness is often practical rather than fundamental. A deterministic system
can behave unpredictably if it is nonlinear and sensitive to initial
conditions. Edward Lorenz’s work on weather and chaos famously showed that very
small differences in initial states can grow into large differences in
outcomes. The equations may be deterministic; the forecast may nevertheless
fail beyond a certain horizon. Here, randomness is not absence of law but
excess sensitivity relative to measurement precision.
In quantum mechanics, the issue
becomes sharper. Standard quantum theory predicts probabilities for measurement
outcomes, and many interpretations treat certain outcomes as genuinely
indeterminate. The Born rule gives probabilities with astonishing empirical
success. At the same time, debates over interpretation remain open:
Copenhagen-type views, many-worlds interpretations, pilot-wave theories and
objective-collapse models do not agree on what quantum probabilities ultimately
mean. What is not responsible is to pretend that quantum randomness has simply
been overthrown. The experimental violation of Bell inequalities places deep
restrictions on local hidden-variable explanations. Whatever lies beneath
quantum probabilities, it cannot be an ordinary local classical mechanism in
disguise.
These diverse approaches show why a
single blunt sentence, such as ‘randomness means lack of pattern’, is
inadequate. Randomness may be formal, statistical, epistemic, informational,
dynamical or physical. It may describe the world, our knowledge of the world,
or the limits of our methods. Any serious discussion must keep these meanings
distinct.
A historical review of the
principal scientific and mathematical approaches
The intellectual history of
randomness begins long before modern probability. Aristotle distinguished
events that occur always or for the most part from events that occur by chance.
For him, chance did not necessarily mean absence of cause; it often meant the
accidental intersection of causal lines. A man goes to the market for one
reason and unexpectedly meets a debtor there: the meeting is by chance, though
each causal strand is intelligible.
The mathematics of probability
arose from games of chance. Blaise Pascal and Pierre de Fermat, in their
seventeenth-century correspondence about gambling problems, helped create the
combinatorial foundations of probability. Christiaan Huygens soon produced one
of the first systematic treatments. Here randomness entered mathematics through
dice, cards and wagers: not as cosmic mystery, but as calculable uncertainty.
Jacob Bernoulli’s Ars Conjectandi
introduced the law of large numbers, showing that relative frequencies
stabilise under repeated trials. This was a major conceptual turn. Individual
outcomes may be unpredictable, yet aggregate behaviour may be regular. The
paradox remains central today: randomness at the level of the single event may
coexist with order at the level of distribution.
Laplace pushed probability into a
grander philosophical frame. His imagined intelligence, later called Laplace’s
demon, knew all forces and positions and could compute the future and past. In
such a universe, chance would be merely a confession of ignorance. Laplace’s
determinism is no longer an adequate description of modern physics, but it
remains a powerful symbol of epistemic randomness: the idea that chance may
reflect the observer’s limitation rather than nature’s indecision.
The nineteenth century brought
probability into physics. James Clerk Maxwell and Ludwig Boltzmann used
statistical reasoning to explain gases. Individual molecular motions were too
numerous to track, but their collective behaviour could be described with
statistical laws. This was not a defeat for science; it was a triumph. A new
kind of explanation emerged: not prediction of every particle, but
understanding of macroscopic regularities from microscopic multiplicity.
Henri Poincaré saw, perhaps more
clearly than most of his contemporaries, that deterministic systems could
exhibit behaviour that looks random because small causes may have large
effects. His insights foreshadowed chaos theory. A century later, Lorenz’s numerical
weather models made the point vivid. Determinism did not guarantee
predictability. This single lesson has immense importance: the absence of
prediction is not proof of absence of law.
In the twentieth century,
Kolmogorov made probability mathematically rigorous through measure theory.
This was a liberation. Probability no longer needed to settle every
philosophical quarrel before being used. It became a formal calculus applicable
to many interpretations. The price of such elegance, however, was that the
formalism alone did not answer whether the probabilities represented
frequencies, beliefs, propensities or objective chances.
Von Mises and later frequentists
attempted to anchor probability in repeated observations. Ronald Fisher, Jerzy
Neyman and Egon Pearson developed statistical inference frameworks that made
randomness central to experimental design, estimation and hypothesis testing.
Random sampling and randomisation became tools not because they created
metaphysical chaos, but because they controlled bias and enabled valid
inference.
Shannon’s information theory
transformed randomness into a measure of uncertainty in communication. Chaitin
and others then connected randomness with computation and incompressibility.
These developments are crucial for the present argument. They show that whether
something is random may depend on the descriptive language, algorithmic
resources and modelling framework available. A sequence may appear random to
one observer and structured to another who knows the generating rule.
Finally, quantum mechanics forced
the deepest confrontation. Max Born’s probabilistic interpretation,
Heisenberg’s uncertainty principle, Bohr’s complementarity and later Bell’s
theorem challenged the classical assumption that every apparent uncertainty must
be removable by better information of a familiar kind. Einstein resisted this
conclusion, famously objecting to a universe in which God plays dice. Yet the
subsequent history of quantum foundations has not restored simple classical
determinism. It has, rather, made the relation between probability,
measurement, locality and reality more subtle than either naïve determinists or
naïve indeterminists might wish.
The central proposal: randomness as
unrecognised model
The central proposal of this
article may now be stated more precisely. Human beings often call a phenomenon
random when one or more of the following conditions hold: the observational
period is too short; the relevant variables are unknown; the measurement
resolution is inadequate; the system is too complex for available computation;
the correct mathematical language has not yet been invented; or the pattern
exists at a scale not yet examined. In such cases, randomness is not
necessarily a property of the phenomenon itself. It is a property of the
relationship between the phenomenon and our present capacity to describe it.
This proposal is not an attack on
probability. On the contrary, probability is precisely what allows science to
proceed when full modelling is not available. But probability may be both a
final theory in some domains and a provisional instrument in others. The
mistake is to assume that the same word, randomness, carries the same
philosophical force in every case.
Consider a sequence of observations
that shows no repeating pattern over ten trials. It would be foolish to infer a
law. Over a thousand trials, a frequency distribution may appear. Over a
million trials, a subtle periodicity may be detectable. Over a century of
observations, a cycle may emerge that was invisible within a decade. Time is
not a passive container of data; it is often the very instrument that reveals
structure.
This matters in fields where the
relevant cycles exceed human patience. Climate oscillations, geological
processes, evolutionary dynamics, demographic transitions and astronomical
variations may all involve timescales that dwarf ordinary observation. A phenomenon
can seem patternless simply because the observational window is too narrow. To
borrow a homely British phrase, one should not declare the orchestra tuneless
after hearing two bars through a closed door.
The same applies to dimensionality.
A distribution may look random in one projection and structured in a
higher-dimensional representation. Modern machine learning repeatedly
demonstrates this point. Data that appear scattered in two variables may become
classifiable when many features are considered. This does not mean that machine
learning discovers metaphysical truth; it means that patterns can be invisible
until the right representation is chosen.
There is also the matter of hidden
variables, though the term must be used carefully. Outside quantum foundations,
hidden variables simply mean unobserved causal factors. A medical outcome may
appear random until genetics, environment, treatment adherence and
comorbidities are included. A traffic jam may seem accidental until road
topology, behavioural response and signal timing are modelled. A biological
mutation may be random with respect to an organism’s adaptive needs, yet its
molecular mechanisms are not magical. Randomness here means independence from a
particular explanatory axis, not absence of physical process.
Even in games of chance, apparent
randomness is conditional. A fair die toss is modelled probabilistically
because its exact outcome depends on initial position, angular velocity,
surface interaction, air resistance and collision dynamics. In principle, classical
mechanics applies. In practice, measurement and computation are insufficient.
The die is not a metaphysical oracle; it is a small chaotic machine. The phrase
‘random toss’ is a practical shorthand for extreme sensitivity and ignorance of
initial conditions.
Examples where apparent randomness
became modelled structure
Brownian motion is perhaps the most
instructive example. To the early observer, tiny particles suspended in liquid
moved irregularly, almost capriciously. Einstein’s 1905 analysis connected this
motion with molecular-kinetic theory, and Jean Perrin’s experimental work
helped confirm the reality of atoms and molecules. The motion remained
stochastic in its mathematical description, but its irregularity was no longer
mysterious. It became intelligible through statistical mechanics. Here
randomness was not abolished; it was domesticated.
Planetary motion offers a more
classical case. Before adequate celestial mechanics, planetary wanderings
across the sky looked irregular against the fixed stars. The very word planet
derives from wandering. Mathematical astronomy, culminating in Kepler’s laws
and Newton’s gravitation, transformed wandering into orbit. The phenomenon did
not change; the model did. What seemed irregular became lawful once the right
geometrical and dynamical framework was available.
Weather provides a third example,
but with a twist. Weather has not become perfectly predictable. Rather,
meteorology has clarified why prediction has limits. The atmosphere is governed
by physical equations, but nonlinear dynamics and sensitivity to initial
conditions impose practical horizons. Here model discovery did not eliminate
uncertainty; it explained the structure of uncertainty. That is an equally
important form of progress.
In epidemiology, the spread of
disease may look chaotic at the level of individual cases, yet population-level
models reveal transmission rates, thresholds and intervention effects. The
basic reproduction number, network structure, contact patterns and immunity
profiles can turn apparent disorder into analysable dynamics. Again, the
individual event may remain uncertain, while the distribution becomes
intelligible.
In statistical physics, the
behaviour of gases cannot be predicted molecule by molecule in ordinary
practice. Yet pressure, temperature and entropy obey robust laws. The
microscopic world may be too complex to track, but the macroscopic world is not
lawless. Disorder at one scale may be order at another.
In number theory and computation,
pseudo-random sequences provide a particularly striking lesson. A deterministic
algorithm can generate outputs that pass many statistical tests for randomness.
Without knowledge of the seed and algorithm, the sequence may be
indistinguishable from random for practical purposes. Once the generator is
known, however, the apparent randomness is revealed as rule-governed. This is
not merely a technical curiosity; it is a philosophical warning. Randomness can
be observer-relative.
Yet not all apparent randomness has
been overturned
The argument must not be allowed to
overreach. It is tempting, once one has seen several examples of order emerging
from apparent disorder, to conclude that all randomness is merely ignorance.
That conclusion is not warranted.
Quantum mechanics remains the
strongest caution. The standard formalism does not merely say that we do not
know which outcome will occur. It provides probabilities that appear, under
many interpretations, to be irreducible. Bell’s theorem and the experimental
violation of Bell inequalities rule out broad classes of local hidden-variable
theories. The 2022 Nobel Prize in Physics recognised experiments with entangled
photons and violations of Bell inequalities, precisely because they reshaped
our understanding of quantum correlations. This does not force one single
philosophical interpretation upon us, but it does rule out the comfortable idea
that quantum randomness is simply classical ignorance with better curtains.
Pilot-wave theory, associated with
Louis de Broglie and David Bohm, shows that deterministic interpretations of
quantum phenomena are possible, but at the cost of nonlocality. Many-worlds
interpretations avoid collapse but reinterpret probability in another way.
Objective-collapse theories modify the dynamics. The lesson is not that
randomness has been refuted; it is that the word randomness sits within a
contested interpretative landscape.
Algorithmic randomness also resists
easy reduction. A formally random sequence is not merely one for which we have
failed to find a pattern; it is one that, under the relevant mathematical
definition, lacks any shorter effective description. Of course, applying this
to finite empirical sequences is difficult, because no finite test can prove
ultimate randomness. Still, the theory shows that randomness can be a rigorous
mathematical property, not merely a placeholder for ignorance.
There is therefore a crucial
distinction between a methodological maxim and a metaphysical doctrine. As a
methodological maxim, it is fruitful to ask: what model have we not yet found?
As a metaphysical doctrine, it is unsafe to declare: every random phenomenon is
definitely modelled underneath. The first statement promotes inquiry. The
second becomes dogma.
Towards a taxonomy of randomness
A mature discussion needs a
taxonomy. We may distinguish at least five forms of randomness.
First, there is epistemic
randomness: uncertainty caused by incomplete knowledge. The die toss, many
engineering uncertainties and numerous medical or economic predictions belong
largely here. More information may reduce uncertainty, even if it never eliminates
it fully.
Second, there is practical or
computational randomness: uncertainty caused by complexity, sensitivity or
computational infeasibility. Chaotic systems are the classic case. The
equations may exist, but prediction is limited by measurement precision and computational
growth.
Third, there is statistical
randomness: behaviour best described by probability distributions, regardless
of whether deeper causes exist. This is the working language of statistics and
experimental science.
Fourth, there is algorithmic
randomness: incompressibility relative to formal computational definitions.
This is not a statement about ordinary ignorance but about description length
and effective procedure.
Fifth, there is ontological or
physical chance: the possibility that nature itself contains irreducible
probabilities. Quantum mechanics is the primary domain in which this
possibility is taken seriously.
The thesis of dynamic modelability
applies most strongly to the first three categories. It applies with caution to
the fourth and fifth. It is entirely reasonable to say that much statistical
randomness may later be explained by richer models. It is much less reasonable
to say that mathematically incompressible sequences or quantum measurement
outcomes are certainly destined to become classical mechanisms.
The role of longer observation and
periodicity
One of the user’s central
intuitions deserves careful development: what appears non-repeating over a
given time span may reveal periodicity over a longer one. This is
scientifically plausible in many domains. Signal processing, astronomy,
geophysics, climatology, economics and biology all contain cases in which
cycles become visible only after sufficient sampling. Periodicity is not always
obvious; it may be noisy, multi-scale, drifting, intermittent or masked by
other processes.
Fourier analysis teaches that
complex signals can be decomposed into frequency components. Wavelet methods go
further, allowing time-localised patterns to be detected. Recurrence analysis,
spectral density estimation and nonlinear time-series methods all exist because
patterns are often hidden beneath irregular appearances. The scientific
question is not simply whether a pattern repeats exactly, but whether there is
structure in correlation, spectrum, scaling, attractor geometry or
distributional form.
However, longer observation is not
a magic wand. With enough data, one can also find spurious patterns. Human
beings are famously good at seeing faces in clouds and conspiracies in
coincidences. Proper statistical testing, out-of-sample validation and model
comparison are indispensable. A proposed periodicity must predict new data, not
merely decorate old data. Otherwise, one has not found order; one has merely
tailored a suit for yesterday’s mannequin.
This point is vital. The argument
for dynamic modelability must not become permission for pattern-hunting without
discipline. The right claim is not that every sufficiently long dataset will
reveal a meaningful periodicity. The right claim is that absence of detected
periodicity within a limited observation window is not proof of fundamental
randomness.
Scientific humility and the
politics of naming
To name something random is also,
in a quiet way, to exercise authority. Scientific communities decide which
explanations count, which models are acceptable, which residuals are tolerable
and which anomalies deserve attention. Such decisions are necessary; without
them, science would drown in speculation. But they are also historically
contingent. What one generation treats as noise may become the research
programme of the next.
The residual is often where
discovery begins. An unexplained deviation, a stubborn scatter, a failure of
fit: these are not embarrassments to be swept under the carpet. They are
invitations. Kepler’s ellipses, Einstein’s relativity, quantum theory and modern
chaos all arose, in different ways, from taking anomalies seriously. The
boundary between noise and signal is not fixed once and for all. It is
negotiated through instruments, mathematics, theory and patience.
This is why the phrase static
randomness is philosophically unsatisfactory. It suggests that randomness is a
final label glued permanently to a phenomenon. Dynamic modelability suggests
something better: every claim of randomness should be indexed to a state of
knowledge. Random with respect to what model? Random for which observer? Random
over what timescale? Random under which measurement resolution? Random in
principle, or random in practice?
These questions do not weaken
science. They make it more exact.
Conclusion: against static
randomness, for dynamic modelability
Randomness is not one thing. It is
a family of concepts linking probability, ignorance, information, complexity,
computation and physical theory. Used carefully, it is indispensable. Used
carelessly, it becomes a veil thrown over what we have not yet understood.
The history of science shows that
apparent randomness is often provisional. Brownian motion, planetary wandering,
chaotic weather, statistical mechanics, pseudo-random computation and many
data-driven discoveries all teach the same lesson: disorder may be the first
appearance of a deeper order. Yet the same history also warns against
triumphalism. Quantum mechanics and algorithmic randomness prevent us from
declaring that all chance is merely ignorance.
The most responsible conclusion is
therefore diplomatic but firm. We should not speak too readily of static
randomness, as though the word closed the file. We should speak instead of
dynamic modelability: the possibility that phenomena now described probabilistically
may, under richer observation, deeper mathematics, longer temporal horizons or
more powerful computation, disclose forms of order not yet available to us.
This position preserves the best of
both worlds. It respects probability theory as a rigorous science of
uncertainty. It respects quantum physics where it resists classical
simplification. But it also preserves the restless spirit of scientific
inquiry. It reminds us that the map is not the territory, the residual is not
always rubbish, and chance may sometimes be the courtesy title we give to a
pattern before we have earned the right to name it.
In the end, the wise scientist
neither worships randomness nor abolishes it by decree. He treats it as a
frontier marker. Beyond it may lie genuine indeterminacy; beyond it may lie
hidden structure; beyond it may lie a mathematics not yet written. The proper
attitude is not certainty, but disciplined expectation: that the universe has
more grammar than our present dictionaries contain.
Selected references and
intellectual landmarks
Aristotle, Physics, Book II, on
chance and necessity.
Bernoulli, J., Ars Conjectandi, on
the law of large numbers.
Boltzmann, L., Lectures on Gas
Theory, on statistical mechanics.
Born, M., on the probabilistic
interpretation of the wave function.
Chaitin, G., Algorithmic
Information Theory, on randomness as incompressibility.
Einstein, A., On the Movement of
Small Particles Suspended in Stationary Liquids Required by the
Molecular-Kinetic Theory of Heat, 1905.
Eagle, A., Chance versus
Randomness, Stanford Encyclopedia of Philosophy.
Hájek, A., Interpretations of
Probability, Stanford Encyclopedia of Philosophy.
Kolmogorov, A. N., Foundations of
the Theory of Probability, 1933.
Laplace, P.-S., A Philosophical
Essay on Probabilities.
Lorenz, E. N., Deterministic
Nonperiodic Flow, 1963.
Shannon, C. E., A Mathematical
Theory of Communication, 1948.
Von Mises, R., Probability,
Statistics and Truth, on collectives and frequency.
Nobel Prize in Physics 2022,
awarded for experiments with entangled photons, establishing the violation of
Bell inequalities and pioneering quantum information science.
