The Moving Frontier of Randomness: Why Chance May Be the Name We Give to Unmodelled Order

 

The Moving Frontier of Randomness: Why Chance May Be the Name We Give to Unmodelled Order

A scientific and philosophical essay on probability, pattern, ignorance and dynamic modelability

Anthony Dernellis

JUN 2026

 

Introduction: the useful scandal of chance

Randomness is among the most useful, and among the most easily misunderstood, words in the scientific vocabulary. It is useful because it gives disciplined language to uncertainty. Without it, modern statistics, thermodynamics, quantum theory, risk analysis, genetics, epidemiology, cryptography, artificial intelligence, insurance, econometrics and experimental science would be almost unthinkable. It is misunderstood because the word often sounds more final than it ought to sound. To say that an event is random may mean several quite different things: that no cause exists; that a cause exists but is inaccessible; that the event belongs to a probability distribution; that no finite description can compress its sequence; that the system is deterministic but practically unpredictable; or, more modestly, that no model presently available to us has succeeded in explaining it.

This ambiguity is not a defect to be brushed aside. It is precisely where the philosophical interest of randomness begins. For the scientist at the bench, randomness is often an operational term: it tells us how to calculate, test, simulate or infer. For the philosopher of science, it is also an epistemological alarm bell: it tells us where knowledge stops, where measurement fails, where computational complexity overwhelms prediction, or where nature itself may resist classical explanation. For the mathematician, randomness can be formalised with great rigour, yet such formalisation does not, by itself, settle what randomness means in the world.

The thesis explored in this article is deliberately bold, but not reckless: perhaps a considerable portion of what human beings call randomness is not the absence of order, but the presence of order that has not yet been recognised, modelled, observed over a sufficiently long interval, or rendered computationally tractable. In plain English, we often call a distribution random when we have not yet found its grammar. Yesterday’s noise may become tomorrow’s equation.

This is not a claim that all phenomena are secretly deterministic in the simple, clockwork sense of nineteenth-century physics. Such a claim would be scientifically incautious, especially in the light of quantum mechanics and the experimental constraints placed on local hidden-variable theories. Nor is it a denial of probability theory, which remains one of the most powerful achievements of modern thought. Rather, the proposal is subtler: randomness should often be treated not as a static verdict, but as a moving frontier. It is the name we give to what, at a given historical moment, lies beyond our current models, instruments, datasets and computational reach.

The distinction matters. If randomness is treated as an ontological finality too quickly, inquiry may stop prematurely. If, conversely, every instance of apparent randomness is treated as a puzzle awaiting a deterministic key, science risks falling into metaphysical overconfidence. The wiser position is neither credulous determinism nor lazy surrender to chance. It is a disciplined agnosticism: a readiness to use probabilistic models with full technical seriousness, while also admitting that scientific history has repeatedly transformed apparent disorder into intelligible structure.

Scientific definitions and the many faces of randomness

Contemporary science does not define randomness in only one way. In probability theory, following the axiomatic work of Andrey Kolmogorov, randomness is handled through probability spaces: a sample space, events and a probability measure satisfying precise mathematical rules. This approach is extraordinarily successful because it avoids unnecessary metaphysics. It does not ask whether a coin toss is truly indeterminate; it asks what formal rules probabilities must obey and how those rules can be used.

In the frequency interpretation associated with Richard von Mises and others, probability is connected with long-run relative frequency. An event has probability one half, for example, if in a sufficiently extended sequence of trials its frequency tends to one half. Von Mises also introduced the idea of a collective: a sequence whose limiting frequency remains stable under admissible selection rules. The intuition is recognisable from gambling: a genuinely random sequence should not allow a system that reliably extracts profit by selecting a favourable subsequence.

Bayesian probability, associated historically with Thomas Bayes and later with thinkers such as Pierre-Simon Laplace, Frank Ramsey, Bruno de Finetti, Leonard Savage and many others, interprets probability as rational degree of belief under uncertainty. On this view, randomness is not necessarily a property of the world alone; it is also related to the information available to an observer. De Finetti’s famous provocation, that probability does not exist, was not a denial of practical probabilistic reasoning, but a rejection of probability as a freestanding physical substance. Probability, for him, belonged to coherent expectation.

In information theory, Claude Shannon made uncertainty measurable through entropy. A message source has high entropy when its outputs are difficult to predict and contain much information. This is not identical with metaphysical chance. It is a measure of informational uncertainty. A perfectly patterned message has low entropy; a message with many equally plausible alternatives has high entropy. Shannon’s framework helped detach randomness from loose intuition and connect it with communication, coding and signal processing.

Algorithmic information theory, developed by Ray Solomonoff, Andrey Kolmogorov, Gregory Chaitin and Per Martin-Löf, sharpened the idea further. A sequence is algorithmically random if there is no shorter effective description of it than the sequence itself. In that sense, randomness is incompressibility. The digits of pi may look irregular, but they are not algorithmically random in this strict sense, because a compact algorithm generates them. By contrast, a truly incompressible sequence cannot be reduced to a rule simpler than itself. This concept is severe, elegant and philosophically unsettling: it shows that apparent irregularity and true formal randomness are not the same thing.

In dynamical systems theory, randomness is often practical rather than fundamental. A deterministic system can behave unpredictably if it is nonlinear and sensitive to initial conditions. Edward Lorenz’s work on weather and chaos famously showed that very small differences in initial states can grow into large differences in outcomes. The equations may be deterministic; the forecast may nevertheless fail beyond a certain horizon. Here, randomness is not absence of law but excess sensitivity relative to measurement precision.

In quantum mechanics, the issue becomes sharper. Standard quantum theory predicts probabilities for measurement outcomes, and many interpretations treat certain outcomes as genuinely indeterminate. The Born rule gives probabilities with astonishing empirical success. At the same time, debates over interpretation remain open: Copenhagen-type views, many-worlds interpretations, pilot-wave theories and objective-collapse models do not agree on what quantum probabilities ultimately mean. What is not responsible is to pretend that quantum randomness has simply been overthrown. The experimental violation of Bell inequalities places deep restrictions on local hidden-variable explanations. Whatever lies beneath quantum probabilities, it cannot be an ordinary local classical mechanism in disguise.

These diverse approaches show why a single blunt sentence, such as ‘randomness means lack of pattern’, is inadequate. Randomness may be formal, statistical, epistemic, informational, dynamical or physical. It may describe the world, our knowledge of the world, or the limits of our methods. Any serious discussion must keep these meanings distinct.

A historical review of the principal scientific and mathematical approaches

The intellectual history of randomness begins long before modern probability. Aristotle distinguished events that occur always or for the most part from events that occur by chance. For him, chance did not necessarily mean absence of cause; it often meant the accidental intersection of causal lines. A man goes to the market for one reason and unexpectedly meets a debtor there: the meeting is by chance, though each causal strand is intelligible.

The mathematics of probability arose from games of chance. Blaise Pascal and Pierre de Fermat, in their seventeenth-century correspondence about gambling problems, helped create the combinatorial foundations of probability. Christiaan Huygens soon produced one of the first systematic treatments. Here randomness entered mathematics through dice, cards and wagers: not as cosmic mystery, but as calculable uncertainty.

Jacob Bernoulli’s Ars Conjectandi introduced the law of large numbers, showing that relative frequencies stabilise under repeated trials. This was a major conceptual turn. Individual outcomes may be unpredictable, yet aggregate behaviour may be regular. The paradox remains central today: randomness at the level of the single event may coexist with order at the level of distribution.

Laplace pushed probability into a grander philosophical frame. His imagined intelligence, later called Laplace’s demon, knew all forces and positions and could compute the future and past. In such a universe, chance would be merely a confession of ignorance. Laplace’s determinism is no longer an adequate description of modern physics, but it remains a powerful symbol of epistemic randomness: the idea that chance may reflect the observer’s limitation rather than nature’s indecision.

The nineteenth century brought probability into physics. James Clerk Maxwell and Ludwig Boltzmann used statistical reasoning to explain gases. Individual molecular motions were too numerous to track, but their collective behaviour could be described with statistical laws. This was not a defeat for science; it was a triumph. A new kind of explanation emerged: not prediction of every particle, but understanding of macroscopic regularities from microscopic multiplicity.

Henri Poincaré saw, perhaps more clearly than most of his contemporaries, that deterministic systems could exhibit behaviour that looks random because small causes may have large effects. His insights foreshadowed chaos theory. A century later, Lorenz’s numerical weather models made the point vivid. Determinism did not guarantee predictability. This single lesson has immense importance: the absence of prediction is not proof of absence of law.

In the twentieth century, Kolmogorov made probability mathematically rigorous through measure theory. This was a liberation. Probability no longer needed to settle every philosophical quarrel before being used. It became a formal calculus applicable to many interpretations. The price of such elegance, however, was that the formalism alone did not answer whether the probabilities represented frequencies, beliefs, propensities or objective chances.

Von Mises and later frequentists attempted to anchor probability in repeated observations. Ronald Fisher, Jerzy Neyman and Egon Pearson developed statistical inference frameworks that made randomness central to experimental design, estimation and hypothesis testing. Random sampling and randomisation became tools not because they created metaphysical chaos, but because they controlled bias and enabled valid inference.

Shannon’s information theory transformed randomness into a measure of uncertainty in communication. Chaitin and others then connected randomness with computation and incompressibility. These developments are crucial for the present argument. They show that whether something is random may depend on the descriptive language, algorithmic resources and modelling framework available. A sequence may appear random to one observer and structured to another who knows the generating rule.

Finally, quantum mechanics forced the deepest confrontation. Max Born’s probabilistic interpretation, Heisenberg’s uncertainty principle, Bohr’s complementarity and later Bell’s theorem challenged the classical assumption that every apparent uncertainty must be removable by better information of a familiar kind. Einstein resisted this conclusion, famously objecting to a universe in which God plays dice. Yet the subsequent history of quantum foundations has not restored simple classical determinism. It has, rather, made the relation between probability, measurement, locality and reality more subtle than either naïve determinists or naïve indeterminists might wish.

The central proposal: randomness as unrecognised model

The central proposal of this article may now be stated more precisely. Human beings often call a phenomenon random when one or more of the following conditions hold: the observational period is too short; the relevant variables are unknown; the measurement resolution is inadequate; the system is too complex for available computation; the correct mathematical language has not yet been invented; or the pattern exists at a scale not yet examined. In such cases, randomness is not necessarily a property of the phenomenon itself. It is a property of the relationship between the phenomenon and our present capacity to describe it.

This proposal is not an attack on probability. On the contrary, probability is precisely what allows science to proceed when full modelling is not available. But probability may be both a final theory in some domains and a provisional instrument in others. The mistake is to assume that the same word, randomness, carries the same philosophical force in every case.

Consider a sequence of observations that shows no repeating pattern over ten trials. It would be foolish to infer a law. Over a thousand trials, a frequency distribution may appear. Over a million trials, a subtle periodicity may be detectable. Over a century of observations, a cycle may emerge that was invisible within a decade. Time is not a passive container of data; it is often the very instrument that reveals structure.

This matters in fields where the relevant cycles exceed human patience. Climate oscillations, geological processes, evolutionary dynamics, demographic transitions and astronomical variations may all involve timescales that dwarf ordinary observation. A phenomenon can seem patternless simply because the observational window is too narrow. To borrow a homely British phrase, one should not declare the orchestra tuneless after hearing two bars through a closed door.

The same applies to dimensionality. A distribution may look random in one projection and structured in a higher-dimensional representation. Modern machine learning repeatedly demonstrates this point. Data that appear scattered in two variables may become classifiable when many features are considered. This does not mean that machine learning discovers metaphysical truth; it means that patterns can be invisible until the right representation is chosen.

There is also the matter of hidden variables, though the term must be used carefully. Outside quantum foundations, hidden variables simply mean unobserved causal factors. A medical outcome may appear random until genetics, environment, treatment adherence and comorbidities are included. A traffic jam may seem accidental until road topology, behavioural response and signal timing are modelled. A biological mutation may be random with respect to an organism’s adaptive needs, yet its molecular mechanisms are not magical. Randomness here means independence from a particular explanatory axis, not absence of physical process.

Even in games of chance, apparent randomness is conditional. A fair die toss is modelled probabilistically because its exact outcome depends on initial position, angular velocity, surface interaction, air resistance and collision dynamics. In principle, classical mechanics applies. In practice, measurement and computation are insufficient. The die is not a metaphysical oracle; it is a small chaotic machine. The phrase ‘random toss’ is a practical shorthand for extreme sensitivity and ignorance of initial conditions.

Examples where apparent randomness became modelled structure

Brownian motion is perhaps the most instructive example. To the early observer, tiny particles suspended in liquid moved irregularly, almost capriciously. Einstein’s 1905 analysis connected this motion with molecular-kinetic theory, and Jean Perrin’s experimental work helped confirm the reality of atoms and molecules. The motion remained stochastic in its mathematical description, but its irregularity was no longer mysterious. It became intelligible through statistical mechanics. Here randomness was not abolished; it was domesticated.

Planetary motion offers a more classical case. Before adequate celestial mechanics, planetary wanderings across the sky looked irregular against the fixed stars. The very word planet derives from wandering. Mathematical astronomy, culminating in Kepler’s laws and Newton’s gravitation, transformed wandering into orbit. The phenomenon did not change; the model did. What seemed irregular became lawful once the right geometrical and dynamical framework was available.

Weather provides a third example, but with a twist. Weather has not become perfectly predictable. Rather, meteorology has clarified why prediction has limits. The atmosphere is governed by physical equations, but nonlinear dynamics and sensitivity to initial conditions impose practical horizons. Here model discovery did not eliminate uncertainty; it explained the structure of uncertainty. That is an equally important form of progress.

In epidemiology, the spread of disease may look chaotic at the level of individual cases, yet population-level models reveal transmission rates, thresholds and intervention effects. The basic reproduction number, network structure, contact patterns and immunity profiles can turn apparent disorder into analysable dynamics. Again, the individual event may remain uncertain, while the distribution becomes intelligible.

In statistical physics, the behaviour of gases cannot be predicted molecule by molecule in ordinary practice. Yet pressure, temperature and entropy obey robust laws. The microscopic world may be too complex to track, but the macroscopic world is not lawless. Disorder at one scale may be order at another.

In number theory and computation, pseudo-random sequences provide a particularly striking lesson. A deterministic algorithm can generate outputs that pass many statistical tests for randomness. Without knowledge of the seed and algorithm, the sequence may be indistinguishable from random for practical purposes. Once the generator is known, however, the apparent randomness is revealed as rule-governed. This is not merely a technical curiosity; it is a philosophical warning. Randomness can be observer-relative.

Yet not all apparent randomness has been overturned

The argument must not be allowed to overreach. It is tempting, once one has seen several examples of order emerging from apparent disorder, to conclude that all randomness is merely ignorance. That conclusion is not warranted.

Quantum mechanics remains the strongest caution. The standard formalism does not merely say that we do not know which outcome will occur. It provides probabilities that appear, under many interpretations, to be irreducible. Bell’s theorem and the experimental violation of Bell inequalities rule out broad classes of local hidden-variable theories. The 2022 Nobel Prize in Physics recognised experiments with entangled photons and violations of Bell inequalities, precisely because they reshaped our understanding of quantum correlations. This does not force one single philosophical interpretation upon us, but it does rule out the comfortable idea that quantum randomness is simply classical ignorance with better curtains.

Pilot-wave theory, associated with Louis de Broglie and David Bohm, shows that deterministic interpretations of quantum phenomena are possible, but at the cost of nonlocality. Many-worlds interpretations avoid collapse but reinterpret probability in another way. Objective-collapse theories modify the dynamics. The lesson is not that randomness has been refuted; it is that the word randomness sits within a contested interpretative landscape.

Algorithmic randomness also resists easy reduction. A formally random sequence is not merely one for which we have failed to find a pattern; it is one that, under the relevant mathematical definition, lacks any shorter effective description. Of course, applying this to finite empirical sequences is difficult, because no finite test can prove ultimate randomness. Still, the theory shows that randomness can be a rigorous mathematical property, not merely a placeholder for ignorance.

There is therefore a crucial distinction between a methodological maxim and a metaphysical doctrine. As a methodological maxim, it is fruitful to ask: what model have we not yet found? As a metaphysical doctrine, it is unsafe to declare: every random phenomenon is definitely modelled underneath. The first statement promotes inquiry. The second becomes dogma.

Towards a taxonomy of randomness

A mature discussion needs a taxonomy. We may distinguish at least five forms of randomness.

First, there is epistemic randomness: uncertainty caused by incomplete knowledge. The die toss, many engineering uncertainties and numerous medical or economic predictions belong largely here. More information may reduce uncertainty, even if it never eliminates it fully.

Second, there is practical or computational randomness: uncertainty caused by complexity, sensitivity or computational infeasibility. Chaotic systems are the classic case. The equations may exist, but prediction is limited by measurement precision and computational growth.

Third, there is statistical randomness: behaviour best described by probability distributions, regardless of whether deeper causes exist. This is the working language of statistics and experimental science.

Fourth, there is algorithmic randomness: incompressibility relative to formal computational definitions. This is not a statement about ordinary ignorance but about description length and effective procedure.

Fifth, there is ontological or physical chance: the possibility that nature itself contains irreducible probabilities. Quantum mechanics is the primary domain in which this possibility is taken seriously.

The thesis of dynamic modelability applies most strongly to the first three categories. It applies with caution to the fourth and fifth. It is entirely reasonable to say that much statistical randomness may later be explained by richer models. It is much less reasonable to say that mathematically incompressible sequences or quantum measurement outcomes are certainly destined to become classical mechanisms.

The role of longer observation and periodicity

One of the user’s central intuitions deserves careful development: what appears non-repeating over a given time span may reveal periodicity over a longer one. This is scientifically plausible in many domains. Signal processing, astronomy, geophysics, climatology, economics and biology all contain cases in which cycles become visible only after sufficient sampling. Periodicity is not always obvious; it may be noisy, multi-scale, drifting, intermittent or masked by other processes.

Fourier analysis teaches that complex signals can be decomposed into frequency components. Wavelet methods go further, allowing time-localised patterns to be detected. Recurrence analysis, spectral density estimation and nonlinear time-series methods all exist because patterns are often hidden beneath irregular appearances. The scientific question is not simply whether a pattern repeats exactly, but whether there is structure in correlation, spectrum, scaling, attractor geometry or distributional form.

However, longer observation is not a magic wand. With enough data, one can also find spurious patterns. Human beings are famously good at seeing faces in clouds and conspiracies in coincidences. Proper statistical testing, out-of-sample validation and model comparison are indispensable. A proposed periodicity must predict new data, not merely decorate old data. Otherwise, one has not found order; one has merely tailored a suit for yesterday’s mannequin.

This point is vital. The argument for dynamic modelability must not become permission for pattern-hunting without discipline. The right claim is not that every sufficiently long dataset will reveal a meaningful periodicity. The right claim is that absence of detected periodicity within a limited observation window is not proof of fundamental randomness.

Scientific humility and the politics of naming

To name something random is also, in a quiet way, to exercise authority. Scientific communities decide which explanations count, which models are acceptable, which residuals are tolerable and which anomalies deserve attention. Such decisions are necessary; without them, science would drown in speculation. But they are also historically contingent. What one generation treats as noise may become the research programme of the next.

The residual is often where discovery begins. An unexplained deviation, a stubborn scatter, a failure of fit: these are not embarrassments to be swept under the carpet. They are invitations. Kepler’s ellipses, Einstein’s relativity, quantum theory and modern chaos all arose, in different ways, from taking anomalies seriously. The boundary between noise and signal is not fixed once and for all. It is negotiated through instruments, mathematics, theory and patience.

This is why the phrase static randomness is philosophically unsatisfactory. It suggests that randomness is a final label glued permanently to a phenomenon. Dynamic modelability suggests something better: every claim of randomness should be indexed to a state of knowledge. Random with respect to what model? Random for which observer? Random over what timescale? Random under which measurement resolution? Random in principle, or random in practice?

These questions do not weaken science. They make it more exact.

Conclusion: against static randomness, for dynamic modelability

Randomness is not one thing. It is a family of concepts linking probability, ignorance, information, complexity, computation and physical theory. Used carefully, it is indispensable. Used carelessly, it becomes a veil thrown over what we have not yet understood.

The history of science shows that apparent randomness is often provisional. Brownian motion, planetary wandering, chaotic weather, statistical mechanics, pseudo-random computation and many data-driven discoveries all teach the same lesson: disorder may be the first appearance of a deeper order. Yet the same history also warns against triumphalism. Quantum mechanics and algorithmic randomness prevent us from declaring that all chance is merely ignorance.

The most responsible conclusion is therefore diplomatic but firm. We should not speak too readily of static randomness, as though the word closed the file. We should speak instead of dynamic modelability: the possibility that phenomena now described probabilistically may, under richer observation, deeper mathematics, longer temporal horizons or more powerful computation, disclose forms of order not yet available to us.

This position preserves the best of both worlds. It respects probability theory as a rigorous science of uncertainty. It respects quantum physics where it resists classical simplification. But it also preserves the restless spirit of scientific inquiry. It reminds us that the map is not the territory, the residual is not always rubbish, and chance may sometimes be the courtesy title we give to a pattern before we have earned the right to name it.

In the end, the wise scientist neither worships randomness nor abolishes it by decree. He treats it as a frontier marker. Beyond it may lie genuine indeterminacy; beyond it may lie hidden structure; beyond it may lie a mathematics not yet written. The proper attitude is not certainty, but disciplined expectation: that the universe has more grammar than our present dictionaries contain.

Selected references and intellectual landmarks

Aristotle, Physics, Book II, on chance and necessity.

Bernoulli, J., Ars Conjectandi, on the law of large numbers.

Boltzmann, L., Lectures on Gas Theory, on statistical mechanics.

Born, M., on the probabilistic interpretation of the wave function.

Chaitin, G., Algorithmic Information Theory, on randomness as incompressibility.

Einstein, A., On the Movement of Small Particles Suspended in Stationary Liquids Required by the Molecular-Kinetic Theory of Heat, 1905.

Eagle, A., Chance versus Randomness, Stanford Encyclopedia of Philosophy.

Hájek, A., Interpretations of Probability, Stanford Encyclopedia of Philosophy.

Kolmogorov, A. N., Foundations of the Theory of Probability, 1933.

Laplace, P.-S., A Philosophical Essay on Probabilities.

Lorenz, E. N., Deterministic Nonperiodic Flow, 1963.

Shannon, C. E., A Mathematical Theory of Communication, 1948.

Von Mises, R., Probability, Statistics and Truth, on collectives and frequency.

Nobel Prize in Physics 2022, awarded for experiments with entangled photons, establishing the violation of Bell inequalities and pioneering quantum information science.