I begin with the definition of power, and find that it is finalistic inasmuch as work directs energy dissipation in the interests of some system. The maximum power principle of Lotka and Odum implies an optimal energy efficiency for any work; optima are also finalities. I advance a statement of the maximum entropy production principle, suggesting that most work of dissipative structures is carried out at rates entailing energy flows faster than those that would associate with (...) maximum power. This is finalistic in the sense that the out-of-equilibrium universe, taken as an isolated system, entrains work in the interest of global thermodynamic equilibration. I posit an evolutionary scenario, with a development on Earth from abiotic times, when promoting convective energy flows could be viewed as the important function of dissipative structures, to biotic times when the preservation of living dissipative structures was added to the teleology. Dissipative structures are required by the equilibrating universe to enhance local energy gradient dissipation. (shrink)

The Maximum Entropy Production Principle (MEPP) stands out as an overarching principle that rules life phenomena in Nature. However, its explanatory power beyond heuristics remains controversial. On the one hand, the MEPP has been successfully applied principally to non-living systems far from thermodynamic equilibrium. On the other hand, the underlying assumptions to lay the MEPP’s theoretical foundations and range of applicability increase the possibilities of conflicting interpretations. More interestingly, from a metaphysical stance, the MEPP’s philosophical status is (...) hotly debated: does the MEPP passively translate physical information into macroscopic predictions or actively select the physical solution in multistable systems, granting the connection between scientific models and reality? This paper deals directly with this dilemma by discussing natural determination from three angles: (1) Heuristics help natural philosophers to build an ontology. (2) The MEPP’s ontological status may stem from its selection of new forms of causation beyond physicalism. (3) The MEPP’s ontology ultimately depends on the much-discussed question of the ontology of probabilities in an information-theoretic approach and the ontology of macrostates according to the Boltzmannian definition of entropy. (shrink)

In the previous papers, it was demonstrated that applying the principle of maximum information entropy by maximizing the conditional information entropy, subject to the constraint given by the Liouville equation averaged over the phase space, leads to a definition of the rate of entropy change for closed Hamiltonian systems without any additional assumptions. Here, we generalize this basic model and, with the introduction of the additional constraints which are equivalent to the hydrodynamic continuity equations, show that (...) the results obtained are consistent with the known results from the nonequilibrium statistical mechanics and thermodynamics of irreversible processes. In this way, as a part of the approach developed in this paper, the rate of entropy change and entropy production density for the classical Hamiltonian fluid are obtained. The results obtained suggest the general applicability of the foundational principles of predictive statistical mechanics and their importance for the theory of irreversibility. (shrink)

In a previous paper, a statistical method of constructing quantum models of classical properties has been described. The present paper concludes the description by turning to classical mechanics. The quantum states that maximize entropy for given averages and variances of coordinates and momenta are called ME packets. They generalize the Gaussian wave packets. A non-trivial extension of the partition-function method of probability calculus to quantum mechanics is given. Non-commutativity of quantum variables limits its usefulness. Still, the general form of (...) the state operators of ME packets is obtained with its help. The diagonal representation of the operators is found. A general way of calculating averages that can replace the partition function method is described. Classical mechanics is reinterpreted as a statistical theory. Classical trajectories are replaced by classical ME packets. Quantum states approximate classical ones if the product of the coordinate and momentum variances is much larger than Planck constant. Thus, ME packets with large variances follow their classical counterparts better than Gaussian wave packets. (shrink)

The current disruption of ecosystems and climate systems can be likened to an increase in entropy within our planet. This concept is often linked to the second law of thermodynamics, which predicts a necessary rise in entropy resulting from all material and energy-related processes, including the intricate organisation of living systems. Consequently, discussions surrounding the ongoing crisis commonly carry an underlying sense of fatalism when referencing thermodynamic principles. In this study, we explore how the understanding of life has (...) been harmonized with thermodynamics to show that entropy production is a consequence of heightened complexity in life rather than its breakdown. Furthermore, it is crucial to perform a thermodynamic analysis of the Earth system as a whole to dispel fatalistic assumptions. The extremum principles linked to thermodynamics do not foretell the precise evolution of complex organisations but rather set the thermodynamic boundaries associated with their development. Ultimately, treating the Earth system as an integrated autonomous entity in which life and human societies play pivotal roles is essential for charting a sustainable path forward for humanity. Understanding how to contribute to thermodynamic states that are more conducive to life, rather than hastening the journey towards chaotic states, is paramount for human survival and well-being in the Anthropocene era. (shrink)

Information is the reduction of uncertainty, and uncertainty is equivalent to entropy. Thus, information is negative entropy. Concepts from information theory can be used to analyse physiological and psychological processes and to probe the nature of abstract ideas such as consciousness and vitality. A previously published model of the Universe indicates that it consists of charge and entropy. Since entropy is energy, then charge and energy are enough to explain the whole of nature. The building blocks (...) of molecules (neutrons, protons and electrons) are formed of negative entropy. Complex high information systems (also negative entropy) lead to living organisms and eventually to complex multicellular organisms that are both vital and conscious. Most of the Universe is a void (maximum entropy) but there are tiny foci, such as the surface of the earth, which are information rich. Everything on the earth is composed of negative entropy—from the simplest physical structure to the most complex product of human thought. There is no fundamental difference between mind and matter. (shrink)

In their book, Darwinism Evolving: Systems Dynamics and the Genealogy of Natural Selection, Depew and Weber argued for the need to address the relationship between self-organization and natural selection in evolutionary theory, and focused on seven “visions” for doing so. Recently, Batten et al. in a paper in this journal, entitled “Visions of evolution: self-organization proposes what natural selection disposes,” picked up the issue with the work of Depew and Weber as a starting point. While the efforts of both sets (...) of authors are to be commended, there are substantive errors in both the presentations of my work and of my work with colleagues that undermine theirs. My purpose here is to correct the errors in question, thereby removing the undermining effects and in so doing reassert the position my colleagues and I first advanced more than two decades ago, and that I still stand by and argue for today. The central points are as follows: Self-organization or spontaneous ordering is a process of selection; this selection process is governed by a “physical selection principle”; this principle is the law of maximum entropy production; and natural selection is a special case where the components are replicating. (shrink)

In recent decades, several theories have claimed to explain the teleological causality of organisms as a function of self-organising and self-producing processes. The most widely cited theories of this sort are variations of autopoiesis, originally introduced by Maturana and Varela. More recent modifications of autopoietic theory have focused on system organisation, closure of constraints and autonomy to account for organism teleology. This article argues that the treatment of teleology in autopoiesis and other organisation theories is inconclusive for three reasons: First, (...) non-living self-organising processes like autocatalysis meet the defining features of autopoiesis without being teleological; second, organisational approaches, whether defined in terms of the closure of constraints, self-determination or autonomy, are unable to specify teleological normativity, that is, the individuation of an ultimate beneficiary; third, all self-organised systems produce local order by maximising the throughput of energy and/or material (obeying the maximum entropy production (MEP) principle) and thereby are specifically organised to undermine their own critical boundary conditions. Despite these inadequacies, an alternative approach called teleodynamics accounts for teleology. This theory shows how multiple self-organising processes can be collectively linked so that they counter each other’s MEP principle tendencies to become codependent. Teleodynamics embraces – not ignoring – the difficulties of self-organisation, but reinstates teleology as a radical phase transition distinguishing systems embodying an orientation towards their own beneficial ends from those that lack normative character. (shrink)

Many religions and religious philosophies say that ultimate reality is a kind of primal energy. This energy is often described as a vital power animating living things, as a spiritual force directing the organization of matter, or as a divine creative power which generates all things. By refuting older conceptions of primal energy, modern science opens the door to new and more precise conceptions. Primal energy is referred to here as ‘spirit’. But spirit is a natural power. A naturalistic theory (...) of spirit is developed using ideas from information theory and thermodynamics, such as the maximum entropy production principle. Spirit drives the evolution of complexity at all levels of existence. (shrink)

The disciplines of cybernetics, semiotics and thermodynamics investigate evolutionary processes quite independently from each other. The aim of this paper is to draw the parallels and point out the possibility and necessity of a reconciliation between these disciplines. The concept of metasystem transition has been proposed by Turchin as a quantum of evolution from a cybernetic point of view. Semiotic processes are of prime importance for the realisation of metasystem transitions in the course of evolution. From a thermodynamic point of (...) view, the emergence of more complex, self-producing agents depends on the emergence of more advanced forms of semiosis. As an evolutionary consequence, more symbolic forms of semiosis that allow higher competence for abstraction and anticipation emerge. (shrink)

Two open questions of inductive reasoning are solved: (1) does the principle of maximum entropy (pme) give a solution to the obverse Majerník problem; and (2) is Wagner correct when he claims that Jeffrey’s updating principle (jup) contradicts pme? Majerník shows that pme provides unique and plausible marginal probabilities, given conditional probabilities. The obverse problem posed here is whether pme also provides such conditional probabilities, given certain marginal probabilities. The theorem developed to solve the obverse Majerník problem demonstrates (...) that in the special case introduced by Wagner pme does not contradict jup, but elegantly generalizes it and offers a more integrated approach to probability updating. (shrink)

This paper is a sequel to an earlier result of the authors that in making inferences from certain probabilistic knowledge bases the maximum entropy inference process, ME, is the only inference process respecting “common sense.” This result was criticized on the grounds that the probabilistic knowledge bases considered are unnatural and that ignorance of dependence should not be identified with statistical independence. We argue against these criticisms and also against the more general criticism that ME is representation dependent. (...) In a final section, however, we provide a criticism of our own of ME, and of inference processes in general, namely that they fail to satisfy compactness. (shrink)

The general attributes of ecosystems are examined and a naturally occurring reference ecosystem is established, comparable with the isolated system of classical thermodynamics. Such an autonomous system with a stable, periodic input of energy is shown to assume certain structural characteristics that have an identifiable thermodynamic basis. Individual species tend to assume a state of least dissipation; this is most clearly evident in the dominant species (the species with the best integration of energy acquisition and conservation). It is concluded that (...) ecosystem structure results from the antagonistic interaction of two nearly equal forces. These forces have their origin in the Principle of Most Action (least dissipation or least entropy production) and the universal Principle of Least Action. Most action is contingent on the equipartitioning of the energy available, through uniform interaction of similar individuals. The trend to Least action is contingent on increased dissipation attained through increasing diversity and increasing complexity. These principles exhibit a basic asymmetry. Given the operation of these opposing principles over evolutionary time, it is argued that ecosystems originated in the vicinity of thermodynamic equilibrium through the resonant amplification of reversible fluctuations. On account of the basic asymmetry the system was able to evolve away from thermodynamic equilibrium provided that it remained within the vicinity of ergodynamic equilibrium (equilibrium maintained by internal work, where the opposing forces are equal and opposite).At the highest level of generalization there appear to be three principles operating: i) maximum association of free-energy and materials; ii) energy conservation (deceleration of the energy flow) through symmetric interaction and increased homogeneity; and iii) the principle of least action which induces acceleration of the energy flow through asymmetrical interaction. The opposition and asymmetry of the two forces give rise to natural selection and evolution. (shrink)

We investigate uncertain reasoning with quantified sentences of the predicate calculus treated as the limiting case of maximum entropy inference applied to finite domains.

It is usually supposed that the central limit theorem explains why various quantities we find in nature are approximately normally distributed—people's heights, examination grades, snowflake sizes, and so on. This sort of explanation is found in many textbooks across the sciences, particularly in biology, economics, and sociology. Contrary to this received wisdom, I argue that in many cases we are not justified in claiming that the central limit theorem explains why a particular quantity is normally distributed, and that in some (...) cases, we are actually wrong. 1 Introduction2 Normal Distributions and the Central Limit Theorem2.1 Normal distributions2.2 The central limit theorem2.3 Terminology3 Explaining Normality3.1 Loaves of bread3.2 Varying variances and probability densities3.3 Tensile strengths and problems with summation3.4 Products of factors and log-normal distributions3.5 Transforming factors and sub-factors3.6 Transformations of quantities3.7 Quantitative genetics3.8 Inference to the best explanation4 Maximum Entropy Explanations5 Conclusion. (shrink)

The new mathematical connection of De Donder’s differential entropy production with the differential changes of thermodynamic potentials (Helmholtz free energy, enthalpy, and Gibbs free energy) was obtained through the linear sequence of equations (direct, straightforward path), in which we use rigorous thermodynamic definitions of the partial molar thermodynamic properties. This new connection uses a global approach to the problem of reversibility and irreversibility, which is vital to global learners’ view and standardizes the linking procedure for thermodynamic potentials (Helmholtz (...) free energy, enthalpy, and and Gibbs free energy)—preferably to the sensing learners. It is shown that De Donder’s differential entropy production in an isolated composite system is equal to the differential change in total entropy and that De Donder’s equation agrees with Clausius’ inequality. The useful work of the irreversible process is discussed, which with the decrease of irreversibility tends towards the hypothetical maximum useful work of the reversible process. (shrink)

The principle of maximum entropy is a general method to assign values to probability distributions on the basis of partial information. This principle, introduced by Jaynes in 1957, forms an extension of the classical principle of insufficient reason. It has been further generalized, both in mathematical formulation and in intended scope, into the principle of maximum relative entropy or of minimum information. It has been claimed that these principles are singled out as unique methods of statistical (...) inference that agree with certain compelling consistency requirements. This paper reviews these consistency arguments and the surrounding controversy. It is shown that the uniqueness proofs are flawed, or rest on unreasonably strong assumptions. A more general class of inference rules, maximizing the so-called Re[acute ]nyi entropies, is exhibited which also fulfill the reasonable part of the consistency assumptions. (shrink)

Several authors have investigated the question of whether canonical logic-based accounts of belief revision, and especially the theory of AGM revision operators, are compatible with the dynamics of Bayesian conditioning. Here we show that Leitgeb's stability rule for acceptance, which has been offered as a possible solution to the Lottery paradox, allows to bridge AGM revision and Bayesian update: using the stability rule, we prove that AGM revision operators emerge from Bayesian conditioning by an application of the principle of (...) class='Hi'>maximum entropy. In situations of information loss, or whenever the agent relies on a qualitative description of her information state - such as a plausibility ranking over hypotheses, or a belief set - the dynamics of AGM belief revision are compatible with Bayesian conditioning; indeed, through the maximum entropy principle, conditioning naturally generated AGM revision operators. This mitigates an impossibility theorem of Lin and Kelly for tracking Bayesian conditioning with AGM revision, and suggests an approach to the compatibility problem that highlights the information loss incurred by acceptance rules in passing from probabilistic to qualitative representations of beliefs. (shrink)

The principle of insufficient reason assigns equal probabilities to each alternative of a random experiment whenever there is no reason to prefer one over the other. The maximum entropy principle generalizes PIR to the case where statistical information like expectations are given. It is known that both principles result in paradoxical probability updates for joint distributions of cause and effect. This is because constraints on the conditional P P\left result in changes of P P\left that assign higher probability (...) to those values of the cause that offer more options for the effect, suggesting “intentional behavior.” Earlier work therefore suggested sequentially maximizing entropy according to the causal order, but without further justification apart from plausibility on toy examples. We justify causal modifications of PIR and MaxEnt by separating constraints into restrictions for the cause and restrictions for the mechanism that generates the effect from the cause. We further sketch why causal PIR also entails “Information Geometric Causal Inference.” We briefly discuss problems of generalizing the causal version of MaxEnt to arbitrary causal DAGs. (shrink)

Sometimes we receive evidence in a form that standard conditioning (or Jeffrey conditioning) cannot accommodate. The principle of maximum entropy (MAXENT) provides a unique solution for the posterior probability distribution based on the intuition that the information gain consistent with assumptions and evidence should be minimal. Opponents of objective methods to determine these probabilities prominently cite van Fraassen’s Judy Benjamin case to undermine the generality of maxent. This article shows that an intuitive approach to Judy Benjamin’s case supports (...) maxent. This is surprising because based on independence assumptions the anticipated result is that it would support the opponents. It also demonstrates that opponents improperly apply independence assumptions to the problem. (shrink)

Objective Bayesian epistemology invokes three norms: the strengths of our beliefs should be probabilities, they should be calibrated to our evidence of physical probabilities, and they should otherwise equivocate sufficiently between the basic propositions that we can express. The three norms are sometimes explicated by appealing to the maximum entropy principle, which says that a belief function should be a probability function, from all those that are calibrated to evidence, that has maximum entropy. However, the three (...) norms of objective Bayesianism are usually justified in different ways. In this paper we show that the three norms can all be subsumed under a single justification in terms of minimising worst-case expected loss. This, in turn, is equivalent to maximising a generalised notion of entropy. We suggest that requiring language invariance, in addition to minimising worst-case expected loss, motivates maximisation of standard entropy as opposed to maximisation of other instances of generalised entropy. Our argument also provides a qualified justification for updating degrees of belief by Bayesian conditionalisation. However, conditional probabilities play a less central part in the objective Bayesian account than they do under the subjective view of Bayesianism, leading to a reduced role for Bayes’ Theorem. (shrink)

This paper concerns the question of how to draw inferences common sensically from uncertain knowledge. Since the early work of Shore and Johnson (1980), Paris and Vencovská (1990), and Csiszár (1989), it has been known that the Maximum Entropy Inference Process is the only inference process which obeys certain common sense principles of uncertain reasoning. In this paper we consider the present status of this result and argue that within the rather narrow context in which we work this (...) complete and consistent mode of uncertain reasoning is actually characterised by the observance of just a single common sense principle (or slogan). (shrink)

This paper proposes a feature extraction algorithm based on the maximum entropy phrase reordering model in statistical machine translation in language learning machines. The algorithm can extract more accurate phrase reordering information, especially the feature information of reversed phrases, which solves the problem of imbalance of feature data during maximum entropy training in the original algorithm, and improves the accuracy of phrase reordering in translation. In the experiment, they were combined with linguistic features such as parts (...) of speech, words, and syntactic features extracted by using the syntax analyzer, and the maximum entropy classifier was used to predict translation errors, and the experimental verification was performed on the Chinese-English translation data set and compared. The experimental results show that different word posterior probabilities have a significant impact on the classification error rate, and the combination of linguistic features based on the word posterior probability can significantly reduce the classification error rate and improve the translation error prediction performance. (shrink)

Jaynes's maximum entropy principle (MEP) is analyzed by considering in detail a recent controversy. Emphasis is placed on the inductive logical interpretation of “probability” and the concept of “total knowledge.” The relation of the MEP to relative frequencies is discussed, and a possible realm of its fruitful application is noted.

The Madelung equations map the non-relativistic time-dependent Schrödinger equation into hydrodynamic equations of a virtual fluid. While the von Neumann entropy remains constant, we demonstrate that an increase of the Shannon entropy, associated with this Madelung fluid, is proportional to the expectation value of its velocity divergence. Hence, the Shannon entropy may grow due to an expansion of the Madelung fluid. These effects result from the interference between solutions of the Schrödinger equation. Growth of the Shannon (...) class='Hi'>entropy due to expansion is common in diffusive processes. However, in the latter the process is irreversible while the processes in the Madelung fluid are always reversible. The relations between interference, compressibility and variation of the Shannon entropy are then examined in several simple examples. Furthermore, we demonstrate that for classical diffusive processes, the “force” accelerating diffusion has the form of the positive gradient of the quantum Bohm potential. Expressing then the diffusion coefficient in terms of the Planck constant reveals the lower bound given by the Heisenberg uncertainty principle in terms of the product between the gas mean free path and the Brownian momentum. (shrink)

This editorial explains the scope of the special issue and provides a thematic introduction to the contributed papers.

The principle of maximum entropy is a method for assigning values to probability distributions on the basis of partial information. In usual formulations of this and related methods of inference one assumes that this partial information takes the form of a constraint on allowed probability distributions. In practical applications, however, the information consists of empirical data. A constraint rule is then employed to construct constraints on probability distributions out of these data. Usually one adopts the rule that equates (...) the expectation values of certain functions with their empirical averages. There are, however, various other ways in which one can construct constraints from empirical data, which makes the maximum entropy principle lead to very different probability assignments. This paper shows that an argument by Jaynes to justify the usual constraint rule is unsatisfactory and investigates several alternative choices. The choice of a constraint rule is also shown to be of crucial importance to the debate on the question whether there is a conflict between the methods of inference based on maximum entropy and Bayesian conditionalization. (shrink)

An important suggestion of objective Bayesians is that the maximum entropy principle can replace a principle which is known to get into paradoxical difficulties: the principle of indifference. No one has previously determined whether the maximum entropy principle is better able to solve Bertrand’s chord paradox than the principle of indifference. In this paper I show that it is not. Additionally, the course of the analysis brings to light a new paradox, a revenge paradox of the (...) chords, that is unique to the maximum entropy principle. (shrink)

To identify relationships among entities in natural language texts, extraction of entity relationships technically provides a fundamental support for knowledge graph, intelligent information retrieval, and semantic analysis, promotes the construction of knowledge bases, and improves efficiency of searching and semantic analysis. Traditional methods of relationship extraction, either those proposed at the earlier times or those based on traditional machine learning and deep learning, have focused on keeping relationships and entities in their own silos: extracting relationships and entities are conducted in (...) steps before obtaining the mappings. To address this problem, a novel Chinese relationship extraction method is proposed in this paper. Firstly, the triple is treated as an entity relation chain and can identify the entity before the relationship and predict its corresponding relationship and the entity after the relationship. Secondly, the Joint Extraction of Entity Mentions and Relations model is based on the Bidirectional Long Short-Term Memory and Maximum Entropy Markov Model. Experimental results indicate that the proposed model can achieve a precision of 79.2% which is much higher than that of traditional models. (shrink)

Kristina Toutanova Christopher D. Manning Dept of Computer Science Depts of Computer Science and Linguistics Gates Bldg 4A, 353 Serra Mall Gates Bldg 4A, 353 Serra Mall Stanford, CA 94305–9040, USA Stanford, CA 94305–9040, USA [email protected] [email protected]..

Legendre transform between thermodynamic quantities such as the Helmholtz free energy and entropy plays a key role in the formulation of the canonical ensemble. In the standard treatment, the transform exchanges the independent variable from the system’s internal energy to its conjugate variable—the inverse temperature of the heat reservoir. In this article, we formulate a microscopic version of the transform between the free energy and Shannon entropy of the system, where the conjugate variables are the microstate probabilities and (...) the energies (scaled by the inverse temperature). The present approach gives a non-conventional perspective on the connection between information-theoretic measure of entropy and thermodynamic entropy. We focus on the exact differential property of Shannon entropy, utilizing it to derive central relations within the canonical ensemble. Thermodynamics of a system in contact with the heat reservoir is discussed in this framework. Other approaches, in particular, Jaynes’ maximum entropy principle is compared with the present approach. (shrink)