This monograph offers a new foundation for information theory that is based on the notion of information-as-distinctions, being directly measured by logical entropy, and on the re-quantification as Shannon entropy, which is the fundamental concept for the theory of coding and communications. Information is based on distinctions, differences, distinguishability, and diversity. Information sets are defined that express the distinctions made by a partition, e.g., the inverse-image of a random variable so they represent (...) the pre-probability notion of information. Then logical entropy is a probability measure on the information sets, the probability that on two independent trials, a distinction or “dit” of the partition will be obtained. The formula for logical entropy is a new derivation of an old formula that goes back to the early twentieth century and has been re-derived many times in different contexts. As a probability measure, all the compound notions of joint, conditional, and mutual logical entropy are immediate. The Shannon entropy and its compound notions are then derived from a non-linear dit-to-bit transform that re-quantifies the distinctions of a random variable in terms of bits—so the Shannon entropy is the average number of binary distinctions or bits necessary to make all the distinctions of the random variable. And, using a linearization method, all the set concepts in this logical information theory naturally extend to vector spaces in general—and to Hilbert spaces in particular—for quantum logical information theory which provides the natural measure of the distinctions made in quantum measurement. Relatively short but dense in content, this work can be a reference to researchers and graduate students doing investigations in information theory, maximum entropy methods in physics, engineering, and statistics, and to all those with a special interest in a new approach to quantum information theory. (shrink)

There is a new theory of information based on logic. The definition of Shannon entropy as well as the notions on joint, conditional, and mutual entropy as defined by Shannon can all be derived by a uniform transformation from the corresponding formulas of logical information theory. Information is first defined in terms of sets of distinctions without using any probability measure. When a probability measure is introduced, the logical entropies are simply the values (...) of the probability measure on the sets of distinctions. The compound notions of joint, conditional, and mutual entropies are obtained as the values of the measure, respectively, on the union, difference, and intersection of the sets of distinctions. These compound notions of logical entropy satisfy the usual Venn diagram relationships since they are values of a measure. The uniform transformation into the formulas for Shannon entropy is linear so it explains the long-noted fact that the Shannon formulas satisfy the Venn diagram relations--as an analogy or mnemonic--since Shannon entropy is not a measure on a given set. What is the logic that gives rise to logical information theory? Partitions are dual to subsets, and the logic of partitions was recently developed in a dual/parallel relationship to the Boolean logic of subsets. Boole developed logical probability theory as the normalized counting measure on subsets. Similarly the normalized counting measure on partitions is logical entropy--when the partitions are represented as the set of distinctions that is the complement to the equivalence relation for the partition. In this manner, logical information theory provides the set-theoretic and measure-theoretic foundations for information theory. The Shannon theory is then derived by the transformation that replaces the counting of distinctions with the counting of the number of binary partitions it takes, on average, to make the same distinctions by uniquely encoding the distinct elements--which is why the Shannon theory perfectly dovetails into coding and communications theory. (shrink)

Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about distinctions, differences and distinguishability and is formalized using the distinctions of a partition. All the definitions of simple, joint, conditional and mutual entropy of Shannon information theory are derived by a uniform transformation from the corresponding definitions at the (...) logical level. The purpose of this paper is to give the direct generalization to quantum logical information theory that similarly focuses on the pairs of eigenstates distinguished by an observable, i.e., qudits of an observable. The fundamental theorem for quantum logical entropy and measurement establishes a direct quantitative connection between the increase in quantum logical entropy due to a projective measurement and the eigenstates that are distinguished by the measurement. Both the classical and quantum versions of logical entropy have simple interpretations as “two-draw” probabilities for distinctions. The conclusion is that quantum logical entropy is the simple and natural notion of information for quantum information theory focusing on the distinguishing of quantum states. (shrink)

A new axiomatic characterization with a minimum of conditions for entropy as a function on the set of states in quantum mechanics is presented. Traditionally unspoken assumptions are unveiled and replaced by proven consequences of the axioms. First the Boltzmann–Planck formula is derived. Building on this formula, using the Law of Large Numbers—a basic theorem of probability theory—the von Neumann formula is deduced. Axioms used in older theories on the foundations are now derived facts.

First comprehensive introduction to information theory explores the work of Shannon, McMillan, Feinstein, and Khinchin. Topics include the entropy concept in probability theory, fundamental theorems, and other subjects. 1957 edition.

Searching for information is critical in many situations. In medicine, for instance, careful choice of a diagnostic test can help narrow down the range of plausible diseases that the patient might have. In a probabilistic framework, test selection is often modeled by assuming that people's goal is to reduce uncertainty about possible states of the world. In cognitive science, psychology, and medical decision making, Shannon entropy is the most prominent and most widely used model to formalize probabilistic uncertainty (...) and the reduction thereof. However, a variety of alternative entropy metrics (Hartley, Quadratic, Tsallis, Rényi, and more) are popular in the social and the natural sciences, computer science, and philosophy of science. Particular entropy measures have been predominant in particular research areas, and it is often an open issue whether these divergences emerge from different theoretical and practical goals or are merely due to historical accident. Cutting across disciplinary boundaries, we show that several entropy and entropy reduction measures arise as special cases in a unified formalism, the Sharma–Mittal framework. Using mathematical results, computer simulations, and analyses of published behavioral data, we discuss four key questions: How do various entropy models relate to each other? What insights can be obtained by considering diverse entropy models within a unified framework? What is the psychological plausibility of different entropy models? What new questions and insights for research on human information acquisition follow? Our work provides several new pathways for theoretical and empirical research, reconciling apparently conflicting approaches and empirical findings within a comprehensive and unified information‐theoretic formalism. (shrink)

Information theory offers a measure of "mutual information" which provides an appropriate measure of tracking efficiency for the naturalistic epistemologist. The statistical entropy on which it is based is arguably the best way of characterizing the uncertainty associated with the behavior of a system, and it is ontologically neutral. Though not appropriate for the naturalization of meaning, mutual information can serve as a measure of epistemic success independent of semantic maps and payoff structures. While not (...) containing payoffs as terms, mutual information places both upper and lower bounds on payoffs. This constitutes a non-trivial relationship to utility. (shrink)

There are aspects of privacy theory that are analogous to quantum theory. In particular one can define distillable key and key cost in parallel to distillable entanglement and entanglement cost. We present here classical privacy theory as a particular case of information theory with adversaries, where similar general laws hold as in entanglement theory. We place the result of Renner and Wolf—that intrinsic information is lower bound for key cost—into this general formalism. Then (...) we show that the question of whether intrinsic information is equal to key cost is equivalent to the question of whether Alice and Bob can create a distribution product with Eve using I M bits of secret key. We also propose a natural analogue of relative entropy of entanglement in privacy theory and show that it is equal to the intrinsic information. We also provide a formula analogous to the entanglement of formation for classical distributions. (shrink)

– The paper explores first the postulational basis and significance of‘measures of information’in current information theory and their possible relations to physical entropy and Brillouin's‘negentropy’regarded as the negative of entropy. For some purposes, the same pattern or formal structure may be abstracted from both‘entropy’and‘information’. The paper analyzes, in the second place, the mathematical analogies which have been traced between information theory and quantum mechanics and argues that the analogies have but a (...) limited value when we come to grips with the deeper, and yet unsolved, problems of quantum theory. Lastly it is urged that the indiscriminate extensions of statistical and information‐theoretic methods, and of quantum‐mechanical analogies, e.g.‘linguistic duality’, to psycholinguistics, literary and aesthetic Gestalten, can result only in crude exaggerations. Dialectical methodology requires the continual reformulation of all scientific descriptions which have to reckon perpetually with the richness of human materials. (shrink)

Rejection is an essential part of the scholarly publishing process, acting as a filter to distinguish between robust and less credible scientific works. While rejection helps reduce entropy and increase the likelihood of disseminating useful knowledge, the process is not devoid of subjectivity. Providing more informative rejection letters and encouraging humility among editors and reviewers are essential to enhance the efficiency of knowledge production as they help ensure that valuable scientific contributions are not overlooked.

A recent study suggests that value and quantum states seem to be governed by the same underlying mechanisms. In our recent book titled "Better economics for the Earth: A lesson from quantum and information theories," specifically Chapter 5, we have proposed an informational entropy-based notion of value, drawing on the granular worldview and primary features of quantum mechanics, Shannon’s information theory, and the mindsponge theory. Specifically, the notion suggests that values are created through the interactions (...) of information. But how do information and its interactions lead to values? This research note aims to contribute to the answer to this question. (shrink)

We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and wherethey are fundamentally different. We discuss and relate the basicnotions of both theories: Shannon entropy, Kolmogorov complexity, Shannon mutual informationand Kolmogorov (``algorithmic'') mutual information. We explainhow universal coding may be viewed as a middle ground betweenthe two theories. We consider Shannon's rate distortion theory, whichquantifies useful (in a certain sense) information.We use the communication of (...) information as our guiding motif, and we explain howit relates to sequential question-answer sessions. (shrink)

William Dembski claims to have established a decision process to determine when highly unlikely events observed in the natural world are due to Intelligent Design. This article argues that, as no implementable randomness test is superior to a universal Martin-Löf test, this test should be used to replace Dembski's decision process. Furthermore, Dembski's decision process is flawed, as natural explanations are eliminated before chance. Dembski also introduces a fourth law of thermodynamics, his “law of conservation of information,” to argue (...) that information cannot increase by natural processes. However, this article, using algorithmic information theory, shows that this law is no more than the second law of thermodynamics. The article concludes that any discussions on the possibilities of design interventions in nature should be articulated in terms of the algorithmic information theory approach to randomness and its robust decision process. (shrink)

Although algorithmic information theory provides a measure of the information content of string of characters, problems of noise and noncomputability emerge. However, if pattern in a noisy string is recognized by reference to a set of similar strings, this article shows that a compressed algorithmic description of a noisy string is possible and illustrates this with some simple examples. The article also shows that algorithmic information theory can quantify the information in complex organized systems (...) where pattern is nested within pattern. (shrink)

Integrated Information Theory (IIT) is currently one of the most influential scientific theories of consciousness. Here, we focus specifically on a metaphysical aspect of the theory’s most recent version (IIT 4.0), what we may call its idealistic ontology, and its tension with a kind of realism about the external world that IIT also endorses. IIT 4.0 openly rejects the mainstream view that consciousness is generated by the brain, positing instead that consciousness is ontologically primary while the physical (...) domain is just “operational”. However, this philosophical position is presently underdeveloped and is not rigorously formulated in IIT, potentially leading to many misinterpretations and undermining its overall explanatory power. In the present paper we aim to address this issue. We argue that IIT’s idealistic ontology should be understood as a specific combination of phenomenal primitivism, reductionism regarding Φ-structures and complexes, and eliminativism about non-conscious physical entities. Having clarified this, we then focus on the problematic tension between IIT’s idealistic ontology and its simultaneous endorsement of realism, according to which there is some kind of external reality independent of our minds. After refuting three potential solutions to this theoretical tension, we propose the most plausible alternative: understanding IIT’s realism as an assertion of the existence of other experiences beyond one’s own, what we call a non-solipsistic idealist realism. We end with concluding remarks and future research avenues. (shrink)

Entropy and information are both emerging as currencies of interdisciplinary dialogue, most recently in evolutionary theory. If this dialogue is to be fruitful, there must be general agreement about the meaning of these terms. That this is not presently the case owes principally to the supposition of many information theorists that information theory has succeeded in generalizing the entropy concept. The present paper will consider the merits of the generalization thesis, and make some (...) suggestions for restricting both entropy and information to specific arenas of discourse. (shrink)

A new logic of partitions has been developed that is dual to ordinary logic when the latter is interpreted as the logic of subsets of a fixed universe rather than the logic of propositions. For a finite universe, the logic of subsets gave rise to finite probability theory by assigning to each subset its relative size as a probability. The analogous construction for the dual logic of partitions gives rise to a notion of logical entropy that is precisely (...) related to Claude Shannon's entropy. In this manner, the new logic of partitions provides a logico-conceptual foundation for information-theoretic entropy or information content. (shrink)

The consolidation of the informational paradigm in molecular biology research concluded on a system to convert the epistemic object into an operational technological object and a stable epistemic product. However, the acceptance of the informational properties of genetic acids failed to clarify the meaning of the concept of information. The “information”’ as a property of the genetic molecules remained as an informal notion that allows the description of the mechanism of inheritance, but it was not specified in a (...) logic–semantic structure. The metaphorical implications associated with the idea of genes as molecules with meaning, questioned the linguistics that seemed too foreign to molecular biology. A reformulation of the concept of information in molecular biology was developed upon the theory of Claude Shannon. The node for the structural coupling between biology, physics and information theory was the identification of an analog structure between the coded messages of Shannon’s theory. (shrink)

Categorical logic has shown that modern logic is essentially the logic of subsets (or "subobjects"). Partitions are dual to subsets so there is a dual logic of partitions where a "distinction" [an ordered pair of distinct elements (u,u′) from the universe U ] is dual to an "element". An element being in a subset is analogous to a partition π on U making a distinction, i.e., if u and u′ were in different blocks of π. Subset logic leads to finite (...) probability theory by taking the (Laplacian) probability as the normalized size of each subset-event of a finite universe. The analogous step in the logic of partitions is to assign to a partition the number of distinctions made by a partition normalized by the total number of ordered pairs |U|² from the finite universe. That yields a notion of "logical entropy" for partitions and a "logical information theory." The logical theory directly counts the (normalized) number of distinctions in a partition while Shannon's theory gives the average number of binary partitions needed to make those same distinctions. Thus the logical theory is seen as providing a conceptual underpinning for Shannon's theory based on the logical notion of "distinctions.". (shrink)

In this paper, we discuss how to conceptual engineer ‘entropy’ and ‘information’ as they are used in information theory or statistical mechanics. Initially, we evaluate the extent to which the all-pervasive entangled use of entropy and information notions can be somehow defective in these domains, such as being meaningless or generating confusion. Then, we assess the main ameliorative strategies to improve this defective conceptual practice. The first strategy is to substitute the terms ‘entropy’ (...) and ‘information’ by non-loaded terms, as it was first argued by Bar-Hillel (1955). A second strategy is to prescribe how these terms should be correctly used to be meaningful, as it was pioneered by Carnap (1952 [1977]) in Two Essays on Entropy. However, the actual implementation of these two ameliorative strategies has been historically unsuccessful due to the low credentials that philosophers as conceptual prescribers have among scientists. Finally, to try to solve these obstacles, we propose a third strategy based on leveraging evidence from the contribution of ‘Philosophy in Science’ (à la Pradeau et al. 2024) to integrate conceptual prescriptions and analyses of entropy and information as part of the scientific practices in which these notions are used. (shrink)

Drowsy driving is a major cause of automobile accidents. Previous studies used neuroimaging based approaches such as analysis of electroencephalogram (EEG) activities to understand the brain dynamics of different cortical regions during drowsy driving. However, the coupling between brain regions responding to this vigilance change is still unclear. To have a comprehensive understanding of neural mechanisms underlying drowsy driving, in this study we use transfer entropy, a model-free measure of effective connectivity based on information theory. We investigate (...) the pattern of information transfer between brain regions when the vigilance level, which is derived from the driving performance, changes from alertness to drowsiness. Results show that the couplings between pairs of frontal, central, and parietal areas increased at the intermediate level of vigilance, which suggests that an enhancement of the cortico-cortical interaction is necessary to maintain the task performance and prevent behavioral lapses. Additionally, the occipital-related connectivity magnitudes monotonically decreases as the vigilance level declines, which further supports the cortical gating of sensory stimuli during drowsiness. Neurophysiological evidence of mutual relationships between brain regions measured by Transfer entropy might elucidate the understanding of cortico-cortical communication during drowsy driving. (shrink)

Daniel R. Brooks and E. O. Wiley have proposed a theory of evolution in which fitness is merely a rate determining factor. Evolution is driven by non-equilibrium processes which increase the entropy and information content of species together. Evolution can occur without environmental selection, since increased complexity and organization result from the likely capture at the species level of random variations produced at the chemical level. Speciation can occur as the result of variation within the species which (...) decreases the probability of sharing genetic information. Critics of the Brooks-Wiley theory argue that they have abused terminology from information theory and t thermodynamics. In this paper I review the essentials of the theory, and give an account of hierarchical physical information systems within which the theory can be interpreted. I then show how the major conceptual objections can be answered. (shrink)

We review some techniques and notions for quantum information theory. It is shown that the dynamical entropies is discussed and some numerical computations of these entropies are carried for several states.

Integrating concepts of maintenance and of origins is essential to explaining biological diversity. The unified theory of evolution attempts to find a common theme linking production rules inherent in biological systems, explaining the origin of biological order as a manifestation of the flow of energy and the flow of information on various spatial and temporal scales, with the recognition that natural selection is an evolutionarily relevant process. Biological systems persist in space and time by transfor ming energy from (...) one state to another in a manner that generates structures which allows the system to continue to persist. Two classes of energetic transformations allow this; heat-generating transformations, resulting in a net loss of energy from the system, and conservative transformations, changing unusable energy into states that can be stored and used subsequently. All conservative transformations in biological systems are coupled with heat-generating transformations; hence, inherent biological production, or genealogical proesses, is positively entropic. There is a self-organizing phenomenology common to genealogical phenomena, which imparts an arrow of time to biological systems. Natural selection, which by itself is time-reversible, contributes to the organization of the self-organized genealogical trajectories. The interplay of genealogical (diversity-promoting) and selective (diversity-limiting) processes produces biological order to which the primary contribution is genealogical history. Dynamic changes occuring on times scales shorter than speciation rates are microevolutionary; those occuring on time scales longer than speciation rates are macroevolutionary. Macroevolutionary processes are neither redicible to, nor autonomous from, microevolutionary processes. (shrink)

The objective of the consistent-amplitude approach to quantum theory has been to justify the mathematical formalism on the basis of three main assumptions: the first defines the subject matter, the second introduces amplitudes as the tools for quantitative reasoning, and the third is an interpretative rule that provides the link to the prediction of experimental outcomes. In this work we introduce a natural and compelling fourth assumption: if there is no reason to prefer one region of the configuration space (...) over another, then they should be “weighted” equally. This is the last ingredient necessary to introduce a unique inner product in the linear space of wave functions. Thus, a form of the principle of insufficient reason is implicit in the Hilbert inner product. Armed with the inner product we obtain two results. First, we elaborate on an earlier proof of the Born probability rule. The implicit appeal to insufficient reason shows that quantum probabilities are not more objective than classical probabilities. Previously we had argued that the consistent manipulation of amplitudes leads to a linear time evolution; our second result is that time evolution must also be unitary. The argument is straightforward and hinges on the conservation of entropy. The only subtlety consists of defining the correct entropy; it is the array entropy, not von Neumann's. After unitary evolution has been established we proceed to introduce the useful notion of observables and we explore how von Neumann's entropy can be linked to Shannon's information theory. Finally, we discuss how various connections among the postulates of quantum theory are made explicit within this approach. (shrink)

Entropy is ubiquitous in physics, and it plays important roles in numerous other disciplines ranging from logic and statistics to biology and economics. However, a closer look reveals a complicated picture: entropy is defined differently in different contexts, and even within the same domain different notions of entropy are at work. Some of these are defined in terms of probabilities, others are not. The aim of this chapter is to arrive at an understanding of some of the (...) most important notions of entropy and to clarify the relations between them, After setting the stage by introducing the thermodynamic entropy, we discuss notions of entropy in information theory, statistical mechanics, dynamical systems theory and fractal geometry. (shrink)

To formalize the human judgment of rhythm complexity, we used five measures from information theory and algorithmic complexity to measure the complexity of 48 artificially generated rhythmic sequences. We compared these measurements to human prediction accuracy and easiness judgments obtained from a listening experiment, in which 32 participants guessed the last beat of each sequence. We also investigated the modulating effects of musical expertise and general pattern identification ability. Entropy rate and Kolmogorov complexity were correlated with prediction (...) accuracy, and highly correlated with easiness judgments. A logistic regression showed main effects of musical training, entropy rate, and Kolmogorov complexity, and an interaction between musical training and both entropy rate and Kolmogorov complexity. These results indicate that information-theoretic concepts capture some salient features of the human judgment of rhythm complexity, and they confirm the influence of musical expertise on complexity judgments. (shrink)

The information age can be dated to the work of Norbert Wiener and Claude Shannon in the 1940s. Their work on cybernetics and information theory, and many subsequent developments, had a profound influence on reshaping the field of psychology from what it was prior to the 1950s. Contemporaneously, advances also occurred in experimental design and inferential statistical testing stemming from the work of Ronald Fisher, Jerzy Neyman, and Egon Pearson. These interdisciplinary advances from outside of psychology provided (...) the conceptual and methodological tools for what is often called the cognitive revolution but is more accurately described as the information-processing revolution. Cybernetics set the stage with the idea that everything ranging from neurophysiological mechanisms to societal activities can be modeled as structured control systems with feedforward and feedback loops. Information theory offered a way to quantify entropy and information, and promoted theorizing in terms of information flow. Statistical theory provided means for making scientific inferences from the results of controlled experiments and for conceptualizing human decision making. With those three pillars, a cognitive psychology adapted to the information age evolved. The growth of technology in the information age has resulted in human lives being increasingly interweaved with the cyber environment, making cognitive psychology an essential part of interdisciplinary research on such interweaving. Continued engagement in interdisciplinary research at the forefront of technology development provides a chance for psychologists not only to refine their theories but also to play a major role in the advent of a new age of science. (shrink)

In a previous paper a theory of program size formally identical to information theory was developed. The entropy of an individual finite object was defined to be the size in bits of the smallest program for calculating it. It was shown that this is − log2 of the probability that the object is obtained by means of a program whose successive bits are chosen by flipping an unbiased coin. Here a theory of the entropy (...) of recursively enumerable sets of objects is proposed which includes the previous theory as the special case of sets having a single element. The primary concept in the generalized theory is the probability that a computing machine enumerates a given set when its program is manufactured by coin flipping. The entropy of a set is defined to be − log2 of this probability. (shrink)

Information theory presupposes the notion of an epistemic agent, such as a scientist or an idealized human. Despite that, information theory is increasingly invoked by physicists concerned with fundamental physics, physics at very high energies, or generally with the physics of situations in which even idealized epistemic agents cannot exist. In this paper, I shall try to determine the extent to which the application of information theory in those contexts is legitimate. I will illustrate (...) my considerations using the case of black hole thermodynamics and Bekenstein's celebrated argument for his formula for the entropy of black holes. This example is particularly pertinent to the theme of the present collection because it is widely accepted as `empirical data' in notoriously empirically deprived quantum gravity, even though the laws of black hole thermodynamics have so far evaded direct empirical confirmation. (shrink)

The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the partition. The logical concept of entropy based on partition logic is the normalized counting measure of the set of distinctions of a partition on a finite set--just as the usual logical notion of probability based on (...) the Boolean logic of subsets is the normalized counting measure of the subsets (events). Thus logical entropy is a measure on the set of ordered pairs, and all the compound notions of entropy (join entropy, conditional entropy, and mutual information) arise in the usual way from the measure (e.g., the inclusion-exclusion principle)--just like the corresponding notions of probability. The usual Shannon entropy of a partition is developed by replacing the normalized count of distinctions (dits) by the average number of binary partitions (bits) necessary to make all the distinctions of the partition. (shrink)

Like all other inventions, advances in the field of digital computational technologies, which we will briefly describe as the information world, have also played an essential role in humanity life. These advances have brought some ethical debates to our individual and social life, as well as the industrial benefit obtained by the digital and analog technological developments that positively or negatively affect and transform all economic and cultural paradigms surrounding human life. The branch of the philosophy of information, (...) which questions the basic issues and discussions of the information age through philosophical paradigms, deals with the ontological, epistemological, and axiological problems of the information age. The subject deals with the analysis and new results of an Aristotelian information ethics theory, which is a theory first proposed to the world of thought in the field of the philosophy of information. The main reason for dealing with this subject in the thesis is to examine the scientific, technological, and philosophical bases of the information revolution. (shrink)

This book is about an important issue which has arisen within two of the branches of physical science - namely thermodynamics and statistical mechanics - where the notion of entropy plays an essential role. A number of scientists and information theorists have maintained that entropy is a subjective concept and is a measure of human ignorance. Such a view, if it is valid, would create some profound philosophical problems and would tend to undermine the objectivity of the (...) scientific enterprise. Whilst the present volume is not a treatise on thermodynamics or statistical mechanics, all relevant steps in the building up of these disciplines are carefully scrutinised and it is concluded that the charge of subjectivity cannot be upheld. The widely adopted view that entropy is a measure of disorder, or of lack of information, is shown to be ambiguous, although it may be of use in certain contexts. (shrink)

Numerous studies have focused on the understanding of rapid automatized naming, which can be applied to predict reading abilities and developmental dyslexia in children. Eye tracking technique, characterizing the essential ocular activities, might have the feasibility to reveal the visual and cognitive features of RAN. However, traditional measures of eye movements ignore many dynamical details about the visual and cognitive processing of RAN, and are usually associated with the duration of time spent on some particular areas of interest, fixation counts, (...) revisited fixation counts, saccadic velocities, or saccadic amplitudes. To cope with this drawback, we suggested an entropy-based method to measure eye movements for the first time, which first mapped eye movements during RAN in a time-series and then analyzed the time-series by a proper definition of entropy from the perspective of information theory. Our findings showed that the entropy was more sensitive to reflect small perturbation of eye movements, and thus gained better performance than traditional measures. We also verified that the entropy of eye movements significantly deceased with the age and the task complexity of RAN, and significantly correlated with traditional eye-movement measures [e.g., total time of naming ] and the RAN-related skills [e.g., selective attention, cognitive speed, and visual-motor integration]. Our findings may bring some new insights into the understanding of both RAN and eye tracking technique itself. (shrink)

In the last 25 years, the concept of information has played a crucial role in communication theory, so much so that the terms information theory and communication theory are sometimes used almost interchangeably. It seems to us, however, that the notion of information is also destined to render valuable services to the student of induction and probability, of learning and reinforcement, of semantic meaning and deductive inference, as~well as of scientific method in general. The (...) present volume is an attempt to illustrate some of these uses of information concepts. In 'On Semantic Information' Hintikka summarizes some of his and his associates' recent work on information and induction, and comments briefly on its philosophical suggestions. Jamison surveys from the sub jectivistic point of view some recent results in 'Bayesian Information Usage'. Rosenkrantz analyzes the information obtained by experimen tation from the Bayesian and Neyman-Pearson standpoints, and also from the standpoint of entropy and related concepts. The much-debated principle of total evidence prompts Hilpinen to examine the problem of measuring the information yield of observations in his paper 'On the Information Provided by Observations'. Pietarinen addresses himself to the more general task of evaluating the systematizing power of hypotheses and theories, a task which quickly leads him to information concepts. Domotor develops a qualitative theory of information and entropy. His paper gives what is probably the first axiomatization of a general qualitative theory of information adequate to guarantee a numerical representation of the standard sort. (shrink)

In many applications of information theory, information measures the reduction of uncertainty that results from the knowledge that an event has occurred. Even so, an item of information learned need not be the occurrence of an event but, rather, the change in probability distribution associated with an ensemble of events. This paper examines the basic account of information, which focuses on events, and reviews how it may be naturally generalized to probability distributions/measures. The resulting (...) class='Hi'>information measure is special case of the Rényi information divergence (also known as the Rényi entropy). This information measure, herein dubbed the variational information, meaningfully assigns a numerical bit-value to arbitrary state transitions of physical systems. The information topology of these state transitions is characterized canonically by a right and left continuity spectrum defined in terms of the Kantorovich- Wasserstein metric. These continuity spectra provide a theoretical framework for characterizing the informational continuity of evolving systems and for rigorously assessing the degree to which such systems exhibit, or fail to exhibit, continuous change. (shrink)

We review some notions for general quantum entropies. The entropy of the compound systems is discussed and a numerical computation of the quantum dynamical systems is carried for the noisy optical channel.

We present an analysis of Szilard's one-molecule Maxwell's demon, including a detailed entropy accounting, that suggests a general theory of the entropy cost of information. It is shown that the entropy of the demon increases during the expansion step, due to the decoupling of the molecule from the measurement information. It is also shown that there is an entropy symmetry between the measurement and erasure steps, whereby the two steps additivelv share a constant (...) entropy change, but the proportion that occurs during each of the two steps is arbitrary. Therefore the measurement step may be accompanied by an entropy increase, a decrease, or no change at all, and likewise for the erasure step. Generalizing beyond the demon, decorrelation between a physical system and information about that system always causes an entropy increase in the joint system comprised of both the original system and the information. Decorrelation causes a net entropy increase in the universe unless, as in the Szilard demon, the information is used to decrease entropy elsewhere before the correlation is lost. Thus, information is thermodynamically costly precisely to the extent that it is not used to obtain work from the measured system. (shrink)

In various publications over the past years, Floridi has developed a theory of semantic information as well-formed, meaningful, and truthful data. This theory is more or less orthogonal to the standard entropy-based notions of information known from physics, information theory, and computer science that all define the amount of information in a certain system as a scalar value without any direct semantic implication. In this context the question rises what the exact relation (...) between these various conceptions of information is and whether there is a real need to enrich these mathematically more or less rigid definitions with a less formal notion of semantic information. I investigate various philosophical aspects of the more formal definitions of information in the light of Floridi’s theory. The position I defend is that the formal treatment of the notion of information as a general theory of entropy is one of the fundamental achievements of modern science that in itself is a rich source for new philosophical reflection. This makes information theory a competitor of classical epistemology rather than a servant. In this light Floridi’s philosophy of information is more a reprise of classical epistemology that only pays lip service to information theory but fails to address the important central questions of philosophy of information. Specifically, I will defend the view that notions that are associated with truth, knowledge, and meaning all can adequately be reconstructed in the context of modern information theory and that consequently there is no need to introduce a concept of semantic information. (shrink)

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result 1 provides a restricted equivalence between the two: where the Bayesian model for MAXENT inference uses an "a priori" probability that is uniform, and where all MAXENT constraints are limited to 0-1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result 2 established a (...) sensitivity of MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacian principle of Insufficient Reason also plague MAXENT theory. Result 3 builds upon the findings of Friedman and Shimony (1971; 1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes's (1978) prescription for his (1963) Brandeis Dice problem. Lastly, Result 4 generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events. (shrink)

Part I of the present work outlined the rigorous application of information theory to a quantum mechanical system in a thermodynamic equilibrium state. The general formula developed there for the best-guess density operator $\hat \rho$ was indeterminate because it involved in an essential way an unspecified prior probability distribution over the continuumD H of strong equilibrium density operators. In Part II mathematical evaluation of $\hat \rho$ is completed after an epistemological analysis which leads first to the discretization ofD (...) H and then to the adoption of a suitable indifference axiom to delimit the set of admissible prior distributions. Finally, quantal formulas for information-theoretic and thermodynamic entropies are contrasted, and the entire work is summarized. (shrink)

[Accepted for publication in Lakatos's Undone Work: The Practical Turn and the Division of Philosophy of Mathematics and Philosophy of Science, special issue of Kriterion: Journal of Philosophy. Edited by S. Nagler, H. Pilin, and D. Sarikaya.] Lakatos’s analysis of progress and degeneration in the Methodology of Scientific Research Programmes is well-known. Less known, however, are his thoughts on degeneration in Proofs and Refutations. I propose and motivate two new criteria for degeneration based on the discussion in Proofs and Refutations (...) – superfluity and authoritarianism. I show how these criteria augment the account in Methodology of Scientific Research Programmes, providing a generalized Lakatosian account of progress and degeneration. I then apply this generalized account to a key transition point in the history of entropy – the transition to an information-theoretic interpretation of entropy – by assessing Jaynes’s 1957 paper on information theory and statistical mechanics. (shrink)

This paper challenges the frequent demonisation of entropy in discourses which attempt to draw a “naturalised” axiology from thermodynamics, information theory, and related sciences. Such discourses include Wiener’s cybernetics, Stiegler’s negathropology, and Floridi’s information ethics, in each of which entropy designates the evil which must be fought in the name of life, information, or some other notion of “the good.” The perspective the paper develops is Nietzschean. Nietzsche himself rejected the consequences of the Second (...) Law, but I wish to argue that it is possible to affirm entropy, for Nietzschean reasons. First, the paper argues that the reason Nietzsche rejected the Second Law is that it provides consolation for the pessimist (an argument made by von Hartmann). Eternal return should be affirmed because it is the more difficult position, and so provides the ultimate existential test. However, metaphysical and existential reasons must give way to the more recent scientific evidence, especially the dating of the universe, which undermines Nietzsche’s argument against heat death. While this is alone sufficient reason to affirm entropy, the position is supported by two further classes of reasons. First, the oppositions which whave supported the traditional ascription of values to negentropy and entropy can be challenged; and 2) entropy can be seen as consonant with the characteristics of existence which Nietzsche sought to affirm, especially becoming. (shrink)

Progress has become a suspect concept in evolutionary biology, not the least because the core concepts of neo-Darwinism do not support the idea that evolution is progressive. There have been a number of attempts to account for directionality in evolution through additions to the core hypotheses of neo-Darwinism, but they do not establish progressiveness, and they are somewhat of an ad hoc collection. The standard account of fitness and adaptation can be rephrased in terms of information theory. From (...) this, an information of adaptation can be defined in terms of a fitness function. The information of adaptation is a measure of the mutual information between biota and their environment. If the actual state of adaptation lags behind the state of optimal adaptation, then it is possible for the information of adaptation to increase indefinitely. Since adaptations are functional, this suggests the possibility of progressive evolution in the sense of increasing adaptation. Keywords: evolution, information, adaptation, fitness, entropy, progress.. (shrink)

The notion of a partition on a set is mathematically dual to the notion of a subset of a set, so there is a logic of partitions dual to Boole's logic of subsets (Boolean logic is usually mis-specified as "propositional" logic). The notion of an element of a subset has as its dual the notion of a distinction of a partition (a pair of elements in different blocks). Boole developed finite logical probability as the normalized counting measure on elements of (...) subsets so there is a dual concept of logical entropy which is the normalized counting measure on distinctions of partitions. Thus the logical notion of information is a measure of distinctions. Classical logical entropy naturally extends to the notion of quantum logical entropy which provides a more natural and informative alternative to the usual Von Neumann entropy in quantum information theory. The quantum logical entropy of a post-measurement density matrix has the simple interpretation as the probability that two independent measurements of the same state using the same observable will have different results. The main result of the paper is that the increase in quantum logical entropy due to a projective measurement of a pure state is the sum of the absolute squares of the off-diagonal entries ("coherences") of the pure state density matrix that are zeroed ("decohered") by the measurement, i.e., the measure of the distinctions ("decoherences") created by the measurement. (shrink)

The Kolmogorov-Sinai entropy is a fairly exotic mathematical concept which has recently aroused some interest on the philosophers’ part. The most salient trait of this concept is its working as a junction between such diverse ambits as statistical mechanics, information theory and algorithm theory. In this paper I argue that, in order to understand this very special feature of the Kolmogorov-Sinai entropy, is essential to reconstruct its genealogy. Somewhat surprisingly, this story takes us as far (...) back as the beginning of celestial mechanics and through some of the most exciting developments of mathematical physics of the 19th century. (shrink)

Pour étendre la notion d'entropie, l'auteur tente de dégager la signification de la “probabilité relative” w de Boltzmann dans sa formule S = k.log.w. Celui-ci introduit implicitement une partition d'un ensemble E de molécules en classes d'équivalence, les divers états macroscopiques du gaz; w est le rapport de la probabilité de l'état dont on cherche à définir l'entropie à celle de l'état le plus improbable.L'auteur propose de généraliser le concept toutes les fois ob sur un ensemble probabilisable E il sera (...) possible de faire une partition.La “complexité d'une structure” peut être définie par le nombre d'aspects différents que peut prendre le graphe sous-jacent à celle-ci. Alors toute stricture a une probabilité d'occurrence, et l'on peut y définir une G-entropie.Appliquant ensuite ce concept à la théorie de l'information de Shannon, l'auteur montre que la quantité d'information apportée par un message est différente de la G-entropie, mais que les deux grandeurs y prennent fortuitement la même valeur. Le concept permet également d'introduire une information liée au message, une autre au “code”, et une information totale.Le concept d'émergence permet ensuite de comprendre la raison pour laquelle celle-ci s'accompagne d'une augmentation de G-neguentropie.L'entropie est considérée depuis Boltzmann comme une borne mesure du désordre d'un gaz, d'où son intérêt en morphogénèse, en théorie des systèmes, et de l'information. Cependant son application directe pose des problèmes, et il parait préférable de tenter auparavant de généraliser le concept en le randant plus abstrait. La première difficulté à laquelle on se heurte alors est l'interprétation du nombre w de la formule de Boltzmann. (shrink)

Michel Serres’s philosophy of entropy takes what he famously calls the ‘Northwest Passage’ between the sciences and the humanities. By contextualizing his approach to entropy and affirming the role of a philosophy of difference, this paper explores Serres’s approach by means of ‘entropic differences’. It claims that entropy – or rather, entropies – provide Serres with a paradigmatic case for critical translations between different domains of knowledge. From his early Hermès series, through to The Birth of Physics (...) and later writings on social and ethical themes, he keeps thermodynamical and informational – or ‘hard’ and ‘soft’ – understandings of entropy apart, while simultaneously exploring their relation. By focusing on the systematic significance of Serres’s ‘entropic difference’, this paper shows how it unfolds not necessarily as an ontological difference but as an operative function between the history and philosophy of science, epistemology, and a theory of negentropic (inter)subjectivity. (shrink)