Information theory and statistical mechanics.

A classic source for understanding the connections between information theory and physics, this text was written by one of the giants of 20th-century physics and is appropriate for upper-level undergraduates and graduate students. Topics include the principles of coding, coding problems and solutions, the analysis of signals, a summary of thermodynamics, thermal agitation and Brownian motion, and thermal noise in an electric circuit. A discussion of the negentropy principle of information introduces the author's renowned examination of Maxwell's (...) demon. Concluding chapters explore the associations between information theory, the uncertainty principle, and physical limits of observation, in addition to problems related to computing, organizing information, and inevitable errors. 1962 ed. 81 figures. 14 tables. (shrink)

The challenges raised in this article are not with information theory per se, but the assumptions surrounding it. Neuroscience isn't sufficiently critical about the appropriate ‘receiver’ or ‘channel’, focuses on decoding ‘parts’, and often relies on a flawed ‘veridicality’ assumption. If these problematic assumptions were questioned, information theory could be better directed to help us understand how the brain works.

This paper presents Integrated Information Theory (IIT) 4.0. IIT aims to account for the properties of experience in physical (operational) terms. It identifies the essential properties of experience (axioms), infers the necessary and sufficient properties that its substrate must satisfy (postulates), and expresses them in mathematical terms. In principle, the postulates can be applied to any system of units in a state to determine whether it is conscious, to what degree, and in what way. IIT offers a parsimonious (...) explanation of empirical evidence, makes testable predictions, and permits inferences and extrapolations. IIT 4.0 incorporates several developments of the past ten years, including a more accurate translation of axioms into postulates and mathematical expressions, the introduction of a unique measure of intrinsic information that is consistent with the postulates, and an explicit assessment of causal relations. By fully unfolding a system's irreducible cause-effect power, the distinctions and relations specified by a substrate can account for the quality of experience. (shrink)

First comprehensive introduction to information theory explores the work of Shannon, McMillan, Feinstein, and Khinchin. Topics include the entropy concept in probability theory, fundamental theorems, and other subjects. 1957 edition.

Tononi et al. claim that their integrated information theory of consciousness makes testable predictions. This article discusses two of the more startling predictions, which follow from the theory’s claim that conscious experiences are generated by inactive as well as active neurons. The first prediction is that a subject’s conscious experience at a time can be affected by the disabling of neurons that were already inactive at that time. The second is that even if a subject’s entire brain (...) is “silent,” meaning that all of its neurons are inactive (but not disabled), the subject can still have a conscious experience. A few authors have noted the implausibility of these predictions—which I call the disabling prediction and the silent brain prediction—but none have considered whether they are testable. In this article, I argue that they are not. In order to make this case, I first try to clarify the distinction between active, inactive (i.e. silent), and inactivated (i.e. disabled) neurons. With this clarification in place, I show that, even putting aside practical difficulties, it is impossible to set up a valid test of either the disabling prediction or the silent brain prediction. The conditions of the tests themselves are conditions under which a response from the subject could not reasonably be interpreted as evidence of consciousness or change in consciousness. (shrink)

Integrated Information Theory (IIT) identifies consciousness with having a maximum amount of integrated information. But a thing’s having the maximum amount of anything cannot be intrinsic to it, for that depends on how that thing compares to certain other things. IIT’s consciousness, then, is not intrinsic. A mereological argument elaborates this consequence: IIT implies that one physical system can be conscious while a physical duplicate of it is not conscious. Thus, by a common and reasonable conception of (...) intrinsicality, IIT’s consciousness is not intrinsic. It is then argued that to avoid the implication that consciousness is not intrinsic, IIT must abandon its Exclusion Postulate, which prohibits overlapping conscious systems. Indeed, theories of consciousness that attribute consciousness to physical systems, should embrace the view that some conscious systems overlap. A discussion of the admittedly counterintuitive nature of this solution, along with some medical and neuroscientific realities that would seem to support it, is included. (shrink)

This book constitutes the refereed proceedings of the International Conference on Spatial Information Theory, COSIT 2005, held in Elliottville, NY, USA in September 2005. The 30 revised full papers presented were carefully reviewed and selected from 82 submissions. The papers are organized in topical sections on vagueness, uncertainty, and gradation; paths and routes; ontologies and semantics; ontologies and spatial relations; spatial reasoning: cognitive maps and spatial reasoning; time, change, and dynamics; landmarks and navigation; geographic information, and spatial (...) behaviour. (shrink)

This paper argues that Information Theoretic Redundancy (ITR) is fundamentally a composite concept that has been continually misinterpreted since the very inception of Information Theory. We view ITR as compounded of true redundancy and partial redundancy. This demarcation of true redundancy illustrates a limiting case phenomenon: the underlying metric (number of alternatives) differs only by degree but the properties of this concept differ in kind from those of partial redundancy. Several other studies are instanced which also imply (...) the composite nature of ITR. We thus provide broadly based but particular support for earlier generalized suggestions that it is the underlying calculus of Information Theory rather than the ill-named concepts themselves that provides something of a unitary language for the description of phenomena. (shrink)

Quantum Information Theory and the Foundations of Quantum Mechanics is a conceptual analysis of one of the most prominent and exciting new areas of physics, providing the first full-length philosophical treatment of quantum information theory and the questions it raises for our understanding of the quantum world. -/- Beginning from a careful, revisionary, analysis of the concepts of information in the everyday and classical information-theory settings, Christopher G. Timpson argues for an ontologically deflationary (...) account of the nature of quantum information. Against what many have supposed, quantum information can be clearly defined (it is not a primitive or vague notion) but it is not part of the material contents of the world. Timpson's account sheds light on the nature of nonlocality and information flow in the presence of entanglement and, in particular, dissolves puzzles surrounding the remarkable process of quantum teleportation. In addition it permits a clear view of what the ontological and methodological lessons provided by quantum information theory are; lessons which bear on the gripping question of what role a concept like information has to play in fundamental physics. Topics discussed include the slogan 'Information is Physical', the prospects for an informational immaterialism (the view that information rather than matter might fundamentally constitute the world), and the status of the Church-Turing hypothesis in light of quantum computation. -/- With a clear grasp of the concept of information in hand, Timpson turns his attention to the pressing question of whether advances in quantum information theory pave the way for the resolution of the traditional conceptual problems of quantum mechanics: the deep problems which loom over measurement, nonlocality and the general nature of quantum ontology. He marks out a number of common pitfalls to be avoided before analysing in detail some concrete proposals, including the radical quantum Bayesian programme of Caves, Fuchs, and Schack. One central moral which is drawn is that, for all the interest that the quantum information-inspired approaches hold, no cheap resolutions to the traditional problems of quantum mechanics are to be had. (shrink)

Integrated Information Theory posits a new kind of information, which, given certain constraints, constitutes consciousness. Searle objects to IIT because its appeal to information relies on observer-relative features. This misses the point that IIT’s notion of integrated information is intrinsic, the opposite of observer-relative. Moreover, Searle overlooks the possibility that IIT could be embraced as an extension of his theory. While he insists that causal powers of the brain account for consciousness, he maintains that (...) these causal powers aren’t tied to protoplasmic material. Whatever these causal powers are, they don’t consist in mere information-processing or computation. IIT agrees, and also positively characterizes the relevant causal powers as those involved in generating integrated information. Examining the further commitments of each theory reveals that IIT renews a fundamentally ontological challenge to information-processing and computational theories of mind. (shrink)

– The paper explores first the postulational basis and significance of‘measures of information’in current information theory and their possible relations to physical entropy and Brillouin's‘negentropy’regarded as the negative of entropy. For some purposes, the same pattern or formal structure may be abstracted from both‘entropy’and‘information’. The paper analyzes, in the second place, the mathematical analogies which have been traced between information theory and quantum mechanics and argues that the analogies have but a limited value when (...) we come to grips with the deeper, and yet unsolved, problems of quantum theory. Lastly it is urged that the indiscriminate extensions of statistical and information‐theoretic methods, and of quantum‐mechanical analogies, e.g.‘linguistic duality’, to psycholinguistics, literary and aesthetic Gestalten, can result only in crude exaggerations. Dialectical methodology requires the continual reformulation of all scientific descriptions which have to reckon perpetually with the richness of human materials. (shrink)

Although algorithmic information theory provides a measure of the information content of string of characters, problems of noise and noncomputability emerge. However, if pattern in a noisy string is recognized by reference to a set of similar strings, this article shows that a compressed algorithmic description of a noisy string is possible and illustrates this with some simple examples. The article also shows that algorithmic information theory can quantify the information in complex organized systems (...) where pattern is nested within pattern. (shrink)

Informational theories of semantic content have been recently gaining prominence in the debate on the notion of mental representation. In this paper we examine new-wave informational theories which have a special focus on cognitive science. In particular, we argue that these theories face four important difficulties: they do not fully solve the problem of error, fall prey to the wrong distality attribution problem, have serious difficulties accounting for ambiguous and redundant representations and fail to deliver a metasemantic theory of (...) representation. Furthermore, we argue that these difficulties derive from their exclusive reliance on the notion of information, so we suggest that pure informational accounts should be complemented with functional approaches. (shrink)

Integrated Information Theory (IIT) stands out as one of the most promising approaches to scientifically understand the emergence of consciousness. Even if it borrows from the phenomenology of consciousness to derive its axiomatic formulation, IIT does not initially adhere to any particular ontological position. However, its founder leans towards Panpsychism. More recently, Owen has studied the pros and cons of different ontologies as a metaphysical basis for IIT, defending a hylomorphic stance where en-grounding, en-forming relations gain the upper (...) hand in the causal narrative. This paper aims at (1) complementing Owen’s work by invoking additional arguments against physicalist IIT, both reductive and non-reductive; and (2) improving Owen’s characterization of non-physicalist causality through a “nested” version of hylomorphism that avoids the combination problem of Panpsychism. (shrink)

This paper is a criticism of full-information theories of welfare. Such theories unsuccessfully attempt to accommodate an internalist intuition (that one's good depends in some way on one's desires or hypothetical desire) with a rationalist intuition (that only fully-informed desires are relevant to one's good).

It is suggested that quantum mechanics is not fundamental but emerges from classical information theory applied to causal horizons. The path integral quantization and quantum randomness can be derived by considering information loss of fields or particles crossing Rindler horizons for accelerating observers. This implies that information is one of the fundamental roots of all physical phenomena. The connection between this theory and Verlinde’s entropic gravity theory is also investigated.

Information theory and statistical mechanics II.

The influence of degree of organization on the ability of Ss to recall lists of syllables immediately after learning was used as a measure in applying the concept of information to the problem of learning. More syllables were correctly recalled from a passage with a lower average rate of information than from a passage with a higher average information rate. The amount of information learned by the Ss was constant when the degree of organization was between (...) 2 and 1.5 bits per syllable. (shrink)

We present a critical discussion of the claim (most forcefully propounded by Chaitin) that algorithmic information theory sheds new light on Godel's first incompleteness theorem.

Tononi has proposed a fundamental theory of consciousness he terms Integrated Information Theory (IIT). IIT purports to explain the quantity of conscious experience by linking it with integrated information: information shared by the system as a whole and quantified by adopting a modified version of Shannon's definition of information. Since the fundamental aspect of IIT is information the theory allows for the multiple realizability of consciousness. While there are several concepts within IIT (...) that need further theoretical development, the main failings of the theory are an absence of a link between conscious experience and awareness and the use of Shannon's limited data based definition of information. These limitations prevent the theory from satisfying Chalmers' principles of structural coherence and organizational invariance which any functionalist theory should obey. It is not clear whether IIT can be repaired without simply reducing the theory to a general statement of computational functionalism. (shrink)

Integrated Information Theory (IIT) is one of the most influential theories of consciousness, mainly due to its claim of mathematically formalizing consciousness in a measurable way. However, the theory, as it is formulated, does not account for contextual observations that are crucial for understanding consciousness. Here we put forth three possible difficulties for its current version, which could be interpreted as a trilemma. Either consciousness is contextual or not. If contextual, either IIT needs revisions to its axioms (...) to include contextuality, or it is inconsistent. If consciousness is not contextual, then IIT faces an empirical challenge. Therefore, we argue that IIT in its current version is inadequate. (shrink)

The Integrated Information Theory is a leading scientific theory of consciousness, which implies a kind of panpsychism. In this paper, I consider whether IIT is compatible with a particular kind of panpsychism, known as Russellian panpsychism, which purports to avoid the main problems of both physicalism and dualism. I will first show that if IIT were compatible with Russellian panpsychism, it would contribute to solving Russellian panpsychism’s combination problem, which threatens to show that the view does not (...) avoid the main problems of physicalism and dualism after all. I then show that the theories are not compatible as they currently stand, in view of what I call the coarse-graining problem. After I explain the coarse-graining problem, I will offer two possible solutions, each involving a small modification of IIT. Given either of these modifications, IIT and Russellian panpsychism may be fully compatible after all, and jointly enable significant progress on the mind–body problem. (shrink)

Integrated information theory is a theory of consciousness that was originally formulated, and is standardly still expressed, in terms of controversial interpretations of its own ontological and epistemological basis. These form the orthodox interpretation of IIT. The orthodox epistemological interpretation is the axiomatic method, whereby IIT is ultimately derived from, justified by, and beholden to a set of phenomenological axioms. The orthodox ontological interpretation is panpsychism, according to which consciousness is fundamental, intrinsic, and pervasive. In this paper (...) it is argued that both components of the orthodox interpretation should be rejected. But IIT should not be rejected since an interpretation-neutral formulation is available. After explaining the neutral formulation, more plausible non-axiomatic epistemologies are defended. The neutral formulation is then shown to be consistent with various contemporary physicalist ontologies of consciousness, including the phenomenal concept strategy, representationalism, and even illusionism. Along the way, instructive connections between interpretations of IIT and interpretations of quantum mechanics are noted. (shrink)

In this paper I argue that the Integrated Information Theory of Consciousness has an underlying emergentist metaphysics, specifically of a kind that has received minimal attention and we may call functionalist emergentism. I will try to show that in this scientific theory conscious experience is a functional-role property possessed by the whole system, not by their parts, which is dependent on, but also (purportedly) causally powerful over and above, the properties of the parts. However, I will argue (...) that depicting conscious experience as a functional-role emergent property threatens the whole coherence of the theory because, by definition, functional-realizers do all the causal work associated with the instantiation of any functional-role property on any occasion. Hence, to preserve the causal power of consciousness, which, as we will see, is one of the fundamental building blocks of the theory, linked to the assertion of the very existence of consciousness, phenomenal properties should be re-thought as realizers of the relevant informationally integrated causal-roles. (shrink)

There are aspects of privacy theory that are analogous to quantum theory. In particular one can define distillable key and key cost in parallel to distillable entanglement and entanglement cost. We present here classical privacy theory as a particular case of information theory with adversaries, where similar general laws hold as in entanglement theory. We place the result of Renner and Wolf—that intrinsic information is lower bound for key cost—into this general formalism. Then (...) we show that the question of whether intrinsic information is equal to key cost is equivalent to the question of whether Alice and Bob can create a distribution product with Eve using I M bits of secret key. We also propose a natural analogue of relative entropy of entanglement in privacy theory and show that it is equal to the intrinsic information. We also provide a formula analogous to the entanglement of formation for classical distributions. (shrink)

The causal and simulation theories are often presented as very distinct views about declarative memory, their major difference lying on the causal condition. The causal theory states that remembering involves an accurate representation causally connected to an earlier experience. In the simulation theory, remembering involves an accurate representation generated by a reliable memory process. I investigate how to construe detailed versions of these theories that correctly classify memory errors as misremembering or confabulation. Neither causalists nor simulationists have paid (...) attention to memory-conjunction errors, which is unfortunate because both theories have problems with these cases. The source of the difficulty is the background assumption that an act of remembering has one target. I fix these theories for those cases. The resulting versions are closely related when implemented using tools of information theory, differing only on how memory transmits information about the past. The implementation provides us with insights about the distinction between confabulatory and non-confabulatory memory, where memory-conjunction errors have a privileged position. (shrink)

Non-technical introduction to Giulio Tononi's Integrated Information Theory of consciousness.

Integrated Information Theory (IIT) of Consciousness is becoming an increasingly popular neuroscientific account of phenomenal experience. IIT claims that consciousness is integrated information in a system. I set this theory against the hard problem of consciousness (Chalmers, 1996, 1995) as the goal for a theory of consciousness to meet. In this essay I look to examine and ultimately critique IIT’s use of the notion of information to base a theory of consciousness. I argue (...) that the notion of information in IIT is a purely structural-dynamical notion, and so falls afoul of the structure and dynamics argument (Chalmers, 2003). I bolster these claims by appeal to the explanatory gap argument and show how IIT succumbs to this argument as well. For these reasons, I call into doubt IIT’s ability to answer the hard problem of consciousness. Although this paper argues against the notion of information in IIT, in a broader context the criticisms which I raise here can be brought against any theory that attempts to explain consciousness as an information-theoretic phenomenon. (shrink)

Integrated Information Theory (IIT) stands out as one of the most promising theories for dealing with the hard problem of consciousness. Founded on five axioms derived from phenomenology, IIT seeks for the physical substrate of consciousness that complies with such axioms according to the criterion of maximally integrated information (Φ). Eventually, IIT identifies phenomenal consciousness with maximal Φ or, what is the same thing, with the strongest cause-effect power in the system. Among the scholars critical of this (...) theory, some point to the so-called Intrinsicality Problem (IP), namely that consciousness cannot be an intrinsic property of the system because maximal Φ crucially depends on the possible existence of bigger values of Φ if the initial system is appropriately linked to or embedded in larger systems. Although proposals in the recent literature aim to solve the IP by going beyond reductionism and physicalism, none of them tackle the real issue, i.e., the insufficiency of IIT’s causal-metaphysical structure. This papers endeavors to provide a solution to the IP in IIT within a hylomorphist ontology that includes formal causation. Complementing IIT with formal causation provides the theory with a criterion of individuation that solves the IP and, by relaxing identification between maximal Φ and consciousness, it lends a more robust metaphysical structure. To wit, maximal Φ is a necessary but not sufficient condition for the existence of consciousness. (shrink)

Integrated information theory (IIT) starts from the essential properties of experience (axioms) and translates them into requirements that any physical system must satisfy to be conscious (postulates; see Integrated Information Theory of Consciousness: An Outline, this volume). The postulates of IIT can be seen as a list of requirements for something to exist ‘for itself’, as an intrinsic entity, and thus have relevance for ontology and metaphysics. Some implications of the theory include the divide between (...) intrinsic and extrinsic entities, the incoherence of ontological reductionism, and the intrinsic nature of meaning. (shrink)

Turing machine An idealized computing device attached to a tape, each square of which is capable of holding a symbol. We write a program (a nite binary string) on the tape, and start the machine. If the machine halts with string o written at a designated place on the tape.

Integrated information theory (IIT) starts from the essential properties of experience and translates them into requirements that any physical system must satisfy to be conscious. It argues that the physical substrate of consciousness (PSC) must constitute a maximum of irreducible, internal cause‐effect power of a specific form, and provides a calculus to determine, in principle, both the quality and the quantity of an experience. Applied to the brain, IIT predicts that the spatio‐temporal grain of the neural units constituting (...) the PSC, and the relevant neural states, are those that maximize cause‐effect power. Moreover, the PSC can shrink, move, split and disintegrate depending on various anatomical and physiological parameters. These predictions are testable with brain stimulation and recording experiments. The theory can explain parsimoniously many known facts about the relationship between consciousness and the brain, including its association with certain cortical structures, its breakdown in deep sleep, anesthesia and seizures, and its return in dreams. (shrink)

The notion of information has figured prominently in much modern evolutionary theorizing. But while theorists usually concede the importance of distinguishing between our ordinary use of this notion and its special acceptation in information theory, some biological theorizing requires "information" to serve a double duty. Lorenz's ethological theorizing is a case in point, and this paper challenges its conceptual underpinnings. Special attention is paid to Lorenz's contention that adaptation to an environment is akin to representation, and (...) it is urged that many purportedly (phylogenetically) adapted behaviors might well instead be better interpreted as non-information-laden (ontogenetically) adaptive behaviors. (shrink)

Quantum information theory has given rise to a renewed interest in, and a new perspective on, the old issue of understanding the ways in which quantum mechanics differs from classical mechanics. The task of distinguishing between quantum and classical theory is facilitated by neutral frameworks that embrace both classical and quantum theory. In this paper, I discuss two approaches to this endeavour, the algebraic approach, and the convex set approach, with an eye to the strengths of (...) each, and the relations between the two. I end with a discussion of one particular model, the toy theory devised by Rob Spekkens, which, with minor modifications, fits neatly within the convex sets framework, and which displays in an elegant manner some of the similarities and differences between classical and quantum theories. The conclusion suggested by this investigation is that Schrödinger was right to find the essential difference between classical and quantum theory in their handling of composite systems, though Schrödinger's contention that it is entanglement that is the distinctive feature of quantum mechanics needs to be modified. (shrink)

Communication is an important feature of the living world that mainstream biology fails to adequately deal with. Applying two main disciplines can be contemplated to fill in this gap: semiotics and information theory. Semiotics is a philosophical discipline mainly concerned with meaning; applying it to life already originated in biosemiotics. Information theory is a mathematical discipline coming from engineering which has literal communication as purpose. Biosemiotics and information theory are thus concerned with distinct and (...) complementary possible meanings of the word ‘communication’. Since literal communication needs to be secured so as to enable semantics being communicated, information theory is a necessary prerequisite to biosemiotics. Moreover, heredity is a purely literal communication process of capital importance fully relevant to literal communication, hence to information theory. A short introduction to discrete information theory is proposed, which is centred on the concept of redundancy and its use in order to make sequences resilient to errors. Information theory has been an extremely active and fruitful domain of researches and the motor of the tremendous progress of communication engineering in the last decades. Its possible connections with semantics and linguistics are briefly considered. Its applications to biology are suggested especially as regards error-correcting codes which are mandatory for securing the conservation of genomes. Biology needs information theory so biologists and communication engineers should closely collaborate. (shrink)

Hi everybody! It's a great pleasure for me to be back here at the new, improved Santa Fe Institute in this spectacular location. I guess this is my fourth visit and it's always very stimulating, so I'm always very happy to visit you guys. I'd like to tell you what I've been up to lately. First of all, let me say what algorithmic information theory is good for, before telling you about the new version of it I've got.