Information Theory

As a neurologist, I spend a lot of time thinking about information.  Sometimes I think about information that the brain produces; typically when I talk about how the neural network forms and uses concepts.  At other times, I think about the information that goes into the brain.  I wonder about the extent to which the genetic code influences brain structure, and wonder about other types of information that might govern the development of the brain, such as fractal geometry.

“Information” is a big word.  In some respects, it strikes at the heart of reality.  Reality is relative; it tends to vary with scale.  On a human scale, reality is defined by time, space, matter and energy.  On a cosmic scale, or a subatomic scale, we see that reality is not defined by any of these things.  Cosmologists vaguely suggest these things are not what they seem.  Quantum physicists spend all day working in a world where matter and energy basically don’t exist.  Both agree that the one true thing in the universe is information.

There are spiritual and philosophical uses of the term “information” as well.  I gripe vigorously and frequently about the imprecise use of the term “consciousness”; about how people should be more aware of how this concept interacts with “intelligence” and “sentience.”  Can’t we all get along?  We could, if we agree to avoid the use of loaded terms, and agree to use the term “information” to refer to… well, everything.  If I’m going to write about neuroscience, spirituality, and physics, it behooves me to use a term all can agree upon.  “Information” is it, I think.

I have suggested that we could all benefit at times by taking the pause that refreshes; we should pause sometimes and ask, “What does it mean to be a human?”  One way of getting a fingernail under that issue is to consider the nature of the human mind-brain.  Without belaboring the point, there is something about the human brain that says something about that question.  “What am I” and “How did I come to be” are intimately related questions; arguably two different ways of asking the same thing.  I think we can all agree that “God made me” is a non-answer.  Even people of deep faith will have follow-ups: what is God, what is my relationship with God, and what does that mean?

Scientists, including those who are people of faith, argue that there is plenty to learn by attempting to answer these questions without resorting to the God principle.  Even if we really believe that God created us, it’s illuminating to ponder the question of how.  Newton was a man of faith, but he was still curious as to the nature of the laws that govern God’s creation.

It follows, then, that the statement “There is no God” could theoretically be a fact, a theory, an opinion, a belief, or an assumption.  It behooves us to be mindful of the differences between these things.  Scientists resist the notion that science is a philosophy or a religion.  They say an assumption does not equal a belief.  We would do well to listen to them; I am of the opinion that the post-modernist mindset, the source of a great deal of existential anguish, represents the elevation of scientific assumption to the level of belief, expressed as fact.  Nowhere is this more apparent, and more destructive, than in the extent to which the theory of evolution is submitted as a grand theory of everything.

To be clear, evolution is a fact of reality.  It is based on thermodynamic law, which is a fundamental fact of reality.  When fundamentalists suggest that evolution isn’t all it’s cracked up to be, they are wrong.  Now, if they were to say that reality isn’t all it’s cracked up to be, they would be barking up the right tree.

Reality is relative.  We all live in our own unique reality, where each individual observer exists in the center of a bubble, the boundary of which is defined by the speed of light.  In this bubble, time only exists as a function of distance, or as a metric of our mortality.  When we share a frame of reference with other observers, we can come to a consensus view of a shared reality.  At this point, time in the sense of Chronos exists, albeit with distortions caused by a tendency to assume cause-and-effect when none exists.  The visible universe is a form of reality, where the laws of physics and mathematics apply to all frames of reference on a given surface (because “space” is probably actually a “space-like surface”).  At this level, time becomes relative in every respect.  We know there is a bigger layer of reality, because we are aware of the existence of gravity, something that does not arise in the visible universe.  Time probably doesn’t exist at all, at this level.   We can also play the movie the other way, by dialing the scale down to the atomic, sub-atomic, or quantum level.  We will find additional layers of reality down there; and the farther we go, the weirder they get.

It behooves us to remember that the laws of nature are scale-dependent.  Joshua Slocum, the first person to sail around the world solo, encountered this concept when he stopped off in South Africa.  By then, he was quite famous; when the president of South Africa found out he was in town, he invited Slocum to dinner.  Apparently it was a grand affair, but there was a particularly uncomfortable moment when the president asked Slocum what, precisely, was his plan.  The room fell silent, for the president was a confirmed flat-earther, and would brook no disagreement.  Slocum was speechless at first.  He wound up eventually admitting his plan, much to the pity of his host.

I would submit he could have finessed the situation by saying, “It’s flat enough.”  In other words, the earth is round, but it’s really big.  On a human scale, it’s hard to perceive the curvature of the earth.  Any given human-size plot of land is almost flat.  Not quite, but close enough for government work.  This is nothing different from what a modern physicist is saying, out of due respect to Newton, by saying that space is a 4-manifold that is locally space-like.  When your high-school physics teacher says space is like a three-dimensional box full of nothing, a physicist could say “close enough,” recognizing that the teacher’s frame of reference looks something like a football field, and is probably happy enough that your teacher was at least smart enough to assimilate the round-earth theory.

The theory of evolution, then, is true enough.  But don’t overestimate what it can do for you, as it pertains to the question of what it means to be human.

The primary issue with the theory of evolution, as it pertains to the human brain, is that it only allows us to study half the information equation.  Evolution is based on the assumption that information follows from matter.  It does not address the idea that information precedes matter, except to the extent that it allows for genetics, which a life scientist will sheepishly assert is a form of matter.  This does not mean that life scientists necessarily believe their assumption is a fact, although some apparently do.  Don’t make the same mistake.

The disconnect is found in the popular definition of “entropy.”  Entropy is typically described as a state of disorder.  It is not.  In fact, entropy is a highly ordered state.  When a life scientist says that entropy is not ordered, he is stating an assumption, not a fact.  One way of stating the third law of thermodynamics is, “anything that can crystallize, will crystallize if it gets cold enough.”  Now, if you want to know something about crystals, don’t talk to a life scientist.  Talk to a crystallographer.  They understand that information precedes form.  They spend all day studying the information space that defines crystal structure.  They have invented machines and mathematical algorithms to infer the nature of that information.  That’s what they do.  That is not what life scientists do, because that ultimately brings God into the equation.

Well, that’s what they say.  I’m not sure I buy into that.  I think the life scientists are making an unwarranted assumption (they do that a lot) that Newtonian reality is all there is. Anything outside the Newtonian universe has to be God.  That’s not technically true; Einstein didn’t find God.  He’s not even close.  We need not fear pulling back, and taking a broader view of reality.

Let’s start with the idea that information can either precede, or follow from matter.  And let’s also stipulate that, inasmuch as time, matter and energy are scale-dependent realities, perhaps we could come up with a more general rule than the rules of thermodynamics, which are locked into a version of reality where energy and time exist.  If we express those laws in terms of information flow, instead of energy flow, I think we might have something a bit more generally true.  Not ultimately true, and not true on a smaller scale, but perhaps truer on a bigger scale.

Earlier, I came up with a swipe at it:

  1.  Information is neither created nor destroyed.
  2. Complexity approaches the attractor.
  3. Any information space is an attractor that defines a conformation of time, space, matter, and energy.

I’m a little uncomfortable with my law of conservation of information. There are two features of this law that deserve debate.  The first has to do with the question of whether or not information can increase in a closed system.  The second has to do with whether we can cut a pie into larger pieces, information-wise.

Neural networks can make information.  A neural network has a simple design relative to the complexity of the concepts it can contain.  Every neural network will form concepts that are meaningful, and there will likely be a high degree of agreement with regard to the nature of this meaning.  But each neural network will do it its own way; the specific pattern of activation, especially of hidden units, will be unique to any given network.  The diversity of concept-formation is probably more apparent than real, but the fact remains, I am more convinced that intelligence in particular is conserved, moreso than information generically.  I do believe it is not possible for a computer to design a computer that is more intelligent than itself.

It follows, I think, that when we make an information space complex, by dividing a unitary information space into numerous smaller information spaces, must necessarily involve a reduction in unit information.  You can cut a pie into smaller pieces, but you can’t cut it into bigger pieces.  This isn’t to say that information is lost; the point of complexification is to improve efficiency.  Sometimes it’s easier to hire a bunch of fairly smart people, and have them work together on teams, that it is to wait for an Einstein to come along who can do the job himself, without help.

I think Godel’s theorem might apply.  He was a mathemetician, and I can’t even begin to explain his theorem in math.  In English, he said that no system of logic can derive its own axiomatic principles.  He stated this in math so he could prove it, and he did.  I think this proves that you can’t cut a pie into bigger pieces, only smaller pieces.  And I think that proves that a closed information space can’t get smarter all by itself, even by cutting itself into a complex system.

If I’m on the right track, it does allow us to reconcile the notion of free will in the age of Minkowski space-time.  Free will, like everything else in reality, is scale-dependent.  Information dynamics suggests that the final state of the universe is implied by its initial conditions.  But who knows how it will get there; that process is chaotic.  We don’t even necessarily know what the universe will look like at the end.  We know the form of the answer, but not the precise answer.  We don’t have much choice in where we are going, but how we get there is up to us.  Within the context of 30 billion years of creation, free will exercised in the context of a single human life doesn’t account for much.  But on the scale of a single lifetime, free will counts for a great deal.

Where does human-kind fit into this scheme?  Heck if I know. Here’s how I look at it.

The human body is a function of thermodynamics and evolution.  It’s a nifty mechanism, but there’s no law says it’s the perfect design.  If that were true, I wouldn’t have a job.

I’m not at all convinced intelligent human life is destined to remain the dominant life form on this planet.  I think intelligence itself is the dominant life form; I think the attractor in this universe strongly favors the evolution of intelligence.  If it can find a better host, I expect it will jump ship.

I think the cool thing about being a human, which is also the frustrating thing, is that intelligence has finally gotten to the point where we can wrap our heads around questions like this.  We can perceive that we are an expression of the information set that defines our universe, and we can formulate a theory of gravity that implies there is a superior information set out there somewhere.  And we can extrapolate from there, to the idea that there is an ultimate information set, or a set of all information sets, which is different but the same.

In other words, we have reached the point where we can say things like, “I think, therefore I am.”  To say that Descartes made an error is to not get the joke, because he stole that line — at least the “I am” part — and it behooves us to remember who first said it. More to the point, we need to remember that when we say Descartes was in error, we are stating an assumption, not a fact.  And you know what happens when people assume.  Low birth rates occur.  Addictions happen.  And people kill themselves.  This is not a trivial matter.