Glitches in the matrix: down the rabbit hole of how reality and your brain interact

Your brain does a better job of distorting reality than AI ever could

If I had to come up with one single reason for a lot of human conflict & confrontational behavior it would be this: everyone feels entitled to receive empathy but no one wants to give it.

The worst part is that its not even anyone’s fault.

You (and I) feel that way because it seems SO intuitive that we are seeing the world objectively and hence our POV and actions are in service of the truth. It then follows that if others are not acting in a similar manner to us, it’s not because they’re seeing a different truth — rather that they have ill-intentions. We even accuse loved ones of ill intentions if their actions are very different from ours.

Underlying this thought pattern is a very powerful belief termed as “Naïve realism”. It’s an old concept but the gist is that one believes fully that their perception (a) matches an objecting & true reality, and (b) this perception (incorrectly labelled as reality) is indisputable.

“Sure, perceptions can sometimes diverge from reality,” one might think, “But those are JUST aberrations that happen to people with biases or with dishonest people. If one is honest and is aware of biases, they can see reality as it exists. You can’t philosophize away common sense…right?”

Wrong.

But let me use a recent example to illustrate this better than any philosophical argument could. Remember this picture that was breaking the internet?

Some people will see this dress as black and blue, while others will see it as white and gold (I’m personally in the white and gold camp).

While you may have been aware of this debate, it died a natural death — registering no more than a curiosity in most minds. What you may not know is that scientists who study brains were also perplexed by this phenomenon.

It was the first case of an “interjacent bistable visual illusion”. Like many illusions (such as the famous duck-rabbit illusion), there were two ways that a brain could interpret the image, but this was the first recorded case where going from duck to rabbit or rabbit to duck became impossible. I.e., once you saw the dress as white and gold, your brain would refuse to see it as blue and black.

Remember a “true objective reality” can be said to exist — but your perception of it cannot be trusted. On your own computer right now, the gold/black bits are actually throwing off this color.

Where as the white/blue bits are this color

I know the color above this sentence isn’t white. But when it’s in the context of the picture, my brain refuses to acknowledge that bias. The reason is likely due to the way that perception works from a biological lens.

Contrary to what seems intuitive, there is no film projector and screen mechanism between your eyes and the brain. Instead the eyes pick up on certain patterns in the world around us (photons), convert those to electric signal & then sends it across to the brain. The brain then recreates a world based off of those electric signals.

The eyes are cool in that they’re a very high bandwidth signal converter that converts light to useful electric signals. But shockingly, the brain doesn’t need the eyes to “see”.

In fact, multiple studies have shown that we can get even the blind to “see” the world using technology that converts vision captured by camera to electrical signals. In the case of technologies like BrainPort, it’s not that the blind are imagining a world the same way you would day-dream. Rather, they are actually seeing and experiencing reality the same way you do when you use your eyes. Of course, the reality created isn’t as high-def as a normal sighted person — but that’s likely because technology hasn’t caught up with the sophistication of the biological systems that capture image & turn it to electric signals.

The brain doesn’t just stop at creating reality from signals. It further optimizes processing power by maintaining high-fidelity archetypes of things it has encountered before. So often it will fill in the blanks and create a personalized reality that ties into past experience — to perhaps save on processing energy. In fact that is the reason some of us see the white goal whereas others see blue/black is because some of us have more experience in natural lighting and others in artificial lighting.

If that sounds hard to believe, look at the following image. Most of you will see this image as red strawberries (although some may not be able to view the strawberries).

But there’s actually no red or white in the picture above. The red is actually this color:

And the white filling (inside the tart) is this

Crazy right?

Just goes to show how your assessment of perception vs reality could be flawed.

Wait a second, something seems off. If our perception can be so different from the truth then how is it that we all see the same thing most of the time?

In other words, how is it that when I show you a green door, you usually see a green door as well (regardless of the images above)? This consensus is important, and in its absence it would be impossible for co-operation or society to exist.

How do we agree on anything?

To be honest — we don’t have a proven answer, but there are some hypotheses.

I’d like to propose one as well — one that views the brain as a “truth filtering machine”. Furthermore, this filtering machine has evolved over millenia with one goal — to filter only those aspects of reality that help an organism survive and thrive in a specific environment.

While it is more common to think of evolution with regards to organs that fulfil physical roles such as legs (carry you around) and stomachs (convert food to energy), there’s no reason that evolution would not favor organs that play a perceptual role as well. Especially when such perceptions could be tied to physical world benefits.

What this means is that the most successful designs of brains over time were ones that led to higher chances of organisms surviving and passing on their genes. And much like genetic code for developing wings, the code for brain perception too was passed on from parent to off-spring. Thereby leading to a sort of common-blue print for perception much like there’s a common blue print for how the stomach works across all humans.

Of course just like how stomachs may vary across species, this also means that different species could encounter different perceptual realities (e.g., dogs and their sense of smell is a great example).

A few interesting things emerge from this theory:

• its almost as if “reality” is a stream of information with an unbelievable amount of “data”.
• However not all of “reality” is necessarily useful for surviving & thriving in a particular environment and there are costs for perceiving reality (you may need more food to better perceive reality)
• The perception filter that wins out will then do the following better than competing filters (in similar environments)

1 Separate out what is the signal (patterns) vs the noise (chaotic elements).

2 Prioritizes signals that are likely the best for surviving and thriving

3 Not showing both the less prioritized signals & the noise

The reason human society can function and collaborate is that our brains have evolved to do the above three things in nearly identical ways, but individual differences could still arise. Much like our stomachs process food in nearly identical way (which is how we can have restaurants), but there are also stomachs out there that can’t process certain foods that other humans have no problem with.

Where perceptions diverge from reality

If the above theory is valid, we can make guesses as to exact reasons that perception and reality can start diverging — both for individuals within species, but also for an entire species itself. They fall under two broad camps

Camp 1: Breakdown in IDing Signal vs noise

One of the brain’s core job could be said to make distinctions between what is signal vs what noise.

But what if the brain doesn’t see a signal even when one exists. This could be because the signal is more complex than the brain’s ability to comprehend or even that the body doesn’t have useful senses to pick up on the signals (e.g., UV or x-ray light can’t be picked up by the photo-receptors in our eyes).

The brain can also observe signals where none exist (this is what happens with schizophrenics for example).

Camp 2: Fault in prioritization order of signals

While human brains have likely evolved to have similar prioritizations of what signals to show (the ones that have been proven to lead to meaningful improvement in genetic propagation), this likely means that there also exist signals that are not being prioritized (and therefore perceived consciously). If our environments were to change, one would think that a re-jigging of the prioritity list would be in order. However, this is unlikely to be the case as genetic propagation may move too slowly to keep up with environmental changes.

In conclusion, while our brains are very similar and follow the same blueprints, they’re not identical. The best analogy would be your fingerprints. And the extent to which the above breakpoints differ between individuals are the reasons why we see a different reality all together — and also why its so hard to be empathetic but so easy to expect empathy.

Further weird implications

We’ve only discussed filtering within the context vision till now. But given that most senses originate in the brain (following a similar process to vision), could this creation of the perceptual ‘matrix’ be applicable to other perceptory senses as well?

One could make the case that the other senses (smell, taste and touch) too follow a similar protocol of converting raw data (from “reality”) to electric signals that the survival-optimizing brain then uses to construct patterns that can be taken advantage of to survive/thrive.

It would almost be like you’re literally giving shape to reality around you based on an evolving filter.

Some strange implications that I think are fun to consider include:

An explanation for the “supernatural”? What if talks of ghosts, apparitions, spooky stuff is actually nothing supernatural, but just signals in “true whole reality” that are usually too weak to be picked up or deprioritized by the brain. On rare occasions, such signals could reach some sort of critical threshold and bubble up into the brain’ zone of signal awareness. They could then be incorporated into our perceptions in a warped way that seems to fit in better with our existing narratives (much the same way that an alarm sound winds up in a dream narrative right before you’re waking up).

The supernatural could thus be some actuals signals that the brain is faintly picking up, but since there very little experience the brain has with these signals, they are being mis-categorized within an existing narrative (spirits that live after death) that the brain is more familiar with.

Are geniuses simply mutations of the brain? Could it be that child prodigies and genius savants (especially the ones that are naturally gifted) are simply the outcomes of a truth filtering machine that is attuned to a slightly different setting?

Why dark energy and dark matter are so confounding? The physical laws that we have discovered are derived or have come on top of various observations that scientist have made with their senses. And scientist and their senses too are limited by their brain’s ability to pick up signals.

What if there are some signals that human brains are incapable of picking up on, no matter what tools we device. And what if dark matter and dark energy are just manifestations of those patterns on our present day physics. In effect, dark matter and dark energy doesn’t have to be matter and energy — it could just be that current laws of physics aren’t incorporating aspects of reality that are ever present all around us, BUT are not perceivable to human brains as signals. I.e., the signals are either too faint or too complicated for our brains to pick up (or perhaps they don’t lead to any evolutionary advantage).

One could argue that the most effective tool scientists use in their investigations does not suffer from sense bias — the tool being mathematics. However, the counter-argument of brain not being sophisticated enough to perceive signals/patterns (even mathematically) could still be said to be valid. The fact that mathematics continues to evolve and hasn’t stagnated hints at the fact that there is a lot out there that is still waiting to be discovered.

What reality do other animals experience? Remember that the brain’s goal is not simply to pick up on signals, but also prioritize the ones that are most necessary for survival. This prioritization is likely dependent on the environmental context as well. For example, signals that will lead to higher chances of surviving/thriving might be difference for life under the sea vs on land.

While the land vs sea distinction may seem obvious, I do not think they represent the greatest contrast — i.e., roughly similar things would be prioritized.

Instead, I believe the greatest differences in prioritization would arise with scale differences. For example, if a species’ context involved sub-atomic particles and picking up nuanced quantum patterns afforded it a better chance at survival, it would experience a different reality. Specifically, you could theorize that it would perceive reality in a way that played up patterns from sub-atomic world and played down patterns of the macro-world.

The crazy thing is that there might actually be proof for this. As an example, a 2021 paper in UChicago studied how bacteria know how to exploit quantum mechanics. While bacteria doesn’t really have a brain, there are still animals such as tardigrades (who have a primitive brain & associated sense organs) and inhabit a tiny tiny world. Do these animals actually perceive an Alice in Wonderland type of world?

The answer is we don’t know, but scientists are testing the possibilities — including trying to quantum entangle a tardigrade.

The brain can be viewed as an instrument that shows us part of the truth. This falls under an umbrella of half-truths that serve an overarching purpose in the universe we inhibit.

I call these half-truths “maya” and explore the topic in greater details here.