What are the challenges in visual scene understanding? There’s a long list: occlusion, the presence of many objects, unfamiliar objects, reflections on surfaces, shadows, blur, etc. Due partially to research breakthroughs but mostly to a massive amount of money and resources, we have computer vision systems that can now perform some version of scene understanding for constrained tasks, such as self-driving cars and autonomous warehouse cart robots.

For now, let’s consider an even simpler problem: object recognition. Suppose you perform 3D transformations on three cars. For each resulting image, you ask someone which car is in the image. This feels like an easy task because it happens almost automatically for us. But consider that the pattern of light that hits a sensor is completely different for each image — there are often no pixels in common between two instances of the same object. Systems must both perform complex operations to transform these seemingly unrelated patterns of light into more abstract concepts that eventually map to recognition.

How do current ML/AI systems solve the problem? Deep learning-based computer vision, with mechanisms inspired by early neuroscience research, has had several breakthroughs in the past ~14 years, including convolutional neural networks, residual networks, and vision transformers. We know they work (where the definition of “work” is based on benchmarks and product usage), but it’s hard to claim that we understand how they work, especially as these models have gotten much more complex. And despite successes, there are still strange failures that indicate that something may be missing.

One way to talk about these failures is excessive sensitivity — when small, perceptually irrelevant changes affect model outputs — and excessive invariance — when large changes that should affect the model do not. Here, different frames in the same movie with very little perceptual difference cause a massive drop in the model’s probability of classifying an otter.

Here, despite massive perturbations of the image that render the image unrecognizable, the model still has high confidence of the correct class.

Although these papers are from a few years ago, if you use any current image or video models, I’m sure you’ve seen a lot of other strange failures as well. So what are these models missing? What properties would an ideal vision system (natural or artificial) have? (I posed these as discussion questions to the class; feel free to ponder or respond.)

Wouldn’t it be great if we could borrow concepts from something we know has robust object recognition abilities — the brain? TOO BAD! We don’t know how the brain does it 🤷🏻‍♀️1. Okay, then let’s think a bit deeper. Suppose we were designing a system for object recognition from scratch: what would be good computational principles to incorporate? There are many answers of course, but in this post I’m going to focus on just a few principles: invariance and equivariance. And more generally, factorization2.

There are formal mathematical definitions for these terms, but for our purposes, I’m going to define them this way: A representation used for recognition should discard transformations irrelevant to object identity (invariant to transformations). For general scene understanding, there should also exist representations equivariant to the transformations, i.e. that change appropriately with the relevant transformation variable3.

On the left, scaling the cat still gets you the cat label, with information about the scaling thrown out. The representation of the cat is invariant to the scaling transform. On the right, scaling gets you a representation that changes along with the scaling of the input; it is equivariant with scaling. Another example of an equivariant representation would be any type of slider in a UI: the position along the bar represents some other relevant variable, e.g. the volume bar on your computer.

Factorization is the process of disentangling relevant factors into explicit representations. Here, an image can be factorized into reflectance — the properties of the object surface — and shading. If you were performing pattern recognition, it would be useful to have the reflectance image, separate from the shading. But if you wanted to understand a general scene, for example, where the light source is, knowing the shading information is necessary. Even though edge 1 and edge 2 have identical pixel values, we know the causes of these values are due to different factors. The brain is clearly doing some kind of factorization.

This example is simple, but many problems in perception can be framed as factorization, such as identifying someone’s voice in a loud room while understanding what they’re saying, or knowing how an object is oriented in order to grab it4.

Now that I’ve introduced factorization at a high level, I’ll focus in on an example of low-level factorization with invariant and equivariant representations in a model: complex-valued sparse coding. My previous blog post has an intro to regular sparse coding, but here’s a quick summary.

Given a set of images X, the goal of sparse coding is to learn a set of basis functions A (each corresponding to a unit) that can be linearly combined to reconstruct the images. The key is that the coefficients s of the linear combination should be sparse, i.e. mostly zeros. We learn an A for the whole dataset and infer an s for each image by minimizing E with respect to the appropriate variable. s is sparse, and A ends up looking like oriented, bandpass, localized filters (Olshausen & Field 1996).

There’s a slight problem with standard sparse coding, however, which is that patterns are entangled with transformations. Here, I’m showing how the sparse code in orange changes with a shifting input pattern.

The representation of the image, the sparse code, is changing quite a lot with each frame. You and I can see easily that a static pattern is being shifted vertically. But if all you saw was this sparse code, a set of neurons firing, it would be impossible to tell whether the changes in the sparse code were caused by the pattern itself changing, a transformation to the pattern, or a combination of both. What might be a better representation that makes the pattern and the transform explicit?

Let’s look at complex-valued sparse coding for movies, described in Cadieu & Olshausen 20125, as an example of factorization.

It has the same core algorithm as sparse coding, but the basis functions are now complex-valued. While the s variable is still responsible for the amount of the feature present and is encouraged to be sparse, the phase variable φ is responsible for interpolating between the basis functions, as you can see in the generative model equation above. Put another way, you can steer the exact appearance of the basis function in the reconstruction using φ.

The energy function to be minimized is almost the same as regular sparse coding, with an extra term that encourages s to be similar between frames. This follows the slowness principle6 which says that because of smoothness in the world, i.e. things don’t change suddenly, representations in neighboring frames should be similar to each other. The authors end up with pairs of receptive fields (each pair is the real and imaginary component of one unit) like the ones below, that vary only in phase.

If you animate them, i.e. apply the generative model above for a unit turned on and fixing the amplitude but varying the phase, you get this. This shows the range of possible features that can be expressed by each unit.

So what does this have to do with factorization? How would this representation allow us to disentangle the pattern from the transform in the example earlier? A key insight in this model is that because of the sparsity and slowness penalties on the amplitudes, representations of transformation are conveyed phases, which can spin freely at each frame. Amplitudes stay relatively stable to incur less cost in the objective function. Here, I’m using the same video but showing the complex-valued sparse coding representation. Orange are the amplitudes, purple is the phase structure of the pattern, and green is the change in phase at each step.

I hope you can see that the information is factorized into two parts: the complex-valued pattern se^{jφ} (orange and purple) whose amplitudes s and phases φ are pretty much constant over time, and the transformation (green) represented by phase shift δ. For this particular transformation, applying local vertical shift does not affect the invariant representation, and the amount of shift is specified by the amount of δ applied — an equivariant representation.

(BTW, I’m not explaining well what about amplitude and phase allows this to work, and I’m actually skipping a portion of the lecture about the importance of phase in images, but this post got too long. I’m planning to write a separate post(s) about phase.)

Furthermore, we can combine patterns and transforms to generalize to novel image sequences. For example, given the complex representation of an unseen pattern, you could apply the appropriate δ to shift it vertically. You can also understand novel sequences: given a known pattern that has been transformed in novel way, you should be able to recover the transform being applied to it, and then apply it to other images. Pretty neat!

Here are example of other transformations, or other δ configurations that can be learned from videos.

The main themes of the lecture were discussing challenges and failures in computer vision, factorization into invariant and equivariant parts as a computational strategy, and complex-valued sparse coding as an example of factorization in a theoretical model of early visual cortex. The subtext of the lecture, however, was that although we live in a society dominated by a very specific type of statistical learning, it is not flawless, and it is not the only way to do things. Although we don’t understand how the brain solves these problems, I think it’s still important to think about fundamental principles of computation for both understanding natural systems and building artificial systems. Maybe we’re wrong: factorization is not how the brain does things, and maybe it’s not the best design for an artificial system. But at least we’ve started to understand the nature of these problems, rather than just throwing data at them7.

I wrote my first blog post during my transition from computer science to neuroscience in my PhD. At the time, neuroscience felt religious to me, and I struggled to understand how neuroscientists often have such immutable convictions in the face of so much conflicting yet reputable evidence1.

Even in computational neuroscience, which appeared more concrete to me than experimental neuroscience, no one really agrees on anything. Unlike machine learning models, it is far less clear what it means for a model of the brain to be “useful”2. Should the goal be to fit to recorded neural data? Even if a model predicted spikes with perfectly, would we understood the brain any better? On the other hand, if we had a principled model that did not fit data as well, would that be useful? One could go around in circles forever arguing like this.

Further contributing to my confusion was the fact that I had not yet mapped out the literature and history of the field for myself. I didn’t know what was obvious or accepted, even from my lab’s point of view. I certainly wasn’t ready to form my own beliefs, and as a result, didn’t have a good idea of the research questions I was interested in. I wrote the first blog post with the naive hope that verbalizing my lack of clarity about the field would help me make more sense of everything.

1.5 years later, I am only slightly less naive and only making a little more sense. The biggest change is simply that I have spent much more time reading, talking, writing, and thinking about the brain. Just like practicing an instrument, playing a sport, or learning a language, I feel myself starting to develop proficiency in this new skill of… neuroscience? Let me explain what I mean.

In the past, questions during presentations or even meetings would often send me into a panic. They didn’t have to be particularly hard questions, just ones I hadn’t yet thought too much about. Attempting to answer involved straining through complex logic, or tortuously sifting through figures and equations in my head. I could not understand how people were able to answer hard questions on the fly, or look at a plot and immediately understand what was going on. It was like they were fluent in a language, and all I could do was string together broken phrases. Was I just too stupid to be doing this?3

Through a slow and agonizing process of pushing my research forward and preparing for my qualifying exam, I managed to start orienting myself. I won’t go through the gory details, but I treated it like how I used to practice violin, or prepare for coding interviews in college. I’m sure everyone’s process is different. But as a result, I’ve started to feel the freedom of moving through a new world, truly like developing fluency in a language. I am more comfortable thinking on my feet, as if concepts are flowing around me, and I just need to grab what’s already there. I don’t have to think so hard all the time.

In other words, I have started forming my intuitions about neuroscience, or at least the subfield I’m focusing on now. For example, someone may ask a confusing question about my project, and I can immediately understand their underlying misconception. Or someone brings up an idea, and a simulation I could run with my model will pop into my head. These are things I couldn’t do before, at least not immediately, through reasoning alone.

A lot of this might seem obvious, and I feel a bit silly writing some of this down. But the reason this feels so profound for me is that I have never experienced this in an academic setting. In undergrad and before grad school, I kept hopping around different disciplines and labs to figure out what I wanted to do. I realized that finally, now at the end of my 4th year, this is the longest I’ve ever spent in one field. I am maybe starting to become an expert in something, for the first time?

I can start to see now that after decades of being in a field, a scientist’s intuition might look a lot like what I called religion, from the outside. Their intuition lets them feel that this problem is a good problem, that this approach is the right one. Even if there is evidence that seems to contradict it. Intuition stems from deep understanding, honed from decades of training. Although it may have started there, it goes beyond rational, quantitative scientific thinking.

In Mathematica, David Bessis argues that contrary to popular belief, mathematical ability is not an intrinsic gift certain people are born with. Rather, it is about developing intuitions in your own way, driven by the desire to understand. Although we’re traditionally taught that math is symbols on a page, that’s just one way to communicate intuition. You could also think about math in terms of of something we all have strong intuitions for, such as the physical world. Terence Tao famously rolled on the floor to think about a problem:

Early in his career, he struggled with a problem that involved waves rotating on top of one another. He wanted to come up with a moving coordinate system that would make things easier to see, something like a virtual Steadi­cam. So he lay down on the floor and rolled back and forth, trying to see it in his mind’s eye. ‘‘My aunt caught me doing this,’’ Tao told me, laughing, ‘‘and I couldn’t explain what I was doing.’’ (NYT)4

There is something very profound about human intuition, and scientists clearly rely on it to guide them. What would science look like without intuition? Unfortunately, understanding intuition itself is a hard problem, certainly harder than next-token prediction (does AI have intuition?). Regardless, it’s part of what makes us human, and more interesting and beautiful than a reasoning machine could ever be.

Let me connect this back to my original point. Similar to religion, intuition is separable from logic and reasoning. Perhaps someone’s intuition is so strong that no evidence they’ve seen so far can convince them otherwise. But ideally, unlike in religion, we learn something when we’re proven wrong. It deepens our understanding, and we modify the intuition accordingly. When someone’s intuition is correct, it may lead to discoveries that would have taken orders of magnitude longer through reasoning.

So for now, I guess I’d revise my opinion from neuroscience as religion to the less extreme neuroscience as intuition5. Through my own attempts to grow as a scientist and seeing others’ processes, I’m realizing how important intuition is in science, and how I might go about developing my own. Maybe these convictions that I see in neuroscientists aren’t as bizarre as they initially seemed. Maybe I will be there too, one day (for better or worse).

This is still just the beginning for me, of course. I still don’t feel comfortable calling myself a neuroscientist (maybe I never will) or a vision scientist (my department, technically), but I’m also definitely not a computer scientist. Whatever I may be, my focus now is not to form beliefs, but to hone my intuition. I guess, until I find a better reframing by the next reflection post.

Although I (obviously) got very behind with blogging, I set a hard deadline for myself to wrap up by… today. We’re finally done with the main topics so I’ll briefly summarize some of the special topics covered in the last four weeks of lecture. All lecture posts from this semester are here.

Factorization and compositionality in scene understanding

Visual objects can appear in infinitely many ways — with varying pose, illumination conditions, occlusion, in a complex scene, etc. — and still be recognizable to us. Deep learning does not process images the same way we do, and some work has highlighted these differences by looking at adversarial examples and metamers, e.g. Excessive Invariance Causes Adversarial Vulnerability (Jacobsen et al. 2018) and Model metamers reveal divergent invariances between biological and artificial neural networks (Feather et al. 2023).

How does this brain handle this? An extreme would be that there is a neuron that responds to every possible combination of every transformation of every entity, which would result in a combinatorial explosion. A theory to address the explosion is that there are mechanisms for factorization (inferring explanatory factors in a scene, e.g. shape from shading) and compositional structure (generalizing to complex scenes from basic components and rules) that allow us to more efficiently disentangle and understand parts of scenes. A few papers that explore this are Shape Recognition and Illusory Conjunctions (Hinton & Lang 1981), Dynamic Routing (Olshausen et al. 1993), and Disentangling Images with Lie Group Transformations and Sparse Coding (Chau et al. 2022).

Sparse Distributed Memory

Pentti Kanerva’s SDM (1988) is a model of human long-term memory using sparse, high-dimensional vectors, and uses a lot of the same concepts as his hyperdimensional computing framework (which I went over in my last post). He later found interesting connections to how it could be used to model the cerebellum. It’s a super rich model that’s been applied to research problems in computer vision, reinforcement learning, and more. Fun fact: attention in transformers has been found to approximate SDM (Bricken 2021).

Maps

The brain is full of maps for vision, audition, touch, space, and more. In the primary visual cortex alone, there are many types of maps, where distance on cortex corresponds to eccentricity in visual field, orientation of features, direction of stimulus movement, and more. Computationally, do these maps serve a purpose, or are they epiphenomena? The classic model is Kohonen’s self-organizing map (SOM, 1982), which starts with units placed on a 2D sheet. Initially the connections between the units are random with the constraint that units excite nearby units and inhibit faraway units. Through Hebbian learning, they eventually learn a map such that neighboring points on the sheet map to nearby points in space. SOM does not answer the question of why maps are needed, as the map part is built into the constraints. There are many theories, but maybe there’s some argument for reduction of wiring length? E.g. Chandra et al. 2024 find that maps and other phenomena emerge from a wiring minimization objective.

Perception and action

Up to this point, we’ve talked about vision as if it exists in isolation. This is wrong! Vision, along with other modes of perception, are active. For example, our eyes move constantly, even when we’re fixating on something, and one hypothesis is that eye movements are essential for high visual acuity. Some research that hints at this are Bayesian model of dynamic image stabilization in the visual system (Burak et al. 2010), Benefits of retinal image motion at the limits of spatial vision (Ratnam et al. 2017), and High-Acuity Vision from Retinal Image Motion (Anderson et al. 2021)1.

A bit more philosophically, there are also theories that “seeing” is an active process that occurs when we probe the world, i.e. due to sensorimotor contingencies. See A sensorimotor account of vision and visual consciousness (O’Regan & Noë 2001) and Is There Something Out There? Inferring Space from Sensorimotor Dependencies (Philipona et al. 2003).

Class projects

Class projects are always fun, and I was genuinely very impressed by these! This year, they ranged from analyzing real neural data to augmenting LLMs. The most popular topics were modeling the cerebellum, memory retrieval, and behavior with SDM; Hopfield networks for different types of data; and hyperdimensional computing with language. There was also a project on retinal waves in development of cortex, sparse coding applied to language, a hardware implementation of the fruit fly head direction circuit, and a review of models in computational psychiatry. Given the diverse background of the students, it was exciting to see what people found interesting enough to investigate on their own, especially in the current age of AI hype.

Reflection

To be honest, blogging through the course was way too much work! But it was also extremely fun, and I learned a lot. Not only was I forced to strengthen my grasp on technical details in order to write about them, I’m starting to get into the habit of thinking more about setting context (I do not know nearly enough history!) and forming a narrative when trying to communicate scientific ideas. People tell me I will be doing a lot of this in my career! I also feel more excited than ever about being in this field and studying these problems, because they are COOL and FUN 🤩! I also realized I kind of like teaching? 👀

I felt this when I took the course, and heard it from students this year: at about the halfway point when we start representation learning, the topics suddenly get more abstract and require more math. The associated blog posts reflected this shift: they were much harder to write, because I had to make more decisions about the precision-clarity tradeoff that is always present in science communication. But I’m happy with how they turned out given my time constraints. Although they are very far from perfect, I hope that even if you don’t have a technical background, you were able to learn something from each post.

Thanks for following along! If you want to stay, the future of dissonances will be a mix of my own research, topics I’m learning about, and random music stuff. See you in 2025! 🥳🥂

Current machine learning relies on manipulation of high-dimensional vectors. The classic Word2vec represents a word as a vector and associates semantic similarity with distance in a vector space. Transformers in large language models (LLMs) convert words to large vectors and perform operations like inner products, concatenation, and thresholding on them. These approaches are in the spirit of the connectionist AI philosophy, inspired by the machinery of neurons inside the brain and training a model to learn from the statistics of data. One weakness of connectionism is that it is unclear whether it can do things we consider important for cognition, such as bind variables, represent relationships between concepts, and compose expressions.

The contrasting old-school approach, symbolic AI, uses high-level logical and human-readable symbols as the basis for intelligence, and explicitly represents data structures (graphs, trees, etc.), variables, and relationships, as in computer programming. It’s a powerful framework, but is not designed to learn from and adapt to the world, and has fallen out of fashion in these data-powered times.

This is a standard narrative of connectionist vs. symbolic AI1, but there’s another more neuroscience-y weakness of connectionism to acknowledge. It was inspired by single-unit recordings in the brain and the idea that recording from individual cells conveys important information about neural computation, borne out of technological limitations that only allowed us to record from one neuron at a time. But the single cell angle is not the only way to think about computation in a network; recorded activity from one neuron is certainly not incorrect, but it does not tell us the whole story2, especially in the highly recurrent circuits of the brain. We also now have technology, such as Neuropixels, that allows us to record from hundreds or thousands of neurons simultaneously.

An approach that attempts to address some of the limitations in both connectionist and symbolic AI is Hyperdimensional computing (HDC, Kanerva 1997)3. In HDC, the fundamental unit is the activity of a population of units, in the form of a vector. It is not just a convenient representation; it is a different computational framework, where the primitives are vectors. Looking at a single value in the vector may tell you something, but it conceals the power of the system. HDC allows us to use the symbolic tools of data structures and logical operators while retaining some desired connectionist properties. We’ll demonstrate this through some examples.

HDC algebra

HDC defines an algebra over vectors, where the values of a vector could be binary, real, complex, etc. Given a concept, like a word, a color, or an image, we assign it a random high-dimensional vector. The main operations, which maintain the same dimensionality, are (random vectors in bold)

• Bundling: u = a + b + c, where + is element-wise addition. This combines a set of discrete items into one vector

• Binding (variable assignment or key-value pairing): d = k ⊙ v, where ⊙ is e.g. element-wise multiplication (Hadamard)

• Permutation (ordering vectors): s = a + ρb + ρ²c, where ρ is a constant

It is also easy to compare similarity between vectors, and a simple example is taking the inner product. u • a would have a nonzero inner product, whereas u • d would likely be close to 0.

The randomness and high dimensionality here are key: if vectors are orthogonal or close to orthogonal, information contained in u, d, and s can easily be decoded, queried, and used to perform reasoning without a serial search through the set. The higher the dimension, the more likely it is that random vectors are orthogonal, and the less likely it is that there will be cross-talk between vectors4.

Although it’s not without limitations, here’s a high-level summary of some advantages of HDC.

I have been vague about what querying and reasoning mean so far, but we’ll try to concretize these in the following examples.

Variable binding and language

Tony Plate developed Holographic Reduced Representations (HRR) as a graduate student with Geoff Hinton. At the time, they were interested in getting connectionist systems to do symbolic reasoning. HRR was inspired by Paul Smolensky’s tensor product variable binding (1990), but instead of keeping the expanded dimensionality, compressed the tensor into a lower dimension using circular convolution.

Plate’s method allows storage of lots of information within one N-dimensional vector, provided N is large enough. Querying the vector is also straightforward. Suppose we want to represent a word x that has part of speech c. We bind via circular convolution (⊛): t = c ⊛ x, and unbind using the correlation operator (⊜5) to get back an estimation of the word x ≈ c ⊜ t. We can compose two bound variables: t = c1 ⊛ x1 + c2 ⊛ x2 and get back an estimation of x1 ≈ c1 ⊜ t, e.g. if we wanted to get the word with part of speech c1 in the phrase t.

Here’s a more concrete example on language. We can represent parts of speech, roles, grammatical structures, and not just superpositions of words, but recursive superposition of words and their roles. For example, to represent the sentence “Spot bit Jane, causing Jane to flee from Spot”, you could construct a highly-structured vector that assigns meaning to each word

agt refers to the agent performing the verb, < > denote unit normalization, and each variable in bold is a random high-dimensional vector (say length=2048). You can query P for who’s doing the biting or fleeing, for example, and also compare the similarity between P and other sentences created using the same vectors. Surprisingly, something as simple as the inner product works quite well as a first-pass similarity measure, as shown in Plate’s thesis (1994), chapter 6.

Although this is a simple example compared to scale of what LLMs are doing now, you can see how easily recursive structure, such as a parse tree, can be stored and accessed efficiently in HDC.

Analogical reasoning

Researchers like Douglas Hofstadter and Melanie Mitchell have long been studying analogical reasoning as a key component of intelligence and reasoning6. The Abstraction and Reasoning Challenge (ARC) and variants have recently gotten a lot of attention7 for being hard to solve with LLMs. Although beating a benchmark is not equivalent to solving a problem, we have not yet saturated these benchmarks.

One basic example of analogical reasoning in HDC comes from Pentti Kanerva (2010). How would we answer a question like What is the dollar of Mexico?

For the variable names, we have Name, Capital, and Monetary unit of the country. For USA, those variables are bound to the values Us, Dc, and $, respectively. The binding operator is * here.

We can bind the two vectors representing the countries

which can also be written as

This can be derived from distributivity and the fact that Pentti uses vectors and a * such that A*(A*X) = X. The noise comes from the fact that all other terms should be approximately orthogonal and result in very small values. To answer the original question, we ask what in Mexico corresponds to $ in USA? and unbind $ with Pair:

to get the correct answer, P for Peso. This is all made possible by this simple algebra in high-dimensional space. This is of course a basic analogy, but we can represent much more complex ones with negligible extra computational cost.

At this point, you might be scratching your head. Of all the increasingly disconnected-from-reality topics we’ve covered in the past few posts, this is by far the most abstract. Despite its simple operations, it’s difficult to wrap your head around the idea of HDC as a distinct computational framework. But even more abstract still is its connection to the brain.

The short answer is that the motivation for HDC comes more from cognitive science and AI than neuroscience. Binding, bundling, and permutation operations are first principles-driven, and not from biology (though perceptrons didn’t come from biology either!). But there is importance in stepping away from neurobiological plausibility (the theoretical neuroscientist’s frenemy) and thinking about what types of computations would be required to perform cognitive reasoning. We’re exploring this framework now, with the ultimate goal of mapping it onto actual implementations in brain circuits. This mindset is not unique to HDC; this is more generally why theory is important8!

HDC is relatively under-explored, but we’re excited to see where it goes in the coming years. Chris Kymn’s recent NeurIPS paper very elegantly uses approaches from HDC and attractor networks to model hippocampus and entorhinal cortex. For a recent review of HDC applications, see Kleyko et al. 2023. And here’s a nice 2023 feature of Bruno and Pentti in Quanta Magazine.

I had just read an article by Scott Kirkpatrick on simulated annealing, and I said, “Geoff, we could heat it up, we could heat up the Hopfield network by adding a temperature”, so instead of always going downhill sometimes you pop up and with that, if you slowly cool it we can find the optimal solution… It turns out something magical happens when you heat up a Hopfield network: it becomes capable of learning the weights to a multi-layer network; which is the first time that had been done.

-Terry Sejnowski via The Artificial Intelligence Papers: Original Research Papers With Tutorial Commentaries by James Stone

At the time, the standard way of thinking about cognition was with rule-based symbolic systems. Hinton and Sejnowski’s 1983 paper Optimal Perceptual Inference was unique in approaching perceptual inference using ideas from physical systems. It was the beginning of what would become the Boltzmann machine: a probabilistic, connectionist model of perception. I this post, I’ll attempt to give a brief introduction to the Boltzmann machine.

What does it mean to heat up a Hopfield network? Recall that the energy function for a Hopfield network (intro in this post) is

where T is a 2D matrix of fully connected, symmetric weights between units with states sᵢ in {-1, 1}.

To recover a state stored in the weights, we want to travel to a low-energy basin of attraction. Another way you could describe the system is by defining the probability of a state s as

which is called a Boltzmann distribution and depends on the energy and the system’s temperature (1/β). High energy states are low probability and vice versa, as desired. If β is very small (high temperature), then we heat up the Hopfield network for a smoother P(s), and we can move easily between low-energy states. In the other extreme, if β is very large, then P(s) will be sharp peaks at low energy states; in this case, it would be hard to move between them. In the Hopfield network, β is quite high, resulting in a deterministic system. In a Boltzmann machine, β is set such that the dynamics become probabilistic. At a high level, this is a more powerful idea than the original Hopfield network, which is just used for storing exact patterns: the Boltzmann machine allows you to learn a distribution over data, and sample new patterns from the distribution.

Learning and updating

The Hopfield network has a well-defined learning rule to store patterns in weights T. From the Boltzmann distribution, they derived the corresponding learning rule

where < > refers to the average over data points, clamped refers to the states set to some input data, and free refers to the states sampled from P(s). There are two stages of learning: “wake” (clamped) and “sleep” (free). When it’s “awake”, it sees data and updates its weights proportionally to pairwise correlations in data. When it’s “asleep”, it samples from its probability distribution P(s), computes pairwise correlations, and does anti-Hebbian learning (due to the negative sign)1. Intuitively, we want to change the weights such that sampling states from the model’s distribution fits the observed data, i.e. when clamped - free = 0. When the model predicts something different from what’s present in the data, the weights are nudged in the correct direction.

There’s a clear problem with this algorithm, however, and it’s perhaps the reason Boltzmann machines never really took off, at least in the same way deep learning has. Due to the Z normalization term, which contains an integral over all possible states, sampling from P(s) is challenging. The way this is normally done is via Gibbs sampling; as s is a vector, this method samples from a series of conditional probabilities, one dimension at a time, to eventually approximate the joint distribution. Writing out the sampling process, you can rearrange terms to eventually get

which looks almost like the update step for a Hopfield network! But rather than a deterministic thresholding function, we now have a sigmoid function σ that outputs values in [0, 1], indicating a probability of flipping the value, dependent on β and a linear combination of all other values. Remarkably, starting from Gibbs sampling, they arrive at the same flavor of update rule as a deterministic model. Furthermore, just as in a Hopfield network, this is a local learning rule requiring only pairwise connections, and could potentially be implemented in biology.

Hidden units

In the basic setting of the Boltzmann machine, the states serve as both input and output. For a more powerful model that can capture higher order structure (more than just pairwise correlations between units), we add latent “hidden” units (white); visible units are outputs.

For the clamped stage, the hidden states are sampled according to P(s^h|s^v) (because there is no data to clamp hidden stages to), and for the free stage, both hidden and visible states are sampled from P(s). Although connections are still only pairwise, marginalizing over hidden units effectively lets us capture higher order structure. Suppose you had one hidden unit s^h connected to three visible units s^v1, s^v2, s^v3 (not shown above). s^h activating would increase the probability that s^v1, s^v2, s^v3 are active together.

Sadly, what’s powerful about the hidden variable model is also what makes it intractable: sampling in the clamped stage becomes very slow due to the hidden units. For each hidden state sample, you need to run a whole Gibbs sampling chain, which must finish before you can sample again. In this form, Boltzmann machines of a nontrivial size are almost impossible to train.

Restricted Boltzmann machine

To address the tractability issue, the Restricted Boltzmann machine (RBM) retains the hidden units, but removes connections between units of the same layer.

The resulting energy function has far fewer dependencies; each hidden unit is only conditioned on a set of visible units and vice versa. This means that we can now sample all hidden units in parallel, without having all the serial dependencies of the original model. This dramatically speeds things up, and allows application to real tasks. For example, Hinton & Salakhutdinov 2006 introduced a “deep” version by stacking RBMs to perform dimensionality reduction via autoencoding. To initialize the network parameters, they trained several small networks separately and connected the outputs of one layer to the inputs of the next. After this pre-training, they used backpropagation to finish fine-tuning the network, and could also perform classification using the pre-trained network.

Don’t listen to your advisor?

Later, going against Hinton (their advisor)2, Alex Krizhevsky and Ilya Sutskever dropped the pre-training part and just did backprop in AlexNet. It worked better than anything else and led to deep learning as we know it today.

The messaging of Hinton’s recent Nobel Prize seemed to suggest that success in deep learning came about as a result of the RBM. In reality, it’s more like it happened in spite of the RBM. This is not to detract from the scientific significance of the RBM or Hinton and Sejnowski’s work, and it may be that the great breakthroughs of the RBM for AI are still yet to come. But the morals of the story are perhaps that 1) great discoveries often result from disobedience, and 2) scientific innovation does not always translate to real impact.

I skipped over details for the RBM, but see Hinton and Sejnowski’s chapter in Parallel Distributed Processing. In the next post, we’ll cover the final main topic of the course: computing with high-dimensional vectors.

You probably got that one, but the next one is harder. If you can’t see it, scroll to the answer at the bottom and come back!

In 1867, Hermann von Helmholtz introduced the idea of visual perception as unconscious inference: that we perceive and understand the visual world immediately and subconsciously. Illusions are interesting because our perceptual deviations from reality give us hints about the inference process. Maybe you stare at the image for a long time, and try to guess what different blobs could be. Maybe it takes knowing the answer, as is usually the case for the second image, to actually make sense of it.

Here’s another example you probably know: we see A and B as different shades, despite identical pixel values. Given the regularity of the grid and the fact that B is in the shadow of the cylinder, we infer the information that B is lighter than A. If all we were doing was processing pixel values, we would perceive A and B as the same color.

Another example is the McGurk effect, which demonstrates the interaction between vision and audition. A different video is enough to skew our perception of the consonant being spoken, even though the audio is the same. If language were only dependent on sound, they would sound identical. But our inference process depends on both sound and vision.

There are endless fun examples (e.g. see the Illusions Index), but how do we formalize perceptual inference? If we have good models of human perception, should they be fooled in similar ways by illusions1? One approach I’ll go over briefly in this post is Bayesian inference and probabilistic models.

Denoising via Bayes’ rule

This is Bayes’ rule (sans normalization term), where H = hypothesis and D = data.

It says that the posterior, the probability of your hypothesis about the world being true given the data you receive, is proportional to the likelihood, the probability that you see the data given your hypothesis, multiplied by your prior knowledge about what the hypothesis could be. There are tons of tutorials online explaining Bayes’ rule (and here’s a more detailed handout by Bruno on Bayes and generative modeling), so I’ll just go through an example application that also relates nicely to natural images statistics: removing noise from an image.

It had been common practice in the television industry to perform coring: noise reduction by removing high-amplitude, high-frequency components from a signal. Previously, this operation was a bit ad-hoc, but Simoncelli & Adelson 1996 showed how to calculate the optimal coring function for a signal using a Bayesian derivation.

Given a simple formulation y = x + n where y is the given noisy signal, x is the clean signal, and n is some Gaussian noise, they wish to recover x̂ , an estimation of x. For example, these clean and noisy images.

Rather than taking x to be the raw pixel values of the image, they perform a 2D steerable pyramid wavelet transform on the image. If you convolve an image with an oriented, bandpass filter, you get a set of coefficients that express roughly the presence/strength of that feature present throughout the image. Doing this with a whole set of filters with different scales and orientations gets you a bunch of coefficients that serve as a summary of the image. In this paper, x, x̂ , and y refer to the coefficients from this transform, not the image pixels. For each filter, they get a distribution of coefficients that looks like the ones below.

We know that natural images do not follow a Gaussian distribution; they look like the left one, with most coefficients at or close to 0 (sparse) and with tails decaying more slowly (high kurtosis) than a Gaussian2. Given this prior knowledge about the images, and assuming that the noise is Gaussian, they want to find a way to go from a distribution with a shape like this

to the original left histogram. In Bayesian terms, they would like an estimator that has a high probability of producing the clean signal x given the noisy signal y. Suppose they want to minimize the mean squared error of the cleaned up image. Then,

which requires P(x|y), which can be calculated via Bayes’ rule:

All that’s needed is P(y|x), which is just P(n) = Normal(0, σₙ) (due to n = y - x), and P(x), which they approximate with P(y).

If one assumes the signal and noise are iid Gaussian with mean 0 and variances σₛ and σₙ, the closed form solution corresponds to a Wiener filter. But since natural images have a peakier distribution, they instead use the Laplacian distribution, and use the noisy image to estimate the distribution parameters for the clean image. See the paper for details, but on a high level, they derive an optimization problem to get coefficients x̂ for each wavelet, then convert back into the cleaned image in pixel space.

The optimal estimator they get out looks like the curvy line below:

Reading out the vertical axis, the coring function pushes small y values down and keeps larger ones approximately the same. It’s called a coring function because you can imagine roughly removing (setting to zero) the “core” (values close to zero), and it induces a distribution more peaked at 0, as desired. Applying it to the noisy image coefficients gets you the estimation of the clean image coefficients. This Bayesian approach shows a nice way to use the statistics of natural images and an assumed noise distribution to clean up an image.

Probabilistic formulation of sparse coding

Here’s another application of Bayes’ rule, applied this time to sparse coding, which I introduced briefly in a previous post. It turns out you can formulate the standard sparse coding energy function probabilistically3. Let’s just focus on the inference step, i.e. determining the sparse coefficients for an image given a dictionary (set of filters). It’s equivalent to finding the most probable configuration of coefficients a given some images I and dictionary Φ. We write the posterior probability using Bayes’ rule, in black:

In practice, we formulate inference as a minimization of the negative log probability, which results in the red equation. The first term |I-Φa|² comes from taking the log of P(I|a; Φ), for which we assume a Gaussian distribution; it’s high if the reconstructed image is different from the actual image, corresponding to the (negative) probability of whether the image I was generated from the given a (remember that sparse coding is a generative model). The second term is the prior P(a), assumed to be sparse based on our knowledge of natural image statistics (e.g. left histogram above). The sum over C(aᵢ) corresponds to the sparse image prior, where C() induces sparsity in a (assumed to be independent) and is chosen to be something like the absolute value or the Cauchy distribution. The standard method of solving is to use maximum a priori (MAP) estimation, which simply takes the mode of the posterior distribution4. I apologize for the whirlwindy explanation, but more details can be found in Olshausen & Field 1997. And if you want more details on perception as inference in general, Bruno wrote a book chapter5.

Okay, I lied last time when I said this post would cover the Restricted Boltzmann machine. I got too wrapped up in sparsity… the next post will cover RBM!

The first image in this post is a Mooney face. The second is a cow that’s often used as an example of “top-down” information influencing perception.

5

Also:

In October, John Hopfield and Geoffrey Hinton won the Nobel Prize in Physics for “foundational discoveries and inventions that enable machine learning with artificial neural networks”. Hopfield was recognized for his theoretical model of associative memory based on the Ising model from physics, and Hinton for a generative, probabilistic neural network1. Though there is no doubt both scientists have contributed significantly to their respective fields, the decision seems to imply that physics-based models like the Hopfield Network and Restricted Boltzmann machine have heavily influenced current ML/AI. This is not true, to my knowledge (if you believe otherwise, I’d like to hear it)2. A different take is, of course:

That said, Nobel Prizes and this weird worship of single scientists doesn’t make much sense to me. Science is only possible through collaboration and building off others’ work3, not to mention all the other factors that allow us to be in the privileged position of doing science in the first place.

Regardless of their relationship to AI, we can learn a lot from these elegant models. Attractor networks (this post) and probabilistic models (next post) are central to theoretical neuroscience.

The brain is highly recurrent. We saw this diagram of cortex and its wirings in a previous lecture:

The connections between cortical areas, represented by the lines, go in both directions. Complex dynamics govern the recurrent interactions between brain areas. In 1982, Hopfield brought ideas from statistical physics as a starting point to model these relationships.

Given a dynamical system, which is a set of variables that change over time according to some equations, we can analyze its state, or the setting of its variables at a moment in time. An attractor network is a dynamical system whose variables converge to a particular state (or a set of states, as we will see later) as time goes on.

The Hopfield Network

An associative memory is a system that takes content as input (as opposed to a memory address), e.g. image pixels, and retrieves the most likely match from stored memories. The Hopfield Network is a recurrent dynamical system that returns a stored pattern given some input pattern that could be noisy or incomplete. The below example shows a noisy binary input and running the dynamics until the correct pattern is recalled.

Implementing an associative or content-addressable memory is a hard problem as the number of patterns increases, but humans are able to robustly recall patterns in this way. For example, if you hear a snippet of a song you know, even if it’s in a noisy environment or a bad karaoke, you can sing the part that follows without much effort. While we do not know how the brain does it, we start by analyzing a simple recurrent system.

The basic Hopfield Network consists of a set of units V that take on values of either -1 or 1. They have all-to-all connections in both directions defined by symmetric weight matrix T, which is where the patterns are stored.

To store, we follow the weight update rule

where α indexes the patterns. For each pattern, this forms a basin of attraction that the values are pushed toward. Note that we store multiple patterns in one weight matrix by simply adding the unit value products (superposition).

Given a set of weights where patterns have already been stored and some initial unit values (input), we select a unit and update its values according to the rules

We then move on to another unit, and repeat until convergence, i.e. when the values stop changing. At this point, we have recovered the stored pattern that is the most similar to the input. But wait, you might think (as I did initially), with these equations, couldn’t the units just flip their values back and forth forever without converging? It turns out the answer is no.

To the system of neurons, we can assign an energy, which we would like to minimize over time. We can prove that Hopfield Network dynamics will always maintain or decrease the energy, and converge to a stable, low-energy fixed point4. Intuitively, we know this energy will be lowest when the product of unit values matches the connection between them. Since the memories are stored in the weights, the units will change values until they all correspond with the weights as well as possible.

Although the dynamics are guaranteed to converge, they are not guaranteed to retrieve the correct pattern because of the way memories are stored. Each network has a capacity for how many memories it can recall accurately, which is ~0.14N where N is the number of neurons in the network. If the patterns are very similar, then this number becomes even smaller. Bruno has a nice handout on attractor networks that goes much more in-depth than I do here, and there are tons of comprehensive tutorials on Hopfield Networks out there, e.g. Bhiksha Raj’s lecture, that can help give more intuition to how these dynamics work.

In the context of neuroscience, an obvious question arises: does the Hopfield Network map onto an actual mechanism in the brain? Not that I know of, but it’s a nice starting point for learning about recurrent neural models. As for the AI side, there are some connections between Hopfield Networks and current machine learning algorithms, such as the attention mechanism in transformers (which are behind LLMs) via the modified Modern Hopfield Network5. But as far as I know, this post-hoc connection was formulated after the success of transformers, and the Hopfield Network has not really played a part in current AI.

A ring attractor for head direction

Here’s an example of an attractor network that we can relate more concretely to a real biological mechanism. Rats have cells that are selective to a specific head direction (an angle from 0 to 360) with respect to some external reference point.

In 1996, Kechen Zhang proposed a ring attractor circuit and dynamics to simulate this head direction-tuned cell activity. Others had previously suggested similar circuits (e.g. Skaggs 1995), but Zhang concretized it with math.

Given a set of neurons, each with a different preferred head direction θ₀ (i.e. each one has a peak like in the above plot, but at a different angle), how can we connect them with some weights and dynamics such that there is a stable “bump” in neural activity that encodes head direction? In the below simulation, 64 neurons are each initialized to some random firing rate. Over time, the dynamics cause convergence to a stable bump, where the peak indicates the neurons that are the most selective to the current head direction. Note that this not the same bump as the above plot; we are modeling the population dynamics of recurrently-connected neurons, not the tuning of a single neuron.

Contrasting with the point attractor dynamics in the Hopfield Network, a ring attractor allows us not only to have multiple stable points of attraction, but also gives us a way to move smoothly between them. Because we are modeling an angular variable, head direction, it is natural to use a ring topology (once we hit the end, we return to the beginning). What if the rat is turning in space? We would like the movement of the bump to reflect this, i.e. shifting the bump corresponds to the rat’s rotation. Below is the model population activity as the rat is turning at a constant velocity in one direction. Despite shifting position, the bump maintains its shape.

The dynamics for the stable bump are quite straightforward. Below, u is a vector that represents the membrane potential of the units, u̇ is the population change over time, and v is a vector of firing rates, produced from a point-wise nonlinear function σ of u. ∗ indicates convolution. w is a vector of weights, which we must define to induce this stable bump structure.

It turns out the correct w looks like the below left: for one unit, we want its neighboring units to be excited, and ones far away to be inhibited. More generally, for a whole set of neurons, we get the matrix on the right, which is symmetric and structured (specifically, a Toeplitz matrix6). This structure lets us formulate the dynamics with the circular convolution above. I won’t go into the derivation of w here, but it’s all in the paper.

To shift the bump, we can take the spatial derivative of the weight vector, shown in the middle row of A below. This basically says to excite neighboring cells to the right and inhibit those to the left, and add it to the original weights. The γ determines how quickly the bump is moved. The derivative helps preserve the bump shape as it’s shifting: this causes movement within the basin of the energy landscape. In a biological circuit, these added derivatives could be inputs from the vestibular system, for example, that indicate to the circuit that the animal is changing head direction.

The head direction circuit is a beautiful example of bridging theory and experimental discoveries. Zhang and Skaggs didn’t originally propose a physical configuration of these units, only a connectivity pattern via the weights. But in 2015, Seelig & Jarayaman showed that head direction activity in the fruit fly ellipsoid body — a donut-shaped structure in the brain — is literally organized on a ring. Below, the bump of neural activity on the upper left is shown via calcium imaging; on the bottom right, the blue trace shows the actual head direction, and the red trace shows the head direction decoded from neural activity.

It’s striking that theory and biology arrived at similar solutions for the dynamics of head direction cells, hinting at some fundamental principles that govern nature.

Also not too long ago, experimental studies (e.g. Green et al. 2017) showed that the bump-shifting mechanism in fruit flies involves asymmetrical connections from different populations, which could connect to Zhang’s formulation. Zhang also notes that one could also extend the 1D dynamics to a 2D sheet; there is a lot of very exciting ongoing theoretical neuroscience work (e.g. Burak & Fiete 2009) on attractor network models of 2D grid cells.

There are clearly many exciting applications in neuroscience for attractor networks, and I regret that I’m really not able to do the topic justice in this post. But we’ll move on in the next post to probabilistic models, and touch on the Restricted Boltzmann machine.

1

Interestingly, Hinton is not a physicist.

In the below cartoon, s and t are 2D points, each representing an image containing A. The most natural transformation between these two As is horizontal translation, where the midpoint between the two should correspond to an A in the middle (top point). If we were to simply linearly interpolate between the pixels of s and t, it would look like the bottom image, a superposition of two As, which is not a valid shift. The red line is the manifold of As shifted along the center horizontal axis of the image. Possible points can lie anywhere in 2D, but all possible horizontal translations lie on a (specific) line, which is 1D. Note also that the position of the point on the red line also tells you the horizontal position: it explicitly parameterizes amount of shift.

Formally, a manifold is a topological space that is locally approximately Euclidean. As I am not a mathematician, I hope you’ll forgive me (and many others) for using manifold here more as a conceptual device (with selected mathematical tools) to help reason about the geometry of data that lie on some high-dimensional space. In 2D, manifolds include lines and circles (1D structures embedded in 2D space), and in 3D, surfaces such as planes, spheres, and toruses (2D structures embedded in 3D space).

What about real data? If you had a set of natural images each of size p pixels, the manifold of natural images would be n-dimensional, where n < p. That is, if you randomly selected p values for an image, this would probably look like noise, and would unlikely fall on the n-dimensional natural image manifold. “Walking” on this manifold would result in natural images undergoing transformations you would expect to see in the real world, such as local translation. Fun fact: Carlsson et al. 2008 showed that the distribution of 3x3 natural image patches parameterized by oriented edges forms a 2D Klein bottle manifold.

Going back to a toy example, shown below is the classic Swiss roll problem, where the manifold is a 2D sheet rolled up in 3D. It’s not immediately clear how you would “unroll” this manifold into a flat 2D sheet. Roweis & Saul 2000 developed local linear embeddings (LLE)1, an algorithm that finds weights to approximate each point as a linear combination of its local nearest neighbors (i.e. in 3D), and then uses those weights to define neighboring relationships in a new low-dimensional (i.e. 2D) space.

While this algorithm unrolls the Swiss roll correctly, we are missing something crucial that the learned weights or final 2D representation don’t tell us: the structure of the original manifold. How might we learn this?

None of the representations we’ve discussed so far in the course, e.g. PCA, winner-take-all, or sparse coding, can capture manifold structure. In standard sparse coding, if you perform inference on frames of a movie, the coefficients at each frame are sparse as intended, but they change extremely quickly and incoherently. They do not reflect the persistence of features or smoothness of motion in the video. Suppose we had an object translating smoothy through space in a video; ideally, the corresponding sparse code should reflect this motion. Similar to the 2D case in the first figure, we would like our representation (sparse code) to parameterize something we care about, such as local position of some feature in the movie.

Inspired by Locally Linear Landmarks from Vladymyrov & Carreira-Perpiñán 2013, Chen, Paiton, & Olshausen 2018 developed the Sparse Manifold Transform, which performs sparse coding of high-dimensional points (resulting in localized features), and then learns a linear projection that induces flatness on a low-dimensional manifold.

I’ll leave the details for another time2, but this is a start in trying to learn and understand manifold structure, rather than just a mapping.

Apart from images, you could also consider manifolds in neural data space. DiCarlo & Cox 2007 proposed that the brain might perform visual object recognition by disentangling manifolds: each object has its own manifold in neural activity space, where movement along the manifold represents different transformations that the object undergoes, such as viewing angle, illumination, etc. The theory is that in early visual cortex, the manifolds are entangled and not separable; in later stages of the pathway, they become disentangled and neurons have explicit representations of object identity.

Though the term manifold is somewhat of a buzzword right now in neuroscience3 and ML, it’s a nice idea. But as far as I know, we don’t currently have specific theories for how the brain does this disentangling.

This post wraps up the unsupervised learning module of the course. Next post, we will discuss attractor dynamics, which includes Hopfield networks!

3

Source.

I found myself at a metal concert last week. I don’t like metal! But I went to see the opener Madeline Johnston, a guitarist and singer who goes by Midwife. I always feel clumsy trying to describe music with words1, but it’s fun to try; I could wander forever in her guitar textures, which range from abrasive overdrive to delicate riffs, accompanied by ethereal vocal effects. Her music envelops you unexpectedly, like a warm hug from a stranger. Devastation and serenity coexist. I’ll defer to the musician: she calls her music heaven metal. I think it’s a perfect descriptor.

I’m impatient and impulsive, and my musical taste reflects this. I tend to prefer lots of movement, rhythmic structure, and interesting tonality. Midwife is none of this! But after listening to her for the first time a couple of months ago, I couldn’t get her slow-burn melodies out of my head. It’s hard to pin down exactly why I like her music so much, but it must have something to do with the tension of tender melodies set against noisy textures. You can find peace, even if things are hard. It’s a patient, deep introspection that is the opposite of almost everything I listen to these days.

Killdozer, from her newest album, has been looping in my head nonstop. Though the song seems angsty at first glance, it is, at its core, peaceful. The warm chords and soft guitar just feel like a hug! And the lyrics deliver a calm acceptance:

All my songs are love songs
All my songs are blue
All my songs are about death
I live my life without you

Midwife pushes the normal boundaries of my listening, and forces me to slow down, in the best way. Her music gives me a chance to breathe, and to sit with complex emotions. It’s what art is about, I suppose.

By the way, I did stay for the main act, Blood Incantation. It was actually a lot of fun, though I did get a pretty nasty shin bruise from a guy who decided to mosh very far from the mosh pit. Which I guess is kind of par for the course?

I’ve never been a fan of metal because I cannot stand the screaming, and the volume of guitar stuff usually overwhelms me. But setting these aside, I found the rhythm and tonality super interesting: shifting, odd time signatures, dissonances (hey that’s the name of the blog!), and unpredictable scales. This is all stuff that I am drawn to normally! I grew up on classical music, and got super into contemporary classical music at one point2. There actually seem to be a lot of similarities to metal3. So maybe I’d be into metal if I could just get past… the way it sounds?

I can imagine this is how people who aren’t familiar with classical music feel: that it’s all just stuffy and boring. There’s certainly lots of classical music that is that, and I don’t blame anyone for thinking it, but there’s also plenty that isn’t. In the end, there are of course only two types of music: the good kind, and the other kind. Anyway, here’s some Blood Incantation (check out the transition at 3:00):

I love hearing crazy textures and rhythms live! I’m glad I got over myself and tried something new. Even if it meant getting kicked in the shin. And I hope everyone starts listening to Midwife.

Last post, we talked about the limits of representation learning in linear neuron models. In PCA, implemented by Sanger’s rule, we only consider pairwise correlations of inputs.

What happens if we want to look at higher-order correlations? A logical next step is nonlinear Hebbian learning. Suppose we now have a neuron with some nonlinear function f. i indexes the inputs x and corresponding weights w.

Our Hebbian update rule becomes

where j indexes all the neurons in the network. If you do a Taylor expansion, your weight updates are now a function of three- and higher-point correlations between units x_i, x_j, x_k… But what you get exactly of course depends on f.

Winner-take-all

How do we pick f? This depends on the structure of the data. Suppose our data come from discrete groups, or clusters, where we do not have ground truth labels for each point. For example, an animal identifying an odorant from olfactory input. Our network looks like this, where black arrows are excitation, and white arrows are inhibition.

We can take f to be the winner-take-all function. That is, y_i is 1 if the input has the highest inner product with the ith weight compared to other weights, and 0 otherwise.

E.g. for output unit ① to turn on, the inner product of its weights and the input would have to be higher than the every other unit’s weights’ inner product with the input. The learning rule is

which says that the cluster center moves toward the points that were assigned to it.

This results in a discrete clustering algorithm: given a dataset, assign each point to a cluster that contains other points similar to it. This is also a form of competitive learning, as seen literally in the winner-take-all function, meaning that neurons have the ability to turn other neurons off in the appropriate situation.

In an abstract neural circuit, you could think of winner-take-all as 1-sparse network: given some input, exactly one neuron fires in response. But there are obvious limitations to the winner-take-all algorithm. What if we can’t assume that each input corresponds to exactly what one neuron is coding? What if activity is distributed across a population of neurons?

Sparse coding

Suppose we could assign multiple clusters to each data point. We could use these assignments (AKA weights) to linearly combine different clusters to describe each data point. Said another way: rather than representing one concept with one neuron, we could represent one concept using a small set of neurons, where the rest of the neurons are silent. This is sparse coding.

In the below figure, we show a range of coding schemes, each with its own pros and cons. In a dense binary code, you can represent 2^N concepts where N is the number of neurons, but it is brittle: changing one unit can completely change the output. Plus, more neurons will be firing at once, which is less energy efficient, and it is nontrivial to decode information from the neural activity. On the other end of the spectrum, local codes are winner-take-all AKA a lookup table AKA “grandmother cells”1: for each concept, exactly one neuron fires. You can only represent up to N concepts, and if one neuron is destroyed, you can no longer represent the corresponding concept. But it requires very little energy, and can be decoded easily by a direct readout.

A sparse code is then somewhere in the middle: it is more robust to noise and perturbations, and can represent up to N choose K patterns, where K « N is the number of active neurons.

In his quest to relate sensory perception to neural activity, Horace Barlow proposed the concept of sparse coding in 1972 as one of five dogmas:

2. The sensory system is organized to achieve as complete a representation of the sensory stimulus as possible with the minimum number of active neurons.

Peter Földiák first created a neural learning algorithm for sparse coding in 1990. Bruno Olshausen & David Field showed in 1996 that maximizing image reconstruction and sparsity of neural activity results in learning localized, oriented, and bandpass features that resemble receptive fields in mammalian primary visual cortex2.

Why do these features look the way they do? Learning them can be thought of abstractly as an evolutionary process to develop optimal representations (i.e. receptive fields). They are adapted to the low-level statistics of natural scenes, which contain localized oriented edges3, and learning oriented features requires seeing correlations beyond pairwise4. Sparse coding provides a way to do this.

Mathematically, the goal of sparse coding is learn the features Φ and infer the neural activity a that minimize the energy function E:

Where x is the input image, and λ is a scalar that controls the strength of the sparsity penalty. The first term encourages reconstruction of the image, and the second encourages sparsity of the coefficients. Note that this is a generative model: we reconstruct the image using a linear combination of features

where ϵ is some random measurement or image noise.

Inferring the activity amounts to producing the sparsest coefficients that best reconstruct the image using the given features5. Unlike in standard machine learning, we use inference here to mean an iterative procedure that outputs relevant factors a given input data6. Although the generative model is linear, inference itself is recurrent and nonlinear: updating the activity of one neuron depends on the activity of all the neurons in the previous time step. Because of the sparsity constraint, this can also be thought of as competitive learning7.

Sparse coding is an unsupervised feature learning algorithm that has lots of advantages to previous methods we discussed. Outside of vision, there has been experimental evidence for sparse coding in other systems in other animals (see Olshausen & Field 2004 for a review). But there are of course limitations. A clear one is the inability of a sparse code to capture spatial transformations, or more abstract similarities. The next post will focus on one method that starts to address this: manifold learning.

4

For example, PCA on natural images (considering only pairwise correlations) results in non-localized and generally non-oriented features.

7

The concept of explaining away is very important in sparse coding inference, and separates it from feedforward filtering algorithms that also produce activity given a set of features. On the left, we see two receptive fields representing two cells, overlaid on an edge. For feedforward neurons (the output is just the inner product between the receptive field and the pixels underneath), the tilted cell would have only a slightly lower firing rate than the upright one, because both receptive fields match decently well with the edge orientation and polarity. However, in sparse coding, the tiled cell would be suppressed by the upright cell, because the upright cell is better aligned with the edge. This suppression also results in a sparser representation.

Competitive learning and explaining away can be better explained through LCA.

After sensing and coding, the brain must reformat information for behavior in the world. For example, to act on an object, the number of photons hitting each rod, or the decorrelated activity across photoreceptors is probably not that relevant. The brain may, however, need to know how close the object is, or which direction it’s moving. Information is reformatted into representations1 that make relevant variables explicit.

In the mammalian brain, we know that primary visual cortex (V1) represents low-level features such as localized oriented edges. But to forage for food, or identify the face of a friend in a crowd, more abstract and complex concepts are needed. How are these representations formed, and how can we understand what they are? If we were to build a brain, what would we put in each box in the cortical diagram below?

We are very far from answering these questions, but they serve as high-level motivation for this post and the next. I’ll describe several learning rules that can be implemented by basic neuron models, their resulting representations, and their limitations.

Rather than tackling the entire problem of visual perception, we’ll start small, with the model that Bruno says “is ironically still with us today”: the perceptron, designed for pattern recognition2 by Frank Rosenblatt in the late 1950s. Why ironic? We spent a few lectures talking about the ways in which neurons are not linear, but the perceptron is fundamentally a linear summing device that takes inputs from other units, multiplies them by some weights, and adds them together. The effectiveness of the perceptron unit lies in outputting a nonlinear function of that sum, which crudely approximates an action potential. This model is also the building block of all current machine learning, so there are a million tutorials and historical accounts out there. I’ll just focus on its limitations.

There’s actually nothing we can say about the limitations of the perceptron that hasn’t already been said. Marvin Minsky and Seymour Papert basically wrote an entire diss track in 1969. But for our purposes, perceptrons have two major weaknesses. First, they require labels, or a teaching signal, to learn the weights mentioned above. We call this a supervised learning problem. For example, if you want to train a model to identify objects in a supervised fashion, you must know the true identity of each example in the training set. Is it reasonable to assume that for any problem we want to solve, in machine learning or in nature, we have explicit ground truth information for every example? Probably not. Second, while the perceptron has nice formal guarantees for labelled data that are linearly separable, these don’t hold for non-linearly separable problems (of course, you can transform your data into a space where it is linearly separable). In the above right figure, imagine if you had a + point to the right of the - points; as this dataset is not linearly separable, you would not be able to find a single line in 2D to separate all + from all -. Unfortunately, pretty much all the problems we care about do not have labels, and are not linearly separable. Very sad!

Even with all its issues, the perceptron is a good starting point because it sets up the problem of learning, or how to find the best representation for a task. Given a set of points and labels, how do we arrive at the correct w? It turns out it’s just basic calculus: assign some kind of differentiable score function to the model, and change w iteratively such that it increases the score. Gradient descent is still the most effective learning algorithm in practice today. There are also a million tutorials on this, so I won’t go into it here.

Other than the tenuous connection between a perceptron and a real neuron, how does this all relate to the brain? Is there a “teaching signal” that helps us modify our synaptic weights? Is gradient descent implemented in the brain3? Up until this point in the course, most concepts have been grounded in physiology or experimental evidence: biophysics, spiking neuron models, sensory and efficient coding. We are about halfway through (I am a few weeks behind with blogging), and the topic of representations marks the start of the more theoretical section of the course. That is, what are possible models and theories that could explain phenomena in the brain, based on mathematical ideas we may not yet have experimental proof for? These theories can then drive future predictions and experiments.

There is also clearly a big overlap between theoretical neuroscience, machine learning, and computer science, which is not coincidental: neuroscience, cybernetics, and computation have been intertwined since their beginnings. And the first well-known linear(ish) neuron model was not actually the perceptron, but the 1943 McCulloch-Pitts neuron, which also inspired early computers4.

I’m rambling, but there is so much to say! We are also rapidly approaching my research area, so I have to restrain myself. Let’s examine other types of lesser-known early models beyond the perceptron, starting with an unsupervised learning method, which does not rely on ground truth labels. Principal components analysis (PCA), which decorrelates data, is one such method. Below, the original data contain correlations: as x values increase, so do y, and vice versa. After linear projection onto the orthonormal basis W, or the directions of highest variance of the data, the decorrelated data on the right no longer have this pattern.

What’s lesser-known is not PCA itself, but the fact that it can be implemented by a neural network using a local update rule called Hebbian learning, or colloquially, the concept that “neurons that fire together wire together”. Let’s assume a simple linear model with some synaptic weight. If two connected neurons fire in response to the same input, then they likely represent some correlation, and Hebbian learning says we increase the strength of the connection.

In 1982, Erkki Oja proposed an implementation of PCA using Hebbian learning, and Terence Sanger generalized this in 1989 for a network with multiple output units. Given a linear neuron

Oja’s learning rule is

where η is the learning rate. The first term in the parentheses tells us that w_i grows proportionally with the correlation between the unit’s output y and input x_i, which is just Hebb’s rule. The second term constrains the growth of w_i (specifically, to be unit norm).

This implements learning the first principle component in PCA. How? If you let a linear neuron follow Hebb’s rule, you can show that the weight vector w_i will grow in the direction of the first eigenvector of the data’s covariance matrix, AKA the first principal component. The problem is that the length of w_i will grow exponentially, which is why Oja’s rule has the second term.

Sanger’s learning generalizes this to a network with multiple output neurons, i.e. multiple dimensions of PCA.

The first term below says we update weight ij proportionally with the correlation between input x_j and output y_i. The second term constrains the growth of w relative to other neurons’ correlations with the output.

I skipped over a ton of details, which you can read in Bruno’s explanation of the derivation here.

If we had a group of neurons implementing PCA, what would be the limitations of this representation? First, suppose we had datasets that had the same mean and variance in each dimension, but one had a Gaussian distribution while the other didn’t. PCA would arrive at the same solution for both datasets, because it considers only pairwise correlations, and produces orthogonal weights.

Unfortunately, most interesting problems are probably non-Gaussian. You’re probably starting to notice a theme: these models make a ton of simplifications, such as supervision, linear neurons, or Gaussian data. We must understand the limitations imposed by these assumptions, but we’ve got to start somewhere. We’d be in trouble if we couldn’t model Gaussian data!

We’ll continue in the next post with nonlinear learning strategies and their resulting representations: clustering and sparse coding.

On the way to the cortex, information from photoreceptors must go through at least two bottlenecks: 1) the conversion from an analog signal to spikes (discussed last time)1, and 2) the optic nerve, which has fewer fibers than the retina has photoreceptors. Analogous to how digital images are encoded to save memory and bandwidth, visual information must also be compressed. This is possible because of redundancies in the signals.

The statistics of natural scenes are quite consistent. If you measure the correlation of grayscale pixel values as a function of spatial distance, pixels that are close together look more similar on average. This makes sense, as they are likely part of the same object or texture.

Because of these correlations, you can imagine that it would be wasteful to have a wire from each photoreceptor down the optic nerve that conveys simply the value at a location. The plot above tells us that given a photoreceptor’s response, you’d already know a fair amount about the responses of neighboring cones.

If we look in the frequency domain, there is another type of correlation: the power spectrum of natural images drops off at a rate of 1/f². Each trace below is a different image, obtained by averaging over orientations. A decorrelated power spectrum would be flat, which is the signature of white noise.

We will illustrate the concept of redundancy reduction by defining whitening, how it’s implemented in the retina, and how it’s been implemented in silicon. These three ways also coincide with the principles of the course outlined at the beginning: mathematical explanations, observations from nature, and how to build a brain.

Theoretical

Suppose you have a 10x10 pixel image. You could plot the image as a point in a 100-dimensional space. Personally, I cannot visualize 100 dimensions, but we can simplify this to a hypothetical 2-pixel “image”, and plot in 2D such that each point with coordinates (x,y) in the plots below represents an image. Across the original images, there are redundancies. For example, looking at the leftmost plot, just knowing that a point has x=2 tells you something about y: it is likely between 0 and 2. Other points with x≈2 probably also have similar y values. In the rightmost plot, we now represent the same data transformed such that the y corresponding to x=2 is evenly distributed around 0. Knowing x does not give you much information about y, or about other points with x≈2. We call this decorrelated and independent.

Here are the whitening steps:

1. Decorrelate: take the eigenvectors of the covariance matrix of the data, and project the data onto these eigenvectors. This amounts to rotating the data such that its main axes are the ones of highest variance.

2. Sphere/whiten: equalize the variance in all directions (divide by the standard deviation) such that the distribution is decorrelated and independent. This is called whitening because if you think of the eigenvector decomposition as a frequency analysis, we’re equalizing power in all directions (which would result in flat power spectra in the earlier plot).

3. Rotate: rotate back to the original orientation to calculate weighted combinations in pixel space.

We can concatenate these linear operations into one matrix W, and plot each weight over the image distribution.

The first weight vector W1 transforms a point (x,y) by calculating a weighted difference between x and y (positive x, negative y). W2 does some amount of y-x2. In 2D, it’s hard to understand what this means beyond the operations. Let’s look at 10x10 natural images: W ends up looking like a set of center-surround filters.

This is a generalized version of the 2D case. Each filter is saying: take some positive amount of one pixel, a negative amount of the pixels around it, and sum those together. Rather than conveying the value at each pixel, which would contain redundancies, it performs a local difference operation. After applying these types of filters, what do actual whitened images look like?

Yay! We hypothesize that this more or less approximates signals being sent down the optic nerve. But how does the retina do this?

Empirical

For a cone in the fovea (the highest resolution part of the retina), there are corresponding “on” and “off” bipolar and retinal ganglion cells (RGCs send information down the optic nerve). They respond to deviations from the mean in the positive and negative direction, as in the schematic below.

Horizontal cells pool over groups of photoreceptors, and serve to inhibit photoreceptor activity relative to the local population. Combined with bipolar cells that drive the center of the receptive field, this gives rise to center-surround RGC receptive fields. These localized filters are also easily wired by the connections between photoreceptors, bipolar cells, horizontal cells, and RGCs.

Remarkably, we can arrive at the same whitening solution through another approach. Karklin & Simoncelli 2012 derive whitening filters by optimizing an objective function: maximize the mutual information between the input and the activity (first term) while penalizing unit activity (second term). X is a set of images, R is RGC firing rates. λ is a weight term that indicates how strongly to penalize activity, and the brackets indicate averaging over images.

The model is constructed as follows: to get a firing rate rᵢ, take an inner product of an image with the wᵢ, pass it through a nonlinearity fᵢ that forces values (firing rates) to be non-negative, and add some noise nᵢ.

If the goal were just to compress the image, W could be any set of independent filters. But with the extra conditions — 1) non-negative firing rates, 2) the noise floor pushing firing rates up3, and 3) the penalty pushing firing rates down — we end up with the center-surround filters below.

There is nothing in the model or optimization that explicitly says to whiten or use localized filters; they emerge from the imposed constraints, trained on natural images. When multiple approaches converge to the same answer, it’s usually a good sign. Statistical, biological, and information-theoretic methods all seem to arrive at this whitening solution!

Synthetic

Other than the retina itself, everything we’ve seen so far is a simulation of whitening, implemented on digital machines. Modern computing architecture is predicated on performing operations without error, with physical constraints abstracted away. But imposing digital precision on an inherently imprecise analog signal is costly. Misha Mahowald and Carver Mead, the inventors of the silicon retina, estimated (in 1991) that

the most efficient digital integrated circuits envisioned will consume about 10^-9 joule per operation, whereas neurons expend only 10^-16 joule. In digital systems, data and computational operations must be converted into binary code, a process that requires about 10,000 digital voltage changes per operation. Analog devices carry out the same operation in one step and so decrease the power consumption of silicon circuits by a factor of about 10,0004.

To build a retina, they construct a hexagonal grid of pixels where each pixel has analogues of a photoreceptor, horizontal cell connections, and a bipolar cell. The key is that each pixel performs a difference operation between the photoreceptor input5 and the circuit average at that location: the resistor network (horizontal connections) calculates the local average voltage, and the amplifier (bipolar cell) outputs the local difference. The voltage at each pixel represents a spatially-weighted average of photoreceptor inputs, where the weighting decays exponentially with distance, like in a network of real neurons modeled by a cable equation. Beautifully, the response of this synthetic retina approximates the human retina (local difference = center-surround AKA whitening), and all of these computations are done at low power, with physics!

For more details, see Chapter 15 of Analog VLSI and Neural Systems.

Phew, this post got way longer than I expected. I think you’d need at least a month to do this topic justice. Sadly, I have some deadlines coming up, and I’ll need to pause the lecture posts for a couple of weeks. We will resume with types of representation learning and their limitations.

3

Decreasing the noise results in more oriented, less localized filters.

Last lecture, we saw how spikes are generated from a fluctuating membrane potential. But why do neurons send discrete spikes instead of continuous voltage? Over long distances, physics tells us that voltage decays exponentially. To maintain the signal, it must be amplified as it travels down the axon. The solution is to send strong signals in the form of action potentials, i.e. spikes.

There are two primary ways to convey information through spikes: a time code and a rate code. Which is the brain using? This is one of the oldest debates in neuroscience, and when you have very informed people on either extreme, the answer is usually somewhere in between, or both. In this post, we’ll take a whirlwind tour through evidence for both sides, and also talk about oscillations (which are… kind of both).

Last time, we described the deterministic leaky integrate-and-fire (LIF) neuron. Although you could still measure a firing rate of such a neuron, information is conveyed in the precise timing of the spikes. Said another way, the LIF would have exactly the same output across different trials of the same experiment. In contrast, the linear-nonlinear Poisson neuron (LNP) generates spikes through a random Poisson process, with nondeterministic timing. Information is conveyed, by design, through the spike rate. For the LNP, different trials would all look different, but their spike rates on average would be about the same. Of course, these neuron models each have various caveats, but we use them to illustrate the opposing regimes.

In physiology, the classic example of single cell recordings comes from the 1960s experiments of David Hubel and Torsten Wiesel in cat primary visual cortex. Neurons are tuned to orientations of bars (A), meaning that there is a tuning curve (B) of firing rate as a function of orientation angle. This supports the rate coding hypothesis. It looks random: if you did this trial again, there would be a different set of spike times with the same firing rate on average.

An implication of the rate coding hypothesis is that there must be noise in the system, shown here by the data points deviating from the fitted curve at higher firing rates. If we model spikes with a Poisson process, the variance should be equal to the mean. Lo and behold, in the middle temporal visual area of macaque monkeys shown movie stimuli, this was found to be roughly empirically true.

In the fly H1 neuron, which responds to horizontal motion, this Poisson relationship was also found in response to a stimulus of constant velocity. However, what if we have a stimulus that varies velocity across time? Suddenly, the the relationship is no longer Poisson. The spike patterns appear to be timed with specific changes in stimulus velocity or direction. This is not a rate code.

In a rat cortical neuron, spike timing response to an injected constant current (left) appears to become noisier as time goes on. The firing rate seems about the same, but the timing looks random. But when a time-varying signal is injected, the spike timing is synchronized to fluctuations in the signal (right). Unlike the time-varying signal, the constant signal gives nothing for the neurons to synchronize with, so their fluctuations in timing are due to other processes.

The results from this experiment should make you uncomfortable, Bruno says. When people saw these results, it certainly had that effect. There were mixed reactions: some people took the left response as confirmation that spiking is governed by a random process under some regimes. Others took it as evidence that spike timing is deterministic when there is signal driving it, and that the seemingly random responses are a result of other processes in the neuron not quantified by us. We don’t have access to the state of the neuron, and we can’t observe all the variables that generate spikes.

Based on studies like these, the answer to the time vs. rate question is unsatisfying. Sometimes, things appear to be random. And sometimes, they appear to be deterministic. We probably need to look at both the timing and the rate?

Here’s an example of this. The nerve fibers in the auditory system seem to be phase-locked, which kind of codes both timing and rate simultaneously: the neurons fire at a specific position in a cycle of the incoming signal (hair cells wiggling). Each row below shows a histogram over many trials of the spikes from an auditory nerve fiber. The rows are ordered by the frequency the fiber is the most sensitive to, determined by the corresponding position on the basilar membrane1. The punchline here is that each nerve fiber fires when there is energy in the signal at its characteristic frequency, but the timing of the spike also reflects the signal frequency!

In addition to the oscillations, the frequency information is transmitted two ways: in the tonotopic code (where the neuron is on the basilar membrane tells you the frequency), and in the phase-locked spike times. Because the tuning of the cells — the frequency they are most sensitive to — is relatively broad, there is some ambiguity to the tonotopic code. A cell could fire in response to 500 Hz, but it could also fire to 510 Hz. The exact timing of the spikes gives much more precision. Downstream processes can then take advantage of either, or both of these coding strategies.

This is not the only place we see oscillations. In the lateral geniculate nucleus (LGN), which sits between the eye and visual cortex in the visual processing stream, intracellular recordings reveal spikes from retinal ganglion cells. The retina was found previously to generate its own oscillations, separate from external signals2. The spikes in A below appear to be noisy, but the authors hypothesized that it was because the stimuli were out of phase with the retinal rhythm. To test this, they shifted all trials to be aligned in phase with the retinal oscillations (B). The spike timings became very precise, and what’s more, they found that there was more information encoded when looking at the timing as opposed to just the rate. Knowing the underlying process behind this initial apparent “noise” revealed that it was not noise after all.

Okay, I’ve just rambled on about a zoo of spike encoding schemes. But how do we determine how much information is actually contained in these spikes? A schematic of the decoding process is shown below, where a signal s(t) is encoded into spikes ρ(t), and then decoded into a reconstruction of the signal ŝ(t).

This is generally a hard problem, because we are trying to reconstruct a smooth, continuous signal from discrete events. The encoder is highly nonlinear, as we saw in the last post, but using a linear decoder is actually a non-absurd assumption3. It turns out that by convolving a kernel with a set of spikes, you can arrive at an approximate reconstruction. Below, we get “on” and “off” spikes via a LIF neuron on the signal and the sign-reversed signal. Then, we convolve the on spikes with the kernel, and the off spikes with the negative kernel. You could also do this with only on spikes and a different kernel.

By showing this decoding procedure, we are not implying that the brain literally reconstructs the signal somewhere downstream of the spikes. This is just one way of quantifying the information contained in the spikes that can be utilized in subsequent processes.

We can also do this using information theory. I won’t go into detail here4, but the remarkable thing about this analysis, applied across many systems and organisms, is that you sometimes get on the order of 2-3 bits of information per spike, rather than just 1 bit, as you’d expect from a binary signal. How? The key is that when the neuron is not spiking, there is also information conveyed; it’s the pattern of spikes over time that matters.

Personally, it’s obvious after seeing all these examples that there is no single answer to the timing vs. rate debate; there is clearly a diverse set of coding schemes dependent on the type of computation being performed. Putting this aside for now, the next lecture will get into coding more abstractly with one example: what is the most efficient way to encode natural scenes based on their statistics, and how does the retina do it?

3

You can see motivation for this assumption in the joint distribution of spikes and signals: the decoding marginal distribution is roughly linear in number of spikes.

4

For more details, the Spikes textbook describes this well! On a high level:

I am very much still learning how to explain technical stuff in this format, so please bear with me! This post kept getting out of hand so I kept it high-level and moved some details to the footnotes. Let me know what worked and what didn’t 😬.

The word “nonlinearity” evokes something specific in the context of artificial neural networks. The classic example is a rectified linear unit (ReLU), which is a function that maps non-positive values to 0 and leaves positive values unchanged1. The initial motivation for using ReLU was to simulate a firing rate (which cannot be negative) of a neuron given some potentially negative signal2. But enforcing biological constraints on deep learning models has not really gotten us anywhere, so the main motivation from machine learning nowadays seems to be that it just… works? (many such cases)

That we use “nonlinear” to refer to these functions reveals that the default operations in artificial neural networks are linear. Linearity is super convenient in engineering and math! But in this post, we’ll see why the brain, starting with its lowest level of computation, cannot be assumed to be linear.

In the last two lectures, we discussed sensory mechanisms that transduce signals into the eventual opening of ion channels. These channels allow the change in voltage, which drive the primary mode of communication in neurons: electrochemical signaling.

The cell membrane encloses the cell and selectively allows ions to pass through, enabling an electrochemical gradient. We typically think of the dendrites as receiving inputs, on which axons from other cells form synapses and change the voltage inside the dendrites. Current accumulates in the cell body, and when the membrane voltage reaches a certain threshold, an action potential (“spike”) is initiated in the axon hillock and is propagated down the axon3.

Let’s zoom into that little circuit diagram on the bottom dendrite to look at the membrane equation. You can think of this whole thing as a battery with a difference in potential (voltage) between the inside and outside of the cell caused by a difference in ion concentrations; the ion pumps serve as battery rechargers that maintain the difference4.

I’ve put this here for those who want to engage with the circuit diagram and its dynamics, but the main point can be made in words. The ΔGs, or changes in conductance, are input currents (channels opening due to synaptic inputs), which change the membrane potential V (the difference between the inside and outside). I’ll draw your attention to the first equation, which says that in addition to a baseline conductance, both the numerator and denominator are affected by conductance changes5. This means that the voltage, which eventually causes spikes, changes nonlinearly with any synaptic input to the cell; the fundamental unit of biophysical computation in the brain is nonlinear.

Shunting inhibition is another nonlinear phenomenon: when only the chlorine channel is open, it does not affect the membrane potential; however, when both chlorine and sodium channels are open simultaneously, the influence of sodium on the membrane potential is inhibited6.

So far, we’ve focused on the membrane potential. But neurons don’t generally convey voltage; they spike. The most basic model of spiking neurons, the leaky integrate-and-fire (LIF) model, is a modification to the membrane equation above, with the addition of a spiking threshold (V_th below), a reset to the resting potential (0 here), and a refractory period (τ^ref). The spikes are the signals sent to other neurons.

Another way of modeling spikes is with the linear-nonlinear Poisson (LNP) model7. A signal is first filtered linearly, then goes through a point nonlinearity, such as ReLU, and is then used as a rate parameter for a Poisson distribution, which gives us spike times. It is nonlinear, and unlike a LIF neuron, the spike times are nondeterministic8.

After seeing all this nonlinearity, if you still believe the brain can be well-modeled with only linear operations, or even mostly linear operations, Bruno will throw tomatoes at you 🍅🫵.

We’ve shown a canonical example of biophysical nonlinearity in the brain, and examples of basic spiking neuron models that are also obviously nonlinear. But how do spikes — a compression of an analog to pulsatile signal — convey information? Next lecture, we will talk about encoding and decoding.

5

To visualize, a basic example is f(x) = (1+x)/x.

Let’s talk about how the cochlea computes!

The tympanic membrane (eardrum) is vibrated by changes in air pressure (sound waves). Bones in the middle ear amplify and send these vibrations to the fluid-filled, snail-shaped cochlea. Vibrations travel through the fluid to the basilar membrane, which remarkably performs frequency separation1: the stiffer, lighter base resonates with high frequency components of the signal, and the more flexible, heavier apex resonates with lower frequencies. Between the two ends, the resonant frequencies decrease logarithmically in space2.

The hair cells on different parts of the basilar membrane wiggle back and forth at the frequency corresponding to their position on the membrane. But how do wiggling hair cells translate to electrical signals? This mechanoelectrical transduction process feels like it could be from a Dr. Seuss world: springs connected to the ends of hair cells open and close ion channels at the frequency of the vibration, which then cause neurotransmitter release. Bruno calls them “trapdoors”. Here’s a visualization:

It’s clear that the hardware of the ear is well-equipped for frequency analysis. Nerve fibers serve as filters to extract temporal and frequency information about a signal. Below are examples of filters (not necessarily of the ear) shown in the time domain. On the left are filters that are more localized in time, i.e. when a filter is applied to a signal, it is clear when in the signal the corresponding frequency occurred. On the right are filters that have less temporal specificity, but are more uniformly distributed across frequencies compared to the left one.

Wouldn’t it be convenient if the cochlea were doing a Fourier transform, which would fit cleanly into how we often analyze signals in engineering? But no 🙅🏻‍♀️! A Fourier transform has no explicit temporal precision, and resembles something closer to the waveforms on the right; this is not what the filters in the cochlea look like.

We can visualize different filtering schemes, or tiling of the time-frequency domain, in the following figure. In the leftmost box, where each rectangle represents a filter, a signal could be represented at a high temporal resolution (similar to left filters above), but without information about its constituent frequencies. On the other end of the spectrum, the Fourier transform performs precise frequency decomposition, but we cannot tell when in the signal that frequency occurred (similar to right filters)3. What the cochlea is actually doing is somewhere between a wavelet and Gabor. At high frequencies, frequency resolution is sacrificed for temporal resolution, and vice versa at low frequencies.

Why would this type of frequency-temporal precision tradeoff be a good representation? One theory, explored in Lewicki 2002, is that these filters are a strategy to reduce the redundancy in the representation of natural sounds. Lewicki performed independent component analysis (ICA) to produce filters maximizing statistical independence, comparing environmental sounds, animal vocalizations, and human speech. The tradeoffs look different for each one, and you can kind of map them to somewhere in the above cartoon.

It appears that human speech occupies a distinct time-frequency space. Some speculate that speech evolved to fill a time-frequency space that wasn’t yet occupied by other existing sounds.

To drive the theory home, one that we have been hinting at since the outset: forming ecologically-relevant representations makes sense, as behavior is dependent on the environment. It appears that for audition, as well as other sensory modalities, we are doing this. This is a bit of a teaser for efficient coding, which we will get to soon.

We’ve talked about some incredible mechanisms that occur at the beginning of the sensory coding process, but it’s truly just the tiny tip of the ice burg. We also glossed over how these computations occur. The next lecture will zoom into the biophysics of computation in neurons.

As we saw last week, animals’ ability to turn sensory information into behavior defines many of the computational problems we’d like to solve. How would one design a neural system to perform these computations? We’ll explore sensory coding with two examples: phototransduction in the eye (this post), and acoustic transduction in the ear (next post).

In every environment, we are constantly bombarded by electromagnetic waves. Anthropocentrically, we call waves at 400-700nm light1. But how do we convert light into electrical signals and glutamate release that eventually drives perception and action? Phototransduction in rods, which are photoreceptors in the retina used in low luminance conditions, illustrates how the brain is able to discriminate between relevant signals — photons from the world — and noise — spontaneous activation in a rod.

Although we think of light as waves, under low luminance, the retina senses light intensity by counting the number of photons. In moonlight, rods catch photons at a rate of about one per second. In darker environments, this rate has been psychophysically determined to be about one every 5000 seconds per rod in humans. To accomplish this, a rod must very reliably detect a single photon!

Inside a rod, rhodopsin molecules catch photons, isomerize (change chemical configuration), and initialize a biochemical cascade. Rhodopsin must be sensitive enough to detect one photon, but because strong collisions between molecules can also cause isomerization, they must be stable enough as they float around in the cell and bump into each other. When a collision does set off the isomerization, it appears to downstream processes identical to a photon catch. This is potentially bad!

Do these collisions happen often enough for us to care? For one rhodopsin molecule, the rate is about once every 700 years. Phew, you might say. BUT! There are hundreds of millions of rhodopsin molecules per rod, so this rate turns out to be about once every 160 seconds for a rod. Even worse, taking into account all the rods in the retina means that these events happen quite frequently. This noise is called spontaneous isomerization, often perceived as a scintillating pattern. And of course the Germans have a word for it: eigengrau, or intrinsic gray. A simulation of the eigengrau on a 300x300 patch of rods:

Here’s the remarkable thing: the one photon per 5000 seconds per rod threshold was determined from the conditions under which a subject can discriminate between actual light and no light with 75% accuracy. How can the threshold be this low if spontaneous isomerizations happen at a much higher rate of once every 160 seconds per rod? How is it possible that a tiny change in rate is enough for the brain to say: this is light!

Let’s express this slightly more formally. We can model the background and signal distributions as Poisson, with λ_b=1/160 as the background spontaneous isomerization rate and λ_s=(1/160 + 1/5000) as the signal rate. Given an event, the brain must determine which distribution caused it. If the two distributions are very separated like in the cartoon below, maybe this is not too hard. You could be correct a high percentage of the time by using the dotted line below as a threshold. Say you perceived x amount of brightness: if it was to the left of the line, you say there’s no real light. Otherwise, you say there’s light. Even if you move the line, there would be some tradeoff between true and false positives, but you would not be that wrong2.

In reality, the two rates only differ by about 3%. Even at a much higher rate of λ_b=10 (where λ_s also increases to maintain the 3% difference), the distributions are very close. At λ_b=100, they are a little bit farther. For both settings, using a threshold line to classify between the two distributions would result in a very high chance of being wrong.

Because the signal and background always differ by 3%, the goal is to increase the absolute distance by raising the rate such that the 3% is enough to get high accuracy by drawing a line. Qualitatively, it should look something like this:

Note that we cannot literally increase the 1/5000 threshold rate; this is a fixed property of the visual system. However, we could integrate over some number of rods and some period of time, i.e. count the number of events that occur in a spatiotemporal region, until we get an aggregated rate that achieves the desired accuracy. I will not give exact numbers in case future versions of this course use the same problem set, but it is a ton of rods and a surprisingly long amount of time3! Amazingly, the brain seems to have solved this problem: it is able to do these statistics constantly and as correctly as necessary.

This balance of sensitivity and stability continues to be important downstream of the rods. Bipolar cells in the retina must reliably detect a photon amongst a pool of rods. If just one rod catches a photon, there must somehow be a reliable change in the voltage of the bipolar cell. The challenge is, as Bruno says, to constantly fight the entropy of the universe, such as random fluctuations in synaptic terminals. If you were to linearly sum the noisy rod responses within each pool together, an individual photon event wouldn’t be detectable. The solution is to place thresholds at synapses, which results in passing only a real light signal (as much as possible) to the bipolar cell. What is the optimal threshold? Work by Sampath, Field, and Rieke4 calculated a theoretical optimal value based on the amount of noise at each synaptic terminal and the signal magnitude in response to actual light. Experimentally, they confirmed that the threshold was exactly where theory said it would be!

Perhaps this all seems like a convoluted, nonsensical system. But in the spirit of building a brain, the challenge would be: could you come up with a better mechanism that operates with such low energy, with such a high dynamic range, under these constraints? Really good cameras don’t have to work like the eye, but it’s still important (and fun, hopefully) to understand the principles underlying how biology solved this problem.

In the next post, we will see a solution to another problem: acoustic transduction.

2

I didn’t want to go into area under the ROC curve (bold black curve below) in detail, but at a high level, the 75% accuracy refers to the area under the curve generated by every location of the threshold line for the two distributions. Each threshold line has a different tradeoff between true positives (hits) and false positives, which each fall somewhere on the ROC curve. If the area under the curve is 0.75 (out of 1), then the distributions are appropriately far apart. As an example, assuming the distributions are far enough apart, the threshold line below corresponds to the dot on the curve.

Chapter 2 of Principles of Neural Design (2017) by Peter Sterling and Simon Laughlin opens with the question: why do we need a brain?

Introductory neuroscience courses generally start at the lowest level, with neurons, synapses, and action potentials. It’s like teaching someone how a car works by starting with spark plugs rather than how the engine makes the car move, or why you need a transmission. Instead of starting with the nuts and bolts, we begin this course as Sterling and Laughlin begin their book: by motivating the investigation into core principles that tell us what the brain is for. And if these principles are truly fundamental, they should hold true for all types of brains.

What we identify here as the brain’s purpose, especially because we are seeking principles, should apply not only to humans but as well to the nematode worm, C. elegans, and to flies. The deep purpose of the nematode’s brain of 302 neurons, the fruit fly’s brain of 105 neurons, and our own brain of 10^11 neurons (Azevedo et al., 2009) must be the same.

Much of the current AI industry likes to impose specific definitions of intelligence, like image classification and next-token prediction1. In our approach, we counter with these questions: Is that what the brain is designed to do? What problems is biology solving? Why did the brain develop anatomically distinct structures; why did we evolve ears and eyes?

Speaking of eyes, the optical properties of animal eyes are beautiful examples of evolution based on animal capabilities and environments2. An improvement to the pit eye (a), the pinhole eye (b) found in the Nautilus is still quite limited due to low retinal illuminance and low resolution. As a solution, various types of lenses (c-h) allow wider apertures and refraction. Some animals developed reflective structure to capture more light (i). And these are just single-chamber eyes! That nature has discovered mechanisms that parallel the optics in our engineered systems is incredible.

To examine behavior, we turn to simpler brains like those of C. elegans, fruit flies, and rats, but what we’ve learned is anything but simple. Some of the big breakthroughs in neuroscience relate to navigation in animals, such as the fruit fly ellipsoid body encoding head direction, and place and grid cells in rat hippocampus (we will go deeper into these topics later in the course). One of Bruno’s favorite examples is jumping spiders3. They have remarkable eyes that swivel back and forth to scan the environment. They stalk their prey, navigating in complex ways in 3D space. Jumping spider psychophysics (yes4) show that they perform object recognition. Supposedly, if you catch one and put it in front of a white computer screen and move your cursor around, it will try to pounce on it like a tiny cat.

Clearly, there is complex computation going on inside a jumping spider. Forget mammalian brains; we don’t even know how these simpler brains work! Until we do, it’s hard to make the case that we have made significant progress on discovering core principles. Surely we cannot discount what underlies these behaviors and structures in our studies of neuroscience and intelligence.

As interesting as animal behavior and biological structure are, this course is about computation, not evolutionary biology. Part of the reason the systems above are so compelling is that we can use mathematical language to describe them. This course will draw on elements of signal processing, statistics, calculus, linear algebra, attractor networks, and manifold representations, just to name a few. While mapping to math is useful, we should, above all, be open to embracing the complexity of biology. Bruno points out that maybe in studying these principles, we will realize we lack the appropriate language, and even develop new mathematical tools.

We’ve talked about animal behavior, but action is largely driven by sensory input. And it is not enough to just sense: signals need to be encoded in ways that are useful for action. For our first technical topic, the next lecture will dive into sensory coding by looking at the computations involved in phototransduction.

This semester, I’m TAing a graduate course called Neural Computation taught by Bruno Olshausen, which is probably my favorite class I’ve taken at Berkeley. I will try to write a short post each week with my annotations of that week’s lectures. It’s going to be a busy semester, so no guarantees, but I’ll do my best 🫡. All course content is on the website.

Throughout history, we have used our newest technologies as metaphors for the brain: hydraulic pumps, steam engines, and computers. I think this urge to project a known system onto the brain tells us how hard it is to reason about its complexity without a concrete foothold. In the context of this class, we are not saying that the brain literally operates like a computer. But we are using ideas of computation, e.g. logic gates, circuits, and encoding principles, to help us ask concrete questions about the brain.

In Bruno’s words, this course is about how brains work, and how to build a brain. These two ideas are intertwined and complementary. The Wright brothers were inspired by observations about bird wings, which also helped us learn more about how birds fly. If we agree with Richard Feynman’s quote “What I cannot create, I do not understand”, we won’t be able to build a brain without understanding it. But trying to build it may help us start to understand it.

With recent technological advances, it’s now possible to record from tens of thousands of neurons in the brain simultaneously1. How do we go from these complex, noisy data to understanding the principles the generate this activity? It isn’t sufficient to only analyze data in a “bottom-up” fashion. We need top-down concepts to inform how we interpret the data. As Horace Barlow said, “A wing would be a most mystifying structure if one did not know that birds flew.”2

As a side note, neuroscientist Konrad Körding made this point in a podcast earlier this year:

“For every neural dataset... there is an infinite set of biologically meaningful, potentially realistic models that will predict exactly that dataset. The neuroscience that we do doesn't actually inform the mechanisms that we want to talk about, which puts large branches of neuroscience into an epistemologically really difficult spot."3

Coming up with a model that predicts a dataset doesn’t mean you definitively understand some aspect of the brain. It’s a nice story to tell, but results are never so clean, and there are so many other possible explanations. Predicting brain activity is also only one type of modeling, which we call descriptive. Can a model that regresses to neural activity tell us something about the actual implementation in the brain, or the normative principles driving these patterns? These are all hairy epistemological questions that I won’t get into here. When it comes to trying to make sense of the massive amount of data and theories put forward about the brain, I like to think of the parable of the blind men and the elephant, where each blind man draws a different conclusion about what an elephant is based on which part they touch. Werner Heisenberg puts it another way: “We have to remember that what we observe is not nature in itself, but nature exposed to our method of questioning.”4 This is why neuroscience is so hard, and why we still don’t understand the brain at all!

Okay, back to the lecture. In the 40s-60s, there were several early efforts to understand the brain. The cybernetics approach tried to understand the mechanisms of the brain from the inside, such as proposing that the brain implemented logic gates, which ended up also contributing to the development of modern computers5. After the famous Dartmouth AI meeting, the organizers were motivated to design a machine to simulate the brain67. Neurobiologists David Hubel and Torsten Wiesel discovered orientation tuning in cat visual cortex8, which still largely dictates how we study visual cortex, and eventually led to the development of convolutional neural networks9. It’s clear that trying to understand the brain and trying to build one have been connected since the beginning of computation.

As Bruno says, nature hides its secrets well. But there is hope for discovering theoretical principles by looking to biology. Biology has already solved complex problems. Can we learn from it? This is one of the main themes of the course, and the next lecture will start by highlighting the rich world of animal behavior.

Millennium Mambo (2001) by Taiwanese director Hou Hsiao-hsien opens with a woman, ethereal, on a fluorescent-lit bridge at night. The music, by Lim Giong, starts with pulsing static and bell-like synths. A warm electric guitar enters, like a hug, accented with a steady kick drum, like a heartbeat. Then a man’s voice echoes in Taiwanese Hokkien: siān-liông (kind) / pîng-huân (ordinary) / khuài-lo̍k (happy) / tan-sûn ê lâng (a pure person). Artifice with sincerity, metronomic with flowing, new textures with an old language; it’s not obvious that these elements should all sound good together. Yet they do.

林強 (Lim Giong), a Taiwanese pop singer turned electronic composer1, has a style characterized by contrasts. Following classic Taiwanese 90s pop, his early music was sentimental melodrama influenced by Western disco and rock. Later, he moved toward electronica, and one of his first mostly-electronic albums (Insects Awaken, 2005) contains spoken and sung Taiwanese, Mandarin, English, and French, as well as sounds of mahjong tiles, a funk track, and his rendition of a Taiwanese folk song. He’s also composed dozens of film soundtracks for Chinese and Taiwanese filmmakers, and won the Cannes Film Festival Soundtrack Award in 2015. His new record 别境 (The Realm of Otherness) combines sounds of nature — waves on a rocky beach, wind in a bamboo forest — with almost any electronic sound you can think of. His versatility is clear, but one thing has remained constant: the nostalgic Taiwanese language, sung, spoken, and sampled.

It’s perhaps not a coincidence that Taiwanese culture and history is also characterized by contrasts. Though best known internationally for semiconductors, Taiwan’s subtropical wildlife and scenery are also famous across Asia2. Throughout its history, the island has been colonized or occupied by southeastern Chinese immigrants, the Dutch and Spanish, the Japanese Empire, and Kuomintang from mainland China. After financial assistance and democratic reforms of the 20th century, Taiwan rapidly developed into what it is today: a mix of southeastern Chinese (where Hokkien comes from) and Japanese culture, democratic yet not recognized as a country, progressive yet steeped in Taoism and tradition, agricultural and food-centered yet technology-focused. Throughout Taiwan’s multiple governing entities and industrialization, Taiwanese Hokkien and Taiwanese Mandarin, distinct from their mainland origins, have developed and persisted.

Lim Giong’s music constantly hints at these themes: outside influences, nostalgia, the power of nature, contrasts and tension, and the Taiwanese identity. But more than anything else, the musical worlds he builds are unique and endlessly fun to listen to.

A few years ago in the winter time, I came across the creaking of a windy bamboo forest outside of Tainan, in southern Taiwan. I had never heard anything like it, so I recorded it.

You can hear something similar at the end of 冬風竹 (Winter Windswept Bamboos).

And 石貝 (Stone Shell) is my favorite from the new record: a dance club on a stone beach.

There is a surreal quality to much of Lim Giong’s music that makes it well-suited for certain types of cinema. This scene from Bi Gan’s 地球最後的夜晚 (Long Day’s Journey Into Night, 2018), for example.

Here’s the opening of Millennium Mambo.

I’ll leave you with a playlist of my favorite Lim Giong.

The only AI product I intentionally use is Copilot, which I adopted about a year ago following a friend’s enthusiastic endorsement1. For my non-programmer readers: GitHub Copilot is an LLM-based code autocompletion tool integrated into software used for programming (IDEs), not to be confused with Microsoft’s general Copilot AI (Microsoft owns GitHub). It automatically generates code given existing code, and you can prompt it the same way you’d use ChatGPT and friends (enemies?). Under the hood, Copilot uses OpenAI’s GPT-3 and 4. Disclaimer: I get Copilot for free because I’m a student.

I will contextualize this reflection with: the majority of my programming these days is small models in Python, basically just moving matrices around (many such cases). It’s also, euphemistically, “research code”, so it’s not exactly what code should aspire to be, and I’m not writing robust tests and infrastructure. On the flip side, research means I often need to write things likely uncommon in LLM training data, e.g. a specific complex-valued matrix operation, which provides an opportunity to explore how the model performs on unseen tasks (spoiler: not well). I don’t know how it performs for other purposes and languages, though it’s supposed to be best at Python2.

Some pros:

• Very well-integrated into VSCode (my IDE): if you stop typing for a few seconds, it will automatically suggest the rest of the line, or a next line/chunk. You can also use Copilot Chat that lets you interact with your codebase and generate more code via prompting. I haven’t tried Chat yet but it seems like it could be useful

• Often good at doing a simple task when you write a comment (prompt) saying what you want to do, like: # get the min and max values of a, or # order b using the descending sorted order of a

• Generally good at completion if you know exactly what you want to do using some common package (e.g. matplotlib), but don’t know the exact way to call a function. You can Google it easily, but this is often faster. E.g. plotting a joint histogram of two variables

• Good at writing documentation/comments for a function you have already written

• Good at writing functions similar to what you have written before (that Copilot has indexed)

Cons:

• Bad at any mathematical operations beyond basic linear algebra, or just generally trying to do something that hasn’t been done before

• Will sometimes make up functions that don’t exist

• For more specific/complex tasks, takes much more involved prompting (breaking the task into small pieces, explaining every step), even with a well-known package, e.g. plotting a list of images in a grid formatted in a particular way. In these cases, doing it all myself would probably take a similar amount of time. I suspect I can get better at prompting, but I’d rather just write it myself?

Overall, I like Copilot, and think it’s a net positive for my work. I have a bad memory, and can never remember if I should use dim or axis in NumPy. Copilot can just do stuff like that for me! Even if it only saves me a few seconds, it reduces friction by offloading trivial tasks without context-switching. But it’s still just a tool: it’s not magic, does not read my mind, and cannot do anything I don’t already know how to do, e.g. come up with a cleverer/faster way of writing something. Like other tools, it won’t be helpful if you don’t already have a good understanding of the task, and using it well takes practice. There’s nothing to verify correctness; using it naively can waste your time instead of helping you.

Although it’s clear these tools cannot generate novel ideas, they are great pattern recognizers, which is still useful! It seems like there is potential for them to help research in other ways. Terence Tao thinks AI will help mathematicians formalize proofs in the future3. Jordan Ellenberg argues certain types of AI proofs will not actually help our mathematical understanding, but thinks program search via LLMs might offer insight4. Tons of people claim Anthropic’s new Claude 3.5 Sonnet is really good at code generation. I am curious to see where these tools go in the next few years.

The big question remains: in this year of code completion, have I become reliant on Copilot and gotten worse at programming? The answer, I think, is yes? Well, I have gotten lazier for sure. But maybe that’s okay, for these unambiguous menial tasks? I don’t know the answer exactly, but I’ll just say I am glad LLMs weren’t around when I was learning how to code and write.

Lastly, if it weren’t free, I wouldn’t pay for it. Microsoft doesn’t care about me. But if most people also feel this way, then LLM tools are hard to monetize, and it becomes clear why Big Tech is incentivized to hype up AI.

I didn’t intend this blog to be just AI rants, but somehow almost everything has turned into that. Don’t you worry, dear reader: upcoming posts will be about SCIENCE.

Bonus Copilot comments (colored is mine, gray is generated):