Venturing out of the weeds

consequentialism 🤝 optimization

Last semester, I took a class that pushed me to think more about the bigger picture of machine learning and decision-making. As graduate students, we spend so much time in the weeds trying to learn and publish that we sometimes lose sight of what’s around us. So I thought it would be fun to take a class this semester on the moral and political philosophy of AI (my first philosophy class!). Inspired by readings and discussions, I’ve started to think about the link between these concepts and my intuitions about math. The purpose of this and the next post are to introduce some basic ideas, and to ask some questions I want to keep in mind. I certainly don’t have any answers!

First, what is consequentialism? Generally, we agree that lying is wrong. But if lying saves someone’s life, was it still wrong? Consequentialism would say it is right because the outcome was good; it is the theory that an action can be judged right or wrong solely by its consequences. It’s extreme, but has practical versions, such as the infamous effective altruism. Put another way, consequentialism takes a nuanced world and converts it to a judgment based on a narrow set of criteria. To me, this recalls something in math: optimization.

Suppose I want to calculate the shortest path from my apartment to campus by bike. But Berkeley drivers are REALLY bad, and the shortest path is actually quite unsafe. Considering routes I feel safe on, the shortest one has a steeper incline, which I do not want! How do I find the best path given my set of preferences? This is an optimization problem. Applied optimization takes real-world problems, formulates them in a solvable way, and then finds the best (or approximately best) solution. Most of machine learning relies on optimization, whether it’s training a model to classify images, or finding the best set of actions for a robot in an environment. Generally, an optimization problem specifies the objective, but you can go about solving it however you want.

On defining intelligence in the 1940s and 50s, Berkeley EECS professor Stuart Russell writes:

…the notion that won out was rationality: a machine is intelligent to the extent that its actions can be expected to achieve its objectives. In the standard model, we aim to build machines of this kind; we define the objectives and the machine does the rest.

What is “the rest”? *shrug* Both consequentialism and optimization would say that no matter the steps taken, as long as the objective is achieved, we’re good. This is often useful, but we must remind ourselves we are throwing away intermediate information, reducing richness to one point.

Guess where people use a lot of optimization, and where effective altruism is popular? Tech! There is nothing inherently wrong with either of these fields, in my opinion. But there is a flavor of black and white-ness to both these, the efficient reduction of information in order to make decisions, that feels cold. We need math when dealing with machines and complicated processes, and apart from that, it can provide us with good intuitions. But to what extent can we rely on math in guiding our decisions as humans, whose complexity of experiences, emotions, and actions don’t come close to being captured by any equations? This is a question I will keep coming back to, probably for a very long time.

Going back to my shortest path example: what if there is sometimes a cute cat on a particular route who I want to see? Or that on Fridays, there is a garbage truck I want to avoid? Or if on certain days, I want to work out more and prefer the steeper path (okay I would never actually want this)? This brings us to another point: the world is ever-changing, and so are we.

If we want to keep going the math route, one way is to model uncertainty. This is not a new idea, but in suggesting a new direction for AI, Russell writes:

It is not rational for humans to deploy machines that pursue fixed objectives when there is a significant possibility that those objectives diverge from our own.

A more sensible definition of AI would have machines pursuing our objectives. Of course, our objectives… are in us, and not in the machines. This means that machines will necessarily be uncertain about our objectives

Russell believes that AI should have some measure of uncertainty about human objectives. This is not a controversial take. But this means thinking about probabilities, and so we prepare to open another can of worms (next post!).

I highly recommend the rest of Russell’s very accessible If We Succeed (2022). I am just beginning to dive into the literature and have a lot of reading to do before I have anything constructive to say. If you have any recommendations, please send!