Brain Chemistry and Neuroeconomics with Read Montague

(music)

Travis

How do feel about dopamine and serotonin? Maybe a better question is, how are dopamine and serotonin affecting how you feel, or your memory, or how you go about making decisions in your day -to -day life? I don't have the answer to any of those questions, but if anyone does, I think it might just be Virginia Tech's Read Montague who was kind enough to join the podcast. Read is the Virginia Tech Carilion -Vernon Mount -Cassel Research Professor and the Director of the Center for Human Neuroscience and Research and the Human Neuroimaging Laboratory of the Freeland Biomedical Research Institute.Read and I chatted about how these chemicals interact in the brain, what some of the historical challenges have been to really getting a grasp on how they impact human behavior, and how some recent advancements at Virginia Tech are providing the potential for us to gain some first -time insights. We also chatted some about the role artificial intelligence is playing in this entire picture, and he tested my knowledge of some of the first video games. And, spoiler alert, I am not an Atari expert.

I'm Travis Williams and this is Virginia Tech's Curious Conversations.

(music)

Travis

I’m interested in dopamine and serotonin and this interplay and some of the work you've done in this space. And I'm just curious, I guess from the get -go, because I hear a lot about these words, how would you basically describe dopamine and serotonin? Like, how should I understand those?

Read

There are chemicals in your brain, there are molecules in your brain that are used in a variety of information -bearing capacities.

And in particular, dopamine and serotonin are involved in things that we know about learning, motivation, memory, mood, the vigor that you put into one decision versus another decision, and the way you feel generally about the outcome of things that are happening to you. Let me give you an example. So serotonin is the system that people have designed molecules around to perturb if you're depressed. SSRIs or selective serotonin reuptake inhibitors. They block the reuptake of serotonin. Serotonin, once it's released in your brain under electrical activity, your brain's an electrochemical device. One of the chemicals that is used to transmit information and to control the states of your brain is serotonin. One of the remedies, it's a variable one. It's not uniformly useful for everyone, but it's a it's a game changer for some people, is to slow the reuptake of serotonin. So you release serotonin, it diffuses in your brain, it binds to receptors and membranes, and it's basically vacuumed up by special proteins called reuptake molecules. And those are blocked by SSRIs, Prozac, for example. There's a famous book called Prozac Nation. mean, we're a nation where there's a lot of depression on board. There's a half a billion people planet wide that suffer from depression and there's a lot of lost productivity from that all the way down to people that you know commit suicide and so it's a very serious worldwide problem. So we know a lot about what the chemicals do at the level of their receptors in the brain. So we know a lot about the molecular parts of that. What we know a lot lessvabout is when you put billions of those molecules together in circuits in your brain, how does that control something like mood?vthat we've considered a system level description. don't know anything about that. We really haven't. And one of the reasons that we haven't known anything about it is we haven't been able to measure serotonin dynamics in your brain through time in a conscious human being. And that's one of the new things that was brought to the table by this recent work that we did where we had people that were afflicted with epilepsy and had wires in their brain. And I'll revisit why they had wires in their brain, to monitor for epilepsy. That's a clinical condition and they were being monitored for clinical reasons in clinically placed electrodes. And we piggyback a new methodology on top of those electrodes, medical approach really, to measuring serotonin and dopamine while the person played a game. When you have epilepsy, so in North America, 3 .4 million people have seizure disorder of some sort. Of those 3 .4 million people, roughly 30 to 35%, the drugs that are used to intervene on their seizures will either stop working altogether or will not keep working the way they were. They won't keep blocking the seizures, you'll start to have them, or they make people feel cloudy, they have cloudy consciousness, they can't think straight, et cetera. Those people become eligible for what's called second level monitoring. In second level monitoring, you...The drugs aren't quite stopping the seizures. They want to monitor how your seizures are starting and progressing. And so you go under anesthesia, little burr holes are put in your head. Micro wires are slipped down into various parts of your brain, depending on the hypothesis of the clinicians for how your seizures are starting and progressing. You're put in a specially outfitted room called an epilepsy monitoring unit, which is basically a hospital room. There's a lot of extra equipment in it. And you're taken off your seizure meds and you're allowed to seize. Okay. It takes often takes several days before they wash out and that happens. But they're just sitting there and they can't get out of the bed because they're a fall risk because they've been taken off, you know, their epilepsy patients off their seizure meds. We come in and use research leads on these electrodes. These are multi -pronged electrodes. They're little tiny microwires really. And we've developed a methodology. based on what you would call modern machine learning that will turn the electrical signals on those wires into estimates of chemical activity in the brain. We can measure dopamine, serotonin, norepinephrine, and lots of things that are involved in disorders that you would know, depression, ADHD, psychosis, dopamine, Parkinson's disease. You lose dopamine neurons in Parkinson's disease. And these things are related to one another, and we haven't been able to clear that up because we hadn't made it with measurement humans. And so we are the group that pioneered that. A former postdoc of mine, Ken Koshida, who's now a professor at White Forest, did this with me initially. Ignacio Saez, who's now a professor at Mount Sinai in New York. This is, I'm getting old now. My trainees are now full professors. Probably rejecting my grants and papers. The bottom line is we can use electrical information to make inferences over chemical activity in your brain and chemicals that are super important for who you are, for how you feel, for the moods that you engender, for the feelings that you have. The thing that really makes you human isn't your ability to do the multiplication tables. It's how you combine that with these emotions that we have that let us do intuitive leaps and feel a certain way and write plays and musicals and things like that. So the particular paper that we were talking about, we took the person that was in the epilepsy monitoring unit. Of course, what we do just to let you know how this goes, you write a human use form. It goes to an institutional review board and they decide that the exact way that you're going to consent people is ethical. People can consent themselves into these experiments. These experiments are promised to them to do absolutely nothing for them. So The only thing that's promised is this may provide information that we've never had before for somebody else that can profit from it. And so these people are, you know, these are social heroes, right? Their social heroes are asked to let us, in a sense, right on the back of their affliction. So we have a very high sign up rate. They can back out at any time. So if they get tired of it or they don't want to do it or there's something about it that's disturbing to them, they can quit and they still get paid for doing the experiments, but it doesn't, it makes no promises to what it might do for their condition. And this started here in Roanoke. Actually, we started all this in Roanoke. It's now spread to grants that we have with Mount Sinai in New York, in Phoenix, Arizona, with the University of Arizona's teaching hospital in Phoenix, which is called Banner Hospital. We're part of a project in the University of Arhus. In Denmark, I have a leading MD -PhD neurosurgeon in Paris, France that has funding to pursue this. And we have a little lab we've built in Paris. And we're now starting up this summer with the Max Planck Institute in Tubingen. So these are leading groups that work on neuromodulation and decision making in people and then also have clinical interest in what goes wrong with this, either in seizure disorder or other kinds of disorders. And so they're necessarily sort of spread out all over the globe. So it makes meetings challenging because I basically have work going on across seven hours of time zones. And so I'm always a little, I'm always late to meetings. I I'm just, terrible about

Travis

Are folks usually angling to try to get that meeting, that Paris meeting? Is that popular?

Read

Yeah, somebody said, why did you do it in Paris? Isn't Paris its own reason? Actually, the person there, Philippe Dominique, So I'm gonna give a talk in Leon, maybe like 2017, and has questions that are both, he's a scientist and he's an MD -PhD, so he's a scientist and a clinician. He has both clinical questions and scientific questions, and he's just, he's very motivated to get whatever piece of this he can get working there. So I'm happy to hand it out.

Travis

Well, one of the things that you were mentioning before we jumped on here to better understand this was a game that you played. I guess with some of the patients and you mentioned a term I'd never heard of before. I believe it was neuroeconomics.

Read

Neuroeconomics is a fusion of neuroscience and economics with the best and the best rendering of it. You want the arrows to point both directions. The best of theoretical and empirical findings in economics impinging on and structuring the way we ask questions about the brain and nervous system.

And then findings in the nervous system informing economic theories. I think the former has been the bigger direction. That is, I think people like me have lifted, stolen, perturbed, or otherwise adapted the best sorts of economic ideas and sort of imported them into structuring experiments in the brain because economic ideas are around how do you make optimal decisions under a certain set of constraints quantitative version of that, you mathematical that you have mathematical models of how to make a decision, how to make a bid in an auction, how to optimize against one parameter another. And so we've used games, economic games where, you know, where I give you a little money, it earns a little interest on the way over to you and you give me some back and we go back and forth with that. The particular game we played is called an ultimatum game.

It's an oddball name. An American would know it as take it or leave it. Okay. Okay. So I'm given an amount of money. We'll play the game. I'm the proposer. You're the responder. Say, I'm given $20. I can keep all of it round over. Okay. Or I can offer you a split of the money. Ten me and ten you. Okay. Control passes to you and you either accept the split or you reject the split. Now the rub is, if you reject it, nobody gets anything. Well, some people go, well, why would you reject any non -zero offers? So I offer you one and me 19. Well, it's because you can feel that. It's like, well, hey, wait, you started with $20. You're giving me one? If that's what you're going to offer me, I'm going to punish you a little bit. I'm going to reject it and you get nothing. OK, so where is the indifference point in that game? 80 -20.

At 80 -20, it's not fixed in stone, but when you play this game with people from lots of different places, and it depends on the culture you're in, it wiggles a little bit, but let's just say it's 80 -20. At 80 -20, you reject half the time, you accept half the time. Okay, well, that's irrational. If your goal, if you're an economic, rational economic agent, Your goal should be to make as much money as you can. And the way you make as much money as you can is you accept all offers. But your instinct built deeply into your nervous system is that maybe if you accept two meager offers, you'll send a signal to me that you're kind of a rube and I can exploit you a little bit and I can get the lion's share of the money. Suppose I'm given $100 and each time I can take in $20 increments, I can offer a split.

Now, my hope is that you'll accept the split. So I have to know how to think about your mind. I have to know how to think about what you think about what I'm doing. We call that second order belief. And those kinds of games probe what we call second order belief. OK, so that game, that ultimatum game, take it or leave it, is what we played in people in the epilepsy monitoring unit. I'm sorry. This was the one intro surgically. This was done in an operating room where they were putting in stimulating electrodes, at Mount Sinai, that was in New York. Same sort of situation. These are people that are having electrodes put into their brains to stimulate, to mitigate the symptoms of either Parkinson's disease or essential tremors. Anyway, we were playing this game in patients having electrodes put into each of their brain that will be turned on that then alleviate many of the symptoms. Anyway, there was a huge difference in dopamine and serotonin signaling whether you were told you were playing a computer or whether you were playing a human being. So there's circuitry in your brain devoted to doing exchange with other humans. And it's separate and distinct from, certainly the patterns of behavior are separate and distinct from what you would get if you're playing a computer. And there was a signature in the chemical signaling as well.

Travis

So if I'm playing a game against you, that releases different stuff in my brain than if I'm just like going back and forth between me and something I know is a computer or fake?

Read

Absolutely. And the degree to which you believe it's a human being will change that. Now imagine that you're taking an SSRI, which blocks serotonin reuptake and causes serotonin to go into dopamine terminals and do other things. And so there's a whole cast of characters there that we've never been able to study in action before. And so it's a small step. That works a small step. But it was within subject meaning these were people that got an electrode put in one side of their brain. On one occasion, the surgeon comes back a month or two later and puts it in the other side. So we're able to ask questions within the same brain that way. That's a long time in the making. The use of these games in people while you're doing neural recordings started in early 2000s. I spent a year at the Institute for Advanced Study at Princeton, working with a guy called Eric Maskin, who is an auction theorist. And this falls under the same sort of umbrella as an auction. How much should I bid for this thing? How much is it worth if I don't know things about you, et cetera? That was 2005, 2006. You won the Nobel Prize in Economics in 2007. I wrote him afterwards and said, isn't it great that I came and helped you win the Nobel Prize? Of course, I had absolutely nothing to do with his Nobel Prize, but I profited from him quite a lot. He's basically an applied mathematician. And I learned a lot of things from him about how to think about these kinds of strategic interactions. And human beings have profited and lost in these kinds of interactions through the millennia, right? mean, you know, wars are like that. Think of mutually assured destruction to the 60s with Russia as we built up our nuclear stockpiles. It's a kind of game theory like stalemate. It's, of course, unstable in many ways. there is a kind of game theory rubric living behind it.

Travis

It sounds like maybe our brains are wired to this human to human game, playing games with one another.

Read

Our brains are wired to want to win. We're good at games. We enjoy games. mean, games have evolved because we like the fundaments of a game. I mean, if you look at this sort of revolution that's happened with neural networks now, it began with Google's company DeepMind in London playing and beating, playing superhuman in all the Atari games that used to run on what's called the Commodore 64.

And then they beat all the go playing champions in the world. If you know the game go, you know go. Have you heard about this?

Travis

I have not. I've heard of Atari and Commodore.

Read

Okay. So Atari has things like breakout, know, breakout. A little ball is going like this and you move it around, you shoot and you try to break out of it.

Travis

I feel like I've heard of breakout. I think I start about Pac -Man. That's where I start.

Read

Pac -Man's 1980 at a kiosk.

Travis

I was born in 82.

Read

you think you start with that?

Travis

I mean, that's my memory.

Read

Yeah, you know, it could be an in utero thing. could have maybe your mom played Pac -Man. You heard it through the... It's very likely. Okay, so the revolution in what they call artificial intelligence now really began with, in my mind, deep mind solving all these problems. So go...is a board playing game. play on a 19 by 19 board. It's, if you're playing the game of fellow, we have black stones and white stones and you capture territory. Well, anyway, that's what you do with go. You flip over black stones and white stones. You're trying to capture territory. It's tremendously popular in Asia. It's nominally popular in the West. Like children in Asia could be everybody in the West. It's just not, you know, it's like, think of the men's soccer team for America. Okay. I it's not like our other kinds of teams. Okay. It has a gigantic state space. That is the number of possible games that you can play. It goes ginormous. It's like 10 to the 150th power. It has a branch factor. So that when you play a game on a board, you have something called a branch factor. Chess has a branch factor of about 34, 35. And that means from any position on the board, they're about They're on average about 34 to 35 legal next moves. Go is 250. OK, so it's just massive. You can't even explore the whole state space. So they trained a neural network to play Go. And long story longer, they beat every champion there was. And they were using Grandmaster input to seed difficult board positions. They removed that, and they just trained it from scratch. And then that version, called AlphaGo Zero, beat the original AlphaGo's like 10 ,000 to zero or something. And they have gamified many sorts of problems like that, including chess. So they can train from zero, they can train up to grand master level in a day. It's probably, if I say that, somebody's going to write me when they see this and say, it's three hours now. mean, it's probably, it's dramatic how fast it is now. So that's part of the whole revolution going on now. Those are models that are part of the old, revolution that you see going on around you now. I guess, you you have been working in this field, you said it's been on like a two decade long process. Three decades. So three decades long process. was a postdoc in 1990, 1991, 92 at the Salk Institute in a guy's lab called Terry Sinovsky. It was a neural network lab. It's called the Computational Neurobiology Lab. neural networks were kind of the lapping stop, and they weren't called AI. AI was really sort of think expert systems on steroids.

Travis

Well, I was going to ask, what's it like for you to have, you know, pursued better understanding the brain over all these years and get into a place where you've made these advancements, you can look at a brain in real time to some of these things.

Read

We're still at the beginning. We're still at beginning. know, the brain's been tinkered with a long, long time. I'm a firm believer that there are clever tricks built deep into the biology of the brain that we haven't even imagined yet these adjuncts to our thinking and neural networks on the outside in order to structure ways that we ask questions or extract, in my case, extract new kinds of signals out of the brain. That's very, very useful, but you know, psychology doesn't have enough ideas, for example, to structure the way you go ask these deep neurobiology problems. Computational theory may, but it's still nascent. It's still at its beginning. stages. I've gravitated to humans through the years, not because I'm not partial to rodents and fruit flies. It's just that I've always wanted to go after humans. so, I mean, my lab in the last 13 years here at Virginia Tech has made real inroads and engineering ways in to the human brain at a level that, you know, rivals sometimes the kinds of signals you can look at in model organisms.

We couldn't have done it without the model organisms. We had to prove our methods out in transgenic rodents with a collaborator at the School of Neuroscience called Matt Howe. That's been essential. I've also been given enormous freedom here, which I also had at my last institution. I had a great run at my last institution, Baylor College of Medicine in Houston and Rice University in Houston. I've had the same kind of freedom and I've moved in a different direction here.

I've enjoyed the, I mean, the administration here has been very open to ideas, both the one that hired me and the one that's here now.

Travis

I'm curious as you look at brains and some of the chemistry and the stuff that's going on in all your work, what's something in this area that really gives you hope?

Read

What is something that gives me hope? Well, I mean, we're now measuring attention deficit, hyperactivity disorder, major depression, various forms of psychosis, et cetera.

They have gigantic influences, impact on systems that deliver the chemicals dopamine, serotonin, and norepinephrine. So just being able to measure the dynamics of these three chemicals and in people that may have some of these pathologies on board gives us for the first time an ability to ask the question, well, what's different about dopamine signaling in this neural structure and this person who's not depressed?

How about this neural structure and this person that is depressed? Many people that have epilepsy have depression on board of varying degrees. That's the kind of thing you can measure behaviorally. And we just haven't had a way to measure these substrates of it, even though we know molecules that perturb them have an impact on them. And so we're this kind of intermediate step that's letting us connect the molecular with the system level description. We're now taking that and appending it to a way to some of this artificial intelligence work that's going on. mean, we use similar kinds of approaches to get the signals out of the brain. And then we can also use artificial intelligence approaches to understand how to send stimuli into the brain to sort of probe it. You know, like you're sending a probe into a computer to see the answer it gives back. So we send something in, we get something back. We send something in, we get something back. So that's been exciting. And I think Virginia Tech opened doors to that and has certainly been transformative in making decisions that way. And we have a lot of young people here too. They've done a lot of great hires over the last six or seven years between the FBRI, the Fralin Biomedical Research Institute, the School of Neuroscience. And you have to invest in the young, you know, can't eat your seed corn take care of it. it sounds like the better we understand what's going on in our brains, maybe the better we can help people with some of those conditions. We can certainly help people and we can on a larger scale have more insights into ourselves. I mean, I think we're kind of in a brainstem age still where people, you know, listen to the egocentric needs of their brainstem that drives them be self -involved and, you know, invade, I have to invade a country to show everybody I'm a big shot. I have to do that. I have to, it seems all to people like me, it seems a bit, unevolved and primitive and I'm hopeful we'll get past it. You know, we now, we're now trying to include everybody in our intellectual resources instead of operating on, you know, 40 % of our intellectual, we're going to let everybody play. That's a good step, but I'm quite hopeful there. I still think we're in kind of the age of brainstem. Hope we make it through it. I think we will.

(Music)

Travis

And thanks to Reed for sharing his insights related to how our brains work, artificial intelligence, and old video games. If you or someone you know would make for a great Curious Conversation, email me at traviskw at vt .edu. I'm Travis Williams, and this has been Virginia Tech's Curious Conversations.