AI and Better Classroom Discussions with Yan Chen

(music)

Travis

It seems the more artificial intelligence advances, the more we find ourselves interacting with non-humans. And while I do think it could be fun to have a robot best friend named Reggie, not that I've thought about it all that much, I'm also curious how artificial intelligence might be enhancing our human-to-human interactions. Thankfully, Virginia Tech's Yan Chen reached out to me to talk about a new project he's working on that he thinks will do just that.

Yan is an assistant professor in the Department of Computer Science, and some of his research interests include human-computer interaction, real-time student learning analytics, and computer science education.

Yan and I talked about how he's been using artificial intelligence to enhance a teaching strategy called peer instruction in his classrooms. He shared that what he's working towards is a project that would allow him as an instructor to monitor classroom discussion and evaluate whether or not it was being productive towards the ultimate goal of learning the material. We talked about some of the general challenges that instructors face, how this may help them overcome it, and how it may help keep students, perhaps like my younger self, from getting sidetracked by a side-conversations about that best friend robot. I'm Travis Williams, and this is Virginia Tech's Curious Conversations.

(music)

Travis

Well, I know you've been doing some research at the forefront of artificial intelligence and how we can be using it in our classrooms. And I think specifically you've been using it with this method called peer instruction. And so I was thinking maybe a good place to start is what exactly is peer instruction?

Yan

Right. So peer instruction is one of these teaching practices that was widely adopted by instructors in the classroom setting. It has been actually shown to be significantly increased students' engagement or comprehension. And essentially, the practice starts with a question posted by an instructor, and the student will first provide the answers to the question independently. And then they will articulate their reasoning behind their answer to their peers, and then to check their answers. So that's where the instruction was coming from the term. And then finally, after the... discussion they're going to resubmit answers if they find that their answers were incorrect or underspecified. This is not a practice that I invented. It was being invented by a physics lecturer who a long time ago they were using this practice in their lectures. So I'm just using this practice on the platform online. And then it has been so far been great. And then it's one of the few that I use so far. And then so far it's been, yeah, fantastic.

Travis

Tell me if I'm understanding it correctly. You ask your students a question, they give an initial response, and then you have them kind of work with other classmates to, hey, why did you think that? And discuss it. And then do you ask them, I guess, again, if they change their answer?

Yan

Yeah, exactly. So the essence of this practice is that by articulating your reasons behind your answer to your peers, and then sometimes your peers have different answer, right? So you guys are creating this, you know, discussions back and forth, and that would help you guys to co-create new knowledge or find misunderstandings in each other's mental models. And finally, can resubmit your answers if you want to change it. And it has been mostly used for multiple choice type of questions. So instructors would post a question on their projector and ask the students to use either iClicker or just a website or phone to vote their answers. And then... And then after that, they can just verbally chat with their neighbors. What I approach, what I use this practice is slightly different. I'm looking at programming lectures where students are, instead of answering a multiple choice question, they write a piece of code. So that requires them to have a laptop open and also an environment that they can write the code, they can...at it, they can submit, they can test it. And then also, instead of just talking to your neighbors, which is not diverse, sometimes you sit in the front row and then probably your neighbors are all pretty good, we're trying to put them in the group virtually so that they can use the chat to ask for help or even just share different approaches to the problem and then they can submit their answers again.

Travis

Yeah, that sounds fascinating. Do you use this in person settings or online or both?

Yan

Majority of the sessions we did so far were in person, in person setting. And we have been collaborating with the faculty in the CS department who usually teach this large in person programming class.

But we also collaborate with some folks online, some teachers who are teaching online courses where essentially the setup is the same. It's just you don't see your neighbors, but instead you're all joined in the virtual room.

Travis

Well, what are some challenges to the person running the class when it comes to just navigating this?

Yan

Yeah. So as you can imagine, this is really hard to scale when it comes to your large in-person classroom. So how do you preserve this same richness of feedback that you can get in a small and highly personal setting? That's the challenge. So typically in today's CS classroom, the number of students enrolled can easily scale to hundreds, if not thousands, right? And then it's really hard for instructors to manage this in-class activities. But the essence of this activity is really on the effective peer communication. Understanding the loopholes and then also co-construct the new knowledge with your peers is really important. And then that's just been really hard for instructors to do at scale.

Travis

Yeah, that sounds super hard with 100 to, that would be hard for me to do with like 10 people. So to do it with 100 people, that sounds super challenging.

Yan

Yeah, and then there has been studies, even our own, that kind of show this unproductive group discussion, where students, some students even left their discussion with more confusion, or formed this false mental model. So we don't want that Happen especially when you had a chance to do this live iteration

Travis

Yeah, so it sounds like a system that has a lot of potential but also some potential pitfalls if it's not kind of harnessed in the right way So how didn't you all come about? Incorporating or using artificial intelligence and large large language models to do this

Yan

Right. So AI or this stays people use large-language models there really good at understanding the meaning and associations in language. And then when you think about collaboration, right, it's fundamentally based on communication, like language plays a key role. So that gives us this new tool to use for engaging with the live students interactions. So, you know, the challenges I mentioned about the scaling issue in the class activity with hundreds of students chatting and coding. And you can imagine that this generates a lot of unstructured data and also hard to grasp what's the key patterns. instructors standing on a stage, what they really want to know is, you know, what's really going on here? Do I do stuff or do I walk around? Right. So it's really hard for them to observe or understand those data that was generated in real time. So that's where my work is actually trying to toggle. So we're trying to use these large-dangling models that we designed to understand the meaning and the structure of this data, and then design these intuitive visualizations, and also guide instructors' attention to quickly understand, interpret, and take corresponding responses. Just to give you one example, right?

This work that we recently got into this HCI venue is a visualization system that can not only intuitive display the real-time status of students' discussions, such as the topics and activity level, but it can also automatically generate or recommend things that teachers should pay attention to, like, you know, are there any students kind of fall behind or are they actually chatting about something that's not really aligned with what they're supposed to be right? Because what you really want to know is instead of wait until the end of the session, can you do something in the middle so that you can make this whole discussion thing more effective, right? So yeah, that's something we're doing right now.

Travis

So it sounds like you're attempting to harness artificial intelligence to just kind of be the monitor over all of these conversations happening and possibly find out if I was in one of those groups, if I was talking about some random movie rather than the topic at hand, which would have been very likely in that situation when I was that situation.

Yeah, so topics and then also sometimes people are talking about related stuff. It's related to the class, related to...things that matter, but it's not really about the things that they are confused about or their peers are confused about, right? So sometimes you see this discussion that was talking about issue A, but what they really were confusing about was issue B. And then we know that because we have access to their code, to their artifacts. So we can sort of see whether...things that you do and things that you say are aligned. And then that's kind of give us a new perspective, gives the instructors a new perspective to say, well, yeah, you guys are talking, but you're not really talking effectively, right? So maybe you guys should lean more towards the issues that you just ran into. Yeah, so that can really help instructors to see the fine-grained understanding of students students' performance.

Travis

I think one of the things that you mentioned was the idea of getting this real-time feedback while you're in the classroom. What does that look like from the professor, from the instructor's perspective? Is it on a computer monitor? it a phone? Do get an alert?

Yan

It's basically a dashboard where they see charts, they see curves, they see notifications, and they see ongoing chat if they want to dive into the details. And then they can click on things they can even type directly to certain groups to give them feedback Yeah, so that's essentially what they're seeing Doing that during the in classroom setting.

Travis

I'm curious. Is it monitoring their verbal communication with one another or just the stuff they type into a system?

Yan

So right now we have different Different types of exercises. So on one type they only chat online meaning that they only type on their screen without any verbal communication. So in that case, we can look at the characters they type in and also the duration of the messages, the duration of the discussions. And for another exercise, which is online, we essentially capture their voice during the live transcription. So still turn their voice into a a digital or text version so that we can use that information to understand the high-level meaning. Yeah, so we have both versions, but it depends on how the class was structured.

Travis

How far along are you in this project?

Yan

So I have been looking at this whole in-class activity since probably the end of my PhD, which is a few years ago, but I wasn't focusing too much on the low level kind of understanding. I was looking at, yeah, maybe give students some communication channel and the hope is that they can talk among themselves. And then later on, are all back in this. online university. So that's when things really start kicking in where you have this large number of students who are taking CS online and we can we are lacking of this tool to really help instructors to manage this in-class activities and then programming is really required to practice do a lot of hands-on practice. So without such tool You just couldn't do that in a very effective way. yeah, then was, me and my colleagues were kind of thinking, how can we give instructors a better support to help them to understand where things are at, where students are at, where the group discussion were at, what are the main difficulties, what are the ways that they can help them before things are already happened. So that's where we say, can we actually do something as things are happening? So that's where we're trying to looking at, is there a way that can help reduce the amount of information to the point that people can react to it, respond to it in a real time fashion. So yeah, that's sort of the trajectory of how we started. This day's My group is developing this platform called VisPI, visualizing peer interactions. And then it allows instructors to basically click just a few buttons to create some exercises that they can run in the class and then be able to distribute the exercise without any barriers. So yeah, that's how things have started and also how things are right now.

Travis

What was going to ask kind of where are you at as far as the development of VizPi?

Yan

 So we have already collaborating with a bunch of faculty in the CS department and then has been used in the class setting dozens of times, if not hundreds. So it's pretty far in terms of getting it to the user's hand. But in terms of the research part, we're still looking at ways to make things more, for example, intuitive, more reliable and then things that instructors can use in a different way. So for instance, right now we're doing this project where we're trying to support instructors to provide feedback in a scalable way, right? So the project I mentioned before was mostly focusing on identifying the key patterns or the insights, but then you have to act on it, right? So that's the part. we're trying to focus in on right now, which is, let's say you have some issues that you saw, what can you do about it? And how do you know the impact of this action that you take? So that's the current research project that we're trying to focus in on. yeah. So not only saying if the AI can evaluate how effective the students are communicating, but how effective the teacher is communicating as well.

Travis

What do you think the potential of artificial intelligence in the classroom is?

Yan

I think it has a huge potential, right? So my go-to recommendation for all those teachers I interact with is trying to use AI as much as you can, right? Because I believe that in the future, AI would deeply integrate into your education, and then especially in the in-person classroom. where you already have this hundreds of students, like diverse mental models and all those motivations that are already there. It just feels like you can leverage more of those energy in a way so that students will feel more engaged and then they can have better access to the content and also access to their neighbors, their peers.

So eventually it's going to create this more immersive environment for them to learn, more personalized environment for them to learn. I think that's what I see the future of AI in the classroom setting. Yeah, well, it sounds like you're definitely an instructor that is not fearful of artificial intelligence in the classroom. No, not at all. We should all embrace it. It's a tool that if you don't use it, yeah, you're going to you just kind of fall behind. And also on the student end, they are going to have to use it in future job. Everybody in this day has some requirements on, know, have you ever interact with AI or of course chat with you, right? But do really know how to use it? Like knowing the limitations, the accuracy, knowing the...ways to manipulate it. So those are something that we see more and more often as a requirement in the job search.

(music)

Travis

Thanks to Yan for sharing his expertise at the intersection of artificial intelligence and classroom learning. If you were someone you know would make for a great curious conversation, email me at traviskw at vt.edu. I'm Travis Williams and this has been Virginia Tech's Curious Conversations.

(music)