Description of graphical content is included between Description Start and Description End. Transcript Start [Silence] Fade up from black. Animation: Text for TSBVI transform into braille cells for TSBVI. Fade to black. Fade up from black. Participant: So these video games, uh I'm thinking of my students, like-- Merabet: Yeah. Participant: You know, they don't know right or left or, you know, directions. That's one of the things we're working on. Merabet: Yeah, yeah. Participant: How can I use that to -- with kids that they don't have that yet? Merabet: Okay. Participant: Would it work? Merabet: Okay, and that's exactly what they are doing. So, my first question to you is are you talking about ocular blind or C-V-I? Participant: Anybody. Merabet: Anyone. So... Certainly the orientation component I think is a very, very interesting one. And one thing that really intrigued me -- uh let me, let me just, sorry, backtrack just a little bit. The big issue here is whether or not we think these skills can even be learned. Right? And there's been a lot of skepticism that if you're born blind you can't possibly understand the spatial relationships around you. In other words, you need vision in order to build that. I think hopefully I've convinced you that that's, that's not true, right? Obviously. Let me give you sort of a counter example. My mom, I keep going back to my mom. Bless her heart, can't find her way out of a paper bag, right? And she's sighted. Right? But if she gets lost we never say anything about her vision. If a blind person gets lost what's the first thing we say? It's because they're blind. It's two different things. Completely different things. Your spatial skills and the information that you use to build those spatial maps in your mind are very, very two different things. So, that's the first sort of myth I think we need, we need to dispel. In the case of teaching, I know that Jaime, Jaime Sanchez in Chile, did do this specifically and they use a variety of ways to try to get that compass aligned. So using cardinal points: north, south, east, west. Another one that's very, very popular for them is using clock hours. So, I don't have a picture of this, but for example the child will sit in a loop with clock hours and using their cane get a sense of 12, six, three, so on. So coordinating a clock that they have in their mind with tactile cues, with the tool that they use very, very often has something to say about integrating various strategies. I think the more you can overlap familiarity, the more powerful it is. And again, I don't want to say that this was a, you know, a very, very structured study, but at least from an observational standpoint the more ways you can get into these maps, which I believe are there, the better it is for them. And some individuals are going to be harder than others. Like I said. But I do believe that it has nothing to do with vision. It is how you integrate spatial information and getting that information is the cue. And the more ways that you can do this or to create a grid system or some sort of spatial register system in your mind, the better, the better it is. Yeah? Participant: Uh, Ed had asked a question earlier about R-O-P, I just wanted to prompt-- Merabet: Yeah, yeah. Participant: --prompt that because that was something that I have interest in, too. Merabet: Yeah. Participant: And along with your, Carolina's um question. It seems like I have students who are typically um challenged with orientation, you know, and I've learned that -- giving enough practice, I mean, a lot of practice, they can learn. Merabet: Yeah, yeah. Participant: But it's like this seems like um a great way to practice something, you know? Merabet: Sure, sure. Participant: And -- and I thought it was interesting, but I, I wondered if you had any other -- um uh anything to say about the R-O-P and people that are very challenged with orientation in general and how -- any implications you could tell us about that. Merabet: So, so, great, great question. This came up earlier as well. I think it's an important thing to kind of discuss. Um. This exactly your comment has come up a number of times with the O&M instructors that I work with as well. It's sort of like this clinical suspicion that you all have that has never been sort of studied systematically. And one of my colleagues came to me and said-- As far as I know, correct me if I'm wrong. One of my colleagues came to me and said, "You know what, this is a chance to look at that once and for all and solve it." You might think that because of the prematurity, you're dealing with quote "a fragile brain" per se in development. And it goes back to this idea of the dorsal stream dysfunction. Why does the dorsal stream seem to be more impaired than ventral stream? And the theory might be is how they're wired, right? If I know your face, it's pretty much your face for life. But how I see space and interact with it is very, very flexible. So, the way that the brain is wired in terms of the dorsal stream might create a certain vulnerability that it doesn't have with say, object-related type tasks. They're wired differently is the first piece to think about. The prematurity may then cause a situation where the dorsal stream is more impaired, so on. The problem with doing that study, which is ultimately the point that I wanted to make, is who do you control against? You control against ocular blind or you control against, which I believe you should do, sighted kids who are born premature but don't have ocular issues. They are tough to find. That's the problem. Finding those controls is the big challenge in terms of trying to validate that study. We've talked about this, I've talked to people at Children's, and so on. And they say, "Yeah, a lot of premature kids, if you're getting them anywhere close to R-O-P age or concerns, have some sort of sequelae." So, disentangling the, the uh, the uh the prematurity component versus the visual component is extremely hard. And that's-- the main, the main study reason why we weren't able to pursue this. So, it wasn't the population of interest, it was finding the right controls, right? But I do think that it's an interesting question. And again, it's come up many, many times, but to answer that properly I think is a lot harder than, than we think. . Participant: [indiscernible] So, the video game is not for sale. A lot of people have asked this as well. And unfortunately, it, it's not up to me. It's Jamie's group that have developed this. Um. We have since split off and started doing our own individual projects, but I can tell you that the Stripp Map, for example, will be made available. They are just in situations now, um some of you are familiar with app development with Apple, um it's a challenge. They take a lot of money out of it. We're trying to raise capital to do that, also. Um. For the case of Jamie's uh software: he, he really does this in terms from an academic standpoint. If you write to him specifically and say, "I'm really interested in that," he will say, "Yes, I'll offer it, it's not a problem." My problem is that you have to find somebody who builds your world, right? That's the big, big challenge. We talked about, also, trying to find an individual who could build just a, an editor or an O&M instructor could just draw various sort of arenas. And then it turns it into the audio world. And say, "Okay, just from a simple standpoint can I do this?" Still in the works. Um, but it's again technical, it's manpower, it's resources that make this challenging. But having said that, I think moving forward we are in a situation now where we will start really uh, uh offering this to the community as well. Participant: Has there been any research done on existing audio games and how they would affect mobility? Participant: Um, not specifically in terms of mobility. I'll give you an example. I, I might actually have this. Yeah. This, this was an example in my mind really sort of rocked, if you will, the, the neuroscience world. Um, this was done by uh Daphne Bevallier, at the University of Rochester, she's now in Switzerland. And this is really the, the, the study that got a lot of press uh in terms of visual function and, and video games. And this is what they did. And again, you'll see when I describe how the study was done, how they were able to figure this out. So, contrast sensitivity, as you all know, is a very good measure of visual function. It's tough to do. But it says a lot about, about how good a visual system is. It's not something that theoretically should get better, right? If anything it's going to get worse over time. So, this is what they did. They took a group of individuals, right, who weren't necessarily game players, and they took their contrast sensitivity function. They randomized one group to action video games, like Call of Duty; where you're very, very engaged, right, playing the game. And the other group was engaged in video games, same amount of time, but it was something like SimCity, where you're sitting in front of a, of a monitor, but you weren't really engaged from a sensory motor standpoint. What they found is your contrast sensitivity improved, right, a lower curve, right, you see-- excuse me -- improved-- a higher contrast sensitivity curve is a, is a expression of higher uh higher contrast sensitivity. And what they found is that it was only in the Action Study Group or the Action Video Game Group-- see how the curve here is much higher versus the control group. So, this was the first real example that video games had an immediate impact or could potentially have an impact on visual function. And theoretically a function that was not supposed to change. It was immutable. So, that was quite dramatic. The other piece of information that, that I'll share with you is that a lot of people were interested-- "Okay, let's turn this now into cognitive games, things like Luminosity and things like that, right? You've seen this. A lot of money spent in this area. The fact of the matter is is there's been no evidence that it has improved cognitive function beyond the task that you're actually doing, right? So, if I play a lot of Sudoku, I get really good at Sudoko. It doesn't help me find my car keys, right? [ Laughter ] Merabet: And that's obviously the idea. Is transferability. Near transfer versus far transfer. Video games are very, very good at getting better at those skills, but ultimately how do they translate into the real world is a tough, tough question. That's why we're excited in terms of this study design. Because then people were saying, "Well, now that these kids play the game and are good at finding their way, are they now better going into buildings they have never seen before after that?" So, in other words, does this somehow increase skills that ultimately translate outside of the context of learning. That's I think the next step. So, there's a near transfer question and a far transfer question, but that has not been answered yet. And I think ultimately that's where I think we'll see benefit, right, outside of the learning context. So that was a little bit of the question I wanted to ask. Um. I'm interested in the interface of the video game and then the haptic sense? Merabet: Yeah. Participant: And that have, have you seen or what's been going -- what's happening right now in terms of the research, is there a huge increase with that haptic inclusion of um applicability or just mapping? Merabet: Great, great question. We don't have enough data to actually answer that question, but that was the goal of the study. By having that piece of haptics are we, is one plus one two or is one plus one five, three or four, for example? And we don't know. And that was our suspicion because there's certainly things you can do auditorily that you can't do haptically and vice versa and that was the suspicion, but it's still very much in the early phases. I can tell you that the haptic component is actually becoming a challenge because it's not intuitive, apparently, for our users. The auditory was very, very intuitive and I think we're not doing it correctly from a haptic standpoint and that's why we're not seeing that benefit just yet. The problem with um, with the Novint Falcon is it's quite limited in its range of motion. And to them this doesn't mean much. Right? Similarly, as I said, with the Wii‑motes if it rumbles and this is the edge it'll start rumbling, but I can still move my arm here, right? So the kinesthetic feedback isn't there either. So I think there are functional constraints to the tools we're using that don't allow us to leverage that information just yet. But we are probing that because one of the control arms is learning with a tactile map, for example. So they explore it tactilely that way versus exploring auditorily. And our suspicion is we can try to disentangle what tactile information gives you versus what is the auditory. The big challenge, of course, with tactile is you need to carry it around. The Perkins School for the Blind has a braille-labeled, uh I'm sure you've seen this, a braille-labeled MTA map of the subway, but it's the size of those two doors. Right? So, I think part of that is a practicality standpoint and the second is can the brain figure things out when it's on large scale like that. You don't have that constraint with auditory. So that is very, very much the goals of the study now. It's to try and disentangle what do those two pieces of information bring and do they act synergistically. But it's too early to answer that. Great question. Tabb: And um with that, I, I, though I could ask questions -- I think we have enough going to keep you for another week ‑‑ Merabet: Sure, no problem. Tabb: Uh, if we could just share a round of applause for Dr. Merabet. [Applause]