ベイズファクター：言うと思ってた

Welcome. My name is Patrick Curran, and along with my third-eye blind friend, Greg Hancock, we make up Quantitude. We are a podcast dedicated to all things quantitative, ranging from the relevant to the completely irrelevant. In this week's episode, Greg and I discuss Bayes' factors, what they are, what they aren't, and how we might use them in practice. Along the way, we also discuss Madam Flora, Big Psychic, Haters, Alaska Cruises, The Gift, Dead Grandfathers, Bait and Switches, Owing Roy Levy a Favor, Daughtering Old Frequentists, Lipstick and Pigs, Kimmy Antonelli, Accordions, Elevators of Blood, Like Choices, and The Titanic. We hope you enjoy this week's episode. I have a question for you. Have you ever been to a psychic? I knew you were going to ask that.

I have not. Ironically, you gave me crap about Ren Fairs last time. And I'm still going to call him Ren Faire because it annoys you. And there's some satisfaction to me in that. but we went with the kids and they had a tarot card reader there all dressed up and I thought it would be so much fun and I was like let's go to the card reader $50 a reading oh I said nope we're going to go to jousting so I have seen a psychic I just was not willing to pay for a psychic From afar. I have seen a psychic. So I drive by this psychic every single day on the way to work and on the way back from work. I pass right by this place. It used to be Madame Flora's, but it's no longer Madame Flora. It's like it got bought out by one of those corporate psychic chains. Big psychic. That's how they're trying to get us, man, is big psychic.

I would not have gone to a psychic, but I took my dad on a cruise for his 80th birthday. It was just the two of us. And when you're on a cruise ship, you are captive, whatever's going on. So there was this wandering psychic who was one of the entertainers on the ship. Are you sure they were part of the cruise? I mean, I'm envisioning some guy. I didn't actually see an advertisement for this person. However, so this person told me many things, but among them told me that I appear to have the gift. The psychic once told me that I'm psychic. I mean, mock it if you will, you lesser being. Apparently, I have the gift. I will prove it to you in the same way that the Alaskan cruise psychic proved it to me. Silence! I'm entering the spirit realm. Patrick, I am sensing that sometimes you enjoy being around some people, but other times you prefer to be alone.

Like now? See? Wow, that's spooky. Okay, okay. And you have experienced disappointment in your life. Like now? Sorry, I mean, wow. Okay, thank you. And? And? And I'm sensing that there is someone who is very important to you in your life whose name contains a vowel. Oh, that one's not true. My best friend is named Erskstvix. Here's the problem. Everything that I say sounds completely accurate. I mean, in that strange, vague kind of way. But pretty much anybody could listen to that and go, oh, yes, oh, yes, that makes sense. So this whole cold read thing is just something that people can find truth in. If we were trying to use them to provide evidence of psychic capabilities, those really just provide horrible evidence. No evidence at all, right?

But given that you are a hater, now I want you to be the psychic. Are you ready, smartass? I am. I have been told before that I'm telepathetic. It's true. You're pathetic even at great distances. Okay, I have made a list of names of people. One, two, three, four, eight names on my sheet. And I am going to focus all of my psychic energy on one of those names. All right. I would like you to channel your psychic energy to figure out which name I am thinking of. All right. Question one. Is it Erkstvix? That's amazing! No. And just to help you out, the name of this person does have vowels. Oh, okay. That will make it easier. Cuts out my friend's whole family. Is the person famous? Yes. Good, Patrick. Good. Good. Because if they weren't, this would be a very long game.

Is the person fictional? No. Ooh, okay. Good. Is the person alive? Yes, Patrick, very good. All right. So, famous, real, alive. Oh, jeez. Are they a musician? Okay, that is uncanny. Yes, this person is a musician. You said I was a hater. I did. About 90 seconds ago? Uh-huh. Is the person Taylor Swift? Can you see a reflection in my glasses right now of what I've written down here? I cannot. Okay. It is Taylor Swift. Is it? It is Taylor Swift. It was very conspicuous that you called me a hater. Cause I play and I play, play, play, play, play, play, and my haters are hate, hate, hate, hate, hate, hate. Oh, my God. I had a tell, and I didn't even know it. You did. Folks, I have no idea what he's doing. It's like we're just making this up as we go along. Uh-huh. Yep, that was your tell. I would suggest not playing poker.

Well, whatever it is, whether I have a tell or you are psychic, I have to give credit where credit is due. That was pretty impressive. And actually, the things that you said were so much better than, you know, you have a friend whose name contains a vowel, even though in the end that turned out to be the bad one. Your questions were actually things that were generating useful evidence along the way. That leads us into what we're going to be talking about today. Wait, wait, wait. Yeah. I'm walking up to the veil. I'm piercing the veil. I see my yellow sticky on the other side, and it says Bayes Factors. Oh, my. Hi, Grandpa. Hi. Close the veil, lad. Today's topic is Bayes Factors, and in order to get into Bayes Factors, I think we need a little bit of foundational material here. Now, we had a recent episode on odds where we unpacked all of the cool things about odds and related kinds of topics.

So can you give us like a really quick recap about odds as we go into this? We have to think about the difference between odds and probability because they are intimately linked with one another. Probability is the likelihood of an elementary event relative to all possible events. Let's say your elementary event is rolling an even number on a six-sided die. Well, there are three ways that you can get that. So the numerator is three. The denominator is six because there are six possible outcomes in the sample space. Three-six is one-half. You have a 50% probability of getting an even number on a die. What an odds is it compares the likelihood of the event with the likelihood that the event will not occur. The odds are super easy to calculate. It's the probability divided by 1 minus the probability. That's all it is. P over 1 minus P. That's the odds. Think about a 50-50 probability. P is 0.5. Well, what are the odds? It's going to be 0.5 divided by 1 minus 0.5, which is 0.5, which is 1.

There's a 1 to 1 odds. The event is equally likely. But get more interesting and say, well, what if the probability is 0.75? All right? So the numerator is 0.75, the denominator is 0.25. That equals 3. The odds are 3 to 1. You're three times more likely for that event to occur. So that's the relation between odds and probability. That really sets us up nicely for talking about what are Bayes' factors. The way that I think about the Bayes' factor, you have two stories about the world. And we could call one of them hypothesis 0 and the other hypothesis is hypothesis 1. Then imagine that you're entertaining these two hypotheses. You would like to know something about the relative merits of those two. The operational question is going to be when something happens which of those two stories which of those two hypotheses would seem less surprised about what happens So if I had one hypothesis that is

Patrick has the inner eye, and the other hypothesis is that Patrick has no inner eye whatsoever, and I told him I'm thinking of a number between 1 and 100, if Patrick gets the number right on the first attempt, the probability of that outcome, given that he has no inner eye, there is a certain probability associated with that. 1 in 100, right? 0.01. And then there is a probability that he gets it right, given that he does have the inner eye, which presumably would be much closer to 1. The ratio of those two things, which would be 1 over 0.01 around 100, that forms what is called a Bayes factor. It will give us a sense of how much more likely or how much less likely a particular observation is given the different hypotheses that we had. Imagine someone does a study on sleep deprivation, in particular the effect that sleep deprivation has on your memory.

One hypothesis, let's say H0, says that sleep deprivation has no effect on memory whatsoever. That is one story about the world. And another hypothesis would be that sleep deprivation does affect your memory. And we'll assume negatively for the purposes of this example. And so we run a particular experiment, and we find that people who are sleep-deprived wind up remembering fewer words. Now, if we were doing some conventional null hypothesis testing, we would say, well, if the null hypothesis were true, how unusual would data like this be? The Bayes factor looks at this as how well would H0 predict these data compared with how well H1 would predict these data. In this example, presumably the alternative hypothesis that sleep deprivation does affect memory would lead us to have a higher probability of predicting that people remember fewer words than the hypothesis that says sleep deprivation has no effect on memory.

So we have a ratio. In the numerator, it's the probability of the data occurring given one of the hypotheses divided by the probability of these data occurring given the other hypothesis. That ratio is a Bayes factor. That's all it is. And when that ratio comes out to be, I don't know, like six, that would mean that the observed evidence was six times more likely under the hypothesis in the numerator than the hypothesis in the denominator. All right, so if I'm going to play straight man here, can I say that H1 has six times greater probability of being true? Use your keen powers of perception. What am I going to say? Taylor Swift. Taylor, hey, hey, hey, hey, hey. That's amazing. Yeah, best predictor of future behavior is past behavior, and Taylor Swift was the right answer before. So you said, does this mean that one of the hypotheses has a six times greater probability of being true?

And we want to be really, really clear on this point. The answer is absolutely no. Bayes' factors tell us about evidence. It doesn't tell us about the probability of some hypothesis being true or not being true. A Bayes' factor of 6, for example, would mean these observations would be six times as likely to have occurred under one model than under the other model. So it's all about the value of evidence. So that is what it means to be a Bayes' factor of 6. Wait, wait, wait, wait, wait. Yes? This is a bait and switch. What? You told me we'll talk about probability, we'll talk about odds, we'll talk about model comparison. You tricked me into having a Bayesian episode. You bastard. Oh, my God! They killed Kenny! You bastard! I owed Roy Levy a favor, so... One of the things that's really cool about Bayes' factors, it aligns with how we change our mind or revise our thinking about certain things.

So if there was some prior odds associated with an event that was one, or as we would say in odds language, one-to-one, and then we gathered some evidence and we got a Bayes factor of six, we literally multiply our prior odds, which would be one-to-one in this example, by the Bayes factor, which in this case could be six, And then the posterior odds of this now become six. We literally use the Bayes factor to multiply our prior odds. And that is really, really cool. That also means, though, that if we had some prior odds of something being very, very unlikely of happening, and then we observed a Bayes factor of six, our posterior odds still might not lead us to believe in favor of that. it would multiply some very, very small odds by six, but they still might wind up being very, very small odds. So it is Bayesian because it's tied to priors and posteriors, but then it's

called a factor because it is literally a multiplicative factor times the original prior odds to yield our posterior odds. So in the end, this is just what the new evidence contributes to the whole argument. I am going to play the dodderly old frequentist in this conversation. So the Bayes factor shows a preference of one model over another. Doesn't frequentists already do that in null hypothesis testing? To some extent, yes. But here's an example that really sets a distinction between the two. Imagine two experiments. And in one experiment, I have this ridiculously small sample, I have, we'll say, 10 people. I compare those two groups in terms of their mean difference, and I essentially get no difference between the groups. And if I were to attach a p-value to that in traditional null hypothesis significance testing, I would get, let's say, a 0.40. Okay. There's not sufficient evidence to reject the null hypothesis.

There you go. Great. So now I run the same study with 10,000 people, and I estimate the difference, and it comes out to be very near zero, and again, I get a p-value of 0.40. As someone who does null hypothesis testing, what am I concluding there? Nothing would change. There is not sufficient empirical evidence to reject the null hypothesis. Okay, so now I need you to go beyond the veil for me and check your post-it. Do you feel differently about those two results, the one based on 10 people that got a P of 0.4 versus the one based on 10,000. Of course, the 10,000 sample estimate is going to be virtually zero between the two groups, and for 10 people, it could be a country mile apart. Exactly right. So in that first study, we don't really have a whole lot of evidence one way or the other, right? Because it was such a weak study to begin with. In that second study, where we have a result that is sitting very, very near zero,

we actually have evidence that there isn't much of an effect. That's one of the nice things about Bayes' factors here. Whereas the null hypothesis significance testing gives you a p-value and we can say, oh, it's not significantly different from zero. With a Bayes' factor, we actually have three conclusions. One is that the evidence strongly favors hypothesis one. The other is that evidence strongly favors hypothesis zero or a null hypothesis. But then there's this third outcome, and that is that the evidence really doesn't serve to distinguish between the two of those particularly well. All right. So are you saying we're comparing the absence of evidence versus the evidence of absence? He's got the inner eye. Yes. No, I have the post-it note, to be completely honest. Close the veil. Close the veil. Grandpa, put your clothes back on. You know, you could argue that science really needs the ability to say,

we don't know yet, that there's strong evidence one way, strong evidence another way, or we are still not certain about something. That's one of the nice things that Bayes' factors help do for us. In the end, we get out a Bayes' factor that is a number, and people attach particular value judgments to the magnitude of the Bayes' factor. And for simplicity, we're just going to assume the bigger values in the numerator, right? So that the Bayes factor we're talking about is greater than or equal to one. We understand that a Bayes factor that's closer to one means that the data that we observe have approximately equal probability of being generated by either of the two hypotheses at hand And we know that Bayes factors farther from one indicate strong evidence in favor of one over the other So we already have that internal sense The categories that people often use are of course if there a Bayes factor near one, it indicates that there's equal evidence for both of them. There's no preference for them. When we are in the one to three zone, that is often considered to be pretty weak evidence.

When we get in the 3 to 10 zone, that is considered moderate evidence. When we get up in the 10 to 30 zone, that's where people will say there's strong evidence, and then the 30 to 100 zone, very strong, and over 100, extremely strong or decisive evidence. Now, somewhere along the way, people have attached these value labels. I think we're pretty comfortable saying that a base factor of 1 means there's really equal evidence for both, And then once we get up to a base factor of 100, the evidence is 100 times more consistent with that hypothesis in the numerator than it is that hypothesis in the denominator. So you totally had me on board all the way up to the cutoffs. Uh-huh. And all I can think of is different lipstick, same pig. Okay. What's bugging you about that? Come on, man. Are you even listening? Is my mic working? Here, do you hear that? Unfortunately, yes, it is.

What we just talked a lot about in the true crime episode with null hypothesis significance testing is the binary decision, the undermining of scientific reasoning and all of that. And are we going to drunkenly find ourselves in exactly the same position where if it's below some value, we make a decision one way. And if it's above some value, we make a decision another way. Thus, lipstick pig. So you're drunkenly stumbling around with a pig that has lipstick on. A different shade of lipstick. Fine, fine. But it's the same pig. Dude, this is not that hard. I love pig. I don't know. No, no, no. You're right, though. Because if someone said, for example, once you hit 10, you consider the evidence to be strongly in favor of one hypothesis over the other. But, yeah, someone has a base factor of 9.95, and they go, well, no, we didn't have very strong evidence, and then it's 10.01, and it's like, hallelujah!

So we absolutely don't want to find ourselves in that position. You're completely right. People have articulated cutoffs, but it's really just to describe some points along the range. One of the things that makes Bayes' factors really difficult for me is that if someone said, I have two hypotheses. One hypothesis is that the mean is 100 in the population. And another hypothesis is that the mean is 150 in the population. And I go out and I gather some data. I get a sample. I get a sample mean. I might be able then to be able to generate a Bayes factor that associates with if the truth were that the population mean is 100, what's the likelihood of these data? If the truth is the population mean is 150, what's the likelihood of these data. I can form that ratio, get my base factor, ta-da. But when we talk about hypothesis testing the way that we typically do it, one of our hypotheses is almost always a null or even a nil hypothesis, right, that some true effect is zero. What is the alternative hypothesis then?

Are we going to be cowardly and say the effect is not zero? You and your lipstick pig are in fact cowardly around this. In fairness to me, it is a fine-looking pig. Do you think I'm pretty? Sorry, I have no response to that. I didn't mean to throw you off. Yeah, you're exactly right. The null hypothesis is not typically a specific hypothesis. It is all the rest, everything else but zero, which could mean that the value is 0.01 or 0.2 or 5 or 100 or minus pi. It's just not zero, being able to compare a point hypothesis, which the null is, versus this hypothesis that it's all this other stuff, is very, very tricky. And in order to do that, I need to have some sort of prior distribution associated with that alternative hypothesis, so I know of all the non-zero possibilities, which ones are more likely and which ones are less likely. And so that's where

a prior starts coming into this to be able to generate the Bayes factor. If I just say, oh, something will happen to you, or someone whose name contains a vowel matters to you. Why are you talking like a ghost? That's the way the guy did it on the cruise ship. Okay, again, was he an employee of the cruise line? Now that I think about it, it's unclear. Did other people see him? I just remember these elevators opening, blood pouring out. That's odd. Usually the blood gets off at the second floor. Those kinds of predictions cover just about anything. And so they really shouldn't get any credit in this comparison of hypotheses. But if I said something like, Patrick, I can't not do it. Patrick, tomorrow at 2.17 p.m., you are going to get hit by a falling accordion.

That is disturbingly specific. Yes, that's the point. If it happens, like, that is freaking amazing. I get my own residency in Vegas at the Sphere because I just predicted something so incredible. Unless you're on my roof with an accordion, which is what I'm more concerned about. Specific predictions are really, really risky, but when they are successful, that's actually impressive. So it means that the prior is relevant, right? And so one of the criticisms of Bayes' factors is that if you have a different prior associated with one of the hypotheses, then that can greatly affect the Bayes factor that you wind up getting out. If someone is going to put a particular prior, like we said on the alternative hypothesis, where it emphasizes some regions of the not zero possibilities more than it emphasizes other regions, you have to be able to justify that prior and, I think, try some reasonable alternatives to the prior.

And I like that about Bayes' factors, actually, that you can treat this like an exercise in sensitivity analysis and see whether or not the conclusions change. So if someone has three different defensible priors that they're putting on the alternative hypothesis, and let's say they get Bayes' factors of 5, 6, 7, something like that, then you would say, oh, okay, our conclusion about the evidence favoring one over the other appears to be fairly robust to the choice of prior if you've articulated reasonable priors. On the other hand, if someone specifies three different possible prior distributions for their alternative, and one of them gives you a base factor of 12, and the other gives you a base factor of 2, and the other one gives you a base factor of 0.7, then what we've learned is that our conclusion is heavily dependent upon the assumptions that we are making about the prior distribution, right? And the data weren't enough to overwhelm that prior information. And the associated issues of using Bayesian programs with default priors.

Yes. That we just don't worry our pretty little head about things like that. Totally true. So we often default in null hypothesis significance testing to things like, oh, P is 0.05. There we go. That makes a decision. You're exactly right that if we just say, oh, hey, JASP uses this particular prior and done and done, it's the same ritual. It's just a different church, right? Oh, that's okay for you to say, but I can't say same pig, different lipstick. Yeah, okay. I see how it is. Geez. Why, yes, Greg, this does involve Formula One racing. I am so glad you brought it up. Here's what's funny. The world champion is often called the greatest race car driver on the face of the planet. I'm offering you an open seat in Formula One. The only place you could say for one day if you win, you are the absolute best in the world. I can't help but think, isn't there a 20-year-old kid who just hasn't had a chance to drive

in go-karting Formula 3, Formula 2, Formula 1, and to say they're the best on the planet might be better to state they are the best of the other people who were able to go into Formula 1 racing. Isn't that kind of what we're talking about here, is if you have two or three alternatives and you're comparing and saying alternative B is superior to alternative A, what about alternative C and D and E and F? It's true. If you only comparing Kimmy Antonelli and that crybaby whiny George Russell It is Antonelli then Russell And here comes Russell He trying to get along side These two about to go wheel to wheel once again. Antonelli stays ahead. Game on between the silver arrows. If it's true and the evidence favors one of those, you have all of these other possible alternatives. You may be comparing two hypotheses, but in the end saying that the data that you have are much more

consistent given one hypothesis over another doesn't mean that that's the hypothesis. There are all of the other potential hypotheses that are out there. The use of Bayes factors to be able to make decisions among things, you have to be able to defend that you actually have reasonable competitors on the table. Incredible for Kimi Antonelli, who wins the Italian Grand Prix from 19th on the grid. So let me run through my checklist real quick that we've talked about accordions, my dead grandfather, lipstick-wearing pigs, churches and rituals, Formula One. I've marked off all my main points that I wanted to raise here. So what I would have for you is, do you see situations where Bayes' factors are more beneficial as we're trying to move forward as a science compared to those that are maybe less so? I would say yes. You know, one of the things that I mentioned earlier was that it gives us this ability to compare things

and maybe reach a conclusion in favor of one, in favor of the other, or to suspend judgment between the two. But the key word here is comparing. So if someone is doing, let's say, a replication study, did the replication actually support previous hypothesis or did it not? Do we not learn anything from it? That's a comparison kind of question. If we have very specific competing theories, maybe there's a theory that says there's a positive effect. Maybe there's a theory that says there's no effect. Maybe there's a theory that says that there's a negative effect, but we've spanned the space with very particular reasonable competitors. Now we can try to make a choice among those things. There's one really nice use when we have sequential designs. You gather some data, look at what the results are, and then you decide whether or not you're going to gather another block of data. Base factors can be used to make decisions along the way in sequential

designs. Whether or not you have, for example, a base factor that is over 10, then you would say, okay, we could stop gathering data. We have evidence that is sufficient. Or if it's not, you say maybe we're going to gather another round of data. The times when I think of Bayes factors as being less useful are when we have primary questions that are things like, well, how big is that effect? Or how much variance do we explain? Or what is the causal effect? So when we're focused more on estimation or prediction or specific causal inference questions, I think that Those are a little bit trickier to get at with a Bayes factor. They don't really solve all of the statistical problems that we have. It's also the case that sometimes Bayes factors can just be computationally heinous. Under the conditions like a t-test, where we are comparing two means to each other, a Bayes factor approach can work really pretty well, fairly straightforward.

When we have more complicated models, like some huge multi-level latent variable model, categorical outcomes or whatever, getting at the necessary probabilities of the data given a particular model gets really, really tricky. And so we have to start worrying about MCMC sampling kinds of issues, bridge sampling issues, a whole bunch of different things to try to get the food that we need to plug into the Bayes factor. Things get a lot more complicated there. That's not to say that people haven't worked on that and come up with ways to approach it, but I think it makes this a lot messier for folks. And it doesn't even have to be like a bonkers model, right, as you identified some really complicated things. It could be a pretty standard SEO. True. If you encounter the term bridge sampling, you need to begin to question some of the decisions that have led your life to where it is today. Because we wouldn't question our life decisions up to this point.

Oh, dude, the Titanic pulled away from the dock a long time ago on that one. Okay, let me reduce all of this down to the key takeaways. And I will use my psychic powers right now to speak directly to the four people who are still listening. Okay, the key takeaways. First one, a Bayes factor compares how well two specified models predicted the data that we observed. Second thing, it can give us evidence in favor of one hypothesis, another, or maybe evidence that doesn't distinguish. Point three, a Bayes factor is not a p-value. It's also not an effect size, and it's not even the probability that a hypothesis is true. It's just evidence favoring one over another. Point number four, it is sensitive to the prior, because the prior actually helped us to determine what the alternative model actually would have predicted. So different priors could lead to different results, and that could be viewed as an Achilles heel for some folks.

And that just generally Bayes factors complement stuff that we already have. It doesn't replace them. So we also have the ability to estimate effect sizes. It doesn't replace good research design. It doesn't replace scientific judgment. It doesn't replace a lot of these other things. It's just a really nice complement to all of these other tools that we have in our research tool belt. So if I shorten all of Bayes' factors down to one sentence, if we had two stories and we observed that a particular thing happened, which story would have been less surprised by what actually happened? That's the Bayes' factor question. I am going to pierce the veil and see that I need to go get more coffee. So I think this is a good stopping point. Okay. So are you getting coffee for just you or for your special pig friend as well? I never drink alone. I knew you'd say that. Take care, everybody.

Thanks a lot. Bye-bye. Thanks so much for joining us. Today's episode has been sponsored by Madam Q, Psychic's Statistical Services. Hello. I am Madam Q. statistical psychic offering assistance in all aspects of quantitative research have you recently experienced something extreme a shockingly high score a devastatingly low score fear not we can gaze deeply into your future and predict that your next observation will probably be closer to the mean conducting an exploratory factor analysis At Madam Q's, we specialize in the unseen. Allow us to put you in touch with the mysterious forces lurking in other dimensions. Troubled by missing data? We can fill in the gaps in your past, present, and future.

And why settle for just one imaginary future when we can impute a thousand? Need an a priori power analysis? Who better to predict your effect size than our certified effect size medium? And yes, she specializes in medium effect sizes. Wondering what might have happened if you'd made a different decision? Forget propensity scores. At Madam Q's we offer extrasensory propensity scores. We don't just estimate your counterfactual. We know your counterfactual. Worried about p-hacking? Nonsense! At Madam Q's, we prefer to think of it as significance manifestation. If your first analysis doesn't reveal your destiny, we'll try another, and another, and another. After all, doesn't your future deserve to be significant? Madam Q's Psychic Statistical Services,

turning data seance into data science. Wow! Be sure to tell your friends to subscribe to Quantitude on Apple Podcasts, Spotify, or wherever they go for things that are predictable in a good way. You can also follow us on X, Blue Sky, or Instagram, where we are at QuantitudePod, and visit our website, Quantitude.org, where you can leave us a message, find organized playlists, show notes, syllabi, and other fun stuff. And finally, get cool Quantitude merch, like T-shirts, hoodies, mugs, stickers, and spiral notebooks from redbubble.com, where all proceeds from non-bootleg authorized Quantitude pod merch go to donorschoose.org to help support low-income schools. You have been listening to Quantitude, the podcast you wish would do us all a favor and go beyond the veil.