潜在変数のスケーリング：「どれを選んでも同じ」とは限らない

Welcome. My name is Patrick Curran, and along with my always optimally scaled friend Greg Hancock, we make up Quantitude. We are a podcast dedicated to all things quantitative, ranging from the relevant to the completely irrelevant. In this week's episode, Greg and I discuss alternative methods for latent variable scaling, a topic about which we are often taught don't make no never mind, but in actuality can potentially make great never mind. Along the way, we also discuss favorite mathematicians, speed squares, drill bits, the quarter pounder, barley corns, butts and hogs heads, one Helen versus one Herbert, micro centuries, parlor tricks, being over served, the great gonzo, but wait, there's less. get on with it and hiding the body. We hope you enjoy this week's episode. All right, I'm almost done here. Who is your favorite mathematician?

What do you mean you're almost done? Don't worry about it. Just who is your favorite mathematician? And why would you think I have a favorite mathematician? You are definitely a guy who has a favorite mathematician. I resent that. And I can't decide anyway between Euler, Leibniz, Newton, or Gauss. or if you want to go to the modern era, it would be John von Neumann or Peter Lacks from the Courant Institute at NYU. Other than that, why would you even ask me that? Okay, folks, he did not prepare those. Like, sometimes we'll Google stuff, and I just want to clarify, he literally generated all of those in the moment. So you're just making my point. So pick one of those. I'd have to go with Newton. Okay, cool. Thank you. That finishes that up. Finish this one up. Don't worry about it. Right, smartass. Who's your favorite mathematician? Michael. Could you be more specific? Actually, I can't. He's my YouTube guy on woodworking. I don't know where Michael is.

I don't know what institute he works at. I don't know what his last name is. Okay. But I got to tell you, I am convinced some of the greatest mathematicians living today are woodworkers and work in the construction industry. Have you seen some of these YouTubes of these guys using, they're called speed squares. And I actually went to Lowe's and I bought myself a speed square, which mostly I just kind of spin around on my finger because I have no idea how to use it. You're probably not using your speed square to its fullest potential. So here are seven uses from the most common knowledge to the least. These guys are like, okay, we've got a 21-degree angle, but we need a 48 oblique coming off at this slope of the roof. So we're going to take three-sevenths of the 716th on the 21-degree. And he's like ripping through this. And I'm like, this guy is brilliant. Take the square and lock it on the pivot point. Subtract three and three-eighths. Flip the measurement in half.

This will give us 19 and a quarter for our run. Then flip the square over. Our rise will be a 612. our rafter length is 21 and a half. He never spent a day in college, and he is one of the greatest mathematicians alive. I know, why are the kids who took woodchop smarter than we are now? But I've got to tell you, as I watch these things, and some of the videos I watch use the imperial system, some use the metric system, and I cannot tell you how much I just want to move to Europe and use the metric system. So I grabbed right before we started recording, I have a pretty nice set of drill bits. All right, and I'm holding them right here. All right, I open it up and there are two sides. I am going to read off the markings that they have in sizes that goes from small to large. And folks in Europe, I want you to listen carefully to this, thinking about your own system.

All right, so here are my drill bits. 1 16th, 5 64th, 3 32nd, 7 64th, 1 8th, 9 64th, 5 32nd, 11 64th, and 3 16th. What is wrong with us? I like that you had to say at the beginning, from smallest to largest, because we wouldn't know. There's a true story. It's not apocryphal. One of the fast food joints wanted to compete with the quarter pounder, and they launched the one-third pounder. And it failed because people thought a third was less than a fourth because three is less than four. That is a real story. That's a real sad story. We're taking on McDonald's. They've got the quarter pounder. We now have the third pounder. Same price. Why would you do that? Three is lower than four. less burger for the same money. No matter marketing, he's going to trick me out of that.

All right, so I'll put this in units that I know for a fact that you like from past conversations. Okay. How many barley corns is 7 32nds of an inch? I believe it's 3 barley corns per inch, so I would need to take 7 32nds of 3 barley corns, which is 21 32nds, which is just slightly less than two-thirds of... Wait, what were we... Damn it, where's my speed square? What are some of your favorite? Well, I really like the non-standard units. Those ones are fun to me. So I did not know this until recently. The butt is an actual unit. Are you familiar with the butt? I am familiar to the extent that in common parlance when I'm talking, I use the term buttload. Is that linked? because I have never known the etymology of that's a buttload of beer. What is a butt? It's a real thing. It's a measure of like 126 gallons of wine or something like that.

And one butt is two hogsheads. But I just like that. Buttload is a real thing. Okay. I've always wondered what a hogshead was. Well, it's half a butt. It's half a butt. Duh. I love some of the sillier ones. We've talked before about the Smoot, the guy's length over Harvard Bridge. That one is great. But you know one I heard recently that I loved? It's a measure of beauty, and it's called a Helen, and it is the one unit of beauty required to launch a single ship. I like that. My all-time favorite measure is the micro-century. Are you familiar with the micro-century? I am not. Okay. Well, the micro-century is one millionth of a century. Supposedly, mathematician John von Neumann said that one millionth of a century is the perfect length of a lecture. Because if you do the math, it comes out to be 52 minutes, 35.7 seconds.

That is the perfect length of a lecture. One micro-century. I like that. I like it, too. And the way this banter is going, this podcast episode is going to be 1.5 microcenturies if we don't get on with it. Get on with it. Yes, get on with it. Yes. So speaking of which, what are we getting on to? Well, I love this conversation because it revolves around units. And so far this season, from time to time, the concept of units of measurement has crept into a lot of things that we've talked about. Well, we got off the rails a bit in the standardization episode where we started talking about how many middle schools a failed rocket represented. How much was that prototype? Four. Million? Middle schools. But the stuff we've talked about before, we're all on measured kinds of things. We have three barley corns. That's right. What happens if we don't have some observed measure?

We've got something latent, but we want to put on some kind of scale or metric to that as well. And that's actually a whole new ballgame. Exactly right. So to lay the groundwork for that, if I gave you a simple regression where we are predicting kids' weight, that's the outcome variable y, from their height, that's the predictor variable x, If I say to you that height is measured in inches and weight is measured in pounds and the slope is 3, what is the standard interpretation of that 3? A 1-inch increase in height is associated with a 3-pound increase in weight. Perfect. And the reason we can estimate that slope the reason we can interpret that slope is precisely because we have chosen units for X and we have chosen units for Y But you right When we get to this latent model and let pick a latent variable model that is really sort of foundational Let's take a confirmatory factor model with one factor, and it has four indicators.

So that model, we have a latent variable with paths coming into each of the indicators. and each of those indicators then has a residual variance associated with it. And if I just asked you to interpret one of those loadings, what words would you put around that? Well, it's no different than the regression. A one-unit change in the factor is associated with a lambda unit change in the indicator, where lambda is the factor loading. Perfect. With the ever so tiny detail that we have no idea what a one unit change in the factor represents, right? Because God does not tell us what units this factor is in. For me and the Lord, we've got an understanding. We're on a mission from God. Right? If it's motivation or it's depression or it's grit or it's growth mindset or whatever it is, it has no inherent units. So for you to be able to say a one unit increase in the factor, we have to know what the hell one unit increase in the factor actually represents.

And if you just tallied up all the parameters in that model, you might think that you're okay. Because if I just walk around that model, the factor is exogenous, so it has a variance. I have four loadings. I have four error variances or residual variances. That adds up to nine parameters. and this model has four variables. How many variances and covariances do I get out of four variables? Quick, quick. Ten. Wow. That's your parlor trick, right? That is my parlor trick, my one and only. So if someone had 20 variables, how many variances and covariances does that give you? 210. Damn it, that was good. I just told my class yesterday, so it's P times P plus 1 divided by 2. Well, obviously, if you have P and you have P plus 1, one of the two numbers is even. So you divide that one by two and then multiply it. So 20 times 21 divided by two, you divide 20 by two is 10. 10 times 21 is 210. So there are 210 variances and covariances in a 20 by 20 covariance matrix.

This is why you are my only friend. And even that is touch and go. The episode is young. Yes, that is Patrick's great parlor trick. I don't know what a parlor is, but that's a great parlor trick. So for the four variable cases, Patrick said we get these 10 unique variances and covariances, and this one-factor CFA model with four indicators has only nine parameters. So from an identification standpoint, we feel like, oh, yeah, this ought to work out. It does not work out when you have these nine parameters and these 10 variances and covariances from which to estimate those. we have a problem. Houston, we have a problem. And the practical problem, as Patrick indicated, is that in order to nail down a loading, we have to know what one unit of the factor represents. Patrick just said lambda generically, but he couldn't attach a number to that because he doesn't know what one unit of the factor represents. Even though we have more than enough pieces of information, this winds up being a problem mathematically because you get this

weird ridge along the likelihood that the optimizer can't settle in on. The Hessian matrix, which defines the curvature around the likelihood, that is technically singular, so we can't get standard errors. It's awful. What do you mean? What do you mean is this real wrath of God type stuff? Fire and brimstone coming down from the sky. Humans sacrifice, dogs and cats living together. Max is tarry up. So for practical interpretational reasons and mathematical reasons, we have to assign a unit to the latent variable. How do you assign a unit to latent variables when you do your modeling? So I learned it in two ways. The key is you have to fix one thing to a predetermined value. All right, we're going to have to go out into my garage. We're going to have to get a hammer and a nail, and we're going to have to pound a nail in to say, this is the metric around which we're going to build the entire model. You can do it one of two ways.

One, you can fix the variance of the latent factor to be 1.0. Now, what's interesting, you can actually fix it to any value you want. Anything at all, you can fix it to a smoot. You can fix it to a barley corn. You can fix it to one Helen. You can have one Helen for your latent factor. But we very often, if not universally, fix it to one because that standardizes the factor, right? So you could do that, but here's where things get interesting. You can randomly pick one of the items and fix that factor loading to one and estimate the variance of the latent variables. Notice all you did was dig up the body from the front yard and rebury it in the backyard, which is something is fixed to one. And here's the tricky part, and it really is hard to get your head around this, especially when you're new to the topic. Those are likelihood equivalent. So you can fix the variance to one, you can fix the loading to one, they fit exactly the same, they have exactly the same standardized solutions, and I was taught it don't make no never mind,

just pick one and get on with your work. But as you said in the true crime episode, the story we'd been told wasn't entirely true. And in fact, software often makes the decision for you, right? In the software that you use, is there a default? The typical default is whatever your first listed item is, and it's based on the mathematical statistical premise of it don't make no never mind. Yeah, and the never mind, it don't make no of. Nice, well done. You did, that was perfect. Your mom was the English teacher. I don't know if that was correct. Yeah, it depends on what that never mind is, right? So like you said, their likelihood equivalent, which means the fit indices that you're going to get out of the same, The standardized solution that you get out is going to be the same, but. And this is a big but. Wait, is it a butler? Is it a butler? Okay. Or is it a hogshead? It's 716 simple hogshead. So I want to create an example with you.

And from this factor model that we have, a single factor with four indicators, I now want to imagine that there is a measured predictor of that factor. All right, so one variable coming in with a path to the latent variable. And I am going to tell you, just for the sake of this example, that I have created a population where the factor for our purposes is in unit variance. The variance of the factor is not literally a parameter in the model, but for the purposes of this example, let's imagine that the metric of that factor is standardized. Let's also imagine that the metric of that measured predictor is standardized, just so we can talk about stuff. The loadings of my four indicators, let's imagine that in a standardized metric, I have varied them from pretty darn good, 0.9 to 0.7 to 0.5 to 0.3. So varying from a good strong loading down to a much, much weaker loading. Imagine I set all the residual variances so that they standardized out the indicators.

and now I have a path coming from that exogenous variable into the endogenous latent factor. Let's imagine it's a standardized 0.4. Not crazy strong like a 0.7 or 0.8, but not super weak like a 0.1 or 0.2. It's a 0.4. And I'm going to actually analyze this using a population analysis. So not a simulation. I am going to feed the model implied matrix for these five variables. It's not four anymore because there's an additional predictor. I'm going to feed the variance covariance matrix for these five variables into an analysis of a population. I'm ignoring the mean structure. And because you have to assume a sample size, I'm going to assume a sample size of 200. And I will tell you what comes out, Mr. It-don't-make-no-nevermind. If I set the best loading to 1, which is the first loading in this example, everything rescales to be aligned with that 1. and I will get out a significance test associated with that structural path

coming in from my predictor to my latent variable. I get a Z value associated with the test of that. When I choose the best indicator, it is a Z of 5.54. When I choose the next best indicator if it didn make no never mind I would expect to get the same thing but no when I choose the next best indicators the scale referent the Z goes down to 4 And when I choose the next weakest indicator, the Z value for that structural path goes down to 4.38. And when I choose the worst of the indicators, it is a Z of 3.27. Now, the don't make no never mind could say, well, those are all significant, so whatever you found it. But you can imagine that they might not be significant, right, that some of them could drift down. And if you think about these from a power standpoint, relative to choosing the best indicator, to achieve the same amount of power choosing the second best indicator, I would need actually an additional 57 subjects.

And now all of a sudden you're like, whoa, to be equal in power? Yes. If I chose the third indicator as my scale referent, I would actually need 122 more subjects to equal the power that I would have with that best loading. If I chose the weakest loading as my scale referent to make it have equivalent power to choosing the best one, I would actually need 382 more subjects to have equal power. So when we say it don't make no never mind, it don't make no never mind in the likelihood, don't make no never mind in the standardized solution or in the fit, but it makes a huge never mind in the tests of the parameters that are directly associated with that latent variable. In my defense, I was over-served. Okay. I think a variation of that applies here. Okay. I've had three or four SEM classes over the years, and in total fairness to some really, really talented instructors,

I may have been daydreaming, I may have missed a class, I, Lord knows, may have misunderstood something. I never walked out of a lecture with any different understanding than it don't make no never mind. Don't worry your pretty little head. Pick whatever you want. Better still, let the computer select whatever it wants to do. This is shocking. And I'm not being hyperbolic about that. that the power of the predictor of your latent factor can vary by several hundred subjects, depending upon which item we set to one. When we are scraping and begging to get 10 more subjects, 20 more subjects, this is mind-boggling. Yeah, and I think one of the reasons that we are told that it doesn't really matter is on the assumption that we are getting comparable quality indicators of our factor, right?

So if you have a bunch of measures, you have a bunch of questions that, you know, are about the same quality in terms of how well they reflect the latent variable, they really shouldn't matter a whole lot. But when you have range among them, any degree of heterogeneity technically, then there is this disparity in terms of the power for the structural parameters. And people did know this, like Jorskog and Sorbaum wrote about it in 79, DePiper and Saris wrote about it in 82, Satora wrote about it in 89. Our friend, Rich Gonzalez, Gonzo, had a great paper in 2001 in Psych Methods with Dale Griffin that was called Testing Parameters in Structural Equation Modeling, Every One Matters. It's a great title. Yeah. I hate pun titles, but that one is pretty good. It's pretty good. But it's also Gonzo who wrote it, and anything he does is cool.

For sure. So this has been known. It's just that we let the software do the thinking for us, and that doesn't seem to be a great idea. We do, but in fairness to me, which is what's important here, is Gonzo's paper only came out 25 years ago. And I'm still trying to catch up on a backlog of readings, so first, cut me a little bit of slack. Okay. But second, there's a circularity as we let the computer make the decision because we were taught in class it doesn't matter. I really like your point that oftentimes if we have a set of indicators that are all more or less the same value in the loading, that this is much less of an issue. That doesn't matter to me because what you're doing is showing the proof of concept that power varies as a function of which lambda you set to 1.0 is kind of staggering. It is. Well, what if I told you there's another way?

When did this become an infomercial? Do you want the set of steak knives or don't you? I do. There are 716th steak knives, too. Well, so there are actually a couple of other options. Well, technically, there are many more options. But one option that was proposed 20 years ago by our friend Todd Little, along with Sleegers and Card, they had a paper in the Structural Equation Modeling Journal in 2006, where they discussed something that they referred to as effects coding. I don't love that name, but the name comes from when we do like contrast codes in ANOVA. And the idea of effects coding is that we do not constrain one of the loadings to equal one. We constrain the average of the loadings to be one. So we are not putting a hard constraint on one. We're doing it on the whole set. And this is enough to resolve the mathematical under-identification problem that we have so that we wind up getting a solution out.

What do you think about that? I think it's incredibly clever, and it's not something I've ever done in my own work. Remember, with the example that Greg gave, with the four-indicator latent factor, we need to impose one restriction. We can set the variance to one. We can set a loading to one. And I'm not exaggerating. Set one of the loadings to the square root of pi. I don't care. It just has to be set to something. And by equating on the four loadings, you actually are imposing that restriction because it's a degree of freedom problem. When you have three initial values, it's not how it's being done in the estimation. But if you have three values and you have a restriction imposed, the fourth has to take on a particular value to fill that restriction. And you have put one restriction into the model. You like it? I think it's terribly clever. It's like cutting the body up and spreading it all over the property instead of having it just in the front yard or in the backyard. So there's that.

There are many advantages to this approach. I don't remember that in the article, but yeah, it's just like that. It might be in the online appendix. Okay. So when we fix a loading to one, the factor inherits the metric of that individual variable. But when we fix the average loading to one, that means then that a one unit increase in the factor corresponds to a one unit increase in the average expected indicator response. So it's aggregating across all of those. And that gives me sort of a plus minus vibe here. If all of my indicator variables are in the same units, you know, like a 10-point scale or something, then that makes some sense to me because it's really just drawing from the same metric across all of the different variables. But if I have one variable that is SAT score, another one that's GPA, another one that's number of days attended in a given school year, you know, whatever these measures might be, then it starts to feel a little bit funnier from an interpretation standpoint.

Now, from an implementation standpoint, that's easy enough, but knowing, if that's important to me, knowing what the heck the factor unit represents, that's a little trickier. I like that I don't have to make a choice and I don't outsource the choice to the software. I think that's a cool thing. It's very easy to implement. You just one line of code in Levon or in M plus, that's pretty easy. You know that example where I had heterogeneous loadings that in the standardized metrics ranged from 0.9 all the way down to 0.3. If I used this constraint, I would need only a couple more subjects than when I had chosen the best of the loadings. That's a nice thing, that when you have such heterogeneous loadings, you come out with something that's pretty darn close. Better than choosing the second, third, or fourth one. That's a good thing. You're exactly right, but it makes me think about something. You putting that average loading of Beam 1 and it has the advantages that you just described you still left with whatever standard error it is that you have on that regression coefficient But instead of going on a snipe hunt and saying what happens if you fix this to one

What happens if you fix this to one? Is there a way of back calculating and saying, this is the one that would maximize the power that I would have for that regression coefficient? Oh, the word maximize. That is disgust. Remember we learned on an earlier episode, if I don't understand where something's going, I'd just say, discuss. So technically, this effects coding method does not maximize your power. This gives you a pretty good power for that Z value of the structural parameter that we're talking about, but it doesn't maximize it, and it is possible to do better in a couple of different ways. One is in a linear sense, and what I mean by that is we just constrained using this effects coding method, we constrained the loadings using a linear combination of them. It was an equally weighted linear combination of those loadings that we constrained to a value. Why not use a different linear combination, right?

What if we had maybe I get a little bit more of variable 1's loading. Maybe I get a little bit less of variable 4's loading. Is there some way that maybe we could come up with an optimal weighting scheme that maximizes the power that we would have for a particular structural parameter? We could call that a linear type of constraint because it's a linear combination of the loadings that we're constraining. Technically, we could have nonlinear combinations, too, where you could just go nuts and say, I want to take a combination of the squared first loading divided by its residual variance plus twice the second loading divided by Patrick's birthday. You could go nuts with all kinds of crazy constraints. If we just keep it in the linear family where we ask, is there some optimal weighting that we could have for these loadings to set that equal to a constant instead of just equally weighted? The answer is yes, there is an optimal weighting. weighting, and it's not too hard to derive, it turns out, and we could figure out what weights

we would need sitting out in front of each of those loadings to give you more power than choosing any one of the loadings. It will give you more power than that equally weighted combination in the effects coding that Little and colleagues came up with. What do you think about that as an idea? So I think that's four Helens in attractiveness. He just launched four ships. I still think it don't make no never mind as long as you pay attention to model fit and standardized effects. But it makes a massive difference when you're talking about study planning, precision, and power. Yeah, definitely four Helens. Okay. I might be able to knock a Helen or two off, though, if I tell you about some of the problems with it. Do you want me to do that, or should we just leave it at four Helens? So give me one Helen worth of problems with it. But wait, there's less!

Okay, here's one bit. If we have a model where that latent variable is embedded in a whole lot of other action, right, and the simple version of that is we've got a predictor of that factor, and we've got an outcome coming out of that factor, When we talked about the effects coding thing, we just said lay that blanket over the factor loadings that forces their average to be one. In this one, we would say, well, what parameter are you optimizing for? The combination of loadings that I would get to be able to optimize my power for, let's say, a path coming into the factor will be a different combination of the loadings than what I would need to maximize the power to detect that path coming out of the factor. And so in this example, there is no one-stop shopping optimization that could occur. You would have to tailor that to every focal parameter associated with the factor. I don't like that at all. three hellens. I got to tell you, we're down to a drunk guy in a rowboat at this point.

Because you're exactly right. Overall fit doesn't change. The standardized solutions don't change. But now you've got to pick your favorite child. I like the idea of this. But yeah, when did you say we're down to a rowboat? Yeah, a drunk guy in a rowboat. Drunk guy in a rowboat. His name is Herbert. It's one Herbert. How many Herberts are you going to launch? Which is the drunk guy in a rowboat. All right. So the idea of it, the idea of optimizing, that doesn't bother you. But, yeah, I'll just leave it at the drunk guy in the rowboat. I won't poke at it too much more because then I'll sink that guy. But if we back off that a little bit and we go back to the effects coding, where all of the loadings average one. A couple more, Helens? I like that. I mean, it's all over the yard. The police dogs are going to have trouble finding everything. But, I mean, it has a beautiful symmetry of you have a set of indicators on the latent factor. You're not picking an only child.

You're allowing all indicators to speak to the setting of the metric of the latent variable. Those with a louder voice are going to be up-weighted. Those with a softer voice are going to be down-weighted. I like that better than picking a single item out of those that you have. That sounds to me like a very reasonable and defensible default. I'm up to like three Herberts on that. Three Herberts in the boat. I think average loading a one, that's three Herberts. Beautiful. Okay. So I think the take-home message for this conversation is that you have options, and you might not have even known that you have options, right? You might have been taught, and I know in my own class, I teach some version of it, don't make no, never mind. And I sort of tuck it underneath the heading of, well, if you've done your homework and chosen good indicators, yada, yada. But when your indicators are not of comparable quality, and there are scenarios where they absolutely might not be of comparable quality,

then you do have choices beyond just, hey, M+, could you decide for me? You have some choices. Exactly. And I would have to say I have not done as good a job in teaching this as I will now, because I really do teach it. Don't make no never mind. I will make a recommendation to either pick your strongest indicator or pick one that is most theoretically important to the underlying construct. But I do not lay out that the standard errors of other things in your model are impacted by that. I say their likelihood equivalent, they have the same chi-square, they have exactly the same standardized solution. Pick the one that helps you sleep at night and get on with it. So your teaching will be a buttload better now. I think it'll be one Herbert better. I don't think it's two hogsheads worth, but I think my teaching has improved by one Herbert. Okay. Take care, everybody. Thanks for being with us.

Thanks so much for joining us. Today's episode has been sponsored by alternative, more academic methods of scaling latent variables. Instead of effects coding, try group project coding. One loading does all the work. Everyone else receives the same credit. Which is not the same as faculty committee coding, where all indicators end up getting equal weight, but only after six meetings, three procedural motions, and two indicators storming out of the room. Or full professor coding, where the oldest indicator gets all the weight, even if it doesn't do very much work anymore and is virtually impossible to remove. Or donor coding, where the indicator with the largest financial contribution gets the factor named after it. And finally, inevitably, reviewer two coding, where the largest weight actually goes to an indicator you didn't originally include in your model and still really don't think it belongs there.

But... Be sure to tell your friends to subscribe to Quantitude on Apple Podcasts, Spotify, or wherever they go to feel like they're the one that matters. You can also follow us on X, Blue Sky, or Instagram, where we are at QuantitudePod, and visit our website, Quantitude.org, where you can leave us a message, find organized playlists, show notes, syllabi, and other fun stuff. And finally, get cool Quantitude merch like t-shirts, hoodies, mugs, notebooks, and stickers from RedBubble.com, where all proceeds from non-bootleg authorized Quantitude pod merch go to DonorsChoose.org to help support low-income schools. You have been listening to Quantitude, the podcast that's been constraining you in non-optimal ways since 2019.