個人内効果と個人間効果の分離:トロイの木馬とトファーキー

S8E08 Disaggregating Within- & Between-Person Effects: The Trojan Tofurkey

Quantitude41分
個人内効果と個人間効果の分離:トロイの木馬とトファーキー

個人内効果と個人間効果の分離:トロイの木馬とトファーキー

Quantitude

0:0041:31

要約

PatrickとGregが、予測変数Xに「個人間(特性的)」と「個人内(状態的)」の2つの情報が混在する問題を、概念面を中心に議論した回。多くの理論は個人内効果を想定する一方、測定・標本・統計モデルは個人間効果に偏っていると指摘する。反復測定による分離の利点や、横断データでの工夫の余地にも触れ、研究課題と分析の対応を確認することを勧めている。

この回のポイント

  • ●Patrickは、ほぼすべての理論が暗黙的・明示的に個人内効果を扱うのに、測定・標本・統計モデルはほぼ個人間効果に集中しており、理論と方法の間に大きなズレがあると述べている。
  • ●1時点のXやYは「トロイの木馬」であり、個人の特性的な平均と、そこからのその時点のずれという2つの情報を同時に含むとGregは説明する。
  • ●問いには、個人間の関連、個人内の関連、因果という階層があり、分析がどの水準の問いに答えているかを意識すべきだとされる。
  • ●運動と心臓発作、自信と成績、SNS利用と青年の幸福度などの例で、個人間と個人内で効果の向きが逆になり、混ぜると打ち消し合うことがあると示される。
  • ●反復測定があれば、時間的先行性の確認や相互効果の検討に加え、個人の特性的水準を推定して個人内の偏差を分離できる。方法は個人平均中心化と、潜在因子を使うモデル内での推定の2種類。
  • ●Patrickのシミュレーションでは、反復測定データから1時点だけ抽出して回帰すると、係数が毎回大きく変わり真の効果とも無関係だった。ただし横断データが無価値だとは言っていない。

章立て

  1. 感謝祭の雑談とトファーキー

    七面鳥とトファーキーの失敗談から、感謝祭で人が普段と違う様子になる話へ移る。

  2. 文脈で変わる自分:個人内の視点

    バンドの例や小さなキリンの例で、集団内での位置づけと自分自身との比較の違いを説明する。

  3. 理論と測定のズレ

    理論は個人内効果を想定するが、測定と統計モデルは個人間に偏っていると指摘。回帰の傾きの解釈の落とし穴も語られる。

  4. トロイの木馬とミルの因果条件

    Xが特性と状態の両方を含む点を示し、問いの階層とJ.S.ミルの3条件に照らして問題点を整理する。

  5. 個人間と個人内が打ち消し合う例

    運動と心臓発作、自信と成績、SNSと幸福度の例で、分離しないと効果が見えなくなることを示す。

  6. 縦断データによる分離の方法

    反復測定の利点と、個人平均中心化や潜在変数モデルによる分離を紹介。文献も挙げられる。

  7. シミュレーションと横断データの工夫

    1時点抽出の回帰が不安定になる実験を紹介。研究課題と分析の対応を確認し、心理測定での工夫の余地を提案する。

解説記事

今回の Quantitude では、Patrick Curran と Greg Hancock が、予測効果を「個人内」と「個人間」に分けて考える問題を議論した。統計の詳細は別の機会に回し、概念面に焦点を当てている。

「普段との違い」と「他人との違い」

導入は感謝祭の雑談だ。Patrick は自分で調理する唯一の品、トファーキー(菜食用の七面鳥風料理)がオーブンで爆発した話を披露する。Greg は感謝祭に集まる人々が普段と別人のようになると語る。ここから、人は日によって、また状況によって変動するという話に移る。

Patrick はバンドの例を挙げる。一方のバンドでは上手い奏者として扱われるが、数メートル歩いて別のバンドに座ると、周囲が熟練者ばかりで自分は上手くなくなる。Greg は、これは集団内での相対位置の話であり、多くの研究上の問いは「自分自身と比べてどうか」だと整理する。「今日は髪型がいい」という言い方も、その人の普段との比較だという例えである。

理論と方法のズレ

Patrick は、ほぼすべての理論が暗黙的または明示的に個人内効果を想定している一方、測定・標本・統計モデルはほぼ個人間効果に集中していると主張する。例えば不安と飲酒の関連を、多数の人から1回ずつ測って回帰する場合だ。傾きは「Xが1単位上がるとYが平均b変わる」と読まれるが、実際には誰のXも上げていない。異なる人どうしを比べただけなのに、私たちは「個人を動かしたらどうなるか」と解釈しがちだと Greg は指摘する。

Xの中のトロイの木馬

Greg は、XとYがそれぞれ複数の情報を含む「トロイの木馬」だと説明する。ある人の不安が5、別の人が6でも、前者の普段の水準が3、後者が8なら、前者は普段より不安で、後者は普段より落ち着いている。1回の測定値は、個人の特性的な平均と、そこからのずれが混ざったものだという。

問いにも階層がある。(1) 普段から不安な人は飲酒も多いか(個人間の関連)、(2) 普段より不安な日に普段より多く飲むか(個人内の関連)、(3) 不安を下げれば飲酒が減るか(因果)、である。分析が実際にはどの水準に答えているかを見極める必要があるという。

番組はジョン・スチュアート・ミルの因果推論の3条件にも触れる。原因が結果に先行すること、両者に関連があること、もっともらしい代替説明がないことだ。1時点の測定ではこの3つとも満たしにくいと述べる。不安が飲酒を生むのか逆か、環境ストレスが両方に影響しているのかも区別できない。

打ち消し合う効果の例

個人間と個人内で効果が逆向きになる例も示される。運動をよくする人は平均的に心臓発作が少ないが、運動中は心臓発作のリスクが高い。両者を混ぜると、効果がゼロ近くに見えることがあるという。自信と学業成績では、平均的に自信のある人ほど成績が良いが、ある試験で過信すると準備不足で成績が下がり得る。SNS利用と青年の幸福度でも、平均的に利用の多い青年ほど幸福度が低いことは、利用が多い日に幸福度が下がることを意味しないと述べる。

縦断データでの分離と、その先

対策の中心は反復測定である。時間的先行性を扱え、相互効果を見られるうえ、十分な回数があれば個人の特性的水準を推定できる点が大きな利点だと Curran は述べる。例えば20日分の測定があれば、その人の普段の水準を見積もり、日々のずれを個人内成分として分析できる。

分離の方法は2つ紹介される。1つは個人平均を計算し、各測定値から引いて分析単位にするデータ処理的な方法。もう1つは、構造方程式モデルで潜在因子やランダム切片を用い、モデル内で個人間成分を推定する方法である。ただし縦断データでも受動的な観察である限り、因果が確定するわけではないと注意している。

Curran は準備中のシミュレーションで驚いたと語る。200人×10回の反復測定データから各人1時点だけを抽出して回帰すると、係数は真の個人間効果にも個人内効果にも対応せず、抽出のたびに向きまで変わった。ただし彼らは、横断データが価値ゼロだと言いたいわけではないと強調する。研究課題、基準となる比較対象、実際の分析が対応しているかを毎回確認することが肝心だという。

Greg は、縦断データを取れない場合のために、心理測定の面で工夫の余地があると提案する。例えば「普段の自分」についての項目と、「普段より孤独なとき」についての項目を組み合わせるような設問設計だ。ただしこれは番組内でも「アイデア」にとどまり、既存研究の有無までは断定していない。関連文献として、Curran と Bauer による2011年の Annual Review of Psychology の論文と、1972年の Joaquin Woolhill による年齢を独立変数として扱った論文が挙げられている。

まとめ

この回の核心は、「XがYを予測する」と書くとき、それが個人間の話なのか個人内の話なのかを明示せよという点にある。日本の研究者や実務家にとっても、行動データやアンケート分析で、1時点の差を「介入したらこう変わる」と読み替えていないか見直す契機になる。ウェアラブルや日次調査などで反復測定が取りやすくなった今、個人平均中心化や潜在変数モデルによる分離は現実的な選択肢だ。一方、横断データしかない場合も、研究課題と分析の対応を確認し、限界を認めたうえで解釈する姿勢が求められる。

文字起こし(英語・自動生成)

Welcome. My name is Patrick Curran, and along with my always stable trait-like friend Greg Hancock, we make up Quantitude. We are a podcast dedicated to all things quantitative, ranging from the relevant to the completely irrelevant. In this week's episode, Greg and I explore the fascinating yet sometimes tricky topic of disaggregating within-person and between-person predictive effects. We primarily focus on the conceptual issues and defer the statistical details to another day. Along the way, we discuss pilot cooking a pre-registered turkey, planting a flag on a tiny hill, elephant suppositories, exploding aliens, unstable family members, birdland, go cubs, tiny Trojan horses, the Patrickverse, grunting corpses, bad hair days, Duolingo, if it isn't Scottish, and squaring the circle. We hope you enjoy this week's episode. So tell me, my friend, how many turkeys have you pre-cooked so far?

I think I was very clear last year that I am never, ever making turkey again. There is no way that I can cook it, that it comes out well. And the year that I made a pilot turkey, which did come out well, the final one sucked. I swear up and down, I am not making turkey this year, even though I saw that one was purchased and it's in the fridge. Damn it. You're so cute when you plant a flag on a hill. I know. That is like the tiniest little hill of all the hills. Folks, you should have seen his face as it was like defiance. This was the Scots defending the border of Scotland. Would you be willing to train all the day from this day to that for one chance to come back here and tell our enemies that they may take our lives, but they'll never take our freedom? And then at the end of that story, he says there's a turkey in the refrigerator.

Okay, let me be clear. At least the hill I'm dying on is made of meat. Oh. Mr. Tofurky. That was wildly unnecessary. I was being nothing but supportive of you in my open mocking on a public forum. Please tell us about your Tofurky. Do I test microwave the Tofurky? A pilot Tofurky? I love that you don't even use the word test cook. That's what everybody uses. You pilot. You're going to pilot your turkey. I have a lovely spouse and two lovely children, and the three of them are vegetarian. Yes, we have a Tofurky. I do not need to pilot test it, get IRB approval. It's pre-registered. It's pre-registered. Oh, God. I never remember what kind I buy every year. So I go to Whole Foods, and they, shockingly, a whole bunch of Tofurky's to choose from. Okay. I do all the cooking, and on Thanksgiving Day, I'm like, okay, I'm going to get the Tofurky, and I pull it out.

I swear to God, it looks like a suppository for an elephant. I don't even want to know how you know. Listeners, picture in your mind's eye, if you were about to insert a suppository into an elephant, this Tofurky would be if. It was wrapped in plastic, and it looked like a giant Tylenol. So I'm like, all right, and I read the instructions, and I'm not kidding. It is, like, hermetically sealed in this plastic tube. I have followed the instructions, I'm ready to take out, and I look in. Now the suppository is just 375 degrees. That's all that's changed. But for some bizarre reason, I decide it's not done yet. I have no basis for this. Like, I've never cooked a suppository before. So I decided to leave it in a little bit longer. Can you not say leave it in? Comes out, you put it back in.

It comes out, you put it back in. So I decided to leave it in a little bit longer. I'm over doing something on the counter, and I hear, I was like, oh, that didn't sound good. And I opened the oven, and, dude, it was like out of an alien movie. The thing had exploded. I heard some squealing or something. Oh, no, everything's fine. But the animal is inside out. I heard that. And it exploded. The inside of the oven was coated with some kind of nutritional paste. I don't know what the hell it was. Two things occurred to me. The first was, yeah, I probably should have followed the instructions. And the second is I start trying to scrape it off the walls of the oven onto a plate. Because that's dinner. That is the marquee piece. And so I've got this like steaming nutritional paste that I have a spatula and a plate that I'm scraping off of the sides.

And then I get what I can out of it and I try to reform it into a turkey. I think this is where I conceded the battle. And I was like, all right, there is no tofurkey. and I literally just threw it away. It didn't matter because we eat nothing but sides anyway. Yeah. That was my pilot testing of the pre-registered elephant suppository Tofurky brought to you by Whole Foods. But I only have three other people to please. You stock the house with more than three people, each of whom has an opinion about something. So I was raised on big Thanksgivings where people come in from out of town. You try to imagine all the different variables you could manipulate to maximize the stress of the event. And yes, the event is much, much more complicated. And I have to say, if I'm looking at this historically, the people who have been a part of my Thanksgivings over the years,

I barely even recognize them at Thanksgiving. My brother and I used to have this game before any major holiday where we would bet which room my mom would lock herself into. God bless you, Mom. But there would be some conflict where Mom would just be off the charts, unable to handle things. You okay? Do I not look okay, Natalie? Not really. Am I okay? People are just very, very different at that time. That's a very nice way of saying it. Yes, very different. There's variability. We see this all the time, though, right? You think about anybody in your life, during your day, they were more talkative in class than they usually are. That's the implication. Sometimes you don't say that second part. Oh, they were really talkative today. Patrick was actually funny. I've heard that. Somebody has said that. Patrick was actually funny. But we see it all the time. And you know what it makes me think about?

really big squirrels and really tiny giraffes. And I've got to tell you, last night I played a gig. I'm in two different bands. Okay. Where the first band, we opened, and then a bunch of us scurried over to the second band that we're in, and we played there. And in the first band, I'm not a great player. I'm a decent player. I'm a reasonably good player. I've got some first parts. I've got some lead parts. I've got a solo in Birdland. I walked 25 feet and sat down in the second band, and I'm not a good player anymore. Because to the left of me is a music school graduate who has played for 40 years. To the right of me is a guy who subs for the North Carolina Symphony. Am I a good player? Well, it depends. Am I playing in the first band, or am I playing in the second band?

I think as a field, it's easier for us to think about this in grouped kind of settings. I think we struggle more when we apply the very same concept, not to an individual within a group, but to the individual itself. And you just made a really important pivot right there. You mentioned one thing very, very quickly about, did you say big squirrels and small giraffes, something like that. There is a standing that each of the squirrels has with reference to the other squirrels, and one squirrel could be absolutely enormous. It's still probably a really small suppository. I can't beat you to that. And then there's the really, really small giraffe, but small relative to the other giraffe. So there's these comparisons within, and those are very important comparisons for understanding the context at a group level. But so many of our questions are really not even about that, right? The questions are about where we are relative to ourselves.

So when I tell Patrick he is having a really good hair day today, for example, what I really mean is that his hair usually looks like crap. And today, it's a very, very nice haircut that you have today, my friend. I'm wearing a baseball cap. Exactly. Go Cubs. Next year. Go Cubs. Next year. That's our year. And two down in the top of the ninth. Kelly swings, grounds it to score. The throw to first is in time, and the Padres have swept the Cubs. But here's the problem. I would argue that nearly, not all, but nearly all of our theories are implicitly or explicitly focused on within-person effects. Yet nearly not all but nearly all of our measures samples and statistical models are focused on between effects And I think there a massive disjoint both theoretically but also statistically But it seems so simple We have a single variable X

and we have a single variable Y. And all the way to high school stats, we are taught that we want to use X to predict Y. So imagine just as an example, we want to know is anxiety and alcohol use related. How easy is that? You get a sample of individuals, you assess their anxiety, you assess their alcohol use, and you try to make an inference on average or higher levels of anxiety associated with higher levels of alcohol use. What could possibly go wrong? Oh, and we are likely part of the problem here, right? Because when we teach people to interpret that slope, there is the true mathematical interpretation, but then there's the little bit more colloquial social science-y interpretation. So that particular slope, then we would say for a one unit increase in x, the expected value for y changes by slope b, whatever that particular value might be. But in that language,

the idea of a one-unit increase in X starts to imply, or at least we vaguely infer, that that means if we increase X, and by the way, we didn't increase X for anybody here, right? We went and got entirely different people who had one unit of X that were higher. And yet our brain makes the leap that if we increase X by one unit, then people will respond by increasing by B units and whatever the outcome is. So our hearts really are gravitating toward much more of a change within individuals, but we haven't even looked at individuals more than once here. Have you seen The Odyssey, the recent movie? I haven't yet. I know I need to. Oh, yeah, he dies at the end. Damn it! Damn it! Look, I saved you three hours. You're welcome. No one can stand between me and home. I live a god.

I think about the thing you just described is we have X and we have Y, and each of those is a little Trojan horse that contains more than one piece of information. And we never like that in statistics. We never like one thing to capture two or more things at the same time. And what is in that little Trojan horse? Well, let's think about the anxiety. Let's say that you and I are in that study, and I report an anxiety of five, and you report an anxiety of six. Well, obviously, you're more anxious than I am. And then we're going to have your alcohol use and my alcohol use, whatever those numbers might be, and that's what goes into the data file. So you're one unit higher in anxiety than I am. But what else is hiding in that little Trojan horse? So I have a five, you have a six. Obviously, you're more anxious. But my trait-like anxiety is maybe three. All right, so I'm two units higher than I usually am.

I'm more anxious than I usually am for me. But let's say that yours is eight. You're just kind of generally an anxious person doing bonkers things like arranging your forks in the dishwasher in a particular way or having your napkins on the table at 90 degrees. Just hypothetically. Just hypothetically. But your six is two units below your treat when I ask you that question. So my score of five is lower than your score of six, but I'm actually more anxious than I usually am, and you're less anxious than you usually are. yet those are both hiding inside that Trojan horse. I don't think we think about that at all in the things that we talk about, that my X at this particular time is made up of my overall average X plus how much my current X deviates from that.

And so I've got this little stable thing and this other little piece, but we only have one snapshot. And the questions that people ask either ignore that entirely or maybe they have some nod to it in the way that they gather the data, but they still might not be analyzing the data in a way to fully unpack that and fully address the question at hand. The way I think about it is that there is this hierarchy of questions that we ask. The general one that you started with, do people who are generally more anxious also drink more? That, to me, is a very between-person association question. That's just asking about different people. But we might not actually mean that question. We might really want to say, when someone is more anxious than usual, do they drink more than usual? That is very much a within-person association question, that there is this distribution of Patrick's own anxiety. There is this distribution of Patrick's own drinking behavior.

And if I went into the Patrick verse and drew out some data points for him, what I'm really asking is, is there an association between Patrick's anxiety and Patrick's drinking, as I would simultaneously be asking that question for me and for everybody else? That is still an association question, to be clear. That is an association question that is within people. There is still another layer of question that we have to wrestle with here, and that is literally, if I went into Patrick and did something to reduce his anxiety, would that then have an effect on his drinking? So there's this hierarchy from between association to within association, and then ultimately to the causal question. And the analyses that we do sometimes fall at one level, sometimes fall at another level, but there's often a mismatch between what we really want to answer and the type of analysis that we choose.

And I think that's the most important point, other than the Tofurky story of this episode, is that mismatch between theory and method. I would argue that the majority of theories implicitly or explicitly predict within-person effects. Between obviously exists, between are important, between are legitimate questions, but if you're talking about I hypothesize that people who are higher on anxiety also tend to drink more is implicitly a within-person effect. Going back to your interpretation of the slope parameter, if you have a slope of 0.5, we literally say a one-unit increase on anxiety is associated with an average 0.5-unit increase on alcohol use. But your observation that I loved is, you're not increasing my anxiety from 5 to 6.

I have a five and we are moving to you who has a six. That's right. And when you start thinking about that, you're like, oh, crap, that is not what I meant to evaluate. No, we do this all the time. We somehow make this leap and pretend that different people are just different versions of us. And they're not. The people who sit at six are not the same people who sit at five. And yet we are trying to make some sort of claim about what would happen if I went into you and moved you up from a five to a six. And that is a massive leap. People do this analytically and then inferentially all the time. And you could argue that if you make this particular constellation of assumptions, that that could still be a valid inference. And that might actually be the case. But there are so many assumptions, so much legal fine print in being able to make that leap. Do you know why I have the scariest house in the neighborhood?

Do tell. So the order that we record these things is a weird timeline. Halloween is tomorrow, and I have the scariest house on the street. And the reason is I take an afternoon, I get a Coors Light, I get a shovel, and I dig up Fisher and Pearson and Socrates and Smitty Stevens, and I prop them up all over the yard. Agatha Christie goes up by the mailbox. I haven't had a trick-or-treater in like 10 years, so it's worked. It has absolutely worked. but one of the scariest ones that I put front and center is John Stuart Mill. And I was recently chatting with him because I went out to bring the recycling bin back, and he expressed some concern that I had dug him up. But in the chat, he reminded me that he proposed three conditions for causal inference. The first is the cause must precede the effect.

The second is that you have to have some observable, demonstrable relation between the cause and the effect. And the third is there are no plausible alternative explanations. The little Trojan horse of anxiety predicting the little Trojan horse of alcohol use violates all three of those. So if we took each one of those in turn, then, the idea that there is this temporal ordering of anxiety and drinking, How do we know it isn't the case that the more you drink, the more anxious you become? Just gathering the data the way you have, we can't even resolve that very first temporal ordering. I don't know if your mic picked it up, but there was a very happy grunt that came from the front yard from that first one. So temporal precedence. The second one is really tricky because that is there needs to be a demonstrable relation between the cause and the effect All right Well let think a little bit more substantively about how these between and

within effects might live together. And what happens if the term we sometimes have is they are confounded, right? That's the Trojan horse. A confound is the Trojan horse as our measure of anxiety contains two pieces of information at once. One is more trait-like, one is more state-like. Think about some classic examples of between and within. What is the relation between physical exercise and heart attacks? On average, people who exercise more are less likely to have a heart attack. However, you are more likely to have a heart attack when you are exercising. Okay, classic finding. Okay. Think about the people who stroke out on a treadmill and you say, see, Never should exercise, I told you. I've never seen a man so desperate to get out of five minutes of calisthenics. One, two, three, four. But there's a negative relation between exercising and your risk of a heart attack on average at a between level.

But there's a positive relation between exercising and the risk of a heart attack at a within-person level. that when you are highly exerting yourself, you are more likely to suffer a cardiac event. What happens when those two little soldiers are together in the Trojan horse? They cancel each other out. You look at the little Trojan horse just asking somebody, how much do you exercise and what is your risk of a heart attack? And you may find a near zero effect. Well, there's a huge effect, but it's being canceled out when you combine the between and the within. And then it's lots of fun to start thinking about other examples. There's empirical evidence about the relation between self-confidence and academic performance. On average, people who have higher confidence tend to perform better academically. But for a given exam, higher confidence might be related to lower performance because you're overestimating your knowledge and you don't prepare. So those are the two little soldiers in the course.

It's fun. This is like a game. I do this in class, thinking about these different things. Another one that I like that is really important in getting a lot of attention right now is social media use in adolescent well-being. Oh, yeah. On average, a demonstrated effect is that adolescents who use more social media tend to report lower well-being. That does not mean that on a day that an adolescent is using more social media than they usually do, that they are having lower well-being than they usually have. That's the within-person effect. So going back to John's grunting corpse out front, we may find no demonstrated relation between the cause and the effect, but that is because we have not disaggregated that within component from that between component. And as part of the fun that you might have in class, just coming up with other potential explanations of what you might have observed, right? If you think about the social media use and well-being, are there other variables that might be influencing people's well-being as well as influencing their social media use?

In order for you to establish that there is a causal relation, as per John Stuart Mill's grunting three criteria, there are probably tons of third variables that you could imagine that are related both to social media use as well as well-being. So this is a very difficult thing to try to unpack. And I love your last point, because right now, as John is grunting up a storm out there, I have never seen John this happy before. Not since Agatha offered him a cigarette. That was so sweet. And it was adorable because John doesn't have a lower jaw, so he can't even smoke it anyway. Go back to the simple anxiety and alcohol use example. Anxiety could predict alcohol use. Alcohol use could predict anxiety. It could be a between-level effect. It could be a within-level effect. And, oh, by the way, it might be that environmental stress is predicting anxiety and environmental stress is predicting alcohol use, and anxiety and alcohol use have nothing to do with each other, right?

That's that plausible alternative hypothesis. It's kind of a wonder that we get out of bed in the morning to try to figure this stuff out given the levels of complexity that are going on in a single predictor and a single outcome. The big thing comes from being able to try to disaggregate these effects, right? If we're ultimately going to want to establish that a change in X is actually leading to a change in Y, then we have to wrestle with the Trojan horse. We're going to have to try to disaggregate these effects. And so we move toward the longitudinal world where we say, okay, we're going to gather longitudinal data, but that doesn't necessarily elevate what we're doing to causal sainthood here, right? Just the fact that you have gathered things longitudinally doesn't mean you're still able to get at that particular question. In large part, because much of what we do in the social sciences is in some way or another passive observational work. And so we're taking a step closer to inferring cause and keeping John's corpse happy.

But we are still observing a process passively. For me, the first biggest step is recognizing we have a problem, right? That's the big thing you always say. Well, you need to recognize you have a problem. Hi, Patrick. Exactly. We have to recognize this is a problem at a theoretical level that when you say X predicts Y, what do you mean? Do you mean a between effect? Do you mean a within effect? Hey, you want to have fun? Do you want to interact the two? Because we can think about the magnitude of the within effect may vary as a function of the between effect. So your higher trait-like level of anxiety, does that actually potentiate the within-level effect? All right, so number one is stand in front of the group and say, my name is Patrick, and I recognize that this is a problem. Then step two is, okay, wise guy, what are you going to do about it?

The big game in town is as you're raising longitudinal, get repeated measures. What does this allow us to do? One, it gives us temporal precedence. Now, it doesn't mean we have the right temporal precedence. Sure. Right? If we're looking at anxiety and alcohol use for a lot of years, we measured anxiety when an adolescent was 15 and looked at their alcohol use when the adolescent was 16 and said, oh, there's no effect. But the theory is, is if a kid is more anxious than they usually are during the day, do they tend to drink more that night, right? So we got to get the timing right. It allows us to varying degrees to get reciprocal effects. If we have repeated assessments on anxiety and repeated assessments on alcohol use, we can look at an anxiety to alcohol use relation and an alcohol use to anxiety relation. Okay, that's cool. But I think one of the biggest things that is not always recognized in the literature as a key advantage is with a sufficient number of repeated measures, maybe we can get a decent measure of that between-person effect of anxiety.

if we have 10 or 20 repeated measures, can we use those in some way to say, here is Greg's trait-like anxiety, and then look at the day-to-day anxiety around that trait? To me, that's one of the biggest advantages of getting repeated measures data. I think that does get us closer to what we tend to want, right? If this is a trait that is stable, then we could imagine that I know what a good hair day is for Patrick because I know what a typical hair day is for Patrick. I know what a high level of anxiety is for Patrick because I know what typical anxiety is for Patrick. So we wind up setting the standard for ourselves, and then we judge high and low relative to our own standards. And we can do that for anxiety, we can do that for drinking, and then we can start to address. Not if you are high on anxiety relative to other people, but if you are high on anxiety relative to yourself, Are you also high on alcohol consumption relative to yourself?

So this kind of disaggregation gets us part of the way there. It certainly gets us there much closer than just a between type of design. Doesn't necessarily get us all the way there, but it's a good start. That's exactly right. And one of the big challenges is how do we get a reliable and valid numerical estimate of that trait-like characteristic? Is one measure enough? Well, no, because it's a Trojan horse and it has both. Well, could you do two measures? Well, yeah, but think about it as a sampling perspective. Are you going to have two students from a school and have that be representative of the school as a whole? You could, and you could take two measures and take the mean of them and have an estimate. Well, what about three? What about five? This is where one of the real advantages of intensive longitudinal designs start to come online. which is, well, what about 20? If I ask Greg on 20 consecutive days about his anxiety,

could I combine those in some way to get a decent estimate of Greg's trait-like anxiety? Think about it from a psychometric standpoint. What is the underlying construct? What are the observable manifestations of things that we can use to infer that construct? And how can we do that in a valid and reliable way? This is not a data analysis episode, but there are kind of two main ways that we can do this. One is I think about it is data management And this is often what done in the traditional multi model All right so let say that I have 20 measures of Greg anxiety over 20 days I literally compute a person mean I compute my mean of 20 I compute your mean of 20 Who's listening? I have a person mean for each, and then I have 20 individual measures, and I just subtract the mean from each of those. So a plus two is your two units above your mean on that day, a negative three, your three units below your mean on that particular day, and then those become the

unit of analysis. We bring those into the model. The other way we can do that is parametrically, and what I mean is we bring them into something like a structural equation model, and we don't compute the person mean, but we infer that between effect using latent factors. Usually, We have random intercepts, and then we can look at lag defects in various ways. There's Rickle Pym and Signor Lickum. Was that it? No, Lickum Signor. I get it mixed up. You made it so international. I like that. I did, right? Well, it's SR, right, as I'm doing my Duolingo. It's Rickle Pym and Lickum Signor. There are different ways of doing it, but we don't get that numerical estimate manually in a data step. We infer it in the model. However we go about doing it, what we need to do is break those into separate components. And the term of the day is we have disaggregated the effects into the within-person component and the between-person component.

You will not make this plug for yourself, but I recommend to absolutely everybody out there the paper that you did with Dan Bauer, I think it was 2011, in Annual Review of Psychology, that really tackles the issue of the disaggregation of within and between and how we need to map our analyses onto our specific research questions and the confounds that can happen when we don't think through that clearly. Everybody out there should take a look at that paper. It is thoughtful. It is very well written. You are very kind. That is one bean in a pile. Other folks have done some really good work, and we'll put show notes on of papers. None of this is new. Going way back, one of my favorite classic papers, a guy named Joaquin Woolhill, wrote in 1972 about age as an independent variable. If you're interested in this stuff, it is required reading, because this dude in the early 70s was telegraphing growth models that were going to come 30 years later.

None of this is new, but we still have not stood in front of the group and said, my name is Patrick, and I recognize that I have a problem, which is if you have one measure of X and one measure of Y, these things are hiding together in that Trojan horse. And one, not everybody can go and get an intensive longitudinal data set to do this, right? If you're a grad student and you have a research budget of $1.50, you can't go buy wearable devices. A lot of really good work can be done on cross-sectional data. But as Clint Eastwood says, man's got to know his limitations. These things are baked in. And I got to tell you, buddy, I accidentally terrified myself when I was prepping for a lecture. I made a little simulation, and it was just to demonstrate within-between effects. So I wrote a couple of lines of code, and I generated repeated measures on Y.

I think there were 8 or 10. I had repeated measures on X that they were matched, 8 or 10. And I had it within-effect and a between-effect, and I linked them together. And I was just going to use it as a demonstration for how to do a latent curve model. and disaggregate these effects. So I thought, okay, as an opening gambit, I'm going to show how important longitudinal is. Imagine in your mind's eye, there are 200 people, 10 repeated measures. That's the real design. And I wrote a little single line of code, and I randomly drew one observation, and what it did was mimic what a cross-sectional study would be. Just one X, one Y. And I did a regression. And I was like, oh, this will be fun. I'll do a little opening. The regression coefficient had nothing in relation to the between effect. It had nothing in the within effect. Wow. And I was like, oh, wow. I drew another one and did the regression, a completely different numerical value, like in a different direction. It wasn't like one was 0.3 and one was 0.4.

I did it again. I did it again. And I leaned back in my chair and I was like, how do we know anything about our field? First of all, that is just foundation rattling, that you get this tremendous variability from one to the other. The other side of that, though, is that I worry that that's going to leave us in this position of the All Things Scottish store. Welcome to All Things Scottish. Our slogan is, if it's not Scottish, it's crap. It's all crap. It's all crap. Right? I don't want to communicate that everything cross-sectional is completely worthless. And there are some people who rattle that saber a fair amount. And I don't think that's necessarily true. But I think you have to be very careful to make sure that you know, and I mean really know what research question you're asking. And you know, really know what analysis you have and how those two link up. And if they don't link up, then you're going to have a problem.

So for me, that winds up being the take-home message. What is your research question? Who is the frame of reference? Is each person their own frame of reference? Are other people the frame of reference? What comparison are you actually hoping to conduct? What relationship are you interested in? And are the parameters that you're estimating really getting at that relationship? Can you really make that claim? I think at the end of the day, that's what we have to do is just do that check for ourselves every single time. And it goes into John's grunting corpse point three, which is what is a plausible alternative hypothesis for your observed effect? And I think we're very quick to say future research would do well to move to longitudinal data to establish temporal precedence. Okay, nobody on the planet is going to argue against that. I think that's almost secondary to moving to longitudinal data to allow us to disaggregate the levels of effect.

Here's the thing, though. Well, you still write papers. I quit writing papers. They seem like a lot of effort. Here's a paper out there for somebody to write. I don't think we've thought creatively enough about how might we address this without going to longitudinal data. There are some situations where longitudinal data are not reasonable. There are some times where we don't have resources to gather them. There are a variety of reasons why we might be interested in cross-sectional data. But we could think about the problem creatively from a measurement standpoint. So imagine instead of saying in the last 30 days, how often have you felt lonely? That's our typical kind of question. Well, that's that little Trojan horse, right? What the hell do you mean? What's inside there? But could we pose a set of questions that tap into more trait-like components? So think about your usual self and respond to some items. But then to ask something like, when you're feeling lonelier than you usually do, which of the following holds?

So maybe you get the person in that state trait space. I have no idea. I'm just saying there might be a creative way of thinking about this from a psychometric standpoint. Undoubtedly, at this very moment, someone is screaming at their steering wheel, I published five papers on that, and you have no freaking idea. I apologize to the steering wheel screamer. I'm just saying, to my knowledge, I have not seen a really focused, dedicated attempt to approach this problem from a psychometric standpoint. Yeah, to give the Trojan horse a suppository, I think is what you're trying. Oh, you brought it back. Huh? So, folks, we always try to say because, and we say, okay, so how are we going to close this out? And we're just like, I don't know, we could do this, do that. All right, that was totally organic, and we squared the circle with the suppository. Well done, Hancock. Thank you very much on that note.

Have a wonderful Thanksgiving. We hope your turkey does not explode. Or Tofurkey. We will catch you on the other side. Take care, everybody. Bye-bye. Thanks so much for joining us. Rather than our traditional post-episode sponsors, this week we have a Thanksgiving message for all our friends out there who celebrate the holiday. Wherever you are this Thanksgiving, We hope somebody remembered to load the cranberry sauce. See what I did there? Please allow me to state a bit of logistic information. Don't be latent for dinner. Even if you're wondering what the chefé is preparing, please stay out of the kitchen and keep your cook's distance. We'll be running a turkeys test on the holiday bird. It's a model-based approach. Every 20 minutes or so, we'll open the oven and say, Hey, Wood, you look at that. It's almost done. Once you start loading up your plates, feel free to do lots of sampling without replacement, please.

And if anybody asks why you're going back for thirds, tell them you needed repeated measures. For dessert, we made a pie chart with a pecan value less than .05. So don't just stand around. Grab a pea slice along with a hot cup of student tea. And be sure to take a box of leftovers when you're going home. Happy Thanksgiving from all of us here at Quantitude. Your favorite family-wise era since 2019.

番組の概要欄(原文)

In this week's episode Patrick and Greg explore the fascinating yet sometimes tricky topic of disaggregating within-person and between-person predictive effects, primarily focusing on the conceptual issues and deferring the statistical details to another day. Along the way they also discuss pilot cooking a pre-registered turkey, planting a flag on a tiny hill, elephant suppositories, exploding aliens, unstable family members, Birdland, go Cubs, tiny Trojan horses, the Patrickverse, grunting c...

出典・この記事について

番組
Quantitude(Greg Hancock & Patrick Curran)
元の回
S8E08 Disaggregating Within- & Between-Person Effects: The Trojan Tofurkey
配信日
要約作成日
公式サイト
quantitudepod.org

この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について

X でシェア

関連エピソード