Read slide 17. The key assumptions are that: the confounder present at bunching point is representative of the confounder present everywhere, and the confounder affects outcome linearly. The second assumption is what strikes me as too strong, and you can read the paper Appendix C for how they address it. The reasoning is quite weak and they even phrase it as such: "Our main empirical findings
therefore do not seem to be an artifact of this linearity assumption". Overall I would take this paper with a grain of salt, from a cognitive science point of view it just makes no sense that doing more reading would have no cognitive benefits (in fact, we know the opposite to be true.)
Also, the paper hasn't even been reviewed yet. We should revisit it after it has been peer reviewed.
These kind of papers (those with the theme of "taking any kind of action for the growth of your cognition will always be in vain") seem to have a strong bias toward biological determinism. But God gave human beings the ability to solve problems for the purpose of overcoming nature, which includes the nature of our cognitive type and our cognitive capacity.
That's not what it's saying though, it's merely saying that structured enrichment activities are no better at improving your cognitive skills than the other things kids choose to spend their time on if they weren't forced to do structured enrichment.
They even point this out as a specific point in the abstract: You can't compare kids doing enrichment activities to kids not doing enrichment activities while controlling for every other variable to be the same, because it is manifestly true that time spent on enrichment activities can not be spent on other activities, such as sleep or socializing, which also have benefits.
> They even point this out as a specific point in the abstract: You can't compare kids doing enrichment activities to kids not doing enrichment activities while controlling for every other variable to be the same, because it is manifestly true that time spent on enrichment activities can not be spent on other activities, such as sleep or socializing, which also have benefits.
Please double-check my reading comprehension of your comment. Are you saying that it is not possible to analyze the benefits of enrichment activities as opposed to what normal kids do, because there is no way to have a group of kids who do exactly what normal kids do + enrichment activities, because taking the time to do enrichment activities necessarily knocks out at least one activity normal kids do, thereby making control impossible?
There’s a difference between taking cello classes for some rich kid to get into Princeton and normal people. I didn’t grow up in a particularly bad neighborhood, but time I spent playing baseball, Boy Scouts, etc kept me away from trouble. I wasn’t directly exposed to drinking, smoking and other not so smart things in middle school. I got the benefit of encountering those things when I was older.
When I was in high school I volunteered as a counselor at a summer program affiliated with my dads job in an inner city environment. The stuff we did help keep those kids, 2-4 grades younger than me, away from (or perhaps delayed) getting recruited into the drug trade.
Asking bankers and economists to measure educational and social outcomes sounds pretty dumb. Enrichment isn’t designed to produce ROI, it’s to enrich your experience as a human being. We don’t have the income mobility we once did, so folks are likely to land in whatever income strata they came from. Using income as a metric to measure whether enrichment activities are worthwhile or not is the same as evaluating the quality of apples based on the price of oranges.
I think it matters if kids enjoy those activities. If they hate them and are forced into it then the answer is quite obvious, they’re not gonna get that much out of it.
At the same time kids at that age don’t know what their best interests are and could do too much of an activity with diminishing returns while not developing other skills.
>We study the effects of enrichment activities such as reading, homework, and extracurricular lessons on children's cognitive and non-cognitive skills. We take into consideration that children forgo alternative activities, such as play and socializing, in order to spend time on enrichment. Our study controls for selection on unobservables using a novel approach which leverages the fact that many children spend zero hours per week on enrichment activities. At zero enrichment, confounders vary but enrichment does not, which gives us direct information about the effect of confounders on skills.
>Using time diary data available in the Panel Study of Income Dynamics (PSID), we find that the net effect of enrichment is zero for cognitive skills and negative for non-cognitive skills, which suggests that enrichment may be crowding out more productive activities on the margin. The negative effects on non-cognitive skills are concentrated in higher-income students in high school, consistent with elevated academic competition related to college admissions.
First, they are looking at a very wide range of activities and calling them enrichment. They include tutoring, music lessons, reading, and other "extracurricular activities" as an "enrichment activity." That seems like too wide of a net, as many extracurricular activities and homework likely are a waste of time, therefore diluting the extracurricular activities with positive effects.
Second, if I understand their methodology, they are checking students with no enrichment activities to children with a few minutes a week of enrichment time. Skill acquisition takes more than a few minutes a week, so of course those children would not be receiving benefit.
Third, the results seem absurd. They are saying there are no cognitive benefits to taking music lessons, but I guarantee that the children who took a music lesson is much better at playing an instrument than a student who didn't.
Fourth, there's already a lot of statistical and neurological evidence that reading is good for your thinking and your brain, and I don't think this study overcomes that evidence.
Their intro shows the bias directly. Quotes between "enrichment", and framing of everything antagonistically, a "vs. enrichment" thing.
So let's start where they should start: what is "enrichment" to the authors?
> Our definition of enrichment intends to capture the kinds of activities that are typically considered to be investments in children’s skills. Therefore, our baseline measure includes only those activities that are unambiguously related to skill development over and above class time in school.
> we aggregate the 300+primitive time-use categories into eight categories: enrichment activities, other enrichment activities, play and social activities, passive leisure, duties/chores, class time, sleep, and other
They go on to show (on page 8) that (1) 3% of children's time is "Enrichment" and 2% is "Other Enrichment" (...), which is all in all a whopping 5% (70 minutes) in the day, and (2) that 66% of that time is homework.
Seems to me like they have a beef with kids having 45 minutes of homework per day. The title of this paper is attractive and the conclusions broad, but really they could have said "let's compare children doing homework to children doing no homework and measure cognitive abilities", and they'd have something more substantial.
But what is cognitive skill vs non-cognitive skill then? Well it's not explained in the paper, but they say that they use the Child Development Supplement (CDS) to aggregate that information, and that the CDS is three "waves" that they look at together. All of those "waves" correspond to years, respectively 1997, 2002 and 2007. I'd like to argue that these last 13 years have seen a wee-bit of change in the world (remember when youtube and twitter were one-year olds?), but whatever.
It takes a text-search to finally find the data, on page 44. Cognitive skills include (and are limited to) Letter Word (?!), Applied Problems, and Passage Comprehension (?!).
So... ok. Let's just say they were really trying to research homework, in which case those only-three-existing-cognitive-skills would mean something (although the jury's out on what Letter Word means). The NON COGNITIVE SKILLS include things like "Curious and exploring, likes new experiences", "Does neat, careful work", "Tries things for himself/herself", "Thinks before acting, not impulsive".
At this point I throw my hands up in the air, I forget that I spent far too much time reading this "paper", and I move on.
> Third, the results seem absurd. They are saying there are no cognitive benefits to taking music lessons, but I guarantee that the children who took a music lesson is much better at playing an instrument than a student who didn't.
They say that since time is finite, the benefit from music lessons seems to be canceled out by lost opportunities of doing other things. It’s totally reasonable to think that, in fact if enrichment was positive in net then that would actually mean kids are doing too little.
Given the very limited readout of "skills", how could it be otherwise? Nothing in their measurements captures the actual skills taught by music lessons (piano-playing ability, music appreciation, etc).
Perhaps music lessons have some second-order effect on "does neat, careful work", but it doesn't seem impossible that might have opposing effects on other so-called skills too: "too fearful or anxious" for stage fright, "disobedient" for not practicing when told.
Yeah, those kids could be honing their fortnite skills instead. You always have to factor in the opportunity cost, particularly when you're talking about how children spend their free time.
I'm guessing they mean cognitive abilities in general. Taking a music lesson won't make the kid any smarter than if he didn't take it. He'll just interact with music up to the level of his general intelligence, and if he played Fortnite instead, he'd do the same.
This is more "chess won't make you smarter." I'm assuming what does is related to family environment, genes, and quality of primary education.
Entire fields (e.g., psychometrics) are devoted to finding ways to accurately measure "skills", so it is also disconcerting that they devote all of three sentences to explaining how the cognitive and non-cognitive skills are measured.
The "non-cognitive skills" come from a factor analysis of a hodgepodge of behavioral ratings ("Cheats or tells lies", "Too dependent on others"), none of which seem similar to what I'd call a skill. It is not clear to me why the loadings on these factors vary across years: if you think that not depending on others is a non-cognitive skill, surely it's the same skill in 1997 and 2002.
> Second, if I understand their methodology, they are checking students with no enrichment activities to children with a few minutes a week of enrichment time.
Did you read the paper? That’s not what they say at all. In fact they say that we would expect no real difference from a few minutes and 0 but because we do find differences we can use that to control for other factors.
One of the things that they get right in Russia, and even places like England, is that its much easier for a poor city family to have a cottage outside the city or an allotment garden that is subsidized by the state.
And I think benefit to city kids is enormous, as far as an entire new environment to be exposed to. And even the growing of food.
This should a program in North America too. Giant farms have taken almost all the land. And some of it is just a waste anyway. If the government can subsidize growing corn to turn into car fuel. An extremely wasteful energy inefficient process.
Government could be buying some of that land back, and letting return to nature or forest, and allowing building even tiny cottages. For poor families, it could be such a quality of life boost, especially for kids.
I agree - homework did nothing for me when I was in school. I could always ace the tests but teachers insisted homework "because" - never made sense and was a constant source of friction. Jackasses.
Homework is totally different for me in university compared to high school. Math in high school was easy. The homework just felt like a waste of time: repeatedly applying what you learned from one example.
In university, homework is everything. If all you do is attend the lectures, you will fail the midterm and final. In order to do well in the course you need to do all the homework which really pushes you to learn. The examples given in lecture are completely trivial compared to the homework problems which can often take hours to solve.
“Enrichment activities” would actually be useful if the content was actually enriched. Advanced students would get a lot more out of school if they were given work that was genuinely challenging for them and took a long time to figure out.
My first collegiate math class, the teacher came in and asked the class, "do you know how to get an A in this class, two hours of math homework a night". I followed this advice and bam, A. Despite the fact that this math class was much harder than anything I did in high school and I was always a B+ math student.
The next semester I followed this advice again but let myself slip a few weeks and bam B+. The next semester I strictly spent the time again, bam, A.
Through upper class years my diligence in school was consistent but I sometimes had to focus on programming over math. I noticed a near perfect correlation between the time I spent on out of class work and my performance.
My experience was similar - in highschool, the class is divided into two parts, the 'lesson' bit where the teacher explains something, then the 'practice' bit where the class tries use the information from the first half to solve some problems. In university, the entire lecture is the 'lesson' bit and the assigned homework is the 'practice' bit.
I made that complaint--as a flippant remark in class--to a high school statistics teacher once so he assigned me the task of writing my own final exam rather than taking a final exam, and gave everybody else the option of taking my exam or his. (Nobody took mine.)
My grades weren't that great (classic B student) and I doubt I had better mastery of the material than the more studious ones in class. But writing the exam was both fun and challenging--I definitely had improved mastery afterward, as well as a newfound appreciation for his job. He gave me a B+, though, out of fairness as I had several days to write the exam, open book. Which was fine by me.
I didn’t like homework that much because there was so much of it. But if I look back, I probably learned the most doing homework. I was alone, concentrating better and having to figure out how to get the homework done.
In college homework was the workhorse of my acquired knowledge.
I think it is quite clear that being inately smart could cause quite a setback as one thinks they don’t need to work hard and winging it would do it. It turns into a bad habit and eventually one falls behind and stays there.
I think I'm not quite understanding what they're saying. In section 5.1 of the PDF tsumnia linked, the author talks about how it's not that enrichment activities have no benefit (despite saying actually writing that), is that the enrichment activities have no greater benefit than leisure activities. Did I get that right?
My older daughter loves to read, so she reads a lot, and her teacher (when school was in session) says she's one of the best readers in the class. So is this paper saying that if my daughter had done activities instead of read (like play video games or run around outside) she'd be just as good a reader? That doesn't seem right based on my experience.
Also, I read a lot about the benefits of having children learn to play music, especially starting with a piano. Some studies referenced here: https://www.lindebladpiano.com/blog/benefits-of-playing-pian.... Doesn't this paper suggest those other studies are wrong? I'm kind of slow when it comes to understanding papers like this.
> So is this paper saying that if my daughter had done activities instead of read (like play video games or run around outside) she'd be just as good a reader?
No, they are saying that your daughters overall cognitive skill would be equal. So maybe she’d be worse at reading but better some other cognitive skill (say, special recognition). Basically there’s no free lunch—if you spend on one thing you aren’t spending it on something else.
> So is this paper saying that if my daughter had done activities instead of read (like play video games or run around outside) she'd be just as good a reader?
I can only imagine that the paper says that in aggregate kids get as much out of play as they get out of enrichment.
Though I’d also say that if your daughter likes reading, that’s pretty much play/leisure to her, not enrichment (or both).
Even if their conclusion is correct, which after a quick skim I didn't see how they measured "cognitive" skills the result is kind of dumb.
Working on math problems might not make you a quicker thinker overall, but it will teach you math! The specific skill you enrich will become stronger at the expense of the rest of your life. I agree that time spent not socializing damages the ability to socialize (it's a joke among engineers in school). Clearly we are not making great tradeoffs, but I'm not sure this result helps us very much beyond what we already knew.
I worry that people read things like this to inform what they do with their children. A child is a person and its reasonable to explore what they need and want through your life with them. We should always remember that children are not simply another of life's projects to be maximised.
I think in my kids it depends on the activity. Some things are better than others I'm sure. Nothing wrong with trying things to find what you like, maybe find your passion.
Is there some perfect version of achieving cognitive and non-cognitive skills. Are people happy if they achieve it?
this paper is very hard to read and full of jargon for the non-initiated.
the conclusion is just one sentence and is very clear. but i was left with the feeling with all these big words that i am not supposed to understand what they are studying or their methodology...
It's a normal paper for its discipline. We are not the target audience.
(something that Nature did in the redesign that led to me dropping the subscription —I use "table of contents on the cover" as a sibboleth— was to introduce a section of summaries, written for a wider audience, for some of the articles, which I found, on the contrary, to be a very positive move.)
I decided to provide my feedback on the paper because judging from some of the comments I've read, some of you did not read it. To those that did read it, remember that the authors are people too and can offer any clarifications or address any concerns you have.
After reading through the paper, I'd like to say the authors did an excellent job addressing a sensitive subject. Part of me likes to think that they, like many of you, thought that enrichment programs were a given positive addition to life. The authors do an excellent job attempting to use mathematics to model human behavior - Sections 2-4 are almost entirely used to explain what they mean and how their equations operate. While reading the paper, I thought about some of the negative comments on this thread about economists. To that, I reasoned that we use mathematics as an attempt to model the world. Physics is our attempt to model the physical world and economics is the closest thing we have to modeling human behavior. If you disagree with that sentiment, become an economist and change the domain or dual major in data science and psychology (or sociology or any other degree used to model humans).
My biggest takeaway from this paper is that additional HOMEWORK does not help in improving a student's cognitive ability. In the paper, enrichment is defined as "investments in children's skills" which 87% of their dataset consisted of homework, reading, or before/after school academic programs. Doing or adding an additional hour of these activities does not provide any benefit to the child. This does not mean that learning is bad, as class time was a separate category, it meant "doing more" didn't help. Again, this is a sensitive subject, and I can agree that the authors don't account for "bad" instruction.
I will say there are limitations to their results, which they do acknowledge. Their dataset is small, but one of the closest datasets we have to mapping long-term human activities. I'd say this limitation grows exponentially because the CDS/PSID dataset they use only addressed 3 days in an individual's life. If I am wrong please correct me, but that is what I assume they used based on the paper. PSID contains 75,000 individuals and they only sampled 4,330 children. However, if you assume a nuclear family (4 people), that means the dataset contains ~18,750 families, which means the sample is ~25%. Again, this is a limitation, but one of the only approximations we have. The other limitation is that they are using self-response surveys, which have limited reliability.
With that, what can we take from this paper? My takeaway is that people are effectively equal but different at the same time. This was addressed with their h(X) value. Using all controllable variables, some h() function exists that produces what they addressed as "ability". This is once again where I, as did the authors, recognize that it is a sensitive topic. It is also one of the additional limitations I would say to their paper - assuming ability is a fixed value.
However, as a discussion point, 'ability' would account for some ugly truths. The g-factor and IQ are methods to address cognitive ability with sensitive undertones. Likewise, it is a commonality that networking is a key factor in successful employment often mentioned on HN. This could be accentuated by their findings that additional enrichment negatively impacted non-cognitive skills. Just because you are smart does not mean you are employable. However, I acknowledge that assessing cognitive ability is a sensitive topic, ripe with misconstruction. Likewise, I think the authors do their best to NOT say your ability is determined at birth.
I would caution against using "ability" as a metric for determining worth. Doing so would encourage early testing to see if someone is "worth the time" and I imagine would be vulnerable to prejudice and confound the features of X, e.g. a low-income child is not worth investing in. Instead, I would argue that this paper doesn't fully address "potential" with their unboservable ε. The job of the individual instead is to identify and maximize their potential (X) or at the very least improve the potential of their future selves (X+1, or children). This would acknowledge the concept of "expertise", or practicing a task makes you better at said task. It still presents an uncomfortable truth that some people aren't able to achieve success in a given task, but does address improving one's self.
I'll end there for some discussion and because it is late.
Table 7 p.43 is the list of "cognitive" and "non-cognitive" skills. Figure 12 p.46 shows the "enrichment activities."
From these the conclusions should not be surprising. "Reading a book" and attending a "before or after school program" may help with the non-cognitive skills, but those are only 10% or so of "enrichment" time, and the only non-cognitive skill homework (unless it has changed greatly from my day?) would significantly help with is "Demands a lot of attention."
(silly authors: enrichment isn't meant to develop skills[1], it's meant, by choice of enrichment activities, to develop the sorts of people the children think are "one of us" as young adults, to influence whom they will hang out with, whom they will date, etc.
In Brave New World this starring of sneetches is managed in infancy by hypnopaedic suggestion:
"… all wear green," said a soft but very distinct voice, beginning in the middle of a sentence, "and Delta Children wear khaki. Oh no, I don't want to play with Delta children. And Epsilons are still worse. They're too stupid to be able to read or write. Besides they wear black, which is such a beastly colour. I'm so glad I'm a Beta."
There was a pause; then the voice began again.
"Alpha children wear grey They work much harder than we do, because they're so frightfully clever. I'm really awfuly glad I'm a Beta[2], because I don't work so hard. And then we are much better than the Gammas and Deltas. Gammas are stupid. They all wear green, and Delta children wear khaki. Oh no, I don't want to play with Delta children. And Epsilons are still worse. They're too stupid to be able …")
[1] I like the french idea of "education" (education), as having components not only of "how to" (savoir faire), but also of "how to live with others" (savoir vivre) and even of "how to live with oneself" (savoir être).
[2] the list of "non-cognitive" skills also shows a distinct bias towards (in Brave New World terms) Beta-Gamma ideals. How many of these soft skills does someone from a high SES background need to become, say, President of the United States?
I doubt it. Mastering an instrument and music reading/writing trains linguistic, focus and cooperation skills. Acting in plays trains memory and cooperation skills and performance skills. Field sports trains spatial reasoning and competition skills. Camping and hiking trains preparation, map reading and cooking skills. And so on. We know what we know and live our lived out metaphors.
Read slide 17. The key assumptions are that: the confounder present at bunching point is representative of the confounder present everywhere, and the confounder affects outcome linearly. The second assumption is what strikes me as too strong, and you can read the paper Appendix C for how they address it. The reasoning is quite weak and they even phrase it as such: "Our main empirical findings therefore do not seem to be an artifact of this linearity assumption". Overall I would take this paper with a grain of salt, from a cognitive science point of view it just makes no sense that doing more reading would have no cognitive benefits (in fact, we know the opposite to be true.)
Also, the paper hasn't even been reviewed yet. We should revisit it after it has been peer reviewed.