After reading the very concise and accessible report on the state of the study, I got the impression that the $1.3 billion had been allocated by Congress but that little or none of it had been spent, basically because the study design never got close enough to being finalized and approved for any large-scale data acquisition to even take place. Unless I'm quite mistaken, the $1.3 billion has been parked on the NIH's balance sheet for the last several years rather than being lost.
This is really a drag, because I was hoping that study (which has been on the wish list of a lot of scientists for a long time) would be fully funded and operated over the long term. Prospective, longitudinal study design is the only valid way to answer a number of important questions about child development. On the other hand, the reason this particular study project was cancelled was a legitimate reason: "The study 'as currently designed is not feasible,' Collins said in a Dec. 12 statement on the NIH’s website."[1] That conclusion was based on concerns raised by the National Academy of Sciences when reviewing the pilot phase of the study.[2]
So of course the next step for scientists would be to learn whatever can be learned from the pilot program in the just-cancelled study, and then design a new study and try again. I will highly support research projects of this kind, which now will have more benefit to my grandchildren than to my children, who are already almost fully grown up into adulthood.
P.S. A big hat tip to the Hacker News participant who found this news story, which is well reported and links to key online documents, for finding a great source about an important story. The key online documents are press releases from government offices, but this story adds a journalist's contacts with other sources and establishes context for the latest news on the study.
[1] "Statement on the National Children’s Study" 12 December 2014
[2] "National Children’s Study Has Great Potential to Expand Understanding of Children’s Health and Well-Being, But Key Design Elements Need Further Development for Study to Be Successful" 16 June 2014
I am glad to see that a decision was made to not throw good money into a bad project. They realized that the study as designed was infeasible. I did not see specific reasons but I can only imagine the difficulty of tracking 100,000 individuals through roughly two decades of life, and accounting for the untold number of uncontroled variables that would be present in such an effort.
With research funding recently, it's important that scarce funding be directed to projects that have a reasonable chance at achieving their research goals. This project didn't.
It should be noted that there are a number of birth cohort studies ongoing in Europe, where single payer healthcare makes it dramatically easier to follow people.
My money would be on a) more people having insurance, b) having resident registration (that database will have errors, but in cohort studies, people spend lots of time trying to correct that for those in a cohort), c) people, in general, to have more trust in government, certainly where it concerns healthcare, and d) government more willing to look ahead further. I do not have supporting data for c) and d)
A fair amount of research has gone into figuring out the study - the NCS was a major undertaking, involved laying down a lot of logistics, figuring out methodological issues, etc.
Also the Vanguard study, while a pilot study, was a fairly serious undertaking.
Most of the researchers are U.S. citizens, so in effect they are.
Also, most NIH-funded researchers have gotten quite used to continual budget cuts, uncertainty, and needing to make every cent count. "Expensive" does not equal wasteful.
Compared to what? What some folks in SV will blow $1.3 billion on?
Cohort studies like this one have led to some major advancements in our understanding of disease - if you know anyone who takes aspirin to reduce their risk of cardiovascular disease, for example, that came out of a cohort study. There is a tremendous amount we don't know about the development of diseases in childhood, life-course trajectories, the effects of environmental exposures that need well conducted, large sized studies - the large bit is important, as they need statistical power if anything will meaningfully change in terms of policy.
I only know about it, though, because my wife & I participated in it with our daughter (who is not on the autism spectrum): https://ncseed.org/public/seed2.php
It seems the organizers are contacting the parents of every child born in the 10 eligible counties of North Carolina, so they must be seeking quite a large number of participants (and this is only one of the five states).
Wow, $1.3 billion is a huge amount for a single study. For comparison, the Framingham Heart Study, going on 65 years, has an annual budget of about $10 million:
(Concerns that the 2013 sequester would cut about $4 million, about 40% of its budget here:
http://www.bostonmagazine.com/health/blog/2013/10/09/framing... )
Calling the NCS a single study is like calling the moon landing a single rocket launch. For example, from the report:
"There are 112 papers resulting from NCS efforts so far, primarily papers about methodological design issues, but also including preliminary results related to environmental exposures."
Am I misunderstanding things? From reading the linked article, there was $1.3 billion spent studying whether we should actually undertake the study. Not the actual study itself (which presumably would cost significantly more?)
- Trying to figure out how the heck to enroll people (vanguard study)
- Decide which variables are worth tracking, and actually trackable. Every variable they track will cost a tonne, if they don't track a variable and decide they need later it's too late.
- Figure out how long they will successfully be able to track people, will they lose 5% a year? 10%? Is there a huge cliff when kids turn 12 where they all move? WHO KNOWS?
- Actually enrolling people and tracking the variables that seemed worthwhile
- Revising surveys based on stuff they're learning.
It was to be a huge study, to be honest if completed I think it would have been the landmark medical advancements of our time.
(I briefly worked on one small piece of software supporting one small aspect of the study)
Thanks for the insights. OK, that makes more sense and I can see the utility of the published results.
Now it sounds to me like when a corporation or large federal agency fails in implementing a new or upgraded software system because the process has become unmanageable. From your perspective in both worlds (setting aside the results published so far) does that analogy capture it?
I don't think it quite captures it, because the NCS might not be "unmanageable", but simply too expensive for what people think its budget will be, or not be able to achieve some of its goals (good followup, a nationally representative sample, etc.).
It would be more like...if you rolled out an upgraded software system in a couple departments, collected some feedback, and decided not to go with the company wide rollout.
In terms of enrolling and tracking children, didn't the Early Childhood Longitudinal Program tackle a lot of those same issues? They managed to track my brother between kindergarten and eight grade which for my family spanned three states over five moves.
No - $1.3 billion was spent laying the groundwork and running the pilot study, from which one of the conclusions was "Running the main study may not work, and will cost a lot".
To use the space program analogy I advanced before, it would be like running Gemini and Mercury, then deciding to scrap the Apollo program because things weren't working out. A lot of science was done just laying the groundwork.
$1.3b to track 100k kids for 7 years. That's about $2k/child/year, which sounds insanely expensive. How can a study cost that much? You could buy each participant an iPad _every_ year and given them $100/month for filling out surveys, or you could hire 1 researcher for every 25 kids, which sounds grossly inefficient -- and you'd still be under $1.3b for 7 years.
Surveys don't cut it. There's clinical costs associated with each subject - getting accurate clinical data, biomarkers, needing the infrastructure to store biological specimens for decades. There's methodological issues, planning that needs to get figured out, and the people who do that are not cheap.
The NCS was planned and known to be a massive undertaking, to try to ask some very serious questions about health that are hard to get out without long-term cohort studies.
I totally agree with you that this is a complex project with a lot of costs. Still, it's hard for me to fathom where so much money could go. I mean, if you give 100k people 5 top-of-the-line wearables over 7 years (maybe $1500/person); hire 250 experts, each of whom charges $250k/year; and allocate $15m/year on AWS hosting bills... you still wind up at "only" about $700m.
I'm sure the money went somewhere. I'd be really curious to see a breakdown of where.
Table down at the bottom. As of 2011, 60% of the spend money was spent at the study sites themselves. From the text of that report, it sounds like recruitment was very expensive, which isn't surprising.
The "small pilot" was a 40 center study enrolling 1,750 pregnant women. There are some fixed costs to a study like that, especially if they're supposed to also apply to the larger study, and there has been a lot of work done in developing the appropriate methods for the NCS - over 100 research papers worth of it.
Also, as has been said, it's not clear if all the allocated money has actually been spent.
I'm not trying to start a USA bash, but I'm pretty sure cost of those bloods is not $600. If you wanted into a clinic and asked for a cash price I'm pretty sure it would be lower. I base this claim on looking at medical bills in a far away country with an utterly different system.
At the same time, the blood draws for this study are looking for things your average clinic isn't going to do for a walk in. And unlike the system in the U.S., or another country, this isn't being done for the patient's health specifically, but rather for research, so there is no subsidy, either from insurance or the government health system.
Basically, we have no idea how much it would cost.
No no wrong wrong please stop applying blanket statements to things like this. DARPA and NASA alone have benefitted humanity and science in ways hard to grasp - from the Internet to vaccines to the microwave frackin' oven.
Please do not lower yourself to blanket statements like this. It's true, government projects sometimes become massive boondoggles like this (or say, the JSF) but just as often the benefits are enormous.
are you kidding me? NASA is your idea of a successful lean government project? Elon Musk is doing what NASA couldn't with a tiny fraction of their budget.
And how far along would Elon Musk and SpaceX be without the decades of research, development, testing, and real-world experimentation done previously by NASA and other government R&D efforts?
Your argument presupposes SpaceX hasn't benefited from the previous work done before. Which they have, immensely. Stop reducing complex systems into simple certainties. It's insulting to the entire community of scientists that have been dedicated to rocket and space research over the past 70 years.
He didn't use the word lean. NASA has gone to the moon. It is as of yet unclear whether the likes of Musk can pull that off. It seems intuitively likely that some things are best done by/funded through the government.
Perhaps you'd like to discuss with us some of the doubtlessly myriad inexpensive long term cohort studies run entirely by private industry?
Or outline the clear problems being suffered by those poor Government fools in Scandinavia running birth cohort studies? Where, perish the thought, Government is not only in charge of research but healthcare.
http://acd.od.nih.gov/reports/NCS_WG_FINAL_REPORT.pdf
edit: this page suggests about 60% of the funding was spent: https://www.nationalchildrensstudy.gov/about/funding/Pages/i...