I studied in India, and had a chance to study abroad as well.

Currently, for reasons that escape me, you lot (Americans) seem to have decided to import those banes of thinking and real learning - standardized tests.

Perhaps in order to have some sort of proxy or measurement for what is inherently a chaotic and un-measurable system of children.

From experience - this is generally a BAD IDEA, but its one whose pernicious subtle problems show up after years and possibly generations of being married to it.At which time it becomes too hard to get rid of it.

Firstly - everyone will stop teaching, and focus on acing the tests. Learning is irrelevant, and students know it. This will bring in rote learning, and soon - tuition classes. I don't know what you guys will call it but expect kids to start studying year round.

Then comes the self fulfilling prophecy - you did well in your tests, so you must be good. You must do well in your tests if you are to be good.

And at this point you aren't even creating or nurturing students, but manufacturing mental athletes with specialization in testing well.

IF you throw in STEM fixation - (Well he should have known better to take a degree in humanities) and you damn students with no interest in science or commerce to do exactly that.

Of course, there are a lot of deficiencies in this approach, which you can see and teachers can see it too. But given centralized, state-driven education system, what else you can do? You can not have absence of quality controls - this would have immense corrupting potential and the system would have incentive to deteriorate very quickly. You can not have individual approach - DoE can't account for each school in each small town, they just don't have resources for that. So with current approach, testing is inevitable.

Its remarkable really. Years, decades of seeing kids die to suicide and cram pressure and this is what you choose.

So that you can test your teachers?

Anyway, your test doesn't test teachers, it tests students.

And then you assume it is a good proxy for teachers.

So just to be clear.

1) Tests are a perfect proxy for taking that test -> 2) Then they are a proxy for test taking ability in the student 3) Then a proxy for student understanding of the subject 4) And thence a proxy for teacher ability.

I'd understand if we were doing chemical analysis with 4 steps involved in the process.

But with human beings? Kids? The fundamental chaotic units of society?

I really have to point out that this is logic that gets most startups killed, or logic on which they pivot to something smarter.

I know you aren't the one responsible, but I really can't help but be perplexed. Once upon a time you guys were known as the place where "stuff just works."

Your older education system did a great job. Sure it may not have made everyone a rock star but frankly - not everyone is cut out to do so at that juncture in their life.

If you say that there is nothing else that can be done, then I accept and understand. But it is a terrible terrible outcome.

>> Years, decades of seeing kids die to suicide

Oh come on. You could at least wait couple of lines before you drop "you are a child killer" bomb. This is complete bullshit. There are very unfortunate cases of child suicide. It has nothing to do with existence and merit of evaluation system whatsoever and assigning to it responsibility for dying kids is a demagogy at its purest.

>>> And then you assume it is a good proxy for teachers.

Again, I don't assume anything - I say this criteria is inevitable, since the job of the teachers is to educate the children, and well-educated child is considered teacher's success, while poorly educated one is considered a failure. Since we don't have a way to test capability of teaches by supplying them with test kids to be taught in lab conditions - we have to test the actual kids to sample their level of education. It's not a choice, it's a logical consequence of current system of assumptions. It's as much choice as for a man jumping from a building it is a choice to hit the pavement.

>>> Your older education system did a great job.

It did great job for some and terrible job for others. It does the same now, only much more expensively. That is another reason why government is so eager to introduce testing - the costs are raising massively, the results aren't improving, and in some places are actually nothing short of disastrous - with schools releasing people that are completely illiterate. Every school complains it's underfunded but more funds do not seem to produce better outcomes. The budgets have limits, so it is no wonder that whoever distributes those budgets wants to see some metrics about how things are going instead of blindly throwing money in an hoping for the best. If something gets startups killed is exactly throwing money out without any way to measure if something was achieved or not.

If you can't measure it there is no way to tell whether it's getting better.

When you run A/B tests on a web site you don't tell the visitors which version they are being shown, or even that there are multiple versions at all. That would completely destroy the value of the testing. In psychology research, where ethics generally dictates that the subjects be informed, they are still not told what specifically is being tested, lest they modify their behavior.

On the other hand, everyone involved in school standardized tests knows about the tests, knows about various biases that exist within the tests, and knows what is at stake based on the results of the tests.

The reason that the current form of standardized testing is destructive is pure economics. If I tell you that I will count how many lines of code you have committed to the repository at the end of each week and that your pay will be based on this figure, what are you and your coworkers going to do? You're going to start writing hundreds of lines of comments, putting each variable declaration on its own line, needlessly re-factoring satisfactory code, etc. This is, as I understand it, a well-documented side-effect of measuring programmer productivity using LoC.

Once I implement this evaluation scheme, two things happen. First, you are being evaluated based on an objective metric. Second, you have direct control over the value that metric takes on. Economics, which is, to a great extent, the study of incentives, tells us that you (and your coworkers, at least in general) will game the daylights out of the metric.

So why is it any different when teachers, once told that their salaries and very jobs depend on the results of an annual test, teach their students exactly what is on the test, by hook or by crook, even at the expense of "real" learning?

The problem is that teachers aren't oblivious (neither are programmers). When they see that they are being evaluated based on the outcome of X, they seek to maximize X. So the trick is, if you insist on quantitative metrics (which is not totally unreasonable, at least as the default) then the metrics must be linked to the behaviors that are desirable, not some vague indicator that approximately tracks those behaviors.

Edit: missing word

There is so much competition you are likely to inevitably work very hard. So much effort that you will ultimately get it even if didn't want to.

I know this is brute force way of getting things done. But it is still far better than getting nothing done.

now, theres something to be aware of - "getting _things_ done" in this context means getting a high score in a standarized test, not getting a job.

Having such a score doesn't necessarily mean you are able. Especially in a creative field that requires genuine interest and passion. This sort of training works for training operators of machinery and/or manufacturing workers that require special knowledge, but for creative fields such as programming (and not the CRUD type!) i think having such brute force method of learning is in fact a barrier.

I hear in china that there are so many people who get excellent scores in tests, and yet is unable to find jobs. Simply because jobs suitable for them isn't there, nor are there that many opportunities. The sort of rote learning is really exacerbating the problem, not helping, because the students studied hard (despite not being their "passion"), they get a good mark, but still find themselves in a rathole.

The sad truth here is that it is possible to "fake it", at every level. For the most part a person can be productive in even the most technical of jobs without fully comprehending most of what they are working with. People talk about cargo cultism a lot, but in the real world cargo cultish behavior often works. A lot of times just going through the motions is enough to get by.

But that's no excuse to continue to wallow in mediocrity. We can and should do better.

We make mental body builders.

But they make terrible people and thinkers. The idea of educated here is lost. Most of the debate the average person can create is middlebrow at best.

Standardized tests aren't the bane of thinking and learning. Drilling children over and over with questions identical to those on some test is the bane of thinking and learning. The United States has had standardized tests at least since I was a child (I took standardized tests in elementary school almost thirty years ago) and U.S. education has continued to be progressive during that time. College admissions were very heavily influenced by standardized tests when I was in high school, but my classmates and I were not taught by rote, because our teachers did not believe rote learning was effective.

Let me give you a concrete example. My ex-girlfriend grew up in South Korea. She did very well in calculus in high school, where she learned to solve problems by rote. They were shown different kinds of problems to solve, they memorized the steps to solve them by rote, and then they practiced solving those problems. That's all they did: memorize the steps and then apply them over and over to thousands of problems. Then they were tested on exactly the same kind of problems. She got excellent grades and test scores, but she cannot tell you what a derivative or an integral are or why you would use them. I was taught entirely differently. I was taught the concept of the derivative and the integral. I was taught from pictures before I was taught with symbols. The motivation for the concepts was discussed. I don't remember if we discussed it in class, but the textbook even described some of the historical context. We often learned computations in the context of applications: a tank filling with water, a car accelerating, etc.[+]

Do you know what? Calculus was tested the same way in my school system here in the U.S. as in Korea. Each test had a bunch of problems. The more problems I solved, the higher my score. That's it! Nothing creative or touchy-feely or magical about it. Not really any different from the way my ex was tested in Korea. At the end of the course, there was a standard test administered uniformly across the nation (the Advanced Placement exam.)

American teachers routinely invoke the idea of "teaching to the test" to imply that there are bad teaching methods that lead to better test scores than good teaching methods. Basically, they threaten to teach more and more poorly the more their students are tested. The truth is that under pressure to produce good scores, teachers continue to use the techniques they believe are effective. Teachers in the U.S. are taught that rote learning is not effective, so they don't use it. Teachers who believe that rote learning is effective will use it regardless of whether their students are tested or not.

tl;dr My ex-girlfriend and I were TESTED the same way, but we were TAUGHT entirely differently, because our teachers were trained differently.

[+] Of course, it turns out that in order to learn how to do the problems, I had to actually do the problems, and to a limited extent, the more problems I did the easier they became. In the United States education system, we are taught to be horrified by this evil, poisonous art of getting better at something by practicing it, but we still do it. Teachers universally disapprove of it, but they have not figured out how to eradicate it from their classrooms. (Getting rid of testing would be a good first step, I suppose, since then we wouldn't have to worry about whether the students can actually demonstrate competence at anything; we could teach them to attain pure, perfect internal understanding untainted by any outward manifestation.)

The problem that "teach to the test" introduces is that students focus on factoids and identifying the correct answer in multiple choice, but in the real world (or in college), I'm less likely to have to pick WW2 generals out of a lineup and more like to need to come up with a few paragraphs on what caused it or how it was won or something. Now AP tests do (at least partially) solve for this with their free response and short answer sections, but Standards of Learning (or other state tests) may not.

As far as I know, there aren't any subjects that are best taught through rote. (The confusion between rote learning and practicing skills is a pet peeve of mine.) Learning and testing don't need to resemble each other, not at all, not any more than a physical exam at the doctor's office needs to include a balanced meal, eight hours of sleep, and some light exercise. I think I established that pretty well in my comment: the way I was taught calculus was very different from how I was tested. I was taught calculus through lecture, demonstration, and practice, using applications to motive a new concept, covering the concept intuitively before learning the mechanics, and finally learning and practicing the relevant computational skills. My ex was the one who was taught according to the philosophy that learning should consist of mimicking the test over and over again.

Most elementary/middle and even high school tests aren't nearly as good as AP tests.... Now AP tests do (at least partially) solve for this with their free response and short answer sections, but Standards of Learning (or other state tests) may not.

In other words, standardized tests are capable of reflecting and encouraging progressive teaching ideals OR regressive fact-regurgitation models of learning. I have no doubt at all that poor tests can be written, have been used in the past, and are currently in use in some places. Tests can't help but reflect the beliefs and competence of the people who write them, but that applies to everything else in education. Nothing is immune to being screwed up by individual or institutional incompetence. If you assume competent, progressive teachers and incompetent, benighted test-writers, it's easy to conclude that tests will drag down educational standards. That isn't fair, though. There's no reason to assume that the test designers are any less enlightened and competent than the teachers manning the classrooms.

in the real world (or in college), I'm less likely to have to pick WW2 generals out of a lineup and more like to need to come up with a few paragraphs on what caused it or how it was won or something

I have to disagree with you there. That NEVER happens in the real world. In real life, you most commonly draw on historical knowledge to generate context for things you read or hear. You often have to rely on little factoids such as names and dates to generate connections between what you're reading and what you already know, because you might be hearing something from somebody who doesn't know the context themselves. (Or you might be reading something by a political pundit who is being selective with the facts.) Knowing dates is highly underrated. In a classroom, you learn that dates are the most sterile and inert form of historical knowledge. That may be true for academics or for students, but for laypeople applying a casual knowledge of history in the real world, dates come in really handy. They're the Kevin Bacon of historical connections.

You are putting the cart before the horse. Standardized tests in America circa 2004 allowed for that flexibility. You had an education system that allowed for progressive thought precisely because you weren't forced to standardize.

as I have argued before. Your standardized test earlier, and your teaching system, was NOT comparable in ANY way to what Korea/Japan/India have.

You are now recreating those systems with Standardized COMPULSORY tests. Your comparisons are superficial and invalid.

> . If you assume competent, progressive teachers and incompetent, benighted test-writers, it's easy to conclude that tests will drag down educational standards. That isn't fair, though. There's no reason to assume that the test designers are any less enlightened and competent than the teachers manning the classrooms.

You are painting a far too rosy picture of test taking limits. And part of that reason is because you aren't straying into the details.

Tests can be only varied on a few axes - if you are testing about the revolt of 1857 or the various ways to calculate capacitance there is only a small subset of problems that you can come up with.

Don't forget - These are STANDARDIZED tests - they have to be graded either through an answer key, or through grader judgement - if you allow too much on grader judgement then results aren't standardized. SO it ALWAYS skews to answer keys.

This results in all tests becoming predictable. You do not have that many degrees of freedom when standardizing.

> I have to disagree with you there. That NEVER happens in the real world.

Studying/testing is never about real life. IT can't be. No one knows what sort of life a student is going to lead, so you can only hope to give everyone a similar base of information which should help them find their own paths eventually.

As a result, data has to be produced in long form story format, and tested to see that key pieces of information were retained.

Real life doesn't come into it.

That's essentially waving a magic wand. You are super privileging teacher training as some sort of silver bullet solution, but missing the larger system that works actively against the outcome. You are undervaluing the various pressures which get applied to education, one important one being parent desires to see their kid succeed.

It doesn't matter if the kid understands or not. It doesn't matter if he can answer more important questions but isn't able to perfectly prove a theorem.

No amount of teacher training can stand in the face of a single determined parent who wants their kid to ace that test.

Heck a more trivial point which is still important? - teacher empowerment must come hand in hand with training, because each student is vastly different even from him or herself depending on what time of the day/month or year it is. Teachers must be able to make graded and customized judgement constantly. Tests don't allow for that.

Moving the power for measurement into the hands of the test dis-empowers teachers. The test becomes/IS all that matters here, not the education.

And Tests only roughly measure learning. They MOST strongly measure whether you could answer that particular test.

The test is all that matters.

As a result, the kind of teaching you value, becomes deprecated. Its just not economically competitive. As the original article posits. You don't need teachers to pass the test. You need drill Sergeants.

Teachers who are good at that are the ones who will come to the fore, other teachers don't matter.

You could love your subject and instill it in students. But that doesn't guarantee a good grade. Loving a subject, being involved in it, and testing well in it are different. Especially if you don't "think the test".

This is the final pressure - loving a subject or enjoying it is irrelevant. Students care even less, they know that as long as that they get the test - its all good. "I'm getting good marks In a stupid subject like chemistry, who cares?".

Do you want those people becoming your computer science engineers?

The system you are setting up is concerning, and no amount of teacher training will solve it.

-----

Further I think you are talking about standardized tests America had 30 or so years ago. Speaking on behalf of student body of India - we would have loved to have tests like that.

At least in America you had the option of making a life if you had some gumption even if you didn't ace those tests. You were a person as opposed to a Frankenstinian mental athlete.

Our tests were/are compulsory, for many people not in their first language, and they determined everything from your social standing to your marriage prospects. You started studying for your tenth standard exam 2 years before they started, you had no vacation and you had no spare time.

Then 2 years later you would do it for your ISC or IIT or CAT.

Do note, as far as standardized tests go - the SATs were fun - I personally loved taking them and practicing them. They weren't compulsory either. And if you didn't take them, well you weren't going to be a loser in life. There were other things you could do.

Please understand, that without NCLB you could have all those "IFs". With NCLB the dynamic has changed very drastically, and those conditional flags are set.

SATs are very different from NCLB mandated tests to measure student performance mate. They are compulsory and everyone gets graded. (Remember the last time someone ran an experiment where they gave some kids fake good grades and some kids fake bad grades?)

Test structure and test making knowledge isn't static either - it evolves. One of the first thing it starts becoming aware of is people gaming the system - cheating, memmorizing and so on. It tries to adapt to this by making the test more complex or variable. But then to standardize it has to create a (publicly available) rubric for graders to follow. So the process just renews.

You've started down an evolutionary path. Your tests are moving down a particular road. What you see today isn't what you are going to have.

Further I don't need colleges to have a single do-or-die test. - I just need parents to start privileging the test. IT doesn't matter what colleges do unless they work actively against the test. All that matters is that parents start pushing their kids to do well in it. This is one of the engines of the self fulfilling prophecy.

Standardized tests aren't being misused. They are achieving the kind of goals they usually achieve.

I just saw your reply. What would you suggest should be done about the problems you think are most prominently mentioned in the submitted letter?

Teachers unions see seniority concessions as a way for school administrators to cut costs by shedding senior teachers and replacing them with underqualified junior teachers.

This, by the way, is a good example of how complex teacher labor issues are. Not every unionized teacher shares the objectives of the teachers union. Younger teachers might favor merit pay and looser seniority rules.

Junior teachers are often more easily pushed around, while senior teachers can have the experience and connections needed to push back against bad administration.

Letting administrators determine who the "best" teachers are can mean that administrators will pick the most obedient, not the most effective. A seniority system has obvious flaws, but I don't see increased bureaucratic power as much of an improvement.

Yet the senior professionals in those fields are valued. I suspect because their employer cares if the organization does a good job. I've seen school administrators range from mildly concerned to completely unconcerned. As long as the bosses' jobs aren't in danger, they don't care, and their jobs tend to only be in danger when something goes really really wrong, like a school shooting. So, like any other bureaucracy, they work hard to clamp down on anything out of the ordinary.

I think school choice is important for getting administrators to care. Of course, if parents pull their kids out of school A and into school B, and some higher power (like a district) simply moves administrators over from A to B, nothing has been accomplished.

"And if the choices have run out for my toddler when he’s ready for school, I will do it myself. Maybe I’ll do it for others, as well. Who knows."

questions about a potential education system hack popped into my head. Maybe many others have already had this idea...

What about uniting the home-schooling and frustrated-teachers movements?

I'm sure there are onerous restrictions on setting up a "school" in the U.S. But what about an intensive tutoring system catering to home schooled children? If you had a tutor (who happens to be a formerly disgruntled public school teacher) in a home, how many other children could be in that home, receiving personal tutoring, before no longer being considered a "home school" environment? What if the formerly disgruntled public school teacher drove to various homes during the day or the week? Can "home school" instruction happen in a place other than the child's residence and still be considered home schooling?

Obviously, this could be supplemented with online curricula.

If anything like this is legally feasible, seems like an online service allowing parents to personally select educators sharing their vision for how they want their children to be educated could be hugely disruptive to the current K12 educational establishment.

Does anything like this already exist?

Seems like a net-negative to me.

Perhaps part of the issue is that people are focusing on a 'silver bullet' for education. Standardized tests, charter schools, and even things like kahn academy have been offered, but it's all based on a false premise. There is no silver bullet. We know the strategies we need to take to improve education, but we always fail at tactics because of the assumption that there is one tactic that can be applied everywhere and it's just going to work.

That said, there might be one tactic that could work everywhere: teacher autonomy. Let the people in the classroom decide the tactics they need to use in order to teach the children they are charged with educating.

Education is far from the only service industry for which there is no silver bullet. Other such industries function by fostering expertise at management levels in addition to the "work" level, but always leaving the final decision in the hands of the consumer. On the other hand, USA public education takes no consumer input at all, and it's crap. The reason we continue to attempt doomed one-size-fits-all strategies (and really, always using a single poor tactic is poor strategy, despite what we think we may know) is because we intentionally ignore the primary input that helps other industries regulate quality.

I actually don't think teachers as a whole are that bad (of course there are individual exceptions), but the management/administration of USA public education is abysmal. If most current administrators were actually responsible for firing underperformers while recruiting talented new teachers, they would fail miserably at those tasks.

I'm all in favor of teacher autonomy when we have sufficient public vouchers and other similar mechanisms in place to guarantee parental choice as well.

* Charter schools would pull out the best students (or at least the ones with the most motivated parents). Research generally suggests weak students get lots of benefit from being in class with strong students.

* Public schools benefit greatly from economies of scale. If 10% of students (and thus money) is taken away, the public school still needs to pay for existing fixed infrastructure

* Charter schools frequently take the cheapest students — special ed students, for example, are still left to public schools.

Taken together, this makes charter schools a net negative.

I'd like to see some data about this "net negative" claim. I think forcing bright kids that want to learn and advance to suffer through substandard school and disruptive environment that suppresses their will to learn and prevents them from achieving better results because the school needs their parents' money and they will be "good influence" on those bullies and delinquents - is plain wrong. A person - even a kid - should not be means to an end. It's just immoral - and judging from the results public school achieve statistically, being more expensive and less successful that private ones - also impractical.

However, I agree that people can be stupid and pay money for products that aren't nearly as good as they are advertised. So given some data I'd be inclined to update my opinion on private and charter schools quality. Where could I see the data that your opinion is based on?

Their overview: "Some charters do better; the majority do the same or worse. CREDO also moved beyond individual student performance to examine the overall performance of charter schools across multiple subject areas. They found that while some charter schools do better than the traditional public schools that fed them, the majority do the same or worse. Almost one-fifth of charters (17 percent) performed significantly better (at the 95 percent confidence level) than the traditional public school. However, an even larger group of charters (37 percent) performed significantly worse in terms of reading and math. The remainder (46 percent) did not do significantly better or worse."

Education is very hard to judge, particularly for a new institution. I'm not at all surprised people pay for private schools or apply to charter schools — exclusivity might seem to provide additional value.

Also, charter schools != private schools. I don't know the data there, but if you have lots more money than a public school, different things are possible.

Public schools benefit much less from economies of scale than their administrators, architects, and accountants (who do so benefit) would have us believe. This is why one of the most effective public-only reforms is to break up large schools into smaller ones. The principals don't have any army of vice-principals behind whom to hide, and there is some accountability for what happens in class. Very few charter, private, or parochial schools ever become the behemoths that public schools are, and few of the parents and students who choose them would want them to.

I agree that students with special challenges should have commensurately larger vouchers they can take with them to whatever school their parents judge to be the best for them. It's hard to imagine that alternative schools would do worse for those with IEPs than most public schools do.

Tragedy of the anti-commons, basically.

Google "homeschool co-ops"

Also, any parent that wants to be involved with their kids education should check out Odyssey of the Mind (getting kids on teams to do creative problem solving. Fun!) -- http://www.odysseyofthemind.com/

Coming up with a rule is very hard. But it's pretty easy to identify the good teachers - everyone at the school knows who they are. Just ask.

How do you think should they do this? If you use test scores, teachers will try to "teach the test", or even encourage students to cheat. Student surveys might work, unless the students just dislike certain teachers (who are effective, but strict). Performance reviews can be rubbish.

I'd bet properly designed teacher assessments (tests, student surveys) could find the worst 10% of teachers, though.

You need to have human judgment involved.

The problem is that the teachers' union does not trust the administrators (probably with good historical reason, but that's neither here nor there) to make those decisions using human judgment. They will only submit to clear rules, and, as with any organization, the rules will tend to benefit the members of the org that are in charge of bargaining.

You underestimate the ability of people to use such systems to end up enforcing a stifling uniform and grey mediocrity.

Turn it on its head - if you switched the voting on HN to find the worst commentors, instead of bringing the best up top - what would happen.

Except it's not: http://blog.fogcreek.com/why-do-we-pay-sales-commissions/

[0] "Building a Better Teacher" http://www.nytimes.com/2010/03/07/magazine/07Teachers-t.html...