Most people don’t know some history. During 1990s, a group of people made a fortune out of consulting gigs where they will be called in by their CTO friends in traditional enterprises to save the late and over budget projects. One of these people was Kent Beck. Kent will use his license to kill to turn things around and eventually generalize his rescue formula and sell it to make 100X more. His crowning glory during those days was XP or eXtreme Programming.
Like with all self-help formulas, Kent will label his solution as magic bullet for all software development problems. He will advertise it as secret medicine that cures all ills. He will be at every conference, write articles after articles, publish books.
Also, like all magic self-help formulas, it wouldn’t quite work. So, Kent will invent something new. His next prescription was TDD and when I first saw it, I thought it was a joke. But people around me started drinking cool aid and if you didn’t join them then you weren’t one of them. Again, Kent and friends will go out on massive marketing spree advertising it as secret talisman. Like all overweight desperate people in need to lose weight, people will enthusiastically start new Kent Beck diet, lose few pounds and endorse the formula. But they will soon find that they had simply traded one problem for another more uglier one.
This went on for long time. For more than two decades, these group of people kept inventing these processes, selling it as magic pill and made millions upon millions in consulting gigs, books, training, certifications and so on. They came up with Agile and 17 people in that group created “agile manifesto”. Their most aggressively marketed prescription was scrum. Like their all previous prescription, world is finally coming off of night of drinking cool aid and feeling severe headache.
I think most of these people have now sort of retired after amassing massive fortunes and hopefully we will not see more of these magic processes pushed to dumb CTOs with promises of curing all ills.
The truth is Scrum was never a magic bullet and it is downright harmful for many projects. It is useful for highly predictable projects where research component is negligible, for example, CRUD websites AND where you are stuck with unmotivated tier-3 talent who failed to get job at insurance company. For everything else, it should never have been used. It is especially going to hurt creativity, originality and novelty if you are in business of making a differentiating unique novel product. It also is very very bad choice if you already had tier-1 highly motivated team.
I think even a tier-1 team could make it work for them, but the key is they would make it work for them (make it their own process).
Once you hire a scrum master to tell you how to do your work, you've sort of lost. They are rarely useful other than as sort of "priest" of the process, who ends up becoming another sort of management, but without management powers (usually).
The meetings etc. can be downright useful, in certain cases, but don't really make sense to follow religiously. I.e. if the devs themselves are running the meetings, for themselves, its a useful form of self-management, and you don't need much management skill to run it (any seasoned dev with any sort of communication capability can do it).
But if its imposed upon you, its just a cookie-cutter sort of management, which, doesn't fit all teams or scenarios.
Second, even XP is not a "magic bullet". Nothing is. It's work that works. (Scrum, on the other hand, is not a "magic bullet" but simply a "bullet". Use it to kill projects very effectively).
Third, at my first real job after uni, we did most of the XP-like practices, and it worked amazingly well. But we didn't know about "XP". Partly because it didn't exist yet, as this was around the same time the Kent Beck started at Chrysler Comprehensive. When the XP books came out it was fun to have a name for what we had been doing so successfully. And also to compare and contrast.
Fourth, I had a great side-by-side natural experiment during my tenure at the BBC. My team did XP-ish things, mostly the technical practices, so test first/TDD, YAGNI etc. Pairing when necessary, but we were co-located around a desk "island" (sort of the way journalist workspaces are organised). My team succeeded far beyond expectations [1]
The team next to us, larger, more important and with way more experienced developers did SCRUM, but not XP. That project had to be rebooted completely after 2 years.
> Their most aggressively marketed prescription was scrum.
I don't think Agile has prescribed this though. Scrum, in my view, is an intermediary 'solution' so non-technical 'bosses' can overlook and micromanage dev teams. I guess it all stems from 'unproductivity' really, those cases you mention, where you end up with sub-par devs trying to deliver complex software products.
If you already know _exactly_ what your code needs to do, you can "just implement it".
I find TDD to be very helpful in the cases where I do _not_ know everything in advance, because it lets me take small steps to explore things and I get very fast feedback if I "misstepped".
All of the methods mentioned are based on a reasonable core. It muddles the waters and make the snake oil marketing - the "this is the cure to it all" discourse - harder to dismiss. Testing is good. Planning is good. Discussing the project is good. But these things are beside the point of op. The point is, these cargo cults are designed to make consultants money. Now they have a lot of inertia because people grow up on it.
It seems like the new generation of software development silver bullets is "microservice", cloud "devops" etc... Managed kubernetes is not a bad thing. Configuration files, software defined infrastructure, etc, not bad things at all. But there is a definite market push in consulting for overtly complicated frameworks as The Way and people who are anxious about their complicated projects gobble it up.
>The truth is Scrum was never a magic bullet and it is downright harmful for many projects. It is useful for highly predictable projects where research component is negligible, for example, CRUD websites AND where you are stuck with unmotivated tier-3 talent who failed to get job at insurance company.
Like with all self-help formulas, Kent will label his solution as magic bullet for all software development problems. He will advertise it as secret medicine that cures all ills. He will be at every conference, write articles after articles, publish books.
Also, like all magic self-help formulas, it wouldn’t quite work. So, Kent will invent something new. His next prescription was TDD and when I first saw it, I thought it was a joke. But people around me started drinking cool aid and if you didn’t join them then you weren’t one of them. Again, Kent and friends will go out on massive marketing spree advertising it as secret talisman. Like all overweight desperate people in need to lose weight, people will enthusiastically start new Kent Beck diet, lose few pounds and endorse the formula. But they will soon find that they had simply traded one problem for another more uglier one.
This went on for long time. For more than two decades, these group of people kept inventing these processes, selling it as magic pill and made millions upon millions in consulting gigs, books, training, certifications and so on. They came up with Agile and 17 people in that group created “agile manifesto”. Their most aggressively marketed prescription was scrum. Like their all previous prescription, world is finally coming off of night of drinking cool aid and feeling severe headache.
I think most of these people have now sort of retired after amassing massive fortunes and hopefully we will not see more of these magic processes pushed to dumb CTOs with promises of curing all ills.
The truth is Scrum was never a magic bullet and it is downright harmful for many projects. It is useful for highly predictable projects where research component is negligible, for example, CRUD websites AND where you are stuck with unmotivated tier-3 talent who failed to get job at insurance company. For everything else, it should never have been used. It is especially going to hurt creativity, originality and novelty if you are in business of making a differentiating unique novel product. It also is very very bad choice if you already had tier-1 highly motivated team.
So exercise caution!