You're in good company with Jonathan Blow and, it appears, John Carmack. Teams pushing TDD might be a good smell test for whether they know how to efficiently code.
Until you mentioned him I had never heard of this Jonathan Blow character. The thing is both of these people seem to do game development. If you write code that is close to the hardware and needs every ounce of performance there might be a point to this. It would make applying TDD both harder to pull off and less beneficial.
When one is writing complex logic that is supposed to receive both frequent new features and keep working correctly there is nothing that beats TDD.
Let me attempt to compare this to bowling even though I know nothing about bowling. If you just occasionally like to go out bowling you can use whatever technique you like and you will get better if you just bowl a bit more often. If you want to play bowling on the national level you may at some point have to drop the technique that you made up yourself when you started out. You need to go through the uncomfortable stage of learning a new an better technique and temporarily be worse at it. I think the people not doing TDD are just refusing to go through this because there is not much of an objective standard of quality for developers. We are in a field where code that lacks any sort of quality is standard and where breaking features three times a week is standard. TDD will fix these things.
TDD works great when the requirements are known and there’s little mystery. Waterfall methods oddly enough lends themselves well for TDD, which is kind of associated with hip styles like agile.
The reality is most projects start as exploration. What you’re thinking might not even be possible. Can you even approximate the classes you’ll need yet? For these scenarios TDD fits in awkwardly.
I prefer to do a little back and forth. A little dev. When things are looking stable, a little resting. Towards the end a lot of testing.
I've found the opposite. When I start to write a test, I often realize I don't know exactly what I need to do because the requirements are unclear or somehow conflict with some other assumption of the system. That realization prompts me to go talk to my product owner, which forces them to clarify their requirements and target a more coherent idea of what the system should be.
There is some point to your comment but also something that seem to me to be misunderstanding. Also, part of it seems self-contradictory to me.
There can indeed be exploration stages where TDD does not help that much. E.g, if you are going to interface with a piece of hardware you have never seen before and you need to get some sort of working communication going there is much point to what you say. As soon as you know what sequence of commands makes the piece of hardware work one has arrived in the territory where TDD is highly beneficial. You can then write a test where you require that your software indeed produces the correct sequence of commands and after that you can refactor your exploratory code while having to check that it still works together with the hardware much less often then otherwise would be needed.
The remark about TDD and waterfall seems much less true to me. In my original post I wrote 'both frequent new features and keep working correctly'. That does not describe waterfall at all. In fact, and this is where you messages seems to get a bit self-contradictory, your post actually sounds much more waterfally to me than mine. It contains the words 'Towards the end'. It is waterfall projects that have an end that was envisioned right from the start. Whatever 'agile' means, it seems to have the feature that we never stop adding features and that stuff is supposed to be in a working state in between adding these features. So, if one is actually doing that there are dragons in the sentence 'Towards the end a lot of testing' because it actually turns into 'always lots of testing' which costs lots of time. In case of the hardware example it would also require that every developer has a species of that hardware on his desk next to his computer. As I happen to work in a field where we interface with hardware that is three stories high, we should realize that this is not always practical.
One misunderstanding that might be going on is that perhaps you are assuming that in TDD one would test just one class at a time. I generally prefer to test a set of classes, i.e., a subsystem of a whole program. In that case the test are about properties of the program that, in many cases, the user could recognize as beneficial. There should be less churn at that level. I suppose some people would call this behaviour driven development instead but I am not so very married to terminology and will just keep calling it TDD. Testing individual classes can also be helpful but if they contain rather simple logic I am not sure writing tests for that is so very helpful. At some point would just be testing whether the processor really is capable of copying values which is not that interesting.
I think as a first order approximation the level to which you can deploy TDD is inversely proportional to how difficult your problem is with regards to computational complexity. The most TDD’able code is a freshman level CS 100 programming assignment. The least TDD’able are things like real time simulations, numerical computing, machine learning, etc. In the latter case, you can TDD the boring parts (reading inputs, simple pure functions, etc) but TDD breaks down quickly if you have a huge state space, side effects, complex outputs, etc.
That depends. A problem may be quite difficult in computational complexity but if you can find simpler less computational complex version of it that can be used for automated tests that can be quick. E.g., test your neural network training procedure with a smaller network which will give a result that performs worse and after that use the thus obtained program on a with a bigger neural net.
It's not about runtime performance. It's about how well you can actually assess your code by matching small and well-defined inputs to small and well-defined outputs.
LeetCode / CS101 problems are almost always like that. In the real world, it's also useful to separate out any tricky "policy" logic into nicely testable pure functions. But you are still going to have a bunch of side-effect munging left over, and unit testing is terrible at that. Tests that only assess their own mocks are a waste of time.
Could you give an example of situations where there would not be small and well-defined inputs and outputs because this certainly does not sound like a common situation to me?
And what do you mean by 'Tests that only asses their own mocks are a waste of time.' Certainly, many side effects can very effecitively be checked by mocks. Now, it could certainly get much more difficult if the side effects were very large and hard to define as you said in the paragraph before that, but I have some difficulty picturing that as a common situation.
> If you write code that is close to the hardware and needs every ounce of performance there might be a point to this. It would make applying TDD both harder to pull off and less beneficial.
It's not really about being close to the hardware[0] - there's many other things that are unrelated to low-level performance tweaks that make TDD or even unit tests a waste of time for experienced game developers.
When someone like Blow or Carmack sees an automated playtest fail, they have a honed intuition for what caused it. This intuition was built up over many years of hard-earned experience. They simply don't need the lower level unit test that says "this matrix multiplication math is wrong" - they'll see a collision fail or a weird scaling thing and _know_ right off the bat the candidates in the codebase where things went wonky and fix it before you can even say "look at the logs" :) Since they get the same results without having to write unit tests everywhere, and especially without writing tests first, it's simply an unnecessary overhead to make them write/maintain it.
I don't think anyone would argue TDD and/or unit tests are valueless, it's just a cost/benefit question. Pretty sure they'd use any technique where the benefit outweighs the cost- and for them, for many reasons in many varied scenarios, it just doesn't.
[0] actually, if anything, I'd bet that the stuff that's really close to the hardware like the core engine stuff (math, memory management, etc.) is the one place they would consider having TDD and/or more unit tests!
If you already know how to code really well, you can easily make a test that tests the things you want to do, and then write code that actually does those things.
If you can’t write code well in the first place, you’ll write terrible tests, and then even more terrible code to make them somehow succeed.
I’d argue TDD has value in the first case, but most teams are more like the second variant.
This all assumes that you know what "the things you want to do" are.
In my experience, coding is the easy part, and pinning the relevant stakeholders down on what they want coded is the hard part. If you're an employee, the hardest part of your job will be in a.) understanding what'll make your boss happy and b.) communicating how to make that realistic. If you're a B2B business, the hardest part of your job is figuring out a.) who your customers are and b.) what they want. If you're a B2C startup, the hardest part of your job is identifying a market with lots of people who want your product.
If you're a maintenance programmer where these things were hammered out a decade ago, then TDD can be a godsend. If you're working with an excellent business analyst / salesperson / cofounder who can hammer these issues out with enough precision that you can trust their answers, TDD is helpful. Neither of these situations describes the gaming industry, nor a number of other very lucrative industries.
If I was arrogant enough to consider myself a good programmer after 30 years of doing it, I would suggest as humbly as possible, that perhaps TDD is something better coders can do in their heads and 'skip over'. How do we know what API to build ? Because the code needs it. How we know the code works ? Because I just ran it.
Which comes right back around to the idea of who you're writing code for. I suspect much, possibly most, of the time it's not purely for yourself, but for your team, or other Co tributes to the project, or thr person that has your job a few years from now, or even you a few years from now.
Just because you have the entire system in your head right now, will you when you revisit it a year later to add a feature or fix a big? How much time will it take to internalize completely? Tests allow you to know less about the system overall and still make assured changes to smaller portions of it because prior you (or someone else) has spent the extra time to make sure any mistakes are caught early and easily.
The go-to counter to this is that you should really know what you're doing and understand the system you're working in, but that's often not only infeasible, but impossible for one person (how well do you know all the libraries you're using?). There are plenty of codebase out there where I think no one person can internalize it all and understand it well enough to always make food choices, and since you'll have to choose what you want to remember, a helpful tool to make sure you don't violate some important facets of how it works is useful.
Now, much of this is about testing in general, but if you think about it, there's a lot similar about writing new code and coming into a large chunk of code that you need to alter but you have limited understanding of or poor recollection of. The same things that keep you from violating how it should function help your design a coherent functionality in the first place.
