Unit tests are for documenting the API contract for the user. You are going to target based on what you are willing to forevermore commit to for those who will use what you have created. Indeed, what happens when two messages come in with the same message ID is something the user needs to be aware of and how it functions needs to remain stable no matter what you do behind the scenes in the future, so you would absolutely want to document that behaviour. How it is implemented is irrelevant, though. The only thing that matters is that, from the public API perspective, it is handled appropriately.
There is a time and place for other types of tests, of course. You are right that unit tests are not the be all and end all. A good testing framework will allow you to mark for future developers which tests are "set in stone" and which are deemed throwaway.
We're in general agreement about the purpose of unit tests. I disagree on a couple of points.
Tests do not document the API. No test is complete, and for that reason alone can't completely document anything. For example, a good API might specify that "the sender must be non-null, and must be valid per RFC blah." There's no way to test that inclusively, to check all possible inputs. You can't use the test cases to deduce "we must meet RFC blah." You might suspect it, but you'd be risking undefined behavior it you stray from input that doesn't exactly match the test cases. And before anyone objects "the API docs can be incomplete too," well, that true. But the point is that a written API has vastly more descriptive power than a set of test cases. (The same applies to "self-documenting code". Bah humbug.) There's also the objection "but you can't guarentee cases you don't test!" Also true. That's reality. _You can never test all your intended behavior._ You pick your test cases to do the best you can, and change your cases as problems pop up.
The other thing I would shy away from is including throwaway tests in the framework. Throwaways are a thing, developers use them all the time, but don't make them unwanted stepchildren--poorly (incompletely?) designed, slapped together, limited scope, confusing for another developer (including time-traveling self) to wade through and decide whether this is a real failure or just bogus test. They're tech debt. Less frequently used tests are another matter. For example, release-engineering tests that only get run against release candidates. But these should be just as much real, set in stone, as any other deliverable.
Which I guess is a third viewpoint nuance difference. I treat tests as being part of the package just as much as any other deliverable. They morph and shift as APIs change, or dependencies mutate, or bugs are found. They aren't something that can be put to the side and left to vegetate.
> There's no way to test that inclusively, to check all possible inputs.
Which means the RFC claim is false and should not be asserted in the first place. The API may incidentally accept valid RFC input, but there is no way to know that it does for sure for all inputs. You might suspect it conforms to the RFC, but to claim that it does with certainty is incorrect. Only what is documented in the tests is known to be true.
Everything else is undefined behaviour. Even if you do happen to conform to an RFC in one version, without testing to verify that continues to hold true, it probably won’t.
This is exactly why unit tests are the expected documentation by users. It prevents you, the author, from make spurious claims. If you try, the computer will catch you in your lies.
> The other thing I would shy away from is including throwaway tests in the framework.
What does that mean? I suspect you are thinking of something completely different as this doesn't quite make sense with respect to what I said. It probably makes sense in another context, and if I have inferred that context correctly, I'd agree... But, again, untreated to our discussion.
OK, one more round. An API spec is a contract, not a guarentee of correctness. You, as the client, are free to pass me any data that fits the spec. If my parsing library does the wrong thing, then I've got a bug and need to fix it. My tests are also defective and need to be adjusted.
If you passed 3.974737373 to cos(x), and got back 200.0, would you be mollified if the developers told you "that value clearly isn't in the unit test cases, so you're in undefined behavior"? Of course not. The spec might be "x is a single-float by value, 0.0 <= x < 2.0 * PI, result is the cosine of X as a single-float." That's a contract, an intent--an API.
The same for a mail parser. If my library croaks with a valid (per RFC) address then I've got a problem. If I try to provide some long, custom, set of cases I will or won't support, then my customer developers are going to be rightfully annoyed. What are they supposed to do when they get a valid but unsupported address? Note we're not talking about carving out broad exceptions reasonable in context ("RFC 5322 except we don't support raw IP addresses foo@[1.2.3.4]", "we treat all usernames as case-insensitive"). And we're not talking about "Our spec (intent) is foo, but we've only tested blah blah blah."
Early in my career I would get pretty frustrated by users who were not concerned with arranging their data and procedures the right way, clueless about what they really were doing. OK, so I still get frustrated by stupid :) But it's gradually seeped into my head that what matters is the user's intentions. Specs are an imperfect simplificaton of those very complex things, APIs are imperfect simplifcations of the specs, and our beautiful code and distributed clusters and redundant networks are extremely limited and imperfect implementations of the APIs. Some especially harmful potential flaws get extra attention during arch, implementation, and testing. When things get too far out we fix them.
There is a time and place for other types of tests, of course. You are right that unit tests are not the be all and end all. A good testing framework will allow you to mark for future developers which tests are "set in stone" and which are deemed throwaway.