You don't have to instrument every single arithmetic operation. Multiplicative overflow can only happens if the value of one or the other operand is >= 2^(N/1), where N is the number of value bits.
On 64-bit machines, this means if you use 32-bit values and properly promote them, or one 32-bit value and one 64-bit value, you don't have to instrument.
Also, often times the better strategy is to design your code to be immune to overflow. Garbage-in is garbage-out, so if somebody gives you garbage then they can expect to get garbage out. All that matters is that you don't allow that garbage to taint the state of your program. I usually use unsigned arithmetic, because then I don't need to worry about undefined behavior in C. And for general bounds checking tasks my only condition is whether something is >= object-size.
Some of us rigorously analyze our code for overflow problems.
Those of you who don't are like the programmers from 20 years ago who thought that it was too tedious to do bounds checking on buffers, and that because of all the existing code it wouldn't improve security.
Well it did matter. Like with buffer overflows, it's a collective action problem. The dilemma you point out doesn't exist if you stop being so resistant to change.
Your comments basically state the same thing: "we avoid these problems in C by conscientiously choosing the data representations and defending against the problems while creating and implementing the original design" which is true. But this is different from going back into a large and complex C program whose design and implementation neglected the issues.
Fixing some realloc calls is possible only because the function has an error return code, and so when the overflow occurs in the multiplication, it can "piggy back" on that null return. The hope is that the caller is already handling null properly and so everything is cool. (Is it? Was a test case ever executed in which that particular realloc call returned null?)
In general, multiplications in C take place in situations where there is no expectation of an error, and so no error handling strategy in place in the surrounding code. It may be difficult to introduce it, and even more difficult to test all the cases.
> I usually use unsigned arithmetic, because then I don't need to worry about undefined behavior in C.
That is wrongheaded, I'm afraid. The unsigned types behave poorly under arithmetic, because they have a large discontinuity immediately to the left of zero. And because their wrapping behavior is well-defined, it basically means that unintended wrapping is not diagnosable as an error (without generating reams of false positives).
It is unfortunate that size_t is an unsigned type. This is for historic reasons. When C ran on machines where size_t was 16 bit, making it signed would have meant limiting the size of an allocation request to 32767, rather than allowing up to 65535. That would have hurt, and induced people into workarounds such as writing their own allocators. (Which, ironically, they do anyway, to this day.)
Unsigned types are best used for things like bit fields, and simulating the arithmetic of particular machines. They are also useful for doing clock arithmetic on a wrapping interrupt timer register. (This is best done with macros or functions that safely wrap it, like "time_is_before(t1, t2)".)
"unsigned char" is useful, but note that values of this type undergo default promotion (on most compilers, by and large) to a signed value of type int.
Signed overflow is undefined in C. Period. That means that detecting overflow for signed types is really messy.
For addition one method would be to subtract the maximum possible positive value. The only way to do that is to use a pre-defined macro. Which means the logic for your check is divorced from the actual operation--in other words, if the type of the operand changes, your check becomes _silently_ worthless.
The alternative exhibits at least _two_ bad security practices: 1) an independent constraint on input divorced from the risky operation is a recipe for bugs because you have no warning when the conditions change such that your input verification fails to properly constrain the values; and more generally, 2) relying on assumptions about a type, rather than relying on the actual behavior of a type or information derived directly from the type, is especially bad form in C because of its loose typing.
I think I'll stick with my strategy, which has worked out well, doesn't rely on undefined behavior, and is less susceptible to bit rot.
Regarding your objection of making changes to existing code, I would just say that we shouldn't let the perfect be the enemy of the good. I also like tedunangst's description of that predisposition: whataboutism.
On 64-bit machines, this means if you use 32-bit values and properly promote them, or one 32-bit value and one 64-bit value, you don't have to instrument.
Also, often times the better strategy is to design your code to be immune to overflow. Garbage-in is garbage-out, so if somebody gives you garbage then they can expect to get garbage out. All that matters is that you don't allow that garbage to taint the state of your program. I usually use unsigned arithmetic, because then I don't need to worry about undefined behavior in C. And for general bounds checking tasks my only condition is whether something is >= object-size.
Some of us rigorously analyze our code for overflow problems.
Those of you who don't are like the programmers from 20 years ago who thought that it was too tedious to do bounds checking on buffers, and that because of all the existing code it wouldn't improve security.
Well it did matter. Like with buffer overflows, it's a collective action problem. The dilemma you point out doesn't exist if you stop being so resistant to change.