The original error could have been checked with a quick code review verifying that the provided number was in fact prime. This should have been done when the original patch was submitted.
I've tested some of them with PARI/GP. There is no probable prime with the Hamming distance 1. There are several (probably 3--400, haven't exhaustively listed) probable primes with the Hamming distance 2 (p ^ (1<<30) ^ (1<<14) is one example).
Well, the provided command takes 2.5 min to for pari to test the given "prime" on my computer. There are 16*256 numbers with a single hex digit mutation from the number given.
The change log message that introduced the new constant is interesting, too. "Socat did not work in FIPS mode because 1024 instead of 512 bit DH prime is required. Thanks to XXX for reporting and sending a patch."
> The power of the Baillie-PSW test comes from the fact that these lists of strong Fermat pseudoprimes and strong Lucas pseudoprimes have no known overlap. There is even evidence that the numbers in these lists tend to be different kinds of numbers.
A primality certificate could be included that allows the prime to be verified quickly during the build. This certificate for the 2048-bit prime took 90 seconds to generate, and can be verified in 3 seconds.
The primality test of the original number using gp that archgoon posted takes 6ms on my Sandy Bridge laptop, that's plenty fast. (The new number takes a minute to check, though, so that might really be too long)
Maxima's primep takes 360ms on the 1024-bit one for me, so seems quick enough to stick into the build.
It sounds like the chance if it passing the test is around 10^-15 using the default values; which seems sufficient for a build test which is run many times.
Kind of interesting that the first nonprime version ended in a comma, as if it originated as a fragment of a longer array of bytes. That's nonstandard C, and I'd expect that whatever tool(s) generate those arrays would know not to add a comma after the final entry.
<cynical thought>That's a nice piece of plausible deniability, huh? I wonder if the committer has a few extra bytes that he can claim were supposed to be there which make a number that passes all the primality tests? That'd be a nice excuse for the NSA/3PLA overlords to have given him...
(Actually I may be confusing the extra comma here with an extra comma at the end of an enum list. Some compilers are OK with that, while others aren't.)
If I'm reading N1256 right (not sure how far back it actually dates, but that's a C standard draft dated 2007), section 6.7.2.3 also explicitly allows a trailing comma in that context as well, so if such a compiler exists its makers should get with the program, so to speak.
