There's acceleration of the read head to move it between the tracks. So it may well be 4ms because shorter distances are penalized by a lower peak speed of the read head as well as constant factors (settling at the end of motion)
That’s a bogus number from an ancient slide deck for a class in 2001 or so, that’s misled generations of folks googling for the answer.
Note also that the outer tracks are longer (google ZCAV) and hold more data, so seeks across uniformly distributed block numbers do not generate uniformly distributed track numbers.