We've seen him have so many adventures (with who knows how many unseen ones in between), and he's had five or so regenerations - all in a time in which he's somewhat consistently claimed to be 900 (or, in a few cases, 1000ish). So if he *does* know what he's talking about regarding his age, all we've seen of him since the late 1970s/early 80s or so happened, for him, in just one century - at *most*. And quite possibly in as little as a few decades. There were much greater age gaps between his earlier incarnations, but nowadays he seems to be dying excessively often, without aging appreciably in between.
Yeah, but that's exactly my point.