I don't think we have a description of the Canada object that is suspected to be this balloon. The car-sized cylindrical object in Alaska could easily be a balloon if the shape was misreported. The octagonal object in Lake Huron trailing cables could have similarly misreported.
It is hard for fighter pilots to observe small objects passing by them at high speed. I also suspect that they will see balloon as interesting object when can't see the payload.
Identification missions are supposed to also photograph the target though, presumably with an appropriate telephoto camera. No need to rely solely on the Mk I eyeball. But admittedly that’s not trivial either at high relative velocities.
It is hard for fighter pilots to observe small objects passing by them at high speed. I also suspect that they will see balloon as interesting object when can't see the payload.