that's largely solved problem. ieee758 defines consistent rules for dealing with infinities. even if don't use the floating-point parts and made a new integer format, it almost certainly would make sense to lift ieee754 rules as-is.
A IEEE754-like arithmetic (transrational arithmetic, or transreal arithmetic) creates new problems due to adding new values. 0*x=0 now requires x≠∞, x≠-∞, and x≠NaN. (x/x)=1 now requires x≠0, x≠∞, x≠-∞, and x≠NaN, so this system doesn't satisfy the field axioms. NaN lacks ordering, so we lose a total order relation.
However, you get cool new results, like x/x=1+(0/x). Definitely some upsides.
