Yes.

If one Yubikey were lost, stolen, or damaged, you would then revoke its access by removing the corresponding entry in ~/.ssh/authorized_keys.

You can load keys into yubikeys if you like (and thus load the same key into multiple devices), but I choose to generate unique keys on-device, so each one has its own (which would make the answer to your question “yes”).

One upside though is that all your keys can go as individual lines in authorized_keys, so there is still only one file to install on remote machines.

There are sites that show how to move your key to multiple Yubikeys. (Basically, backup your keyring before moving to Yubikey, then restore and repeat move to a new Yubikey).

This is the guide I followed: https://github.com/drduh/YubiKey-Guide

Unless you put the same OpenPGP key on all of them, or you want them to have a different levels of access.