I've never seen a fully ROM-based PC, but I have worked with ISA NE2000 NICs with boot ROMs that allowed them to diskless boot into a DOS-based Novell NetWare client. It wouldn't surprise me if there were ROM-based local boot environments on ISA cards.
The Cisco PIX firewall, in it's original incarnation, booted from non-volatile memory (I don't know if it was EEPROM or flash) an ISA card.
Fully ROM/Flash PC-ish systems are/were pretty common in the embedded and industrial control space. Usually the boards emulate one or more floppy drives. I've got a half dozen variations, including PROM, Flash and battery backed SRAM. Even one with bubble memory, which is pretty nifty.
The first-gen PIX you mention had an 8M or 16M ISA flash card it booted from, containing the whole OS.
The Cisco PIX firewall, in it's original incarnation, booted from non-volatile memory (I don't know if it was EEPROM or flash) an ISA card.