The MIDI pins on game ports were an add-on on sound cards to unused pins from the game port, though. The original game port spec doesn't have anything to do with MIDI or UARTs, and joysticks and the like did not use those extra pins.
And iirc game-ports were usually their own expansion card. It wasn’t until Sound Blaster decided to include it on their cards that we started seeing it integrated into the sound card (part of a sound cards job is Analog to digital conversation, why not do the ADC conversation needed for joysticks as well? Also, games normally also want sound so seemed like a nice why to bundle everything you need onto a single card).
