The handheld firearm was mindbogglingly inaccurate. Hitting a target farther than 50 to 60 yards away was virtually impossible. So think about that. Your average bow or crossbow might not have the sheer force, but you were far more likely to hit what you were aiming at. That was the non-rifled musket. And there was the likelihood of cracking the weapon or even explosion if you used too much powder. It was not uncommon for a musket to misfire or break even under the best conditions, and the inclusion of wooden materials in the making of the weapon was to facilitate easy repair as much as it was to lighten the weapon weight.
The average loading time for the musket for a trained musketeer was twice a minute, or so. Some claimed 3 shots in a minute, but I've rarely seen this accomplished enough to make it believable. So imagine how much happens in the 20 to 30 second that you're there loading your weapon between shots. Many units had musketeers fight in pairs or trios to keep up the fire, and some irregular forces had a pair of people with a skilled musketman to hold up a tower shield to prevent return fire kills by more accurate archers and crossbowmen.
The pistol (which was far larger than our modern pistols) were even less accurate, and if you shot a pistol at an opposing force about 100 feet away, you had little likelyhood of hitting the man you were aiming at, but maybe you might hit someone in the line he was standing in, perhaps 25-50 feet on either side of him. So the pistol was usually a bit of flash and a near-point blank form of death. It shouldn't surprise you that some people held a few loaded pistols in a belt or bandolier, though. When you could see the man across from you was a better swordsman, you could ruin his day with a quick pistol shot, and then turn your attention to someone else who you had better chances against. If you had three pistols, loaded, tamped, and ready to fire, that could potentially be three trump cards to turn a melee in your favor. But you wouldn't waste your time fighting exclusively with your pistol. It was actually quite common to drop the pistol rather than waste time waiting for the barrel to cool so you could tuck it in your belt. Time enough to retrieve it once your enemies were incapacitated.
And that's the last part. Ball shot was catastrophically damaging to the human body. Sometimes you killed your enemy. But more often you maimed them, and shock or infection killed them. So if your foe was crumpled, screaming, and clutching broken, shattered, or mangled body parts, they were out of the battle. And no game is going to simulate the sheer shock and incapacity of a pistol or musket shot. No one is going to want to have their avatar incapacitated for hours, days, or weeks trying to recover and relearn to use their body. It's just not game style. Create the whine and deafness of a near-miss shot? Sure. Create the flash and fire? Can do. It wouldn't be hard to make the fire and brimstone scent for a live performance, or the concussive thump of the report. But the incapacity of the injury is just not something people would accept.