The door panels actually can be removed quite easily. First off you need to remove all of the screws that hold the door panel on. I think there are about 4 or 5 of them. Two of them are kind of hidden, one in the black plastic area near the rearview mirror, and one behind a plastic plug just behind the door opening lever. Once you have all of the screws out use your fingers to slowly pry the door panel away from the metal door frame, beginning with the outside edge of the door. What you are doing here is trying to carefully remove the plastic connectors that hold the door panel in place. With some gentle force they should pop away from the frame. Work your way around the door panel along the rest of the outside edge, then the bottom edge, then the inside edge, and finally the top of the door. When you have all of the connectors free the only things holding the door panel to the door frame should be the wire connectors for the power window switch, and the courtesy lamp (you may or may not have these connectors in your jeep) Simply disconnect the wires and you're done. If you have manual windows you'll probably need a special tool to remove the window crank handle. At this point you should have easy access to the door speaker. You'll want to check all of those plastic connectors before you reinstall the door panel to see if any of them are broken. If some of them are broken, then pay a visit to a local body shop of see if they won't sell you some. Often they'll even give them to you in hopes of future business. Shucks auto supply doesn't have them, I know that for sure.
1993 ZJ Limited 5.2 - no mods - Red with Gold (what were they thinking) trim.