You really do need the fenders, otherwise you'll be throwing rocks and crap in all directions, not just at the people behind you. If you are running without a top, you could conceivably put a rock into the back of your head, or a passenger's. That, and if you ever take it wheeling in a state with fender laws, I doubt that the po-po will be impressed by your out of state license plate.
Actually, even with fenders it's possible for your Jeep to bounce a rock off the front of your trailer, and back at the Jeep. That cost my insurance company the lift glass for my hard top a few years ago. Sounded like an explosion, for a few seconds I thought somebody was shooting at us. Every once in a while I still find a small piece of glass back there. After that happened I built a plywood shield for the back window that I can attach to the roof rack when pulling a trailer with the Jeep.