I have mine removed, though I feel a certain way about it both positive and negative. And also while I feel the whole "doors are structural" thing is total nonsense, I do have unibody rail stiffeners, though I did that for other reasons.
I drilled hinges to 3/8" and used 3/8" hardware welded to the door side and as a result have ZERO door sag. Doors off is great for wheeling, the increase in visibility and comfort plays a big role in finding the right line. One thing though is, with a couple exceptions here and there, I typically DO NOT run on the street doorless.
I have my reasons, though most are aesthetic more so than overall safety. My biggest two are:
1. Kinda gives off a Wrangler wannabe vibe. While Im a former TJ owner and I dont personally view it that way, it is a bit undeniable.
2. Certain angles (i.e. dead on side view) look a little funny. I have carried my two tone bedliner theme over to my sills and b pillar to lessen the effect but its hard to deny it looks like something is missing... well because it is, well over 200lbs of something actually.
My plan is to build a set of tube doors. Will give me something to hang onto without limiting visibility, and will make things look a little more finished and less like some redneck who had his doors fall off.