I hear ya. That is why my DD is a 24 year old YJ. The only creature comforts it has are a center console with a nice padded armrest , radio and a hardtop with a good heater lol.
I don't need to have luxuries and all this nice stuff, in fact I prefer not too because I lose focus and either stop paying attention, or I start falling asleep at the wheel. I don't like driving the Nissan I'm stuck in right now because it's boring to drive, and too quiet and smooth so to speak. AC sure as **** is nice on days like this though, hasn't been this humid out here in a while.
When I drive a Jeep, pickup, semi, or any other vehicle bigger than a sedan, I'm paying attention.
I prefer a manual in a true sports car (of which very few are made today if any) but otherwise I want an Auto. Three years ago I was attacked by "sweet" and "wouldn't hurt anybody" pit bulls while riding my bicycle. this resulted in my femur being broken in four places and my hip. As a result, I am a little lame now (and a lot richer for it) and while I can do most things just fine, repeatedly pushing a heavy clutch would be a PITA.
You have a perfectly good reason, more than just personal preference or opinion. But, if curiosity gets the best of you, you can always float the gears. As long as you don't grind or jump when you shift, it won't damage the transmission.