Go over to Pirate4x4. It's been done hundreds of times over. But yes, without reinforcing with the likes of a cage, it will be structurally weak. It may be more effort than it's worth, but tell that to the score of people who have done so in their wheeling rigs. I'm sure an adequate solution can be made for the X tube in the back.
"Chilton's and Haynes are for brake jobs, oil changes and starting campfires." - CJ7-Tim
'98 2dr/5spd Cherokee rockin' on tons, 5.9L V8 Magnum, HP60/14b, 5.13:1, locked, 14" coilovers, Dana 300 4:1 twin stick... oink oink!