There's no reason to eliminate that bracket unless you are going to replace it with a weld on bracket that fits your lift better.
You say you have a three inch lift, I would cut the top hole off and just use the bottom or add one lower.
You will have better handling if you use a bracket that raises the roll center.
My bracket is raised 3" and makes the track-bar close to horizontal.
This also helps to minimize the axle shift due to changes in cargo weight.
Edit; On second look that is higher than I thought and it might not be possible to drill it low enough!