The tensioner will move up and down slightly at idle. About 1/4" in either direction. But if yours moves more than that, to remove there will be a bolt in that center hole on the silver part that you can back out after removing the tension on the belt (take the belt off just one pulley) once removed you'll see a pin on the back that hold the tensioner in the right location.
Cheap belts will also cause that...so it might not be the tensioner at all. Belts of uneven thickness will appear tighter then looser at the same point in the run causing the tensioner to move back and forth and causing extra wear on it. Had that on my last Alfa V6. Bought genuine belt...sorted.
Just an idea!
If you can't fix it with a hammer, you've got yourself an electrical problem!