I do just wonder if this is all about those who have him and those who do not? I have him but not used him and not in PVP yet so no ideas as to his importance but what I can see is that (as others pointed out) he is quite rare and he has not much else at all going for him other than the passive? So I can see why he works well in PVP and I can see how he works well in grind. If you mess with the passive it ruins both aspects of the game surely? So really it is why have this toon if you remove the only thing he is good for? Or am I missing something?
If there are special talismans or a revised passive on a different character that can help against a Prisoner why not do that as that means he stays as is for everyone. If the future of the game is PVP (not so sure that it is if new content comes out in the shape of new worlds or other new content) then Prisoner (being quite rare) should still be available as he is I think.....or if the consensus id to nerf him then he needs some other boost to what he does on other attacks or passives I think.
I am sure there are other characters that have the limelight in PVP but not much idea as not played it yet - so if Prisoner gets hit then does other strong PVP toons need to come next and if so which ones? Red troll, hellhound, red bat.......?