Curious.. did the enemy start with The Prisoner.. or did your Anubis die and get revived?
Both of these can mean passives are disabled permanently for the PLAYER.. but NOT the AI. This especially true if Anubis died and was revived.. no passives work for the PLAYER after that.