As a lower-level arena player (I've never been above 2400), just one set is enough to determine the match. If one side has them and the other doesn't, the have-nots will have a hard time to even make a dent in the other team.
As someone who hasn't managed to get a single talisman (let a lone a whole set) it's not even worth trying. Right now, I simply skip all teams featuring either A) the prisoner or B) multiple yellow chars. High risk of a promordial set, therefore no chance of winning.