Done right, a Turing Test is a difficult game for a bot
The way people informally talk about "passing a Turing test" describes a weak test, but the imitation game that Turing described isn't weak if the human players are skilled. It's not just "chatting like a human.” It's more like playing the Werewolf party game. You could learn to be better at the imitation game and play to win, and a bot that could defeat you would be far more impressive than today’s bots.
Alice and Bob don’t know each other, but they want to communicate. A bot is pretending to be Bob. How can Alice pick out Bob over the bot?
It depends on what shared secrets they have. If they cheated and picked a shared password, it would be easy. Or if they went to the same school and they figured that out in conversation, then they could talk about what happened at school and that would be hard to beat.
To make this a fair test, we need to assume Alice and Bob don't know each other and don't cheat. But if they didn’t even speak the same language, they would find it hard to do anything. There needs to be some shared culture. How much?
Suppose there is a pool of players who come from the same country and don’t know each other, but they have studied previous games, learned the same basic strategies, and practiced.
One strategy is to find a subject in common that they don't think the bot is good at. The first thing each player should do is propose subjects that they think they could do better than the bot at discussing, and then decide on one that they’re both good at. If they're both mathematicians then talk about math, or they're both cooks, talk about cooking. They could discuss previous games too, since they always have that in common.
The bot would have to act in a similar way to appear to be a skilled player. That probably means proposing subjects that the human players think it would be bad at, but it’s actually good at. To avoid having known weaknesses, it would need to be able to talk about previous games too, and somehow have become better at any subjects that it previously did badly at.
If the players are reasonably skilled and they’re playing to win, it will be difficult for the team building the bot to compete. It has to be good at conversation for many possible subjects and not have weaknesses known to the human players.