A mixed strategy in game theory involves players randomizing over possible moves to prevent opponents from predicting their actions, thereby maximizing their own expected payoff. It is particularly useful in games where no pure strategy Nash equilibrium exists, ensuring that players remain unpredictable and potentially gain a strategic advantage.