In and out of equilibrium I: Evolution of strategies in repeated games with discounting

In the repeated prisoner's dilemma there is no strategy that is evolutionarily stable, and a profusion of neutrally stable ones. But how stable is neutrally stable? We show that in repeated games with large enough continuation probabilities, where the stage game is characterized by a conflict between individual and collective interests, there is always a neutral mutant that can drift into a population that is playing an equilibrium, and create a selective advantage for a second mutant. The existence of stepping stone paths out of any equilibrium determines the dynamics in finite populations playing the repeated prisoner's dilemma.