When and how does mutation-generated variation promote the evolution of cooperation?

Mutation-generated variation in behavior is thought to promote the evolution of cooperation. Here, we study this by distinguishing two effects of mutation in evolutionary games of the finitely repeated Prisoner’s Dilemma in infinite asexual populations. First, we show how cooperation can evolve through the direct effect of mutation, i.e., the fitness impact that individuals experience from interactions with mutants before selection acts upon these mutants. Whereas this direct effect suffices to explain earlier findings, we question its generality because mutational variation usually generates the highest direct fitness impact on unconditional defectors (AllD). We identify special conditions (e.g., intermediate mutation rates) for which cooperation can be favored by an indirect effect of mutation, i.e., the fitness impact that individuals experience from interactions with descendants of mutants. Simulations confirm that AllD-dominated populations can be invaded by cooperative strategies despite the positive direct effect of mutation on AllD. Thus, here the indirect effect of mutation drives the evolution of cooperation. The higher level of cooperation, however, is not achieved by individuals triggering reciprocity (‘genuine cooperation’), but by individuals exploiting the willingness of others to cooperate (‘exploitative cooperation’). Our distinction between direct and indirect effects of mutation provides a new perspective on how mutation-generated variation alters frequency-dependent selection.