In this paper, Geoffroy et al.  deal with partner choice as a mechanism of maintaining cooperation, and argues that rather than being unequivocally a force towards improved payoffs to everyone through cooperation, partner choice can lead to “over-cooperation” where individuals can evolve to invest so much in cooperation that the costs of cooperating partially or fully negate the benefits from it. This happens when partner choice is consequential and effective, i.e., when interactions are long (so each decision to accept or reject a partner is a bigger stake) and when meeting new partners is frequent when unpaired (so that when one leaves an interaction one can find a new partner quickly). Geoffroy et al.  show that this tendency to select for overcooperation under such regimes can be counteracted if individuals base their acceptance-rejection of partners not just on the partner cooperativeness, but also on their own. By using tools from matching theory in economics, they show that plastic partner choice generates positive assortment between cooperativeness of the partners, and in the extreme case of perfectly assortative pairings, makes the pair the unit of selection, which selects for maximum total payoff.
This study is a nice contribution to the literature that illustrates potential complexities with partner choice as a mechanism for cooperation, including how the proximate mechanisms of partner choice can significantly alter the evolutionary trajectory of cooperation. Modeling choice as a reaction norm that depends on one’s own traits also adds a layer of realism to partner choice theory.
The authors are also to be commended for the revisions they made through the review process. Earlier versions of the model somewhat overstated the tendency for fixed partner choice strategies to lead to over cooperation, missing some of the important features in previous models, notably McNamara et al.  that can counter this tendency. In this version, the authors acknowledge these factors, mainly, mortality during partner choice (which increases the opportunity cost of forgoing a current partner) and also the fact that endogenous distribution of alternative partners (which will tend to be worse than the overall population distribution, because more cooperative types spend more time attached and less cooperative types more time unattached). These two factors can constrain cooperation from “running away” as the authors put it, but the main point of Geoffroy et al.  that plastic choice can create selection against inefficient cooperation stands.
I think the paper will be very stimulating to theoretical and empirical researchers working on partner choice and social behaviors, and happy to recommend it.
 Geoffroy, F., Baumard, N., & Andre, J.-B. (2019). Why cooperation is not running away. bioRxiv, ver. 5 peer-reviewed and recommended by PCI Evol Biol. doi: 10.1101/316117
 McNamara, J. M., Barta, Z., Fromhage, L., & Houston, A. I. (2008). The coevolution of choosiness and cooperation. Nature, 451, 189–192. doi: 10.1038/nature06455
In this revision, the authors have made a number of the changes I and the reviewers asked for and I think the paper has improved as a consequence. But there is still one major issue, which the authors have not adequately addressed. This relates to how the model builds on previous work, in particular McNamara et al. 2008. Reviewer 2 had pointed out in their review that it was unclear why the current model yields different results than McNamara et al, specifically, why McNamara et al do not get runaway in their model (and sometimes get efficient outcomes). The authors in their reply make it sound like this is purely a modeling artifact, b/c McNamara et al constrain cooperation (with linear costs and benefits) to be limited to a finite value. But this is clearly incorrect from Figure 3b of McNamara et al: it is true that for some parameter values the ES value of cooperation effort seems cut off at the boundary, but clearly for others there is an internal ESS that is not at the boundary.
This seemingly little detail about the discussion of McNamara et al made me look at their paper more closely, and realize that their model has two very crucial elements completely missing from the current work: (a) in McNamara et al, agents experience constant mortality across their lives (and also it’s an overlapping generations model), (b) in McNamara et al, agents’ current accept/reject decisions affects the supply of future partners. Both of these factors will work against run-away choosiness. I think these crucial differences need to be discussed in detail. Currently, they aren’t mentioned at all and lead me — as well as the reviewers— think that there was a much closer semblance between the current work and McNamara et al. Substantively, they also make me less convinced about the bold statements about how “fixed” partner choice will always run away (which I already thought were somewhat overstated).
On to the gritty details: the neglect of mortality in the current model basically amounts to no discounting of future payoffs. If there is a positive probability that I die before I get to choose again (and therefore leave no offspring), I will be more motivated to take a lower current match and get some payoff rather than have no payoff. This should reduce my choosiness threshold. Even if you don’t think about literal mortality, humans (and other animals) clearly value present rewards more than future ones. The authors remove this discounting completely, and therefore create an unrealistic scenario that promotes the run-away dynamics.
The second piece important difference between the current model and McNamara et al is that if you are choosy and reject a match, chances are your next match is going to be worse. This is because the high quality individuals will not have been rejected, so the supply of new partners will be skewed towards lower quality individuals. This is true even in an infinite population, since even if there is always an infinite number of good quality matches, there will be proportionally more bad matches if the good ones are taken, so you’re more likely to encounter the bad quality ones after the initial choice. McNamara et al do take this into account because their life-cycle only pairs individuals that are single (and therefore dismissed or “widowed” or newborn, page 3 of their supplement). The effect of this is also expected to work against run-away choosiness, since rejecting the current partner does not guarantee a better partner in the future (or from a different perspective, the steady state distribution of potential partners is going to be worse quality overall than the underlying distribution of quality). It is not clear whether this effect is taken into account in the agent-based model, since there isn’t really an adequate description of it, or the code, but it seems clear at least the first effect is not there (since there is no mortality parameter anywhere).
In short, both of these effects would likely counteract the run-away dynamics of choosiness, and probably explain why McNamara et al don’t get such a result. It is true that McNamara et al also don’t always get efficient outcomes, but neither do the authors in their model. Rather, their main conceptual argument is only true quantitatively in the “frictionless” market regime (which as per above is actually quite a bit more frictionless than they make it out to be): there, runaway dynamics happen with fixed choice, but efficiency (or something close to it) obtains with plastic choice. Away from that regime, both conclusions are altered depending on parameters, and as I argue above, trying to make a “fair” comparison with McNamara et al would likely strengthen this conclusion.
In conclusion, I still like the contribution of the paper, and believe that the plastic choice aspect is a welcome addition to the discussion of partner choice in the literature. But I think at a minimum the issues above need to be explicitly acknowledged and discussed. As it is, I think the ms creates a misleading view of how it differs from McNamara et al. but more importantly, it also glosses over some biologically/socially important aspects of partner choice that they take into account and the current paper doesn’t. Ideally, one would take into account these two effects as well, but that might involve some substantial new modeling work.
Minor comments: Incidentally, the authors talk (in their SI) of two potential but equivalent implementations of quality variation: one where it doesn’t change through life, and one it “resets” after every interaction. But in fact the two would yield differences, since the latter implementation would not necessarily have this selection effect for the single-pool.
The comment on Akcay and Van Cleve 2012 is also slightly off: what we showed there was in fact that you can build proximate mechanisms (e.g., families of preference functions) that can yield efficiency (or close to efficiency, see our Figure 2) with very low relatedness. So, it is a bit awkward to cite it as showing that one function requires relatedness as contrast to the current paper.
In addition to the modifications described in the attached file, the authors have made the following two modifications.
1) Two references have been added.
 Alger I and Weibull JW. Evolution and Kantian morality. Games and Economic Behavior 98 (2016), 56–67. DOI: 10.1016/j.geb.2016.05.006.
 Alger I and Weibull JW. Homo Moralis—Preference Evolution Under Incomplete Information and Assortative Matching. Econometrica 81 (2013), 2269–2302. DOI: 10.3982/ECTA10637.
2) A paragraph in the Discussion section has been modified:
"Previous work has shown that assortative matching favours the evolution of cooperation [25, 48, 58]. For instance, in kin selection, assortment between relatives drives the evolution of cooperation [57, 83]. To our knowledge, Wilson & Dugatkin  first discussed the consequences of assortative matching for the evolution of socially efficient levels of cooperation. Alger & Weibull [6, 7] have studied the evolution of social preferences, rather than strategies, under assortative matching. However, both analyses did not explicitly model a partner choice strategy, let alone the evolution of this strategy, but merely assumed that assortment occurs in one way or another. In contrast, here, we have studied the joint evolution of choosiness and cooperation, showing how a positive assortative matching can emerge from a simple partner choice mechanism."
First off, I apologize to the authors that the review process took somewhat longer than anticipated, and thank them for their patience. I have now obtained two detailed reviews from experts in social evolution theory, and evaluated the paper myself. Both reviewers find the paper to be an interesting and potentially valuable contribution, but raise some important issues that the authors need to address. Based on my own reading, I agree with these comments, and have some concerns myself. I found the paper to be a very interesting read, and agree that it introduces a nice twist to the partner choice story for the evolution of cooperation. That said, I think the paper needs a revision in order to better reflect connections to previous work, the specific contribution of this paper, and also explain its results more transparently.
One issue that Reviewer 2* points out, and I agree, that considering quantitative evolution of cooperation (with diminishing benefits and/or increasing costs of cooperation) per se is a relatively weak argument for novelty. I don’t think this is really the authors’ intention, but at times the introduction does read like that is the main contribution. Even the more specific point of focusing on efficiency has been considered before, not least the McNamara et al. paper that looks at it in the precise setting of this paper, as do Akcay and Van Cleve 2012 (where it’s in the title). It’s true that continuous investment models are outnumbered by discrete strategy papers, but if the framing is “most papers ignore these issues,” I think in fairness the papers that don’t ignore them deserve mention in the framing, and the more specific contribution needs to be fleshed out. I think reviewer 2 in particular points to a helpful framing (their second paragraph), where the real contribution is investigating the effects of different mechanisms of choice on the level of cooperation.
I also find the main narrative of the paper to be somewhat misleading, or at least, incomplete. In particular, the abstract, introduction, and discussion are all state that the “base model” with non-plastic choice generates so much cooperation that it cancels out the benefits. This result is stated unconditionally in most places in the paper (e.g., abstract, l. 21-23, introduction l. 100-102, discussion l. 365-377). But Fig 2 shows that the zero-benefit cooperation result only obtains in the relatively large phenotypic variance limit. At least for medium and high market fluidity, there exists an intermediate level of phenotypic variation where ESS choosiness and cooperation seemingly reach efficient levels. It is true that the variance required to obtain near efficient cooperation is low in absolute value, so perhaps the authors would argue that biologically relevant levels of variance is high relative to these values, but this would need to be spelled out explicitly. As it is, the statements in the lines referenced above strictly speaking false, because they sound like there is an impossibility result, whereas all that there is cooperation evolving to one level or another depending on the parameter set. More generally, it looks like there is quantitative, rather than qualitative differences between the models in how much cooperation can be sustained in equilibrium. So I think this quantitative variation needs to be addressed in a more systematic fashion.
Reviewer 1 and 2 both make a number of useful suggestions in both elucidating the results in more depth and also making the connections to the prior literature more explicit. I think the authors would do well to heed these suggestions. One question I had while reading the model was what would happen if choice was also noisy: after all, in a scenario where cooperation phenotype is noisy, choice being so as well (through errors in perception or execution of choice). In the fixed choice scenario, I can see this further relaxing the constraints to the evolution of cooperation, though intuition is a dangerous guide there. In any case, I feel like this is a relatively straightforward extension of the analysis that would also make the story in this paper more complete.
In sum, I think this has the makings of a nice paper and I think the mechanism proposed here is a nice twist on classic models of partner choice. At the same time, I hope the authors will take these comments at heart and improve the presentation and analysis of their results.
This paper studies partner choice as a mechanism for the evolution of cooperation. This is not a new endeavour, but the model introduces different variations and perspectives that can potentially enrich the literature on cooperation and partner choice.
The model starts by assuming individuals endowed with a trait (xi, yi ), where xi is the amount of cooperation and yi is the minimum level of cooperation that individuals are willing to accept in a partner. When an interaction is accepted, individuals interact for an amount of time 1/\tau. Interactions happen at a rate \beta. A payoff S per unit of time is given to solitary individuals. It is assumed that cooperation has diminishing returns, specifically, the payoff for i when interacting with j is \Pi(xi, xj)= xj - c xi^2. Naturally, there is a socially optimal level of cooperation, which is not full cooperation. The possibility of having a sub-optimal level of cooperation will play a key role in the narrative of the paper.
Under this setup the paper proposes two models, i) hard-wired choosiness and ii) plastic choosiness.
For hard-wired choosiness the authors use the framework of adaptive dynamics to numerically compute exact trajectories. The main result is that cooperation evolves beyond what is socially optimal at the group level. This is interesting but not very surprising because selection is acting on individuals, not collectives. I would certainly not call this paradoxical. I find the choice of language also a little confusing, because choosiness is changing in this model and not fixed.
The second model posits that the "paradox" outlined earlier is solved when choosiness is plastic. The value xi determines the expected cooperation, with the actual value following a normal distribution around xi. The plasticity arises from the choosiness not being a single value, but a function y(x), of the cooperation level. It is hard to derive an equilibrium y(), but it can be shown (from a result in matching theory) that y will be monotonously increasing for the chosen payoff function. This setup is studied via simulations, with the main finding being that the afore-mentioned paradox is resolved -- cooperation stabilises closer to social optimum.
I like the idea of this paper but I have a few basic concerns:
I wonder if this is the simplest model to get to the main result. I find the need for S, \tau and interactions through time not core to the argument. Why do we need them?
I would have liked to have a more in depth study of the first model, using adaptive dynamics. How do the results depend on the choice of the payoff function? I find the efficiency argument is interesting, but rather underdeveloped. Is it only due to diminishing returns to cooperation?
How can we be sure that the results of the model with plasticity do not arise from noise alone? I understand that the evolution of functional traits is hard to tackle analytically, but numerical results are sometimes attainable, see for example reference 2. I would love to see a more robust theoretical underpinning for the simulations, or in lack of that, a more detailed analysis of the simulations results. Since the evolving traits are x and a function of x, one wonders why do the results differ from the first model when x and y vary freely. Under reasonable assumptions with minimal noise I would expect these to be very similar.
I would hope the authors can revise the paper to address this suggestions or queries.
Other minor details:
The abstract talks about ESS, but there is not ESS calculation, only adaptive dynamics. I would not expect these to be equivalent in general.
Other studies have combined simulations with adaptive dynamics to look at the coevolution of traits (e.g., Reference 1). How simulations match predictions -- in the same setup, is a useful way to produce robust results. How does this paper relate to those?
"It takes grouping and cooperation to get sociality." Journal of theoretical biology 264.4 (2010): 1240-1253.
"The adaptive dynamics of function-valued traits." Journal of theoretical Biology 241.2 (2006): 370-389.