Neurocomputational mechanisms of adaptive learning in social exchanges

Polina M. Vanyukov, Michael Nelson Hallquist, Mauricio Delgado, Katalin Szanto, Alexandre Y. Dombrovski

Research output: Contribution to journalArticle

Abstract

Prior work on prosocial and self-serving behavior in human economic exchanges has shown that counterparts’ high social reputations bias striatal reward signals and elicit cooperation, even when such cooperation is disadvantageous. This phenomenon suggests that the human striatum is modulated by the other’s social value, which is insensitive to the individual’s own choices to cooperate or defect. We tested an alternative hypothesis that, when people learn from their interactions with others, they encode prediction error updates with respect to their own policy. Under this policy update account striatal signals would reflect positive prediction errors when the individual’s choices correctly anticipated not only the counterpart’s cooperation but also defection. We examined behavior in three samples using reinforcement learning and model-free analyses and performed an fMRI study of striatal learning signals. In order to uncover the dynamics of goal-directed learning, we introduced reversals in the counterpart’s behavior and provided counterfactual (would-be) feedback when the individual chose not to engage with the counterpart. Behavioral data and model-derived prediction error maps (in both whole-brain and a priori striatal region of interest analyses) supported the policy update model. Thus, as people continually adjust their rate of cooperation based on experience, their behavior and striatal learning signals reveal a self-centered instrumental process corresponding to reciprocal altruism.

Original languageEnglish (US)
Pages (from-to)985-997
Number of pages13
JournalCognitive, Affective and Behavioral Neuroscience
Volume19
Issue number4
DOIs
StatePublished - Aug 15 2019

Fingerprint

Corpus Striatum
Learning
Reversal Learning
Altruism
Social Values
Policy Making
Reward
Economics
Magnetic Resonance Imaging
Brain

All Science Journal Classification (ASJC) codes

  • Cognitive Neuroscience
  • Behavioral Neuroscience

Cite this

Vanyukov, Polina M. ; Hallquist, Michael Nelson ; Delgado, Mauricio ; Szanto, Katalin ; Dombrovski, Alexandre Y. / Neurocomputational mechanisms of adaptive learning in social exchanges. In: Cognitive, Affective and Behavioral Neuroscience. 2019 ; Vol. 19, No. 4. pp. 985-997.
@article{44c63ebeb4c64fd6bb39900f38f57ebb,
title = "Neurocomputational mechanisms of adaptive learning in social exchanges",
abstract = "Prior work on prosocial and self-serving behavior in human economic exchanges has shown that counterparts’ high social reputations bias striatal reward signals and elicit cooperation, even when such cooperation is disadvantageous. This phenomenon suggests that the human striatum is modulated by the other’s social value, which is insensitive to the individual’s own choices to cooperate or defect. We tested an alternative hypothesis that, when people learn from their interactions with others, they encode prediction error updates with respect to their own policy. Under this policy update account striatal signals would reflect positive prediction errors when the individual’s choices correctly anticipated not only the counterpart’s cooperation but also defection. We examined behavior in three samples using reinforcement learning and model-free analyses and performed an fMRI study of striatal learning signals. In order to uncover the dynamics of goal-directed learning, we introduced reversals in the counterpart’s behavior and provided counterfactual (would-be) feedback when the individual chose not to engage with the counterpart. Behavioral data and model-derived prediction error maps (in both whole-brain and a priori striatal region of interest analyses) supported the policy update model. Thus, as people continually adjust their rate of cooperation based on experience, their behavior and striatal learning signals reveal a self-centered instrumental process corresponding to reciprocal altruism.",
author = "Vanyukov, {Polina M.} and Hallquist, {Michael Nelson} and Mauricio Delgado and Katalin Szanto and Dombrovski, {Alexandre Y.}",
year = "2019",
month = "8",
day = "15",
doi = "10.3758/s13415-019-00697-0",
language = "English (US)",
volume = "19",
pages = "985--997",
journal = "Cognitive, Affective and Behavioral Neuroscience",
issn = "1530-7026",
publisher = "Springer New York",
number = "4",

}

Neurocomputational mechanisms of adaptive learning in social exchanges. / Vanyukov, Polina M.; Hallquist, Michael Nelson; Delgado, Mauricio; Szanto, Katalin; Dombrovski, Alexandre Y.

In: Cognitive, Affective and Behavioral Neuroscience, Vol. 19, No. 4, 15.08.2019, p. 985-997.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Neurocomputational mechanisms of adaptive learning in social exchanges

AU - Vanyukov, Polina M.

AU - Hallquist, Michael Nelson

AU - Delgado, Mauricio

AU - Szanto, Katalin

AU - Dombrovski, Alexandre Y.

PY - 2019/8/15

Y1 - 2019/8/15

N2 - Prior work on prosocial and self-serving behavior in human economic exchanges has shown that counterparts’ high social reputations bias striatal reward signals and elicit cooperation, even when such cooperation is disadvantageous. This phenomenon suggests that the human striatum is modulated by the other’s social value, which is insensitive to the individual’s own choices to cooperate or defect. We tested an alternative hypothesis that, when people learn from their interactions with others, they encode prediction error updates with respect to their own policy. Under this policy update account striatal signals would reflect positive prediction errors when the individual’s choices correctly anticipated not only the counterpart’s cooperation but also defection. We examined behavior in three samples using reinforcement learning and model-free analyses and performed an fMRI study of striatal learning signals. In order to uncover the dynamics of goal-directed learning, we introduced reversals in the counterpart’s behavior and provided counterfactual (would-be) feedback when the individual chose not to engage with the counterpart. Behavioral data and model-derived prediction error maps (in both whole-brain and a priori striatal region of interest analyses) supported the policy update model. Thus, as people continually adjust their rate of cooperation based on experience, their behavior and striatal learning signals reveal a self-centered instrumental process corresponding to reciprocal altruism.

AB - Prior work on prosocial and self-serving behavior in human economic exchanges has shown that counterparts’ high social reputations bias striatal reward signals and elicit cooperation, even when such cooperation is disadvantageous. This phenomenon suggests that the human striatum is modulated by the other’s social value, which is insensitive to the individual’s own choices to cooperate or defect. We tested an alternative hypothesis that, when people learn from their interactions with others, they encode prediction error updates with respect to their own policy. Under this policy update account striatal signals would reflect positive prediction errors when the individual’s choices correctly anticipated not only the counterpart’s cooperation but also defection. We examined behavior in three samples using reinforcement learning and model-free analyses and performed an fMRI study of striatal learning signals. In order to uncover the dynamics of goal-directed learning, we introduced reversals in the counterpart’s behavior and provided counterfactual (would-be) feedback when the individual chose not to engage with the counterpart. Behavioral data and model-derived prediction error maps (in both whole-brain and a priori striatal region of interest analyses) supported the policy update model. Thus, as people continually adjust their rate of cooperation based on experience, their behavior and striatal learning signals reveal a self-centered instrumental process corresponding to reciprocal altruism.

UR - http://www.scopus.com/inward/record.url?scp=85061508340&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061508340&partnerID=8YFLogxK

U2 - 10.3758/s13415-019-00697-0

DO - 10.3758/s13415-019-00697-0

M3 - Article

C2 - 30756349

AN - SCOPUS:85061508340

VL - 19

SP - 985

EP - 997

JO - Cognitive, Affective and Behavioral Neuroscience

JF - Cognitive, Affective and Behavioral Neuroscience

SN - 1530-7026

IS - 4

ER -