TY - JOUR
T1 - Expectancies in decision making, reinforcement learning, and ventral striatum
AU - Van der Meer, Matthijs A.A.
AU - Redish, A. David
PY - 2010
Y1 - 2010
N2 - Decisions can arise in different ways, such as from a gut feeling, doing what worked last time, or planful deliberation. Different decision-making systems are dissociable behaviorally, map onto distinct brain systems, and have different computational demands. For instance, "model-free" decision strategies use prediction errors to estimate scalar action values from previous experience, while "model-based" strategies leverage internal forward models to generate and evaluate potentially rich outcome expectancies. Animal learning studies indicate that expectancies may arise from different sources, including not only forward models but also Pavlovian associations, and the flexibility with which such representations impact behavior may depend on how they are generated. In the light of these considerations, we review the results of van der Meer and Redish (2009a), who found that ventral striatal neurons that respond to reward delivery can also be activated at other points, notably at a decision point where hippocampal forward representations were also observed. These data suggest the possibility that ventral striatal reward representations contribute to model-based expectancies used in deliberative decision making.
AB - Decisions can arise in different ways, such as from a gut feeling, doing what worked last time, or planful deliberation. Different decision-making systems are dissociable behaviorally, map onto distinct brain systems, and have different computational demands. For instance, "model-free" decision strategies use prediction errors to estimate scalar action values from previous experience, while "model-based" strategies leverage internal forward models to generate and evaluate potentially rich outcome expectancies. Animal learning studies indicate that expectancies may arise from different sources, including not only forward models but also Pavlovian associations, and the flexibility with which such representations impact behavior may depend on how they are generated. In the light of these considerations, we review the results of van der Meer and Redish (2009a), who found that ventral striatal neurons that respond to reward delivery can also be activated at other points, notably at a decision point where hippocampal forward representations were also observed. These data suggest the possibility that ventral striatal reward representations contribute to model-based expectancies used in deliberative decision making.
KW - Actor-critic
KW - Pavlovian-instrumental transfer
KW - Planning
KW - Reinforcement learning
KW - Reward
UR - http://www.scopus.com/inward/record.url?scp=79960218850&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=79960218850&partnerID=8YFLogxK
U2 - 10.3389/neuro.01.006.2010
DO - 10.3389/neuro.01.006.2010
M3 - Review article
C2 - 21221409
AN - SCOPUS:79960218850
SN - 1662-4548
VL - 4
JO - Frontiers in Neuroscience
JF - Frontiers in Neuroscience
IS - MAY
ER -