Team decision making under stress involving multiple contexts is an extremely challenging issue faced by various real world application domains. This research is targeted at coupling cognitive agent technology and human-centered teamwork to address the informational challenges associated with Command and Control (C2) teams in contemporary military operations. Two sets of experiments, each with various settings of context switching frequencies and tasking complexities, were conducted. To ensure that the human subjects were familiar with the C2 context, they were selected from US Army ROTC (Reserve Officer Training Corps) students. Experiments on C2 teams that involve human subjects only were conducted first. We observed the decision making behavior of human subjects and incorporated expert behaviors into R-CAST - an agent architecture built upon a naturalistic decision making model that captures how domain experts make decisions based on experiences and situational similarity recognition. We then conducted another set of experiments with R-CAST agents as teammates and decision aids for human subjects. The results show that RPD-enabled agents can significantly improve the tasking capacity of C2 teams in multi-context decision making under stress. It also suggests that higher demand situations require more competent teammates.