This paper presents a stochastic control framework for optimizing datacenter power management. The paper focuses on datacenters employing lithium-ion batteries for demand response. The use of batteries for demand response can reduce electricity costs, at the expense of battery degradation. We minimize this degradation using a control policy that takes into account uncertainties in power demand. We perform this optimization using a second-order model of battery charge dynamics, coupled with a physics-based model of battery aging via solid-electrolyte interphase (SEI) growth. To the best of our knowledge, this is the first study that uses battery models capturing diffusion dynamics and nonlinear aging effects, together with a model of demand uncertainty, for datacenter energy management. We formulate this as a stochastic dynamic programming (SDP) problem, where uncertain power demand is modeled as a Markov chain. The resulting control policy keeps grid power within a predefined range while minimizing battery degradation.