This paper derives analytic expressions for both the mean and variance of battery state of charge (SOC) estimation error, assuming a least squares estimation law. The paper examines three sources of estimation error, namely: (i) voltage measurement errors (both bias and noise), (ii) current measurement bias, and (iii) mismatch between the order of the battery model used for estimation and the true order of the battery's dynamics. There is already a rich literature on quantifying battery SOC estimation errors for different estimator designs. The novelty of this paper stems from its extensive examination of both the expected SOC estimation bias and noise, for a least squares estimation algorithm, in the presence of three different fundamental sources of these estimation errors. We show, both analytically and using Monte Carlo simulation, that under reasonable operating conditions, the expected bias in SOC estimation for lithium-ion batteries is dominant compared to the expected estimation variance. This leads to the important insight that quantifying SOC estimation variance using Fisher information furnishes overly optimistic predictions of achievable SOC estimation accuracy.