Let Ω ⊂ ℝd, d ≥ 1, be a bounded domain with piecewise smooth boundary ∂Ω and let U be an open subset of a Banach space Y. Motivated by questions in "Uncertainty Quantification," we consider a parametric family P = (Py)y∈U of uniformly strongly elliptic, second order partial differential operators Py on Ω. We allow jump discontinuities in the coefficients. We establish a regularity result for the solution u: Ω x U → ℝ of the parametric, elliptic boundary value/transmission problem Py u y = fy, y ∈ U, with mixed Dirichlet-Neumann boundary conditions in the case when the boundary and the interface are smooth and in the general case for d = 2. Our regularity and well-posedness results are formulated in a scale of broken weighted Sobolev spaces K̂ a+1m+1 (Ω) of Babuška-Kondrat'ev type in Ω, possibly augmented by some locally constant functions. This implies that the parametric, elliptic PDEs (Py)y∈U admit a shift theorem that is uniform in the parameter y ∈ U. In turn, this then leads to hm-quasi-optimal rates of convergence (i. e., algebraic orders of convergence) for the Galerkin approximations of the solution u, where the approximation spaces are defined using the "polynomial chaos expansion" of u with respect to a suitable family of tensorized Lagrange polynomials, following the method developed by Cohen, Devore, and Schwab (2010).