We present a modular approach toward poly(vinylidene fluoride) based ferroelectric polymers with high dielectric constants and energy densities. This strategy is based on a two-step reaction including the co-polymerization of vinylidene fluoride (VDF) and chlorotrifluoroethylene (CTFE) and a subsequent hydrogenation reaction. Due to the similar reactivity of VDF and CTFE in free radical polymerization and quantitative yield of dechlorination reaction, the chemical compositions of the resulting terpolymers can be precisely controlled, leading to tunable Curie temperatures and dielectric constants. A library of the ferroelectric polymers with dielectric constants varying from 11 to 50 measured at 1 kHz and room temperature has been prepared. The structural characteristics including microstructure, chain conformation, and crystallinity of the polymers have been carefully elucidated as a function of the chemical composition by 1H and 19F NMR, Fourier transform infrared spectroscopy, and wide angle X-ray diffraction. The influence of the polymer compositions on thermal transitions and dielectric constants has also been investigated.