We introduce a general technique for frequency stability characterization of Fabry-Perot etalons that are being explored for astronomical spectrograph calibration. In our approach a frequency-stabilized laser frequency comb is employed as a reference for a scanning CW laser measurement of the temperature sensitivity of a fiber Fabry-Perot interferometer (FFP). For an in-house constructed, actively stabilized FFP, we observe the thermal sensitivity of a resonance mode at 1319 nm of ∼7.4 GHz C-1, which corresponds to a fractional thermal sensitivity of ∼3.2 × 10-5 C-1. We compare these results to a simple model and discuss further the materials construction and stabilization of the FFP. Our measurement technique is one step toward a broad characterization of Fabry-Perot instruments, and this FFP in particular is currently being investigated as a wavelength calibration source in precision radial velocity spectroscopy to discover terrestrial-mass exoplanets.