The objective of this paper is to estimate millimeter wave (mmWave) channels based on the magnitude of the received complex measurements. This is motivated by the observation that in practice, a random carrier frequency offset usually occurs for each transmission and corrupts the phase of the measurement. The problem is cast into a non-convex optimization framework, and solved through a thresholded Wirtinger flow (TWF) based approach. Due to the complex signals involved in the problem, and the constraints imposed by phase shifters, existing design and analysis of TWF are not directly applicable. To overcome these challenges, a structured design of the combiner (precoder) is investigated, which naturally leads to a novel initialization scheme for the TWF algorithm. Simulation results indicate that the proposed approach can successfully recover multiple paths in an mmWave channel and the required number of measurements scales mildly with the dimension of the array size.