Recent popularization of personal hand-held mobile devices makes it important to characterize the mobility pattern of mobile device users, so as to accurately predict user mobility in the future. Currently, the user mobility pattern is mostly characterized at a coarse-grained level, in the form of transition among wireless Access Points (APs). There is limited research effort on the fine-grained characterization of geographical user movement. In this paper, we present a novel approach to characterize the steady-state and transient-state user mobility behaviors at a fine-grained level, based on the Hidden Markov Model (HMM) formulation of user mobility. By applying our approach on both realistic mobility traces and synthetic mobility scenarios, we show that our approach is effective in characterizing user mobility pattern and making accurate mobility prediction. We also experimentally demonstrate that fine-grained user mobility knowledge is more effective to improve the performance of a variety of mobile computing applications.