Although streaming video at a higher bitrate (resolution) can lead to better Quality of Experience (QoE), a larger amount of data will have to be downloaded and processed on smartphones and thus consuming more energy. On a moving bus where the wireless signal is weak, more energy will have to be spent on maintaining high bitrate video streaming than at a static environment such as at home or a cafe where the wireless signal is strong. On the other hand, the user perceived QoE does not increase too much by watching high bitrate videos in a vibrating environment (i.e., a moving vehicle), because the perception of video quality is affected by the environment such as the vibration or shaking on a moving bus. To address this problem, we propose to save energy by considering the context (environment) of video streaming. To model the impact of context, we exploit the embedded sensors (e.g., accelerometer) in smartphones to record the vibration level during video streaming. Based on quality assessment experiments, we collect traces and model the impacts of video bitrate and vibration level on QoE, and model the impacts of video bitrate and signal strength on power consumption. Based on the QoE model and the power model, we formulate the energy-aware and context-aware video streaming problem as an optimization problem. We present an optimal algorithm which can maximize QoE and minimize energy. Since the optimal algorithm requires perfect knowledge of future tasks, we further propose an online bitrate selection algorithm. Through real measurements and trace-driven simulations, we demonstrate that the proposed algorithm can significantly outperform existing approaches when considering both energy and QoE.