An energy harvesting transmitter with an inefficient energy storage device, i.e., battery or capacitor, is considered, where a fraction of the stored energy is lost in the process. An optimal transmission policy maximizing the average rate within a finite time horizon is found for a given energy harvesting scenario. In contrast to previous results for optimal power allocations with energy harvesting transmitters, it is observed that storage losses encourage instantaneous consumption of harvested energy without storage for moderate harvest rates. It is also shown that this policy converges to the known constant power transmission policies when storage efficiency goes to unity. Based on the optimal offline policy, an online policy based on constant charging and discharging thresholds is proposed. The performance of the optimal offline and proposed online algorithms are simulated along with naive transmission policies, and the behavior of the algorithms for various storage efficiencies is discussed.