In this paper, we study the optimal energy management policy of an energy harvesting transmitter by taking both battery degradation and finite battery constraints into consideration. We consider a scenario where the sensor is able to harvest energy from the ambient environment and use it to power its transmission. The harvested energy can be used for transmission immediately without entering the equipped battery, or charged into the battery and discharged later for transmission. When the battery is charged or discharged, a cost will be incurred to account for its impact on battery degradation. We impose a long-term average cost constraint on the battery, which is translated to the average number of charge/discharge operations per unit time. At the same time, we assume the capacity of the battery is finite, and the total amount of energy stored in the battery cannot exceed its capacity. Our objective is to develop an online energy management policy to maximize the long-term average throughput of the transmitter under both the battery usage constraint and finite battery constraint. We propose an energy-aware adaptive transmission policy, which is a modified version of the optimal policy for the infinite battery case. Our analysis indicates that the energy-aware adaptive transmission policy is asymptotically optimal when the battery size is sufficiently large. Simulation results corroborate the theoretical analysis.