In this paper, a two-hop channel is considered with energy harvesting transmitter nodes. In particular, the offline throughput maximization problem is solved for a constant power relay, and a relay with one energy arrival, in both cases assuming a finite buffer is available at the relay for temporarily storing data received from the source. The focus is on assessing the impact of this data buffer at the relay on optimal transmission policies. The solution is found indirectly, by first assuming that the relay has an infinite size buffer, and then proving that an optimal policy exists that does not require any data buffer at the relay, thus solving the problem regardless of the data buffer size at the relay. Numerical results demonstrate that the proposed solution performs significantly better than naïve policies, and a constant relay rate limits the average throughput as the peak energy harvest rate for the source increases.