Content Delivery Networks (CDN) design entails the placement of server replicas to bring the content close to the users, together with an efficient and content aware request routing. In this paper we address the problem of dynamic replica placement to account for users demand variability while optimizing the costs paid by a CDN provider and the overall performance of the distributed replica servers architecture. We formulate the dynamic replica placement problem as a Semi Markov Decision Process accounting for the traffic conditions, the users level of satisfaction, as well as the costs paid to install, maintain or remove a replica from a site. The proposed model applies to general network topologies and considers realistic constraints on network and servers capacity. The optimal strategy derived by means of the decision model provides the ground for designing a centralized heuristic and is used as a benchmark for the heuristic evaluation. Simulation results show that the proposed heuristic has a behavior close to that of the optimal strategy and achieves very good performance in terms of low average distance from a user to the serving replica, low average number of replicas and high probability of being able to serve a request.