A facile route to synthesis silver nanoparticles into polyvinylpyrrolidone (PVP) or silica (SiO2) matrices using sol-gel method has been developed. The influence of matrices on the size and optical properties of silver nanoparticles were studied. Because the particle size increases at different rate with silver nitrate (AgNO3) concentration in different matrices, there is an up-limit for AgNO3 concentration in each given matrix to keep the particle size below 100 nm. This limit is 8% in silica matrix and can be as high as 50% for PVP matrix. The effects of aging time on the size of nanoparticle and optical absorption spectrum were also studied. The absorption peak occurs at 425 in PVP matrix and at 442 in SiO2 matrix. Based on Scanning Electron Microscopy (SEM) and Atomic Force Microscopy (AFM) images, Ag nanoparticles are well-dispersed into the film and the mean particle size of the nanosilver increases with the AgNO3 content at different rate in different matrix. For the same 1.6% AgNO3 concentration, the mean particle size is 65 nm in SiO2 matrix but only 10 nm in PVP matrix.