Sharing personal information and documents is pervasive in Web 2.0 environments, which creates the need for properly controlling shared data. Most existing authorization and policy management systems are for organizational use by IT professionals. Average Web users, however, do not have the sophistication to specify and maintain privacy policies for their shared content. In this paper, we aim to utilize personal and social annotations to develop automatic tools for managing content sharing, and demonstrate a new application of social annotations in access control. We use annotation data to predict privacy preferences of users and automatically derive policies for shared content. We carry out a series of user studies to evaluate the accuracy of our predicted techniques. We also perform extensive analysis on static and dynamic approaches of analyzing semantic similarities of tags, which is of independent interest. Our analysis gives encouraging results on the feasibility of using annotations for privacy management in Web 2.0.