Self-interaction is a central problem for the accuracy of density-functional approximations in describing the electronic structure of atoms and molecules. In this work, we discuss the different types of self-interaction errors commonly encountered in density-functional calculations, providing precise definitions for each of them. Based upon these definitions, we derive an orbital-dependent density-functional method, called the Koopmans-compliant approach, which simultaneously corrects the different self-interaction errors, by enforcing piecewise linearity with respect to fractional particle counts and by imposing the correct asymptotic behavior of the one-electron potential in approximate energy functionals. We illustrate the very good performance of this new method in predicting the electronic properties of atoms and molecules, while preserving or improving the prediction of total energies and equilibrium geometries. These results highlight the accuracy and efficiency of Koopmans-compliant functionals as an attractive solution to the self-interaction problem.