This research presents a new method to localize an aircraft without GPS using fixed, generic landmarks observed from an optical sensor. Computer vision object detectors are trained to detect abundant generic landmarks referred as buildings, fields, trees, and road intersections from aerial perspectives. Various landmark attributes and spatial relationships to other landmarks are used to help associate observed landmarks with reference landmarks, which are processed offline before a flight. During a flight, the aircraft navigates with attitude, heading, airspeed, and altitude measurements and obtains measurement corrections by processing aerial photos with similar generic landmark detection techniques. Landmark coordinates extracted from the aircraft’s camera images and inertial measurements are combined into an unscented Kalman filter to obtain an estimate of the aircraft’s position and wind velocities. The objective is to achieve practical navigation performance using available autopilot hardware and a downward pointing camera. The method is demonstrated with an airplane simulation and high resolution orthoimagery. The simulations indicate feasible localization results for outdoor visual flight conditions. Finally, real world flight test results demonstrate localization accuracy within two feet for a quad-rotor flying in an indoor lab at an altitude of ten feet.