Recently, biologically inspired vision systems have been the focus of intense research effort to emulate the high energy-efficiency, performance and robustness of mammalian vision systems. However, previous vision accelerators have only focused on speeding up computationally intense portions of the system without exploiting effects seen in the human brain that demonstrate the task influence in the vision mechanism. In this paper, we propose a task-oriented two-level vision system which is composed of Saliency and SURF. To the best of our knowledge, our design is the first embedded system that utilizes task influence in the computation of visual attention and recognition. As a result, we show that the new system can achieve at most 12.75% accuracy improvement while saving 25% computation work.