In real-time video mining applications it is desirable to extract information about human subjects, such as gender, ethnicity, and age, from grayscale frontal face images. Many algorithms have been developed in the Machine Learning, Statistical Data Mining, and Pattern Classification communities that perform such tasks with remarkable accuracy. Many of these algorithms, however, when implemented in software, suffer poor frame rates due to the amount and complexity of the computation involved. This paper presents an FPGA friendly implementation of a Gaussian Radial Basis SVM well suited to classification of grayscale images. We identify a novel optimization of the SVM formulation that dramatically reduces the computational inefficiency of the algorithm. The implementation achieves 88.6% detection accuracy in gender classification which is to the same degree of accuracy of software implementations using the same classification mechanism.