The use of classification techniques for machine health monitoring and fault diagnosis has been popular in recent years. System response in form of time series data can be used to identify type of defect, severity of defect etc. However, a central issue with time series classification is that of identifying appropriate features for classification. In this paper, we explore a new feature set based on a delay differential equations (DDEs). DDEs have been used recently for extracting features for classification but have never been used to classify system responses. The Duffing oscillator and Van der Pol-Duffing (VDP-D) oscillator are used as dynamic systems, and the responses are classified into self-similar groups. Responses with the same period should belong to the same group. Misclassification rate is used as an indicator of the efficacy of the feature set. The proposed feature set is compared to a statistical feature set, a power spectral coefficient feature set and a wavelet coefficient feature set. In work described in this paper, a density estimation algorithm called DBSCAN is used as the classification algorithm. The proposed DDE-based feature set is found to be significantly better than the other feature sets for the classifying responses generated by the Duffing system. The wavelet and the power spectral coefficient data sets are not found to be significantly better than the statistical feature set for the Duffing system. None of the feature sets tested are discerning enough on the VDP-D system.