Recursive system identification is the heart of many industrial applications that include tracking systems, time-varying behaviors, and fault-detection systems. Two significant challenges in the recursive system identification are a prior specification of the model order and computation complexity of recursive system identification. Further, when controller design ensues the model identification, it is desirable to compute the lowest order model that explains the input-output data with reasonable computations. Therefore this paper presents a Recursive Parsimonious System Identification (RPSI) algorithm for recursively identifying the lowest order model from measurements. To simplify the computations the method uses the recently developed concept of an atomic norm. The main advantage of the method is that, it provides a way to recursively estimate a lowest order model without the use of Riccati recursions on covariance matrices and other such computations. Also, it does not require any assumptions on the model order as with other methods in the literature. The proposed method is illustrated on two examples-first a simulation and second an industrial example from the cement industry.