Probabilistic finite state automata (PFSA) are constructed from symbol sequences for modeling the behavior of dynamical systems. This paper presents construction of finite history automata from symbol sequences; such automata, called D-Markov machines, are structurally simple and computationally efficient. The construction procedure is based on: (i) state splitting that generates symbol blocks of different lengths according to their relative importance; and (ii) state merging that assimilates histories from symbol blocks leading to the same symbolic behavior. A metric on probability distribution of symbol blocks is introduced for trade-off between modeling performance and the number of PFSA states. These algorithms have been tested by two examples.