This paper describes an aggregate imaging based Discrete Element Modeling (DEM) methodology successfully employed to model both clean and coal dust fouled railroad ballast behavior. Laboratory direct shear box tests were conducted on granite type clean and fouled ballast aggregate samples. The size, shape and angularity properties of the aggregate particles were obtained through image analysis. Accordingly, direct shear box DEM simulations were conducted for different aggregate contact friction conditions. Both the clean ballast behavior and the heavily fouled case of coal dust filling completely all the voids in ballast were adequately modeled using the DEM approach by assigning a lower contact friction angle as the controlling discrete element model parameter in the latter case. In accordance with the direct shear test results, much lower shear stress-shear deformation curves, i.e., lower shear strengths, were predicted the coal dust fouled case using the DEM simulations. Copyright ASCE 2009.