Deep Spiking neural architectures are becoming increasingly popular tools in complex pattern recognition tasks. However, implementation of such algorithms in conventional CMOS hardware entails huge area and power consumption due to the significant mismatch between the computational units and the corresponding CMOS devices. In this paper, we explore the design of an All-Spin Deep Spiking Neural Network where we demonstrate the mapping of synaptic and neuronal functionalities to domain wall dynamics in ferromagnets. We evaluate the potential advantages offered by such spintronic devices by performing micromagnetic simulations calibrated to experimental results. In order to investigate the benefits of such a spintronic design for large-scale neuromorphic systems, we perform device-circuit-algorithm co-design for a standard digit recognition problem on the MNIST dataset. Results indicate 250 × improvements in energy consumption and 56× improvement in EDP of the spintronic deep network over a baseline CMOS implementation in commercial 45nm technology.