Many large scale system blackouts involve relay misoperations. Traditional relay algorithms and settings need to be evaluated under a variety of fault and no-fault system-wide scenarios to better understand the causes for misoperations. New fault diagnosis algorithms also need to be developed to assure improved relaying performance and then evaluated under various scenarios. This paper introduces advanced fault analysis simulation software based on the interactive MATLAB and ATP simulation. The software consists of two major parts, power system simulation and relay algorithm evaluation. The former part can automatically generate thousands of system-wide events at one time and extract the transients for fault studies. The latter part includes the traditional distance relay model and two new advanced fault diagnosis algorithms. The structure of the software enables easy simulation setup for different power system models.