High and reproducible dielectric breakdown strength is essential requirement for high energy pulse power capacitors. It is important to understand which features of a dielectric are instrumental in determining breakdown strength. Modeling of the dielectric breakdown complements experimental work in achieving deeper insights into dielectric breakdown and helps to identify parameters controlling breakdown strength for a given dielectric. We have performed Monte Carlo modeling of breakdown strength in ceramics, polymers, and layered organic/inorganic composites. In the case of ceramics, the effect of uniform and nonuniform porosity on the breakdown strength indicated that both the level and size of porosity influence dielectric strength. Furthermore, porosity has stronger influence on the breakdown strength as the pore size-to-dielectric thickness ratio becomes larger. It was also determined that the prediction of the model agrees well with experimentally observed area dependence of dielectric breakdown of commercial capacitors. In the case of semicrystalline polymers, the modeling work strongly suggests positive contribution to the breakdown strength from the interfaces between the crystalline and amorphous phases. This conclusion was reached through the comparison of Weibull modulus obtained from the experimental breakdown strength measurements of biaxially oriented polypropylene capacitors with that of Monte Carlo modeling of the same dielectrics. Only in the case of interfaces dominating the breakdown strength there was a good agreement between experimental and modeled Weibull modulus. The modeling work also indicate that unlike ceramics for which infinite Weibull modulus is theoretically achievable for the defect-free dielectric, in the case of semicrystalline dielectrics the Weibull modulus is finite in all cases, and is controlled by polymer microstructure and capacitor geometry. In the case of layered ceramic - polymer composites we focused on understanding tree propagation as a function of the dielectric contrast between the dielectrics and the placement of a single high K ceramic layer within the polymer. Both dielectric contrast between the layers and placement of the layer turned out to be important in determining the time required for sample destruction. Interestingly, these effects are observed in the electrostatic modeling without introduction of defects and space charges at the interfaces. A comparison of breakdown in heterogeneous ceramics, polymers, and composites indicates a great diversity and richness of observed behavior. In particular, both the beneficial and detrimental effects of heterogeneity were deduced from the modeling.