Advanced sensing technologies, driven by the Internet of Things, have caused a sharp increase in data availability within the healthcare system. The newfound availability of data offers an unprecedented opportunity to develop new analytical methods to improve the quality of patient care. Data availability, however, is a double-edged sword. Malicious attacks and data breaches are increasingly seen in the healthcare field, which result in costly disruptions to operations. Adversaries exploit analytic models to infer participation in a dataset or estimate sensitivity attributes about a target patient. This paper is aimed at developing a differentially private gradient-based mechanism and assessing its utility in mitigating the impact of these attack risks within the context of the intensive care units. Experimental results showed that this methodology is capable of greatly reducing the risk of model inversion while retaining model accuracy. Thus, health systems that employ this technique can be given more peace of mind that high-quality services can be delivered in such a way that privacy is preserved.