Rapid advancements of sensing and mobile technology provide an unprecedented opportunity to empower smart and connected healthcare. Realizing the full potential of connected care depends, however, to a great extent on the capability of data analytics. Our previous study proposed a next-generation mobile health system, namely, the Internet of Heart (IoH). The IoH embeds patients into a dynamic network, where the distance between network nodes is determined by the dissimilarity of patients' conditions. Dynamics of the network reveal the change of clinical status of patients. However, it poses a great challenge for real-time recognition of disease patterns when a considerably large number of patients are involved in the IoH. In this present investigation, we develop a novel scheme to optimize the network in a parallel, distributed manner, thereby improving the efficiency of computation. First, a stochastic gradient descent approach is designed to embed patients with similar conditions into a local network. Second, local networks are optimally pieced together to obtain a global network. As opposed to directly embed all patients into one network, the proposed scheme distributes the network optimization into multiple processors for parallel computing. This, in turn, enables the IoH to handle large amount of patients and timely recognize disease patterns in the early stage. Experimental results demonstrated the effectiveness of the proposed scheme, e.g., it achieves 80-fold faster than conventional algorithms for optimizing a network with 20000 patients. The developed scheme is effective and efficient for realizing smart connected healthcare in large-scale IoH contexts.