As a proactive means of preventing struck-by accidents in construction, many studies have presented proximity monitoring applications using wireless sensors (e.g., RFID, UWB, and GPS) or computer vision methods. Most prior research has emphasized proximity detection rather than prediction. However, prediction can be more effective and important for contact-driven accident prevention, particularly given that the sooner workers (e.g., equipment operators and workers on foot) are informed of their proximity to each other, the more likely they are to avoid the impending collision. In earlier studies, the authors presented a trajectory prediction method leveraging a deep neural network to examine the feasibility of proximity prediction in real-world applications. In this study, we enhance the existing trajectory prediction accuracy. Specifically, we improve the trajectory prediction model by tuning its pre-trained weight parameters with construction data. Moreover, inherent movement-driven post-processing algorithm is developed to refine the trajectory prediction of a target in accordance with its inherent movement patterns such as the final position, predominant direction, and average velocity. In a test on real-site operations data, the proposed approach demonstrates the improvement in accuracy: for 5.28 seconds' prediction, it achieves 0.39 meter average displacement error, improved by 51.43% as compared with the previous one (0.84 meters). The improved trajectory prediction method can support to predict potential contact-driven hazards in advance, which can allow for prompt feedback (e.g., visible, acoustic, and vibration alarms) to equipment operators and workers on foot. The proactive intervention can lead the workers to take prompt evasive action, thereby reducing the chance of an impending collision.