The safety of online transactions is paramount in the modern world, mainly since technology develops at a dizzying rate. This study aims to shed light on the numerous threats that users of online transaction systems face. The study used a mixed-methods research strategy to investigate the experiences and perspectives of 400 individuals from various backgrounds. Worryingly, the results show a significant knowledge gap on the many types of cyber hazards. The research reveals a troubling lack of awareness about various cyber risks, including fraud, phishing, and identity theft. It highlights the user’s common functional difficulties. The study proposes a novel framework named COTSEF: A Comprehensive Framework for Enhancing Security in Online Transactions to enhance online transaction security alongside these findings. This comprehensive framework aims to provide a safer and more dependable environment for online commerce by mitigating the identified risks and challenges. The demographic breakdown of the users is also investigated, with the results indicating the increased vulnerability of some age groups and professions to various hazards. It also highlights the need for educational activities to address the significant need for more awareness about data protection rules. The study is a critical resource for policymakers, corporations, and educational institutions, offering actionable insights for developing more secure and user-friendly online transaction systems.