Investigating factors and good practices to improve the effectiveness of phishing awareness

Abstract

Phishing is a targeted attack that uses fraudulent messages to deceive users to obtain restricted information or install malicious software, making it one of the main tools cybercriminals use in the digital environment. Employees from various organizations are often the target of phishing attacks, representing a significant threat to themselves and their organizations. In response, organizations invest resources, time, and effort in structured initiatives aimed at enhancing users’ ability to identify and respond to such threats, described as Security Awareness Training (SAT), which includes simulated phishing attacks and training to help individuals recognize phishing attempts. However, the actual effectiveness of these initiatives remains underexplored. To investigate the factors contributing to the effectiveness of SAT programs, we conducted a case study at a public organization, allowing us to assess the impact of the intervention on users’ ability to recognize phishing attempts. The case study was performed in four phases. In the first phase, we planned the design, implementation, and evaluation processes of the intervention. In the second phase, we conducted a quantitative study with 4,457 participants to measure individuals' susceptibility to phishing attacks and their engagement with Security Awareness Training designed for phishing prevention using the KnowBe4 platform. In the third phase, we conducted qualitative interviews with 20 participants from the studied organization to analyze their experiences, perceptions, and motivations regarding phishing prevention efforts within the SAT program. In the fourth phase, we proposed a set of good practices informed by the findings from both the quantitative and qualitative studies. Our case study highlights the main factors influencing SAT effectiveness and presents good practices designed to improve phishing prevention strategies.