Social engineering attacks go beyond just phishing and no longer limited to PCs, but most solutions don't distinguish between different types of attacks or platforms.
The existing methods are based around self-reported measures, attack simulations, and training (with some mitigation).
But the self-reported method is biased and resource intensive, so cannot be done continuously. The attack simulations are typically limited to classic phishing, and cannot be used to evaluate users vulnerability to other attack vectors. The training workshops are great, but unlikely to reflect the users normal behaviour - as they know they are in training. Additionally, employees are not big fans of forced training, and may not be engaged.
Most technological mitigations are limited to specific environments (like the office, specific browser).
The researchers have created a new toolkit: SafeMind. The researchers looked into specific areas of awareness models, and worked with other security researchers to help rate the importance of the criteria, which helped them narrow down the most critical areas to measure.
Created an endpoint solution, attack simulator and network solution. The endpoint solution looks at a lot of things on the endpoint - sensors on social media activity, security settings, certificate management - to create a profile of the user. Using this profile, could target attack simulations for that user.
Over 7 weeks they experimented on 162 subjects. They could see that those users with lower security knowledge were less successful at mitigating some attacks. Users self-reported behaviour may differ significantly from their actual behaviour, whereas their research could predict more accurately their actual behaviour.