See Huntress in Action
Our platform combines a suite of powerful managed detection and response tools for endpoints and Microsoft 365 identities, science-backed security awareness training, Managed SIEM, and the expertise of our 24/7 Security Operations Center (SOC).
Schedule a demo to:
- Explore the Huntress dashboard and platform
- Learn about core services, features, and incident reports
- Ask Huntress experts questions and chat live with our team
- See how our platform can impact your unique security and business needs
Deepfakes and Human-Targeted Social Engineering
The technology for creating deepfake videos is now a commodity item and available as a SaaS service. As a result, deepfakes are now being used at scale globally by bad actors attempting to conduct Social Engineering attacks on unsuspecting victims.
Deepfakes are about to reach their 10th birthday. By this we mean that the term “deepfake” was first used back in 2017 by a Reddit user who – in stereotypical internet style – had been using open source technology to swap people’s faces in adult videos. For those not familiar with the concept, the Merriam-Webster dictionary defines the term deepfake as: “a digital image, video, voice, etc. that has been convincingly generated or altered to misrepresent someone as doing or saying something that was not actually done or said”.
Anyone who used video editing software on their home PC back in the 2010s will remember that it could be a slow, clunky business. Processing video has always required lots of processing power and vast amounts of RAM as the data behind high-resolution digital video can be huge and the algorithms used to modify video content convincingly are complex and therefore slow. The other key ingredient, even if you could find a beefy enough PC, was human talent: anything but the most basic video editing, particularly if you needed to change the content of the images rather than just stitching video and sound together and fixing brightness and contrast problems, required human talent and a lot of time.
A Lower Barrier to Entry
Today, the technology for creating deepfake videos is now a commodity item and available as a SaaS service. A quick hop around the web reveals sites that will use artificial intelligence (AI) to create a one-off, two-and-a-half minute, 4K deepfake for the vast sum of $10. Another wants around $35 per month for unlimited videos, but you can run up to 15 minutes in each movie. No skill required – just pay, upload and go.
As a result, deepfakes are now being used globally by bad actors attempting to conduct social engineering attacks on unsuspecting victims. Publicly reported examples include: a fake video call that led a member of a multi-national company’s finance team to dispatch HK$200 million (US$25.5 million at the time); the CEO of a UK energy firm transferred £200,000 ($255,000) to a bogus supplier in Hungary after wrongly believing that the voice on the end of the phone really was his German boss; and a Maryland school staffer who landed in jail for four months after various disreputable activities including making and distributing a racist, antisemitic deepfake video of the school principal.
Fighting Fire with Fire
Ironically, the concept is being used recursively: deepfakes are also being used to warn people about deepfakes. A favorite of ours was produced by Lisa Townsend, the Police and Crime Commissioner for Surrey in the U.K., along with her team as part of their Question EVERYTHING crime prevention campaign. It’s a sobering watch and a tremendous training tool we could all use in our awareness training activities.
Deepfakes are, of course, simply a modern extension of the long-running threat of social engineering attacks which threaten, pressure, entice or guilt-trip victims into helping the attacker get what they want. The problem is, though, that social engineering in general has become harder to distinguish from genuine communications. Take phishing as an example: AI is enabling attackers to produce phishing emails that are not just word-perfect in whatever language they wish but which can write in a style of the attacker’s choosing and can even adapt the content to how an individual’s manager would phrase things so long as there are examples of the latter’s material available to train the model.
And as we know, social engineering relies on common traits that everyone knows about but which many seem to forget when it matters most: responding to perceived urgency; acting on instructions from perceived authority; giving into the fear of reprisal or other negative outcome; trusting familiar faces and voices. We simply cannot train people out of such behaviors, as some (for example, fear) are intrinsic parts of human nature while others (respecting the wishes of seniors) are simply the good manners we have been brought up with.
The human factors that deepfakes mainly rely on are authority and trust: the victim is simply convinced that the person telling them to do something is the person they know is entitled to do so. This brings an interesting change of direction for the outcomes of social engineering attacks: traditionally the downside has been financial (transferring money to bad actors), legal (prosecution for losses of personal data), reputational (ditto) or emotional (individuals being mentally impacted by their unwitting part in the incident). Deepfakes accentuate a fifth impact: trust. By this, we do not simply mean trust in an organization or an individual – these are synonymous with reputational impact. We are referring to trust in our own ability to deal with an attack that any reasonable person would probably not have fallen for.
Realistic Expectations
What do we consider reasonable, though? In reality we should expect a reasonable person, with some awareness training, to be smart enough in the average case to spot physical issues with a deepfake: mismatches between mouth motions and sound; lighting and/or shadows that look odd; odd patterns of blinking; or just good old glitches in picture continuity. Just as we are entitled to expect a user to spot a typo in a fake Microsoft 365 login pages (other faked web sites are available), we should be able to expect them not to fall for a video of the CEO with lip movements that look like an episode of Friends dubbed into German. Of course, there are plenty of software solutions that can analyse video to check for uneven edits, odd lighting and the like.
Detection alone cannot be relied on. For any attack that relies on people, our collection of defense tools needs to include the people that may be attacked. Which leads us back to all the lessons we learned and which we keep teaching our people. True, the attackers can now use AI to make emails and audio tracks sound like people we know. This does not stop us doing things like putting four-eye checks in place to avoid having single points of failure or ensuring that if our policy says payments may only be made using the MFA-defended payment system we do not permit so-called senior people to instill fear and get people to side-step that process.
More importantly, though, we need to empower and encourage our people to be skeptical, to think before acting, to speak up with concerns and to be confident to say “no” without fear of reprisals. Ask yourself: if your CEO stomped into your office and demanded you do something reckless, unusual or unethical, would you do so? If not, why would you do anything different if he asked you on a Teams call, no matter how real it is (or appears to be). If you would, then you have a problem – one that is mostly out of the hands of the security team and can only be solved by your senior management team. (While they are dealing with it, consider resources such as the ISC2 code of ethics under which you should always “act honorably, honestly, justly, responsibly and legally”.
Let us close, though, with a small reprise of our earlier reference to trust, as there is one more area we need to be mindful of: false positives. That is, because our people will increasingly doubt themselves when viewing materials, the chances of them considering those materials fake rather than real. There are approaches available to us such as more training defining specific channels that we can rely on and showing people how to identify them, and trying to limit the outflow of materials from our senior executives onto the internet where they can be used for training AI fakery.
Because as much as we need our people to distrust the fakes, we also need them to trust the genuine.
