How Object Detection is Used in Anti-Phishing

How Object Detection is Used in Anti-Phishing

Reading Time: 3 minutes

Object Detection plays an important role in phishing protection

We have discussed previously how logo detection and text detection work with anti-phishing software, and now it’s time to focus on the oft unsung hero of computer vision’s efficacy in phishing protection: Object Detection!

With bad actors employing every visual evasion technique they can come up with, anti-phishing software developers must make it a priority to utilize all computer vision technologies necessary in order to stay a step ahead. Object detection is one such technology that cannot be ignored when it comes to phishing protection. 

Object Detection used in Anti-Phishing

How Does Object Detection Work? 

Bad actors use a lot of code-based and graphical techniques to muddy things like buttons and forms or by using encrypted scripts that only build forms when a user’s browser renders the page. Object detection can see through these tactics because Visual-AI looks at an email or web page after it has been rendered. It is firstly captured as an image, then processed to look for any anomalies, which are then flagged for investigation.

Applying object detection in phishing protection

Cybercriminals quite literally make it their business to know how anti-phishing software tracks and stops their attack attempts, so they do all they can to evade detection. The reverse must always be true; developers and engineers at cybersecurity companies make it their business to always be ready for the next popular attack vector.  Object detection is one technology that can help with this in two ways. As mentioned before, code-based and graphical techniques are used by bad actors to evade detection. With code-based obfuscation, they can use javascript to hide/disguise forms and buttons among other data capture elements. A typical programmatic-based detection system will not flag these elements because they are simply not seen.  With graphical obfuscation, bad actors disguise forms and buttons as images, which when clicked will take users to a harmful webpage or open up a data collection form in another window. Traditional phishing protection software cannot detect these as they are primed to read specific code elements that denote the presence of a form or button.  Object detection, however, can spot either of these techniques and detect these elements once the computer vision engine renders them as images.

Be Sure To Also Read ->Phishing Detection Use Case Overview

The full protection suite 

A full protection suite will combine traditional, machine learning-based anti-phishing technology with computer vision technology including logo detection, text detection and objection to ensure that all bases are covered. The goal should be to leave as few gaps in the fence as possible through which a phishing threat can succeed.  With bad actors constantly finding themselves under and over that fence, it’s absolutely paramount that cybersecurity companies, and their partners, do all they can to keep up to date with the latest technology. 

Want To Talk? 

Visual-AI, including object detection, should be an integral part of any anti-phishing system. December 2021 saw the highest number of phishing attacks since 2004, and it would be fair to estimate that this trend will continue upwards. Cybersecurity companies can show scammers that they mean business by introducing computer vision into the fold.  VISUA works with leading cybersecurity companies to empower their technology with Visual-AI. If this is something you are interested in, we’re ready to talk! Fill in the form below, and in the meantime, you can watch our video in which our CTO explains how Computer Vision and Cybersecurity work together. 

Book A Demo


Are Website CMS, Email Marketing, and Survey Platforms Accountable For Their Part In The Phishing Epidemic?

Reading Time: 7 minutes TLDR: Phishing attacks have reached the highest levels ever seen. Bad Actors are abusing convenient and well-known platforms to craft emails, web […]

Anti-Phishing Cybersecurity
APWG Phishing Trends Report: Year on Year Review (2022)

Reading Time: 5 minutes A close examination of the APWG Phishing Trends Report (Q1 2022) TLDR: Our previous article on this subject focused on the historical […]

Anti-Phishing Cybersecurity
How Visual Search is Used in Anti-Phishing

Reading Time: 2 minutes Visual Search in Phishing Protection – an effective combination Visual search is a powerful piece of computer vision technology that can enhance […]


Trusted by the world's leading platforms, marketplaces and agencies

Integrate Visual-AI Into Your Platform

Seamlessly integrating our API is quick and easy, and if you have questions, there are real people here to help. So start today; complete the contact form and our team will get straight back to you.

  • This field is for validation purposes and should be left unchanged.