Real-world applications of Computer Vision

Computer vision takes advertising intelligence to the next level. Leaving guesswork on the sidelines, computer vision powers ad monitoring platforms to deliver the most accurate and complete intelligence possible. It enables granular competitor analysis, to-the-second placement tracking and the ability to classify and categorise ad types and placements. 

With Computer Vision, ad agencies can create a log of which brands advertised across which channels and geographic regions. What products or services they were advertising and what their message was, including pricing and offers. This can then be compared with their client’s brand advertising, allowing them to review and adjust future ad placements and messaging.

It’s an advertiser’s dream come true. Platforms that utilise computer vision are the ones that lead their market because they can provide a level of intelligence that’s impossible without it.

It’s no secret that cybersecurity businesses are in an arms race with cybercriminals. It seems as though every time the former adapts, the latter finds another way to slip through the anti-phishing net. Computer vision enables anti-phishing software to add a new layer of key signals that can help them better detect phishing attacks. Computer Vision takes a different approach to traditional programmatic email and web page analysis in that it doesn’t read code or look at the technical detail of emails/pages. Instead, it uses a purely visual approach, just as a human would, only at machine speed. It converts emails and web pages to flat images and then extracts the key signals from it. This includes brand information, the presence of data collection and payment forms, along with links and buttons. It can also flag where an email or page is the same or similar to a known authentic version, i.e. a copy of a Microsoft login page.

This approach is key because bad actors will often use obfuscation techniques to hide elements when analysed programmatically. But all their tinkering finishes when the email/page is rendered in the reader/browser – and that’s what Computer Vision analyses.

It grants anti-phishing software the ability to spot and flag high-risk elements as it finds them, allowing detection systems to prioritise further and analyse deeper than would otherwise be possible. 

Watch our Chief Technology Office, Alessandro Prest explain it in more detail below.

As with advertising monitoring, some brand monitoring platforms haven’t kept up with how, where and what, brand managers need to monitor their brands. 

Brand monitoring platforms traditionally used text analysis to detect brand mentions in the main post text and comments. But a recent report by VISUA highlighted that 88% of brand mentions are missed when analysing only text. This is because the majority are visual brand mentions, i.e. locked in images and videos, with no corresponding text-based mention. 

Computer Vision allows brand monitoring/social listening platforms to not only detect significantly more brand data than ever before, but also to provide more contextual information related to the brand mention. Couple this with the fact that most social platforms now focus on visual media sharing, and it means that with computer vision, brand managers can unlock more than 1000 times more valuable brand insights than textual analysis alone.

Counterfeiters exploit every opportunity to fool unsuspecting consumers. Computer Vision technology has the ability to spot even the smallest anomalies, whether in the logo, the packaging, the size or aspects of the product itself.

Ideally used by marketplaces, and brand protection service providers, Computer Vision can analyse every image of a listing, and even the listing itself, to provide key signals that can help identify the fakes from the authentic products.

Importantly, potentially counterfeit products can be highlighted at upload, which not only ensures these products never make it onto the platform, but avoids regulatory issues and reduces the strain on compliance teams.

Protecting intellectual property has become increasingly challenging in the digital age. It’s easier than ever for bad actors to make use of trademarked and copyrighted imagery and sell them on unlicensed products online. There are sellers taking trademarked characters and images, incorporating them into their own designs and selling them on creative marketplaces and print on demand sites. And of course there is the age-old issue of piracy, which seems to still thrive despite trojan attempts from copyright holders to put a stop to it. 

Computer vision applied in these situations makes it simple to detect and stop copyright infringed and pirated content in its tracks as it can instantly detect the use of logos, images and characters. Importantly, it can do so at time of upload by the seller, which not only ensures these products never make it onto the platform, but avoids regulatory issues and reduces the strain on compliance teams.

Counterfeiters exploit known markings and labels on products to evade detection and fool unsuspecting consumers. Most recently, they have even taken to creating fake holograms to be placed on their fake products; all in an effort to make their counterfeit product as close as possible to the genuine articles. Computer vision technology has the ability to detect all these elements and validate them against known good product examples to spot even the smallest anomalies.

It is ideally used to authenticate products at key touch points throughout the supply chain. With a simple app installed on a smartphone and the Computer Vision technology running on-device, key stakeholders, from logistics and customs, all the way to the consumer, can scan a product to confirm its authenticity in seconds. 

Delivering actionable sponsorship data means having access to all the data, rather than  looking at only partial metrics. Brand impressions, much like ad impressions, form the foundation of every ROI measurement challenge. 

The key is to be able to monitor and deliver key insights, including accurate brand impressions, in real-time from all relevant sources. Once an aspirational goal, this is now possible with computer vision. 

Computer Vision allows you to measure broadcast media, online coverage, social media and even print impressions from images and videos alike, all using the same methodology. Once you have complete impressions data from all these sources, combined with other data, such as net promoter scores and brand and sponsorship recall, and even sales data, you can then truly assess the ROI of sponsorship spend – but it all begins with accurate impressions data.

But, Computer Vision is not limited to detecting the presence of brands in visual media. It can also be used to enhance sponsorship opportunities by looking for prominent areas that lack any brand exposure. This is called Sponsorship Opportunities Research Tracking (SORT) or more generally Whitespace Analysis. This would allow venue owners, event organisers and teams to maximise the revenues from sponsorship for themselves and their sponsors.

Most user generated content today is visual, either as images, video or both. From social platforms to video chat platforms, gaming platforms, social marketplaces, and even the metaverse, user generated visual content is hard to find and moderate. Whether your problem is simply blocking infringing/non-compliant products from your social marketplace or the more complex issue of NSFW, racist or other hate speech content – the main limitation is humans vs. massive volumes. But not for computer vision!

Content moderation is crucial to protect the public and the platforms to which content is being uploaded. Computer vision improves the moderation process by allowing moderation software to analyse images and videos, at machine speed, for elements that contravene content policies or should simply be flagged as potentially harmful. Many automated systems currently in use are purely text-based, leaving humans to do most of the visual work. It is widely reported that this kind of work is leaving employees with symptoms of depression and PTSD. Computer vision allows businesses to protect their employees, their business interests and the end-user in one go. 

Watch the video below to see how it applies to various industries.

Computer vision has the capability to enhance retail in numerous ways, both logistically and from the customer perspective. As the image above shows, managing stock on shelves can be critical to meeting sales figures. Computer Vision can automatically alert staff as products run low on shelves, ensuring there are always products for customers to buy.

Technologies like Visual Search can also help customers match a product a custom has an image of to an identical or similar product in-store.

Automated stores can be made possible with object and label detection. No more awkward scanning of barcodes, simply show the product to a camera and the cost can be added to your cart.

Managing traffic flow, identifying popular traffic areas and detecting theft are further uses of Computer Vision in retail.

Computer vision can be applied to a number of areas to ensure public, worker, and road safety. By processing live camera feeds with Visual-AI potential hazards, incidents, and even dangerous driving conditions, can be detected and action taken.

These actions can vary from alerting the necessary bodies, such as safety personnel and traffic control, to updating digital road signage to alert drivers of upcoming difficulties or changing the sequencing of traffic lights during heavy foot-traffic periods.

Computer vision is the core technology behind autonomous vehicles. Self-Driving cars use object detection in combination with sensors to analyse surroundings in real-time. With the power of Visual-AI, a self-driving car can recognize pedestrians & other road users, animals, road signs & markings, barriers and other road-side furniture, red lights and other road vehicles, so that they can safely navigate roads.

There are many use cases to which computer vision can be applied when it comes to security, with many finding it comforting while others find it concerning. 

But it doesn’t have to be the Orwellian concept of tracking individual people across cities (although that is possible). The more important and valuable uses are things like building security, retail theft reduction, and crowd control & disaster prevention. Computer Vision, in most cases, has the potential to be the hero rather than the villain when it comes to public safety and security.

Applications in medicine and healthcare are virtually limitless. Any situation that requires a trained medical professional to visually analyse a scan or plate, etc, can be done more accurately, more reliably, and faster than any human could.

The applications for computer vision include: 

• detecting early stage cancer cells in imaging scans, such as MRIs and X-Rays, even if they are being carried out for other purposes. 
• Spotting nerve issues and early stage tooth decay in dentistry 
• Spotting ocular discrepancies at the earliest stages in order to prevent deterioration. 
• Scanning photographs of areas of the body to detect possible malignant melanomas or other conditions.
• Embedded Computer Vision in AR eyewear that can enhance what the practitioner or surgeon is seeing in real-time and offer recommended actions.

Because most healthcare diagnoses are generated through visual means, the widespread application of computer vision is pretty much a given. It’s just a matter of time and budget before all imaging machines are equipped with Visual-AI which can immediately alert doctors of anything unusual or suspicious, greatly reducing the risk of misdiagnosis and hugely improving treatment efficacy. 

However, a key barrier of entry is the lengthy process of proving the technology through studies and trials before any implementation or product can enter the market. This can take years and so requires significant investment.

Computer vision has been a wonderful innovation for production/manufacturing facilities. It enables production machinery to extract information and report or take action on flaws or mechanical problems. For example, in a factory that mass printed circuit boards, the machine will be equipped with computer vision so that it can spot poor solder joints, missing or badly placed components, and remove them from the line immediately. It can also be applied to the production of ice cream wafers to ensure they are the same size, to contact lenses to scan for miniscule imperfections, to pharmaceuticals to detect imperfections in pills and poorly sealed packaging, or to ensure bottles are all properly filled. This is called “optical sorting”, a process which is celebrated as an enormous cost saver in the production industry.

Optical sorting has, in recent years, also revolutionised the waste management industry. Powered by computer vision technology, optical waste sorting machinery ensures waste is sorted accurately, dismissing any non-recyclables from the conveyor system and sorting plastics, paper and tins into the right areas. 

The result is a much more efficient waste management system that allows for a wider scale of product recycling; something that is so important in this day and age. 

Computer vision is widely used across a variety of military applications. Detection of enemy soldiers and vehicles and missile guidance are applications you may already be familiar with. Missile guidance can reach specific targets or areas through computer vision aided target selection which uses locally acquired image data to recognise the target location. Another advanced application is the usage of visual sensors that provide soldiers with live information about a combat scenario which can be used for making to-the-second decisions, potentially saving lives. Finally, smart drones that can fly autonomously. It may sound like something from a futuristic sci-fi film but it is very much present-day technology.