Is Visual-AI the Key to Detecting Sarcasm on Social Media?

How to Detect Sarcasm on Social Media

Sarcasm has always been prevalent on social media. What isn’t so prevalent, however, is the ability for Social Media Monitoring companies to measure it. This is particularly important for brands to be aware of. In order to find a solution to this, start by thinking of any brand. Now imagine that one of their customers uploads a photo of themselves with that brand’s product to Twitter. The customer looks completely unamused, maybe even appalled by the product, but the caption tells the opposite story. Isolated from the image, the accompanying text would express positivity towards the brand, but combined with the image, it’s clear to any Twitter user that this is a sarcastic tweet. It, therefore, makes sense that a visual-based solution could be the most effective when it comes to classifying sarcasm on social media.

Before we get into any possible solutions to monitoring sarcasm online, we need to first take a look at how sentiment analysis has worked for detecting sarcasm thus far in text-based monitoring.

Sarcasm and the importance of context:

Any Social Media Monitoring and Social Listening tool focused on analyzing text-based mentions would detect a tweet that had a seemingly positive caption as good customer engagement with a brand, and may even classify it as UGC (user generated content). But we know that this isn’t the case if the user’s intention was to be sarcastic. There have been some technologies specializing in sentiment analysis that have attempted to rectify this, but it is still a new area so there has yet to be an accurate text-based solution for detecting sarcasm on the market. One way in which brands would be able to detect sarcasm through text-based Social Media Monitoring is if the user included the words “sarcasm” or “sarcastic” in the caption.

Something that detecting sarcasm needs is context. Without it, it’s very difficult for humans, let alone machines, to interpret sarcasm. That’s why text-only tweets usually need a #sarcasm hashtag included within the statement, otherwise the reader will find it very difficult to determine whether or not the tweeter is being genuine or sarcastic. When the reader knows the author of the tweet and is aware of their sense of humor based on previous tweets, this is also usually enough context to make that distinction as the intended tone can be established. However, there is another factor that these studies didn’t take into consideration in terms of context: an accompanying image. But we’ll get to that later.

When you splash out on extra legroom on Ryanair: pic.twitter.com/ofeOch92Bc

— Louie Stowell (@Louiestowell) April 13, 2018

There have been three particularly noteworthy academic studies conducted on this topic and each of them are very different.

1. Faculty of Industrial Engineering & Management, Technion-Israel Institute of Technology:

Graduate student, Lotam Peled developed the Sarcasm SIGN (Sentimental Interpretation GeNerator) that translates sarcastic tweets that contain #sarcasm hashtag into honest ones. According to Peled, this system is designed for people with Autism and Aspergers “who have difficulty interpreting sarcasm, irony, and humor.” He has also stated:

“Despite the vast development in this field, and the successes of sentiment analysis applications on “social media intelligence”, existing applications do not know how to interpret sarcasm, where the writer writes the opposite of what (s)he actually means.”

2. School of Computer Science, Carnegie Mellon University:

David Bamman and Noah A. Smith also tested tweets that contained “#sarcasm” or “#sarcastic” in the text when conducting their study entitled Contextualized Sarcasm Detection on Twitter. Two takeaways in particular highlighted the importance of context when detecting sarcasm:

• Prediction accuracy improves when any aspect of the environment was included in the tweet, i.e. features that originated from the thread, the author in question, or their audience.
• Tweeters who are less familiar with their audience are more likely to include a hashtag #sarcasm in their tweets.

Sephora employee: “you have 70000 points, you can choose from any of these things” pic.twitter.com/zVHQW9obkR

— Jel (@JelianMercado) February 16, 2018

3. School of Communication & Information Rutgers, The State University of New Jersey:

Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder conducted this research entitled Identifying Sarcasm in Twitter: A Closer Look. They tested their three studies using both humans and automatic classification methods. They found that while accuracy is still low, automatic classification can be as good as human classification. Their results highlighted the difficulty of sarcasm classification for both humans and machine learning methods.

The joys of Internet shopping 😂 pic.twitter.com/zccFoEuVNA

— Andy Leeman (@AndyLeeman91) March 23, 2017

We’ve often talked about the importance of visuals on social media, including images. In order to understand the value of images when it comes to Social Media Monitoring. Images could be the solution to this everlasting problem of sarcasm classification on social media. Going back to the above academic studies, none have been able to achieve an accuracy of over 80%, and even then there has needed to be a hashtag #sarcasm within the text in order to attain that result.

So what do images have to do with detecting sarcasm? Well, logo detection and image recognition combined with sentiment analysis in images and text could be the answer. If a Twitter user posts a photo of themselves or their friends with a brand’s product and the logo is visible, it will be much easier for Social Media Monitoring companies to determine whether or not the tweet is sarcastic using some form of image recognition technology. While it’s still early days for sarcasm detection, investigating the effectiveness of image recognition in classifying sarcasm could be worth looking into.