Conducting Social Media Sentiment Analysis: A Working Example

Before we launch into why conducting sentiment analysis of your brand on social media is important, and how to go about doing sentiment analysis with Python, here’s a quick lowdown on sentiment analysis itself.

Why Sentiment Analysis?

Till a few years ago, analyzing a customer’s sentiments was not something that was taken seriously. Today, thanks to advancements in technology and also in the thinking of businesses, sentiment analysis is emerging as a viable tool. What makes it interesting and rather different from the other forms of data analytics is that this one deals with emotions, and as we all know, emotions are never black or white.

Express Analytics puts the voice of the customer at the heart of the business. Want to learn more about sentiment analysis? Fill up this short form and our experts will touch base with you.

Sentiment analysis tells a brand or an enterprise what the world or the consumer feels about it. The sentiments could be positive, negative, or even neutral. Sentiment analysis is an algorithm-driven process, with the algorithms having access to a dictionary of words, each of them holding a positive, negative or neutral sentiment. Example: happy, sad, annoying, rewarding, lovely, wonderful, creative, etc.

To conduct such an analysis, a business must have the required tools in place and a clear idea of how to use them.

What Is Social Media Sentiment Analysis?

As the name signifies, sentiment analysis of social media means just that – to understand using data what people feel or think about your product, service, or even brand. Today, this has gone beyond the vanity metrics like likes, mentions, etc. Social media sentiment analysis means collecting and analyzing information in the posts people share about your company on social media.

Why is this form of analysis important? You get to know exactly what people think of your brand on social media. What are they saying? Are the majority happy or unhappy with your product? Why? Social media sentiment analysis helps you answer these types of questions.

Increasing Use Of AI in Sentiment Analysis

Because of advancements in Natural Language Processing (NLP), data analysts can segment client feedback as: positive, negative, or neutral with natural language.

NLP was largely being used in text analytics. With NLP, this form of analytics groups words into a defined form before extracting meaning from the text content. NLP is used to derive changeable inputs from the raw text for either visualization or as feedback to predictive models or other statistical methods.

But with the advent of new tech, there are analytics vendors who now offer NLP as part of their business intelligence (BI) tools. From the text, for example, NLP is now used to make sense of “voice” in interfaces such as digital voice assistants, or smart speakers like Amazon’s Alexa, as the latter becomes more and more interactive.

In sentiment analytics, to get a relevant result, everything needs to be put in a context or perspective. When a human uses a string of commands to search on a smart speaker, for the AI running the smart speaker, it is not sufficient to under “understand” the words. It also needs to bring in a context to the spoken words used, and try and understand the “searcher’s”, eventual aim behind the search.

What keeps happening in enterprises is the constant inflow of vast amounts of unstructured data generated from various channels – from talking to customers or leads to social media reactions, and so on. Now, to make sense of all this unstructured data you require NLP for it gives computers machines the wherewithal to read and obtain meaning from human languages.

Here’s How To Conduct Social Media Sentiment Analysis Python (With Twitter As An Example)

The process, sentiment analysis python, by and large, can be duplicated across almost all social media networks. Data scientist Shubham Shankar shows you the way. You need to start by:

Understanding Your Market

Any analysis starts from this point: understand your audience. Which means you need to understand how your audience feels about not only your brand but also about your various marketing campaigns.

Choose The Right Keywords, Hashtags, And Labels

Your business needs to choose specific keywords that sentiment analysis software detects like:

  • It’s great or it’s wonderful
  • It’s cheap or it’s expensive
  • It’s easy to use or
  • I didn’t like it

Then, you need to assign a sentiment score to each of these words. Obviously, it means positive feedback gets a higher sentiment score while negative feedback a lower one.

Two Techniques Of Sentiment Analysis

Before going ahead with this post on sentiment analysis with Python, you must understand that there are two techniques of this form of analysis.

Very simply put, this method uses a dictionary of words labeled by sentiment to determine the feeling put forth in a sentence. Like we said above, after a sentiment score is assigned, it needs to be combined with additional rules.

Machine learning-based

This takes out most of the human interactivity. In this technique, an ML algorithm is trained to recognize the thought based on the words and their order using a labeled training set. The technique is dependent on the type of algorithm and the quality of the training data. The “label” will be a measure of how positive or negative the sentiment is.

Sentiment analysis of Twitter with Python 

For fetching the real-time data from Twitter, we will first need to get our key and access token using the official Twitter API Tweepy from the Twitter developer tool website. Once we have the keys, then we can proceed towards tweet extraction and creating the sentiment model.

Importing Necessary Libraries

We have to import all the required libraries.

sentiment analysis python

Getting Credentials

Now to extract the tweets from Twitter, we need to fill the consumer_key, consumer_secret, access_key, and access_secret and initialize the API as shown.

sentiment analysis using python

Before we start extracting the tweets, we must define the username of accounts for which we want to analyze the tweets. We can also define query words related to which we want our tweet data. Here, we will analyze tweets of 5 global organizations from their official Twitter accounts.

Extracting Tweets

Now, after defining organizations, we are ready to extract the tweets. Using the below-written code, we are extracting 100 tweets from each organization’s Twitter feed and store all the tweets in sentiment dictionary along with the user and date when the tweet gets posted.

extracting twitter data using python

As the tweets data is in the dictionary, we convert it into a data frame for further processing. Now you can see the data pulled out from Twitter with tweets, User along post-date.

Twitter Sentiment Analysis using Python

extracting twitter data sentiment analysis

Cleaning the Twitter data

We are discussing conducting sentiment analysis with Python in this post. Well, we fetched real-time tweets from Twitter, but now we have to extract meaningful insights from the tweet’s data. For that, we have to clean data because it contains lots of URLs, numbers, and user_ids, which get challenging to analyze tweets.

cleaning twitter data in python

VADER Sentiment Analysis Model

VADER (Valence Aware Dictionary for Sentiment Reasoning) is a model used for text sentiment analysis that is sensitive to both polarity (positive/negative) and intensity (strength) of emotion.VADER sentiment analysis relies on a dictionary that maps lexical features to emotion intensities known as sentiment scores. The sentiment score of a text can be obtained by summing up the intensity of each word in the text.

Vader takes in a string and returns a dictionary of scores in each of four categories:

  • negative
  • neutral
  • positive
  • compound (computed by normalizing the scores above

Using VADER Sentiment Analysis

VADER is available in the NLTK package and can be applied directly to unlabeled text data as shown below.

vader sentiment analysis python

After initializing the analyzer, we can use polarity_scores() to find all the positive, negative, and neutral scores for each tweet.

Now using the compound score we will assign a sentiment label to each tweet.

sentiment analysis python vader

sentiment analysis

Now using the compound score we will assign a sentiment label to each tweet.

sentiment analysis algorithm

sentiment analysis applications


VADER classifies the sentiments very well. The ready-made model can be used across multiple domains, social-media texts, review texts, etc. It also does not require any training data and is very easy to use.

Build sentiment analysis models with Oyster

Whatever be your business, you can leverage Express Analytics’ customer data platform Oyster to analyze your customer feedback. To know how to take that first step in the process, press on the tab below.

Liked This Article?

Gain more insights, case studies, information on our product, customer data platform

Leave a comment

Your email address will not be published. Required fields are marked *

Copy link
Powered by Social Snap