How Natural Language Processing is Helping Democratize Business Intelligence
In the 1st part, we looked at the various stages of development of NLP, and its uses, especially in search. In the 2nd and final part of the blog post, we will look at how exactly is NLP used in BI, and the hurdles it still faces.
A minuscule percentage of those in the business of providing BI solutions have adopted NLP and adapted it to generate results for Enterprise clients. While the figure may be small today, advancements in the field are bound to push the number up.
While writing on how BI requires NLP to transform data and analysis into an intelligent and human-sounding language, to encourage more and more people into adopting data analytics, Stuart Frankel, CEO of Narrative Science (2), says the BI & analytics vendors have “realized the power of language.”
Stuart writes that instead of forcing workers to interpret the visualization of data, all they need to do is read the written summary. He calls NLP “an additive element” that tells a deeper story about the data.
There’s still healthy cynicism around NLP because of the many hurdles that remain (one more reason why NLP did not make it to our 2018 ‘trends’ list), but if one were to see the growth curve of NLP from the past half a century or so, clearly many obstacles have also already been surmounted.
Yet, there’s no doubt that the promise of ‘ease of business it offers as an application in data analytics will ensure its en masse adoption in the years to come.
With more and more companies adopting data analytics, to make the technology more democratic and help percolate it down to the lowest corporate level, there’s an increasing need to move away from complex data visualization dashboards.
This will help many more within an Enterprise to understand the analysis and take decisions, preferably instantly. That’s why NLP is being looked at as a viable option.
Enhance Your Business Efficiency with Our BI Experts
Where NLP Will Score – Unstructured Data
Enterprises have started collecting humungous amounts of data, and the challenge is to make sense of it all.
For the longest time, Enterprises chose to mostly ignore unstructured or semi-structured data since the tools and skills required to derive meaning from it were not sophisticated nor flexible enough. Till recently, companies would adopt either manual data entry or a template-based approach to analyze unstructured data.
One of the factors that have limited the use of BI is the need to understand how to compose queries and correctly interact with BI products to achieve the desired results.
But NLP is set to make analysis more user-friendly by translating natural language queries into the language needed to obtain results. It can help extract vast amounts of unstructured data that have been made available by the ever-increasing use of social media use, online reviews, etc.
NLP now allows a computer to understand the language of humans, thus making sense of customer conversations, and categorizing them.
This means the use of NLP in online social conversations can help recognize a sentiment on a particular subject, probably in real-time, thus allowing the brand to change course on a product, midway through its marketing campaign.
One of the biggest applications of NLP has been in the health sector, where it is used as a tool to unlock critical clinical data.
A large proportion of the health sector consists of unstructured data, which is vital for complete and accurate quality measure analytics.
Because of the limitations of using free text in analytical initiatives, health organizations often miss out on critical patient-related information. (3) Here’s an example: A patient’s ‘smoking status’ may have been marked as, “smoked in the past” or “smoking 1 pack a day”, or, “now down to 1 cigarette a day.”
NLP can help identify the relevant qualitative statements within that patient’s record, and derive the meaning to categorize phrases into structured observations such as, “is a smoker,” “ex-smoker,” or “not a smoker.”
Thus, it can help healthcare organizations to extract such information from free text to ultimately help the physician prescribe the best medication choices and therapies.
Challenges Before NLP
There’s no denying the fact that NLP will change the world of BI. But full-scale integration is still some time away for these reasons (4):
The dynamism of human language: Bridging the gap between machine and human languages is a complicated process. For one, understanding the syntax of the search query inputted in human language remains one of the biggest challenges.
When contextual interpretation is removed from a query, only bare words remain behind. It’s easy then to misinterpret the meaning behind the words. As of now, the immediate challenge seems to get over syntactical complexities in human languages.
The other is context and emotion. For example, the AI-run processor must be able to decipher the difference between, “Will Apple become a trillion-dollar company?” and, “Will an apple export company work in San Francisco valley?” to throw up a correct response.
It must be able to understand word/phrase order variation, suffixes that transform one part of speech to another, inflection (eg: cough and coughed), and so on.
Hardware limitations: While there’s no doubt that computing power and storage capacity has multiplied over the years, will a computer ultimately be able to mimic the way a human brain works? The latter’s complex neural network allows it to analyze millions of variables within seconds and spew out the output, in real-time.
Also, humans rely on emotional and physical relationships linked to other humans, so a single word can mean different things at different times vis a vis for a human being.
Computers, of course, are unable to apply the same contextual meaning to unstructured data. Nuances of sarcasm, wit, and anger are something that computers today are unable to grasp.
In Conclusion
User interfaces using natural language will only proliferate, and so will NLP applications in analytics. As AI technology matures, the computer will get better at “understanding” human language queries, learn about the various semantic relations and inferences in a query, and start delivering even real-time business intelligence to users, irrespective of their level of technical expertise.
For now, a beginning has been made, with AI helping some cutting-edge Enterprises turn complex data into actionable business intelligence, although with some amount of time lag. By transforming analytics results into stories, NLP will speed up the implementation of analytics in every sphere of life, augment human potential, and help Enterprises of all scales achieve their goals much faster.
References:
Business Intelligence Requires Natural Language Generation
NLP: Unlocking the potential of unstructured text in healthcare
An Engine That Drives Customer Intelligence
Oyster is not just a customer data platform (CDP). It is the world’s first customer insights platform (CIP). Why? At its core is your customer. Oyster is a “data unifying software.”
Liked This Article?
Gain more insights, case studies, information on our product, customer data platform