How Artificial Intelligence Turns Data into Useful Information

How Artificial Intelligence turns data into useful information? We are often asked – explain AI from the viewpoint of a data lifecycle, or just how does artificial intelligence (AI) convert data into output that`s beneficial to a business? Machine learning (ML), which can be best described as a subset of AI, is a technology that “fits right into” the traditional data analysis cycle.

ML works on the premise that Machine Learning algorithms, given enough time and experience, should be able to “learn and adapt”. This post deals with the use of machine learning in business intelligence.

Andrew Ferlitsch, Google`s Developer Programs Engineer Machine Learning explains that the ML lifecycle is made up of 3 components – planning, data engineering, and modeling. The latter is the primary thrust area for deploying ML, a method of data analysis that automates analytical model building.

Traditional data analysis requires a model that`s built on past data to establish a link between the variables. It`s sort of an “inside-out” type of model. ML, on the other hand, starts with the outcome variables*, and then automatically looks for predictor variables^ and their interactions, and is like an “outside-in” model.

ML is of great help when your business manager knows what is required but doesn`t know the important input variables to arrive at that inevitable decision. So, you give the ML algorithm the end-goal(s), and then it “learns” from the data about the factors that are important in achieving that target.

(*Outcome variables are usually the dependent variables that are monitored and measured by changing the independent variables. These variables decide the effect the result (independent) variables have when their values are changed. The dependent variables are the results of the investigations determining what was caused or what changed.)

(^A predictor variable is one used in a regression model to predict another variable. In simple linear regression, for example, based on an X & Y-axis, we predict scores on one variable from the scores on a second variable. The variable we are predicting is called the “criterion variable” and is referred to as Y. The variable we are basing our predictions on is called the “predictor variable” and is referred to as X. When there is only one predictor variable, the prediction method is called simple regression)

In his blog post in the Data Science Central, Ajit Jaokar explains ML further:

ML can be seen as a form of applied statistics with increased use of computing and data to statistically estimate complicated functions. It allows us to tackle tasks that are too difficult to solve with fixed programs that are written manually. Instead, machine learning programs depend on learning from patterns of data to make new predictions.”

So here it is. To understand the use of it, it is imperative to understand the distinction between traditional data analytics and ML.

Both are used for analytics but while one is driven by human beings, and can be limiting because there`s only so much data a human can handle, the other is a powerful IT-driven tool that takes copious amounts of data and generates useful insights to help an organization.

While the data models built using traditional data analytics are static, Machine Learning algorithms constantly improve over time as more data is integrated.

What it means is that the ML algorithm can make predictions, observe the outcome, compare against its predictions, then modify to become more accurate.

In fact, Machine Learning Algorithms are used a lot these days in predictive analytics, but more on that later.

Our readers may recall a previous post on recommendation engines where we had written about how Amazon or Netflix recommends “you might also like” products or movies based on past behaviors. That`s ML for you.

As you would have guessed by now, in the ML process, the more data you provide, the more the algorithm learns, returning all the clues you were looking for.

Without such prodigious amounts of big data, ML is not optimized simply because, with fewer data, the machine has fewer examples to learn from, which could have a bearing on the outcome.

Collecting big data and looking for clues within becomes a challenge for your human analysts, more so if the decisions are needed in real-time. ML provides them that extra help in this process.

Here’s the actual process of how Machine Learning algorithms are used to transform data into beneficial information:

First, the input data is collected and formatted based on the data translation measurements such as “categorical data” or “numerical data”. This input can be in any of these forms – RDBS, CSV, JSON, Excel, HTML, text, image, etc.

Then, such input data is imported into a machine learning application programming interface (API). The latter has 3 procedures: the preparation code for accessing data, pre-processing data, and ML algorithms.

The data structure and data type need to be differently manipulated and processed depending on the algorithms before the data is passed into the ML models.

At the third stage, the input data, which does not represent a clear meaning for humans, is transformed into beneficial information by going through the machine learning APIs.

Generally, Machine Learning Algorithms are of 2 types:

Supervised Machine Learning algorithms 

A majority of Machine Learning algorithms fall in this category.

Supervised learning is where you have input variables (x) and an output variable (Y) and you use an algorithm to learn the mapping function from the input to the output.

Y = f(X)

The goal is to estimate the mapping function so well that when you have new input data (x) you can predict the output variables (Y) for that data.

So why is it called supervised learning? That`s because it mimics a teacher supervising the learning method of her student.

Unsupervised machine learning algorithms:

In this, data points are not associated with any labels. Data here is organized into a group called “clusters”. Also, under this, you only have input data (X) and no corresponding output variables.

The goal for unsupervised ML is to model the underlying structure or distribution in the data to learn more about the data.

It`s called unsupervised because there`s no teacher and no correct answers. The Machine Learning algorithms are left to their own means to offer an interesting structure in the data.

One use case of this form of ML in market segmentation for targeting the right customers, i.e. people with the same interests.

Now that we have understood the concept of ML and how the algorithms work, we will look at the use of machine learning in business intelligence in the next part.

References:

datasciencecentral

towardsdatascience

machinelearningmastery

thoughtspot