Data-based knowledge - Handling Data

In the modern company, two parallel and interconnected worlds of information and knowledge coexist.

In the first, the more operative world, the company exploits its know-how based on current insights to achieve its objectives as an organisation. At this level, operational efficiency is king, and data is mainly used to control and try to optimise execution to increase efficiency within the quality parameters chosen as standard (the aspired levels of effectiveness).

On the second plane, in parallel, the company analyses the vast amount of information available to increase its operational efficiency and find new patterns. These new patterns may lead to new know-how, new ways of doing things that change its operating models, and the effectiveness of the solutions it offers to the market.

These two levels are connected at times. In these moments, this ambidextrous management, exploitative and exploratory, of data-driven knowledge allows for continuously updating the operational model of knowledge exploitation by exploring new knowledge.

This module will discuss both planes and how these connections occur.

Learning objectives

Upon completing this sub-module, you should be able to do the following:

  • Describe and Analyze the Composition of a Company Information System
  • Examine the new landscape for data result of bridging Big Data and Artificial Intelligence
  • Distinguish between the different Artificial Intelligence Methodologies.
  • Judge the Impact of Artificial Intelligence tools in business processes.

Glossary

Company Information System. Is the set of elements (technical, logical and human), that allows a company to manage the internal and external data needed to improve the business results.

Big Data. Massive datasets should be worked to extract manageable data to source the company information system to reach business goals.

Parallel Data Processing. Is a processing technique based on spilling a big dataset into small pieces, processing them in many small and cheap computers, and after processing the pieces, reunifying the results? This avoids the invested need to build expensive computer processing units (CPUs) capable to process such amounts of big data.

Data Set. Is a group of data coming from a source? The bigger the dataset, the bigger the capability of learning from it.

Algorithm. Is a sequence of logical actions where the subsequent action depends on the result of the precedent action/data that acts as its input, the processing of an input generates an output, a logical decision over that output and so on. This logical string conducts to a final decision and could feed a learning process as well.

Artificial Intelligence. Is to try to make computers take decisions in a similar way to humans. Artificial intelligence combines computer science and robust datasets, to enable problem-solving, decision-making, etc.

RPA (Robot Process Automation). Is the application of AI to a repetitive process where human actions are replaced by machine actions, most of them based on different technological solutions (voice and image recognition, character recognition, etc.

Intro video

Learning content

Describing and Analyzing the Composition of a Company Information System

A Company Information System is a set of elements (technical, logical and human), that allows a company to manage the internal and external data needed to improve its business results.

When a company defines a product to be manufactured and sold, it starts to work and store data about:

  • The suppliers of the necessary raw materials: their prices, delivery terms and conditions.
  • Logistical input data: how the raw materials are received and stored.
  • Planning and concrete production operations: who will do what, for how long, with which tools, with which costs and with which final result.
  • Quality, quality control and quality assurance.
  • Customer to whom the product will be sold: their orders, prices and the sales conditions we agreed with them.
  • Outbound logistics data: where, how, when and in what quantities we serve the orders, delivering the products to points of sale or end customers.
  • Returns and after-sales services provided.
  • Supplier invoices payable and customer invoices receivable.
  • Financial data: profitability achieved, cash flows for the period and operational funding requirements to keep the business running.

Data, hundreds of data enabling the business processes to meet their specific objectives, helping the business achieve its overall objectives. All these data are located in the Data Map in the company's information system. That Data Map has the aspect described in the below chart.

Source: Dynacrongroup.

In this Information Map, there are some main parts to focus on that you will probably find in a lot of companies from different sectors:

  • eCommerce (Online shop/portal): for online selling data.
  • CRM (Customer Relationship Management): for data related to any interaction with the customer out of the selling data (registered in the company ERP)
  • ERP (Enterprise Resources Planning): for all the company transactional data (orders, invoices, warehouse movements, etc.), you get info from the ERP launching queries to it. The ERP will answer about the register in its databases that accomplish the conditions established in the query. This kind of data exploitation takes a very long time when you work with massive data.
  • DW (Data Warehouse): it contains the same ERP data but is organized in an easy-to-use way, and with periodic data consolidation that avoids the query way that implies losing a big amount of time to get results.
  • BI Analytics (MS Power BI, etc.): it means organizing the Data Analysis into dashboards with KPIs that allow a precise view of your business goals.
  • Collaborative Tools (MS Sharepoint, MS Teams, Slack, etc.): this part of the system has the data and the tools to collaborate on projects, processes and activities. These tools use to combine data from communication (video calls, emails, messages and chats), project management (Kanban), scheduling (calendar), people organization (teams management) and work/project-related file storage.

What is Big Data

Big Data is a massive amount of data that could be managed in order to improve some business process results. As stated before, this data came from very different sources and in very different forms.

Image by Olga Hmelevskaya on Freepik

Big data is the ability to manage an immense volume of disparate data, at the right speed and within the right timeframe to enable real-time analysis and reaction. Big data is broken down into three characteristics (the 3Vs):

  • Volume: How much data (more than a normal computer can process).
  • Velocity: How fast the data is produced and processed (some cannot even be stored for processing).
  • Variety: The various types of data.
  • The fourth V is Veracity: the triangulation of information that is possible to perform.

The main benefits of Big Data for a business are:

  • Better customer insight
  • Improved operations
  • More insightful market experience
  • Agile Supply Chain Management
  • Data-driven innovation
  • Smarter recommendations and targeting.

The real breakthrough in Big Data came when companies like Yahoo!, Google and Facebook realised that they needed help monetising the huge amounts of data that their offerings were creating.

These start-ups needed to find new technologies that would allow them to store, access and analyse vast amounts of data in near real-time so that they could monetise the benefits of owning this amount of data about the participants in their networks. Their resulting solutions are transforming the data management market. In particular, the MapReduce, Hadoop and Big Table innovations proved to be the sparks that led to a new generation of data management. These technologies address one of the most fundamental problems: the ability to process large amounts of data in an efficient, cost-effective and timely manner.

What is Artificial Intelligence?

Intelligence is the ability to use language, mental abstractions and concepts to solve problems and improve. Four main things that AI enables a machine to do:

Artificial Intelligence is spreading across all sectors and is accompanying us in more and more situations as the cost of obtaining data (sensors) and its storage and processing (cloud and parallel computing) becomes cheaper and faster. The potential demand for this new world of knowledge has brought to the fore two professions specifically related to data management, data scientists (data set-oriented) and data engineers (algorithms oriented).

Source: europarl.eu

The New Data Landscape: Bridging Big Data and Artificial Intelligence.

We have seen in the previous section that an Information System consists of the interaction of several elements, remarking between them: the data, the computer system and the user. In this interaction, human users establish how to get and process data, analyse the process output, make decisions, and very often introduce those decisions in the form of new input data to be processed or communicated. In this process, the user analyses the results and takes action continuing the process flow with their analysis result, but not only this. The human system user learns during the analysis process and changes how the user analyses the data; in fact, the user changes the algorithm used in the analysis and, sometimes, how the data set is organised as well.

This data knowledge updating process (over algorithm and data set) is based on experience and judgement about these experiences. Experience depends on how many times you realise the process and how big the amount of data you process is. The bigger the data, the bigger the frequency of processing, and the bigger the learning is.

This learning potential has been known for a long time ago. The term “artificial intelligence” was coined as long ago as 1956. In 1950, Alan Turin stated: “Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child´s? If this were then subjected to an appropriate course of education one would obtain the adult brain.”

Source: Ted Talks

And here comes the machines, the big data and the processing technologies. In the last decade, it produced the confluence of:

  • The vast amount of available data (coming from social media interaction, online shopping, transactional data from banks and physical stores, sensors, etc.) is called Big Data.
  • The development of new processing methodologies that allow getting actionable information results without the need to invest in expensive processing central units (parallel computing and cloud computing).
  • The development of the sensors sector allows exponential growth of the Internet of Things (IoT). IoT lets your freezer know if you need to purchase something better and earlier than yourself or allows your house to make efficient use of energy, enlightening or warming the areas of the home that you are using or going to use soon.
  • AI methodologies, depending on data and work complexity, could be applied looking for:Automation. Is the application of AI to replace human decision in a repetitive process. There is efficiency improvement derived from its application but no learning.Augmentation. Is the application of AI to improve the scope or the efficacy of the process derived to the algorithm training exposing it to new and bigger datasets and extracting learning conclusions from the right or wrong decisions offered by the algorithm.

Source: Accenture

Artificial Intelligence Methodologies and Tools Used over business processes

AI Methodologies

Machine Learning means the ability of the AI-based tool to learn. ML is defined by Computer Scientist and machine learning pioneer Tom M. Mitchell: Machine learning is the study of computer algorithms that allow computer programs to automatically improve through experience.” ML is one of the ways we expect to achieve AI. Machine learning relies on working with small to large datasets by examining and comparing the data to find common patterns and explore nuances.

The function of a machine learning system can be:

  • descriptive, meaning that the system uses the data to explain what happened (e.g. Social media usage and engagement data such as Instagram or Facebook likes);
  • predictive, meaning the system uses the data to predict what will happen (e.g. managing performance expectations and avoiding risks); or
  • prescriptive, meaning the system will use the data to make suggestions about what action to take (e.g. investment decisions).

Deep learning is about computers learning to think using structures modelled on the human brain. In fact, all deep learning is machine learning, but not all machine learning is deep learning.

The Deep Learning approach uses logical structures that are more similar to the organisation of the nervous system of mammals, with layers of processing units (artificial neurons) that specialise in detecting certain characteristics of perceived objects. Deep Learning computational models mimic these architectural features of the nervous system, allowing networks of processing units within the overall system to specialise in detecting certain hidden features in the data.

When our brain's predictions do not accurately predict the future, we become aware of the anomaly and this information is fed back to the brain, which updates its algorithm, allowing it to learn and improve the model.

Source: edureka.co

AI Impact on General Business Processes

Depending on the economic activity sector and its position in the sector value chain, each organisation has a different Process Map.

A Process Map is the structured set of processes that a Company uses to create value. Examples of processes are R&D, Marketing, Financials, Manufacturing, etc.

In each one of those processes, AI could create an improvement. Here we show you some of that processes with some AI standards solutions available on the market.

Product Development & Sourcing

Manufacturing

Demand Planning, Inventory Management & Order Fulfillment

Sales & Marketing

  • Mass product customization
  • Trend prediction
  • Ingredient Discovery
  • DNA sequencing for supply chain traceability
  • Robotic manufacturing
  • Quality control análisis
  • Predictive factory maintenance
  • Demand forecasting
  • Anti-counter feiting
  • Micro-fulfillment
  • Product recommendations
  • Customerv lifetime value modeling
  • Conversational agents and product Discovery
  • Shelf intelligence
  • AI-enabled IoT CPG products

Case study

Source: www.ebir.com

THE CONTEXT:

EBIR bathroom lighting is a Spanish company that sells light fixtures and illuminated bathroom mirrors throughout Europe.

The company uses different brands (EBIR, FOCCO and SPEHO), to sell its products in the various channels in which it works (Industrial Bathroom Furniture Manufacturers, DIY stores and Contract Projects for hotels).

Source: www.ebir.com

To develop its product portfolio EBIR has an R&D department, which has been commissioned to develop a project to obtain up-to-date and actionable knowledge of the markets in which it competes. The EBIR I+D goals are:

Product Portfolio Development

Cover all customer needs and trends with the best product range, achieve industrial protections and promote diversification of risks in product categories, channels and customers.

Market surveillance

Business intelligence through systematic identification of trends, consumer insights, anticipation of new regulations and alert systems for the early detection of threats and opportunities.

Opportunity management

Support for the introduction into new markets, new categories, new clients and assessment on large account customers.

Source: EBIR bathroom lighting I+D

THE PROBLEM:

EBIR has information needs for decision-making that go beyond what its internal data provides:

THE PROBLEM STATEMENT

We need less data and more intelligence.

Knowledge needed:

  • Brand reputation & drill down of S/W’s by topics
  • Business O/T’s (gaps in assortment, price or configuration, replenishment lead efficiency,... )
  • Product & Service benchmarking (availability and lead times for online delivery and store pick & collect)
  • The whole buying experience (from assessment till after sales) including omni-channel product POS display (featured, promoted, procurement classification) and value added services
  • Understanding customer needs or unmet needs by country
  • Pricing analysis
  • Validation of the communication (sales pitch, keywords, images, etc.) by analyzing user contents & keywords and monitoring trends and topics over time, pricing by

Source: EBIR bathroom lighting I+D

THE SOLUTION:

EBIR has decided to implement a solution for capturing, processing and analysing market data based on the web scraping methodology.

Web scraping tools are programs specifically developed to simplify extracting data from websites. Data extraction is quite a helpful and commonly used process; however, it can also quickly become a complicated and messy business and require a great deal of time and effort.

So what does a web scraper do?

A web scraper uses bots to extract structured data and content from a website, extracting the underlying HTML code and data stored in a database.

There are many sub-processes involved in data extraction in data extraction, from preventing your IP from being banned to correctly parsing the source website, generating the data in a compatible format to data cleansing. Fortunately, web scrapers and data extraction tools make this process easy, fast and reliable.

The data pipeline

Source: EBIR bathroom lighting I+D

DATA Analytics

Source: EBIR bathroom lighting I+D

LESSON LEARNT:

You need to apply new data capturing tools and data analysis methodologies to extend the range of your data knowledge boundaries making it possible to answer to daily Marketing Mix questions such as:

  • Mostly rated categories and why
  • People’s choice for each category and why
  • Regarding product configuration, which features are more sensitive to price and why?
  • Top keywords and topics by positive, neutral and negative reviews
  • Extract mostly liked/disliked features by category, brand and topic
  • Quantify the relevance of certain topics of product development for the user
  • Understand the reasons why a product does not succeed

EBIR describes its project lesson learnt as follows:

The pains...

  • Web scraping, data science and ML needs specialized and qualified IT resources
  • No commercial with a holistic approach and those that solve part, are very expensive
  • We have some knowledge, but we have no experience
  • More Budget is required

The gains ...

  • improved & market validated concept design definition for 2 new categories
  • more objective decisions for the selection of projects
  • new opportunities managed with intelligence support help customers understand better proposals

Questions and answers

Q1: In data analysis execution, what should we ensure first, the efficiency or the effectiveness of the result?

A1: If we have to work on efficiency and efficacy, we should always put efficacy first. We must first ensure that we are in the right direction of the solution (effectiveness) before accelerating and/or reducing the effort to achieve the solution (efficiency).

Q2: What are the significant benefits of using RPA (Robot Process Automation) tools?

A2: RPA is a software technology that makes it easy to build, deploy, and manage software robots that emulate human actions interacting with digital systems and software. Like people, software robots can understand what’s on a screen, complete the correct keystrokes, navigate systems, identify and extract data, and perform a wide range of defined actions. But software robots can do it faster and more consistently than people, without the need to get up and stretch or take a coffee break

Q3: What are the essential elements on which a Machine Learning tool is built?

A3: The data set and the algorithms.

References

Ajay K. Agrawal, Avi Goldfarb y Joshua Gans Prediction Machines: The Simple Economics of Artificial Intelligence. Harvard Business Review Press (2018)

https://www.accenture.com/us-en/insights/artificial-intelligence-summary-index

Accenture “What is AI”

https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ai-readiness-model-whitepaper.pdf

Intel “The AI Readiness Model”

https://www.gartner.com/smarterwithgartner/3-barriers-to-ai-adoption

Gartner “Barriers to AI Implementation”

34 ARTIFICIAL INTELLIGENCE COMPANIES BUILDING A SMARTER TOMORROW Alyssa Schroer https://builtin.com/artificial-intelligence/ai-companies-roundup