One of the first things that one encounters when studying Machine Learning is a barrage of terms like inferential statistics, statistical test, statistical hypothesis, null hypothesis, alternative hypothesis, p-value, probability distribution…the list goes on and on.
This may prove discouraging for those who are not familiar with Statistics. The present post aims to provide a gentle introduction to all those terms and concepts.
In many of my previous articles, I have posted about Association Rule Learning, what it’s about, and how it is performed. In this article, we are going to use Association Rule Learning to actually see it in action, and for this purpose, we are going to use KSAI, a machine learning library purely written in Scala. So, let’s begin.
You can add KSAI in your SBT project using the following import line:
Of all of the technologies that underpin the 4th Industrial Revolution, AI has undoubtedly received the most hype in terms of its ability to transform the workplace. Breathless tomes have predicted the demise of millions of jobs as automated systems frogmarch their way through workplace after workplace.
Except the reality is somewhat different. A recent report on the state of digital transformation by MIT Sloan Management Review and Deloitte Digital suggests that progress is glacial. This is despite executives telling INSEAD researchers that AI and big data have the biggest potential for disruptive change.
In a classical prediction use case, the predicted output is either a number (for regression) or category (for classification). A set of training data (x, y) where x is the input and y is the labeled output is provided to train a parameterized predictive model.
This problem is typically viewed as an optimization problem and uses a gradient descent approach to solve it.
This post represents views on why Machine Learning systems or models are termed as non-testable from quality control/quality assurance perspectives. Before I proceed, let me humbly state that data scientists and the Machine Learning community have been saying that ML models are testable as they are first trained and then tested using techniques such as cross-validation etc. based on different techniques to increase the model performance and optimize the model. However, "testing" the model is referred with the scenario during the development (model building) phase when data scientists test the model performance by comparing the model outputs (predicted values) with the actual values. This is not the same as testing the model for any given input for which the output (expected) value is not known beforehand. In this post, I am rather talking about ML models testability from the overall traditional software testing perspective.
Given that Machine Learning systems are non-testable, it can be said that performing QA or quality control checks on Machine Learning systems is not easy, and, thus, a matter of concern given the trust, the end-users need to have on such systems. Project stakeholders must need to understand the non-testability aspects of Machine Learning systems in order to put appropriate quality controls in place to serve trustable Machine Learning models to end users in production. This applies greatly to healthcare and financial systems where a couple of false negatives or type-II error could lead to havoc or troubles for the stakeholders.
While previous years have seen an awful lot of discussion given to the potential of AI-based technology, this year has, thus far, seen a tremendous amount of effort given over to the ethical development of AI.
Very much in keeping with this trend was a report, published earlier this year, by the UK’s House of Lords, which made a number of principles around which they urge the development of AI to revolve:
Data that has been collected, collated, and cleansed is ripe for analysis and insight generation. Advances in Machine Learning and AI are helping deliver on the promises of augmented analytics to produce actionable insights. Pairing Machine Learning techniques with prepared data enables organizations to achieve more accurate predictions and measurable analysis on all kinds of business functions.
A growing number of BI and Analytics tools vendors are responding to the need for augmented BI by opening their platforms through APIs and making stored data more easily accessible. This is a critical first step that gives IT the ability to build connections from Machine Learning products to raw, cleaned, and prepared data.
Everything is derivative. Take advantage of that. "New" ideas are the next step in an extensive network of existing people and ideas. If we can get the data and reconstruct the network, we can analyze it and understand where the branches of a network have the potential for innovation.
Great ideas do not need to be created. They can be discovered.
As AI has taken on ever greater importance in the priority of organizations around the world, it is understandable that efforts are underway to protect the intellectual property of algorithms that have strategic importance.
A recent paper from IBM Research highlights one strategy being worked on to provide this protection. Their approach takes inspiration from the digital watermarking that helps to protect video, audio and photos.
In this post, you will learn about how metamorphic testing could be used for performing quality control checks/testing on Machine Learning models. It is primarily meant for data science specialists to plan the test cases to test the Machine Learning (ML) model implementation from a QA perspective.
Testing Machine Learning models from a quality assurance perspective is different from testing Machine Learning models for accuracy/performance.
