Last time, we looked at how to leverage the SAP HANA R integration, which opens the door to about 11,000 packages. So, if you feel like the built-in libraries (APL and PAL) don’t offer what you need or if you feel like doing something your way too, now you can!
I hope you all managed to try this out, and probably some of you already started comparing the PAL implementation with R algorithms. Feel free to share your feedback!
In this post, we will try and understand the concepts behind evaluation metrics such as sensitivity and specificity, which is used to determine the performance of the Machine Learning models. The post also describes the differences between sensitivity and specificity. The concepts have been explained using the model for predicting whether a person is suffering from a disease or not.
Sensitivity is a measure of the proportion of actual positive cases that got predicted as positive (or true positive). Sensitivity is also termed as Recall. This implies that there will be another proportion of actual positive cases, which would get predicted incorrectly as negative (and, thus, could also be termed as the false negative). This can also be represented in the form of a false negative rate. The sum of sensitivity and false negative rate would be 1. Let’s try and understand this with the model used for predicting whether a person is suffering from the disease. Sensitivity is a measure of the proportion of people suffering from the disease who got predicted correctly as the ones suffering from the disease. In other words, the person who is unhealthy actually got predicted as unhealthy.
R Data science includes data analysis. It is an important component of the skill set required for many jobs in this area. But it’s not the only necessary skill. They play active roles in the design and implementation work of four related areas:
Learn more about Data Science.
Last time, we looked at how to import data in SAP HANA express, and we used the dataset provided by the SAP Predictive Analytics tools (and available online).
But the main idea was to show you how you can import more or less any kind of text/CSV files in your HXE instances.
Established in 2018, the Machine Intelligence Technology Laboratory comprises of a group of outstanding scientists and engineers, with research centers located in Hangzhou, Beijing, Seattle, Silicon Valley, and Singapore. Machine Intelligence Technology Laboratory is Alibaba’s core team responsible for the research and development of artificial intelligence technologies. Relying on Alibaba’s valuable massive data and machine learning/deep learning technologies, the lab has developed image recognition, speech interaction, natural language understanding, intelligent decision-making, and other core artificial intelligence technologies. It fully empowers Alibaba Group’s important businesses such as e-commerce, finance, logistics, social interaction, and entertainment, and also provides outputs to ecosystem partners to jointly build a smart future.
If you’ve spent any time job hunting, you’ll no doubt be well aware of how frustrating and hopeless the task can seem at times.
The haphazardous process of getting yourself noticed in a sea of applications that may be prioritized despite carrying less relevance is tricky enough. But then you may find that when you eventually do accept a role, it’s nothing like you were led to believe it would be.
One of the first things that one encounters when studying Machine Learning is a barrage of terms like inferential statistics, statistical test, statistical hypothesis, null hypothesis, alternative hypothesis, p-value, probability distribution…the list goes on and on.
This may prove discouraging for those who are not familiar with Statistics. The present post aims to provide a gentle introduction to all those terms and concepts.
This post represents views on why Machine Learning systems or models are termed as non-testable from quality control/quality assurance perspectives. Before I proceed, let me humbly state that data scientists and the Machine Learning community have been saying that ML models are testable as they are first trained and then tested using techniques such as cross-validation etc. based on different techniques to increase the model performance and optimize the model. However, "testing" the model is referred with the scenario during the development (model building) phase when data scientists test the model performance by comparing the model outputs (predicted values) with the actual values. This is not the same as testing the model for any given input for which the output (expected) value is not known beforehand. In this post, I am rather talking about ML models testability from the overall traditional software testing perspective.
Given that Machine Learning systems are non-testable, it can be said that performing QA or quality control checks on Machine Learning systems is not easy, and, thus, a matter of concern given the trust, the end-users need to have on such systems. Project stakeholders must need to understand the non-testability aspects of Machine Learning systems in order to put appropriate quality controls in place to serve trustable Machine Learning models to end users in production. This applies greatly to healthcare and financial systems where a couple of false negatives or type-II error could lead to havoc or troubles for the stakeholders.
Data that has been collected, collated, and cleansed is ripe for analysis and insight generation. Advances in Machine Learning and AI are helping deliver on the promises of augmented analytics to produce actionable insights. Pairing Machine Learning techniques with prepared data enables organizations to achieve more accurate predictions and measurable analysis on all kinds of business functions.
A growing number of BI and Analytics tools vendors are responding to the need for augmented BI by opening their platforms through APIs and making stored data more easily accessible. This is a critical first step that gives IT the ability to build connections from Machine Learning products to raw, cleaned, and prepared data.
In this post, you will learn about how metamorphic testing could be used for performing quality control checks/testing on Machine Learning models. It is primarily meant for data science specialists to plan the test cases to test the Machine Learning (ML) model implementation from a QA perspective.
Testing Machine Learning models from a quality assurance perspective is different from testing Machine Learning models for accuracy/performance.
