Practical Automated Machine Learning on Azure: Using Azure Machine Learning to Quickly Build AI Solutions

by Deepak Mukunthu, Parashar Shah, Wee Hyong Tok

Overview

Develop smart applications without spending days and weeks building machine-learning models. With this practical book, you'll learn how to apply Automated Machine Learning, a process that uses machine learning to help people build machine learning models. Deepak Mukunthu, Parashar Shah, and Wee Hyong Tok provide a mix of technical depth, hands-on examples, and case studies that show how customers are solving real-world problems with this technology.

Building machine learning models is an iterative and time-consuming process. Even those who know how to create these models may be limited in how much they can explore. Once you complete this book, you'll understand how to apply Automated Machine Learning to your data right away.

• Learn how companies in different industries are benefiting from Automated Machine Learning
• Get started with Automated Machine Learning using Azure
• Explore aspects such as algorithm selection, auto featurization, and hyperparameter tuning
• Understand how data analysts, BI professionals, and developers can use Automated Machine Learning in their familiar tools and experiences
• Learn how to get started using Automated Machine Learning for use cases including classification and regression.

Product Details

ISBN-13:	9781492055594
Publisher:	O'Reilly Media, Incorporated
Publication date:	10/08/2019
Pages:	196
Product dimensions:	6.90(w) x 9.10(h) x 0.50(d)

About the Author

Deepak Mukunthu is a product leader with more than 16 years of experience. With his experience in big data, analytics, and AI, Deepak has played instrumental leadership roles in helping organizations and teams become data-driven and to adopt machine learning. He brings a good mix of thought leadership, customer understanding, and innovation to design and deliver compelling products that resonate well with customers. In his current role of principal program manager of the automated ML in Azure AI platform group at Microsoft, Deepak drives product strategy and roadmap for Automated ML with the goal of accelerating AI for data scientists and democratizing AI for other personas interested in machine learning. In addition to shaping the product direction, he also plays an instrumental role in helping customers adopt Automated ML for their business-critical scenarios. Prior to joining Microsoft, Deepak worked at Trilogy where he played multiple roles—consultant, business development, program manager, engineering manager—successfully leading distributed teams across the globe and managing technical integration of acquisitions.

Parashar Shah is a senior program/product manager on the Azure AI engineering team at Microsoft, leading big data and deep learning projects to help increase adoption of AI in enterprises especially automated ML with Spark. At Microsoft and at Alcatel-Lucent/Bell Labs prior to that, his contributions increased global adoption of AI/analytics platform contributing to customers' growth in retail, manufacturing, telco, and oil and gas verticals. Parashar has an MBA from the Indian Institute of Management Bangalore and a B.E. (E.C.) from Nirma Institute of Technology, Ahmedabad. He also cofounded a carpool startup in India. He has also coauthored Hands-On Machine Learning with Azure: Build Powerful Models with Cognitive Machine Learning and Artificial Intelligence (Packt), published in November 2018. He has filed for five patents. He has presented at multiple Microsoft and external conferences, including Spark summit and KDD. His interests span the subjects of photography, AI, machine learning, automated ML, big data, and the internet of things (IoT).

Wee Hyong Tok is part of the AzureCAT team at Microsoft. He has extensive leadership experience leading multidisciplinary team of engineers and data scientists, working on cutting-edge AI capabilities that are infused into products and services. He is a tech visionary with a background in product management, machine learning/deep learning and working on complex engagements with customers. Over the years, he has demonstrated that his early thought leadership whitepapers on tech trends have become reality, and deeply integrated into many products. His ability to strategize, and turn strategy to execution, and hunting for customer adoption has enabled many projects that he works on to be successful. He is continuously pushing the boundaries of products for machine learning and deep learning. His team works extensively with deep learning frameworks, ranging from TensorFlow, CNTK, Keras, and PyTorch. Wee Hyong has worn many hats in his career—developer, program/product manager, data scientist, researcher, and strategist—and his range of experience has given him unique superpowers to lead and define the strategy for high-performing data and AI innovation teams. Throughout his career, he has been a trusted advisor to the C-suite, from Fortune 500 companies to startups.

Table of Contents

Foreword ix

Preface xi

Part I Automated Machine Learning

1 Machine Learning: Overview and Best Practices 1

Machine Learning: A Quick Refresher 2

Model Parameters 4

Hyperparameters 5

Best Practices for Machine Learning Projects 5

Understand the Decision Process 5

Establish Performance Metrics 6

Focus on Transparency to Gain Trust 7

Embrace Experimentation 8

Don't Operate in a Silo 8

An Iterative and Time-Consuming Process 10

Feature Engineering 11

Algorithm Selection 12

Hyperparameter Tuning 12

The End-to-End Process 13

Growing Demand 15

Conclusion 17

2 How Automated Machine Learning Works 19

What Is Automated Machine Learning? 19

Understanding Data 19

Detecting Tasks 21

Choosing Evaluation Metrics 23

Feature Engineering 23

Selecting a Model 26

Monitoring and Retraining 30

Bringing It All Together 30

Automated ML 31

How Automated ML Works 31

Preserving Privacy 32

Enabling Transparency 33

Guardrails 34

End-to-End Model Life-Cycle Management 34

Conclusion 35

Part II Automated ML on Azure

3 Getting Started with Microsoft Azure Machine Learning and Automated ML 39

The Machine Learning Process 39

Collaboration and Monitoring 40

Deployment 40

Setting Up an Azure Machine Learning Workspace for Automated ML 41

Azure Notebooks 48

Notebook VM 57

Conclusion 58

4 Feature Engineering and Automated Machine Learning 59

Data Preprocessing Methods Available in Automated ML 61

Auto-Featurization for Automated ML 61

Auto-Featurization for Classification and Regression 64

Auto-Featurization for Time-Series Forecasting 69

Conclusion 74

5 Deploying Automated Machine Learning Models 75

Deploying Models 75

Registering the Model 77

Creating the Container Image 80

Deploying the Model for Testing 84

Testing a Deployed Model 88

Deploying to AKS 89

Swagger Documentation for the Web Service 91

Debugging a Deployment 92

Web Service Deployment Fails 93

Conclusion 95

6 Classification and Regression 97

What Is Classification and Regression? 97

Classification and Regression Algorithms 100

Using Automated ML for Classification and Regression 101

Conclusion 116

Part III How Enterprises Are Using Automated Machine Learning

7 Model Interpretability and Transparency with Automated ML 119

Model Interpretability 119

Model Interpretability with Azure Machine Learning 121

Model Transparency 129

Understanding the Automated ML Model Pipelines 129

Guardrails 130

Conclusion 131

8 Automated ML for Developers 133

Azure Databricks and Apache Spark 133

ML.NET 147

SQL Server 149

Conclusion 149

9 Automated ML for Everyone 151

Azure Portal UI 152

Power BI 161

Preparing the Data 161

Automated ML Training 163

Understanding the Best Model 166

Understanding the Automated ML Training Process 169

Model Deployment and Inferencing 170

Enabling Collaboration 170

Azure Machine Learning to Power BI 170

Power BI Automated ML to Azure Machine Learning 172

Conclusion 173

Index 175