Machine Learning for Business: Using Amazon SageMaker and Jupyter

by Doug Hudgeon, Richard Nichol

Overview

Summary

• Imagine predicting which customers are thinking about switching to a competitor or flagging potential process failures before they happen
• Think about the benefits of forecasting tedious business processes and back-office tasks
• Envision quickly gauging customer sentiment from social media content (even large volumes of it).
• Consider the competitive advantage of making decisions when you know the most likely future events

Machine learning can deliver these and other advantages to your business, and it’s never been easier to get started!

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the technology

Machine learning can deliver huge benefits for everyday business tasks. With some guidance, you can get those big wins yourself without complex math or highly paid consultants! If you can crunch numbers in Excel, you can use modern ML services to efficiently direct marketing dollars, identify and keep your best customers, and optimize back office processes. This book shows you how.

About the book

Machine Learning for Business teaches business-oriented machine learning techniques you can do yourself. Concentrating on practical topics like customer retention, forecasting, and back office processes, you’ll work through six projects that help you form an ML-for-business mindset. To guarantee your success, you’ll use the Amazon SageMaker ML service, which makes it a snap to turn your questions into results.

What's inside

• Identifying tasks suited to machine learning
• Automating back office processes
• Using open source and cloud-based tools
• Relevant case studies

About the reader

For technically inclined business professionals or business application developers.

About the author

Doug Hudgeon and Richard Nichol specialize in maximizing the value of business data through AI and machine learning for companies of any size.

 

Table of Contents:

PART 1 MACHINE LEARNING FOR BUSINESS

1 ¦ How machine learning applies to your business

PART 2 SIX SCENARIOS: MACHINE LEARNING FOR BUSINESS

2 ¦ Should you send a purchase order to a technical approver?

3 ¦ Should you call a customer because they are at risk of churning?

4 ¦ Should an incident be escalated to your support team?

5 ¦ Should you question an invoice sent by a supplier?

6 ¦ Forecasting your company’s monthly power usage

7 ¦ Improving your company’s monthly power usage forecast

PART 3 MOVING MACHINE LEARNING INTO PRODUCTION

8 ¦ Serving predictions over the web

9 ¦ Case studies

Product Details

ISBN-13:	9781638353973
Publisher:	Manning
Publication date:	12/24/2019
Sold by:	SIMON & SCHUSTER
Format:	eBook
Pages:	280
File size:	5 MB

About the Author

Doug Hudgeon runs a business automation consultancy, putting his considerable experience helping companies set up automation and machine learning teams to good use. In 2000, Doug launched one of Australia's first electronic invoicing automation companies.

Richard Nichol has over 20 years of experience as a data scientist and software engineer. He currently specializes in maximizing the value of data through AI and machine learning techniques.

Table of Contents

Preface xiii

Acknowledgments xv

About this book xvii

About the authors xx

About the cover illustration xxi

Part 1 Machine Learning for Business 1

1 How machine learning applies to your business 3

1.1 Why are our business systems so terrible? 4

1.2 Why is automation important now? 8

What is productivity 9

How will machine learning improve productivity 9

1.3 How do machines make decisions? 10

People: Rules-based or not? 10

Can you trust a pattern-based answer? 11

How can machine learning improve your business systems? 12

1.4 Can a machine help Karen make decisions? 12

Target variables 13

Features 13

1.5 How does a machine learn? 14

1.6 Getting approval in your company to use machine learning to make decisions 17

1.7 The tools 18

What are AWS and SageMaker, and how can they help you? 18

What is a Jupyter notebook? 19

1.8 Setting up SageMaker in preparation for tackling the scenarios in chapters 2 through 7 19

1.9 The time to act is now 20

Part 2 Six Scenarios: Machine Learning for Business 23

2 Should you send a purchase order to a technical approver? 25

2.1 The decision 26

2.2 The data 27

2.3 Putting on your training wheels 28

2.4 Running the Jupyter notebook and making predictions 29

Part 1 Loading and examining the data 32

Part 2 Getting the data into the right shape 36

Part 3 Creating training, validation, and test datasets 39

Part 4 Training the model 41

Part 5 Hosting the model 43

Part 6 Testing the model 44

2.5 Deleting the endpoint and shutting down your notebook instance 46

Deleting the endpoint 46

Shutting down the notebook instance 47

3 Should you call a customer because they are at risk of churning? 49

3.1 What are you making decisions about? 50

3.2 The process flow 50

3.3 Preparing the dataset 52

Transformation 1 Normalizing the data 53

Transformation 2 Calculating the change from week to week 54

3.4 XGBoost primer 54

How XGBoost works 54

How the machine learning model determines whether the function is getting better or getting worse AUC 57

3.5 Getting ready to build the model 58

Uploading a dataset to S3 59

Setting up a notebook on SageMaker 60

3.6 Building the model 61

Part 1 Loading and examining the data 62

Part 2 Getting the data into the right shape 65

Part 3 Creating training, validation, and test datasets 65

Part 4 Training the model 67

Part 5 Hosting the model 70

Part 6 Testing the model 70

3.7 Deleting the endpoint and shutting down your notebook instance 73

Deleting the endpoint 73

Shutting down the notebook instance 74

3.8 Checking to make sure the endpoint is deleted 74

4 Should an incident be escalated to your support team? 76

4.1 What are you making decisions about? 77

4.2 The process flow 77

4.3 Preparing the dataset 78

4.4 NLP (natural language processing) 79

Creating word vectors 80

Deciding how many words to include in each group 82

4.5 What is BlazingText and how does it work? 83

4.6 Getting ready to build the model 84

Uploading a dataset to S3 85

Setting up a notebook on SageMaker 86

4.7 Building the model 86

Part 1 Loading and examining the data 87

Part 2 Getting the data into the right shape 90

Part 3 Creating training and validation datasets 93

Part 4 Training the model 93

Part 5 Hosting the model 95

Part 6 Testing the model 96

4.8 Deleting the endpoint and shutting down your notebook instance 97

Deleting the endpoint 97

Shutting down the notebook instance 97

4.9 Checking to make sure the endpoint is deleted 97

5 Should you question an invoice sent by a supplier? 99

5.1 What are you making decisions about? 100

5.2 The process flow 101

5.3 Preparing the dataset 103

5.4 What are anomalies 104

5.5 Supervised vs. unsupervised machine learning 105

5.6 What is Random Cut Forest and how does it work? 106

Sample 1 106

Sample 2 109

5.7 Getting ready to build the model 114

Uploading a dataset to S3 114

Setting up a notebook on SageMaker 115

5.8 Building the model 115

Part 1 Loading and examining the data 116

Part 2 Getting the data into the right shape 120

Part 3 Creating training and validation datasets 121

Part 4 Training the model 121

Part 5 Hosting the model 122

Part 6 Testing the model 123

5.9 Deleting the endpoint and shutting down your notebook instance 126

Deleting the endpoint 126

Shutting down the notebook instance 126

5.10 Checking to make sure the endpoint is deleted 126

6 Forecasting your company's monthly power usage 128

6.1 What are you making decisions about? 129

Introduction to time-series data 130

Kiara's time-series data: Daily power consumption 132

6.2 Loading the Jupyter notebook for working with time-series data 133

6.3 Preparing the dataset: Charting time-series data 134

Displaying columns of data with a loop 137

Creating multiple charts 138

6.4 What is a neural network? 139

6.5 Getting ready to build the model 140

Uploading a dataset to S3 141

Setting up a notebook on SageMaker 141

6.6 Building the model 141

Part 1 Loading and examining the data 142

Part 2 Getting the data into the right shape 144

Part 3 Creating training and testing datasets 147

Part 4 Training the model 150

Part 5 Hosting the model 152

Part 6 Making predictions and plotting results 153

6.7 Deleting the endpoint and shutting down your notebook instance 158

Deleting the endpoint 158

Shutting down the notebook instance 158

6.8 Checking to make sure the endpoint is deleted 159

7 Improving your company's monthly power usage forecast 161

7.1 DeepAR's ability to pick up periodic events 161

7.2 DeepAR's greatest strength: Incorporating related time series 163

7.3 Incorporating additional datasets into Kiara's power consumption model 164

7.4 Getting ready to build the model 165

Downloading the notebook we prepared 165

Setting up the folder on SageMaker 166

Uploading the notebook to SageMaker 166

Downloading the datasets from the S3 bucket 166

Setting up a folder on S3 to hold your data 167

Uploading the datasets to your AWS bucket 167

7.5 Building the model 168

Part 1 Setting up the notebook 168

Part 2 Importing the datasets 169

Part 3 Getting the data into the right shape 170

Part 4 Creating training and test datasets 172

Part 5 Configuring the model and setting up the sewer to build the model 175

Part 6 Making predictions and plotting results 179

7.6 Deleting the endpoint and shutting down your notebook instance 182

Deleting the endpoint 182

Shutting down the notebook instance 183

7.7 Checking to make sure the endpoint is deleted 183

Part 3 Moving Machine Learning into Production 185

8 Serving predictions over the web 187

8.1 Why is serving decisions and predictions over the web so difficult? 188

8.2 Overview of steps for this chapter 189

8.3 The SageMaker endpoint 189

8.4 Setting up the SageMaker endpoint 190

Uploading the notebook 191

Uploading the data 193

Running the notebook and creating the endpoint 194

8.5 Setting up the serverless API endpoint 197

Setting up your AWS credentials on your AWS account 198

Setting up your AWS credentials on your local computer 199

Configuring your credentials 200

8.6 Creating the web endpoint 201

Installing Chalice 202

Creating a Hello World API 204

Adding the code that serves the SageMaker endpoint 205

Configuring permissions 207

Updating requirements.txt 207

Deploying Chalice 208

8.7 Serving decisions 208

9 Case studies 211

9.1 Case study 1: WorkPac 212

Designing the project 214

Stage 1 Preparing and testing the model 215

Stage 2 Implementing proof of concept (POC) 216

Stage 3 Embedding the process into the company's operations 216

Next steps 217

Lessons learned 217

9.2 Case study 2: Faethm 217

AI at the core 217

Using machine learning to improve processes at Faethm 217

Stage 1 Getting the data 219

Stage 2 Identifying the features 220

Stage 3 Validating the results 220

Stage 4 Implementing in production 220

9.3 Conclusion 220

Perspective 1 Building trust 221

Perspective 2 Geting the data right 221

Perspective 3 Designing your operating model to make the most of your machine learning capability 221

Perspective 4 What does your company look like once you are using machine learning everywhere? 221

Appendix A Signing up for Amazon AWS 222

Appendix B Setting up and using S3 to store files 229

Appendix C Setting up and using AWS SageMaker to build a machine learning system 238

Appendix D Shutting it all down 243

Appendix E Installing Python 247

Index 249