Deep Learning for Search

by Tommaso Teofili

Overview

Summary

Deep Learning for Search teaches you how to improve the effectiveness of your search by implementing neural network-based techniques. By the time you're finished with the book, you'll be ready to build amazing search engines that deliver the results your users need and that get better as time goes on!

Foreword by Chris Mattmann.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the Technology

Deep learning handles the toughest search challenges, including imprecise search terms, badly indexed data, and retrieving images with minimal metadata. And with modern tools like DL4J and TensorFlow, you can apply powerful DL techniques without a deep background in data science or natural language processing (NLP). This book will show you how.

About the Book

Deep Learning for Search teaches you to improve your search results with neural networks. You'll review how DL relates to search basics like indexing and ranking. Then, you'll walk through in-depth examples to upgrade your search with DL techniques using Apache Lucene and Deeplearning4j. As the book progresses, you'll explore advanced topics like searching through images, translating user queries, and designing search engines that improve as they learn!

What's inside

• Accurate and relevant rankings
• Searching across languages
• Content-based image search
• Search with recommendations

About the Reader

For developers comfortable with Java or a similar language and search basics. No experience with deep learning or NLP needed.

About the Author

Tommaso Teofili is a software engineer with a passion for open source and machine learning. As a member of the Apache Software Foundation, he contributes to a number of open source projects, ranging from topics like information retrieval (such as Lucene and Solr) to natural language processing and machine translation (including OpenNLP, Joshua, and UIMA).

He currently works at Adobe, developing search and indexing infrastructure components, and researching the areas of natural language processing, information retrieval, and deep learning. He has presented search and machine learning talks at conferences including BerlinBuzzwords, International Conference on Computational Science, ApacheCon, EclipseCon, and others. You can find him on Twitter at @tteofili.

Table of Contents

PART 1 - SEARCH MEETS DEEP LEARNING
1. Neural search
2. Generating synonyms
PART 2 - THROWING NEURAL NETS AT A SEARCH ENGINE
3. From plain retrieval to text generation
4. More-sensitive query suggestions
5. Ranking search results with word embeddings
6. Document embeddings for rankings and recommendations
PART 3 - ONE STEP BEYOND
7. Searching across languages
8. Content-based image search
9. A peek at performance

Product Details

ISBN-13:	9781638356271
Publisher:	Manning
Publication date:	06/02/2019
Sold by:	SIMON & SCHUSTER
Format:	eBook
Pages:	328
File size:	7 MB

About the Author

Tommaso Teofili is a software engineer with a passion for open source and machine learning. As a member of the Apache Software Foundation, he contributes to a number of open source projects, ranging from topics like information retrieval (such as Lucene and Solr) to natural language processing and machine translation (including OpenNLP, Joshua, and UIMA).

He currently works at Adobe, developing search and indexing infrastructure components, and researching the areas of natural language processing, information retrieval, and deep learning. He has presented search and machine learning talks at conferences including BerlinBuzzwords, International Conference on Computational Science, ApacheCon, EclipseCon, and others. You can find him on Twitter at @tteofili.
Tommaso Teofili is a software engineer at Adobe Systems with a passion for open source and artificial intelligence. He is a long-time member of the Apache Software Foundation, where he contributes to many projects on topics like information retrieval, natural language processing, and distributed computing.

Table of Contents

Foreword xiii

Preface xv

Acknowledgments xvii

About this book xviii

About the author xxi

About the cover illustration xxii

Part 1 Search Meets Deep Learning 1

1 Neural search 3

1.1 Neural networks and deep learning 4

1.2 What is machine learning? 7

1.3 What deep learning can do for search 9

1.4 A roadmap for learning deep learning 12

1.5 Retrieving useful information 13

Text, tokens, terms, and search fundamentals 14

Relevance first 22

Classic retrieval models 23

Precision and recall 24

1.6 Unsolved problems 24

1.7 Opening the search engine black box 25

1.8 Deep learning to the rescue 27

1.9 Index, please meet neuron 31

1.10 Neural network training 31

1.11 The promises of neural search 34

2 Generating synonyms 36

2.1 Introduction to synonym expansion 37

Why synonyms? 39

Vocabulary-based synonym matching 40

2.2 The importance of context 49

2.3 Feed-forward neural networks 51

2.4 Using word2vec 54

Setting up word2vec in Deeplearning4j 63

Word2vec-based synonym expansion 64

2.5 Evaluations and comparisons 66

2.6 Considerations for production systems 67

Synonyms vs. antonyms 69

Part 2 Throwing Neural Nets at a Search Engine 73

3 From plain retrieval to text generation 75

3.1 Information need vs. query: Bridging the gap 77

Generating alternative queries 77

Data preparation 79

Wrap-up of generating data 86

3.2 Learning over sequences 86

3.3 Recurrent neural networks 88

RNN internals and dynamics 91

Long-term dependencies 94

Long short-term memory networks 95

3.4 LSTM networks for unsupervised text generation 95

Unsupervised query expansion 103

3.5 From unsupervised to supervised text generation 106

Sequence-to-sequence modeling 107

3.6 Considerations for production systems 110

4 More-sensitive query suggestions 112

4.1 Generating query suggestions 113

Suggesting while composing queries 113

Dictionary-based suggesters 114

4.2 Lucene Lookup APIs 115

4.3 Analyzed suggesters 118

4.4 Using language models 124

4.5 Content-based suggesters 128

4.6 Neural language models 129

4.7 Character-based neural language model for suggestions 131

4.8 Tuning the LSTM language model 134

4.9 Diversifying suggestions using word embeddings 142

5 Ranking search results with word embeddings 146

5.1 The importance of ranking 147

5.2 Retrieval models 149

TF-IDF and the vector space model 151

Ranking documents in Lucene 154

Probabilistic models 156

5.3 Neural information retrieval 158

5.4 From word to document vectors 159

5.5 Evaluations and comparisons 165

Similarity based on averaged word embeddings 166

6 Document embeddings for rankings and recommendations 170

6.1 From word to document embeddings 171

6.2 Using paragraph vectors in ranking 174

Paragraph vector-based similarity 177

6.3 Document embeddings and related content 177

Search, recommendations, and related content 178

Using frequent terms to find similar content 179

Retrieving similar content with paragraph vectors 188

Relieving similar content with vectors from encoder-decoder models 191

Part 3 One Step Beyond 192

7 Searching across languages 195

7.1 Serving users who speak multiple languages 196

Translating documents vs. queries 197

Cross-language search 199

Querying in multiple languages on top of Lucene 200

7.2 Statistical machine translation 202

Alignment 204

Phrase-based translation 205

7.3 Working with parallel corpora 206

7.4 Neural machine translation 208

Encoder-decoder models 209

Encoder-decoder for MT in DL4J 212

7.5 Word and document embeddings for multiple languages 219

Linear projected monolingual embeddings 219

8 Content-based image search 225

8.1 Image contents and search 227

8.2 A look back: Text-based image retrieval 229

8.3 Understanding images 231

Image representations 233

Feature extraction 235

8.4 Deep learning for image representation 243

Convolutional neural networks 245

Image search 253

Locality-sensitive hashing 258

8.5 Working with unlabeled images 261

9 A peek at performance 267

9.1 Performance and the promises of deep learning 268

From model design to production 269

9.2 Indexes and neurons working together 284

9.3 Working with streams of data 287

Index 295