Programmer's Guide to Apache Thrift

by Randy Abernethy

Overview

Summary

Programmer's Guide to Apache Thrift provides comprehensive coverage of the Apache Thrift framework along with a developer's-eye view of modern distributed application architecture.

Foreword by Jens Geyer.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

About the Technology

Thrift-based distributed software systems are built out of communicating components that use different languages, protocols, and message types. Sitting between them is Thrift, which handles data serialization, transport, and service implementation. Thrift supports many client and server environments and a host of languages ranging from PHP to JavaScript, and from C++ to Go.

About the Book

Programmer's Guide to Apache Thrift provides comprehensive coverage of distributed application communication using the Thrift framework. Packed with code examples and useful insight, this book presents best practices for multi-language distributed development. You'll take a guided tour through transports, protocols, IDL, and servers as you explore programs in C++, Java, and Python. You'll also learn how to work with platforms ranging from browser-based clients to enterprise servers.

What's inside

• Complete coverage of Thrift's IDL
• Building and serializing complex user-defined types
• Plug-in protocols, transports, and data compression
• Creating cross-language services with RPC and messaging systems

About the Reader

Readers should be comfortable with a language like Python, Java, or C++ and the basics of service-oriented or microservice architectures.

About the Author

Randy Abernethy is an Apache Thrift Project Management Committee member and a partner at RX-M.

Table of Contents

1. Introduction to Apache Thrift
2. Apache Thrift architecture
3. Building, testing, and debugging
4. Moving bytes with transports
5. Serializing data with protocols
6. Apache Thrift IDL
7. User-defined types
8. Implementing services
9. Handling exceptions
10. Servers
11. Building clients and servers with C++
12. Building clients and servers with Java
13. Building C# clients and servers with .NET Core and Windows
14. Building Node.js clients and servers
15. Apache Thrift and JavaScript
16. Scripting Apache Thrift
17. Thrift in the enterprise

Product Details

ISBN-13:	9781617296161
Publisher:	Manning
Publication date:	04/14/2019
Edition description:	1st Edition
Pages:	592
Product dimensions:	7.30(w) x 9.20(h) x 1.30(d)

About the Author

Randy Abernethy is an Apache Thrift Project Management Committee member and a partner at RX-M.

Table of Contents

Foreword xv

Preface xvii

Acknowledgments xix

About this book xxi

About the author xxiv

About the rover illustration xxv

Part 1 Apache Thrift Overview 1

1 Introduction to Apache Thrift 3

1.1 Polyglotism, the pleasure and the pain 4

1.2 Application integration with Apache Thrift 6

Type serialization 6

Service implementation 9

1.3 Building a simple service 11

The Hello IDL 11

The Hello sewer 12

A Python client 13

A C++ client 15

A Java client 17

1.4 The communications toolkit landscape 19

SOAP 19

REST 20

Protocol Buffers 21

Apache Avro 22

Strengths of Apache Thrift 23

Takeaway 25

2 Apache Thrift architecture 27

2.1 Transports 28

The transport interface 30

Endpoint transports 30

Layered transports 31

Server transports 32

2.2 Protocols 33

2.3 Apache Thrift IDL 35

User-defined types and serialization 36

RPC services 38

2.4 Servers 41

2.5 Security 42

3 Building, testing, and debugging 44

3.1 Installing the Apache Thrift IDL compiler 45

Platform installers 47

VMs and containers 49

Building from source 51

3.2 The Apache Thrift source tree 51

3.3 Apache Thrift tests 55

3.4 Debugging RPC services 57

Examining packets on the wire 58

Unbuffered interfaces 62

Interface misalignment 65

I/O stack misalignment 68

Instrumenting code 69

Additional techniques 70

Part 2 Programming Apache Thrift 73

4 Moving bytes with transports 75

4.1 Endpoint transports, part 1: Memory & disk 77

Programming with memory transports 78

Programming with file transports 85

4.2 The transport interface 89

Basic transport operations 90

4.3 Endpoint transports, part 2: Networks 91

Network programming with TSocket 91

4.4 Server transports 97

Programming network servers with server transports 98

The server transport interface 100

4.5 Layered transports 101

Message framing 102

5 Serializing data with protocols 106

5.1 Basic serialization with the binary protocol 108

Using the C++ TBinaryProtocol 109

Using the Java TBinaryProtocol 110

Using the Python TBinaryProtocol 111

Takeaway 112

5.2 The TProtocol interface 112

Apache Thrift serialization 113

C++ TProtocol 118

Java TProtocol 120

Python TProtocolBase 121

5.3 Serializing objects 123

Struct serialization 123

Struct de-serialization 126

Struct evolution 128

5.4 TCompactProtocol 130

5.5 TJSONProtocol 132

5.6 Selecting a protocol 133

6 Apache Thrift IDL 139

6.1 Interfaces 139

6.2 Apache Thrift IDL 144

IDL file names 144

Element names 145

Keywords 145

6.3 The IDL compiler 148

Compilation phases and error messages 148

Command line switches 150

6.4 Comments and documentation 158

6.5 Namespaces 160

6.6 Built-in types 161

Base types 161

Container types 162

Literals 165

6.7 Constants 166

C++ interface constant implementation 167

Java interface constant implementation 168

Python interface constant implementation 168

6.8 Typedefs 169

6.9 Enum 169

6.10 Structures, unions, exceptions, and argument-lists 171

Structs 172

Fields 172

Exceptions 177

Unions 177

6.11 Services 178

6.12 Including external files 180

6.13 Annotations 185

7 User-defined types 188

7.1 A simple user-defined type example 190

7.2 Type design 193

Namespaces 196

Constants 196

Structs 196

Base types 197

Typedefs 197

Field IDs and retiring fields 197

Enums 198

Collections 199

Unions 199

Requiredness and optional fields 200

7.3 Serializing objects to disk 201

7.4 Under the type serialization hood 209

Serializing with write() 212

De-serializing with read() 213

7.5 Type evolution 215

Renaming fields 216

Adding fields 216

Deleting fields 217

Changing a field's type 217

Changing a field's requiredness 218

Changing a field's default value 219

7.6 Using Zlib compression 219

Using Zlib with C++ 220

Using Zlib with Python 224

8 Implementing services 227

8.1 Declaring IDL services 229

Parameter identifiers 230

Parameter requiredness 230

Default parameter values 231

Function and parameter types 232

8.2 Building a simple service 233

Interfaces 233

Coding service handlers and test harnesses 234

Coding RFC servers 236

Coding RFC clients 237

8.3 Service interface evolution 239

Adding features to a service 241

8.4 RFC services in depth 245

Under the hood 245

One-way functions 247

Service inheritance 248

Asynchronous clients 254

9 Handling exceptions 257

9.1 Apache Thrift exceptions 258

9.2 TTransportException 259

C++ exception processing 260

Java exception processing 262

Python exception processing 265

Error processing with out exceptions 267

9.3 TProtocolException 270

9.4 TApplicationException 270

9.5 User-defined exceptions 272

User-defined exception IDL example 273

C++ user-defined exception client 273

C++ user-defined exception server 274

Java user-defined exception client 276

Python user-defined exception client 278

Servers 280

10 Servers 280

10.1 Building a simple server from scratch 282

10.2 Using multithreaded servers 287

10.3 Server concurrency models 290

Connection-based processing 291

Task-based processing 293

Multithreading vs. multiprocessing 299

Server summary by language 299

10.4 Using factories 303

Building I/O stacks with factories 304

Processor and handler factories 309

In/out factories 313

Building sewers with custom factories and transports 314

10.5 Server interfaces and event processing 319

TServer 320

TServerEventHandler 320

Building a C++ thread pool sewer with server events 322

10.6 Servers and services 333

Building multiservice servers 334

Building a multiplexed few a threaded selector server 335

Part 3 Apache Thrift Languages 341

11 Building clients and servers with C++ 343

11.1 Setting up Apache Thrift for C++ development 344

Apache Thrift C++ versions and Boost 345

Building Apache Thrift C++ libraries 346

Building Apache Thrift C++ libraries on Windows 352

11.2 A simple client and server 354

The Hello IDL 354

Building a simple C++ client 355

Creating a simple RFC server 358

11.3 C++ transports, protocols, and servers 360

C++ transports 360

C++ protocols 363

Runtime versus compile lime polymorphism 364

C++ servers 366

11.4 The C++ TNonBlockingServer 366

12 Building clients and servers with Jam 381

12.1 Setting up Apache Thrift for Java development 382

Apache Thrift and SLF4J 384

12.2 A simple client and server 384

The Hello IDL 384- Building a simple Java client 385

Creating a simple RTC server 387

Building with Ant 388

Building with Maven 391

12.3 Using Apache Thrift in other JVM languages 396

12.4 Java transports, protocols, and servers 397

Java transports 397

Java protocols 399

Java sewers 399

12.5 Asynchronous Java RPC 400

13 Building C# clients and servers with .NET Core and Windows 410

13.1 Setting up Apache Thrift on Windows 412

13.2 A simple client and server 415

Creating a Visual Studio RFC solution 415

Creating the interface library 417

Creating the RPC sewer 422

Creating the RFC client 424

Testing the RFC application 425

13.3 C# transports, protocols, and servers 426

C# transports 426

C# protocols 428

C# servers 428

13.4 Long polling with named pipes 429

A long polling interface 430

Installing Apache Thrift support through NuGet 431

Creating a named pipe server 434

Budding the long polling sewer 444

Building a named pipe client 444

14 Building Node.js clients and servers 447

14.1 A simple client and server 447

Generating client/server slabs 448

Creating a Node.js server 448

Creating a Node.js client 450

14.2 Q 452

14.3 Node.js servers 454

14.4 Multiplexed services 455

14.5 Apache Thrift IDL and Node.js 458

Creating full-featured IDL handlers 463

Creating a full-featured Node.js client 464

Apache Thrift andJavaScript 468

15 Apache Thrift and JavaScript 468

15.1 Apache Thrift JavaScript quick start 469

15.2 A simple client and server 470

Installing Apache Thrift for JavaScript 470

The Hello World IDL 472

The Hello World Node.js server 473

The Hello World web client 476

Running the Hello World example 478

Node.js HTTP clients 478

15.3 Asynchronous browser client calls 479

15.4 RPC error handling 480

15.5 Browser RPC and jQuery 484

15.6 Apache Thrift and web security 486

Cross Origin Resource Sharing (CORS) 487

Content Security Policy (CSP) 491

X-Frame-Options 495

Transport security 496

15.7 Using the WebSocket transport 500

16 Scripting Apache Thrift 504

16.1 Apache Thrift and Ruby 505

A Ruby server 507

A Ruby client 508

Ruby features 509

16.2 Apache Thrift and PHP 511

A PHP program 512

A PHP Apache Thrift client 514

PHP features 517

16.3 Apache Thrift and Perl 518

16.4 Apache Thrift Perl clients 519

16.5 Apache Thrift Perl servers 521

Apache Thrift Perl features 523

16.6 Apache Thrift and Python 523

17 Thrift in the enterprise 526

17.1 Polyglot systems 527

17.2 Service tooling and considerations 527

Services 527

Interface comparisons 528

17.3 Messaging 539

17.4 Best practices 544

IDL 544

Interface evolution 545

Service design 545

Type design 546

Coding practices 547

Index 549