MySQL Database Design and Tuning

by Robert Schneider

Overview

The authoritative, hands-on guide to advanced MySQL programming and administration techniques for high performance is here. MySQL Database Design and Tuning is the only guide with coverage of both the basics and advanced topics, including reliability, performance, optimization and tuning for MySQL. This clear, concise and unique source for the most reliable MySQL performance information will show you how to:

• Deploy the right MySQL product for your performance needs.
• Set up a performance management and monitoring environment using tools from MySQL.
• Implement the right indexing strategy
• Apply good performance strategy when developing software to work with the MySQL database.
• Configure dozens of variable to correctly tune the MySQL engine.

If you deal with the intricacies and challenges of advanced MySQL functionality on a daily basis, you will be able to build on your knowledge with author Robert Schneider's real-world experiences in MySQL Database Design and Tuning.

Product Details

ISBN-13:	9780672332692
Publisher:	Pearson Education
Publication date:	06/13/2005
Series:	Developer's Library
Sold by:	Barnes & Noble
Format:	eBook
Pages:	432
File size:	8 MB
Age Range:	18 Years

About the Author

MySQL® Database Design and TuningAbout the Author

Robert D. Schneider has more than 15 years of experience developing and delivering sophisticated software solutions worldwide. He has provided database optimization, distributed computing, and other technical expertise to a wide variety of enterprises in the financial, technology, and government sectors. Clients have included Chase Manhattan Bank, VISA, HP, SWIFT, and the governments of the United States, Brazil, and Malaysia.

He is the author of Optimizing Informix Applications and Microsoft SQL Server: Planning and Building a High Performance Database. He has also written numerous articles on technical and professional services topics. He can be reached at [e-mail].

Table of Contents

The Purpose of This Book.

    Who Should Read This Book?

    Benefits of Using This Book

    How to Use This Book

    Information About MySQL Software and Environments

    Information About Examples

I. INTRODUCTION.

1. Setting Up an Optimization Environment.

    Pretesting Preparation

    Hardware

    Connectivity

    Software

    Data

    Your Test Plan

    Change One Variable at a Time

    Organizational Support

    After Testing Is Under Way

    Posttesting Implementation

    Recording Your Results

    Making Improvements

2. Performance Monitoring Options.

    MySQL Tools and Utilities

    Character-Based Tools and Utilities

    Graphical Tools

    Logging

    Operating System Tools

    Windows

    Linux/Unix

3. MySQL Overview and New Performance-Related Features.

    MySQL Products

    MySQL Core Technologies

    Database Storage Engines and Table Types

    Distributed Computing Technologies

    Graphical Tools and Assistants

    Connectors

    APIs

    Utilities

    Performance-Related Tools

    About the Versions Used for This Book

    MySQL Versions

    Operating Systems

    Performance-Boosting Features from Version 4.0 Onward

II. ADVANCED DATABASE DESIGN CONCEPTS.

4. Designing for Speed.

    Choosing the Right Storage Engine and Table Type

    InnoDB

    MyISAM

    MEMORY

    MERGE

    CSV

    ARCHIVE

    BerkeleyDB (BDB)

    MaxDB

    NDB

    FEDERATED

    Optimizing Table Structure

    Specifying Row Format

    Specifying Table Size

    Index Key Compression

    Checksum Integrity Management

    Column Types and Performance

    Using Views to Boost Performance

    Reduced Returned Columns

    Reduced Returned Rows

    Reduced Inefficient Joins

    Simpler Updating with Integrity Checking

    When Not to Normalize

5. Using Constraints to Improve Performance.

    UNIQUE Constraints

    PRIMARY KEY

    FOREIGN KEY

    DEFAULT and NOT NULL

    ENUM

    SET

6. Understanding the MySQL Optimizer.

    What Is an Optimizer?

    Optimizer Diagnostics and Tuning

    The ANALYZE TABLE Command

    The OPTIMIZE TABLE Command

    The EXPLAIN Command

    Helping the Optimizer

7. Indexing Strategies.

    Index Key Terms and Concepts

    Index Reports and Utilities

    SHOW INDEX

    Assisting the Optimizer with Indexes

    Index-Related Logging

    MyISAM Indexing Features

    Key Cache

    The myisamchk Utility

    Index and Table Compression

    InnoDB Indexing Features

    Index Structure and Storage

    Buffer Pool

    Memory Pool

    Adaptive Hash Index

    Automatic Foreign Key Index Generation

    Indexing Scenarios

    PRIMARY KEY

    Filter Columns

    Join Columns

    Index Cardinality

    Character Versus Numeric Indexes

    Multicolumn Indexes

    Partial Indexes

    Ascending Versus Descending Indexes

    Storing Tables in Column-Sorted Order

III. OPTIMIZING APPLICATION CODE.

8. Advanced SQL Tips.

    Improving Searches

    Leveraging Internal Engine Caches

    Controlling Data Retrieval

    Reducing Security Costs

    Off-loading Processing Work

    Boosting Join Performance

    Substring Searches

    Improving Temporary Table Performance

    Managing View Performance

    Subqueries

    Using Math Within SQL

    UNIONs

    Sorting

    HANDLER

    Improving Data Modification

    Improving INSERT

    Improving DELETE

9. Developing High-Speed Applications.

    Understanding Locking

    General Locking Overview

    InnoDB Locking Overview

    Optimal Transactions

    Key Transaction Terms and Concepts

    Cost of Transactions

    Transaction Performance Suggestions

    Stored Procedures and Functions

    Difference Between Stored Procedures and Functions

    When to Use Stored Procedures or Functions

    When Not to Use Stored Procedures or Functions

    Stored Procedure Optimization

    Triggers

    Current Trigger Limitations

    When to Use Triggers

    When Not to Use Triggers

    Writing Efficient Application Code

    General Suggestions

IV. OPTIMIZING AND TUNING THE MYSQL ENGINE.

10. General Server Performance Parameters and Tuning.

    Server Settings Scope

    Connectivity

    Creating a Connection

    Managing a Connection

    Memory Management

    Locking mysqld in Memory

    Thread Memory Settings

    MEMORY Tables

    Caching

    Application Control

    Resources

    Optimizer Control

    Concurrency

    User Resource Control

11. MyISAM Performance Enhancement.

    Optimal Data Storage

    Table Reporting and Control

    Table Maintenance and Repair

    Controlling Sorting Resources

    MyISAM and Memory

    The Key Cache

    Speeding MyISAM Operations

    Loading Information

    Improving FULLTEXT Searches

    Concurrency Considerations

12. InnoDB Performance Enhancement.

    InnoDB Storage Architecture and I/O

    Indexes

    Pages

    Extents and Segments

    Tablespaces

    Built-in Performance Enhancers

    Data Storage

    Log Files

    InnoDB and Memory

    Buffer Pool

    InnoDB Operational Control

    Thread Performance

    Improving Concurrency

    Improving Large-Scale Operations

    Speeding Up Shutdown

13. Improving Disk Speed.

    General Suggestions

    Deploying the Fastest Possible Disk Drives

    Configuring Memory Caches Correctly

    Implementing a RAID Strategy

    Distributing Temporary Directories

    MyISAM Suggestions

    Symbolic Links

    Table Compression

    InnoDB Suggestions

    Choosing the Right autoextend Setting

    Using Raw Devices

    Alternative Disk Writing Methods

    Table Defragmentation

    Operating System–Specific Disk Suggestions

    Linux/Unix Suggestions

    Windows Suggestions

14. Operating System, Web Server, and Connectivity Tuning.

    Operating System Considerations

    Assumptions

    Linux/Unix

    Windows

    Web Server Performance Considerations

    Choosing the Right Server Topology

    General PHP Suggestions

    Apache/PHP Suggestions

    Internet Information Services (IIS)/PHP Tips

    Connectivity Tuning

    Protocol Choices

    Costs of Hostname Resolution

    Costs of Creating and Destroying Connections

    Costs of SSL

15. Improving Import and Export Operations.

    Speeding Data Exports

    The Importance of Regular Exports

    Avoiding Excessive Memory Consumption

    Concurrency and Exports

    Retrieving Subsets of Information

    Copying Table Structures Without Data

    Delaying Index Re-creation—MyISAM

    Delaying Index Re-creation—InnoDB

    Preparing for Data Reload

    Accelerating Data Loading

    Managing Concurrency

    Handling Errors

    Engine-Specific Tips

V. DISTRIBUTED COMPUTING.

16. Optimal Replication.

    Introduction

    How Does Replication Work?

    Is Replication Right for You?

    Creating a Replication Strategy

    High-Performance Replication

    Network Considerations

    Master Server Performance Considerations

    Implementing Best Tuning Practices

    Managing Slave Resources

    Slave Server Performance Considerations

17. Optimal Clustering.

    Introduction

    How Does Clustering Work?

    Nodes

    Shared-Nothing

    Cluster Clients

    Storage Engine

    Transporter

    Data Distribution

    Checkpoints

    Is Clustering Right for You?

    Clustering Versus Replication

    Creating a Clustering Strategy

    Choosing the Right Version

    Cluster Topology

    Configuring MySQL Cluster for High Performance

    General Clustering Best Practices

    Network Considerations

    Transports

    Management Node Considerations

    SQL Node Considerations

    Data Node Considerations

VI. CASE STUDIES.

18. Case Study: High-Hat Delivers!

    Problem Queries

    Package Status Lookup

    Shipping Option Lookup

    Random Transaction Bottlenecks

    Diagnosis

    Solution

    Implementing These Solutions

19. Case Study: Friends Fly Free-for-All—A Promotion Gone Wrong.

    Server Availability

    Diagnosis

    Solution

    Application and Transaction Issues

    Diagnosis

    Solution

20. Case Study 3: Practice Makes Perfect.

    Data Importing

    Diagnosis

    Solution

    Clustering

    Diagnosis

    Solution

    Stored Procedures

    Diagnosis

    Solution

Index.

Preface

Introduction

The Purpose of This Book

MySQL is the foundation of many of today's highest quality, most robust database-driven solutions. However, users continually grow more demanding: A rich application that runs sluggishly is unacceptable, no matter what functionality it offers. Unfortunately, the cost of job cutbacks and outsourcing means that most overloaded IT professionals barely have enough time to deliver a minimal solution, much less master the minutiae of their database engine, which, in the case of MySQL, grows richer and more complex with each release.

Performance often takes a backseat to merely hitting the schedule. "Just ship it now and make it faster later" is a refrain heard around the world today. Regrettably, "later" rarely arrives, and designers, developers, and administrators end up delivering suboptimal solutions and then bracing themselves for the inevitable complaints from users. In many cases, these problems would never have come to pass if the overloaded designers and developers had enough time to fully research and exploit the numerous avenues to boost MySQL response. This book was written to help these beleaguered professionals jump-start their knowledge of MySQL performance tuning, letting them design, develop, and maintain the fastest MySQL solutions in the shortest amount of time.

Who Should Read This Book?

MySQL Database Design and Tuning is aimed at busy database designers, database and system administrators, webmasters, and developers who want to build, deploy, and manage the fastest possible solutions based on MySQL.

To take full advantage of this book, application developers should understand the following:

• Standard Structured Query Language (SQL)—Because SQL is the underpinning of MySQL database access, it's important that you at least comprehend basic SQL (simple queries, INSERT/UPDATE/DELETE operations, and so on).

• A programming language such as Java, C++, and so on—SQL is often paired with a procedural programming or scripting language to create MySQL-based applications. Although MySQL Performance Optimization is not a programming guide, it does discuss the performance interplay between the programming language and the MySQL engine.

Readers who are responsible for designing and/or administering databases should understand the following:

• Basic relational database design theory—To make the most of this book, you should be aware of how to design a simple database, as well as understand data normalization, constraints, and so on.

• Intermediate or advanced SQL—As a database administrator/analyst (DBA), you are often called on to produce more sophisticated SQL on behalf of your users. MySQL Database Design and Tuning will help you improve the responsiveness of your more complex join, subquery, index construction, and transactional challenges.

• MySQL engine architecture—This book does not expect you to have downloaded the MySQL source code or to construct hashing algorithms in your spare time. However, if you are familiar with the elegant MySQL product line architecture, you are likely to make better use of this book's recommendations.

System and web administrators should be familiar with the following:

• MySQL's engine architecture—Database performance issues are often misdiagnosed as hardware and/or operating system flaws. By understanding the components that make up the MySQL database, it's more likely that you will be able to accurately identify and fix (or even prevent) database response concerns.

• Configuring server operating systems such as Windows and Linux—There is interplay between the MySQL database and the underlying operating system. Fortunately, you can employ a number of operating system–specific settings to make your operating system more MySQL-friendly.

Finally, some words for the open source developer. Unlike any other widely available database, MySQL's open source architecture, customer-friendly licensing, and source code availability means that you can programmatically change the behavior of the engine itself. This book, however, focuses on using the commercially available versions of MySQL; this book does not expect any reader to have the time, skill, or inclination to alter MySQL's source code.

Benefits of Using This Book

Readers of MySQL Database Design and Tuning will help themselves and their organizations in at least five ways:

• Simplicity—There's a common belief that the simplest solution is by definition a slow solution. However, this is often incorrect. In fact, a well-designed, optimized database application is frequently cleaner and more elegant than an overengineered product. Good performance and simple design are rarely mutually exclusive. This book always opts for the simplest, fastest solution.

• Productivity—Database-centric applications often serve as the cornerstone of an organization's operations. A nonoptimized solution wastes each user's time. When multiplied by the large number of internal and external users affected by these delays, the cost to the organization's productivity can be staggering. MySQL Performance Optimization is filled with easily implemented tips and tricks that will have a dramatic, positive impact on performance.

• Reliability—Response and reliability problems often are mistaken for each other. For example, imagine that you are a user of a popular website that is built on MySQL. You have just submitted an e-commerce payment form. Unfortunately, the site database and application designers never planned for large numbers of users inserting information at the same time, so your browser appears to "hang" while waiting for a response. From the user's perspective, is this a flaw in response or reliability? It really doesn't matter because you now have an unhappy customer who thinks that your website is "buggy." Faster systems equal more reliable systems.

• Cost control—Hardware prices continue to fall; memory and disk drives are now so cheap that you might wonder if they will be offered in cereal boxes before too long. Nevertheless, many organizations that employ MySQL are cost conscious. Squeezing an extra 10% or 20% from your applications might spell the difference between your raise money being spent on you or on a new CPU.

• User satisfaction—Although your users will likely love your MySQL-based solutions, it's doubtful that they will want to spend any more time staring at their screens awaiting response than is absolutely necessary. Each year, users grow more sophisticated and less tolerant of delay. In some cases, a perfectly good system is abandoned because of performance problems. MySQL Performance Optimization is designed to help you delight your users by delivering the fastest, most reliable solutions.

How to Use This Book

This book is divided into six major sections:

• Part I: Introduction—This section describes how to get the most from this book, its structure and examples, and MySQL's architecture, as well as how to set up an optimization environment.

• Part II: Advanced Database Design Concepts—Your MySQL solution is only as solid and responsive as the database design upon which it rests. This section explores how to choose the correct options when designing your tables, the impact of constraints on response, the excellent diagnostics offered by the MySQL optimizer, and how to leverage indexes for exceptional performance.

• Part III: Optimizing Application Code—Your applications will likely interact with MySQL through one or more programming/scripting languages, even if it's just SQL. For that reason, this section examines how to write the most efficient SQL, as well as special considerations when using popular programming technologies, such as Java, C, ODBC, Perl, and so on. Because newer versions of MySQL offer stored procedures and triggers, this section also looks at how to make these exciting new technologies as speedy as possible.

• Part IV: Optimizing and Tuning the MySQL Engine—Administrators have dozens of choices to make when setting up and maintaining their MySQL environments. These choices range from engine and operating system parameters to disk management and bulk data operations. This section is devoted to helping these administrators make intelligent decisions when confronted with these situations.

• Part V: Distributed Computing—Replication and clustering are two key MySQL technologies for spreading your processing load among multiple computers. This section of the book examines when to deploy these powerful capabilities, as well as how to tune them for optimal performance.

• Part VI: Case Studies—The final section of this book ties the information found in the earlier chapters together in a series of multidimensional, real-world case studies.

Whether you choose to read all sections will likely be determined by your MySQL-oriented responsibility. For example, if you are an application developer, it's likely that you'll focus on Parts I, II, III, and V; you are probably not going to spend much time on Part IV's engine configuration. However, if you are tasked with tuning the database engine or the operating system, Part IV will be of supreme interest to you. In any case, the optimization techniques and examples have been purposely designed to be as independent of each other as possible.

Information About MySQL Software and Environments

The frequency with which MySQL AB releases new versions and products can be overwhelming. This book uses products from both the 4.1 series (versions 4.1.7 through 4.1.10), along with newer versions, from 5.0 through 5.0.3. When applicable, the performance-specific features of the 5.1 release are discussed. This book also highlights some of the excellent new graphical products, such as the MySQL Administrator and MySQL Query Browser.

MySQL's open source architecture means that the product runs on a tremendous variety of hardware and operating system environments. However, aside from the sections in which operating system tuning in a MySQL environment is discussed, this book always strives to make the examples as platform-independent as possible.

Information About Examples

Founded during the height of the dot-com boom in 1999, High-Hat Airways has grown into the world's largest airline. Its unique blend of hip, in-your-face marketing, low prices, and creative corporate accounting helped propel the airline past sluggish industry dinosaurs.

Its fleet has expanded from one 20-year-old rented Boeing 737 to more than 550 planes. During the next year, more than 275 million customers will board a High-Hat flight and be greeted by the familiar "Go sit down over there!" from a friendly flight attendant.

The year 2002 saw the indictments of the CEO and CFO, along with the mysterious disappearance of four members of the Board of Directors. However, these setbacks proved to be mere road bumps on the path to success: The value of High-Hat's stock has grown thirtyfold in less than three years.

As part of its ongoing commitment to cost control, High-Hat Airways has decided to convert all internal systems to open source. Naturally, MySQL will be a cornerstone of this new philosophy. However, the already overburdened IT staff has a monumental task ahead: to consolidate 27 internal applications running on 11 different database platforms. The most important applications are as follows:

• Reservations

• Flight logistics and scheduling

• Catering

• Luggage management

• Loyalty programs

To make matters worse, the CIO has promised both internal users and the general public that High-Hat's systems will have the fastest response time in the industry. The CIO is also refereeing a religious war between Microsoft zealots and Linux enthusiasts. The result is that the new solutions will be evenly split between the two operating systems.

As you proceed through this book, you'll notice it continually refers to the challenges faced by the High-Hat Airways IT staff as they struggle to deliver quality, high-performance solutions based on MySQL. Luckily, very simple, easily implemented changes can result in massive speed gains, and that is what this book plans to help you achieve.

To keep things simple, this book introduces High-Hat's new applications, database structures, and engine configurations on an as-needed basis, in concert with the specific performance concept being examined at the time. This means that you can focus on MySQL optimization, rather than becoming an expert in the internal systems of High-Hat Airways. Finally, the database design, SQL, and code examples are always as simple and to-the-point as possible while still conveying the appropriate performance messages.

© Copyright Pearson Education. All rights reserved.