Operating System

To perform system cloning of an AIX 3.2.x system, follow the same procedure explained for mksysb restores and Sysback restores. The new system will become an image of the system backed up on tape. No special procedure must be followed since AIX 3.2.x supports only the channel architecture.

AIX 4.x Operating System

The newer version of AIX requires special attention since not all the drivers are installed with the BOS. In this case, you are left with two choices:

• Install all device drivers for all available architectures before taking the mksysb or Sysback backup of the system. That allows the tape to be booted from and restored onto any IBM RS/6000 and is the only viable option for Sysback.

• If device drivers were not installed, you can simply boot the new machine from the CD- ROM containing the AIX 4.x operating system. After the boot, choose the

"Restore from tape backup" option, insert the mksysb tape, and start the restore. This last method is the one recommended by IBM in its documentation; this option works only for mksysb tapes and is not available with Sysback.

Page 339

V

DATABASE BACKUP & RECOVERY

Part V consists of the following four chapters which discuss database backup and detail the steps to back up and recover each of these widely used database:

• Chapter 13, Backing Up Databases, provides an overview of database backup, concepts, terminology, and procedures.

• Chapter 14, Informix Backup & Recovery, discusses backup and recovery facilities and provides step-by-step recovery instructions.

• Chapter 15, Oracle Backup & Recovery, discusses Oracle's backup and recovery facilities and provides step-by-step recovery instructions.

• Chapter 16, Sybase Backup &Recovery, discusses Sybase's backup and recovery facilities and provides step-by-step recovery instructions.

Page 341

13

Backing Up Databases

Performing regular database backups is one of the hardest tasks that lies before today's system administrator. The primary reason is that database are infinitely larger and more complex than simple filesystem files. In order to properly back up a database, you first need to:

• Understand the internal structure of your database

• Understand the available utilities

• Have an excellent working relationship between system administrators and database administrators Once you've accomplished all of that, you'll need to choose among your various options:

• Buy an expensive commercial utility

• Find or write your own utility

• Perform cold backups without a utility

Almost anyone who reads this list will find at least one of these steps daunting. Many people work with database that operate 24 hours, seven days a week. They can't shut them down for hours at a time to back them up. Even if they could, if. a database uses raw devices it can't be backed up with a regular dump. Of course dd would work, but that would mean doing one thing for filesystems and a different thing for databases. A common theme throughout this book is that different is bad. Every special case is a chance for failure. It's something else you have to code for, something else you have to watch-something else to break. The result is that database backups are not easy.

Page 342 Part of the problem is the design process of the actual database engine itself. His-torically, the need for bigger storage and faster queries drove the design of a product much more than its ability to back itself up. (This goes for filesystems too. Most Unix vendors have added support for a multiple-terabyte filesystem that break the 2-GB file-size barrier, but at least one man page for dump says "WARNING: dump will not back up a filesystem containing large files.") Over the last few years, database have grown from a gigabyte or so to an average size that is daily growing closer to a terabyte. This growth in size and performance happened because the customer base screamed for it. Unfortunately, they weren't simultaneously screaming for a backup utility to support those huge database.

The Nature of the Beast

About five years ago, I had a conversation with a developer at Informix, in which I was

complaining that they were selling us a 150-GB database without a decent utility to back it up. (We were using onarchive. See Chapter 14, Informix Backup & Recovery, for details.) His response was, "Good backup utilities don't sell databases. Good databases do. Developers have to have their priorities." Wonderful. You can't really hold this against Informix, though. All the database companies did about the same thing at the same time. They are all finally starting to come out with utilities that are much more functional. Informix is now the easiest database to back up and recover; onbar is so much easier than rman. And yet, Informix still doesn't have Oracle's market share. I guess the Informix developer was right.

Can It Be Done?

Think about database backups from a big-picture perspective, comparting them to filesystem backups. There are a number of good backup utilities on the market now.

Why aren't there just as many for database backups? There is certainly a demand for it.

One of the reasons is the complexity of the task. In order to release a database backup product, a company would need to consider several factors:

Multiple moving targets

How do you get a database to hold still? Have you ever tried to take a picture of 100 people? Designing a backup utility for a database is very hard, since you have to

"take a picture" of hundreds of files all at once.

Interrelationship between the files

A database backup needs to understand all of the database elements and how they relate.

Page 343 Necessity to work with the database engine

This is essential. If the database understands that you are running a backup, it can help you. If you don't interface with the database, you'll be backing up blind.

The size of the job

How do you get one terabyte of data to a backup drive in one hour? That's what some of the backup requirements are! The only answer to that is a multithreaded backup program. (See "Simultaneous Backup of One Client to Many Drives" in Chapter 5, Commercial Backup Utilities, for more details.)

Recoverability versus cost

You need all of the preceding, but you don't want to mortgage your house to get it.

Differing levels of automation

Different customers want different levels of automation. Some want everything managed by the library, while others would rather do it themselves.

With these kinds of requirements, is there any hope of getting commercial utilities that are up to the challenge? The answer is "Yes!" Database companies have finally recognized that backup utilities do have an effect on the overall sales of a database engine. They have finally started producing good utilities that interface with other commercial backup products. Vendors have even taken an active lead in making sure that customers use a utility that works properly on both the backup and the restore sides. Now that there are decent backup utilities, though, you have another problem-confusion.

Confusion: The Mysteries of Database Architecture

Any system administrator who has been in the business for any length of time can probably tell you how to back up the home directories on any system. Start asking about backing up databases, though, and even the most seasoned veterans may start to squirm. To many administrators, the architecture of a database is a mystery.

Unfortunately, you really need to understand how the database works to properly recover from a disaster. They know how to back up a filesystem, but ask them to find the backups for data space A in database B in instance C, and they look at you with fear in their eyes! They just don't have any experience in database design-nor do they have time to get that experience. If they work in a heterogeneous shop with more than one database, it gets even harder. Their only hope is that the database administrators(DBAs) know what they are doing.

Page 344 DBAs spend much of their time normalizing databases, designing user forms, and making sure the database performs adequately. They know how to back up their database disk, maybe even to a standalone backup drive, but they don't have any experience with commercial backup software or large, automated libraries. If the database is too big to back up to disk and they don't have a standalone backup drive to back up to, they have to work with the system administrators(SAs) to get the backup done. Their only hope is that the SAs know what they are doing.

Database products also differ from one another. Try to get an Informix DBA and an Oracle DBA to agree on what a tablespace is!* One of the reason this is difficult is that different products use the same term for different logical elements. What Informix calls a tablespace, Oracle calls a segment. Don't confuse that, however, with a Sybase segment, which is closer to what Oracle calls a tablespace. Are you confused yet? Don't worry.

The Muck Stops Here: Databases in Plain English

This chapter explains everything you need to know in order to learn more about database backup and recovery. You then will be prepared to read and understand the vendor-specific database chapters in this book, as well as any of the appropriate sections of your favorite DBA book or manual. This chapter assumes no prior knowledge, beginning with some of the very basic elements such as tables and rows, so that even the most junior person can understand everything. However, that does not mean that a seasoned SA or DBA should not read this chapter. It took several experienced DBAs of a number of different types of databases to come up with the information you see here, so almost any DBA should be able to learn something from this chapter.

This chapter includes an extremely useful table that lists all of the elements of database storage for each of three big databases:** Oracle, Informix, and Sybase. If you're not a DBA, the information here will allow you to discuss backup matters intelligently with your DBA. If you are a DBA for one product, this will help you to discuss backup and storage strategies with DBAs for other products-without confusing one another! The desired result for your organization is that you can all agree on why and how to protect your database data. What a beautiful world that will be!

* I know. It took weeks to compile the comparison table in this chapter. It would be easier to get Unix administrators to agree on which editor is the best!

** DB2 and Ingres are not included in this edition. The reason for this is lack of research material. If I can find some good books to read and DBAs to work with, they might be included in a later revision of this book. If I do, the updated table will be put on the web page for your enjoyment.

Page 345

What's the Big Deal?

Why, all of a sudden, are we hearing so much about database backups? Why are they so hard? Why don't utilities currently exist to do all this? Can't I just shut down the database and back up the whole system? These are all questions that may be going through your head. If you already know the answers, feel free to skip to the next section.

Why, all of a sudden, are we hearing so much about database backups?

The demand for Relational Database Management Systems (RDBMSs) has grown exponentially in the last few years. Not only are there more databases, they are faster, larger, and more complex than ever before. Companies are relying increasingly on bigger and bigger databases to store their information-information that, if lost, would be irreplaceable. Customers have started to recognize the importance of safeguarding their data, and the demand for better backup and recovery utilities has followed. Database companies and backup product companies have finally responded with utilities that are up to the task at hand.

Why is backing up a database so hard?

Actually, backing up a database isn't that hard. It's restoring the database that has caused many people to go insane! Seriously, though, the reason it is so difficult is that you need a good SA and a good DBA to design a good backup plan. Most people know only one side well. If you or your people know both sides, then consider yourself very lucky. In some companies, it's tough to get both sides to work together.

Why aren't there utilities already available to do all this?

Much to the chagrin of a certain backup book author, backups aren't sexy. Historically, customers have asked for faster databases, or easier-to-program databases.

Therefore, the existence of a good backup utility was not even considered until well after a product was purchased, installed, and quite often in production. Many times it even took a disaster to get some people to realize that backup and recovery is an essential part of any database system.

This finally has changed. Maybe it was because customers finally realized that they needed to ask about backup and recovery when they were evaluating a database product. Maybe the database companies' support departments beat up the developers because they were spending all their time on down system calls.

(Hell hath no fury like a customer who has suffered data loss that could have been prevented by a better backup utility.)

One of the most important tasks that customers needed was the ability to integrate their database backups into their commercial backup utility. Remember, though, that these products haven't been on the market for that long. Until a

Page 346 few years ago, if you asked your database vendor if its database worked with product X, they were likely to say, "Who are they and why should we?" It's not clear who broke the barrier first, but all three big vendors did the same thing within a year or two of one another-they cooperated with commercial backup companies to develop and release their own utility that was designed specifically to work with third-party backup products.

Can't I just shut down the database and back up the whole system?

In a small numbers of cases, yes. There are a number of considerations that might prevent you from doing this, including your database platform, whether you are using raw or filesystem files and whether you need point-in-time recovery. Those details are covered in appropriate chapters in this book.

Database Structure

There are many terms thrown around when one is discussing RDBMSs. The good news is that you don't need to know all of them to properly back up and recover databases. You do need to know some of them, though-about 20 individual terms. It's helpful to know:

• What all the different storage elements are and what they are called

• How these elements are logically organized within the RDBMS

• What facilities are in place to protect and back up the data

This information can be complex, because it depends a lot on how you look at the data. This chapter presents this information from first a power user's, then a DBA's, point of view. The various building blocks of a database are defined, although we may have to go up and down the building a bit before we're done!

The Power User's View: Logical Elements of a Database

Before looking at how databases are stored on disk, let's look at the "user's" view of a database. This is necessary since some of these terms are used in the definition of the storage elements. It might be more appropriate to call this a "power user's view," since many users will have little or no knowledge of any of these terms. But unless they want to start doing the DBA's job of putting a database together, these terms may be all that they would ever need. The terms are presented in no particular order, since it is very difficult to define one term without using another one. Therefore, it may help some readers to read this section more than once.

Page 347 This view also could be called the ''logical" view, since many of the elements described in this view don't exist in a physical sense. That fact is one of the many things that differentiate an RDBMS from a simple spreadsheet. A spreadsheet has one table that resides in one physical file. An RDBMS table, on the other hand, gives the appearance that its data is all sitting in one place, but it may be spread out all over the system.

Instance

Term Informix Oracle Sybase

Instance Instance Instance Server

Instance is probably the most difficult term to explain because it means different things to different people-and to different database platforms. The simplest definition is that an instance is a process (or set of processes) on one machine, through which the database on that machine* communicate with shared memory. There often can be multiple databases within an instance, and a database also can be distributed across multiple instances on the same machine or on separate machines within a cluster. (In order to distribute a database across multiple instances, you must use a parallel database product such as Informix Extended Parallel Option (XPO) or Oracle's Cluster Server (OCS).) Therefore, an instance and a database are two entirely different concepts.

The Sybase term "server" stems from the original intent that each machine/server would have one Sybase instance/server on it, although it is now quite common to have more than one instance on each machine. Informix occasionally uses the term in this manner, and it can be rather confusing. They tend to use server when speaking about the software and instance when speaking about a running engine environment and especially when discussing running multiple instantiations of the server.

If an instance needs to be shut down and restarted for any reason, all databases within that instance are unavailable during the shutdown.** Perhaps this helps to understand the original definition, since all of the databases within an instance have a single connection to shared memory, which is provided by the instance. If the instance is shut down, that connection is no longer available. (See Figure 13-1 for a graphical representation of an instance.)

* An instance can actually be spread across multiple machines.

** That is why it is not correct to say that you are going to stop/start a database. You are actually stopping/starting an instance that contains the database.

Page 348

Figure 13-1.

A database instance

Most database backup utilities are able to back up only an entire instance, although they may be able to restore a single database within that instance.

Page 349 Database

Term Informix Oracle Sybase

Database Database Database Database

A database is a collection of database objects. It may be a very simple database with one table and no indexes, or it could contain many tables, indexes, and other database objects. (All database products have the ability to have more than one database object and more than one type of database object.) For example, the

"customer" database may contain a table that has customer addresses and an index for that table. It also may contain a Binary Large OBject (BLOB) table that contains a scanned-in image of the customer's contract, a regular table that contains the data from that contract, and an index for that table. Then there might be another

"customer" database may contain a table that has customer addresses and an index for that table. It also may contain a Binary Large OBject (BLOB) table that contains a scanned-in image of the customer's contract, a regular table that contains the data from that contract, and an index for that table. Then there might be another