Beware of Class.forName()!
2.4. Executing Statements and Retrieving Results
Problem
You want a program to send an SQL statement to the MySQL server and retrieve its result.
Solution
Some statements return only a status code; others return a result set (a set of rows).
Some APIs provide different methods for executing each type of statement. If so, use the appropriate method for the statement to be executed.
Discussion
You can execute two general categories of SQL statements. Some retrieve information from the database; others change that information. Statements in the two categories are handled differently. In addition, some APIs provide multiple routines for executing statements, complicating matters further. Before we get to examples demonstrating how to execute statements from within each API, I’ll describe the database table the examples use, and then further discuss the two statement categories and outline a general strategy for processing statements in each category.
In Chapter 1, we created a table named limbs to try some sample statements. In this chapter, we’ll use a different table named profile. It’s based on the idea of a “buddy list,” that is, the set of people we like to keep in touch with while we’re online. The table definition looks like this:
CREATE TABLE profile (
id INT UNSIGNED NOT NULL AUTO_INCREMENT, name VARCHAR(20) NOT NULL,
birth DATE,
color ENUM('blue','red','green','brown','black','white'),
foods SET('lutefisk','burrito','curry','eggroll','fadge','pizza'), cats INT,
PRIMARY KEY (id) );
The profile table indicates the things that are important to us about each buddy: name, age, favorite color, favorite foods, and number of cats—obviously one of those goofy tables used only for examples in a book! (Actually, it’s not that goofy. The table uses several different data types for its columns, and these come in handy to illustrate how to solve problems that pertain to specific data types.)
The table also includes an id column containing unique values so that we can distinguish one row from another, even if two buddies have the same name. id and name are declared
2.4. Executing Statements and Retrieving Results | 65
as NOTNULL because they’re each required to have a value. The other columns are im‐
plicitly permitted to be NULL (and that is also their default value) because we might not know the value to assign them for any given individual. That is, NULL signifies “un‐
known.”
Notice that although we want to keep track of age, there is no age column in the table.
Instead, there is a birth column of DATE type. Ages change, so if we store age values, we’d have to keep updating them. Storing birth dates is better: they don’t change and can be used to calculate age any time (see Recipe 6.13). color is an ENUM column; color values can be any one of the listed values. foods is a SET, which permits the value to be any combination of the individual set members. That way we can record multiple fa‐
vorite foods for any buddy.
To create the table, use the profile.sql script in the tables directory of the recipes dis‐
tribution. Change location into that directory, then run this command:
% mysql cookbook < profile.sql
The script also loads sample data into the table. You can experiment with the table, then restore it if you change its contents by running the script again. (See the final section of this chapter on the importance of restoring the profile table after modifying it.) The contents of the profile table as loaded by the profile.sql script look like this:
mysql> SELECT * FROM profile;
+----+---+---+---+---+---+
| id | name | birth | color | foods | cats | +----+---+---+---+---+---+
| 1 | Sybil | 1970-04-13 | black | lutefisk,fadge,pizza | 0 |
| 2 | Nancy | 1969-09-30 | white | burrito,curry,eggroll | 3 |
| 3 | Ralph | 1973-11-02 | red | eggroll,pizza | 4 |
| 4 | Lothair | 1963-07-04 | blue | burrito,curry | 5 |
| 5 | Henry | 1965-02-14 | red | curry,fadge | 1 |
| 6 | Aaron | 1968-09-17 | green | lutefisk,fadge | 1 |
| 7 | Joanna | 1952-08-20 | green | lutefisk,fadge | 0 |
| 8 | Stephen | 1960-05-01 | white | burrito,pizza | 0 | +----+---+---+---+---+---+
Although most of the columns in the profile table permit NULL values, none of the rows in the sample dataset actually contain NULL yet. (I want to defer the complications of NULL value processing to Recipes 2.5 and 2.7.)
SQL statement categories
SQL statements can be grouped into two broad categories, depending on whether they return a result set (a set of rows):
• Statements that return no result set, such as INSERT, DELETE, or UPDATE. As a general rule, statements of this type generally change the database in some way. There are
66 | Chapter 2: Writing MySQL-Based Programs
some exceptions, such as USEdb_name, which changes the default (current) database for your session without making any changes to the database itself. The example data-modifying statement used in this section is an UPDATE:
UPDATE profile SET cats = cats+1 WHERE name = 'Sybil'
We’ll cover how to execute this statement and determine the number of rows that it affects.
• Statements that return a result set, such as SELECT, SHOW, EXPLAIN, or DESCRIBE. I refer to such statements generically as SELECT statements, but you should under‐
stand that category to include any statement that returns rows. The example row-retrieval statement used in this section is a SELECT:
SELECT id, name, cats FROM profile
We’ll cover how to execute this statement, fetch the rows in the result set, and determine the number of rows and columns in the result set. (To get information such as the column names or data types, access the result set metadata. That’s Recipe 10.2.)
The first step in processing an SQL statement is to send it to the MySQL server for execution. Some APIs (those for Perl, Ruby, and Java, for example) recognize a distinc‐
tion between the two categories of statements and provide separate calls for executing them. Other APIs (such as the one for Python) have a single call used for all statements.
However, one thing all APIs have in common is that no special character indicates the end of the statement. No terminator is necessary because the end of the statement string terminates it. This differs from executing statements in the mysql program, where you terminate statements using a semicolon (;) or \g. (It also differs from how this book usually includes semicolons in examples to make it clear where statements end.) When you send a statement to the server, be prepared to handle errors if it did not execute successfully. Do not neglect this! If a statement fails and you proceed on the basis that it succeeded, your program won’t work. For the most part, this section does not show error-checking code, but that is for brevity. The sample scripts in the recipes distribution from which the examples are taken do include error handling, based on the techniques illustrated in Recipe 2.2.
If a statement does execute without error, your next step depends on the statement type.
If it’s one that returns no result set, there’s nothing else to do, unless you want to check how many rows were affected. If the statement does return a result set, fetch its rows, then close the result set. In a context where you don’t know whether a statement returns a result set, Recipe 10.3 discusses how to tell.
Perl
The Perl DBI module provides two basic approaches to SQL statement execution, de‐
pending on whether you expect to get back a result set. For a statement such as IN 2.4. Executing Statements and Retrieving Results | 67
SERT or UPDATE that returns no result set, use the database handle do() method. It executes the statement and returns the number of rows affected by it, or undef if an error occurs. If Sybil gets a new cat, the following statement increments her cats count by one:
my $count = $dbh->do ("UPDATE profile SET cats = cats+1 WHERE name = 'Sybil'");
if ($count) # print row count if no error occurred {
$count += 0;
print "Number of rows updated: $count\n";
}
If the statement executes successfully but affects no rows, do() returns a special value,
"0E0" (the value zero in scientific notation, expressed as a string). "0E0" can be used for testing the execution status of a statement because it is true in Boolean contexts (unlike undef). For successful statements, it can also be used when counting how many rows were affected because it is treated as the number zero in numeric contexts. Of course, if you print that value as is, you’ll print "0E0", which might look odd to people who use your program. The preceding example makes sure that doesn’t happen by adding zero to the value to coerce it to numeric form so that it displays as 0. Alternatively, use printf with a %d format specifier to cause an implicit numeric conversion:
if ($count) # print row count if no error occurred {
printf "Number of rows updated: %d\n", $count;
}
If RaiseError is enabled, your script terminates automatically for DBI-related errors, so you need not check $count to find out whether do() failed and consequently can simplify the code:
my $count = $dbh->do ("UPDATE profile SET cats = cats+1 WHERE name = 'Sybil'");
printf "Number of rows updated: %d\n", $count;
To process a statement such as SELECT that does return a result set, use a different approach that involves these steps:
1. Specify the statement to be executed by calling prepare() using the database han‐
dle. prepare() returns a statement handle to use with all subsequent operations on the statement. (If an error occurs, the script terminates if RaiseError is enabled;
otherwise, prepare() returns undef.)
2. Call execute() to execute the statement and generate the result set.
3. Loop to fetch the rows returned by the statement. DBI provides several methods for this; we cover them shortly.
68 | Chapter 2: Writing MySQL-Based Programs
4. If you don’t fetch the entire result set, release resources associated with it by calling finish().
The following example illustrates these steps, using fetchrow_array() as the row-fetching method and assuming that RaiseError is enabled so that errors terminate the script:
my $sth = $dbh->prepare ("SELECT id, name, cats FROM profile");
$sth->execute ();
my $count = 0;
while (my @val = $sth->fetchrow_array ()) {
print "id: $val[0], name: $val[1], cats: $val[2]\n";
++$count;
}
$sth->finish ();
print "Number of rows returned: $count\n";
The row array size indicates the number of columns in the result set.
The row-fetching loop just shown is followed by a call to finish(), which closes the result set and tells the server to free any resources associated with it. If you fetch every row in the set, DBI notices when you reach the end and releases the resources for you.
Thus, the example could omit the finish() call without ill effect.
As the example illustrates, to determine how many rows a result set contains, count them while fetching them. Do not use the DBI rows() method for this purpose. The DBI documentation discourages this practice because rows() is not necessarily reliable for SELECT statements—due not to a deficiency in DBI, but to differences in behavior among database engines.
DBI has several methods that fetch a row at a time. The one used in the preceding example, fetchrow_array(), returns an array containing the next row, or an empty list when there are no more rows. Array elements are present in the order named in the SELECT statement. Access them as $val[0], $val[1], and so forth.
The fetchrow_array() method is most useful for statements that explicitly name the columns to select. (With SELECT*, there are no guarantees about the positions of col‐
umns within the array.)
fetchrow_arrayref() is like fetchrow_array(), except that it returns a reference to the array, or undef when there are no more rows. As with fetchrow_array(), array elements are present in the order named in the statement. Access them as $ref->[0],
$ref->[1], and so forth:
while (my $ref = $sth->fetchrow_arrayref ()) {
print "id: $ref->[0], name: $ref->[1], cats: $ref->[2]\n";
}
2.4. Executing Statements and Retrieving Results | 69
fetchrow_hashref() returns a reference to a hash structure, or undef when there are no more rows:
while (my $ref = $sth->fetchrow_hashref ()) {
print "id: $ref->{id}, name: $ref->{name}, cats: $ref->{cats}\n";
}
To access the elements of the hash, use the names of the columns selected by the state‐
ment ($ref->{id}, $ref->{name}, and so forth). fetchrow_hashref() is particularly useful for SELECT* statements because you can access elements of rows without knowing anything about the order in which columns are returned. You need know only their names. On the other hand, it’s more expensive to set up a hash than an array, so fet chrow_hashref() is slower than fetchrow_array() or fetchrow_arrayref(). It’s also possible to “lose” row elements if they have the same name because column names must be unique. Same-name columns are not uncommon for joins between tables. For sol‐
utions to this problem, see Recipe 14.10.
In addition to the statement execution methods just described, DBI provides several high-level retrieval methods that execute a statement and return the result set in a single operation. All are database-handle methods that create and dispose of the statement handle internally before returning the result set. The methods differ in the form in which they return the result. Some return the entire result set, others return a single row or column of the set, as summarized in the following table:
Method Return value
selectrow_array() First row of result set as an array
selectrow_arrayref() First row of result set as a reference to an array selectrow_hashref() First row of result set as a reference to a hash selectcol_arrayref() First column of result set as a reference to an array selectall_arrayref() Entire result set as a reference to an array of array references selectall_hashref() Entire result set as a reference to a hash of hash references
Most of these methods return a reference. The exception is selectrow_array(), which selects the first row of the result set and returns an array or a scalar, depending on how you call it. In array context, selectrow_array() returns the entire row as an array (or the empty list if no row was selected). This is useful for statements from which you expect to obtain only a single row. The return value can be used to determine the result set size. The column count is the number of elements in the array, and the row count is 1 or 0:
my @val = $dbh->selectrow_array ("SELECT name, birth, foods FROM profile WHERE id = 3");
my $ncols = @val;
my $nrows = $ncols ? 1 : 0;
70 | Chapter 2: Writing MySQL-Based Programs
You can also invoke selectrow_array() in scalar context, in which case it returns only the first column from the row (especially convenient for statements that return a single value):
my $buddy_count = $dbh->selectrow_array ("SELECT COUNT(*) FROM profile");
If a statement returns no result, selectrow_array() returns an empty array or undef, depending on whether you call it in array or scalar context.
selectrow_arrayref() and selectrow_hashref() select the first row of the result set and return a reference to it, or undef if no row was selected. To access the column values, treat the reference the same way you treat the return value from fetchrow_arrayr ef() or fetchrow_hashref(). The reference also provides the row and column counts:
my $ref = $dbh->selectrow_arrayref ($stmt);
my $ncols = defined ($ref) ? @{$ref} : 0;
my $nrows = $ncols ? 1 : 0;
my $ref = $dbh->selectrow_hashref ($stmt);
my $ncols = defined ($ref) ? keys (%{$ref}) : 0;
my $nrows = $ncols ? 1 : 0;
selectcol_arrayref() returns a reference to a single-column array representing the first column of the result set. Assuming a non-undef return value, access elements of the array as $ref->[i] for the value from row i. The number of rows is the number of elements in the array, and the column count is 1 or 0:
my $ref = $dbh->selectcol_arrayref ($stmt);
my $nrows = defined ($ref) ? @{$ref} : 0;
my $ncols = $nrows ? 1 : 0;
selectall_arrayref() returns a reference to an array containing an element for each row of the result. Each element is a reference to an array. To access row i of the result set, use $ref->[i] to get a reference to the row. Then treat the row reference the same way as a return value from fetchrow_arrayref() to access individual column values in the row. The result set row and column counts are available as follows:
my $ref = $dbh->selectall_arrayref ($stmt);
my $nrows = defined ($ref) ? @{$ref} : 0;
my $ncols = $nrows ? @{$ref->[0]} : 0;
selectall_hashref() returns a reference to a hash, each element of which is a hash reference to a row of the result. To call it, specify an argument that indicates which column to use for hash keys. For example, if you retrieve rows from the profile table, the primary key is the id column:
my $ref = $dbh->selectall_hashref ("SELECT * FROM profile", "id");
Access rows using the keys of the hash. For a row that has a key column value of 12, the hash reference for the row is $ref->{12}. That row value is keyed on column names,
2.4. Executing Statements and Retrieving Results | 71
which you can use to access individual column elements (for example,
$ref->{12}->{name}). The result set row and column counts are available as follows:
my @keys = defined ($ref) ? keys (%{$ref}) : ();
my $nrows = scalar (@keys);
my $ncols = $nrows ? keys (%{$ref->{$keys[0]}}) : 0;
The selectall_XXX() methods are useful when you need to process a result set more than once because Perl DBI provides no way to “rewind” a result set. By assigning the entire result set to a variable, you can iterate through its elements multiple times.
Take care when using the high-level methods if you have RaiseError disabled. In that case, a method’s return value may not enable you to distinguish an error from an empty result set. For example, if you call selectrow_array() in scalar context to retrieve a single value, an undef return value is ambiguous because it may indicate any of three things: an error, an empty result set, or a result set consisting of a single NULL value. To test for an error, check the value of $DBI::errstr, $DBI::err, or $DBI::state. Ruby
As with Perl DBI, Ruby DBI provides two approaches to SQL statement execution. With either approach, if a statement-execution method fails with an error, it raises an excep‐
tion.
For statements such as INSERT or UPDATE that return no result set, invoke the do database-handle method. Its return value indicates the number of rows affected:
count = dbh.do("UPDATE profile SET cats = cats+1 WHERE name = 'Sybil'") puts "Number of rows updated: #{count}"
For statements such as SELECT that return a result set, invoke the execute database-handle method. execute returns a statement handle for fetching result set rows. The statement handle has several methods of its own that enable row fetching in different ways. After you are done with the statement handle, invoke its finish method. (Call finish for every statement handle that you create, unlike Perl DBI where finish need be invoked only if you fetch a partial result set.) To determine the number of rows in the result set, count them as you fetch them.
The following example executes a SELECT statement and uses the statement handle’s fetch method in a while loop:
count = 0
sth = dbh.execute("SELECT id, name, cats FROM profile") while row = sth.fetch do
printf "id: %s, name: %s, cats: %s\n", row[0], row[1], row[2]
count += 1 end
sth.finish
puts "Number of rows returned: #{count}"
72 | Chapter 2: Writing MySQL-Based Programs
row.size tells you the number of columns in the result set.
fetch can also be used as an iterator that returns each row in turn:
sth.fetch do |row|
printf "id: %s, name: %s, cats: %s\n", row[0], row[1], row[2]
printf "id: %s, name: %s, cats: %s\n", row[0], row[1], row[2]