Sun Blueprint : MySQL in Solaris Containers

While the costs of managing a data center are becoming a major concern with the increased number of under-utilized servers, customers are actively looking for solutions to consolidate their workloads to:

• improve server utilization
• improve data center space utilization
• reduce power and cooling requirements
• lower capital and operating expenditures
• reduce carbon footprint, ..

To cater those customers, Sun offers several virtualization technologies such as Logical Domains, Solaris Containers, xVM at free of cost for SPARC and x86/x64 platforms.

In order to help our customers who are planning for the consolidation of their MySQL databases on systems running Solaris 10, we put together a document with a bunch of installation steps and the best practices to run MySQL inside a Solaris Container. Although the document was focused on the Solaris Containers technology, majority of the tuning tips including the ZFS tips are applicable to all MySQL instances running [on Solaris] under different virtualization technologies.

You can access the blueprint document at the following location:

        Running MySQL Database in Solaris Containers

The blueprint document briefly explains the MySQL server & Solaris Containers technology, introduces different options to install MySQL server on Solaris 10, shows the steps involved in installing and running Solaris Zones & MySQL, and finally provides few best practices to run MySQL optimally inside a Solaris Container.

Feel free to leave a comment if you notice any incorrect information, or if you have generic suggestions to improve documents like these.

Acknowledgments

Many thanks to Prashant Srinivasan, John David Duncan and Margaret B. for their help in different phases of this blueprint.

Posted at 06:14PM Feb 18, 2009 by Giri Mandalika in MySQL  |  Comments[0]

Demonstrating the Features of MySQL Native Driver for PHP, mysqlnd

Support for Persistent Connections

ext/mysqli does not support persistent connections when built with libmysql. However ext/mysqli does support persistent connections when built with mysqlnd. To establish a persistent connection with the MySQL server using ext/mysqli and mysqlnd, prepend the database host with the string "p:" (p stands for persistent) as shown below.

$host="p:localhost";
$port=3306;
$socket="/tmp/mysql.sock";
$user="root";
$password="admin";
$dbname="test";

$cxn = new mysqli($host, $user, $password, $dbname, $port, $socket)
	or die ('Could not connect to the database server' . mysqli_connect_error());

ext/mysql, ext/mysqli and PDO_MySQL support persistent connections when built with mysqlnd.

The new API call mysqli_fetch_all()

mysqlnd extends the ext/mysqli API with one brand new method, mysqli _fetch_all().

mysqli_fetch_all() fetches all result rows and return the result set as an associative array, a numeric array, or both. The method signature is shown below for both procedural as well as object oriented style of programming.

Procedural style:

	mixed mysqli_fetch_all (mysqli_result $result [, int $resulttype])

Object oriented style:

	mixed mysqli_result::fetch_all ([int $resulttype])

where: $result is a result set identifier returned by mysqli_query(), mysqli_store_result() or mysqli_use_result(), and $resulttype is an optional constant indicating what type of array should be produced from the current row data. The possible values for this parameter are the constants MYSQLI_ASSOC, MYSQLI_NUM, or MYSQLI_BOTH. Defaults to MYSQLI_NUM.

Because mysqli_fetch_all() returns all the rows as an array in a single step, it may consume more memory than some of its counterparts like mysqli_fetch_array(). mysqli_fetch_array() returns one row at a time from the result set, hence consumes less memory relative to mysqli_fetch_array(). Besides, if you need to iterate over the result set, you may need a foreach() loop and this approach might be little slower compared to the result set retrieval using mysqli_fetch_array(). Hence consider using mysqli_fetch_all() only in those situations where the fetched result set will be sent to another layer for post processing. If you have to process the fetched result set in the same layer with the help of iterators, then the benefit of using the mysqli_fetch_all() method might be minimal, if there is any.

Statistical Data Collection

mysqlnd collects a lot of statistics which you can use to tune your application. mysqlnd enhances ext/mysqli API with three mysqli_get_XX_stats() methods for easy monitoring and to simplify the bottleneck analysis. For example, using a combination of mysqli_get_XX_stats() methods, one can easily identify a PHP client script that is opening more database connections than it needs or selecting more rows than it consumes.

Accessing Client Statistics:

To access per process client statistics, simply call mysqli_get_client_stats() with no arguments. Similarly to access client statistics per connection, call mysqli_get_connection_stats() with the database connection handle as the argument. Both of these methods return an associated array with the name of the statistic parameter as the key and the corresponding data as the value.

Alternatively per process client statistics can be accessed by calling the phpinfo() method.

The above methods return statistics like bytes_sent, bytes_received to represent the number of bytes sent to and received from the MySQL server, result_set_queries to show the number of queries which generated a result set, buffered_sets, unbuffered_sets to show the number of buffered and unbuffered result sets for the queries generating a result set but not run as a prepared statement. rows_fetched_from_server_normal shows the number of rows that have been fetched from the server using buffered and unbuffered result sets. rows_buffered_from_client_normal shows the number of rows fetched from the server and buffered on the client-side, and rows_skipped_normal shows the number of rows generated by the server but not read from the client.

Accessing Zval Cache Statistics:

mysqlnd collects statistics from its internal zval cache, that you can access by calling mysqli_get_cache_stat() method. This method returns an associative array with the name of the statistic as the key and the corresponding data as the value. The zval cache statistics might be useful to tweak zval cahe related php.ini settings for better performance.

Sample PHP Script Demonstrating mysqlnd's Features

The following sample PHP script demonstrates how to:

• establish persistent connections
• use mysqli_fetch_all() to fetch and display the result set
• access client, connection and zval cache statistics using mysqli_get_client_stats(), mysqli_get_connection_stats() and mysqli_get_cache_stat() methods

The code sample in this tutorial try to retrieve the data from the City table in the MySQL test database. The table structure and the data from the City table are shown below by using the mysql client. MySQL server is running on the default port 3306.

bash# mysql -u root -p
Enter password: admin
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.1.24-rc-standard Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> USE test;
Database changed

mysql> DESCRIBE City;
+----------+-------------+------+-----+---------+-------+
| Field	   | Type	 | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+-------+
| CityName | varchar(30) | YES	|	| NULL	|	|
+----------+-------------+------+-----+---------+-------+
1 row in set (0.07 sec)

mysql> SELECT * FROM City;
+--------------------+
| CityName		|
+--------------------+
| Hyderabad, India   |
| San Francisco, USA |
| Sydney, Australia  |
+--------------------+
3 rows in set (0.17 sec)

The main purpose of the following example is only to illustrate the syntactical use of the new features of mysqlnd. The sample code does not represent any real world scenarios.

bash# cat PHPmysqliClientmysqlnd.php

<?php

	/* create a persistent connection to the MySQL server */
	$cxn = new mysqli("p:localhost", "root", "admin", "test", 3306, "/tmp/mysql.sock")
		or die ('Could not connect to the database server' . mysqli_connect_error());

	$query = "SELECT * FROM City";
	
	/* execute the query */
	if ($cxn->real_query ($query)) {

		/* initiate the result set retrieval */
		if ($result = $cxn->store_result()) {

			/* find the number of rows in the result set */
			$nrows = $result->num_rows;

			echo "\nRetrieved $nrows row(s).\n\n";
			echo "CityName\n--------\n";

			$all_rows = $result->fetch_all(MYSQLI_ASSOC);

			for($i = 0; $i < count($all_rows); $i++) {
				echo $all_rows[$i][CityName] . "\n";
			}
		}

		/* close the result set */
		$result->close();
	}

	echo "\n\nClient Statistics After One Query\n---------------------------------";
	$client_stats = mysqli_get_client_stats();
	#var_dump($client_stats);
	foreach ($client_stats as $key=>$value) {
		if ($value > 0) {
                       	echo "\n$key : $value";
		}
	}

	echo "\n\nStatistics for Connection #1\n----------------------------";
	$conn_stats = mysqli_get_connection_stats($cxn);
	#var_dump($conn_stats);
	foreach ($conn_stats as $key=>$value) {
		if ($value > 0) {
                       	echo "\n$key : $value";
		}
	}

	echo "\n\nCache Statistics After One Query\n--------------------------------";
	$cache_stats = mysqli_get_cache_stats();
	#var_dump($cache_stats);
	foreach ($cache_stats as $key=>$value) {
		if ($value > 0) {
                       	echo "\n$key : $value";
		}
	}

	echo "\n\n=================================\n\n";
	echo "\nEstablishing connection #2 to the MySQL server ..\n\n";

	/* create a non-persistent connection to the MySQL server */
	$cxn2 = new mysqli("localhost", "root", "admin", "mysql", 3306, "/tmp/mysql.sock")
		or die ('Could not connect to the database server' . mysqli_connect_error());

	$query = "SELECT Host, User FROM user";

	/* execute the query */
	if ($cxn2->real_query ($query)) {

		/* initiate the result set retrieval */
		if ($result = $cxn2->store_result()) {

			/* find the number of rows in the result set */
			$nrows = $result->num_rows;
			echo "\nRetrieved $nrows row(s).\n\n";

			echo "Host\t\tUser\n----\t\t----\n";

			$all_rows = $result->fetch_all(MYSQLI_ASSOC);

			for($i = 0; $i < count($all_rows); $i++) {
				echo $all_rows[$i][Host] . "\t" . $all_rows[$i][User] . "\n";
			}
		}

                /* close the result set */
		$result->close();
	}

	echo "\n\nClient Statistics After Two Queries\n-----------------------------------";
	$client_stats = mysqli_get_client_stats();
	#var_dump($client_stats);
	foreach ($client_stats as $key=>$value) {
		if ($value > 0) {
			echo "\n$key : $value";
		}
	}

	echo "\n\nStatistics for Connection #2\n----------------------------";
	$conn_stats = mysqli_get_connection_stats($cxn2);
	#var_dump($conn_stats);
	foreach ($conn_stats as $key=>$value) {
		if ($value > 0) {
			echo "\n$key : $value";
		}
	}

	echo "\n\nCache Statistics After Two Queries\n----------------------------------";
	$cache_stats = mysqli_get_cache_stats();
	#var_dump($cache_stats);
	foreach ($cache_stats as $key=>$value) {
		if ($value > 0) {
			echo "\n$key : $value";
		}
	}

	echo "\n";

	//phpinfo();

	/* close the database connections */
	$cxn->close();
	$cxn2->close();

?>

bash# /export/home/php53/bin/php PHPmysqliClientmysqlnd.php

Retrieved 3 row(s).

CityName
--------
Hyderabad, India
San Francisco, USA
Sydney, Australia


Client Statistics After One Query
---------------------------------
bytes_sent : 90
bytes_received : 222
packets_sent : 2
packets_received : 9
protocol_overhead_in : 36
protocol_overhead_out : 8
bytes_received_ok_packet : 11
bytes_received_eof_packet : 9
bytes_received_rset_header_packet : 5
bytes_received_rset_field_meta_packet : 54
bytes_received_rset_row_packet : 70
packets_sent_command : 1
packets_received_ok : 1
packets_received_eof : 1
packets_received_rset_header : 1
packets_received_rset_field_meta : 1
packets_received_rset_row : 4
result_set_queries : 1
buffered_sets : 1
rows_fetched_from_server_normal : 3
rows_buffered_from_client_normal : 3
rows_fetched_from_client_normal_buffered : 3
rows_skipped_normal : 3
copy_on_write_performed : 3
connect_success : 1
active_connections : 1
active_persistent_connections : 1
explicit_free_result : 1
mem_erealloc_count : 1
mem_efree_count : 2
mem_realloc_count : 1
proto_text_fetched_string : 3

Statistics for Connection #1
----------------------------
bytes_sent : 90
bytes_received : 222
packets_sent : 2
packets_received : 9
protocol_overhead_in : 36
protocol_overhead_out : 8
bytes_received_ok_packet : 11
bytes_received_eof_packet : 9
bytes_received_rset_header_packet : 5
bytes_received_rset_field_meta_packet : 54
bytes_received_rset_row_packet : 70
packets_sent_command : 1
packets_received_ok : 1
packets_received_eof : 1
packets_received_rset_header : 1
packets_received_rset_field_meta : 1
packets_received_rset_row : 4
result_set_queries : 1
buffered_sets : 1
rows_fetched_from_server_normal : 3
rows_buffered_from_client_normal : 3
rows_skipped_normal : 3
connect_success : 1
active_connections : 1
active_persistent_connections : 1
explicit_free_result : 1
proto_text_fetched_string : 3

Cache Statistics After One Query
--------------------------------
put_misses : 3
get_hits : 3
size : 2000
free_items : 1997
references : 3

=================================


Establishing connection #2 to the MySQL server ..


Retrieved 5 row(s).

Host		User
----		----
127.0.0.1	root
localhost
localhost	root
unknown
unknown root


Client Statistics After Two Queries
-----------------------------------
bytes_sent : 190
bytes_received : 501
packets_sent : 4
packets_received : 21
protocol_overhead_in : 84
protocol_overhead_out : 16
bytes_received_ok_packet : 22
bytes_received_eof_packet : 18
bytes_received_rset_header_packet : 10
bytes_received_rset_field_meta_packet : 148
bytes_received_rset_row_packet : 157
packets_sent_command : 2
packets_received_ok : 2
packets_received_eof : 2
packets_received_rset_header : 2
packets_received_rset_field_meta : 3
packets_received_rset_row : 10
result_set_queries : 2
buffered_sets : 2
rows_fetched_from_server_normal : 8
rows_buffered_from_client_normal : 8
rows_fetched_from_client_normal_buffered : 8
rows_skipped_normal : 8
copy_on_write_performed : 13
connect_success : 2
active_connections : 2
active_persistent_connections : 1
explicit_free_result : 2
mem_erealloc_count : 1
mem_efree_count : 2
mem_realloc_count : 4
proto_text_fetched_string : 13
Statistics for Connection #2
----------------------------
bytes_sent : 100
bytes_received : 279
packets_sent : 2
packets_received : 12
protocol_overhead_in : 48
protocol_overhead_out : 8
bytes_received_ok_packet : 11
bytes_received_eof_packet : 9
bytes_received_rset_header_packet : 5
bytes_received_rset_field_meta_packet : 94
bytes_received_rset_row_packet : 87
packets_sent_command : 1
packets_received_ok : 1
packets_received_eof : 1
packets_received_rset_header : 1
packets_received_rset_field_meta : 2
packets_received_rset_row : 6
result_set_queries : 1
buffered_sets : 1
rows_fetched_from_server_normal : 5
rows_buffered_from_client_normal : 5
rows_skipped_normal : 5
connect_success : 1
active_connections : 1
explicit_free_result : 1
proto_text_fetched_string : 10

Cache Statistics After Two Queries
----------------------------------
put_misses : 13
get_hits : 13
size : 2000
free_items : 1987
references : 4

Before concluding, be adviced that some of the experimental functions that are available with ext/mysqli and libmysql are not available with ext/mysqli and mysqlnd. eg., mysqli_embedded_*(), mysqli_*rpl*_()

Related Blog Post:
MySQL Native Driver for PHP, mysqlnd

Acknowledgments
Andrey Hristov & Ulf Wendel, Sun-MySQL AB

Posted at 01:42AM Jan 22, 2009 by Giri Mandalika in MySQL  |  Comments[0]

MySQL Native Driver for PHP, mysqlnd

In order to communicate with the MySQL database server from a PHP application, ext/mysql, ext/mysqli and the PDO MYSQL driver rely on the MySQL client library, libmysql - that has the required implementation for the client-server protocol. The MySQL native driver for PHP (will simply be referred as mysqlnd from this point), is an additional, alternative way to connect from PHP 5 and PHP 6 to the MySQL Server 4.1 or later versions. mysqlnd is a replacement for the MySQL client library, libmysql; and it is tightly integrated into PHP starting with the release of PHP 5.3. That is, from PHP 5.3 onwards the developers can choose between libmysql and mysqlnd when using mysql, mysqli or PDO_MySQL extensions to connect to the MySQL server 4.1 or newer. Due to the tight integration into PHP 5.3 (and later), mysqlnd eliminates the dependency on the MySQL client programming support when the database extension(s) and the database driver are built with the support for mysqlnd.

mysqlnd is not another PHP extension like mysqli nor it has an exposed API to the userland. It is a library that provides almost similar functionality as that of the MySQL client library, libmysql. mysqlnd and libmysql libraries implement the MySQL communication protocol - hence both of those libraries can be used to connect to the MySQL Server.

Since mysqlnd is neither a new extension nor a programming API, but just an alternative to libmysql to connect from PHP to the MySQL Server, there is no need to make changes to the existing PHP scripts. The existing scripts which were running properly with the mysql, mysqli and PDO_MySQL extensions built with libmysql support, continue to run with the exact same behavior even when the mysql, mysqli and PDO_MySQL extensions are built with the mysqlnd support.

From the performance perspective, mysqlnd might be as fast as libmysql; and may even outperform libmysql in some cases. The generic recommendation is to try mysqlnd with your PHP application and to decide based on the performance results.

Some of the advantages of using mysqlnd are listed below:

• Easy to compile: no linking against libmysql, and no dependency on the MySQL client programming support.
• may outperform libmysql in certain cases
• persistent connections for ext/mysqli
• uses PHP memory management, supports PHP memory limit
• reduced memory footprint -- keeps every row only once in memory, where as with libmysql you have it twice in memory
• keeps a long list of performance related statistics for bottle-neck analysis
• client-side result set cache (still in experimental stage)

Installing PHP with the MySQL native driver, mysqlnd

As most of the pre-packaged PHP binary installations may not have the support for mysqlnd enabled by default, it is recommended to build PHP and the required database extensions with mysqlnd from the source code. The rest of this section focuses on the installation of PHP 5.3 from the source code. Check the php-mysqlnd web page at dev.mysql.com for the installation instructions for the prior versions of PHP with mysqlnd support.

1. Get the source code for PHP 5.3 or later versions from http://www.php.net or from http://snaps.php.net/.

2. Make sure that Autoconf 2.13 or later and GNU M4 1.4.7 or later are installed on the machine. Adjust the PATH environment variable to include the path to the autoconf and m4 tools.

   eg.,

   
   bash# ls /usr/local/bin/autoconf ; ls /usr/local/bin/m4
   /usr/local/bin/autoconf
   /usr/local/bin/m4
   
   bash# export PATH=.:/usr/local/bin:$PATH
   
   

3. On Sun Solaris, create a soft link to gmake in the source directory.

   
   bash# ln -s /usr/bin/gmake make
   
   

4. Navigate to the source directory and run buildconf.

   eg.,

   
   bash# cd php5.3-200811132130
   bash# ./buildconf --force
   
   

5. For building PHP 5.3 or later with mysqlnd support on UNIX/Linux systems, you can decide for all three MySQL extensions (ext/mysql, ext/mysqli, PDO_mysql) whether they should be built using mysqlnd or libmysql. When choosing mysqlnd, use "mysqlnd" as path to the mysql client library. If you don't specify "mysqlnd" as library location, by default, PHP tries to use libmysql. It is possible to build one extension with one library, and another extension with another library. For example, you can build mysqli extension with mysqlnd support, and PDO_MYSQL with libmysql.

   The configure option shown below builds all the three extensions with mysqlnd support.

   eg.,

   
   bash# ./configure --prefix=/export/home/php53 --enable-mysqlnd \
          --with-mysqli=shared,mysqlnd --with-mysql=shared,mysqlnd \
   	--with-pdo-mysql=shared,mysqlnd --with-zlib=shared [other options]
   
   

   configure script in PHP 6.0 and some builds of PHP 5.3 may not recognize the --enable-mysqlnd option, so check the configure options by running ./configure --help before specifying --enable-mysqlnd in the list of configure options when building PHP.

   On Windows platform, mysqli extension uses the MySQL Native Driver by default in PHP versions 5.3 and newer. Hence you don't need to configure libmysql.dll

6. On Sun Solaris, pass the -z muldefs option to the linker to pro-actively defend against linker errors like ld: fatal: symbol `<symbol>' is multiply-defined.

   
   bash# export LDFLAGS="-z muldefs"
   
   

7. Build PHP.

   
   bash# make
   
   

8. Install PHP in the destination location.

   
   bash# make install
   
   

9. Enable the required database extensions in php.ini

   eg.,

   
   bash# grep extension php.ini | grep -v \;
   extension_dir=/export/home/php53/lib/php/extensions/no-debug-non-zts-20071006
   extension="mysql.so"
   extension="mysqli.so"
   extension="pdo_mysql.so"
   
   

10. Finally verify the new PHP installation by checking the list of PHP modules.

    eg.,

    
    bash# cd /export/home/php53/bin
    
    bash# ./php -m | grep mysql
    mysql
    mysqli
    mysqlnd
    pdo_mysql
    
    bash# ./php -i | grep -i mysql
    ..
    mysql
    MySQL Support => enabled
    Client API version => mysqlnd 5.0.5-dev - 081106 - $Revision: 1.3.2.18 $
    ..
    mysqli
    MysqlI Support => enabled
    Client API library version => mysqlnd 5.0.1-beta - 070402 - $Revision: 321 $
    ..
    mysqlnd
    mysqlnd => enabled
    Version => mysqlnd 5.0.5-dev - 081106 - $Revision: 1.3.2.18 $
    ..
    pdo_mysql
    PDO Driver for MySQL => enabled
    Client API version => mysqlnd 5.0.5-dev - 081106 - $Revision: 1.3.2.18 $
    ..
    
    

Acknowledgments

Andrey Hristov & Ulf Wendel, Sun-MySQL

Posted at 02:56AM Jan 18, 2009 by Giri Mandalika in MySQL  |  Comments[1]