Internet Super Hero

Langweiler

Wenn ein Blog ein Abbild des Lebens darstellt, dann sollte ich das wordle.net Warnsignal beachten.

MySQL Workbench is using Connector/C++ pre-alpha snapshot

Eat your own dog food. The latest development version of MySQL Workbench successfully runs a pre-alpha snapshot of the MySQL Driver for C++ since a few weeks. Enjoy your pizza at my costs, Andrey (Hristov). I lost my bet. Less than five bugs have been found when migrating MySQL Workbench to Connector/C++.

MySQL Connector/C++ is now being used by two "internal customers": MySQL Workbench and MySQL Connector/OpenOffice.org. And our internal development version of Connector/OpenOffice.org runs on Connector/C++ as well. The preview version of Connector/OpenOffice.org was using the MySQL Client Library (C-API)

Weiter lesen »

PHP: How mysqlnd async queries help you with sharding!

Recipe for conference talks: add a buzzword like "sharding" and show how mysqlnd helps you with asynchronous queries. Only two PHP database extensions give you asynchronous queries: mysqlnd and Postgres. Slides (OpenOffice) from the International PHP Conference 2008 have the details about the asynchronous query API in ext/mysqli (using mysqlnd).

Asynchronous, non-blocking IO is an old favourite. Although its an old idea, and it is a well known technology that is supported in many environments, it is still new for PHP database extensions. A year ago, I blogged about parallel and asynchronous queries in mysqlnd. Shortly after we had it implemented but never tested it and therefore never pushed it forward. Parallel queries, using a background thread in PHP, did not turn out to fly. But asynchronous queries have some potential.

Weiter lesen »

Kücken, die vergebens den Schnabel aufreissen

Ich habe gerade frei und räume die Festplatten auf. Beim Aufräumen fand ich auf diese Fotos von Jungtieren der Küstenseeschwalbe, welche vergebens den Schnabel aufreissen.

PHP: New network traffic, CPU and memory savings with mysqlnd

BIT and CPU squeezing with Facebook, Tuenti and Andrey - for the die hard experts… The MySQL Client Server Protocol supports two flavours of sending database results to the client send as text and as binary. Unlike the text protocol, the binary protocol avoids converting data into strings whenever possible. For example, the binary protocol send a TINYINT column value as one byte and not as a length coded string of a length of one to four bytes. This saves CPU cycles on the server by avoiding a cast operation and it saves at least one byte of network traffic - sometimes. Furthermore, ext/mysqli will use an (PHP) integer variable instead of a string to hold the column value. This saves memory when caching database results in APC (see below for memcached). A MySQL server patch and an experimental mysqlnd branch allows you to use the binary protocol for all statements - not only prepared statements. Furthermore, you can get native types support even without touching the server.

Here is an example of the difference it makes to store a hash in APC which uses strings to hold integer values versus a hash that uses integers to hold the same integer values. If the hash would have been created by fetching results from MySQL using mysqli_query(), it would store all the integers using strings and it would require 102 kB. The same database result would require only 100 kB, if ext/mysqli would return integers as PHP integers and not as strings. Not a big deal, but this is just one example and even small differences can sum up. We have new fine tuning screws to play with…

$integers = array();
$strings = array();
for ($i = 0; $i < 169; $i++) {
  $integers[$i] = array($i, $i, $i);
  $strings[$i] = array((string)$i, (string)$i, (string)$i);
}
	
apc_clear_cache("user");
	
apc_store("integers", $integers, 10);
apc_store("strings", $strings, 10);
	
$info = apc_cache_info("user");
foreach ($info['cache_list'] as $details) {
	printf("Cache entry %-10s requires %d bytes\n", $details['info'], $details['mem_size']);
}

Cache entry integers   requires 100318 bytes
Cache entry strings    requires 102014 bytes

Weiter lesen »

Slides (german) from MySQL Connectors talk at the Hamburg MySQL Meetup

On monday I gave a presentation at MySQL user group meeting in Hamburg on the team I’m working for: the MySQL Connectors team. The presentation was given in german. Unfortunately about 99.99999% of all PlanetMySQL had no chance to join the meetup, but you can find the slides (german language) here for download (PDF). The slides give an overview on the team, its products and its activties. You will also find introduction material on those Connectors (drivers) I am working on: Connector/OpenOffice.org, Connector/C++ and all the PHP stuff.

Weiter lesen »

Duo mit Kopfschmerzen

Unsere Duo war im früheren Leben eine Schrotflinte. Der Zylinderkopf zeigt deutliche Spuren von Geschossen. Wahrscheinlich gab es Tage an denen sich Kugellager aus dem Inneren des Motors sich ihren Weg nach draußen gebahnt haben. Die vermeintlichen Fluchthelfer sind der Zylinder und der Auspuff.

Der Motor ist im OP: Zylinder einschleifen, neuer Kolben, neue Kurbelwelle, neue Lager. Nach 30 Jahren und 5.500 Kilometern erblicken die Original DDR-Teile zum ersten Mal wieder die Sonne. Ob sie 1978 beim Einbau damit gerechnet haben ihren Lebensabend unter der Westsonne zu verbringen?

(Not) Using OpenSolaris 2008.05 for Connector/OpenOffice.org development (I)

While Andrey made good progress with the Connector/OpenOffice.org I wasted my time trying to use OpenSolaris 2008.05 as an OpenOffice.org development platform. OpenSolaris is not on the list of platforms used my the maintainers and developers of OpenOffice.org. Another open technology - JDK 1.5 - has lowered my openness towards new development platforms.

If you are an OpenOffice.org end-user, skip this blog posting: grab a binary package and be happy! That’s better for the world-wide climate because your power consumption will be lower. An Intel Core 2 Duo E6750 class PC needs some 4-6 hours to compile OpenOffice.org. As a plain vanilla user, its better for your notebook hard disk to use a pre-compiled binary. When Georg started his OpenOffice.org initiative he burned his notebook hard disk within short time. Burned means burned: overheated. OpenOffice.org is a monster: du -h […] 2,0G oo.org-m28/.

The posting might be of more interest for developers.

Weiter lesen »

In the (german) Press: Linux Technical Review 09/2008 on MySQL

The german print magazine Technical Review features two MySQL articles in his latest issue 09/2008. Kaj gets interviewed on the question what the future plans of Sun are with regards to MySQL. And I give answers to five technical questions on MySQL and upcoming features.

Weiter lesen »

MySQL Connector/C++ preview is available!

A new MySQL product is born. The preview release of MySQL Connector/C++ is available for download at http://downloads.mysql.com/forge/connector_cpp_preview . Connector/C++ is a new database driver to connect C++ applications to the MySQL Server. The preview mimics parts of the JDBC 3.0 API and is licensed under the terms of the GPL (plus FLOSS License Exception). Have a look at the announcement and the MySQL Connector/C++ wiki page for further “official” information.

Motivation and Advantages

The Connector/C++ is based on an initial development from MySQL GUI team. The GUI team needed a C++ based connector with a JDBC like API. Andrey Hristov from the MySQL Connectors team had been assigned to take the initial code over from Vladimir Kolesnikov. He worked on it - to a different degree - over several months. Andrey is now the main developer of two Connectors for two of the five most common programming languages according to the TIOBE Programming Community Index - PHP and C++. Andrey, that is outstanding work! The TIOBE Index is in line with a MySQL quick poll asking for the primary programming language used for developing MySQL applications: C++ is very common and deserves a dedicated Connector.

Weiter lesen »

PDO_MYSQLND: The new features of PDO_MYSQL in PHP 5.3

PDO_MYSQLND is in the PHP CVS repository at php.net: PDO_MYSQL has been patched (PHP 5.3, PHP 6.0). Try out PDO_MYSQL with the MySQL native driver for PHP (mysqlnd). Its has new features.

Since Kaj’s announcement of the MySQL native driver for PHP (mysqlnd), we promised to to patch all three PHP MySQL extensions (ext/mysql, ext/mysqli and PDO_MYSQL) optionally support it. Almost exactly a year ago, on July 27th 2007, mysqlnd and updated versions of ext/mysql and ext/mysqli have been checked into the PHP 6 (HEAD) branch of the PHP CVS repository. PDO_MYSQL was the last extension to be upgraded. We have choosen PDO_MYSQLND as a working title for the upgrade PDO_MYSQL. On Monday, July 21st 2008, Johannes has committed the patch into the CVS with a short comment.

 Log:
  - Add mysqlnd support for PDO_mysql, fixes at least bug#41997,#42499,  pecl#12794, pecl#12401

Weiter lesen »

PDO_MYSQLND: Monday morning murders

I am a member of the monday morning murder association. I murdered bugs. As said earlier, the modification of PDO_MYSQL to support both the MySQL native driver for PHP (mysqlnd) and the MySQL Client Library (AKA libmysql) is progressing. A good number of known and unknown bugs has been killed. But some new have been found as well - an overview.

Weiter lesen »

PDO_MYSQLND: Mighty multiple-queries and PDOStatement::nextRowset()

MySQLs multiple-queries feature makes it possible to run two or more SQL commands with one function call: SQL injections made easy. The PDO design is said to be optimized for security. For security reasons PDO has gone the stony road of prepared statements. If security matters, why does PDO_MYSQL explicitly enable multiple-queries? If portability matters, why do you use a MySQL specific feature?

Mighty multiple-queries feature: performance

Multi-statement is yet another example of how optimization works: trade in one feature (security) for another (performance). Sending several SQL command with one function call saves communication round-trips. You need only one round-trip for executing n > 1 SQL statements. This can be a mighty feature if you need to tune MySQL to the extreme. The price you pay is that you have to check carefully what SQL commands you send.

Weiter lesen »

PDO: Learn how the PDO SQL parser causes bug reports

The PDO SQL parser causes many bugs. All drivers suffer from it. By default the parser replaces placeholder with '<bound_value>'.'<bound_value>' is a string. This can clash with the SQL syntax. PDO can make an INSERT fail. Please learn why and stop reporting bogus bugs.

Why does PDO parse my SQL commands?

PDO has chosen Prepared Statements as its preferred way of running database queries. The entire API is build around prepared statements. Wez has explained that primarily security considerations have caused this decision. Read PDO_MYSQLND: Prepared Statements, again for a discussion of prepared statements in the context of PDO.

Not every database system supports prepared statements. And those who do use a different syntax for placeholder. Some systems support named parameters (SELECT * FROM test WHERE id = :name), some support positional/ordinal parameter (SELECT * FROM test WHERE id = ?) and some support both. The cause of the differences is the lack of a standard. PDO tries to hide the differences from the user. This is done by a prepared statement emulation. The emulation includes a SQL parser which allows you to use whatever format you prefer for placeholder. If the underlying database system does not support the placeholder format you have chosen, PDO emulates it for you.

In many cases the emulation works flawless. But the bug databases on bugs.php.net and pecl.php.net show plenty of reports that boil down to the PDO SQL parser. Please learn how the SQL parser works. This should help avoiding duplicates in the bug database.

Weiter lesen »

PDO_MYSQLND: Calling Stored Procedures works fine with mysqlnd

Mike is asking I`m hoping that MySQL stored procedures will work properly. Especially multiple calls in one request. in reply to PDO_MYSQLND: R[a|u]mbling and a breeze of progress. Mike, it depends what you mean by "properly". If you use the API properly it works fine with mysqlnd. Read on for code examples.

Executing a basic stored procedure twice

Code take from the internal test pdo_mysql_attr_oracle_nulls.phpt. Status: PASS. NOTE the use of $stmt->nextRowset(). If you do not call $stmt->nextRowset() in a loop to fetch all result sets you will get the usual 2014 error. MySQLs Stored Procedures always return n (here: 1) result sets for the actual data followed by another result set with the status code. You must use PDOStatement::nextRowset() to fetch them.

Weiter lesen »

PDO_MYSQLND: R[a|u]mbling and a breeze of progress

The modification of PDO_MYSQL to support the MySQL native driver for PHP (mysqlnd) is progressing. We are using the project title “PDO_MYSQLND” for the modification. The goal of PDO_MYSQLND is to provide a PDO driver for MySQL which can be compiled either against the MySQL Client Library or against the MySQL native driver for PHP. This is the same type of modification we did with ext/mysql and ext/mysqli already.

The use of any of the libraries is transparent for the PHP user. You may continue to use the MySQL Client Library, like you do today with PDO_MYSQL, or give mysqlnd a try. The MySQL native driver for PHP (mysqlnd) is easier to compile as its tightly integreated into the PHP internals and ships with PHP as of version PHP 5.3. In case of ext/mysql and ext/mysqli our benchmarks and first user feedback indicates that mysqlnd gives you at least the same performance of libmysql. And sometimes it is faster and more memory efficient. Read below and see my blog archive for more on mysqlnd.

PDO_MYSQLND test results

Finally, the PDO_MYSQLND test results are looking fine. On a 64bit Linux box PDO_MYSQLND compiled against mysqlnd passes 98.8% of the tests and the code coverage is above 90%. When compiled against libmysql, the results are a little worse: 97.7%. In case of mysqlnd, the remaining 1.2% test failures are PDO (not driver) bugs. We are making use of the new --XFAIL-- feature of the PHP test tool run-tests.php to mark them as expected failures as long as we do not have the time to fix PDO itself as we need to finish our driver itself.

Weiter lesen »

PDO_MYSQLND: Prepared Statements, again

Server-side Prepared Statements are an outdated technology from ancient times, are they? Brian gives a long list of arguments in his blog posting “Prepared Statements, Musings” why one should think twice before using server-side prepared statements. PDO does enforce the use of prepared statements for all statements which return a result set. Good or bad?

• What are (server-side) Prepared Statements?
• Pro’s and Con’s
• MySQL myths
• MySQL specifics
• What enforcing Prepared Statements means
• PDO’s prepared statement emulation
• "Abusing" Prepared Statements - 10% performance loss
• Using a Prepared Statement for the INSERT - 10% gain
• Using a Prepared Statement for the SELECT
• Prepared statements a win?

Weiter lesen »

Unser neues Knattert: Duo 4/1!

Jetzt ist es endlich sicher. Sie sitzt im Flieger auf dem Rückflug vom Kurzurlaub. Ich kann endlich über unser neues Knattert bloggen ohne den Überraschungseffekt aufs Spiel zu setzen. Seit nunmehr zehn Tagen möchte ich es in die Welt rausrufen und fiebere auf den Moment hin an dem Steffi das Knattert zum ersten Mal zu sehen, hören und riechen bekommt. Ein Simson Duo 4/1 spricht einfach alle Sinne an! Typisch für einen 2-Takter mit Baujahr 1978. Ich scheine irgendwie immer bei diesem Baujahr zu landen. Meine Gummikuh, eine BMW R80/7, die leider nach einem Ventilabriß bei Tempo 100 nur noch als Organspender diente, stammte auch aus dem Baujahr.

Der Vorbesitzer mochte sich kaum vom Duo trennen. Er hat es, als passionierter Hobbyschrauber, aus einem rollenden Teilelager vor drei Jahren wieder aufgebaut. Dadurch zeigt es eine gute Substanz. Bei der ersten Durchsicht sind nur ein fehlender Außenspiegel, ein falsch montierter Auspuff - möglicherweise die Ursache fehlender Leistung - , ein defektes Standlicht und das Fehlen der Scheibenwaschanlage aufgefallen. Es tut sich etwas schwer beim Kaltstart, fuhr den Vorbesitzer jedoch lange Zeit treu zur Arbeit und hat die ersten 100km nach einjähriger Pause klaglos gemeistert. Einzig der für die Lackierung verwendete Kunstharzlack wurmt mich etwas, weil dies eine (professionelle) Lackierung mit 2-K Lacken verhindert. Ja, richtig erkannt, ich habe ein neues Spielzeug… - und ich habe bereits rausgefunden, daß es in Kiel eine Simson-Szene gibt.

Vielleicht wird es ja als 78er-Familienmitglied genehmigt und begleitet uns ein paar Jahre. Es kommt zwar nicht wie eine Ape oder ein Fiat 500 aus Italien, aber eine zweisitzige Ape ist ebenso unbezahlbar wie ein gut restaurierter Fiat 500.

Nachtrag (19.06.2008): Das Knattert wurde angenommen. Zur Kennzeichnung seiner Vollwertigkeit hat eine nette Politesse ihm sein erstes Knöllchen verpasst wegen Parkens ohne Parkschein. Knattert, jetzt bist Du der Stadt Kiel vorgestellt worden.

PHP: PDO_MYSQLND 1.0.2-alpha released

I am glad to announce the availability of the first alpha version of PDO_MYSQLND. PDO_MYSQLND is a PHP PDO driver for MySQL based on the MySQL native driver. PDO_MYSQLND 1.0.2-alpha is available for download on http://downloads.mysql.com/forge/pdo_mysqlnd_preview. Please read the announcement and check the MySQL Forge project page on PDO_MYSQLND for further information.

What is PDO_MYSQLND?

PDO (PHP Data Objects) is a data-access abstraction layer for PHP. PDO consists of a core and a rich set of database specific drivers. Traditionally there has been one PDO driver for MySQL called PDO_MYSQL. PDO_MYSQL is a community developed and community driven PDO driver. PDO_MYSQLND is a new PDO driver for MySQL based on it. PDO_MYSQLND is developed by MySQL and will be maintained by MySQL.

Weiter lesen »

Gehört die Unconference zum Establishment?

Die zweite Ausgabe der PHP-Unconference in Hamburg am Wochenende (26./27. April 2008) übertraf die kühnsten Erwartungen.

Auf dem Rückweg vom ersten PHP Vikinger im Sommer 2006 durfte ich Darren Cooper (PHP UG Frankfurt) kennenlernen. Wir verbrachten einen wunderbaren Nachmittag in Oslo bei strahlendem Sonnenschein und sinnierten über Community-Events wie die Vikinger-Unconference und Usergroups. Wir hielten noch einige Wochen Kontakt per E-Mail, verloren uns dann etwas aus den Augen, chatteten gelegentlich mussten uns jedoch zwei Jahres bis zum letzten Wochenende gedulden, bevor wir uns wiedersahen.

Unsere alten Diskussionen und Ideen kann man in sechs Worten zusammenfassen: PHP-Unconference in Hamburg - 2008 und 2009. Die PHP-Unconference Hamburg ist bereits im zweiten Jahr im Establishment angekommen. Es ist dem lokalen Team gelungen 120 Gäste aus dem gesamten Bundesgebiet zu mobilisieren und zu betreuen. In vier Räumen wurden an zwei Tagen über 30 Vorträge gehalten und unzählige Diskussionen in den sinnvoll bemessenen Freiräumen und Pausen durchgeführt. Die Liste der Referenten steht der einer kommerziellen Konferenz nicht nach. Wohl aber der Unkostenbeitrag von 10 Euro. Unter den Referenten waren:

• … Zend Deutschland
• … die Väter von ext/session
• … die Nummer 60 der “100 Most Influential People in IT”
• … Consultants und Trainer, die pro Minute 5 Euro kosten
• … zwei Mitglieder der PHP-Group
• … Gewinner des red dot Design Awards, Designpreis der Bundesrepublik und Nominierte für den Grimme Online Award
• … und so weiter!

Ich will fast ausrufen und feststellen: Konferenzen haben ausgedient! Dem Organisationsteam und all seinen Helfern ist es gelungen, die Unconference auf das Radar prominenter Sponsoren zu bringen:

• Mayflower
• MySQL
• O’Reilly
• PACKT Publishing
• Sektion Eins
• Soprado

Kommerzielle Konferenzen haben aufgrund ihres ungleich höheren Budgets die Möglichkeit mehr Annehmlichkeiten für die Teilnehmer zu bieten. Aber wer mit dem Anspruch des Konsums an eine Unconference herantritt ist am falschen Platz. Zugeschaut, mitgebaut ist das Motto. Wer schon keinen Vortrag hält, der kann zumindest Freunde und Bekannte bei sich beherbergen. Wer keinen Platz hat für Gäste, der kann Vorträge im Wiki protokollieren oder mit dem im Freundeskreis ausgeliehenen Camcorder die Sessions aufzeichnen.

Manchen kommerziellen Konferenzen gelingt es aufgrund des h¨heren Budgets spannendere Veranstaltungsorte anzubieten. Gerüchten zu Folge ist auch dieser letzte Gedanke, den die zwei Sonnenanbeter aus Oslo nie in die Tat umgesetzt haben, für 2009 im Gespräch.

Wie ich schon letztes Jahr sagte: ich bin ein Fan dieser Veranstaltung, es ist ein Pflicht-Veranstaltung, die Veranstaltung ist hiermit etabliert. Ich wünsche mir, daß das Orga-Team auch nächstes Jahr mit so viel Elan bei der Sache ist. Community Events stehen und fallen mit der Existenz einer Keimzelle, mit dem Vorhandensein eines aktiven Orga-Teams. Danke an das Orga-Team und alle Helfer!