PHP and two phase commit transactions

12 years ago by Christian Ferrari — view source

unread

Dear all,
I'm the author of LIXA project (http://lixa.sourceforge.net/) and I would discuss a little about "two phase commit" transactions for PHP applications.
"Two
phase commit" means I can write a program that uses two (or more)
Resource Managers (for example databases, like MySQL and PostgreSQL)
with ACID transactions that span all the databases. With two phase
commit I can perform a global commit that commits the changes inside
two or more databases at the same time. This is quite different than
performing two (or more) distinct commits: with distinct commits it may
happen the first one completes and the second one catches an error.

LIXA
project implements two X/Open standards: "XA specification" and "TX
(transaction demarcation) specification". XA describes an interface
between the Resource Managers (databases) and the Transaction Manager
(for example LIXA); TX describes an interface between the Application
Program (your own custom software) and the Transaction Manager (for
example LIXA).

The "TX (transaction demarcation) specification"
is a C API very easy to understand and use when you are developing a C
(or C++) software.

When I started developing LIXA, I figured out
it would have been straightforward to wrap it for other languages like
PHP, Perl, Python, Ruby and so on...

Unfortunately I discovered
it's not easy to extend the TX API to other languages: creating a
wrapper for TX API is trivial because it's just the matter of some SWIG (http://www.swig.org/) hacking.
Integrating the TX wrapped interface with the environment is a completely different task.

These are the issues I partially solved integrating LIXA with PHP.

XA interface is a C level interface: the PHP database drivers must be
changed at C source level to avoid many leaps between C and PHP code.
The most important PHP database drivers are distributed with PHP core
itself => patching their source code is equivalent to patch PHP core.

As a consequence of the first point, the LIXA PHP wrapper itself should
be distributed with PHP core to avoid dependencies between some core
extensions and an external extension.

The TX API does not deal
with the concept of "connection" because it's implicit: every
process/thread has its own connection with every Resource Manager
(database). Better explained: tx_open(), tx_begin(), tx_commit(),
tx_rollback(), tx_close() (and so on...) do not specify the handlers
necessary to distinguish connection objects.
The TX API design does not take into account the concept of "persistent connections" and "connection pooling".

I
afforded issues 1 and 2 creating a global experimental patch for PHP
core itself (you can find it in "ext/php" directory of LIXA tarball or
directly at SVN repository. http://lixa.svn.sourceforge.net/viewvc/lixa/ext/php/).
Issue
number 3 was bypassed and, for the sake of trivial "hello world
equivalent examples", the integration seems to work properly.
Issue 4
is quite difficult to address: I ignored "persistent connection"
disabling LIXA feature when the application asks a persistent
connection, but I was not able to deal with Oracle "connection pools" of oci8
driver.

It's time to made a balance of LIXA and PHP integration: I
"successfully" integrated mysqli (with MySQL supplied library) and pgsql
drivers. There are some examples and 7 case tests available.

I was not able to figure out a smart patch for "oci8" driver and I stopped my effort.

To
conclude, I think 98% of the use cases does not need two phase
commit, but if you are developing the other 2% of the use cases -
without two phase commit - you probably could create some clumsy
"solution" to avoid the "two phase commit issue".

What do you
think about PHP and two phase commit? Do you think the game isn't worth
the candle? Do you think there's room for two phase commit inside the
PHP userland?

In the next future I will concentrate my LIXA efforts
in expanding the number of different Resource Managers for C
programmers because "patching" PHP for LIXA integration is a too hard
task for me myself. Is there someone interested in picking-up LIXA PHP
experimental extension and going further with the task? Is there someone
interested in "two phase commit" and "distributed transaction
processing" in general?

Every comment/hint/suggestion would be appreciated.

Regards
Ch.F.

Decent workarounds outperform poor solutions

12 years ago by Gustavo Lopes — view source

unread

On Tue, 29 May 2012 22:27:53 +0200, Christian Ferrari camauz@yahoo.com
wrote:

These are the issues I partially solved integrating LIXA with PHP.

XA interface is a C level interface: the PHP database drivers must be
changed at C source level to avoid many leaps between C and PHP code.
The most important PHP database drivers are distributed with PHP core
itself => patching their source code is equivalent to patch PHP core.

As a consequence of the first point, the LIXA PHP wrapper itself
should
be distributed with PHP core to avoid dependencies between some core
extensions and an external extension.

The TX API does not deal
with the concept of "connection" because it's implicit: every
process/thread has its own connection with every Resource Manager
(database). Better explained: tx_open(), tx_begin(), tx_commit(),
tx_rollback(), tx_close() (and so on...) do not specify the handlers
necessary to distinguish connection objects.

The TX API design does not take into account the concept of
"persistent connections" and "connection pooling".

I afforded issues 1 and 2 creating a global experimental patch for PHP
core itself (you can find it in "ext/php" directory of LIXA tarball or
directly at SVN repository.
http://lixa.svn.sourceforge.net/viewvc/lixa/ext/php/).
Issue number 3 was bypassed and, for the sake of trivial "hello world
equivalent examples", the integration seems to work properly.
Issue 4 is quite difficult to address: I ignored "persistent connection"
disabling LIXA feature when the application asks a persistent
connection, but I was not able to deal with Oracle "connection pools" of
oci8 driver.

I have some concerns about the approach you're taking. I don't think a
compile time dependency to LIXA is acceptable.

Can't you add some hooks to the several database extensions? From a glance
at your patch, it seems you only need to change the behavior when getting
a new connection and closing one. This shouldn't be much work. Once that
were done, one would be able to plug the LIXA extension (or any similar
extension). I don't think this approach would get objections from the
extension maintainers and it is more versatile.

--
Gustavo Lopes

12 years ago by Christian Ferrari — view source

unread

These are the issues I partially solved integrating LIXA with PHP.

XA interface is a C level interface: the PHP database drivers must be
changed at C source level to avoid many leaps between C and PHP code.
The most important PHP database drivers are distributed with PHP core
itself => patching their source code is equivalent to patch PHP core.

As a consequence of the first point, the LIXA PHP wrapper itself should
be distributed with PHP core to avoid dependencies between some core
extensions and an external extension.

The TX API does not deal
with the concept of "connection" because it's implicit: every
process/thread has its own connection with every Resource Manager
(database). Better explained: tx_open(), tx_begin(), tx_commit(),
tx_rollback(), tx_close() (and so on...) do not specify the handlers
necessary to distinguish connection objects.

The TX API design does not take into account the concept of "persistent connections" and "connection pooling".

I afforded issues 1 and 2 creating a global experimental patch for PHP
core itself (you can find it in "ext/php" directory of LIXA tarball or
directly at SVN repository. http://lixa.svn.sourceforge.net/viewvc/lixa/ext/php/).
Issue number 3 was bypassed and, for the sake of trivial "hello world
equivalent examples", the integration seems to work properly.
Issue 4 is quite difficult to address: I ignored "persistent connection"
disabling LIXA feature when the application asks a persistent
connection, but I was not able to deal with Oracle "connection pools" of oci8 driver.

I have some concerns about the approach you're taking. I don't think a compile time dependency to LIXA is acceptable.

Can't you add some hooks to the several database extensions? From a glance at your patch, it seems you only need to change the behavior when getting a new connection and closing one. This shouldn't be much work. Once that were done, one would be able to plug the LIXA extension (or any similar extension). I don't think this approach would get objections from the extension maintainers and it is more versatile.

--Gustavo Lopes

Dear Gustavo,
I've appreciated you feedback and the approach I followed to produce some experimental code could be changed for sure.
You correctly understood the only necessary changes are related to open/close connection to the resource manager (database); this is a consequence of XA and TX specifications.
I've not completely understood your statement "add some hooks to the several database extensions" and I would need some more details; is this mailing list the right place to discuss about this type of details or is there a better place?
Before starting to implement the interface in "the right way" I have to collect some feedback about the following issues:

Is there any interest in "two phase commit" inside PHP community? Without a real interest, every effort would be useless.
Is there anyone available to help me in implementing the interface between LIXA and database extensions in "the right way"? After some analysis I discovered some database extensions are really sophisticated and injecting a bug is simpler than adding a well implemented feature. I wouldn't break some working code.

Any hint/suggestion would be appreciated.
Ch.F.

12 years ago by Sanford Whiteman — view source

unread

Is there any interest in "two phase commit" inside PHP
community? Without a real interest, every effort would be useless.

I can't speak to a "critical mass" of interest, but as PHP and MySQL
are closely coupled in the real world, until relatively recently (when
Inno became the default) that meant that PHP and MyISAM were best
buds. I don't know how you could do 2PC between two
transaction-unaware back ends. I could see one transaction-aware and
one transaction-unaware working by running synchronously w/the
transactional one first (?).

So my sense is that PHP, because of MyISAM's ubiquity, isn't the ideal
language target for 2PC (compared to Java/.NET where the most
"enterprise" back end is assumed, however inaccurately!).

That doesn't mean there wouldn't be some interest, though.

-- S.

12 years ago by Christian Ferrari — view source

unread

Is there any interest in "two phase commit" inside PHP

community? Without a real interest, every effort would be useless.

I can't speak to a "critical mass" of interest, but as PHP and
MySQL
are closely coupled in the real world, until relatively recently (when
Inno became the default) that meant that PHP and MyISAM were best
buds. I don't know how you could do 2PC between two
transaction-unaware back ends. I could see one transaction-aware and
one transaction-unaware working by running synchronously w/the
transactional one first (?).

LIXA can be coupled only with resource managers that supports some form of XA protocol. When the resource manager is truly XA compliant (Oracle DBMS, IBM DB2, IBM WebSphere MQ) LIXA code is just a wrapper of the XA switch file provided by the resource manager. When the resource manager is partially XA compliant (PostgreSQL, MySQL with InnoDB) LIXA code adds some logic to emulate a standard XA switch file. MySQL itself is affected by this serious bug http://bugs.mysql.com/bug.php?id=12161 that violates XA specification. Despite it, LIXA tries to do its best to support XA for MySQL.
MySQl and MyISAM can not be used in conjuction with LIXA for distributed transaction processing.

So my sense is that PHP, because of MyISAM's ubiquity, isn't the ideal
language target for 2PC (compared to Java/.NET where the most
"enterprise" back end is assumed, however inaccurately!).

That doesn't mean there wouldn't be some interest, though.

-- S.

Ch. F.

12 years ago by Sanford Whiteman — view source

unread

MySQl and MyISAM can not be used in conjuction with LIXA for
distributed transaction processing.

Should be very clear about this in your documentation, as MySQL as RM
will be unsupported in practice in the majority of *MP installations.

You didn't mention MSSQL, I assume because you develop only on Ubuntu
and/or because of other issues, nor SQLite. These cut out wide slices
of general PHP users. (As a side note, I am very frustrated when
extensions act like PHP on Windows doesn't exist so it isn't even
worth a "not supported" note.)

Again, none of these rule out interest, even gotta-get-it-now
interest, in a LIXA extension for PHP, but it mitigates the portion of
users who have the technical environment to act on that interest.

-- S.

12 years ago by Gustavo Lopes — view source

unread

On Wed, 30 May 2012 22:10:56 +0200, Christian Ferrari camauz@yahoo.com
wrote:

You correctly understood the only necessary changes are related to
open/close connection to the resource manager (database); this is a
consequence of XA and TX specifications.
I've not completely understood your statement "add some hooks to the
several database extensions" and I would need some more details; is this
mailing list the right place to discuss about this type of details or is
there a better place?

What I meant was that you should not couple the database extensions to
your C library. Instead of having:

struct conection *conn;

#ifdef LIXA
if (lixa_is_active()) {
conn = lixa_whatever();
} else {
#endif
conn = do_default();
}

prefer something like:

extern struct connection (*get_connection_override)(void);
...
if (get_connection_override) {
	conn = get_connection_override();
} else {
	conn = do_default();
}

You can do this in different ways -- by having a PHP wide variable and
generic function to retrieve and close connections that would be passed
the type of resources an other parameters; you could have a different
scheme for eachDB extension, etc. -- those details are relatively
unimportant.

Is there any interest in "two phase commit" inside PHP community?
Without a real interest, every effort would be useless.

It is very difficult to determine the future demand for something that
does not exist yet. Microsoft introduced filesystem transactions in
Windows Vista, and it's considering deprecating them. But given the
prevalence of JTA in the Java world and the developments in PHP in the
last years I would not be surprised if there was significant demand for
this. And given the current tendency to use more specialized databases
(see JCR client implementations), it may become more important in the
future.

Is there anyone available to help me in implementing the interface
between LIXA and database extensions in "the right way"? After some
analysis I discovered some database extensions are really sophisticated
and injecting a bug is simpler than adding a well implemented feature. I
wouldn't break some working code.

I can review your patches and help you along the way.

Note that stable branches generally do not receive new features, so you
should develop against the development branch (master). If it's your
thing, you can fork PHP on github: https://github.com/php/php-src

--
Gustavo Lopes

12 years ago by johannes@schlueters.de — view source

unread

Dear all,
I'm the author of LIXA project (http://lixa.sourceforge.net/) and I
would discuss a little about "two phase commit" transactions for PHP
applications.

Besides all technical discussions etc one legal comment, on the project
page I read:

    Which type of license applies to LIXA project?
    
    The whole LIXA project is released under the terms of GNU Public
    License (GPL) version 2. 
    http://sourceforge.net/apps/mediawiki/lixa/index.php?title=FAQ:_Frequently_Asked_Questions#License

It then mentions "bla bla bla" about restrictions imposed by the GPL.
One restriction is that the GPL is not compatible with the current PHP
License. Therefore lixa can't be combined with PHP. See
http://www.gnu.org/licenses/license-list.html#PHP-3.01

johannes

12 years ago by Christian Ferrari — view source

unread

Dear all,

I'm the author of LIXA project (http://lixa.sourceforge.net/) and I
would discuss a little about "two phase commit" transactions for
PHP
applications.

Besides all technical discussions etc one legal comment, on the project
page I read:

Which type of license applies to LIXA project?

The whole LIXA project is released under the terms of GNU Public
License (GPL) version 2.

http://sourceforge.net/apps/mediawiki/lixa/index.php?title=FAQ:_Frequently_Asked_Questions#License

It then mentions "bla bla bla" about restrictions imposed by the GPL.
One restriction is that the GPL is not compatible with the current PHP
License. Therefore lixa can't be combined with PHP. See
http://www.gnu.org/licenses/license-list.html#PHP-3.01

johannes

I do know this restriction is in place, but the solution will be quite easy: I could split the project licensing policy: the lixad code (state server) might be GPL and lixac code (client libraries & embedded transaction manager) might be LGPL. The IP of the project is mine, so there would be no issue on this side.
If I did understand there's a really interest about LIXA & PHP, I might move in this direction.

Ch.F.