Hello,
We recently successfully embedded PHP into our application using an
approach based on the embed SAPI.
However, our application is large and complex, written entirely in
C++, requires a bit more functionality than the embed SAPI offers.
(As an example, however, it was invaluable.) So we needed a
customized SAPI, but there was no way it was going to build our live
inside the PHP source tree.
Our solution was to develop a new pseudo-SAPI we call the null SAPI.
All it does is build a complete libphp5.so with no SAPI-related
structures or functions in it at all. Then we can build our real
embedded SAPI -- with all its extra dependencies and goofy custom I/O
handling -- outside the PHP tree and just link to the shared library,
rather than entangling our app's source tree and the PHP source tree.
This was really very straightforward; creating an empty SAPI was
pretty easy and beyond that it only required changes to a couple of
build files.
Is this something that would be of more general interest? It's not
clear if this is exactly the recommended approach, but it certainly
did work well for us and we would be happy to contribute the "work" (I
use this term loosely given the lack of difficulty) done.
Thanks!
Apart from managing lifecycles, the SAPI is also resposible for things like
directing I/O between the host application, how does null-sapi handle this?
Hello,
We recently successfully embedded PHP into our application using an
approach based on the embed SAPI.However, our application is large and complex, written entirely in
C++, requires a bit more functionality than the embed SAPI offers.
(As an example, however, it was invaluable.) So we needed a
customized SAPI, but there was no way it was going to build our live
inside the PHP source tree.Our solution was to develop a new pseudo-SAPI we call the null SAPI.
All it does is build a complete libphp5.so with no SAPI-related
structures or functions in it at all. Then we can build our real
embedded SAPI -- with all its extra dependencies and goofy custom I/O
handling -- outside the PHP tree and just link to the shared library,
rather than entangling our app's source tree and the PHP source tree.This was really very straightforward; creating an empty SAPI was
pretty easy and beyond that it only required changes to a couple of
build files.Is this something that would be of more general interest? It's not
clear if this is exactly the recommended approach, but it certainly
did work well for us and we would be happy to contribute the "work" (I
use this term loosely given the lack of difficulty) done.Thanks!
Apart from managing lifecycles, the SAPI is also resposible for things like
directing I/O between the host application, how does null-sapi handle this?
It doesn't. It provides no SAPI functionality at all (hence "null").
Its only purpose is to allow building SAPIs separately from building
PHP by leaving all that stuff out. The sapi_module_struct and all its
callbacks -- including but not limited to I/O handling -- are provided
later by the externally-built SAPI.
This offers several benefits:
- it makes embedding PHP easier and more flexible.
- it allows new SAPIs to be developed and distributed independently of
the PHP source tree (i.e. as part of a web server or as a separate
package) - it (hypothetically) makes SAPIs modular and able to be selected at
runtime rather than build time - (therefore) it allows multiple SAPIs to coexist in a single installation
Obviously our use case is the first/second.
The last two benefits are hypothetical, since they would require
existing SAPIs to be tweaked.
E.g. currently the CLI SAPI and the apache2 SAPI can coexist, but they
do so by building the entire PHP runtime into both …/bin/php and
…/apache/libexec/mod_php5.so. If they were tweaked to build with
null-sapi:
- the "main" build would be null-sapi which produces only libphp5.so
with no SAPI code - the CLI and apache2handler SAPI's would be built after/separately
from the "main" build, like extensions - the CLI binary and mod_php5.so would both link to the same shared
libphp5.so at runtime, instead of the current approach where they both
contain the entirety of PHP.
Modifying the existing SAPI's would be a bigger step and may not be
desirable for other reasons. It isn't something we have pursued or
consider necessary; the first two benefits were sufficient for us to
justify doing this. But the issue of building multiple competing
SAPIs from one tree does come up from time to time, so if you want to,
imagine a possible bright future world of arbitrary happily coexisting
SAPIs. :)
Thanks!
Ah, I see. I wasn't taking "null" quite literally enough. :)
That sounds pretty awesome to me. Assuming the implementation looks good
you'd get my vote.
Apart from managing lifecycles, the SAPI is also resposible for things
like
directing I/O between the host application, how does null-sapi handle
this?It doesn't. It provides no SAPI functionality at all (hence "null").
Its only purpose is to allow building SAPIs separately from building
PHP by leaving all that stuff out. The sapi_module_struct and all its
callbacks -- including but not limited to I/O handling -- are provided
later by the externally-built SAPI.This offers several benefits:
- it makes embedding PHP easier and more flexible.
- it allows new SAPIs to be developed and distributed independently of
the PHP source tree (i.e. as part of a web server or as a separate
package)- it (hypothetically) makes SAPIs modular and able to be selected at
runtime rather than build time- (therefore) it allows multiple SAPIs to coexist in a single installation
Obviously our use case is the first/second.
The last two benefits are hypothetical, since they would require
existing SAPIs to be tweaked.E.g. currently the CLI SAPI and the apache2 SAPI can coexist, but they
do so by building the entire PHP runtime into both …/bin/php and
…/apache/libexec/mod_php5.so. If they were tweaked to build with
null-sapi:
- the "main" build would be null-sapi which produces only libphp5.so
with no SAPI code- the CLI and apache2handler SAPI's would be built after/separately
from the "main" build, like extensions- the CLI binary and mod_php5.so would both link to the same shared
libphp5.so at runtime, instead of the current approach where they both
contain the entirety of PHP.Modifying the existing SAPI's would be a bigger step and may not be
desirable for other reasons. It isn't something we have pursued or
consider necessary; the first two benefits were sufficient for us to
justify doing this. But the issue of building multiple competing
SAPIs from one tree does come up from time to time, so if you want to,
imagine a possible bright future world of arbitrary happily coexisting
SAPIs. :)Thanks!
Ah, I see. I wasn't taking "null" quite literally enough. :)
That sounds pretty awesome to me. Assuming the implementation looks good
you'd get my vote.
"Implementation" might be a little generous… it's a config.m4 file, a
header file designed to be included by the real SAPI and s some other
header files, and an empty source file. But you're most welcome to
look at it:
Thanks!
To try it out, here's a hacky little Makefile you can use to build the
CLI SAPI against the null shlib:
Resulting in:
$ ls -l phpn
-rwxrwxr-x 1 me staff 113669 Aug 17 00:13 phpn
$ ldd phpn
phpn:
libphp5.so => /usr/local/php/5.6-dev/lib/libphp5.so (0x28080000)
libc.so.7 => /lib/libc.so.7 (0x28487000)
libcrypt.so.5 => /lib/libcrypt.so.5 (0x285ab000)
libpcre.so.3 => /usr/local/lib/libpcre.so.3 (0x285ce000)
librt.so.1 => /usr/lib/librt.so.1 (0x28633000)
libm.so.5 => /lib/libm.so.5 (0x28639000)
libthr.so.3 => /lib/libthr.so.3 (0x28653000)
libz.so.6 => /lib/libz.so.6 (0x28673000)
libssl.so.8 => /usr/local/lib/libssl.so.8 (0x28687000)
libcrypto.so.8 => /usr/local/lib/libcrypto.so.8 (0x286e1000)
libxml2.so.5 => /usr/local/lib/libxml2.so.5 (0x2885e000)
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x2898c000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x28a84000)
$ ./phpn -v
PHP 5.6.0-dev (cli) (built: Aug 17 2013 00:06:53)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.6.0-dev, Copyright (c) 1998-2013 Zend Technologies
Thanks!
Right, gotcha. I think ideally someone should put in the grunt work to do
what you suggested: Build libphp5.so all the time, then link up php/
mod_php5.so/etc... against that as a shared system library (which in turn
other programs or SAPIs could link against).
I'm not sure if anyone has the time and patience to do that (for its
relatively small return), but it'd get my vote.
If nothing else, nice poetry selections.
-Sara
To try it out, here's a hacky little Makefile you can use to build the
CLI SAPI against the null shlib:Resulting in:
$ ls -l phpn
-rwxrwxr-x 1 me staff 113669 Aug 17 00:13 phpn
$ ldd phpn
phpn:
libphp5.so => /usr/local/php/5.6-dev/lib/libphp5.so (0x28080000)
libc.so.7 => /lib/libc.so.7 (0x28487000)
libcrypt.so.5 => /lib/libcrypt.so.5 (0x285ab000)
libpcre.so.3 => /usr/local/lib/libpcre.so.3 (0x285ce000)
librt.so.1 => /usr/lib/librt.so.1 (0x28633000)
libm.so.5 => /lib/libm.so.5 (0x28639000)
libthr.so.3 => /lib/libthr.so.3 (0x28653000)
libz.so.6 => /lib/libz.so.6 (0x28673000)
libssl.so.8 => /usr/local/lib/libssl.so.8 (0x28687000)
libcrypto.so.8 => /usr/local/lib/libcrypto.so.8 (0x286e1000)
libxml2.so.5 => /usr/local/lib/libxml2.so.5 (0x2885e000)
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x2898c000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x28a84000)
$ ./phpn -v
PHP 5.6.0-dev (cli) (built: Aug 17 2013 00:06:53)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.6.0-dev, Copyright (c) 1998-2013 Zend TechnologiesThanks!
Right, gotcha. I think ideally someone should put in the grunt work to do
what you suggested: Build libphp5.so all the time, then link up
php/mod_php5.so/etc... against that as a shared system library (which in
turn other programs or SAPIs could link against).I'm not sure if anyone has the time and patience to do that (for its
relatively small return), but it'd get my vote.
That is something I would volunteer to undertake (including RFC, etc),
but in order to do so in a productive way, someone else would have to
volunteer to do the Windows portion of the work. That's well beyond
my knowledge, ability, and available development hardware. :(
The big preliminary question for me would be, "Is there a specific
design reason why it isn't currently done this way?" PHP already
requires shlib's that depend on shlib's, so that functionality is
probably universally available, but I can't shake the suspicion that
maybe there is some has-to-be-supported platform or use case hiding at
the periphery that requires static linking. (Which could
hypothetically be addressed with a libphp5.a, but that isn't something
I've looked into at all.)
On the flip side of that, if the elsewhere-proposed OS/SAPI supported
list reduction for 5.6 were to come to pass, that might be the right
time for this change as well. (It would also allow those
lesser-used/unused SAPIs to be independently supported if necessary,
without carrying them all along in the tree forever.)
Thanks!
The big preliminary question for me would be, "Is there a specific
design reason why it isn't currently done this way?" PHP already
requires shlib's that depend on shlib's, so that functionality is
probably universally available, but I can't shake the suspicion that
maybe there is some has-to-be-supported platform or use case hiding at
the periphery that requires static linking.
I think it all was about to be as standalone as possible, e.g. you
could have a CLI with builtin readline/pcntl/whatelse extensions,
while mod_php could be kept lean of that and include an
opcache/whatelse instead.
So, I'm not sure if a libphp.so is actually something we want
unconditionally... maybe if it wasn't --enable-null but something like
--enable-libphp, which would create a libphp.so which would be used
to link all of the specified SAPIs.
Examples:
Creates three static SAPIs, all with the same configure options, though:
./configure --with-apxs --enable-cli --enable-cgi ...
Create each SAPI with their own configure options:
./configure --with-apxs ...
./configure --disable-cgi --enable-cli ...
...
Create SAPIs which link against a libphp.so:
./configure --enable-libphp --enable-cli --enable-cgi --with-apxs ...
Now that example actually points to a problem, what if I mix case 2 and 3?
./configure --enable-libphp --enable-cli --disable-cgi ...
./configure --enable-libphp --with-apxs ...
Where "..." could be f.e. enable-maintainer-zts or enable-debug?
--
Regards,
Mike
I think it all was about to be as standalone as possible, e.g. you
could have a CLI with builtin readline/pcntl/whatelse extensions,
while mod_php could be kept lean of that and include an
opcache/whatelse instead.
To stick with your example, of CLI and mod_php -- a good choice as
it's likely the most common combo -- when linking to a common
libphp5.so, the recommended methodology would be to build in only the
functionality required by both, then load the rest as extensions from
php-cli.ini or php.ini, respectively. This seems like the way it is
generally done anyway?
Is doing it this other way a thing that people actually do?
So, I'm not sure if a libphp.so is actually something we want
unconditionally... maybe if it wasn't --enable-null but something like
--enable-libphp, which would create a libphp.so which would be used
to link all of the specified SAPIs.
That's essentially how I interpreted Sara's suggestion, with possibly
some variance on whether or not this would be the default behavior.
If the --enable-shared-phplib (or whatever) flag weren't present, the
alternative would be to build a static library and then link your SAPI
selections against that. The only difference in that scenario is that
the static library could be (optionally and probably not by default)
installed as libphp5.a to allow statically-linked independent SAPI's.
That should produce default behavior almost indistinguishable from
today.
Now that example actually points to a problem, what if I mix case 2 and 3?
./configure --enable-libphp --enable-cli --disable-cgi ...
./configure --enable-libphp --with-apxs ...Where "…" could be f.e. enable-maintainer-zts or enable-debug?
This concern (coexistence of conflicting settings/versions) seems very
similar to this:
On Mon, Aug 19, 2013 at 4:15 PM, Johannes Schlüter
johannes@schlueters.de wrote:
For Unix I'm not sure it is good - it makes it complex to run multiple
PHP versions next to each other which is not only relevant for
developers but also hosters offering multiple versions.
From the standpoint of major versions, versioning the library (e.g.
libphp55.so, libphp-5.5.so) seems sensible. It doesn't seem reasonable
to expect ABI/SPAI compatibility between branches. One could go as
far as libphp55.so.2 for 5.5.2, though that might be overkill.
Centrally installing one libphp55.so (e.g. into /usr/lib or
/usr/local/lib) would still limit one to one set of build options, but
as that is already largely true of centralized installations.
Typically, php.ini needs to be different between development and
production builds of the same version, for example. Or some/all of the
extensions may need to be built with different settings, etc.
However, in practice, this is not a problem we have encountered while
running 5.3, 5.4, 5.5 and master alongside each other, often with
multiple revisions within each major version and devel/debug vs.
production variants of the same version.
That's because we use a different --prefix for each. Then, to link to
a specific build's libphp5.so created with null-sapi, we use the rpath
flag at link time. This has proved to be a good strategy. However,
we were doing this before we switched to this build methodology,
because the speed bump of inter-version collisions had already been
hit. :)
In the worst case scenario, an additional flag like
--with-shlib-suffix=55xyzpdq (producing/linking to libphp55xyzpdq.so)
passed to PHP or SAPI builds would probably allow sufficiently
infinite variety. :) Such an option would choose a sensible default
("55") for most people.
Thanks!
To stick with your example, of CLI and mod_php -- a good choice as
it's likely the most common combo -- when linking to a common
libphp5.so, the recommended methodology would be to build in only the
functionality required by both, then load the rest as extensions from
php-cli.ini or php.ini, respectively. This seems like the way it is
generally done anyway?Is doing it this other way a thing that people actually do?
Ah, there's another thing to mind: Extensions can be marked as "cli
extensions", if statically compiled they become only part of the CLI
binary, not other SAPIs. Think readline & pcntl.
johannes
On Mon, Aug 19, 2013 at 7:49 PM, Johannes Schlüter
johannes@schlueters.de wrote:
Ah, there's another thing to mind: Extensions can be marked as "cli
extensions", if statically compiled they become only part of the CLI
binary, not other SAPIs. Think readline & pcntl.
The logical expectation then would be not to include them in the shlib
build, as that is not the CLI binary. Of course nothing would prevent
making them part of a CLI binary that links to the shlib.
But definitely, in any case where statically-linked anything is the
goal, using an option that builds almost all the code into a shared
library may cause an endless and entertaining variety of problems and
conflicts. The solution for such cases would appear to be fairly
straightforward: do not use that option in those cases where it
produces a non-viable result.
Thanks!
Right, gotcha. I think ideally someone should put in the grunt work to do
what you suggested: Build libphp5.so all the time, then link up
php/mod_php5.so/etc... against that as a shared system library (which in
turn other programs or SAPIs could link against).I'm not sure if anyone has the time and patience to do that (for its
relatively small return), but it'd get my vote.That is something I would volunteer to undertake (including RFC, etc),
but in order to do so in a productive way, someone else would have to
volunteer to do the Windows portion of the work. That's well beyond
my knowledge, ability, and available development hardware. :(The big preliminary question for me would be, "Is there a specific
design reason why it isn't currently done this way?" PHP already
requires shlib's that depend on shlib's, so that functionality is
probably universally available, but I can't shake the suspicion that
maybe there is some has-to-be-supported platform or use case hiding at
the periphery that requires static linking. (Which could
hypothetically be addressed with a libphp5.a, but that isn't something
I've looked into at all.)
On Windows we do this already. We have php.exe apache dlls with the SAPI
and then php5[ts].dll.
For Unix I'm not sure it is good - it makes it complex to run multiple
PHP versions next to each other which is not only relevant for
developers but also hosters offering multiple versions.
johannes
Our solution was to develop a new pseudo-SAPI we call the null SAPI.
All it does is build a complete libphp5.so with no SAPI-related
structures or functions in it at all. Then we can build our real
embedded SAPI -- with all its extra dependencies and goofy custom I/O
handling -- outside the PHP tree and just link to the shared library,
rather than entangling our app's source tree and the PHP source tree.
I was successful by using embedded SAPI for that and ignoring all things
from php_embed.[ch]. Sure this gives few extra symbols and a few bytes
of unneeded code but works fine.
johannes
On Fri, Aug 16, 2013 at 12:55 PM, Johannes Schlüter
johannes@schlueters.de wrote:
I was successful by using embedded SAPI for that and ignoring all things
from php_embed.[ch]. Sure this gives few extra symbols and a few bytes
of unneeded code but works fine.
Yes, exactly. This is a cleaner/more formal way of doing that,
without the extra symbols and unneeded code.
This is a small change that is no trouble to keep on our local branch.
My question is basically whether it's of value to anyone but us and
"no" is a perfectly valid answer. :)
Thanks!