Porting PECL to userspace

Hi,

1. Is this even a viable approach? It seems like it, but to my
   knowledge no one else has done this to any serious extent which makes me
   wonder if there's a reason the road less traveled is less traveled.

This is a case by case thing. Most modules talk to a C library. You'd
have to recreate the functionality. Some things can be done. Some things
can be done but will be slooooowww. Some things won't work.

2. Is anyone else doing this? No sense doing it ourselves if someone
   else already is.

There are some things where a similar solution exists already in PEAR or
some other repository. They often have a different interface (leveraging
PHP while extensions are formed more or less after a C lib) but can
easily be swapped.

3. What would be the cleanest way to do so?

Case by case.

We had the thought of
partially automating the process by having PHP auto-generate at the very
least the subs of any classes and functions that the module provides.
However, when my colleague tried using the same parser as is used for
generating documentation he says he got several times as many classes as
the manual says the module has. We were using the PECL HTTP module as
our sample (http://www.php.net/http). (I don't know the exact details
of what he did at the moment.) Is that not a viable approach? Would we
be better off using reflection? Is there some other tool we're not
aware of?

Well, HTTP is a doable case, I assume. Creating stubs from reflection
shouldn't be too hard. While lots of HTTP classes already exist. Why
build yet another one?

In other cases I think you shouldn't follow necessarily the extensions
interface but build one that makes sense[tm].

If viable I'd love if this would start a trend, but we'll see where it
goes. I know it wouldn't work for all PECL modules, obviously, but I
suspect it could work for several, and provide an easy way for different
PHP projects to share backend code without needing lots of C developers.

Well, in general PHP is grown-up and usable enough that more and more
stuff can be done in pure PHP. Main reasons for using C are that
existing C libraries can be used and C code is still way faster. Look at
the stuff frameworks provide ... the way should actually be the other
way round than what you describe: Stuff is done in PHP andportedtoC when
needed.

johannes

Hi all.

I'm working with a fellow developer on an experimental project.  There are
some PECL modules that we want to try and use in an open source project
where we cannot guarantee that PECL modules will be available, since it's
intended for widespread distribution on both shared hosts and custom
hosting. The thought we had was to do a user-space port of the PECL module
to include in the project and rely on that.  Then if the PECL module is
installed, we don't include the library (either via an extension_loaded()
check or just relying on autoload) and the PECL implementation gets used
instead.  Poof, nice speed boost.

The questions I have are:

1. Is this even a viable approach?  It seems like it, but to my knowledge no
   one else has done this to any serious extent which makes me wonder if
   there's a reason the road less traveled is less traveled.

2. Is anyone else doing this?  No sense doing it ourselves if someone else
   already is.

3. What would be the cleanest way to do so?  We had the thought of partially
   automating the process by having PHP auto-generate at the very least the
   subs of any classes and functions that the module provides.  However, when
   my colleague tried using the same parser as is used for generating
   documentation he says he got several times as many classes as the manual
   says the module has.  We were using the PECL HTTP module as our sample
   (http://www.php.net/http).  (I don't know the exact details of what he did
   at the moment.)  Is that not a viable approach?  Would we be better off
   using reflection?  Is there some other tool we're not aware of?

If viable I'd love if this would start a trend, but we'll see where it goes.
 I know it wouldn't work for all PECL modules, obviously, but I suspect it
could work for several, and provide an easy way for different PHP projects
to share backend code without needing lots of C developers.

--Larry Garfield

If you using an extension that exists as glue to a third party
library, then you are going to need PHP to communicate at a very low
level. Something I'm not sure would be possible at the moment.

If the extension is self contained, then this would more likely be a
good candidate.

You can use reflection to build the stubs easily enough

But I'm not sure all the things you can do in C can be done in PHP,
not without a LOT of effort. Talking to third party libraries or
making c calls to the OS is out.

Richard.

Richard Quadling
Twitter : EE : Zend : PHPDoc
@RQuadling : e-e.com/M_248814.html : bit.ly/9O8vFY : bit.ly/lFnVea

AMF-PHP does something similar to this.
It is a PHP implementation of AMF (Action Message Format) so it allows you
to accept and respond to AMF messages from Flash apps.

Its native PHP encoder is extremely slow but it works. However, if you
install amfext from PECL it will use that for encoding instead.

AMF-PHP is more than just an encoder, though; whereas the extension is only
used for the encoding part.
So, in that regard, it may not be the best example to go by for what you
want to do.

• Andrew

On Fri, May 20, 2011 at 12:15 PM, Larry Garfield larry@garfieldtech.comwrote:

Hi all.

I'm working with a fellow developer on an experimental project. There are
some PECL modules that we want to try and use in an open source project
where we cannot guarantee that PECL modules will be available, since it's
intended for widespread distribution on both shared hosts and custom
hosting. The thought we had was to do a user-space port of the PECL module
to include in the project and rely on that. Then if the PECL module is
installed, we don't include the library (either via an extension_loaded()
check or just relying on autoload) and the PECL implementation gets used
instead. Poof, nice speed boost.

The questions I have are:

1. Is this even a viable approach? It seems like it, but to my knowledge
   no one else has done this to any serious extent which makes me wonder if
   there's a reason the road less traveled is less traveled.

2. Is anyone else doing this? No sense doing it ourselves if someone else
   already is.

3. What would be the cleanest way to do so? We had the thought of
   partially automating the process by having PHP auto-generate at the very
   least the subs of any classes and functions that the module provides.
   However, when my colleague tried using the same parser as is used for
   generating documentation he says he got several times as many classes as the
   manual says the module has. We were using the PECL HTTP module as our
   sample (http://www.php.net/http). (I don't know the exact details of what
   he did at the moment.) Is that not a viable approach? Would we be better
   off using reflection? Is there some other tool we're not aware of?

If viable I'd love if this would start a trend, but we'll see where it
goes. I know it wouldn't work for all PECL modules, obviously, but I
suspect it could work for several, and provide an easy way for different PHP
projects to share backend code without needing lots of C developers.

--Larry Garfield

--

To unsubscribe, visit: http://www.php.net/unsub.ph<http://www.php.net/unsub.php

Hi all.

I'm working with a fellow developer on an experimental project. There
are some PECL modules that we want to try and use in an open source
project where we cannot guarantee that PECL modules will be available,
since it's intended for widespread distribution on both shared hosts
and custom hosting. The thought we had was to do a user-space port of
the PECL module to include in the project and rely on that. Then if
the PECL module is installed, we don't include the library (either via
an extension_loaded() check or just relying on autoload) and the PECL
implementation gets used instead. Poof, nice speed boost.

The questions I have are:

1. Is this even a viable approach? It seems like it, but to my
   knowledge no one else has done this to any serious extent which makes
   me wonder if there's a reason the road less traveled is less traveled.

One reason would be because it requires fixing bugs and implementing
features twice, so it effectively doubles the required work.

2. Is anyone else doing this? No sense doing it ourselves if someone
   else already is.

There's a Drupal extension that does something like this, but in reverse:
http://drupal.org/project/drupal_php_ext

Cheers,
David

On Fri, May 20, 2011 at 12:15 PM, Larry Garfield larry@garfieldtech.comwrote:

I'm working with a fellow developer on an experimental project. There are
some PECL modules that we want to try and use in an open source project
where we cannot guarantee that PECL modules will be available, since it's
intended for widespread distribution on both shared hosts and custom
hosting. The thought we had was to do a user-space port of the PECL module
to include in the project and rely on that. Then if the PECL module is
installed, we don't include the library (either via an extension_loaded()
check or just relying on autoload) and the PECL implementation gets used
instead. Poof, nice speed boost.

I do a fair amount of this with Flourish (http://flourishlib.com), since my
goal is to have all functionality work with only the extensions that come
with a standard PHP install (plus at least one database extension for the
desired database). I also back-port some functionality since my goal is for
PHP 5.1+ compatibility. When the PHP extension in question is available and
known not to contain show-stopper bugs, I'll use it. Otherwise I fall back
to a native PHP implementation of the functionality. Granted, the native PHP
is much slower, but people who are really concerned about performance can
always do the work of installing the extensions. In practice this seems to
work fairly well.

So far I have the following implemented with native PHP fallbacks:

mbstring (UTF-8 only): fUTF8
json (for 5.1): fJSON
bcmath: fNumber

I've also implemented imap/pop3 in native PHP to get around performance
issues, segfaults and similar issues. I've backported HTTP-only cookies to
5.1. There are also a whole bunch of other compatibility/portability
elements.

The questions I have are:

1. Is this even a viable approach? It seems like it, but to my knowledge
   no one else has done this to any serious extent which makes me wonder if
   there's a reason the road less traveled is less traveled.

It has seem to work fine for Flourish, although I don't have anywhere near
the traction/usage of any of the big frameworks or component libraries. I
think part of the reason this road is less traveled is because most
developers who care about more advanced functionality are more than capable
of finding an environment that has the extensions they need.

2. Is anyone else doing this? No sense doing it ourselves if someone else
   already is.

As I mentioned, I do this is Flourish, however I do wrap the functionality
up in my own API since I am kind of particular about the usability of APIs.
Sometimes wrapping it in my own API provides the ability to skip 30% of the
functionality I don't see being frequently used, other times it allows me to
fix bugs in specific versions of PHP, such as with stream_select() in some
versions of PHP 5.2.

3. What would be the cleanest way to do so? We had the thought of
   partially automating the process by having PHP auto-generate at the very
   least the subs of any classes and functions that the module provides.
   However, when my colleague tried using the same parser as is used for
   generating documentation he says he got several times as many classes as the
   manual says the module has. We were using the PECL HTTP module as our
   sample (http://www.php.net/http). (I don't know the exact details of what
   he did at the moment.) Is that not a viable approach? Would we be better
   off using reflection? Is there some other tool we're not aware of?

Like I said, I create my own APIs. I normally will have used the C
extensions some and have some ideas about how I think they can be improved
and how I normally see them used. This then informs me about what I think
the custom API should be. Sometimes I do a pretty straight-forward
implementation though. For instance, I've implemented all of the PHP string
functions to work against UTF-8, even things like ucwords().

If viable I'd love if this would start a trend, but we'll see where it
goes. I know it wouldn't work for all PECL modules, obviously, but I
suspect it could work for several, and provide an easy way for different PHP
projects to share backend code without needing lots of C developers.

I agree that not having to rely on C code is helpful, but in my opinion it
isn't because developing in C is hard/annoying/there are many C developers,
but because it is much harder to get special C extensions installed on many
shared hosts. Even for people who do have full access to install extensions,
it is often a pain to deal with compared to dropping in an updated PHP
script.

So from my experience implementing various semi-CPU-intensive work, the key
aspects to making native PHP work is trying to rely on built-in functions
that do as much as possible and to be sure to think about algorithms being
used. For instance, I found I was able to implement the porter stemming
algorithm with much better performance by using preg* functions for string
manipulation. For the diff/patch library I wrote, I found using the patience
diff algorithm it was possible to write a diffing library that wouldn't
consume 128MB of ram for 3000 line files, and it was an order of magnitude
faster to boot.

Will