[VOTE] Weak References

Hello,

 I've talked to Etienne and he still believes we should vote on this so the
vote is still up. He just replied to quickly. If the vote fails the patch
can still be implemented in a PECL extension. There's no point canceling the
vote or the RFC - it's still valid and ongoing, and there is still time for
discussion.

.. snip ...

I think at this point everyone is well aware that you would like this
feature implemented.

I would like to make a couple points here, first I is that PECL is not
a place for a extension to rot, that statement makes absolutely no
sense to me. PDO I recall years ago getting from PECL, Http a
extension I often use, was recently on this list if I recall to be
moved into the core. I'm sure there are plenty of people on this list
that were part of extension's PECL -> CORE lifecycles that could add
several more examples to the list. I believe PECL Would be a great
place for something like this that was is entirely understood or has
overwhelming support in my opinion. I am really not sure how
comfortable I am with a feature as new as this being in the PHP on my
servers. Some form of incubation, testing and use feedback only makes
sense to me.

Second I would like to talk a little bit about weak references for
those who are not familiar with them or have thus far only read your
very strong pro-opinion. I would like to make it very clear with the
simple fact that weak references are not required, ever by any
language. They are a convenience, not a solution to a unsolvable
problem. I believe that if anything weak references encourage laziness
in design, it causes the referential integrity that I am accustomed to
is broken, and can often lead to more code and checks then it
relieves. It is very easy to run into unexpected bugs and in a
language like PHP, where you don't get a NPE when you try to access a
NULL method of a non object, you get a fatal error, this doesn't seem
very appealing to me. Only under very specific cases of caching
patterns as you have previously mentioned is such a pattern even
recognized to -have a place for use that isn't just plain wrong- -not
a requirement- by me.

In PHP, I believe a web or cli app that uses enough resources to bomb
memory limits on a server, needs to be re-factored. I have definitely
written multi-process PHP daemons that go to disk or over the network
for taxing operations that need cached and had very little trouble
with rudimentary memory management patterns, so from my perspective I
can't really see why given the toolset PHP provides that this is such
a problem.

This is just my opinion, I don't have a strong opposition to this
feature, anything "new" to PHP I usually get excited about, I simply
wanted to give a few counter arguments as the fact that this feature
is being presented as a dire and long over due missing feature to the
language is a bit dramatic. Not saying its useless and doesn't belong
in PHP or I would never want it to exists, I just would hate to see
something rushed through that didn't get a bit of testing and typical
QA from a language that affects millions of people.

-Chris

Adding a new feature to the latest version of PHP will not make it
available to users of shared hosting. Many of these are running on older
versions and it takes time for them to get updated to the latest.

• Frank

I've talked to Etienne and he still believes we should vote on this so the
vote is still up. He just replied to quickly. If the vote fails the patch
can still be implemented in a PECL extension. There's no point canceling the
vote or the RFC - it's still valid and ongoing, and there is still time for
discussion.

The problem is that PECL is not an option for all users. If you're using
shared hosting and otherwise don't have control over what extensions is
loaded or not then essentially you'll be unable to use a framework that
takes advantage of weak references. That's the reality of many freelancing
developers and small-scale sites out there. It's especially problematic for
framework designers that builds ORM-layers - they can't tell their users "oh
btw you need to install this non standard PECL extension for it to work". Or
rather, you can but then the user will be angry.

I don't think "because they can" is a good argument for why features should
generally be outsourced as PECL extensions. Complex, experimental stuff that
solves specific integration or performance problems is great for PECL. But
generic object orientation design fundamentals should be implemented in the
core since people rather have their framework leak a bit more memory (and
risk running out of it in edge cases) or design their application more
clumsy than depend on a non-standard PECL extension. It's about allowing
people to do elegant design. A good developer would choose compatibility
over elegant design any day - so if the language wants people to do design
great applications the language needs to provide those tools natively.

I realize I need to provide some kind of realistic scenario for you to
understand so let's put ourselves in the shoes of Joe Codeguy - he's
currently designing the an application that will edit inventory details at a
small company with 10 employees. It is hosted at a shared hosting provider
since they are running a tight budget. They can't afford their own servers
or an IT guy. The employers regularly login to the application and updates
the inventory. Joe Codeguy has designed the application so that there are
one Product class that is loaded from the database where each "products"
table row corresponds to a product. Certain function in the application runs
a MySQL procedure that updates a certain complex selection of products in
the products database rendering the currently loaded products in memory
obsolete since they might have been updated. After this procedure runs all
Product class objects in memory will need to be reloaded. Joe Codeguy fixes
this by implementing the observer pattern (PEAR Event_Dispatcher) so that
whenever a Product is loaded or created it registers itself on a global
event called "EVENT_PRODUCT_RELOAD". He then proceeds to trigger the event
whenever the complex MySQL procedure updates the database table. Everything
seems to be working fine so he deploys the application. One days later the
customer complains that a certain critical operation no longer works. Joe
then find a subroutine while debugging that loops trough a quite large
selection of thousands of products one by one and makes some complicated
calculations. During this loop the application faces a out of memory
condition. Joe thinks this is strange because it only operates on one
product at a time. He then realizes that the observer registration prevents
the products from being garbage collected properly since they are strongly
referenced by Event_Dispatcher. Thankfully PHP implements weak references so
he can correct Event_Dispatcher to use weak referencing instead. However
it's outsourced into a PECL extension and since Joe cannot install
extensions into the production server he is forced to use unset hacks in the
faulty loops to correct the issue. Joe deploys the application again and can
only hope that this was the only loop that triggered the memory problem and
that the application doesn't crash anywhere else when the products table
grows larger.

Also even if an application doesn't run out of memory there might be a
serious performance impact from the memory leaked by objects that can't be
collected properly. If an application peaks at 15 MB instead of 25 MB, the
server can run 40% more child threads in parallel.

Regards,

~Hannes

The fast development root does seem to have been a little screwed up
recently, but if there is nothing in an extension that requires direct
access to the core and that extension CAN simply be enabled as required,
then pecl is the way to go? We do need the pecl area to be properly included
in the build process with windows builds being created just as they used to
be, but the process is working fine for APC and perhaps the plan should be
to moving some more extensions that are being worked on between PHP releases
into the pecl process? It used to be that many of the extensions I used were
managed in pecl ... and I'd be quite happy if they moved back there. There
is nothing stopping 'Weak References' being added by those who want them ...
but I can't see the point myself when running scripts live.

--
Lester Caine - G8HFL

Contact - http://lsces.co.uk/wiki/?page=**contacthttp://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/**index.php<http://www.firebirdsql.org/index.php

Hannes Landeholm wrote:

Certain function in the application runs a MySQL procedure that updates a
certain complex selection of products in the products database rendering the
currently loaded products in memory obsolete since they might have been
updated.

Then you are caching the wrong data in memory?
Following an update like that one would either have to commit the changes or
rollback. So the data has to be reloaded from the server anyway. You can't rely
on any of the data already loaded.

Joe deploys the application again and can only hope that this was the
only loop that triggered the memory problem and that the application
doesn't crash anywhere else when the products table grows larger.

So you are trying to load lots of unnecessary data from the database through
poor design. Redesign the data so it works properly ... or switch to a database
that handles business logic via triggers properly ;) You should only be
downloading the data you need to look at, leave the volume data on the server.

The majority of my business logic is done in the database (Firebird) and I
simply can't see how I would ever fall into the trap you seem to be creating?
Perhaps I am doing something wrong, but my systems all run multiuser real time
and handle large numbers of records without these sorts of problem.

--
Lester Caine - G8HFL

Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/index.php

Hi!

I'm referring to the fact that PHP has a major update once a year.
Looking at it from an agile perspective this is insanely slow. I think
you should use smaller iterations and release faster. The new Mozilla
release schedule is a good example.

Having major changes more frequent would mean people that use PHP would
be forced to constantly update their systems to keep up - or would not
use most of the new features. It would also mean most of new releases
will be unstable for most of their lifecycle - as once they'd stabilize
we'd have new version on the way. And we'd need massive quantity of
resources to test and fix those.

Just for perspective - we have a lot of PHP 4 code still out there and
many still run 5.1. Most projects still can't afford not to have 5.2
support - even though this version is officially dead. Now let's suppose
we are "agile" and have 3 versions in a year. That means developers
would have to test on 6 or 9 versions of PHP instead of 2 or 3 now. Or
people would just ignore all new stuff - and what's the point then?

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

I have no problem compiling extensions and I also don't have any
customers on share hosting. Although it wastes a bit more of my time
I'm personally fine with PECL. I'm contributing to open source though
so I'm concerned with the users that won't be able to run my code
because they can't install PECL extensions. I'm also worried that it
will takes years for WR to be moved to the core and that nobody will
notice or use it. PHP deserves weak references. Not in a year but
today.

Honestly, pecl is kind of self-selecting. APC is installed on millions
of servers and it lives in pecl. It is installed everywhere because it
is an extremely useful extension and distros tend to package it up
promptly for that reason as well. If weak references is a highly
desirable feature that will make every complex framework use less
memory, then I don't see why it wouldn't follow APC's path and end up on
millions of servers.

Note also that there are distros that unbundle core extensions and move
them into separate packages which makes them indistinguishable from pecl
extensions as far as the user is concerned. Realistically, very few end
users are installing from the php.net sources. Just about everyone
installs their distro's version of PHP with whatever extensions the
distro maintainer has decided to include by default.

The major frameworks/CMS systems are also packaged up and carry along
dependencies which sometimes includes extension packages. So again, the
line between core and pecl is very fuzzy here since you could easily
have your framework have a dependency on a pecl extension.

And the final point is that large shared hosting farms is kind of old
fashioned these days. Modern hosting is done with dedicated machine
slices, but there are of course a lot of these legacy shared hosting
farms around. But, given these are legacy systems they are not likely to
rush to upgrade to 5.4 and by the time 5.4 becomes prevalent how many
users are really still going to be on a shared web server config?

I think the only valid point for putting this in core is that it does
feel more like a core feature than a pecl add-on. However, a number of
us are uncomfortable with the impact of it and would like to understand
better how people other than yourself would use it, or more likely,
mis-use it.

-Rasmus