Reference handling change and PHP 4.4.0

 You do understand the difference between 4.3.x and 4.4.x? :)
 (yes, this issue has been beaten to death already)

 --Jani

Hello all,

I know this issue has most likely been discussed to death here so I apologise
in advance for starting a new thread about it. I just need to get my head
around the reasoning for introducing this change to PHP4 (4.4 branch). I can
understand making the change to PHP5, but can someone bring me up to speed as
to why it was done to PHP4? As I'm sure you're aware, it breaks heaps and
heaps of existing code out there.

We found this out the hard way yesterday when our server administrator
upgraded our production server to PHP 4.4.0 (even though the Debian package
description said it was a PHP 4.3.x release). We ended up with hundreds of
errors and many, many vhosts stopped working correctly due to "variable
references should be returned by reference" problems, both in my code and in
third-party code, such as PHPBB instances.

And now I read that this will not be "fixed" in future PHP4 releases due to
breaking backwards compatibility? I just don't understand... a change that
breaks backwards compatibility in a large portion of the existing PHP4 code
base will not be fixed because it breaks backwards compatibility? It's doing
my head in, please can someone explain?

This means that either we have to (potentially) change thousands of lines of
code and upgrade many instances of third-party PHP-based sites just to use PHP
4.4.0 and above, or miss out on using this and future PHP4 releases and any
security vulnerability fixes they may include. Why could the change not be
made optional by using a php.ini directive?

Kind regards,

Colin.

 1) I'm not a bot.
 2) I'm not bogusing automatically anything (except for few exceptions [1])
 3) I spend a LOT of my free time, without getting paid
    testing hundreds of wierd code pieces people send,
    configuring and compiling PHP with wierd configure lines,
    etc.

 Never forget: MOST of us (me included) do NOT get paid for this.
 And handling bug reports is propably the most ungrateful "jobs"
 there is.

 I can count the times I've been thanked personally with one hand.
 Most of the time people send me hate email. :)

 It's the community who should support the developers, not the other
 way around. We owe the community nothing, the community owes us everything.

 Don't lie. You aimed quite well. :)

 --Jani

 [1] People seem to think their problem is the most important one
 and even if it's been reported before, still send it as new one..
 in which case their report gets very quickly bogused. :)
 There are several other reasons why to do that, but for _valid_
 bug reports this happens VERY rarely. (Try piss me off and you'll see)

 You really think I have _time_ to spend reading some rants?
 That'll all be out of handling the bogus reports. :)

 Why not? a) the guy uses configure options he doesn't know what they
 really do b) I couldn't reproduce it with proper configure line.
 Hence he's doing something stupid.

 You don't obviously understand how much time it takes to go
 through the reports. Try simulating it by taking the 10 first
 open reports and try reproduce all of them with the given
 information in the initial comment..

 Yes, should I have lied I don't do that?

 That's what the s.c. quickfixs are for. Our "how-to-report" etc. pages
 specifically ASK people to FIRST check if the bug has already been
 reported. If they ignore our instructions how to reports, we don't have
 the right to ignore their (duplicate) report? Especially in cases where
 there is an OPEN one. And even more in case there is already 10 bogused
 ones where the reason for bogusing has been explained by 2-3 different
 PHP developers?!

 Most of the reports are user errors and plain lazyness in reading the
 friendly manual. Not about making the "product" better. (since when is
 an opensource project a product? :)

 I guess this is coming back to the fact that nobody notices the positive
 things, only the negative. And think that "bogus" as status always
 means negative thing. Instead of 2nd hand rants, try read the bug
 mailing list for a week. That should give you better insigh what goes
 on on daily basis rather than these separate and very few cases where
 even more arrogant people than me think their civil rights have been
 violated and that they actually have the right to decide what is a bug
 and what's not..

 I take it more as fan mail. I have a special folder for it too. (/dev/null :)

 --Jani

 Show me the report where I have called someone stupid. Then I can explain
 you why I did so. :)

 10 seconds? Sheesh. If english was my native language, maybe I could do it
 that fast but as it isn't, and as I have to first think the phrase in finnish
 then traslate it in my head into english and then write it and then fix the typos,
 it takes at least 15 seconds! :)

 Job? I haven't earned any money directly from dealing with the reports in the
 5 years I have been "working" with PHP so I wouldn't call it a job yet. :)

 --Jani

return $instance;

return NULL;

if ($this->hasIndex($index))
{
   $element =& $this->elements[$index];
}

return $element;

Colin Tucker wrote:

I know this issue has most likely been discussed to death here so I
apologise in advance for starting a new thread about it. I just need to
get my head around the reasoning for introducing this change to PHP4
(4.4 branch). I can understand making the change to PHP5, but can
someone bring me up to speed as to why it was done to PHP4? As I'm sure
you're aware, it breaks heaps and heaps of existing code out there.

We found this out the hard way yesterday when our server administrator
upgraded our production server to PHP 4.4.0 (even though the Debian
package description said it was a PHP 4.3.x release). We ended up with
hundreds of errors and many, many vhosts stopped working correctly due
to "variable references should be returned by reference" problems, both
in my code and in third-party code, such as PHPBB instances.

And now I read that this will not be "fixed" in future PHP4 releases due
to breaking backwards compatibility? I just don't understand... a
change that breaks backwards compatibility in a large portion of the
existing PHP4 code base will not be fixed because it breaks backwards
compatibility? It's doing my head in, please can someone explain?

This means that either we have to (potentially) change thousands of
lines of code and upgrade many instances of third-party PHP-based sites
just to use PHP 4.4.0 and above, or miss out on using this and future
PHP4 releases and any security vulnerability fixes they may include. Why
could the change not be made optional by using a php.ini directive?

We have been looking to see if there is a way to fix the memory
corruption issue in a way that has less of an impact on existing code.
This doesn't change the fact that every error you get is actually an
error in the code you are running. The broken code is effectively doing
something similar to:

function foo(&$arg) {
$arg = 4;
}
foo(strlen("abc"));

which makes very little sense. There are many variations of this, but
in every instance it is something that should be fixed, and in some of
the cases it actually leads to memory corruption.

There are many examples of this sort of thing working just fine in PHP
4.3.x because people are passing things in by reference and never
modifying the passed in value. They'll do something along the lines of:

function foo(&$arg) {
return $arg[0] + 1;
}
echo foo(array(1,2,3,4,5,6,7,8,9,10));

possibly because they believe that they are saving memory by not making
a copy of the array with that reference. Of course that is quite
misguided as no copy is done until the arg is modified so if you don't
modify it, you are better off passing it by value.

Right now the change in 4.4 and 5.x is to complain loudly when you pass
a reference to something without any associated storage. In both of the
above examples there is no permanent storage associated with either of
the passed arguments, so trying to get a reference to this storage makes
very little sense. The only feasible way we might be able to work
around this is to make a copy of such bogus reference attempts and
effectively just pretend they were passed by value. I think it is a bit
of a hack, but at the same time the breakage of existing apps has been
more widespread than I think anybody anticipated. I'd still want to
throw a notice to let people know they are doing something odd though.

-Rasmus

Hello all,

I know this issue has most likely been discussed to death here so I apologise
in advance for starting a new thread about it. I just need to get my head
around the reasoning for introducing this change to PHP4 (4.4 branch). I can
understand making the change to PHP5, but can someone bring me up to speed as
to why it was done to PHP4? As I'm sure you're aware, it breaks heaps and
heaps of existing code out there.

It already broke heaps of code - and this problems was affecting
application with hard to find crashes. So yes, it is worth it.

We found this out the hard way yesterday when our server administrator
upgraded our production server to PHP 4.4.0 (even though the Debian package
description said it was a PHP 4.3.x release). We ended up with hundreds of
errors and many, many vhosts stopped working correctly due to "variable
references should be returned by reference" problems, both in my code and in
third-party code, such as PHPBB instances.

Uhm, they're only notices - they shouldn't show up at the website in
production, now should they?

And now I read that this will not be "fixed" in future PHP4 releases due to
breaking backwards compatibility? I just don't understand... a change that
breaks backwards compatibility in a large portion of the existing PHP4 code
base will not be fixed because it breaks backwards compatibility? It's doing
my head in, please can someone explain?

PHP 4 is not going to change the behavior anymore.

Derick

--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

Hi again from an "ordinary user"!

This thread has some opponents sharpening their knives in the corners.
Hey guys, cool down! It's only ones and zeroes! ;-)

First of all let's find some agreements points:

• I think most of you agree that it was a bigger problem
  fixing it with warning in PHP4 than it was in PHP5.
• Not fixing it was out of question because it caused
  memory corruption.

Some time ago there were default options in the php.ini that are
considered very harmful nowadays (register_globals etc) but they are
still there, just carefully defaulting to off. See the similarities? If
some people just want to use it this way the put in an option:

unstored_references = NOTICE|IMPLICIT_TEMP_STORE

So people relying on using it the old way can just switch it to
automagically assigning a temp variable inbetween (or is this standard
procedure since 4.4? Then just allow to switch the log level for such
events.).

On a side note it would help people if there was a possibility to run
all kinds of PHP versons as Apache module in parallel which nowadays has
to be done by patching the sources. Some PHP coders don't like the CGI
variant and prefer the module and just for the sake of using the module
they resign to doing all stuff with a single PHP version. Being able to
run a 4.3.10 module in parallel with a 4.4 module and choosing the one
to use on a per-directory-basis via .htaccess could calm a lot of voices.

AllOLLi

---

Time is an illusion. Lunchtime doubtly so.
[Douglas Adams]

Hello again,

Wow, this thread generated more heat than I thought it would, I
basically just wanted to know the reasoning behind making the change to
PHP4, unfortunate I guess for all involved that it went downhill. I
have learned much in the past few days, at any rate.

As mentioned, my code actually broke following the switch to 4.4.0, in
addition to logging hundreds of E_NOTICE errors. It's taken me some
time to track down where it's going wrong, but I think I've found out
the culprits involved. It's complicated, I'll try to explain it all.

The problem was related to components of my web app framework (which is
similar to Struts, and incidently, now being phased out for a MUCH more
lightweight, PHP5-based framework), in particular, an array of
attributes, view helper classes and the creation of DAOs (data access
objects) at runtime via XML configuration, which would look like this:

<daos> <dao name="dao.freight" class="FreightDAO" component="Generic/Commerce/DAO/FreightDAO.php" /> <dao name="dao.customer" class="CustomerDAO" component="Generic/Commerce/DAO/CustomerDAO.php" /> <dao name="dao.order.status" class="OrderStatusDAO" component="Generic/Commerce/DAO/OrderStatusDAO.php" /> </daos>

This XML would be parsed and the relevant DAO instances would be created
and recorded in attribute scope for the current application module
instance. For each dao node, the following code would be called (where
$daoClass and $daoComponent are extracted from the XML attributes):

$dao =& $this->_createDAO($daoClass,$daoComponent);

if ($dao)
{
$this->module->setAttributeByReference($name,$dao);
}

Notice it assigns by reference, however I discovered that the
_createDAO() method was not returning by reference:

function _createDAO($daoClass=null,$daoComponent=null)
{

// Obtain Database Object:

$database =& $this->module->findDatabase();

if (is_null($database))
{
   trigger_error($this->error(ERROR_DB_COULD_NOT_OBTAIN));
}
else
{

// Obtain Database Connection:

   $connection =& $database->getConnection();

   if (is_null($connection))
   {
      trigger_error($this->error(ERROR_DB_COULD_NOT_CONNECT));
   }
   else
   {

   // Create DAO Instance:

      $cmp =& new ClassComponent($daoClass,$daoComponent);

      $dao =& $cmp->getInstance();

      if ($dao)
      {
         $dao->setConnectionByReference($connection);

         return $dao;
      }
   }
}

return NULL;

}

What was happening under PHP 4.4.0 is when the newly created DAO was
added to the module as an attribute (setAttributeByReference), a new
attribute would be created, but ALL existing DAOs in the attribute scope
would become the newly added DAO, for example:

(create and set the dao.freight attribute):

attributes (array)
{
[dao.freight] => FreightDAO Object
}

(create and set the dao.customer attribute):

attributes (array)
{
[dao.freight] => CustomerDAO Object
[dao.customer] => CustomerDAO Object
}

(create and set the dao.order.status attribute):

attributes (array)
{
[dao.freight] => OrderStatusDAO Object
[dao.customer] => OrderStatusDAO Object
[dao.order.status] => OrderStatusDAO Object
}

Simply by adding the ampersand to the _createDAO() method, like so:

function &_createDAO($daoClass=null,$daoComponent=null)
{
}

...rectified the problem (however did not make hundreds of E_NOTICE's
disappear, of course ;)).

The question is, why did this problem not occur before 4.4.0? Is this
an example of the "memory corruption"?

Regards,

Colin.

Hi

I was lost in this thread. Could someone post resume from this
discussion? How can I deal with this in 4.3, 4.4, 5+?

Thanks

--
Ondrej Ivanič
(ondrej@kmit.sk)