restore user opcode handler in PHP

13 years ago by yoram bar haim — view source

unread

I first saw this problem while addapting extension to PHP 5.4 but the code is
the same in 5.3 :
when you call zend_set_user_opcode_handler() it set's a pointer (handler) in
zend_user_opcode_handlers and a uint opcode value in zend_user_opcodes to
ZEND_USER_OPCODE .

you can call zend_set_user_opcode_handler with the original handler to restore
it, but if the original state did not include ZEND_USER_OPCODE you have no way
to restore it.

Why restoring ?
On mac, static variables are not re-initialized on dlclose()+dlopen(). that
means that apache reload does not re-create zend_user_opcode_handlers and
zend_user_opcodes.
if your extension sets user_opcode_handler and is not reloaded in apache
reload (because you commented it in php.ini, for example), your handlers array
will actually point to non-exist handler.
if you will call zend_set_user_opcode_handler() with the original handler at
MSHUTDOWN, it will restore the handler but zend_user_opcodes[<opcode] will
still be ZEND_USER_OPCODE so you will end up calling a NULL handler.

we should eather allow zend_set_user_opcode_handler to also set the value in
zend_user_opcodes[<opcode>] or we should build those arrayes from scratch at
php_module_init.

currently an extension tha calls zend_set_user_opcode_handler() will make PHP
crash on mac after disabling the extension and apache reload.

13 years ago by Laruence — view source

unread

Hi:
could you file a feature req on bugs.php.net?

I think it's okey to add a zend_restore_user_handler .

like:

ZEND_API int zend_restore_user_opcode_handler(zend_uchar opcode) {
zend_user_opcodes[opcode] = opcode;
zend_user_opcode_handlers[opcode] = NULL;
return SUCCESS;
}

but it still need dmitry to review :)

thanks

I first saw this problem while addapting extension to PHP 5.4 but the code is
the same in 5.3 :
when you call zend_set_user_opcode_handler() it set's a pointer (handler) in
zend_user_opcode_handlers and a uint opcode value in zend_user_opcodes to
ZEND_USER_OPCODE .

you can call zend_set_user_opcode_handler with the original handler to restore
it, but if the original state did not include ZEND_USER_OPCODE you have no way
to restore it.

Why restoring ?
On mac, static variables are not re-initialized on dlclose()+dlopen(). that
means that apache reload does not re-create zend_user_opcode_handlers and
zend_user_opcodes.
if your extension sets user_opcode_handler and is not reloaded in apache
reload (because you commented it in php.ini, for example), your handlers array
will actually point to non-exist handler.
if you will call zend_set_user_opcode_handler() with the original handler at
MSHUTDOWN, it will restore the handler but zend_user_opcodes[<opcode] will
still be ZEND_USER_OPCODE so you will end up calling a NULL handler.

we should eather allow zend_set_user_opcode_handler to also set the value in
zend_user_opcodes[<opcode>] or we should build those arrayes from scratch at
php_module_init.

currently an extension tha calls zend_set_user_opcode_handler() will make PHP
crash on mac after disabling the extension and apache reload.

--

--
Laruence Xinchen Hui
http://www.laruence.com/

13 years ago by yoram bar haim — view source

unread

I think the following patch is a better solution, currently trying it :
--- Zend/zend_execute.c.orig 2012-02-12 14:32:07.000000000 +0200
+++ Zend/zend_execute.c 2012-02-12 14:31:33.000000000 +0200
@@ -1512,7 +1512,11 @@
ZEND_API int zend_set_user_opcode_handler(zend_uchar opcode,
user_opcode_handler_t handler)
{
if (opcode != ZEND_USER_OPCODE) {

          zend_user_opcodes[opcode] = ZEND_USER_OPCODE;

```
          if (handler == NULL) { 
```

                  zend_user_opcodes[opcode] = opcode;

```
          } else {
```

                  zend_user_opcodes[opcode] = ZEND_USER_OPCODE;

          }
          zend_user_opcode_handlers[opcode] = handler;
          return SUCCESS;
  }

Hi:
could you file a feature req on bugs.php.net?

I think it's okey to add a zend_restore_user_handler .

like:

ZEND_API int zend_restore_user_opcode_handler(zend_uchar opcode) {
zend_user_opcodes[opcode] = opcode;
zend_user_opcode_handlers[opcode] = NULL;
return SUCCESS;
}

but it still need dmitry to review :)

thanks

I first saw this problem while addapting extension to PHP 5.4 but the
code is the same in 5.3 :
when you call zend_set_user_opcode_handler() it set's a pointer (handler)
in zend_user_opcode_handlers and a uint opcode value in
zend_user_opcodes to ZEND_USER_OPCODE .

you can call zend_set_user_opcode_handler with the original handler to
restore it, but if the original state did not include ZEND_USER_OPCODE
you have no way to restore it.

Why restoring ?
On mac, static variables are not re-initialized on dlclose()+dlopen().
that means that apache reload does not re-create
zend_user_opcode_handlers and zend_user_opcodes.
if your extension sets user_opcode_handler and is not reloaded in apache
reload (because you commented it in php.ini, for example), your handlers
array will actually point to non-exist handler.
if you will call zend_set_user_opcode_handler() with the original handler
at MSHUTDOWN, it will restore the handler but zend_user_opcodes[<opcode]
will still be ZEND_USER_OPCODE so you will end up calling a NULL
handler.

we should eather allow zend_set_user_opcode_handler to also set the value
in zend_user_opcodes[<opcode>] or we should build those arrayes from
scratch at php_module_init.

currently an extension tha calls zend_set_user_opcode_handler() will make
PHP crash on mac after disabling the extension and apache reload.

13 years ago by Dmitry Stogov — view source

unread

In case this patch works, I think it's fine to include it.

Thanks. Dmitry.

I think the following patch is a better solution, currently trying it :
--- Zend/zend_execute.c.orig 2012-02-12 14:32:07.000000000 +0200
+++ Zend/zend_execute.c 2012-02-12 14:31:33.000000000 +0200
@@ -1512,7 +1512,11 @@
ZEND_API int zend_set_user_opcode_handler(zend_uchar opcode,
user_opcode_handler_t handler)
{
if (opcode != ZEND_USER_OPCODE) {
          zend_user_opcodes[opcode] = ZEND_USER_OPCODE;
          if (handler == NULL) {
                  zend_user_opcodes[opcode] = opcode;
          } else {
                  zend_user_opcodes[opcode] = ZEND_USER_OPCODE;
          }
           zend_user_opcode_handlers[opcode] = handler;
           return SUCCESS;
   }
Hi:
could you file a feature req on bugs.php.net?

I think it's okey to add a zend_restore_user_handler .

like:

ZEND_API int zend_restore_user_opcode_handler(zend_uchar opcode) {
zend_user_opcodes[opcode] = opcode;
zend_user_opcode_handlers[opcode] = NULL;
return SUCCESS;
}

but it still need dmitry to review :)

thanks

I first saw this problem while addapting extension to PHP 5.4 but the
code is the same in 5.3 :
when you call zend_set_user_opcode_handler() it set's a pointer (handler)
in zend_user_opcode_handlers and a uint opcode value in
zend_user_opcodes to ZEND_USER_OPCODE .

you can call zend_set_user_opcode_handler with the original handler to
restore it, but if the original state did not include ZEND_USER_OPCODE
you have no way to restore it.

Why restoring ?
On mac, static variables are not re-initialized on dlclose()+dlopen().
that means that apache reload does not re-create
zend_user_opcode_handlers and zend_user_opcodes.
if your extension sets user_opcode_handler and is not reloaded in apache
reload (because you commented it in php.ini, for example), your handlers
array will actually point to non-exist handler.
if you will call zend_set_user_opcode_handler() with the original handler
at MSHUTDOWN, it will restore the handler but zend_user_opcodes[<opcode]
will still be ZEND_USER_OPCODE so you will end up calling a NULL
handler.

we should eather allow zend_set_user_opcode_handler to also set the value
in zend_user_opcodes[<opcode>] or we should build those arrayes from
scratch at php_module_init.

currently an extension tha calls zend_set_user_opcode_handler() will make
PHP crash on mac after disabling the extension and apache reload.

13 years ago by Dmitry Stogov — view source

unread

Hi Yoram,

Can we just fill this arrays with zeros on php startup?

Thanks. Dmitry.

I first saw this problem while addapting extension to PHP 5.4 but the code is
the same in 5.3 :
when you call zend_set_user_opcode_handler() it set's a pointer (handler) in
zend_user_opcode_handlers and a uint opcode value in zend_user_opcodes to
ZEND_USER_OPCODE .

you can call zend_set_user_opcode_handler with the original handler to restore
it, but if the original state did not include ZEND_USER_OPCODE you have no way
to restore it.

Why restoring ?
On mac, static variables are not re-initialized on dlclose()+dlopen(). that
means that apache reload does not re-create zend_user_opcode_handlers and
zend_user_opcodes.
if your extension sets user_opcode_handler and is not reloaded in apache
reload (because you commented it in php.ini, for example), your handlers array
will actually point to non-exist handler.
if you will call zend_set_user_opcode_handler() with the original handler at
MSHUTDOWN, it will restore the handler but zend_user_opcodes[<opcode] will
still be ZEND_USER_OPCODE so you will end up calling a NULL handler.

we should eather allow zend_set_user_opcode_handler to also set the value in
zend_user_opcodes[<opcode>] or we should build those arrayes from scratch at
php_module_init.

currently an extension tha calls zend_set_user_opcode_handler() will make PHP
crash on mac after disabling the extension and apache reload.

13 years ago by yoram bar haim — view source

unread

Hi Dmitry.
the patch does work, in sense of actually restoring the handlers state when
you call zend_set_user_opcode_handler() with original handler.
the patch does not solve the basic problem of no re-initialization of static
arrayes on mac.

As for feeling the array with zero's, if we do that, we also need to fill it
with original opcodes, something like :

static zend_base_user_opcodes = {0,1,2,3,4,....};

and then in php_module_startup :

memset (zend_user_opcode_handlers,0,sizeof(zend_user_opcode_handlers);
memcpy
(zend_user_opcodes,zend_base_user_opcodes,sizeof(zend_base_user_opcodes);

So basically, every modifiably variable should be initialized as startup and
not only by static initializer.

Hi Yoram,

Can we just fill this arrays with zeros on php startup?

Thanks. Dmitry.

I first saw this problem while addapting extension to PHP 5.4 but the
code is the same in 5.3 :
when you call zend_set_user_opcode_handler() it set's a pointer (handler)
in zend_user_opcode_handlers and a uint opcode value in
zend_user_opcodes to ZEND_USER_OPCODE .

you can call zend_set_user_opcode_handler with the original handler to
restore it, but if the original state did not include ZEND_USER_OPCODE
you have no way to restore it.

Why restoring ?
On mac, static variables are not re-initialized on dlclose()+dlopen().
that means that apache reload does not re-create
zend_user_opcode_handlers and zend_user_opcodes.
if your extension sets user_opcode_handler and is not reloaded in apache
reload (because you commented it in php.ini, for example), your handlers
array will actually point to non-exist handler.
if you will call zend_set_user_opcode_handler() with the original handler
at MSHUTDOWN, it will restore the handler but zend_user_opcodes[<opcode]
will still be ZEND_USER_OPCODE so you will end up calling a NULL
handler.

we should eather allow zend_set_user_opcode_handler to also set the value
in zend_user_opcodes[<opcode>] or we should build those arrayes from
scratch at php_module_init.

currently an extension tha calls zend_set_user_opcode_handler() will make
PHP crash on mac after disabling the extension and apache reload.

13 years ago by Stas Malyshev — view source

unread

Hi!

Hi Dmitry.
the patch does work, in sense of actually restoring the handlers state when
you call zend_set_user_opcode_handler() with original handler.
the patch does not solve the basic problem of no re-initialization of static
arrayes on mac.

Could you please describe in more details what's going on on mac? It
seems strange that dlopen() would not initialize vars - that'd be pretty
bad behavior. Do you have an easy way to reproduce it?

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

13 years ago by Dmitry Stogov — view source

unread

I'm not familiar with Mac, but I saw such behavior on Linux very long
time ago, when on PHP restart static variables were still keep values
from old process.

Thanks. Dmitry.

Hi!

Hi Dmitry.
the patch does work, in sense of actually restoring the handlers state
when
you call zend_set_user_opcode_handler() with original handler.
the patch does not solve the basic problem of no re-initialization of
static
arrayes on mac.

Could you please describe in more details what's going on on mac? It
seems strange that dlopen() would not initialize vars - that'd be pretty
bad behavior. Do you have an easy way to reproduce it?

13 years ago by Stas Malyshev — view source

unread

Hi!

I'm not familiar with Mac, but I saw such behavior on Linux very long
time ago, when on PHP restart static variables were still keep values
from old process.

That might happen if the shared library was not unloaded (dlopen will
not re-load library that is already loaded) however I don't understand
how it can happen on Apache - it's supposed to dlclose all modules. Is
it reproducible on any Apache version on Linux or Mac?

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

13 years ago by yoram bar haim — view source

unread

Here is a simple test program that reproduce the issue on mac:

lib.h:
#include "lib.h"
void set_index(int index, int value);
int get_index(int index);

lib.c:
#include "lib.h"

static int glob_var[] = {0,0,0,0};

void set_index(int index, int value) {
glob_var[index] = value;
}

int get_index(int index) {
return glob_var[index];
}

main.c:
#include <stdio.h>
#include <dlfcn.h>
#include "lib.h"

int main() {
void *handle = dlopen ("lib.so",0);
set_index(0,1);
dlclose(handle);
handle = dlopen ("lib.so",0);
printf("%d\n",get_index(0));
return 0;
}

shell> gcc -o lib.so -shared lib.c
shell> gcc -o main main.c lib.so
shell> ./main

Hi!

I'm not familiar with Mac, but I saw such behavior on Linux very long
time ago, when on PHP restart static variables were still keep values
from old process.

That might happen if the shared library was not unloaded (dlopen will
not re-load library that is already loaded) however I don't understand
how it can happen on Apache - it's supposed to dlclose all modules. Is
it reproducible on any Apache version on Linux or Mac?

13 years ago by Gwynne Raskind — view source

unread

Here is a simple test program that reproduce the issue on mac:
[snip]
shell> gcc -o lib.so -shared lib.c
shell> gcc -o main main.c lib.so
shell> ./main

This example is flawed.

The manpage for dlclose(3) on Darwin explicitly states that a library
linked by the executable will never be unloaded. You link directly to
the library here. Also, you use prototypes of the library's functions
rather than calling dlsym(3), forcing you to link to the library,
defeating the entire point of loading it with dlopen(3) in the first
place. I've included a test which does not show the issue at the
bottom of this message.

(As a side note of interest, OS X has historically always had issues
with unloading images from the address space; for example, even in
Lion, if you unload an image containing Objective-C symbols, you stand
a pretty good chance of crashing. Image unloading simply didn't exist
at all in any useful form before Leopard.)

-- Gwynne Raskind

/* dylib.c - clang dylib.c -o dylib.dylib -dynamiclib */
static int var = 0;
extern int getVar(void);
extern void setVar(int value);
int getVar(void) { return var; }
void setVar(int value) { var = value; }

/* loader.c - clang loader.c -o loader */
#include <stdio.h>
#include <dlfcn.h>
int main(int argc, char **argv)
{
void *libh = dlopen("dylib.dylib", 0);
int (*getfunc)() = dlsym(libh, "getVar");
void (*setfunc)(int) = dlsym(libh, "setVar");

    printf("First load value: %d\n", getfunc());
    setfunc(5);
    printf("First load set value: %d\n", getfunc());
    dlclose(libh);
    libh = dlopen("dylib.dylib", 0);
    getfunc = dlsym(libh, "getVar");
    setfunc = dlsym(libh, "setVar");
    printf("Second load value: %d\n", getfunc());
    return 0;

}

/* output */
$ uname -v ; clang --version ; ./loader
Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012;
root:xnu-1699.24.23~1/RELEASE_X86_64
clang version 3.0 (tags/RELEASE_30/final)
First load value: 0
First load set value: 5
Second load value: 0

13 years ago by yoram bar haim — view source

unread

Indeed, when using dlsym the value is recreated.
now the question is what happens at apache restart (on mac) that causes
incorrect behavior.
a-priory to that question, static initializers can not be trusted on some
situations.

Here is a simple test program that reproduce the issue on mac:
[snip]

shell> gcc -o lib.so -shared lib.c
shell> gcc -o main main.c lib.so
shell> ./main

This example is flawed.

The manpage for dlclose(3) on Darwin explicitly states that a library
linked by the executable will never be unloaded. You link directly to
the library here. Also, you use prototypes of the library's functions
rather than calling dlsym(3), forcing you to link to the library,
defeating the entire point of loading it with dlopen(3) in the first
place. I've included a test which does not show the issue at the
bottom of this message.

(As a side note of interest, OS X has historically always had issues
with unloading images from the address space; for example, even in
Lion, if you unload an image containing Objective-C symbols, you stand
a pretty good chance of crashing. Image unloading simply didn't exist
at all in any useful form before Leopard.)

-- Gwynne Raskind

/* dylib.c - clang dylib.c -o dylib.dylib -dynamiclib */
static int var = 0;
extern int getVar(void);
extern void setVar(int value);
int getVar(void) { return var; }
void setVar(int value) { var = value; }

/* loader.c - clang loader.c -o loader */
#include <stdio.h>
#include <dlfcn.h>
int main(int argc, char **argv)
{
void *libh = dlopen("dylib.dylib", 0);
int (*getfunc)() = dlsym(libh, "getVar");
void (*setfunc)(int) = dlsym(libh, "setVar");
    printf("First load value: %d\n", getfunc());
    setfunc(5);
    printf("First load set value: %d\n", getfunc());
    dlclose(libh);
    libh = dlopen("dylib.dylib", 0);
    getfunc = dlsym(libh, "getVar");
    setfunc = dlsym(libh, "setVar");
    printf("Second load value: %d\n", getfunc());
    return 0;
}

/* output */
$ uname -v ; clang --version ; ./loader
Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012;
root:xnu-1699.24.23~1/RELEASE_X86_64
clang version 3.0 (tags/RELEASE_30/final)
First load value: 0
First load set value: 5
Second load value: 0

13 years ago by Stas Malyshev — view source

unread

Hi!

Indeed, when using dlsym the value is recreated.
now the question is what happens at apache restart (on mac) that causes
incorrect behavior.
a-priory to that question, static initializers can not be trusted on some
situations.

I think we need to figure out why. This is standard behavior for dynamic
objects, before we start patching around I think we need to understand
why it doesn't work - maybe we are doing something wrong in some other
place.
So right now do we have something simple on Mac that looks like mod_php
and demonstrates the problem?

Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

13 years ago by Dmitry Stogov — view source

unread

I thought Yoram sent some very simple test case yesterday.

Thanks. Dmitry.

Hi!

Indeed, when using dlsym the value is recreated.
now the question is what happens at apache restart (on mac) that causes
incorrect behavior.
a-priory to that question, static initializers can not be trusted on some
situations.

I think we need to figure out why. This is standard behavior for dynamic
objects, before we start patching around I think we need to understand
why it doesn't work - maybe we are doing something wrong in some other
place.
So right now do we have something simple on Mac that looks like mod_php
and demonstrates the problem?

13 years ago by Stas Malyshev — view source

unread

Hi!

I thought Yoram sent some very simple test case yesterday.

Yes, as Gwynne commented, this test case doesn't do what PHP does, since
it compiles .so into the binary, and mod_php is not compiled into
Apache's binary, and this changes the way it works.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

13 years ago by yoram bar haim — view source

unread

The last reproduction was executed on darwin 10.8 with gcc 4.2.1

Hi!

I'm not familiar with Mac, but I saw such behavior on Linux very long
time ago, when on PHP restart static variables were still keep values
from old process.

That might happen if the shared library was not unloaded (dlopen will
not re-load library that is already loaded) however I don't understand
how it can happen on Apache - it's supposed to dlclose all modules. Is
it reproducible on any Apache version on Linux or Mac?

13 years ago by yoram bar haim — view source

unread

Hi Stas
reproduction is indeed easy - just change a static variable (e.q. call
zend_set_user_opcode_handler()), then send SIGHUP to apache parent process and
check the value in zend_user_opcode_handlers or zend_user_opcodes.

it is possible that this problem is related to specific gcc versions but the
bottom line is that static initializers occur at first dlpopen but don't re-
occur after dlclose()+dlopen().
on linux it behaves ok, unless you have refrence count bug or RTLD_NODELETE
on your link flags. however, I also saw this happens on linux once, probably
because of glibc bug in specific version.

let me return with more detailes about mac.

Hi!

Hi Dmitry.
the patch does work, in sense of actually restoring the handlers state
when you call zend_set_user_opcode_handler() with original handler.
the patch does not solve the basic problem of no re-initialization of
static arrayes on mac.

Could you please describe in more details what's going on on mac? It
seems strange that dlopen() would not initialize vars - that'd be pretty
bad behavior. Do you have an easy way to reproduce it?

restore user opcode handler in PHP

Could you please describe in more details what's going on on mac? It seems strange that dlopen() would not initialize vars - that'd be pretty bad behavior. Do you have an easy way to reproduce it?

That might happen if the shared library was not unloaded (dlopen will not re-load library that is already loaded) however I don't understand how it can happen on Apache - it's supposed to dlclose all modules. Is it reproducible on any Apache version on Linux or Mac?

Could you please describe in more details what's going on on mac? It
seems strange that dlopen() would not initialize vars - that'd be pretty
bad behavior. Do you have an easy way to reproduce it?

That might happen if the shared library was not unloaded (dlopen will
not re-load library that is already loaded) however I don't understand
how it can happen on Apache - it's supposed to dlclose all modules. Is
it reproducible on any Apache version on Linux or Mac?