# restore user opcode handler in PHP - Thread: https://externals.io/message/57808 - Original archive: https://news-web.php.net/php.internals/57808 - Started by yoram bar haim on 2012-02-12T12:09:54+00:00 - Messages: 17 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #57808 — yoram bar haim — 2012-02-12T12:09:54+00:00 URL: https://externals.io/message/57808#57808 I first saw this problem while addapting extension to PHP 5.4 but the code is the same in 5.3 : when you call zend_set_user_opcode_handler() it set's a pointer (handler) in zend_user_opcode_handlers and a uint opcode value in zend_user_opcodes to ZEND_USER_OPCODE . you can call zend_set_user_opcode_handler with the original handler to restore it, but if the original state did not include ZEND_USER_OPCODE you have no way to restore it. Why restoring ? On mac, static variables are not re-initialized on dlclose()+dlopen(). that means that apache reload does not re-create zend_user_opcode_handlers and zend_user_opcodes. if your extension sets user_opcode_handler and is not reloaded in apache reload (because you commented it in php.ini, for example), your handlers array will actually point to non-exist handler. if you will call zend_set_user_opcode_handler() with the original handler at MSHUTDOWN, it will restore the handler but zend_user_opcodes[] or we should build those arrayes from scratch at php_module_init. currently an extension tha calls zend_set_user_opcode_handler() will make PHP crash on mac after disabling the extension and apache reload. --- ## #57809 — Laruence — 2012-02-12T12:41:44+00:00 URL: https://externals.io/message/57808#57809 · In reply to: #57808 Hi: could you file a feature req on bugs.php.net? I think it's okey to add a zend_restore_user_handler . like: ZEND_API int zend_restore_user_opcode_handler(zend_uchar opcode) { zend_user_opcodes[opcode] = opcode; zend_user_opcode_handlers[opcode] = NULL; return SUCCESS; } but it still need dmitry to review :) thanks > I first saw this problem while addapting extension to PHP 5.4 but the code is > the same in 5.3 : > when you call zend_set_user_opcode_handler() it set's a pointer (handler) in > zend_user_opcode_handlers and a uint opcode value in zend_user_opcodes to > ZEND_USER_OPCODE . > > you can call zend_set_user_opcode_handler with the original handler to restore > it, but if the original state did not include ZEND_USER_OPCODE you have no way > to restore it. > > Why restoring ? > On mac, static variables are not re-initialized on dlclose()+dlopen(). that > means that apache reload does not re-create zend_user_opcode_handlers and > zend_user_opcodes. > if your extension sets user_opcode_handler and is not reloaded in apache > reload (because you commented it in php.ini, for example), your handlers array > will actually point to non-exist handler. > if you will call zend_set_user_opcode_handler() with the original handler at > MSHUTDOWN, it will restore the handler but zend_user_opcodes[ still be ZEND_USER_OPCODE so you will end up calling a `NULL` handler. > > we should eather allow zend_set_user_opcode_handler to also set the value in > zend_user_opcodes[] or we should build those arrayes from scratch at > php_module_init. > > currently an extension tha calls zend_set_user_opcode_handler() will make PHP > crash on mac after disabling the extension and apache reload. > > -- > > > -- Laruence  Xinchen Hui http://www.laruence.com/ --- ## #57810 — yoram bar haim — 2012-02-12T12:55:44+00:00 URL: https://externals.io/message/57808#57810 · In reply to: #57809 I think the following patch is a better solution, currently trying it : --- Zend/zend_execute.c.orig 2012-02-12 14:32:07.000000000 +0200 +++ Zend/zend_execute.c 2012-02-12 14:31:33.000000000 +0200 @@ -1512,7 +1512,11 @@ ZEND_API int zend_set_user_opcode_handler(zend_uchar opcode, user_opcode_handler_t handler) { if (opcode != ZEND_USER_OPCODE) { - zend_user_opcodes[opcode] = ZEND_USER_OPCODE; + if (handler == NULL) { + zend_user_opcodes[opcode] = opcode; + } else { + zend_user_opcodes[opcode] = ZEND_USER_OPCODE; + } zend_user_opcode_handlers[opcode] = handler; return SUCCESS; } > Hi: > could you file a feature req on bugs.php.net? > > I think it's okey to add a zend_restore_user_handler . > > like: > > ZEND_API int zend_restore_user_opcode_handler(zend_uchar opcode) { > zend_user_opcodes[opcode] = opcode; > zend_user_opcode_handlers[opcode] = NULL; > return SUCCESS; > } > > but it still need dmitry to review :) > > thanks > > > > […] --- ## #57814 — Dmitry Stogov — 2012-02-13T06:31:21+00:00 URL: https://externals.io/message/57808#57814 · In reply to: #57808 Hi Yoram, Can we just fill this arrays with zeros on php startup? Thanks. Dmitry. > I first saw this problem while addapting extension to PHP 5.4 but the code is > the same in 5.3 : > when you call zend_set_user_opcode_handler() it set's a pointer (handler) in > zend_user_opcode_handlers and a uint opcode value in zend_user_opcodes to > ZEND_USER_OPCODE . > > you can call zend_set_user_opcode_handler with the original handler to restore > it, but if the original state did not include ZEND_USER_OPCODE you have no way > to restore it. > > Why restoring ? > On mac, static variables are not re-initialized on dlclose()+dlopen(). that > means that apache reload does not re-create zend_user_opcode_handlers and > zend_user_opcodes. > if your extension sets user_opcode_handler and is not reloaded in apache > reload (because you commented it in php.ini, for example), your handlers array > will actually point to non-exist handler. > if you will call zend_set_user_opcode_handler() with the original handler at > MSHUTDOWN, it will restore the handler but zend_user_opcodes[ still be ZEND_USER_OPCODE so you will end up calling a `NULL` handler. > > we should eather allow zend_set_user_opcode_handler to also set the value in > zend_user_opcodes[] or we should build those arrayes from scratch at > php_module_init. > > currently an extension tha calls zend_set_user_opcode_handler() will make PHP > crash on mac after disabling the extension and apache reload. --- ## #57815 — Dmitry Stogov — 2012-02-13T06:33:54+00:00 URL: https://externals.io/message/57808#57815 · In reply to: #57810 In case this patch works, I think it's fine to include it. Thanks. Dmitry. > I think the following patch is a better solution, currently trying it : > --- Zend/zend_execute.c.orig 2012-02-12 14:32:07.000000000 +0200 > +++ Zend/zend_execute.c 2012-02-12 14:31:33.000000000 +0200 > @@ -1512,7 +1512,11 @@ > ZEND_API int zend_set_user_opcode_handler(zend_uchar opcode, > user_opcode_handler_t handler) > { > if (opcode != ZEND_USER_OPCODE) { > - zend_user_opcodes[opcode] = ZEND_USER_OPCODE; > + if (handler == NULL) { > + zend_user_opcodes[opcode] = opcode; > + } else { > + zend_user_opcodes[opcode] = ZEND_USER_OPCODE; > + } > zend_user_opcode_handlers[opcode] = handler; > return SUCCESS; > } > > > > > […] --- ## #57817 — yoram bar haim — 2012-02-13T08:56:03+00:00 URL: https://externals.io/message/57808#57817 · In reply to: #57814 Hi Dmitry. the patch does work, in sense of actually restoring the handlers state when you call zend_set_user_opcode_handler() with original handler. the patch does not solve the basic problem of no re-initialization of static arrayes on mac. As for feeling the array with zero's, if we do that, we also need to fill it with original opcodes, something like : static zend_base_user_opcodes = {0,1,2,3,4,....}; and then in php_module_startup : memset (zend_user_opcode_handlers,0,sizeof(zend_user_opcode_handlers); memcpy (zend_user_opcodes,zend_base_user_opcodes,sizeof(zend_base_user_opcodes); So basically, every modifiably variable should be initialized as startup and not only by static initializer. > Hi Yoram, > > Can we just fill this arrays with zeros on php startup? > > Thanks. Dmitry. > > > > […] --- ## #57818 — Stas Malyshev — 2012-02-13T09:02:20+00:00 URL: https://externals.io/message/57808#57818 · In reply to: #57817 Hi! > Hi Dmitry. > the patch does work, in sense of actually restoring the handlers state when > you call zend_set_user_opcode_handler() with original handler. > the patch does not solve the basic problem of no re-initialization of static > arrayes on mac. Could you please describe in more details what's going on on mac? It seems strange that dlopen() would not initialize vars - that'd be pretty bad behavior. Do you have an easy way to reproduce it? -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #57819 — Dmitry Stogov — 2012-02-13T09:11:20+00:00 URL: https://externals.io/message/57808#57819 · In reply to: #57818 I'm not familiar with Mac, but I saw such behavior on Linux very long time ago, when on PHP restart static variables were still keep values from old process. Thanks. Dmitry. > Hi! > > > […] > > Could you please describe in more details what's going on on mac? It > seems strange that dlopen() would not initialize vars - that'd be pretty > bad behavior. Do you have an easy way to reproduce it? --- ## #57820 — Stas Malyshev — 2012-02-13T09:16:41+00:00 URL: https://externals.io/message/57808#57820 · In reply to: #57819 Hi! > I'm not familiar with Mac, but I saw such behavior on Linux very long > time ago, when on PHP restart static variables were still keep values > from old process. That might happen if the shared library was not unloaded (dlopen will not re-load library that is already loaded) however I don't understand how it can happen on Apache - it's supposed to dlclose all modules. Is it reproducible on any Apache version on Linux or Mac? -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #57821 — yoram bar haim — 2012-02-13T09:23:11+00:00 URL: https://externals.io/message/57808#57821 · In reply to: #57818 Hi Stas reproduction is indeed easy - just change a static variable (e.q. call zend_set_user_opcode_handler()), then send `SIGHUP` to apache parent process and check the value in zend_user_opcode_handlers or zend_user_opcodes. it is possible that this problem is related to specific gcc versions but the bottom line is that static initializers occur at first dlpopen but don't re- occur after dlclose()+dlopen(). on linux it behaves ok, unless you have refrence count bug or RTLD_NODELETE on your link flags. however, I also saw this happens on linux once, probably because of glibc bug in specific version. let me return with more detailes about mac. > Hi! > > > […] > > Could you please describe in more details what's going on on mac? It > seems strange that dlopen() would not initialize vars - that'd be pretty > bad behavior. Do you have an easy way to reproduce it? --- ## #57823 — yoram bar haim — 2012-02-13T09:38:00+00:00 URL: https://externals.io/message/57808#57823 · In reply to: #57820 Here is a simple test program that reproduce the issue on mac: lib.h: #include "lib.h" void set_index(int index, int value); int get_index(int index); lib.c: #include "lib.h" static int glob_var[] = {0,0,0,0}; void set_index(int index, int value) { glob_var[index] = value; } int get_index(int index) { return glob_var[index]; } main.c: #include #include #include "lib.h" int main() { void *handle = dlopen ("lib.so",0); set_index(0,1); dlclose(handle); handle = dlopen ("lib.so",0); printf("%d\n",get_index(0)); return 0; } shell> gcc -o lib.so -shared lib.c shell> gcc -o main main.c lib.so shell> ./main > Hi! > > > […] > > That might happen if the shared library was not unloaded (dlopen will > not re-load library that is already loaded) however I don't understand > how it can happen on Apache - it's supposed to dlclose all modules. Is > it reproducible on any Apache version on Linux or Mac? --- ## #57824 — yoram bar haim — 2012-02-13T09:39:54+00:00 URL: https://externals.io/message/57808#57824 · In reply to: #57820 The last reproduction was executed on darwin 10.8 with gcc 4.2.1 > Hi! > > > […] > > That might happen if the shared library was not unloaded (dlopen will > not re-load library that is already loaded) however I don't understand > how it can happen on Apache - it's supposed to dlclose all modules. Is > it reproducible on any Apache version on Linux or Mac? --- ## #57825 — Gwynne Raskind — 2012-02-13T09:44:15+00:00 URL: https://externals.io/message/57808#57825 · In reply to: #57823 > Here is a simple test program that reproduce the issue on mac: [snip] > shell> gcc -o lib.so -shared lib.c > shell> gcc -o main main.c lib.so > shell> ./main This example is flawed. The manpage for dlclose(3) on Darwin explicitly states that a library linked by the executable will never be unloaded. You link directly to the library here. Also, you use prototypes of the library's functions rather than calling dlsym(3), forcing you to link to the library, defeating the entire point of loading it with dlopen(3) in the first place. I've included a test which does not show the issue at the bottom of this message. (As a side note of interest, OS X has historically always had issues with unloading images from the address space; for example, even in Lion, if you unload an image containing Objective-C symbols, you stand a pretty good chance of crashing. Image unloading simply didn't exist at all in any useful form before Leopard.) -- Gwynne Raskind /* dylib.c - clang dylib.c -o dylib.dylib -dynamiclib */ static int var = 0; extern int getVar(void); extern void setVar(int value); int getVar(void) { return var; } void setVar(int value) { var = value; } /* loader.c - clang loader.c -o loader */ #include #include int main(int argc, char **argv) { void *libh = dlopen("dylib.dylib", 0); int (*getfunc)() = dlsym(libh, "getVar"); void (*setfunc)(int) = dlsym(libh, "setVar"); printf("First load value: %d\n", getfunc()); setfunc(5); printf("First load set value: %d\n", getfunc()); dlclose(libh); libh = dlopen("dylib.dylib", 0); getfunc = dlsym(libh, "getVar"); setfunc = dlsym(libh, "setVar"); printf("Second load value: %d\n", getfunc()); return 0; } /* output */ $ uname -v ; clang --version ; ./loader Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 clang version 3.0 (tags/RELEASE_30/final) First load value: 0 First load set value: 5 Second load value: 0 --- ## #57826 — yoram bar haim — 2012-02-13T09:57:31+00:00 URL: https://externals.io/message/57808#57826 · In reply to: #57825 Indeed, when using dlsym the value is recreated. now the question is what happens at apache restart (on mac) that causes incorrect behavior. a-priory to that question, static initializers can not be trusted on some situations. > > > […] > [snip] > > > […] > > This example is flawed. > > The manpage for dlclose(3) on Darwin explicitly states that a library > linked by the executable will never be unloaded. You link directly to > the library here. Also, you use prototypes of the library's functions > rather than calling dlsym(3), forcing you to link to the library, > defeating the entire point of loading it with dlopen(3) in the first > place. I've included a test which does not show the issue at the > bottom of this message. > > (As a side note of interest, OS X has historically always had issues > with unloading images from the address space; for example, even in > Lion, if you unload an image containing Objective-C symbols, you stand > a pretty good chance of crashing. Image unloading simply didn't exist > at all in any useful form before Leopard.) > > -- Gwynne Raskind > > /* dylib.c - clang dylib.c -o dylib.dylib -dynamiclib */ > static int var = 0; > extern int getVar(void); > extern void setVar(int value); > int getVar(void) { return var; } > void setVar(int value) { var = value; } > > /* loader.c - clang loader.c -o loader */ > #include > #include > int main(int argc, char **argv) > { > void *libh = dlopen("dylib.dylib", 0); > int (*getfunc)() = dlsym(libh, "getVar"); > void (*setfunc)(int) = dlsym(libh, "setVar"); > > printf("First load value: %d\n", getfunc()); > setfunc(5); > printf("First load set value: %d\n", getfunc()); > dlclose(libh); > libh = dlopen("dylib.dylib", 0); > getfunc = dlsym(libh, "getVar"); > setfunc = dlsym(libh, "setVar"); > printf("Second load value: %d\n", getfunc()); > return 0; > } > > /* output */ > $ uname -v ; clang --version ; ./loader > Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; > root:xnu-1699.24.23~1/RELEASE_X86_64 > clang version 3.0 (tags/RELEASE_30/final) > First load value: 0 > First load set value: 5 > Second load value: 0 --- ## #57834 — Stas Malyshev — 2012-02-13T18:23:48+00:00 URL: https://externals.io/message/57808#57834 · In reply to: #57826 Hi! > Indeed, when using dlsym the value is recreated. > now the question is what happens at apache restart (on mac) that causes > incorrect behavior. > a-priory to that question, static initializers can not be trusted on some > situations. I think we need to figure out why. This is standard behavior for dynamic objects, before we start patching around I think we need to understand why it doesn't work - maybe we are doing something wrong in some other place. So right now do we have something simple on Mac that looks like mod_php and demonstrates the problem? -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227 --- ## #57845 — Dmitry Stogov — 2012-02-14T07:01:34+00:00 URL: https://externals.io/message/57808#57845 · In reply to: #57834 I thought Yoram sent some very simple test case yesterday. Thanks. Dmitry. > Hi! > > > […] > > I think we need to figure out why. This is standard behavior for dynamic > objects, before we start patching around I think we need to understand > why it doesn't work - maybe we are doing something wrong in some other > place. > So right now do we have something simple on Mac that looks like mod_php > and demonstrates the problem? --- ## #57847 — Stas Malyshev — 2012-02-14T07:05:37+00:00 URL: https://externals.io/message/57808#57847 · In reply to: #57845 Hi! > I thought Yoram sent some very simple test case yesterday. Yes, as Gwynne commented, this test case doesn't do what PHP does, since it compiles .so into the binary, and mod_php is not compiled into Apache's binary, and this changes the way it works. -- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227