# PHP 7 : Cannot use stream wrappers during MINIT - Thread: https://externals.io/message/89091 - Original archive: https://news-web.php.net/php.internals/89091 - Started by francois@php.net on 2015-11-06T00:08:06+00:00 - Messages: 6 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #89091 — francois@php.net — 2015-11-06T00:08:06+00:00 URL: https://externals.io/message/89091#89091 Hi, First, I don't know if this must be considered as a bug, because a lot of features are not available during MINIT but it seems quite inconsistent. I precise I'm using the 7.x master branch (I don't know if it works on 5.x or not). I am writing an extension where I need to read a file tree during the MINIT phase. So, I am using php_stream_scandir() to read some directories. At this time, php_init_stream_wrappers() has run, so I consider stream wrappers are available. Unfortunately, this is not true because stream wrappers register resources in EG(regular_list), which is not initialized before init_compiler() has run. If I call zend_hash_init(&(EG(regular_list))) before calling php_stream_scandir(), it seems to works. But it is not the solution because the hashtable is never destroyed and there are probably other side effects. The same potentially applies to EG(persistent_list). So, does it mean that stream wrappers cannot be used during MINIT, or do you know a workaround ? I can revert to the C stdio library but, for portability reasons, I'd prefer using the plain wrapper. Every suggestion is welcome. Another suggestion to help detecting such issues : can we consider, in _zend_is_inconsistent(), that a HashTable whose nTableSize is null is inconsistent and raises an alert. An uninitialized HashTable generally is filled with 0s. Today, using an uninitialized HashTable goes undetected, even in debug mode (because HT_OK == 0), and is very hard to track. Regards François --- ## #89098 — Joe Watkins — 2015-11-08T14:11:41+00:00 URL: https://externals.io/message/89091#89098 · In reply to: #89091 It doesn't sound like what you are doing belongs in MINIT, it seems to belong in RINIT. MINIT is only for module setup with regard to zend (and other deps perhaps), but you are preparing for the user, that's request stuff I think ... Cheers Joe On Fri, Nov 6, 2015 at 12:08 AM, François Laupretre wrote: > Hi, > > First, I don't know if this must be considered as a bug, because a lot of > features are not available during MINIT but it seems quite inconsistent. I > precise I'm using the 7.x master branch (I don't know if it works on 5.x or > not). > > I am writing an extension where I need to read a file tree during the > MINIT phase. So, I am using php_stream_scandir() to read some directories. > At this time, php_init_stream_wrappers() has run, so I consider stream > wrappers are available. > > Unfortunately, this is not true because stream wrappers register resources > in EG(regular_list), which is not initialized before init_compiler() has > run. If I call zend_hash_init(&(EG(regular_list))) before calling > php_stream_scandir(), it seems to works. But it is not the solution because > the hashtable is never destroyed and there are probably other side effects. > The same potentially applies to EG(persistent_list). > > So, does it mean that stream wrappers cannot be used during MINIT, or do > you know a workaround ? I can revert to the C stdio library but, for > portability reasons, I'd prefer using the plain wrapper. Every suggestion > is welcome. > > Another suggestion to help detecting such issues : can we consider, in > _zend_is_inconsistent(), that a HashTable whose nTableSize is null is > inconsistent and raises an alert. An uninitialized HashTable generally is > filled with 0s. Today, using an uninitialized HashTable goes undetected, > even in debug mode (because HT_OK == 0), and is very hard to track. > > Regards > > François --- ## #89116 — Dmitry Stogov — 2015-11-09T07:51:23+00:00 URL: https://externals.io/message/89091#89116 · In reply to: #89091 > Hi, > > First, I don't know if this must be considered as a bug, because a lot of > features are not available during MINIT but it seems quite inconsistent. I > precise I'm using the 7.x master branch (I don't know if it works on 5.x or > not). > MINIT wasn't designed to run any php code. You can't create objects and resources in this stage. > > I am writing an extension where I need to read a file tree during the > MINIT phase. So, I am using php_stream_scandir() to read some directories. > At this time, php_init_stream_wrappers() has run, so I consider stream > wrappers are available. > > Unfortunately, this is not true because stream wrappers register resources > in EG(regular_list), which is not initialized before init_compiler() has > run. If I call zend_hash_init(&(EG(regular_list))) before calling > php_stream_scandir(), it seems to works. But it is not the solution because > the hashtable is never destroyed and there are probably other side effects. > The same potentially applies to EG(persistent_list). > > So, does it mean that stream wrappers cannot be used during MINIT, or do > you know a workaround ? I can revert to the C stdio library but, for > portability reasons, I'd prefer using the plain wrapper. Every suggestion > is welcome. > It seems, you can't use wrappers. Probably, nobody tried this before :) You may try to use php_scandir() wrapper. > > Another suggestion to help detecting such issues : can we consider, in > _zend_is_inconsistent(), that a HashTable whose nTableSize is null is > inconsistent and raises an alert. An uninitialized HashTable generally is > filled with 0s. Today, using an uninitialized HashTable goes undetected, > even in debug mode (because HT_OK == 0), and is very hard to track. > Uninitialized HashTables are not necessary filled with 0s. We can add more `assert()`s, but this is going to be a protection from our selves. Anyway, attempts of destruction of uninitialized HashTables most probably should lead to crash. Thanks. Dmitry. > > Regards > > François --- ## #89123 — Dan Ackroyd — 2015-11-09T15:41:57+00:00 URL: https://externals.io/message/89091#89123 · In reply to: #89091 > Hi, > > An uninitialized HashTable generally is > filled with 0s. Today, using an uninitialized HashTable goes undetected, > even in debug mode (because HT_OK == 0), and is very hard to track. Uninitialized variables should be pretty easy to check by setting `export USE_ZEND_ALLOC=0` to make allocs go straight through to malloc and then running the program through valgrind. That should report all uninitialized data. Additionally using the --malloc-fill=255 (or appropriate value) should make the code blow up pretty quickly. I'd be interested to know if that's doesn't report the issues for you - as that's the backup to check that I haven't forgotten to initialize anything, and if that's not reliable....I might need to check some stuff. cheers Dan --- ## #89285 — francois@php.net — 2015-11-21T00:51:47+00:00 URL: https://externals.io/message/89091#89285 · In reply to: #89116 Le 09/11/2015 08:51, Dmitry Stogov a écrit : > > > On Fri, Nov 6, 2015 at 3:08 AM, François Laupretre > wrote: > > Hi, > > First, I don't know if this must be considered as a bug, because a > lot of features are not available during MINIT but it seems quite > inconsistent. I precise I'm using the 7.x master branch (I don't > know if it works on 5.x or not). > > > MINIT wasn't designed to run any php code. You can't create objects > and resources in this stage. > > > I am writing an extension where I need to read a file tree during > the MINIT phase. So, I am using php_stream_scandir() to read some > directories. At this time, php_init_stream_wrappers() has run, so > I consider stream wrappers are available. > > Unfortunately, this is not true because stream wrappers register > resources in EG(regular_list), which is not initialized before > init_compiler() has run. If I call > zend_hash_init(&(EG(regular_list))) before calling > php_stream_scandir(), it seems to works. But it is not the > solution because the hashtable is never destroyed and there are > probably other side effects. The same potentially applies to > EG(persistent_list). > > So, does it mean that stream wrappers cannot be used during MINIT, > or do you know a workaround ? I can revert to the C stdio library > but, for portability reasons, I'd prefer using the plain wrapper. > Every suggestion is welcome. > > > It seems, you can't use wrappers. Probably, nobody tried this before :) > You may try to use php_scandir() wrapper. Thanks for this. It is the workaround I needed. I should have thought of ini settings because these also need to scan directories at MINIT time. > Another suggestion to help detecting such issues : can we > consider, in _zend_is_inconsistent(), that a HashTable whose > nTableSize is null is inconsistent and raises an alert. An > uninitialized HashTable generally is filled with 0s. Today, using > an uninitialized HashTable goes undetected, even in debug mode > (because HT_OK == 0), and is very hard to track. > > > Uninitialized HashTables are not necessary filled with 0s. > We can add more `assert()`s, but this is going to be a protection from > our selves. > Anyway, attempts of destruction of uninitialized HashTables most > probably should lead to crash. Of course, it won't trap every case but I prefer a clean failing assertion as soon as possible than a cryptic error much later. That's the purpose af `assert()`, after all. What I'm proposing is that, since (nTableSize = 0) should never happen, it is checked by an additional assert when checking array consistency. A better protection would be, in ZEND_DEBUG mode only, to write a special value at init time somewhere in the structure (a typical #55aa for instance). Then, hash consistency check would check this value. This would be a much better way of detecting uninitialized hash tables. Thanks François --- ## #89286 — francois@php.net — 2015-11-21T01:10:48+00:00 URL: https://externals.io/message/89091#89286 · In reply to: #89123 Le 09/11/2015 16:41, Dan Ackroyd a écrit : > > > […] > Uninitialized variables should be pretty easy to check by setting > `export USE_ZEND_ALLOC=0` to make allocs go straight through to malloc > and then running the program through valgrind. That should report all > uninitialized data. Additionally using the --malloc-fill=255 (or > appropriate value) should make the code blow up pretty quickly. > > I'd be interested to know if that's doesn't report the issues for you > - as that's the backup to check that I haven't forgotten to initialize > anything, and if that's not reliable....I might need to check some > stuff. > > cheers > Dan Your method works for dynamically allocated data but, in my case, the uninitialized HashTable is EG(regular_list), which is not dynamically allocated in non-ZTS mode. For data coming from the data or bss sections of the executable, I think that valgrind cannot detect anything (not sure for bss, not tested). The problem with uninitialized hash struct is that, for various reasons, the issue may remain unnoticed during a long time before the final crash and, because of this delay, it is quite hard to go back to the primary cause. That's why I propose to add an assert. This is not perfect but, in many cases, this will avoid the program to return happily from a failed operation on an uninitialized hash. Regards François