Hi,
First, I don't know if this must be considered as a bug, because a lot
of features are not available during MINIT but it seems quite
inconsistent. I precise I'm using the 7.x master branch (I don't know if
it works on 5.x or not).
I am writing an extension where I need to read a file tree during the
MINIT phase. So, I am using php_stream_scandir() to read some
directories. At this time, php_init_stream_wrappers() has run, so I
consider stream wrappers are available.
Unfortunately, this is not true because stream wrappers register
resources in EG(regular_list), which is not initialized before
init_compiler() has run. If I call zend_hash_init(&(EG(regular_list)))
before calling php_stream_scandir(), it seems to works. But it is not
the solution because the hashtable is never destroyed and there are
probably other side effects. The same potentially applies to
EG(persistent_list).
So, does it mean that stream wrappers cannot be used during MINIT, or do
you know a workaround ? I can revert to the C stdio library but, for
portability reasons, I'd prefer using the plain wrapper. Every
suggestion is welcome.
Another suggestion to help detecting such issues : can we consider, in
_zend_is_inconsistent(), that a HashTable whose nTableSize is null is
inconsistent and raises an alert. An uninitialized HashTable generally
is filled with 0s. Today, using an uninitialized HashTable goes
undetected, even in debug mode (because HT_OK == 0), and is very hard to
track.
Regards
François
It doesn't sound like what you are doing belongs in MINIT, it seems to
belong in RINIT.
MINIT is only for module setup with regard to zend (and other deps
perhaps), but you are preparing for the user, that's request stuff I think
...
Cheers
Joe
On Fri, Nov 6, 2015 at 12:08 AM, François Laupretre francois@php.net
wrote:
Hi,
First, I don't know if this must be considered as a bug, because a lot of
features are not available during MINIT but it seems quite inconsistent. I
precise I'm using the 7.x master branch (I don't know if it works on 5.x or
not).I am writing an extension where I need to read a file tree during the
MINIT phase. So, I am using php_stream_scandir() to read some directories.
At this time, php_init_stream_wrappers() has run, so I consider stream
wrappers are available.Unfortunately, this is not true because stream wrappers register resources
in EG(regular_list), which is not initialized before init_compiler() has
run. If I call zend_hash_init(&(EG(regular_list))) before calling
php_stream_scandir(), it seems to works. But it is not the solution because
the hashtable is never destroyed and there are probably other side effects.
The same potentially applies to EG(persistent_list).So, does it mean that stream wrappers cannot be used during MINIT, or do
you know a workaround ? I can revert to the C stdio library but, for
portability reasons, I'd prefer using the plain wrapper. Every suggestion
is welcome.Another suggestion to help detecting such issues : can we consider, in
_zend_is_inconsistent(), that a HashTable whose nTableSize is null is
inconsistent and raises an alert. An uninitialized HashTable generally is
filled with 0s. Today, using an uninitialized HashTable goes undetected,
even in debug mode (because HT_OK == 0), and is very hard to track.Regards
François
Hi,
First, I don't know if this must be considered as a bug, because a lot of
features are not available during MINIT but it seems quite inconsistent. I
precise I'm using the 7.x master branch (I don't know if it works on 5.x or
not).
MINIT wasn't designed to run any php code. You can't create objects and
resources in this stage.
I am writing an extension where I need to read a file tree during the
MINIT phase. So, I am using php_stream_scandir() to read some directories.
At this time, php_init_stream_wrappers() has run, so I consider stream
wrappers are available.Unfortunately, this is not true because stream wrappers register resources
in EG(regular_list), which is not initialized before init_compiler() has
run. If I call zend_hash_init(&(EG(regular_list))) before calling
php_stream_scandir(), it seems to works. But it is not the solution because
the hashtable is never destroyed and there are probably other side effects.
The same potentially applies to EG(persistent_list).So, does it mean that stream wrappers cannot be used during MINIT, or do
you know a workaround ? I can revert to the C stdio library but, for
portability reasons, I'd prefer using the plain wrapper. Every suggestion
is welcome.
It seems, you can't use wrappers. Probably, nobody tried this before :)
You may try to use php_scandir() wrapper.
Another suggestion to help detecting such issues : can we consider, in
_zend_is_inconsistent(), that a HashTable whose nTableSize is null is
inconsistent and raises an alert. An uninitialized HashTable generally is
filled with 0s. Today, using an uninitialized HashTable goes undetected,
even in debug mode (because HT_OK == 0), and is very hard to track.
Uninitialized HashTables are not necessary filled with 0s.
We can add more assert()s, but this is going to be a protection from our
selves.
Anyway, attempts of destruction of uninitialized HashTables most probably
should lead to crash.
Thanks. Dmitry.
Regards
François
Le 09/11/2015 08:51, Dmitry Stogov a écrit :
On Fri, Nov 6, 2015 at 3:08 AM, François Laupretre <francois@php.net
mailto:francois@php.net> wrote:Hi, First, I don't know if this must be considered as a bug, because a lot of features are not available during MINIT but it seems quite inconsistent. I precise I'm using the 7.x master branch (I don't know if it works on 5.x or not).MINIT wasn't designed to run any php code. You can't create objects
and resources in this stage.I am writing an extension where I need to read a file tree during the MINIT phase. So, I am using php_stream_scandir() to read some directories. At this time, php_init_stream_wrappers() has run, so I consider stream wrappers are available. Unfortunately, this is not true because stream wrappers register resources in EG(regular_list), which is not initialized before init_compiler() has run. If I call zend_hash_init(&(EG(regular_list))) before calling php_stream_scandir(), it seems to works. But it is not the solution because the hashtable is never destroyed and there are probably other side effects. The same potentially applies to EG(persistent_list). So, does it mean that stream wrappers cannot be used during MINIT, or do you know a workaround ? I can revert to the C stdio library but, for portability reasons, I'd prefer using the plain wrapper. Every suggestion is welcome.It seems, you can't use wrappers. Probably, nobody tried this before :)
You may try to use php_scandir() wrapper.
Thanks for this. It is the workaround I needed. I should have thought of
ini settings because these also need to scan directories at MINIT time.
Another suggestion to help detecting such issues : can we consider, in _zend_is_inconsistent(), that a HashTable whose nTableSize is null is inconsistent and raises an alert. An uninitialized HashTable generally is filled with 0s. Today, using an uninitialized HashTable goes undetected, even in debug mode (because HT_OK == 0), and is very hard to track.Uninitialized HashTables are not necessary filled with 0s.
We can add moreassert()s, but this is going to be a protection from
our selves.
Anyway, attempts of destruction of uninitialized HashTables most
probably should lead to crash.
Of course, it won't trap every case but I prefer a clean failing
assertion as soon as possible than a cryptic error much later. That's
the purpose af assert(), after all. What I'm proposing is that, since
(nTableSize = 0) should never happen, it is checked by an additional
assert when checking array consistency. A better protection would be, in
ZEND_DEBUG mode only, to write a special value at init time somewhere in
the structure (a typical #55aa for instance). Then, hash consistency
check would check this value. This would be a much better way of
detecting uninitialized hash tables.
Thanks
François
Hi,
An uninitialized HashTable generally is
filled with 0s. Today, using an uninitialized HashTable goes undetected,
even in debug mode (because HT_OK == 0), and is very hard to track.
Uninitialized variables should be pretty easy to check by setting
export USE_ZEND_ALLOC=0 to make allocs go straight through to malloc
and then running the program through valgrind. That should report all
uninitialized data. Additionally using the --malloc-fill=255 (or
appropriate value) should make the code blow up pretty quickly.
I'd be interested to know if that's doesn't report the issues for you
- as that's the backup to check that I haven't forgotten to initialize
anything, and if that's not reliable....I might need to check some
stuff.
cheers
Dan
Le 09/11/2015 16:41, Dan Ackroyd a écrit :
Hi,
An uninitialized HashTable generally is
filled with 0s. Today, using an uninitialized HashTable goes undetected,
even in debug mode (because HT_OK == 0), and is very hard to track.
Uninitialized variables should be pretty easy to check by setting
export USE_ZEND_ALLOC=0to make allocs go straight through to malloc
and then running the program through valgrind. That should report all
uninitialized data. Additionally using the --malloc-fill=255 (or
appropriate value) should make the code blow up pretty quickly.I'd be interested to know if that's doesn't report the issues for you
- as that's the backup to check that I haven't forgotten to initialize
anything, and if that's not reliable....I might need to check some
stuff.cheers
Dan
Your method works for dynamically allocated data but, in my case, the
uninitialized HashTable is EG(regular_list), which is not dynamically
allocated in non-ZTS mode. For data coming from the data or bss sections
of the executable, I think that valgrind cannot detect anything (not
sure for bss, not tested).
The problem with uninitialized hash struct is that, for various reasons,
the issue may remain unnoticed during a long time before the final crash
and, because of this delay, it is quite hard to go back to the primary
cause. That's why I propose to add an assert. This is not perfect but,
in many cases, this will avoid the program to return happily from a
failed operation on an uninitialized hash.
Regards
François