Fwd: zend_std_get_method : Address 0xffffffff is not stack'd, malloc'd or (recently) free'd

11 years ago by Kajetan Dolinar — view source — reply

unread

Hi there,

I may have missed the right list but please consider my problem as it
needs Zend internals expert attention and may be a Zend bug. I am
developing a PHP extension and have stmbled over a fault. The excerpt
from the valgrind output on that fault says

Invalid read of size 4
at 0x44EAA4: zend_std_get_method (zend_object_handlers.c:847)
by 0x47A035: ZEND_INIT_METHOD_CALL_SPEC_VAR_CONST_HANDLER
(zend_vm_execute.h:10461)
by 0x452D1D: execute (zend_vm_execute.h:107)
by 0x42C398: zend_execute_scripts (zend.c:1236)
by 0x1: ???
Address 0xffffffff is not stack'd, malloc'd or (recently) free'd

The reason most probably lies within my code, but I don't have enough
PHP Zend engine hacking experience to get to it by myself - it's my
first PHP extension. Much obliged for intuitions from experienced for
what could bring about such kind of fault.

The code for my extension comes from a project already thoroughly
debugged, working in production for quite some time. I have dilligently
used the emalloc/erealloc/efree res. pemalloc/perealloc/pefree instead
of the original malloc/realloc/free. I have gone through a separate
evaluation of my code in the setting of the PHP extension using my own
memory management inspection tools as well as valgrind. For what I can
tell is that there are no violations of any obvious kind.

However, there are leaks because I have turned off my own system for
memory control. I am using quite a complicated interplay of allocated
memory as well as references to static data. I use

void * phxy_data_start() { return (void *) &etext; }
void * phxy_variable_start() { return (void *) &edata; }
void * phxy_heap_start() { return (void *) &end; }
void * phxy_stack_start() { return sbrk(0); }
unsigned char phxy_is_heap(void * data) {
return (phxy_heap_start() <= data && data < phxy_stack_start());
}
unsigned char phxy_is_data(void * data) {
return (data >= phxy_data_start());
}

to find out which data need freeing and which not (i.e. before any
free() I check with phxy_is_heap()). But for my PHP extension I dear not
use this so that phxy_is_heap() always returns false for any data -
hence memory leaks. Please advise me of a safe PHP Zend analogue for
that approach.

Thanks
Kajetan

-------- The total valgrind output ------

valgrind --tool=memcheck --time-stamp=yes
--read-var-info=yes --trace-children=yes --leak-check=full
--show-possibly-lost=yes --undef-value-errors=yes
--track-origins=yes php index.php
(The same problem comes with USE_ZEND_ALLOC=0.)

==00:00:00:00.000 24826== Memcheck, a memory error detector
==00:00:00:00.000 24826== Copyright (C) 2002-2011, and GNU GPL'd, by
Julian Seward et al.
==00:00:00:00.000 24826== Using Valgrind-3.7.0 and LibVEX; rerun with -h
for copyright info
==00:00:00:00.000 24826== Command: php index.php
==00:00:00:00.000 24826== Parent PID: 6209
==00:00:00:00.000 24826==
==00:00:00:37.429 24826== Invalid read of size 4
==00:00:00:37.430 24826== at 0x44EAA4: zend_std_get_method
(zend_object_handlers.c:847)
==00:00:00:37.430 24826== by 0x47A035:
ZEND_INIT_METHOD_CALL_SPEC_VAR_CONST_HANDLER (zend_vm_execute.h:10461)
==00:00:00:37.431 24826== by 0x452D1D: execute
(zend_vm_execute.h:107)
==00:00:00:37.431 24826== by 0x42C398: zend_execute_scripts
(zend.c:1236)
==00:00:00:37.431 24826== by 0x1: ???
==00:00:00:37.431 24826== Address 0xffffffff is not stack'd, malloc'd
or (recently) free'd
==00:00:00:37.431 24826==
==00:00:00:37.431 24826==
==00:00:00:37.432 24826== Process terminating with default action of
signal 11 (SIGSEGV)
==00:00:00:37.432 24826== General Protection Fault
==00:00:00:37.432 24826== at 0x44EAA4: zend_std_get_method
(zend_object_handlers.c:847)
==00:00:00:37.432 24826== by 0x47A035:
ZEND_INIT_METHOD_CALL_SPEC_VAR_CONST_HANDLER (zend_vm_execute.h:10461)
==00:00:00:37.432 24826== by 0x452D1D: execute
(zend_vm_execute.h:107)
==00:00:00:37.432 24826== by 0x42C398: zend_execute_scripts
(zend.c:1236)
==00:00:00:37.432 24826== by 0x1: ???
==00:00:00:37.630 24826==
==00:00:00:37.632 24826== HEAP SUMMARY:
==00:00:00:37.632 24826== in use at exit: 2,041,044 bytes in 21,055
blocks
==00:00:00:37.632 24826== total heap usage: 30,045 allocs, 8,990
frees, 3,978,081 bytes allocated
==00:00:00:37.632 24826==
==00:00:00:38.077 24826== 120 bytes in 1 blocks are definitely lost in
loss record 8,039 of 8,869
==00:00:00:38.078 24826== at 0x482C40D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==00:00:00:38.078 24826== by 0x77: ???
==00:00:00:38.078 24826==
==00:00:00:38.082 24826== 144 (80 direct, 64 indirect) bytes in 2 blocks
are definitely lost in loss record 8,162 of 8,869
==00:00:00:38.083 24826== at 0x482C40D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==00:00:00:38.083 24826== by 0x50C7CCD:
hypx_inquiry_radios_object_create_handler (hypx.c:293)
==00:00:00:38.083 24826== by 0x42E276: _object_and_properties_init
(zend_API.c:1092)
==00:00:00:38.083 24826== by 0x496593:
zend_do_fcall_common_helper_SPEC (zend_vm_execute.h:320)
==00:00:00:38.083 24826== by 0x452D1D: execute
(zend_vm_execute.h:107)
==00:00:00:38.083 24826== by 0x42C398: zend_execute_scripts
(zend.c:1236)
==00:00:00:38.083 24826== by 0x1: ???
==00:00:00:38.083 24826==
==00:00:00:38.084 24826== 548 (28 direct, 520 indirect) bytes in 1
blocks are definitely lost in loss record 8,587 of 8,869
==00:00:00:38.085 24826== at 0x482C40D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==00:00:00:38.085 24826== by 0x50EB69C: hypx_expert (expert.c:420)
==00:00:00:38.085 24826== by 0x50EB7A3: hypx_expert_make_inquiry
(expert.c:451)
==00:00:00:38.085 24826== by 0x50CC423: zim_HypxExpert_makeInquiry
(hypx.c:1745)
==00:00:00:38.085 24826== by 0x496593:
zend_do_fcall_common_helper_SPEC (zend_vm_execute.h:320)
==00:00:00:38.085 24826== by 0x452D1D: execute
(zend_vm_execute.h:107)
==00:00:00:38.085 24826== by 0x42C398: zend_execute_scripts
(zend.c:1236)
==00:00:00:38.085 24826== by 0x1: ???
==00:00:00:38.085 24826==
==00:00:00:38.090 24826== 19,591 bytes in 13 blocks are possibly lost in
loss record 8,857 of 8,869
==00:00:00:38.090 24826== at 0x482C40D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==00:00:00:38.090 24826== by 0x50DFDB8: phxy_cbase_find_last_part
(cbase.c:1193)
==00:00:00:38.090 24826== by 0x50E176A:
phxy_cbase_determine_spare_ids (cbase.c:1589)
==00:00:00:38.090 24826== by 0x50E1A7F: phxy_cbase_open
(cbase.c:1650)
==00:00:00:38.090 24826== by 0x50CD2EB: hypx_engine (engine.c:247)
==00:00:00:38.090 24826== by 0x50CD3EA: hypx_engine_start
(engine.c:275)
==00:00:00:38.090 24826== by 0x50C8178: zm_startup_hypx (hypx.c:413)
==00:00:00:38.091 24826== by 0x42CA8B: zend_startup_module_ex
(zend_API.c:1617)
==00:00:00:38.091 24826== by 0x438B8B: zend_hash_apply
(zend_hash.c:674)
==00:00:00:38.091 24826== by 0x43073F: zend_startup_modules
(zend_API.c:1666)
==00:00:00:38.091 24826== by 0x3CCB1C: php_module_startup
(main.c:2066)
==00:00:00:38.091 24826== by 0x4C421A: php_cli_startup
(php_cli.c:398)
==00:00:00:38.091 24826==
==00:00:00:38.091 24826== 79,254 (888 direct, 78,366 indirect) bytes in
11 blocks are definitely lost in loss record 8,865 of 8,869
==00:00:00:38.091 24826== at 0x482C40D: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==00:00:00:38.091 24826== by 0x50DFDB8: phxy_cbase_find_last_part
(cbase.c:1193)
==00:00:00:38.091 24826== by 0x50E176A:
phxy_cbase_determine_spare_ids (cbase.c:1589)
==00:00:00:38.091 24826== by 0x50E1A7F: phxy_cbase_open
(cbase.c:1650)
==00:00:00:38.091 24826== by 0x50CD2EB: hypx_engine (engine.c:247)
==00:00:00:38.091 24826== by 0x50CD3EA: hypx_engine_start
(engine.c:275)
==00:00:00:38.091 24826== by 0x50C8178: zm_startup_hypx (hypx.c:413)
==00:00:00:38.091 24826== by 0x42CA8B: zend_startup_module_ex
(zend_API.c:1617)
==00:00:00:38.091 24826== by 0x438B8B: zend_hash_apply
(zend_hash.c:674)
==00:00:00:38.092 24826== by 0x43073F: zend_startup_modules
(zend_API.c:1666)
==00:00:00:38.092 24826== by 0x3CCB1C: php_module_startup
(main.c:2066)
==00:00:00:38.092 24826== by 0x4C421A: php_cli_startup
(php_cli.c:398)
==00:00:00:38.092 24826==
==00:00:00:38.093 24826== LEAK SUMMARY:
==00:00:00:38.093 24826== definitely lost: 1,116 bytes in 15 blocks
==00:00:00:38.093 24826== indirectly lost: 78,950 bytes in 38 blocks
==00:00:00:38.093 24826== possibly lost: 19,591 bytes in 13 blocks
==00:00:00:38.093 24826== still reachable: 1,941,387 bytes in 20,989
blocks
==00:00:00:38.093 24826== suppressed: 0 bytes in 0 blocks
==00:00:00:38.093 24826== Reachable blocks (those to which a pointer was
found) are not shown.
==00:00:00:38.093 24826== To see them, rerun with: --leak-check=full
--show-reachable=yes
==00:00:00:38.093 24826==
==00:00:00:38.093 24826== For counts of detected and suppressed errors,
rerun with: -v
==00:00:00:38.093 24826== ERROR SUMMARY: 6 errors from 6 contexts
(suppressed: 0 from 0)

If you need anything more, please tell me about it ...

11 years ago by johannes@schlueters.de — view source — reply

unread

Hi there,

I may have missed the right list but please consider my problem as it

This is a good place, usually I suggest using pecl-dev as there are less
mails, so less chance of being overseen.

needs Zend internals expert attention and may be a Zend bug. I am
developing a PHP extension and have stmbled over a fault. The excerpt
from the valgrind output on that fault says

Invalid read of size 4
at 0x44EAA4: zend_std_get_method (zend_object_handlers.c:847)
by 0x47A035: ZEND_INIT_METHOD_CALL_SPEC_VAR_CONST_HANDLER
(zend_vm_execute.h:10461)
by 0x452D1D: execute (zend_vm_execute.h:107)
by 0x42C398: zend_execute_scripts (zend.c:1236)
by 0x1: ???
Address 0xffffffff is not stack'd, malloc'd or (recently) free'd

This won't help much, without any further information like code or at
least PHP version. Apparently 5.3 was the last version where
zend_std_get_method is lose to line 847 in later versions it's starting
somewhere in the thousands ... but that's too little to guess the fault.

The reason most probably lies within my code, but I don't have enough
PHP Zend engine hacking experience to get to it by myself - it's my
first PHP extension. Much obliged for intuitions from experienced for
what could bring about such kind of fault.

Reduce you ode till it works or share more details ..

However, there are leaks because I have turned off my own system for
memory control. I am using quite a complicated interplay of allocated
memory as well as references to static data. I use

void * phxy_data_start() { return (void *) &etext; }
void * phxy_variable_start() { return (void *) &edata; }
void * phxy_heap_start() { return (void *) &end; }
void * phxy_stack_start() { return sbrk(0); }
unsigned char phxy_is_heap(void * data) {
return (phxy_heap_start() <= data && data < phxy_stack_start());
}
unsigned char phxy_is_data(void * data) {
return (data >= phxy_data_start());
}

to find out which data need freeing and which not (i.e. before any
free() I check with phxy_is_heap()). But for my PHP extension I dear not
use this so that phxy_is_heap() always returns false for any data -
hence memory leaks. Please advise me of a safe PHP Zend analogue for
that approach.

Depends on what you want to use that memory for and what you optimize
for. i.e. if you want to stick data in PHP data structures it should be
emalloc'd with one block per item so PHP an free it.

johannes

11 years ago by Kajetan Dolinar — view source — reply

unread

Johaness, thank you for your attention ...

The reason most probably lies within my code, but I don't have enough
PHP Zend engine
hacking experience to get to it by myself - it's my first PHP
extension. Much obliged
for intuitions from experienced for what could bring about such kind
of fault.

Reduce you ode till it works or share more details ..
Yes, that may now be the only possible way to procede.

Depends on what you want to use that memory for and what you optimize
for. i.e. if you want to stick data in PHP data structures it should be
emalloc'd with one block per item so PHP an free it.

I would like to see from the address value itself whether that address
was produced by
e{m|re}alloc() so that I have to efree() it, or by pe{m|re}alloc() that
I have to pefree(),
or whether it falls within some other data segment such as static data
which I should not
free() in any way. For example:

struct record {
int id;
char * name;
. . .
};

. . .

struct record * a = emalloc(sizeof(struct record));
a->id = 1; a->name = estrdup("name1");

struct record * b = emalloc(sizeof(struct record));
b->id = 2; b->name = "name2";

Clearly, there should be efree(a->name) as well as efree(a). However, in
case of b, only
efree(b) is appropriate whereas efree(b->name) would produce an error.
How can I find
out whether an address belongs to efree() or to pefree() or to none of
that?

Kajetan

11 years ago by johannes@schlueters.de — view source — reply

unread

Johaness, thank you for your attention ...

The reason most probably lies within my code, but I don't have enough
PHP Zend engine
hacking experience to get to it by myself - it's my first PHP
extension. Much obliged
for intuitions from experienced for what could bring about such kind
of fault.

Reduce you ode till it works or share more details ..
Yes, that may now be the only possible way to procede.

Well, you could also provide the PHP version number you are using ...
maybe an experienced developer might then point you in the right
direction. I simply have no clue which exact line
zend_object_handlers.c:847 is. So which structure is being read. Maybe
it also helps to use --db-attach=yes for valgrind to inspect the
state ... the only information the given valgrind output gives is that
something weird is happening while trying to call a method on an object.

Best, of course, would be to show code.
(Mind: It is legal to have an open source extension on top a closed
library if you don't want to share some library ... benefit then is that
it might be put in PECL, eventually you get windows builds and bugfixes
from PHP maintainers aside from improved visibility to potential
users ...)

Depends on what you want to use that memory for and what you optimize
for. i.e. if you want to stick data in PHP data structures it should be
emalloc'd with one block per item so PHP an free it.

I would like to see from the address value itself whether that address
was produced by e{m|re}alloc() so that I have to efree() it, or by
pe{m|re}alloc() that I have to pefree(), or whether it falls within
some other data segment such as static data which I should not
free() in any way. For example:

struct record {
int id;
char * name;
. . .
};

. . .

struct record * a = emalloc(sizeof(struct record));
a->id = 1; a->name = estrdup("name1");

struct record * b = emalloc(sizeof(struct record));
b->id = 2; b->name = "name2";

Clearly, there should be efree(a->name) as well as efree(a). However,
in case of b, only efree(b) is appropriate whereas efree(b->name)
would produce an error.
How can I find out whether an address belongs to efree() or to
pefree() or to none of that?

The typical pattern we use is either to add a flag to the structure,
something like

struct record {
int id;
zend_bool need_free; /* zend_bool is a uint */
char * name;
. . .
};

this obviously increases memory usage by sizeof(zend_bool) [+alignment]
for ach element and requires more care when creating/copying these
structures.

Some prefer doing this with

struct record {
int id;
void * to_free; /* equal to name if needs to be free'd, else NULL */
char * name;
. . .
};

The other approach is to enforce ownership inside the structure, thus
requiring more copies but it ensures that ownership is fully clear and
prevents double frees which might occur with your approach if a record
is copied but one doesn't copy the name. This trades some CPU and memory
for safety which in most cases is preferred.

johannes

11 years ago by Kajetan Dolinar — view source — reply

unread

Johannes, all,

Best, of course, would be to show code.
(Mind: It is legal to have an open source extension on top a closed
library if you don't want to share some library ... benefit then is
that
it might be put in PECL, eventually you get windows builds and bugfixes
from PHP maintainers aside from improved visibility to potential
users ...)
I have no reservations over showing my code. I just find it hard to
expect
from anybody that they would actually go through it instead of me. And,
after
all, code reduction is a fair approach.

However, it may be that I have problems already within my PHP extension
file
with declaring or assigning PHP classes and the like. When I get to my
machine
I will gladly disclose that file as well as the PHP version ...

The typical pattern we use is either to add a flag to the structure,
something like

struct record {
int id;
zend_bool need_free; /* zend_bool is a uint */
char * name;
. . .
};
So there is no chance that something like

if (may_be_efreed(address)) efree(address);

would fit within Zend?

Kajetan

11 years ago by Kajetan Dolinar — view source — reply

unread

Hi Johannes

I have succeeded in producing a reduced code for the fault. It should be
fairly simple for anybody to understand it. My particular logic has been
reduced to a mere dummy, so you won't need any deep thoughts on
anything.

But I have a problem: the lists.php.net will reject the attachment. The
attachment is a mere 8KB, that is quite small, so it must be for the
general policy. How can I deliver the code?

Kajetan

11 years ago by Stas Malyshev — view source — reply

unread

Hi!

But I have a problem: the lists.php.net will reject the attachment. The
attachment is a mere 8KB, that is quite small, so it must be for the
general policy. How can I deliver the code?

You could put it on gist.github.com if it's just text files.

--
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227

11 years ago by Kajetan Dolinar — view source — reply

unread

Hi Stas,

You could put it on gist.github.com if it's just text files.

I'll do it this way ... the original mail with the link comming soon ...

Kajetan