Recurring core dump on y2

Minor update. The latest core is the first I have seen not on the mysql
page. Still in highlight_string(), but this time it is on the first
user note at:

http://www.php.net/manual/pt_BR/function.register-shutdown-function.php

$4 = {value = {lval = 9162776, dval = 4.5270128421386736e-317, str = {
val = 0x8bd018 "I have discovered a change in behavior from PHP
5.0.4 to PHP 5.1.2 when using a shutdown function in conjunction with an
output buffering callback.\n\nIn PHP 5.0.4 (and earlier versions I
believe) the s"..., len = 732}, ht = 0x8bd018, obj = {handle = 9162776,
handlers = 0x2dc}}, refcount = 3, type = 6 '\006', is_ref = 0 '\0'}

-Rasmus

Rasmus Lerdorf wrote:

I am out of ideas on this one. I started chasing a problem yesterday
where we were crashing repeatedly on y2. I removed APC and a couple of
other things to rule those out. The crash is consistently in exactly
the same place. It is always on a request to:

http://www.php.net/manual/en/ref.mysql.php

in the highlight_string() call from layout.inc:

function highlight_php($code, $return = FALSE)
{
// Using OB, as highlight_string() only supports
// returning the result from 4.2.0
ob_start();
highlight_string($code);
$highlighted = ob_get_contents();
ob_end_clean();

and it is always on the second note on that page. I yanked the raw note
data out of the backend files and tried to reproduce it with:

http://www.php.net/~rasmus/note.php

note.phps for source

But no luck. Somehow CG(op_array) is getting set to crap coming into
that call, but only sometimes. Yet if it is random memory corruption
like that, why is the crash always exactly the same? It looks like this:

(gdb) bt
#0 0x0000000800b59953 in strlen () from /lib/libc.so.6
#1 0x000000080109dac6 in lex_scan (zendlval=0x7fffffff8d20) at
zend_language_scanner.l:1301
#2 0x00000008010b1214 in zend_highlight
(syntax_highlighter_ini=0x7fffffff8e10) at
/home/rasmus/php51/Zend/zend_highlight.c:178
#3 0x00000008010a03b4 in highlight_string (str=0x7fffffff8d80,
syntax_highlighter_ini=0x7fffffff8e10,
str_name=0x981d18
"/home/local/Web/sites/www.php.net/include/layout.inc(21) : highlighted
code") at zend_language_scanner.l:621
#4 0x00000008010072ac in zif_highlight_string (ht=40,
return_value=0x986c58, return_value_ptr=0xc, this_ptr=0x978340,
return_value_used=19651040)
at /home/rasmus/php51/ext/standard/basic_functions.c:2537
#5 0x00000008010d9407 in zend_do_fcall_common_helper_SPEC
(execute_data=0x7fffffff91e0) at zend_vm_execute.h:200
#6 0x00000008010d8d21 in execute (op_array=0x893f00) at
zend_vm_execute.h:92
#7 0x00000008010d8ff5 in zend_do_fcall_common_helper_SPEC
(execute_data=0x7fffffff9440) at zend_vm_execute.h:234
#8 0x00000008010d8d21 in execute (op_array=0x8f5400) at
zend_vm_execute.h:92
#9 0x00000008010d8ff5 in zend_do_fcall_common_helper_SPEC
(execute_data=0x7fffffff9b80) at zend_vm_execute.h:234
#10 0x00000008010d8d21 in execute (op_array=0x842300) at
zend_vm_execute.h:92
#11 0x00000008010d8ff5 in zend_do_fcall_common_helper_SPEC
(execute_data=0x7fffffffa370) at zend_vm_execute.h:234
#12 0x00000008010d8d21 in execute (op_array=0x842000) at
zend_vm_execute.h:92
#13 0x00000008010d8ff5 in zend_do_fcall_common_helper_SPEC
(execute_data=0x7fffffffa7a0) at zend_vm_execute.h:234
#14 0x00000008010d8d21 in execute (op_array=0x816b00) at
zend_vm_execute.h:92
#15 0x00000008010d8ff5 in zend_do_fcall_common_helper_SPEC
(execute_data=0x7fffffffc750) at zend_vm_execute.h:234
#16 0x00000008010d8d21 in execute (op_array=0x7e8a18) at
zend_vm_execute.h:92
#17 0x00000008010bc4a9 in zend_execute_scripts (type=8, retval=0x0,
file_count=3) at /home/rasmus/php51/Zend/zend.c:1109
#18 0x000000080107d17d in php_execute_script
(primary_file=0x7fffffffdfd0) at /home/rasmus/php51/main/main.c:1726
#19 0x0000000801130d4f in apache_php_module_main (r=0x28,
display_source_mode=0) at /home/rasmus/php51/sapi/apache/sapi_apache.c:53
#20 0x0000000801131995 in send_php (r=0x7bf060, display_source_mode=0,
filename=0x0) at /home/rasmus/php51/sapi/apache/mod_php5.c:661
#21 0x0000000000427cec in ap_invoke_handler ()
#22 0x0000000000439a6c in process_request_internal ()
#23 0x0000000000439bad in ap_process_request ()
#24 0x000000000043132f in child_main ()
#25 0x00000000004316b9 in make_child ()
#26 0x000000000043295e in standalone_main ()
#27 0x0000000000433453 in main ()

(gdb) up
#1 0x000000080109dac6 in lex_scan (zendlval=0x7fffffff8d20) at
zend_language_scanner.l:1301
1301 zendlval->value.str.len = strlen(func_name);
(gdb) p func_name
$3 = 0x28 <Address 0x28 out of bounds>

(gdb) frame 2
#2 0x00000008010b1214 in zend_highlight
(syntax_highlighter_ini=0x7fffffff8e10) at
/home/rasmus/php51/Zend/zend_highlight.c:178
178 token.type = 0;
(gdb) l
173 break;
174 }
175 } else if (token_type == T_END_HEREDOC) {
176 efree(token.value.str.val);
177 }
178 token.type = 0;
179 }
180 done:
181 if (last_color != syntax_highlighter_ini->highlight_html) {
182 zend_printf("</span>\n");
(gdb) p token
$8 = {value = {lval = 12225272, dval = 6.0400869062649078e-317, str =
{val = 0xba8af8 "::", len = 2}, ht = 0xba8af8, obj = {handle = 12225272,
handlers = 0x2}}, refcount = 4294937984, type = 0 '\0', is_ref =
127 '\177'}

(gdb) frame 3
#3 0x00000008010a03b4 in highlight_string (str=0x7fffffff8d80,
syntax_highlighter_ini=0x7fffffff8e10,
str_name=0x981d18
"/home/local/Web/sites/www.php.net/include/layout.inc(21) : highlighted
code") at zend_language_scanner.l:621
621 zend_highlight(syntax_highlighter_ini TSRMLS_CC);
(gdb) p *str
$5 = {value = {lval = 9998360, dval = 4.9398461907532858e-317, str = {
val = 0x989018 "This is a small function I wrote to handle queries
on a table.\nIt can query a table, order and sort, and supports inner
joins.\n\nThis function also returns the result as a single row or all
rows.\n\nEnjo"..., len = 3374}, ht = 0x989018, obj = {handle = 9998360,
handlers = 0xd2e}}, refcount = 3, type = 6 '\006', is_ref = 0 '\0'}

(gdb) frame 4
#4 0x00000008010072ac in zif_highlight_string (ht=40,
return_value=0x986c58, return_value_ptr=0xc, this_ptr=0x978340,
return_value_used=19651040)
at /home/rasmus/php51/ext/standard/basic_functions.c:2537
2537 if (highlight_string(expr, &syntax_highlighter_ini,
hicompiled_string_description TSRMLS_CC) == FAILURE) {
(gdb) p *expr
$2 = {value = {lval = 9994264, dval = 4.9378224978679201e-317, str = {
val = 0x988018 "This is a small function I wrote to handle queries
on a table.\nIt can query a table, order and sort, and supports inner
joins.\n\nThis function also returns the result as a single row or all
rows.\n\nEnjo"..., len = 3374}, ht = 0x988018, obj = {handle = 9994264,
handlers = 0xd2e}}, refcount = 3, type = 6 '\006', is_ref = 0 '\0'}

I have also tried to reproduce this running under valgrind, but no luck
there either. So it could be 64-bit specific, or somehow I didn't
recreate the exact same environment for it. Since it is the second user
note with code to highlight on that page that always crashes, it could
be some sort of re-entrancy problem as well. Or perhaps it is totally
unrelated, although the fact that it is always the exact same crash is
extremely suspicious.

If you have an account on y2, do:

gdb /local/httpd/bin/httpd /local/httpd/httpd.core

This is running PHP_5_1 as of yesterday.

-Rasmus