Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:86465
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain gmail.com designates 209.85.192.175 as permitted sender)
Message-ID: <556BC5A5.8060705@gmail.com>
Date: Sun, 31 May 2015 19:38:29 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
MIME-Version: 1.0
To: PHP Internals <internals@lists.php.net>, 
 Dmitry Stogov <dmitry@zend.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Subject: Big hashes problem in master
From: smalyshev@gmail.com (Stanislav Malyshev)

Hi!

Looking into bug #69295	I've noticed the following problem with hashes
on master: if we try to create very large hashtable, on 64-bit build
(e.g. one with 1126626666 elements) then _zend_hash_init would set the
following:

ht->nTableSize = 0x80000000
ht->nTableMask = 0xfffffffe
(I'm using hex so that the numbers would be more clear). Then when we
come to zend_hash_real_init_ex, we do allocation of HT_SIZE(ht). But
then after this line:

 HT_SET_DATA_ADDR(ht, pemalloc(HT_SIZE(ht), (ht)->u.flags &
HASH_FLAG_PERSISTENT));

I see this:
(gdb) p ht->arData
$37 = (Bucket *) 0xffffffff0a000000

Looks like what happened is that whet HT_HASH_SIZE was calculated,
(-(int32_t)(ht)->nTableMask) was calculated as 0x80000000 as signed it,
and then promoted to size_t to multiply it with sizeof, which produces
0xffffffff80000000. Of course, adding this to a pointer wouldn't do much
good. So the next line, 		
HT_HASH_RESET(ht);
crashes because the pointer arData is broken. I think this:

#define HT_HASH_SIZE(ht) \
	((uint32_t)(-(int32_t)(ht)->nTableMask) * sizeof(uint32_t))

Should fix it but would like second pair of eyes on this.

There's also another problem in the code in zend_hash_real_init_ex - the
initialized flag is set before the address is actually set, so if the
allocation fails, the dtor may get broken array marked as initialized.
This however seems to be easily fixed if flags are moved to be after
allocation.
-- 
Stas Malyshev
smalyshev@gmail.com