Hi everyone,
I would like to use this list to address the major players in bytecode
caching tools: Zend, APC, Xcache...
One problem I and others have run into is that from time to time we need
to store extra information for specific opcode arrays. For simple values
it is possible to use one of the reserved slots in the op_array
structure, but in the past that has been unrelieable because APC for
example simply overwrote the first slots without asking the Zend Engine
to reserve some space.
The next problem is that the amount of data you can store is not that big.
Leaving a pointer in the reserved field is also not a good idea, because
this will break as soon the opcode array is shared among processes or
was stored on the disk.
Therefore it would be great if we can come up with a modification of the
op_array structure that allows extensions to append arbitrary sized data
to an op_array, that gets also cached by all the opcode cachers...
What do you think?
Stefan Esser
One problem I and others have run into is that from time to time we need
to store extra information for specific opcode arrays. For simple values
it is possible to use one of the reserved slots in the op_array
structure, but in the past that has been unrelieable because APC for
example simply overwrote the first slots without asking the Zend Engine
to reserve some space.
Have a pointer on how this is done? I sortof need/want to do this in
Xdebug as well, however I don't want opcode caches to remember this
value.
The next problem is that the amount of data you can store is not that big.
Leaving a pointer in the reserved field is also not a good idea, because
this will break as soon the opcode array is shared among processes or
was stored on the disk.Therefore it would be great if we can come up with a modification of the
op_array structure that allows extensions to append arbitrary sized data
to an op_array, that gets also cached by all the opcode cachers...What do you think?
Sounds like a good idea to me - but the op code caches do need to some
special trickery for this I guess. However, in my case I do not want
an opcode cache to remember the stored data, so that perhaps needs to be
taken into account as well.
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
Derick Rethans schrieb:
One problem I and others have run into is that from time to time we need
to store extra information for specific opcode arrays. For simple values
it is possible to use one of the reserved slots in the op_array
structure, but in the past that has been unrelieable because APC for
example simply overwrote the first slots without asking the Zend Engine
to reserve some space.Have a pointer on how this is done? I sortof need/want to do this in
Xdebug as well, however I don't want opcode caches to remember this
value.
Well the "official" way to use the reserved area of an opcode array is
to get an handle from the zend engine for reserved space.
This is done by
ZEND_API int zend_get_resource_handle(zend_extension *extension)
it returns an index you can simply use. (I think meanwhile APC will
honor this and will not overwrite the first bytes).
Sounds like a good idea to me - but the op code caches do need to some
special trickery for this I guess. However, in my case I do not want
an opcode cache to remember the stored data, so that perhaps needs to be
taken into account as well
The basic idea would be to add API functions like
zend_op_array_add_data(key, data, size, flags)
zend_op_array_remove_data()... zend_op_array_get_data,
zend_op_array_get_all_data
In the flags one could choose: Needs to be cached etc...
Well and this will be stored in a datastructure that will just be
pointed at by the op_array struct.
So op_arrays will not really grow (just a single pointer added) and the
whole overhead is only in the
extensions that actually require such extra data and in the opcode
caches that need to cache this.
And I think the opcode cachers would just need to traverse the list of
all data and store it with the op_array. I doubt that would take much
changed in the cacher software. And well if the cache authors speak up,
PHP can pretty much implement all the API functions required for such a
step...
Stefan Esser
Derick Rethans schrieb:
One problem I and others have run into is that from time to time we need
to store extra information for specific opcode arrays. For simple values
it is possible to use one of the reserved slots in the op_array
structure, but in the past that has been unrelieable because APC for
example simply overwrote the first slots without asking the Zend Engine
to reserve some space.Have a pointer on how this is done? I sortof need/want to do this in
Xdebug as well, however I don't want opcode caches to remember this
value.Well the "official" way to use the reserved area of an opcode array is
to get an handle from the zend engine for reserved space.
This is done by
ZEND_API int zend_get_resource_handle(zend_extension *extension)
it returns an index you can simply use. (I think meanwhile APC will
honor this and will not overwrite the first bytes).
Yes, it does:
void apc_zend_init(TSRMLS_D)
{
zend_extension dummy_ext;
#ifdef ZEND_ENGINE_2
APCG(reserved_offset) = zend_get_resource_handle(&dummy_ext);
assert(APCG(reserved_offset) == dummy_ext.resource_number);
assert(APCG(reserved_offset) != -1);
assert(sizeof(apc_opflags_t) <= sizeof(void*));
#endif
For Xdebug I don't need to use the dummy ext though as it's a real zend
extension.
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
The basic idea would be to add API functions like
zend_op_array_add_data(key, data, size, flags)
zend_op_array_remove_data()... zend_op_array_get_data,
zend_op_array_get_all_data
In the flags one could choose: Needs to be cached etc...Well and this will be stored in a datastructure that will just be
pointed at by the op_array struct.
Some questions here:
- Do I understand right that that this is supposed to be arbitrary
length list of (void *, size) pairs? - If opcode cache gets just size, that means the data structure would
have to be in some kind of serialized position-independent form - which
most of C structures that use any kind of pointers aren't. - What happens with this if there's no bytecode cache? How the data is
supposed to be retrieved from the cache? - How modifications are handled - i.e., if extension modifies the data,
bytecode cache is supposed to be notified of the change, or the data is
supposed to be immutable once created and cached?
--
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
One problem I and others have run into is that from time to time we need
to store extra information for specific opcode arrays. For simple values
it is possible to use one of the reserved slots in the op_array
structure, but in the past that has been unrelieable because APC for
example simply overwrote the first slots without asking the Zend Engine
to reserve some space.
I think it's a bug in APC. Each extension has its own slot, by allocated
slot number, and should never touch other slots (unless it has some
protocol to talk to the extension which owns other slot). So touching
slot that is not owned by the code is a bug like touching unallocated
memory.
The next problem is that the amount of data you can store is not that big.
Leaving a pointer in the reserved field is also not a good idea, because
this will break as soon the opcode array is shared among processes or
was stored on the disk.
Well, we can't have pointers to disk in memory :), so disk is another
problem. As for shared memory, I don't see how one can support arbitrary
length and structure data without using pointers in one way or another,
so once you share that you'd have to use shared memory pointers anyway.
Maybe I misunderstand something in what do you mean - could you tell
some more on what would be the improvement you are thinking of?
Therefore it would be great if we can come up with a modification of the
op_array structure that allows extensions to append arbitrary sized data
to an op_array, that gets also cached by all the opcode cachers...
Note that reserved space in op_array is meant for general extension
usage, not just for bytecode caches. So at least some of that data
doesn't have to be cached. Now, we could add some other data that would
be cached, but here we need to understand how we can do arbitrary length
and arbitrary structure data that can be stored in shared memory and
concurrently used. It looks to me rather complex task, not sure it
belongs to the engine even - unless I'm missing some obvious easy solution.
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com