# Fwd: How to create Persistent zval?

- Thread: https://externals.io/message/71249
- Original archive: https://news-web.php.net/php.internals/71249
- Started by Daniel Lowrey on 2014-01-18T19:19:34+00:00
- Messages: 8
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #71249 — Daniel Lowrey — 2014-01-18T19:19:34+00:00

URL: https://externals.io/message/71249#71249

> > […]
request,
> > […]
zval
> > […]

> For persistent alloc, you should use pemalloc() , which is just a
> wrapper leading to libc's malloc.
> If you use emalloc() , the Zend Memory Manager will free the storage
> at the end of the request, thus leading to use-after-free crash if you
> reuse your pointer on a next request.

--- Disclaimer ---
I'm passing this along; Joe is having technical difficulties and asked for
help sending this to the list. I provide no warranties or refunds ;)
---

All zvals passed into the engine must be allocated by the mm, you cannot
pemalloc anything and pass it into the engine safely.

You don't want to store a persistent zval, you want to store a persistent
resource entry, which should be allocated with pemalloc(size, 1)

The reason for this is when cleaning up, the engine has no means by which
to tell if a zval has been pemalloc or emalloc'd, zval_ptr_dtor works the
same for everything.

Cheers
Joe

---

## #71250 — Lin Yo-An — 2014-01-18T19:52:47+00:00

URL: https://externals.io/message/71249#71250 · In reply to: #71249

Daniel Lowrey <rdlowrey@php.net> 於 2014年1月19日星期日寫道：

> > […]
> request,
> > […]
> zval
> > […]
>
> > […]
>
> --- Disclaimer ---
> I'm passing this along; Joe is having technical difficulties and asked for
> help sending this to the list. I provide no warranties or refunds ;)
> ---
>
> All zvals passed into the engine must be allocated by the mm, you cannot
> pemalloc anything and pass it into the engine safely.
>
> You don't want to store a persistent zval, you want to store a persistent
> resource entry, which should be allocated with pemalloc(size, 1)
>
> The reason for this is when cleaning up, the engine has no means by which
> to tell if a zval has been pemalloc or emalloc'd, zval_ptr_dtor works the
> same for everything.



Thank you so much! That is what i was guessing..  - ref cnt and pemalloc
does not work for persistent zval.

And zend engine cleans up zvals at the end of request no matter the ref cnt
is?

so zval is not persistent, then is HashTable persistent?

It's because I need to store a zval object which has several properties.
 Is there a way to do make the object persistent? Or do i need to serialize
it just like apc?



> Cheers
> Joe
>


-- 
Best Regards,

Yo-An Lin

---

## #71251 — Lin Yo-An — 2014-01-18T20:01:53+00:00

URL: https://externals.io/message/71249#71251 · In reply to: #71250

By the way, Is this the reason of date ext always check the global
HashTable and re-initialize the tzdata in every request?

Lin Yo-An <cornelius.howl@gmail.com> 於 2014年1月19日星期日寫道：

>
>
> Daniel Lowrey <rdlowrey@php.net <javascript:_e({}, 'cvml',
> 'rdlowrey@php.net');>> 於 2014年1月19日星期日寫道：
>
> > […]
>
>
>
> Thank you so much! That is what i was guessing..  - ref cnt and pemalloc
> does not work for persistent zval.
>
> And zend engine cleans up zvals at the end of request no matter the ref
> cnt is?
>
> so zval is not persistent, then is HashTable persistent?
>
> It's because I need to store a zval object which has several properties.
>  Is there a way to do make the object persistent? Or do i need to serialize
> it just like apc?
>
>
>
> > […]
>
>
> --
> Best Regards,
>
> Yo-An Lin
>
>

-- 
Sent from Gmail Mobile

---

## #71252 — Julien Pauli — 2014-01-18T20:12:11+00:00

URL: https://externals.io/message/71249#71252 · In reply to: #71251

> By the way, Is this the reason of date ext always check the global
> HashTable and re-initialize the tzdata in every request?
>
> Lin Yo-An <cornelius.howl@gmail.com> 於 2014年1月19日星期日寫道：
>
> > […]

Read the shutdown sequence, and you will figure it out by yourself.
Non-persistent pointers are freed at request shutdown.

http://lxr.php.net/xref/PHP_5_5/Zend/zend_alloc.c#1601

Julien

---

## #71265 — Lin Yo-An — 2014-01-19T06:16:11+00:00

URL: https://externals.io/message/71249#71265

Thanks again for the detailed answers and your patience. =]

Just want to make sure the mechanism here:

1. an object can not be persistent even the object is allocated with
pemalloc?
   I actually have tried using pemalloc then init an object few days ago,
and it doesn't work.

2. an zend object can not be allocated persistently because its property
HashTable or handlers is not allocated persistently by default?

3. if so, objects must be serialized to string?

4. what about array? can I make the array persistent with pemalloc
allocation and without serialization?


Thank you so much
Yo-An Lin
http://github.com/c9s




> 
> > […]
> VM.
> > […]
> context.
> > […]
> for
> > […]
> cannot
> > […]
> persistent
> > […]
> which
> > […]
> the
> > […]
> serialize
> > […]
>
> A HashTable can be persistent:
>
>  define zend_hash_init(ht, nSize, pHashFunction, pDestructor, persistent)
>
>  HashTable *ht = pemalloc(sizeof(HashTable), 1);
>
>  zend_hash_init(ht, 8, NULL, NULL, 1);
>
>  When you write the table with zend_hash_update/add/insert the buckets
> it creates are also persistently allocated.
>
>  You don't need to serialize the data, or not necessarily anyway, a
> hashtable can be copied bitwise - which in actual fact is what APC does
> do. APC only serializes objects, normal arrays - HashTables are copied
> bitwise, bucket by bucket.
>
>  Of course, if the persistent table contains references to objects (as
> in PHP objects in object_store or anything else emalloc'd) you will have
> difficulties.
>
>  Probably best to design the tables in such a way that they only need
> ever store persistent data.
>
>  Hope you're getting somewhere :)
>
> Cheers
> Joe
>



-- 
Best Regards,

Yo-An Lin

---

## #71268 — Lester Caine — 2014-01-19T10:09:25+00:00

URL: https://externals.io/message/71249#71268 · In reply to: #71265

Lin Yo-An wrote:
> 3. if so, objects must be serialized to string?
'must' -> 'could' ...

An object consists of a set of software to handle it which is persistent in so
far as it exists in a certain version on the server, and an instance of the data
created for that object ( or an 'array' of them ). Passing that data to later
runs of the code can either be done by producing a 'static' serialization of the
data, or a more flexible approach is store the data in a database. When the
object is recreated it can either be repopulated from the serialization, or from
the data. The first method requires that managing the data is handled directly
by the objects code, while the second allows management from other paths. The
first also introduces restrictions on how modifications can be applied as all
stored serializations would need to be updated, while the second can simply
create additional properties on the fly.

Which method of persistence is used simply depends on other factors, and it's
not uncommon for the serialized data to be stored in a database anyway.

-- 
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

---

## #71270 — Sherif Ramadan — 2014-01-19T10:52:02+00:00

URL: https://externals.io/message/71249#71270 · In reply to: #71265

> Thanks again for the detailed answers and your patience. =]
>
> Just want to make sure the mechanism here:
>
> 1. an object can not be persistent even the object is allocated with
> pemalloc?
>    I actually have tried using pemalloc then init an object few days ago,
> and it doesn't work.
>
> 2. an zend object can not be allocated persistently because its property
> HashTable or handlers is not allocated persistently by default?
>
> 3. if so, objects must be serialized to string?
>
> 4. what about array? can I make the array persistent with pemalloc
> allocation and without serialization?
>
>
> Thank you so much
> Yo-An Lin
> http://github.com/c9s
>
>

Sending this message on behalf of Joe as he's having technical difficulties
sending to the list...


----




1) We have no support for a persistent objects store, whatever you put in
the objects store has a request lifecycle.

2) Correct, a zend object cannot be persistent, as 1).

3) Actual objects, in the sense of zend_objects in the object store
(EG(objects_store)), must be serialized.

4) A HashTable, one that is not used as the properties of an object in
EG(objects_store) can be persistent, and doesn't have to be serialized.

https://github.com/krakjoe/apcu/blob/simplify/apc_cache.c#L1164

Above is the function that copies HashTables to shared memory for APC

https://github.com/krakjoe/apcu/blob/simplify/apc_cache.c#L1309

Above is the callback function passed to my_copy_hashtable as copy_fn, note
that only objects are serialized by this code, nothing else.

Cheers
Joe

---

## #71304 — Stas Malyshev — 2014-01-20T08:00:40+00:00

URL: https://externals.io/message/71249#71304 · In reply to: #71250

Hi!

> And zend engine cleans up zvals at the end of request no matter the ref cnt
> is?

Yes and no. Zend Engine will clean up all emalloc-ed memory. It also
will clean up all symbol tables, objects, resources, etc. that are
stored in system tables. However, if you just create a zval structure
with persistent pemalloc, it would not be cleaned - but you could not
put this zval into any object, structure, etc., and passing it to engine
functions may be dangerous too. If you're knowing what you're doing, you
can pull it off (opcache sometimes does stuff like that IIRC) but you
need to be extremely careful so that the engine does not store your zval
and does not try to free it.

> so zval is not persistent, then is HashTable persistent?
> 
> It's because I need to store a zval object which has several properties.
>  Is there a way to do make the object persistent? Or do i need to serialize
> it just like apc?

Serializing it one way or another (doesn't have to be full serialize -
serialize is pretty complex since it has to account for all corner
cases, you can have something much simpler and use serialize handlers or
custom code) may be the simplest way. It is possible to also keep
objects in memory as is if you're very careful but it is not as simple
to do it right.
-- 
Stanislav Malyshev, Software Architect
SugarCRM: http://www.sugarcrm.com/
(408)454-6900 ext. 227