# serialize and cache handling

- Thread: https://externals.io/message/28944
- Original archive: https://news-web.php.net/php.internals/28944
- Started by Mathias Bank on 2007-04-21T07:30:36+00:00
- Messages: 14
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #28944 — Mathias Bank — 2007-04-21T07:30:36+00:00

URL: https://externals.io/message/28944#28944

Hallo,

I use pear cache (light) to save calculated values. The package uses 
serialize to transform the content of a variable into a writable form. 
This transformed value will be saved to disk.

This could be improved! Would it be possible to rewrite the serialize 
function to make it possible to write directly to disk? If you have a 
value, which needs 10MB you need also ca. 10MB to serialize and than you 
can write this data to disk. So, you needs 20MB. If serialize (and of 
course unserialize) would be able to write directly to disk (or read 
directly from disk), you only needs 10MB.

I think, that could perform much better for every file cache than 
current implementation does. I have such a scenario, in which a variable 
needs 100MB. Because the calculation of this value needs much time, I 
wanted to save this value to cache (with the help of (un)serialize). But 
than, I need 200MB and that doesn't look nice, because it is not really 
necessary.

As I understand the serialize structure, it would be no problem to 
extend it to directly save to disk. But my c knowledge is not as good as 
it should be. I would be very happy, if this idea could be realized.

Mathias

---

## #29296 — Stanislav Malyshev — 2007-05-07T21:53:20+00:00

URL: https://externals.io/message/28944#29296 · In reply to: #28944

> can write this data to disk. So, you needs 20MB. If serialize (and of 
> course unserialize) would be able to write directly to disk (or read 
> directly from disk), you only needs 10MB.

Actually having serialize/unserialize be able to write directly to a 
stream and read directly from a stream might be interesting, would 
probably improve working with things like large sessions or caching 
large data substantially.

-- 
Stanislav Malyshev, Zend Products Engineer
stas@zend.com  http://www.zend.com/

---

## #29299 — Lukas Kahwe Smith — 2007-05-07T22:24:01+00:00

URL: https://externals.io/message/28944#29299 · In reply to: #29296

Stanislav Malyshev wrote:
> > […]
> 
> Actually having serialize/unserialize be able to write directly to a 
> stream and read directly from a stream might be interesting, would 
> probably improve working with things like large sessions or caching 
> large data substantially.

Indeed, especially since this is the most common use case. Maybe it 
should optionally also return an md5 of the written data.

regards,
Lukas

---

## #29303 — Derick Rethans — 2007-05-08T06:17:28+00:00

URL: https://externals.io/message/28944#29303 · In reply to: #29299

> Stanislav Malyshev wrote:
> > […]
> 
> Indeed, especially since this is the most common use case. Maybe it should
> optionally also return an md5 of the written data.

If we're to add this, make sure writes to the files are atomic.

regards,
Derick

-- 
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

---

## #29329 — Stanislav Malyshev — 2007-05-08T12:39:37+00:00

URL: https://externals.io/message/28944#29329 · In reply to: #29303

> If we're to add this, make sure writes to the files are atomic.

Does PHP now ensure fwrite is atomic? If it doesn't than writing on 
serialize doesn't change a thing.
-- 
Stanislav Malyshev, Zend Products Engineer
stas@zend.com  http://www.zend.com/

---

## #29330 — Derick Rethans — 2007-05-08T12:48:21+00:00

URL: https://externals.io/message/28944#29330 · In reply to: #29329

> > […]
> 
> Does PHP now ensure fwrite is atomic? If it doesn't than writing on serialize
> doesn't change a thing.

Only "a" mode is atomic - per write call; normal fwrites are not. 
However, you'd need to write the *whole* file to disk atomically and not 
on every fwrite. And you can not first cache it in memory as you then 
lose the whole advantage of this idea.

regards,
Derick

---

## #29331 — Stanislav Malyshev — 2007-05-08T13:01:18+00:00

URL: https://externals.io/message/28944#29331 · In reply to: #29330

> on every fwrite. And you can not first cache it in memory as you then 
> lose the whole advantage of this idea.

IIRC sessions are locked by php anyway, and for other uses if locking is 
important it is already implemented anyway, so we shouldn't really try 
to solve all the world's problems with this one.
-- 
Stanislav Malyshev, Zend Products Engineer
stas@zend.com  http://www.zend.com/

---

## #29332 — Derick Rethans — 2007-05-08T13:08:51+00:00

URL: https://externals.io/message/28944#29332 · In reply to: #29331

> > […]
> 
> IIRC sessions are locked by php anyway, and for other uses if locking is
> important it is already implemented anyway, so we shouldn't really try to
> solve all the world's problems with this one.

? Nobody is talking about sessions here, just about the `serialize()` 
function that is also used for a myriad of other things...

regards,
Derick

---

## #29333 — Stanislav Malyshev — 2007-05-08T13:16:09+00:00

URL: https://externals.io/message/28944#29333 · In reply to: #29332

> ? Nobody is talking about sessions here, just about the `serialize()` 

You mean you are not talking about sessions. I, however, do. Sessions 
are one of the obvious examples where such functionality could improve 
performance.

> function that is also used for a myriad of other things...

Oh really? I guess that's why we talk about _adding_ stuff to it in 
order to make it more efficient *in certain scenarios* and not replacing 
it with the new one. Precisely the scenarios where external locking 
would be happening anyway - when either using sessions or caching.

-- 
Stanislav Malyshev, Zend Products Engineer
stas@zend.com  http://www.zend.com/

---

## #29335 — Lukas Kahwe Smith — 2007-05-08T13:29:11+00:00

URL: https://externals.io/message/28944#29335 · In reply to: #29333

Stanislav Malyshev wrote:
> > […]
> 
> You mean you are not talking about sessions. I, however, do. Sessions 
> are one of the obvious examples where such functionality could improve 
> performance.

Well as the topic implies I am quite sure that the user request was 
about caching into a custom file and not inside the session. Both are 
frequent use cases.

regards,
Lukas

---

## #29338 — Stanislav Malyshev — 2007-05-08T15:25:21+00:00

URL: https://externals.io/message/28944#29338 · In reply to: #29335

> Well as the topic implies I am quite sure that the user request was 
> about caching into a custom file and not inside the session. Both are 
> frequent use cases.

If his cache had no locking before, what changed?
-- 
Stanislav Malyshev, Zend Products Engineer
stas@zend.com  http://www.zend.com/

---

## #29344 — Richard Lynch — 2007-05-08T17:42:44+00:00

URL: https://externals.io/message/28944#29344 · In reply to: #29303

> 
>
> > […]
>
> If we're to add this, make sure writes to the files are atomic.

Is this suggesting that the entire 80M upload has to be done in a
single operation?...

Or is the md5/sha1 computed chunk by chunk, in parallel, with writing
buffered data to the disk?

Cuz if it's the former, I don't see that working out too well for
ginormous uploaded files...

Which people probably shouldn't be doing over HTTP anyway, but they
do, and that's the reality one has to deal with...

Apologies if I'm being alarmist and totally mis-reading this through
my ignorance.

-- 
Some people have a "gift" link here.
Know what I want?
I want you to buy a CD from some indie artist.
http://cdbaby.com/browse/from/lynch
Yeah, I get a buck. So?

---

## #29346 — Derick Rethans — 2007-05-08T17:48:09+00:00

URL: https://externals.io/message/28944#29346 · In reply to: #29344

> 
> > […]
> 
> > […]

Wrong thread ;-) This is on serialize, not on hashes.

regards,
Derick

-- 
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org

---

## #29383 — Mathias Bank — 2007-05-09T17:47:20+00:00

URL: https://externals.io/message/28944#29383 · In reply to: #29338

> If his cache had no locking before, what changed?

Well, I have been using several cache classes. A good cache class is the
pear cache light.

This cache is serializing your data and write this data to a file - of
course with file locking.

I could imagine, that a improved serialize-function could work like this:

$fp = @fopen($filename, "wb");
if ($fp) {
  @flock($fp, LOCK_EX);
  serialize($data,$fp);
  @flock($fp, LOCK_UN);
  @fclose($fp);
}


This method makes it possible to decide, if you want to use a locking
function or not.

The same technique could be applied to imagepng (and others):
instead of writing

  imagepng($img,"/path/to/file");

it would be better to write

$fp = @fopen($filename, "wb");
if ($fp) {
  @flock($fp, LOCK_EX);
  imagepng($img,$fp);
  @flock($fp, LOCK_UN);
  @fclose($fp);
}

This way, you can decide, if you want to use locking functions or not. I
think, this could be a big improvement (until now you have to use
imagepng in combination with ob_start / ob_get_contents / ob_end_clean
to cache images in a secure way).

Mathias