# New functions `hash_serialize` and `hash_unserialize`? - Thread: https://externals.io/message/110422 - Original archive: https://news-web.php.net/php.internals/110422 - Started by Eddie Kohler on 2020-06-08T13:01:52+00:00 - Messages: 6 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #110422 — Eddie Kohler — 2020-06-08T13:01:52+00:00 URL: https://externals.io/message/110422#110422 Hello internals! Thanks for PHP! I'm writing to gauge interest in two new functions to the PHP `hash` extension, `hash_serialize` and `hash_unserialize`. These functions would serialize and unserialize the internals of a HashContext object, allowing a partially-computed hash to be saved, then restored and completed in a later run. EXAMPLE: Multi-part upload. Say that a very large file is uploaded in pieces, `big.001` through `big.999`, and it is necessary to compute the SHA256 of the final concatenated file. Current PHP must compute the hash in one go: $ctx = hash_init("sha256"); for ($i = 1; $i <= 999; ++$i) { hash_update_file($ctx, sprintf("big.%.03d", $i)); } $hash = hash_final($ctx); This in turn requires that all pieces be on the filesystem simultaneously. With hash_serialize and hash_unserialize, the hash can be computed gradually, allowing pieces to be deleted as they are uploaded elsewhere. $ctx = hash_init("sha256"); hash_update_file($ctx, "big.001"); SAVE_TO_DATABASE(hash_serialize($ctx)); ... $ctx = hash_unserialize(LOAD_FROM_DATABASE()); hash_update_file($ctx, "big.002"); SAVE_TO_DATABASE(hash_serialize($ctx)); ... etc. *** I am happy to write up an RFC for these functions. An initial implementation with tests is visible here: https://github.com/kohler/php-src/commit/5a3a828f90b88cd7f660babec7db531cfc04b0a1 New functions `hash_serialize` and `hash_unserialize` appear to fit the existing API well, and simplify implementation, but it's possible that `__serialize/__unserialize` or the internal `serialize/unserialize` functions would be preferred. I'd be grateful for any feedback. Thanks! Eddie Kohler --- ## #110423 — johannes@schlueters.de — 2020-06-08T13:28:43+00:00 URL: https://externals.io/message/110422#110423 · In reply to: #110422 > I'm writing to gauge interest in two new functions to the PHP `hash` > extension, `hash_serialize` and `hash_unserialize`. These functions > would serialize and unserialize the internals of a HashContext > objectallowing a partially-computed hash to be saved, then restored > and completed in a laterrun. I would suggest to make the HashContext Serializable, then serialize($hash_context); works. Then it also fits when stored in other objects or something ... johannes --- ## #110487 — Eddie Kohler — 2020-06-11T16:58:49+00:00 URL: https://externals.io/message/110422#110487 · In reply to: #110423 Thanks for this suggestion. I've updated the implementation to make HashContext implement Serializable. I'd still be grateful for more feedback, or perhaps I should just create an RFC? Eddie On Mon, Jun 8, 2020 at 9:28 AM Johannes Schlüter wrote: > > > […] > > I would suggest to make the HashContext Serializable, then > > serialize($hash_context); > > works. Then it also fits when stored in other objects or something ... > > johannes --- ## #110491 — Christoph M. Becker — 2020-06-11T21:07:52+00:00 URL: https://externals.io/message/110422#110491 · In reply to: #110487 > Thanks for this suggestion. I've updated the implementation to make > HashContext implement Serializable. > > I'd still be grateful for more feedback, or perhaps I should just create an > RFC? Not sure if that would need an RFC; maybe just start by submitting a pull request. :) Thanks, Christoph > Eddie > > > On Mon, Jun 8, 2020 at 9:28 AM Johannes Schlüter > wrote: > > > […] --- ## #110492 — Sara Golemon — 2020-06-11T21:41:51+00:00 URL: https://externals.io/message/110422#110492 · In reply to: #110487 On Thu, Jun 11, 2020 at 11:59 AM Eddie Kohler wrote: > Thanks for this suggestion. I've updated the implementation to make > HashContext implement Serializable. > > I'd still be grateful for more feedback, or perhaps I should just create > an RFC? > > Be careful what you ask for. :) Overall +1 on the concept with a few notes: 1. Please put this on a branch and make it a PR so we can comment on it directly. 2. Consider using zend_parse_parameters_throws() and family so that the exception which is thrown contains the type error information rather than the generic RETURN_THROWS() macros. 3. Consider using hex or base64 to serialize the contexts. This will reduce various transport/storage issues. 4. It's great that you've thought about endianness, but the current implementation simply bails on endian mismatch. It'd be a nice-to-have for the user if these serializations were portable. I know this represents a lot of work for sort of an edge case so I won't hold it against you if you say 'no' and/or save this for later work if demand surfaces. 5. Storing $key makes me nervous. I don't have a good solution to this since the deserialization doesn't actually give us a chance to specify it in the deserialization process. I wish I'd made $key/hmac an option to hash_final rather than hash_init. Maybe we can think about allowing that to be specified at either end. Let's expand on this topic while you work on your RFC. 6. Yeah... I think you need an RFC because of #5. Sorry. 7. TABS v SPACES indentation issues. -Sara --- ## #110691 — Eddie Kohler — 2020-06-21T13:12:14+00:00 URL: https://externals.io/message/110422#110691 · In reply to: #110492 Hi all, I've opened up a pull request and responded to this message there. I'd love any further comments. https://github.com/php/php-src/pull/5702 Eddie > WARNING: Harvard cannot validate this message was sent from an authorized > system. Please be careful when opening attachments, clicking links, or > following instructions. For more information, visit the HUIT IT Portal and > search for SPF. > ------------------------------ > On Thu, Jun 11, 2020 at 11:59 AM Eddie Kohler > wrote: > > > […] > > Overall +1 on the concept with a few notes: > > 1. Please put this on a branch and make it a PR so we can comment on it > directly. > 2. Consider using zend_parse_parameters_throws() and family so that the > exception which is thrown contains the type error information rather than > the generic RETURN_THROWS() macros. > 3. Consider using hex or base64 to serialize the contexts. This will > reduce various transport/storage issues. > 4. It's great that you've thought about endianness, but the current > implementation simply bails on endian mismatch. It'd be a nice-to-have for > the user if these serializations were portable. I know this represents a > lot of work for sort of an edge case so I won't hold it against you if you > say 'no' and/or save this for later work if demand surfaces. > 5. Storing $key makes me nervous. I don't have a good solution to this > since the deserialization doesn't actually give us a chance to specify it > in the deserialization process. I wish I'd made $key/hmac an option to > hash_final rather than hash_init. Maybe we can think about allowing that > to be specified at either end. Let's expand on this topic while you work > on your RFC. > 6. Yeah... I think you need an RFC because of #5. Sorry. > 7. TABS v SPACES indentation issues. > > -Sara