Performance of buffer based functionality (JSON, AES, serialize())

I think there is much to gain by improving the serialization speed in
PHP. It is used everywhere from caches like memcache, to sessions or
manual data input into DB. I would say that there are very few
non-trivial apps that would not benefit from a more compact and faster
serializer.

In our specific work-use-case switching to igbinary improved the speed
of the overall page generation by 2-3%.

Hi,

Completely different topic :)

I've been looking a bit into performance around json encoding, hashing+encryption (aes) and serialize()/unserialize(). Data that is marshaled and often transmitted over the wire.

I know there have been some high-end apps that have benefited from some custom serializers, etc... (typically platform dependent).
I wonder if people here think improvements in these areas would move the needle for the majority of mainstream apps or not.

Thanks,

Andi

Hi,

I think it would, a lot of sites/apps nowadays rely a lot on JSON encoding/decoding, plus a lot of technologies are relying on serialization/json encoding (Memcached, Redis, to name a few) at the PHP level, which can be a really big performance eater if you use it a lot.
On the other hand, it's not too difficult to get IGBinary setup as serializer instead of the default one, and it does some pretty good improvement.
Ever considered making it a part of PHP? What would be the implication (as I'm not too familiar with all the internals of PHP besides extensions...)?

Michel Bartz

Lead Developer
Manwin Canada
Skype: michel.php

-----Original Message-----
From: Andi Gutmans [mailto:andi@zend.com]
Sent: Thursday, November 25, 2010 12:47 PM
To: internals@lists.php.net
Subject: [PHP-DEV] Performance of buffer based functionality (JSON, AES, serialize())

Hi,

Completely different topic :)

I've been looking a bit into performance around json encoding, hashing+encryption (aes) and serialize()/unserialize(). Data that is marshaled and often transmitted over the wire.

I know there have been some high-end apps that have benefited from some custom serializers, etc... (typically platform dependent).
I wonder if people here think improvements in these areas would move the needle for the majority of mainstream apps or not.

Thanks,

Andi

This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately.

Ce courrier ?lectronique est confidentiel et prot?g?. L'exp?diteur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) d?sign?(s) est interdite. Si vous recevez ce courrier ?lectronique par erreur, veuillez m'en aviser imm?diatement, par retour de courrier ?lectronique ou par un autre moyen.

hi,

For the record here, igbinary is a very good example of such optimization:

http://opensource.dynamoid.com/

Hi,

Completely different topic :)

I've been looking a bit into performance around json encoding, hashing+encryption (aes) and serialize()/unserialize(). Data that is marshaled and often transmitted over the wire.

I know there have been some high-end apps that have benefited from some custom serializers, etc... (typically platform dependent).
I wonder if people here think improvements in these areas would move the needle for the majority of mainstream apps or not.

Thanks,

Andi

--
Pierre

@pierrejoye | http://blog.thepimp.net | http://www.libgd.org

I know there have been some high-end apps that have benefited from
some custom serializers, etc... (typically platform dependent).
I wonder if people here think improvements in these areas would move
the needle for the majority of mainstream apps or not.

Like people have mentioned, improving (un)serialize speed would be a huge
benefit, especially for caching data sets or large objects.

From experience, it would seem valuable to have:

1. serialize_text($var)

The existing serialize() minus the NULL bytes on private properties. It has
been a source problems for developers serializing an object with private
properties and storing it in a database (the string may get cutoff).

I'm not sure why there's a NULL byte in 'zend_mangle_property_name', instead
the char "" could be used to mark a private property in the serialized
text.
The unserialize could be BC compatible accepting both NULL and "" around a
private property.

2. serialize_binary($var)

An efficient and compact serialization using techniques from igbinary.

A potential problem with igbinary I've noticed is it packs a double as a 64
bit integer.
That could be a problem if you serialize on a platform that has an IEEE 754
binary representation and unserialize on a non-IEEE platform but I don't
know if php compiles on architectures that are non-IEEE.

It could also be interesting to pack integers as varints:
http://code.google.com/apis/protocolbuffers/docs/encoding.html#varints
http://protobuf-c.googlecode.com/svn/trunk/src/google/protobuf-c/protobuf-c.
c

That's most likely slower though then what igbinary does with integers