# Support for writing and executing compiled opcode - Thread: https://externals.io/message/89208 - Original archive: https://news-web.php.net/php.internals/89208 - Started by Stephen Coakley on 2015-11-13T21:35:52+00:00 - Messages: 13 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #89208 — Stephen Coakley — 2015-11-13T21:35:52+00:00 URL: https://externals.io/message/89208#89208 Hello all, I've been thinking about PHP optimization and distribution, and I would like to hear some opinions on introducing a new feature for a future PHP 7 version. Here's the concept: allow PHP opcode to be both saved after parsing a file, and to be loaded and executed, similar to the bcompiler extension and the APC bin functions back in the day. The advantages are clear: libraries and applications could be compiled to opcode ahead of time so PHP wouldn't have to compile it again and again (assuming you're not already using OPcache). A new function could be provided to parse a PHP file, but instead of executing it, the compiled opcode would be saved to a file. It might part of the OPcache extension if it makes sense, and could be called opcache_compile_to_file() or something. Another option would be to add a command-line flag to the interpreter to write compiled opcode instead of executing it after parsing. Another part of the feature would be to enable the interpreter to execute compiled opcode scripts directly. This would work for both compiled scripts passed to the interpreter, and scripts loaded with `include` and `require`. We would probably need to introduce a new file extension to specify opcode files. I'd recommend *.phpo or *.phpc. This is quite similar to Python's ability to execute Python scripts compiled to bytecode as *.pyc files. The feature has seen great success in Python, mostly for distributing releases of software or deploying to a server. I'm not at this moment planning an RFC, but I'd like to gauge your opinions and reactions first. -- Stephen --- ## #89209 — Sebastian Bergmann — 2015-11-13T21:45:03+00:00 URL: https://externals.io/message/89208#89209 · In reply to: #89208 > This is quite similar to Python's ability to execute Python scripts > compiled to bytecode as *.pyc files. The feature has seen great success in > Python, mostly for distributing releases of software or deploying to a > server. Correct me if I'm wrong, but this should already be possible with OpCache and its filesystem backend in PHP 7.0. See http://talks.php.net/froscon15#/php7pcache1 and following for details. --- ## #89210 — Stephen Coakley — 2015-11-13T22:00:12+00:00 URL: https://externals.io/message/89208#89210 · In reply to: #89209 > > > […] > > Correct me if I'm wrong, but this should already be possible with OpCache > and its filesystem backend in PHP 7.0. > > See http://talks.php.net/froscon15#/php7pcache1 and following for > details. That's great! That's about halfway toward what I'm looking for. That means that the engine is likely already capable of doing these things -- the next step is to be able to execute any given .php.bin file like in that talk. The idea would be to be able to bypass the caching mentality by executing an already compiled file, instead of checking if the original .php file has a corresponding bin file in the cache -- Stephen --- ## #89211 — Rasmus Lerdorf — 2015-11-13T23:46:15+00:00 URL: https://externals.io/message/89208#89211 · In reply to: #89210 > > > […] > > That's great! That's about halfway toward what I'm looking for. That means that the engine is likely already capable of doing these things -- the next step is to be able to execute any given .php.bin file like in that talk. The idea would be to be able to bypass the caching mentality by executing an already compiled file, instead of checking if the original .php file has a corresponding bin file in the cache You could simply deploy both the .php and the .bin files to achieve this today. -Rasmus --- ## #89212 — Stephen Coakley — 2015-11-14T02:32:52+00:00 URL: https://externals.io/message/89208#89212 · In reply to: #89211 > > > […] > > You could simply deploy both the .php and the .bin files to achieve this today. > > -Rasmus > Would the bin files not have to be placed in the special OPcache file store location though? That seems sub-optimal. I'm glad it's already possible though. Thanks for the tip. -- Stephen --- ## #89213 — Sara Golemon — 2015-11-14T04:43:59+00:00 URL: https://externals.io/message/89208#89213 · In reply to: #89208 > I've been thinking about PHP optimization and distribution, and I would like > to hear some opinions on introducing a new feature for a future PHP 7 > version. Here's the concept: allow PHP opcode to be both saved after parsing > a file, and to be loaded and executed, similar to the bcompiler extension > and the APC bin functions back in the day. > Back up. What problem are you /actually/ trying to solve? OPCache does a very good job of caching the result of compilations, and it's a standard, bundled extension in common use. Is there a hit for that first load? Yes, but webservers run a very long time, and a priming step during deployment is more robust that pushing pre-compiled script databases. > The advantages are clear: libraries and applications could be compiled to > opcode ahead of time so PHP wouldn't have to compile it again and again > (assuming you're not already using OPcache). > That's a REALLY strange assumption to make. Who runs a PHP site where performance is any kind of concern, but doesn't use OPCache? Are these hypothetical people going to fail at that, but succeed at adding an additional step to their deployment process? That seems unlikely. > A new function could be provided to parse a PHP file, but instead of > executing it, the compiled opcode would be saved to a file. It might part of > the OPcache extension if it makes sense, and could be called > opcache_compile_to_file() or something. > I'm not inherently against this as a concept, but again I have to ask: "What problem does this solve?" because it certainly creates a new one. Here's my guess: You want to be able to distribute a library or application without giving your users source code. Apart from being a very non-open thing to do, it's also pointless. Disassembling an opcode stream to PHP source code is trivial. Many folks have tried to "encrypt" such bundles, and they've all been broken, because eventually PHP has to be able to see the bytecodes. Or am I wrong, and there's a motivation I'm just not seeing? -Sara --- ## #89214 — francois@php.net — 2015-11-14T14:16:00+00:00 URL: https://externals.io/message/89208#89214 · In reply to: #89208 Hi Stephen, Le 13/11/2015 22:35, Stephen Coakley a écrit : > Hello all, > > I've been thinking about PHP optimization and distribution, and I > would like to hear some opinions on introducing a new feature for a > future PHP 7 version. Here's the concept: allow PHP opcode to be both > saved after parsing a file, and to be loaded and executed, similar to > the bcompiler extension and the APC bin functions back in the day. > > The advantages are clear: libraries and applications could be compiled > to opcode ahead of time so PHP wouldn't have to compile it again and > again (assuming you're not already using OPcache). > > A new function could be provided to parse a PHP file, but instead of > executing it, the compiled opcode would be saved to a file. It might > part of the OPcache extension if it makes sense, and could be called > opcache_compile_to_file() or something. > > Another option would be to add a command-line flag to the interpreter > to write compiled opcode instead of executing it after parsing. > > Another part of the feature would be to enable the interpreter to > execute compiled opcode scripts directly. This would work for both > compiled scripts passed to the interpreter, and scripts loaded with > `include` and `require`. We would probably need to introduce a new > file extension to specify opcode files. I'd recommend *.phpo or *.phpc. > > This is quite similar to Python's ability to execute Python scripts > compiled to bytecode as *.pyc files. The feature has seen great > success in Python, mostly for distributing releases of software or > deploying to a server. > > I'm not at this moment planning an RFC, but I'd like to gauge your > opinions and reactions first. > AFAIK, generated bytecodes are not compatible across PHP versions. It may work for relatively similar versions and preferably in acscending order (executor more recent than compiler) but nothing is guaranteed. So, in practice, you cannot distribute pre-compiled code if you must impose a specific PHP version to execute it. Python chose to maintain a backwards compatibility between compiler and executor. PHP chose to cache transient opcodes. It may seem weaker but, IMO, this is a better approach because it allows a total freedom when creating, deprecating, and managing opcodes. Working on the engine would be much harder if we had to maintain BC on pre-compiled code. In practice, distributing pre-compiled PHP scripts wouldn't have any noticeable impact on performance because anyone whi cares about performance will enable opcache first (which should even be enabled by default, IMO). As Sara noticed, the only benefit would be obfuscation, which probably deserves a more powerful and reliable mechanism. Regards François --- ## #89217 — Rasmus Lerdorf — 2015-11-15T00:03:24+00:00 URL: https://externals.io/message/89208#89217 · In reply to: #89212 > > > […] > > Would the bin files not have to be placed in the special OPcache file store location though? That seems sub-optimal. But that is just a matter of copying the .bin files into a configurable directory. However, as others have said, this is completely pointless. If you are worried about initial compile latency on initial requests after a deeply of new code then simply run a couple of warmup requests before flipping your symlink to the new code. Beyond that I can't picture what possible use this could be. Hiding source code? No chance, getting PHP code from bytecode is trivial. -Rasmus --- ## #89241 — Jefferson Gonzalez — 2015-11-16T20:13:47+00:00 URL: https://externals.io/message/89208#89241 · In reply to: #89217 > Beyond that I can't picture what possible use this could be. I also think that the ability to have a .phpc and .php side by side would be nice, but not for hiding the source code. It would be useful especially for php-cgi and secondary for php-cli. In php-cgi case theres people who still host multiple sites on a single server and use the php-cgi method because it doesn't requires to have an always running instance like with php-fpm. Lets say you are hosting 100 sites on a single server, imho it would be less resourceful to run php-cgi processes on demand than having more than 100 fpm processes idling for requests. So in this scenario, having the flexibility to load byte code for php-cgi process would be a nice performance boost. --- ## #89243 — Rasmus Lerdorf — 2015-11-16T20:22:44+00:00 URL: https://externals.io/message/89208#89243 · In reply to: #89241 > > > […] > > I also think that the ability to have a .phpc and .php side by side > would be nice, but not for hiding the source code. It would be useful > especially for php-cgi and secondary for php-cli. In php-cgi case theres > people who still host multiple sites on a single server and use the > php-cgi method because it doesn't requires to have an always running > instance like with php-fpm. > > Lets say you are hosting 100 sites on a single server, imho it would be > less resourceful to run php-cgi processes on demand than having more > than 100 fpm processes idling for requests. So in this scenario, having > the flexibility to load byte code for php-cgi process would be a nice > performance boost. But that is exactly what the file-based opcache does by itself. The only speedup you achieve by trying to distribute the .bin files would be a minor boost the first time a cli script is executed. All subsequent runs of the script would hit the cache. The added complexity and potential version conflicts of trying to distribute the .bin files doesn't seem like it would be worth the trouble for such a minor one-time performance benefit. -Rasmus --- ## #89244 — Jefferson Gonzalez — 2015-11-16T21:11:04+00:00 URL: https://externals.io/message/89208#89244 · In reply to: #89243 > But that is exactly what the file-based opcache does by itself. The only > speedup you achieve by trying to distribute the .bin files would be a > minor boost the first time a cli script is executed. All subsequent runs > of the script would hit the cache. The added complexity and potential > version conflicts of trying to distribute the .bin files doesn't seem > like it would be worth the trouble for such a minor one-time performance > benefit. > > -Rasmus > > But as far as I know thats only possible with php-cli and not with php-cgi, is it? --- ## #89245 — Rasmus Lerdorf — 2015-11-16T21:22:24+00:00 URL: https://externals.io/message/89208#89245 · In reply to: #89244 > > > […] > > But as far as I know thats only possible with php-cli and not with > php-cgi, is it? There is nothing cli-specific about this feature, so yes, it works fine with php-cgi. You can set opcache.file_cache_only if you like to not even try to create a shm cache and only use the file-based one for your php-cgi setup. -Rasmus --- ## #89252 — Sara Golemon — 2015-11-17T00:06:43+00:00 URL: https://externals.io/message/89208#89252 · In reply to: #89241 > > > […] > > > I also think that the ability to have a .phpc and .php side by side would be > nice, but not for hiding the source code. It would be useful especially for > php-cgi and secondary for php-cli. In php-cgi case theres people who still > host multiple sites on a single server and use the php-cgi method because it > doesn't requires to have an always running instance like with php-fpm. > Which OpCache's file-backend covers...