Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:108135
User-Agent: Cyrus-JMAP/3.1.7-754-g09d1619-fmstable-20200113v1
Mime-Version: 1.0
Message-ID: <a599c53d-665b-4e45-9d3e-c5deeac2d5f4@www.fastmail.com>
In-Reply-To: <A053F3BB-97CC-4CB3-99DE-275438628E8C@newclarity.net>
References: 
 <CAMuFt-6=FaqRpTEvZDtW_OsLmq3R171nDc7wH0D5itJUosMtdQ@mail.gmail.com>
 <d2348c4b-7993-489e-86ed-01932a864ec9@www.fastmail.com>
 <CD056374-BE4F-4B7F-A0F1-5481D024AE8A@newclarity.net>
 <f0a08ad9-abfa-4155-941c-4efd2cf61842@www.fastmail.com>
 <FF6699B7-39E7-4AD7-8FC6-5EE02BBC8E05@newclarity.net>
 <2502e68d-a1bc-41c6-9763-8c0e2ac9e6f4@www.fastmail.com>
 <A053F3BB-97CC-4CB3-99DE-275438628E8C@newclarity.net>
Date: Tue, 14 Jan 2020 18:50:57 -0600
To: "php internals" <internals@lists.php.net>
Content-Type: text/plain
Subject: =?UTF-8?Q?Re:_[PHP-DEV]_Introducing_compile_time_code_execution_to_PHP_p?=
 =?UTF-8?Q?reloading?=
From: larry@garfieldtech.com ("Larry Garfield")

On Tue, Jan 14, 2020, at 1:22 PM, Mike Schinkel wrote:
> On Jan 14, 2020, at 1:10 PM, Larry Garfield <larry@garfieldtech.com> wrote:
> >> I get your concern about recursion, but if that kind of issue is really 
> >> a concern I don't see why we could not artificially limit recursion on 
> >> preload to a configurable amount, with 100 being the default?
> > 
> > It's not recursion itself that's an issue.  It's that the behavior of the code changes between preload and non-preload contexts in subtle, non-obvious ways.
> > 
> > I don't mean a recursive function that runs during preload.  I mean if a function that runs in a normal request is written recursively, it will fail at high recursion levels only if the file it is in was not preloaded.  So whether or not foo.php was run through opcache_compile_file() changes when and how code in that file will fail.  That's the situation we want to avoid like the plague, IMO.
> 
> With respect, that feels like you are concerned about some real edge 
> cases, and ignoring the probability that most code run during 
> preloading would be written bespoke for preloading.

I think you're still missing my point here.  AST manipulation during preload would exist to manipulate code that would be run *later*.  That's the point.  Sure, the code in the run_once/preload/whatever block would be written to run in that context, but its entire point is, presumably, to improve the running of the *rest* of the code later.

That's where the footgun exists.  The recursion example was simply to demonstrate that even seemingly "safe" optimizations may introduce inconsistency, so I don't expect most userland developers to be able to get it right the first 5 times.  (And again, I include myself in that.)

> ALSO, I think I may have caused confusion by calling it "preloading."  
> In my mind that meant "run once" vs. "run for each page load" but I 
> think you are (rightly) conflating what I have called "preloading" with 
> the new preloading feature in PHP 7.4?
> 
> What I have been envisioning is a `preload` (or some other) keyword 
> that we could apply to functions, methods, and expressions. When 
> `preload` modified  functions/methods/expressions are reached during 
> OpCode generation, rather than generate a call to runtime code PHP 
> would just run the code and then take the return value and generate 
> OpCode for that return value. 

That... is a completely different thing, yes, that has nothing to do with the PHP 7.4 feature called preloading.  Which would mean we've been talking about two different things this whole damned time.  *sigh*

> > That... sounds a lot like the new FFI extension?  Write code in C or Rust or anything that can produce a .so file, plug it into PHP, go?
> 
> Unless I am misunderstanding, using an .so requires a sysadmin to add 
> an extension and update php.ini. Which is a non-starter on hosted sites 
> like Pantheon, WPEngine, Kinsta, Pagely, Flywheel, PressLabs and 
> similar.

No, the whole point of the FFI extension is that a user-land developer can load a .so file into PHP directly; basically doing the lib->PHP glue code in PHP rather than in C.  It does require the FFI extension itself to be enabled, and to do safely requires using the preloader, which does mean setting an ini setting.

Not all hosts support that.  The one I work for does, which is why I've been playing with it lately to document it all for our customers. :-)  But the fact that such advanced features are not universally available is... exactly why I am extremely cautious about having functionality that results in subtle behavioral differences between those environments where it's available and those where it's not.

The existing preload logic has no impact on behavior other than skipping the autoloader.  Any behavioral change is limited to those oddball libraries that do black magic during the autoloader, which are already well off the beaten path.  That makes it a "safe" optimization.

My underlying point, and then I will bow out of this thread as I am tired of repeating myself, is that I am all for enabling more "safe" optimizations, even letting userland developers build them, but we need to be extremely careful about them being "safe" and not "a time bomb that will introduce subtle behavioral differences between different run modes that a developer is not expecting, thus slowly creating libraries that will only function in one run mode or the other".

If we can enable more optimizations while avoiding that trap, I'm all for it.  But avoiding that trap is more important than giving developers more foot-guns.

--Larry Garfield