# [mini-RFC] Disable opcache per script using "declare(cache=0)" - Thread: https://externals.io/message/103480 - Original archive: https://news-web.php.net/php.internals/103480 - Started by Dmitry Stogov on 2018-11-23T10:04:05+00:00 - Messages: 22 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #103480 — Dmitry Stogov — 2018-11-23T10:04:05+00:00 URL: https://externals.io/message/103480#103480 Hi internals, I like to propose an ability to disable opcache caching per script, using declare(cache=0) at start of PHP file. The idea was inspired by preloading usage, when some scripts are actually preloading scenarios and have to be executed, but don't have to be cached. https://github.com/php/php-src/pull/3678 The change is really small in both syntax and implementation, so I wouldn't like to waste time going through the full RFC process. I'm going to commit this into master on Wednesday (November 28), if no objections. Thanks. Dmitry. --- ## #103481 — Nikita Popov — 2018-11-23T10:08:53+00:00 URL: https://externals.io/message/103480#103481 · In reply to: #103480 > Hi internals, > > > I like to propose an ability to disable opcache caching per script, using > declare(cache=0) at start of PHP file. > > The idea was inspired by preloading usage, when some scripts are actually > preloading scenarios and have to be executed, but don't have to be cached. > > > https://github.com/php/php-src/pull/3678 > > > The change is really small in both syntax and implementation, so I > wouldn't like to waste time going through the full RFC process. > > I'm going to commit this into master on Wednesday (November 28), if no > objections. > > > Thanks. Dmitry. > Can you please provide an example where this would be used? Should all scripts involved in preloading (including the main one) be marked this way? I'm not sure I fully understand the purpose. Nikita --- ## #103482 — Dmitry Stogov — 2018-11-23T12:48:40+00:00 URL: https://externals.io/message/103480#103482 Hi Nikita, Especially, the main preload.php, usually should be marked, to disable its caching. Thanks. Dmitry. --- ## #103483 — Pierre Joye — 2018-11-23T14:36:29+00:00 URL: https://externals.io/message/103480#103483 · In reply to: #103482 Hi Dmitry, > Hi Nikita, > > > Especially, the main preload.php, usually should be marked, to disable its > caching. > should this specific case not be done automatically then? best, > > > Thanks. Dmitry. > > ________________________________ > From: Nikita Popov > Sent: Friday, November 23, 2018 1:08:53 PM > To: Dmitry Stogov > Cc: PHP internals > Subject: Re: [PHP-DEV] [mini-RFC] Disable opcache per script using > "declare(cache=0)" > > On Fri, Nov 23, 2018 at 11:04 AM Dmitry Stogov dmitry@zend.com>> wrote: > Hi internals, > > > I like to propose an ability to disable opcache caching per script, using > declare(cache=0) at start of PHP file. > > The idea was inspired by preloading usage, when some scripts are actually > preloading scenarios and have to be executed, but don't have to be cached. > > > https://github.com/php/php-src/pull/3678 > > > The change is really small in both syntax and implementation, so I > wouldn't like to waste time going through the full RFC process. > > I'm going to commit this into master on Wednesday (November 28), if no > objections. > > > Thanks. Dmitry. > > Can you please provide an example where this would be used? Should all > scripts involved in preloading (including the main one) be marked this way? > I'm not sure I fully understand the purpose. > > Nikita --- ## #103487 — Sara Golemon — 2018-11-23T16:23:01+00:00 URL: https://externals.io/message/103480#103487 · In reply to: #103483 > > […] > its > > […] > > should this specific case not be done automatically then? > > I'm with Pierre on this. During the preload phase, we should probably have implicit caching disabled and only cache things brought in via `opcache_compile_file()`. Outside of the preload pragma, I'd worry that this declare would be mistakenly overused in most cases. Question though... who wins if a file is explicitly loaded with opcache_compile_file *and* it has the "don't cache me, bro" pragma? I'd expect the file wouldn't be cached in that case. That said, I'm of the "here's your gun, try not to shoot yourself" school on most things, and I see no harm in this when used judiciously, so no objections from me. -Sara --- ## #103488 — Rowan Collins — 2018-11-23T23:17:55+00:00 URL: https://externals.io/message/103480#103488 · In reply to: #103482 >Especially, the main preload.php, usually should be marked, to disable >its caching. Sorry, could you explain for those of us in the peanut gallery why this is the case? I'm struggling to understand what it means to preload something but not cache it, and suspect I'm missing something fundamental. Regards, -- Rowan Collins [IMSoP] --- ## #103489 — Zeev Suraski — 2018-11-23T23:49:50+00:00 URL: https://externals.io/message/103480#103489 · In reply to: #103488 On Sat, Nov 24, 2018 at 1:18 AM Rowan Collins wrote: > On 23 November 2018 12:48:40 GMT+00:00, Dmitry Stogov > wrote: > > […] > > Sorry, could you explain for those of us in the peanut gallery why this is > the case? I'm struggling to understand what it means to preload something > but not cache it, and suspect I'm missing something fundamental. > > I'm not sure if you're missing anything fundamental - it's just that the preload.php file Dmitry's referring to (the one that's responsible to loading all the other files) - is one file that's pretty much by definition, will be of no use at any later point in the lifetime of the server. It preloads other files (which of course, should very much persist in the opcache memory) - but once it's done, it's done. That's why it kind of makes sense not to cache it. That said, I'm not sure whether having this feature as a declare() makes sense - at least in the context of the new preload feature. The preload file is likely to be fairly small, wouldn't it consume negligible amounts of shared memory and just one slot in the files table? And like Pierre suggested, perhaps we can simply default to not caching the main preload file (i.e. the one referenced by php.ini). Outside the context of preloading, there's probably some value in having this new declare() option. There are situations where you simply don't want to cache certain files (what I usually bumped into was in case of auto-generated, short-TTL files). That's why a lifetime or two ago we added the blacklist feature - the declare() syntax would offer a new way to do that that is configurationless and built into the code. In that sense, it's not a bad thing. I'm just seeing it as not so much related to preloading, and more of a general-purpose feature. Unless of course I'm missing something..? Zeev --- ## #103490 — Stanislav Malyshev — 2018-11-24T07:09:31+00:00 URL: https://externals.io/message/103480#103490 · In reply to: #103482 Hi! > Especially, the main preload.php, usually should be marked, to disable its caching. Why should it be marked as non-cacheable? I am feeling there's something important I am missing here. Could you explain a bit more about this? -- Stas Malyshev smalyshev@gmail.com --- ## #103491 — Rowan Collins — 2018-11-24T13:19:15+00:00 URL: https://externals.io/message/103480#103491 · In reply to: #103489 > I'm not sure if you're missing anything fundamental - it's just that > the preload.php file Dmitry's referring to (the one that's responsible > to loading all the other files) - is one file that's pretty much by > definition, will be of no use at any later point in the lifetime of > the server.  It preloads other files (which of course, should very > much persist in the opcache memory) - but once it's done, it's done.  > That's why it kind of makes sense not to cache it. Ah, OK, so it's more a case of "don't bother caching" rather than "must avoid caching" then? I think making it automatically uncached might be a bit confusing, and a source of future "gotchas". While it's *logical* for the INI setting to reference a stub file with nothing but includes, the feature as described in the RFC doesn't actually *require* this; it would be perfectly possible for someone to put a bunch of definitions in "preloaded_functions.php" and reference that one file directly in the preload directive. On the flip side, it's likely that frameworks will start shipping with their own preload helpers, so that you'd have "preload_all.php" reference "symfony/preload.php", and want to exclude *both* from the cache. Thinking about it, I'm not clear exactly what a "do not cache" directive would mean in a preloaded file, since preloading is more than just caching. For instance, if "class A" is declared in a preloaded file with declare(cache=0), then "class B extends A" is declared in a preloaded file with caching enabled, how will class B be treated? Will the file containing class B be cached, but need to be referenced in an "include" statement in each request? Again, people will *probably* be using file structures and autoloaders that make this trivial, but the language doesn't require that. Regards, -- Rowan Collins [IMSoP] --- ## #103492 — Stanislav Malyshev — 2018-11-24T19:57:37+00:00 URL: https://externals.io/message/103480#103492 · In reply to: #103491 Hi! > > […] Ah, ok, this makes sense. Though usually such file would be rather small and adding whole new directive just for it may not be that necessary, I think. > Thinking about it, I'm not clear exactly what a "do not cache" directive > would mean in a preloaded file, since preloading is more than just > caching. For instance, if "class A" is declared in a preloaded file with > declare(cache=0), then "class B extends A" is declared in a preloaded > file with caching enabled, how will class B be treated? Will the file Yes, that's what I am wondering too - how things defined directly in preload file then will be treated? While file itself is not useful later in server's lifetime, the things defined in it - like classes or functions - can be? -- Stas Malyshev smalyshev@gmail.com --- ## #103493 — Marco Pivetta — 2018-11-25T05:03:01+00:00 URL: https://externals.io/message/103480#103493 · In reply to: #103492 Adding to the pile of "it's an edge case", since the preload scripts will be procedural, wouldn't it be sufficient to call `opcache_invalidate(__FILE__)` at the end of them? > Hi! > > > […] > > Ah, ok, this makes sense. Though usually such file would be rather small > and adding whole new directive just for it may not be that necessary, I > think. > > > […] > > Yes, that's what I am wondering too - how things defined directly in > preload file then will be treated? While file itself is not useful later > in server's lifetime, the things defined in it - like classes or > functions - can be? > -- > Stas Malyshev > smalyshev@gmail.com --- ## #103494 — Sara Golemon — 2018-11-25T16:04:18+00:00 URL: https://externals.io/message/103480#103494 · In reply to: #103493 > Adding to the pile of "it's an edge case", since the preload scripts will > be procedural, wouldn't it be sufficient to call > `opcache_invalidate(__FILE__)` at the end of them? > I suppose the only negative there would be some very minor fragmentation, though again, since we can reasonably expect preload scripts to be fairly simple affairs, I'm not massively bothered by this approach. On the other hand, I love its simplicity. --- ## #103495 — Larry Garfield — 2018-11-25T17:34:49+00:00 URL: https://externals.io/message/103480#103495 · In reply to: #103493 > Adding to the pile of "it's an edge case", since the preload scripts will > be procedural, wouldn't it be sufficient to call > `opcache_invalidate(__FILE__)` at the end of them? Are we certain that it will always be procedural? Is there a part of the preload script design that prevents it from defining a function? I'd be very surprised if someone defined a class in a preload script, but I can definitely see a function getting defined somewhere. And if it includes some other file (eg, Composer may be generating one, frameworks may build one, etc.) then who knows what that file will have defined. Would that cause issues in this case? --Larry Garfield --- ## #103496 — Marco Pivetta — 2018-11-25T18:06:43+00:00 URL: https://externals.io/message/103480#103496 · In reply to: #103495 Even if there were definitions in said script, the same issue would arise with a directive. > > > […] > > Are we certain that it will always be procedural? Is there a part of the > preload script design that prevents it from defining a function? > > I'd be very surprised if someone defined a class in a preload script, but > I > can definitely see a function getting defined somewhere. And if it > includes > some other file (eg, Composer may be generating one, frameworks may build > one, > etc.) then who knows what that file will have defined. > > Would that cause issues in this case? > > --Larry Garfield --- ## #103497 — Zeev Suraski — 2018-11-25T21:34:28+00:00 URL: https://externals.io/message/103480#103497 · In reply to: #103493 > Adding to the pile of "it's an edge case", since the preload scripts will > be procedural, wouldn't it be sufficient to call > `opcache_invalidate(__FILE__)` at the end of them? > > That would actually not do anything useful - as the file will (in all likelihood) never be accessed again anyway. Invalidating a file only ensures it won't be fetched again from the opcache - it doesn't actually free up the space it consumes. Invalidate it or not, in both cases the outcome is the same - a bit of memory consumed by a file that will never again be fetched from the opcache. I believe this is what Dmitry's proposal was about - not putting the file into the opcache to begin with, to save the bit of memory that it would otherwise consume (but I could be wrong, as the savings are likely to be so small I'm not sure why we would care about them...). Zeev --- ## #103498 — Marco Pivetta — 2018-11-25T21:43:15+00:00 URL: https://externals.io/message/103480#103498 · In reply to: #103497 Is that space rrrrrrreeeeeally a problem? Take the example ZF loader from the RFC: that barely makes any difference at all. A stronger reasoning for another language construct (that changes engine behaviour) is kinfa required. > Adding to the pile of "it's an edge case", since the preload scripts will > be procedural, wouldn't it be sufficient to call > `opcache_invalidate(__FILE__)` at the end of them? > > That would actually not do anything useful - as the file will (in all likelihood) never be accessed again anyway. Invalidating a file only ensures it won't be fetched again from the opcache - it doesn't actually free up the space it consumes. Invalidate it or not, in both cases the outcome is the same - a bit of memory consumed by a file that will never again be fetched from the opcache. I believe this is what Dmitry's proposal was about - not putting the file into the opcache to begin with, to save the bit of memory that it would otherwise consume (but I could be wrong, as the savings are likely to be so small I'm not sure why we would care about them...). Zeev --- ## #103499 — Dmitry Stogov — 2018-11-26T07:51:25+00:00 URL: https://externals.io/message/103480#103499 OK. I didn't think, this feature required explanation. I saw this like a simple way to separate "data" and "code". Anyway, now I see, this can't be accepted without RFC etc Thanks. Dmitry. --- ## #103500 — Zeev Suraski — 2018-11-26T09:15:03+00:00 URL: https://externals.io/message/103480#103500 · In reply to: #103498 > Is that space rrrrrrreeeeeally a problem? > > Take the example ZF loader from the RFC: that barely makes any difference > at all. > > A stronger reasoning for another language construct (that changes engine > behaviour) is kinfa required. > The emails I've written on this thread already suggested that I tend to agree. I just spoke with Dmitry to better understand why he cared about so little space, and turns out he didn't either - the rationale is different and is pretty straightforward. The goal of declare(cache=0) would be to avoid persisting utility functions/classes that have to do with a particular preload.php implementation - so that they don't become a part of the app's memory context and 'pollute' its scope. For example, you may implement a preload_file() function in preload.php, or perhaps even a class Preload{} with some utility functions that will be responsible for include()ing all the various files you want to preload into your server. Currently, function preload_file() / class Preload would become permanent parts of the app's scope - and pollute it to some degree. They're sort of meta-code that has no business sticking around in the app's memory space once the preloading stage is over. With declare(cache=0) - these will execute just as before, but won't be persisted into the permanent scope (or into the OPcache at all, for that matter). There's no way to 'automatically' apply this 'do-not-cache', as in different implementations - the preload.php file might actually include classes/functions folks would actually want to persist, and in other cases files include()ed by preload.php may in fact include functions/classes people do not want to persist. This has to be a manually-applied feature. As I mentioned in another reply on this thread, it's also a nice runtime/configurationless alternative for the blacklist feature, in case you want to prevent certain files from being cached for whatever reason (that use case is unrelated to the preload capability). I'm personally fine with this being admitted w/o a full fledged RFC if there are no objections. Zeev --- ## #103501 — Zeev Suraski — 2018-11-26T09:18:28+00:00 URL: https://externals.io/message/103480#103501 · In reply to: #103494 > > > > […] > > > > […] > I suppose the only negative there would be some very minor fragmentation, > One thing that's worth mentioning is that there's no fragmentation ever with OPcache - because we actually never reclaim freed space (only during a restart, where, well, we start over completely with a clean slate...). Once a file is in the cache, the space it consumes is never freed and can never be (re)used by another file. Excellent strategy to eliminate fragmentation, not quite as good for reducing memory consumption though :) Zeev --- ## #103503 — Rowan Collins — 2018-11-26T10:46:59+00:00 URL: https://externals.io/message/103480#103503 · In reply to: #103500 > The goal of declare(cache=0) would be to avoid persisting utility > functions/classes that have to do with a particular preload.php > implementation - so that they don't become a part of the app's memory > context and 'pollute' its scope. > Thanks for the clarification, that makes a lot more sense than the performance impact. As I mentioned before, though, the behaviour of the following case is not obvious to me: # preload.php require 'foo.php'; require 'bar.php'; # foo.php declare(cache=0); class Foo {} # bar.php class Bar extends Foo {} At the time it is first parsed, Bar is a complete class, and is being defined during preload, so ought to be defined permanently; but it depends on Foo, which will not be cached. Will this be an error, or will class Bar simply not be defined on future requests? Regards, -- Rowan Collins [IMSoP] --- ## #103505 — Nikita Popov — 2018-11-26T11:14:40+00:00 URL: https://externals.io/message/103480#103505 · In reply to: #103500 > > > > […] > > The emails I've written on this thread already suggested that I tend to > agree. I just spoke with Dmitry to better understand why he cared about so > little space, and turns out he didn't either - the rationale is different > and is pretty straightforward. > > The goal of declare(cache=0) would be to avoid persisting utility > functions/classes that have to do with a particular preload.php > implementation - so that they don't become a part of the app's memory > context and 'pollute' its scope. > > For example, you may implement a preload_file() function in preload.php, or > perhaps even a class Preload{} with some utility functions that will be > responsible for include()ing all the various files you want to preload into > your server. Currently, function preload_file() / class Preload would > become permanent parts of the app's scope - and pollute it to some degree. > They're sort of meta-code that has no business sticking around in the app's > memory space once the preloading stage is over. With declare(cache=0) - > these will execute just as before, but won't be persisted into the > permanent scope (or into the OPcache at all, for that matter). > > There's no way to 'automatically' apply this 'do-not-cache', as in > different implementations - the preload.php file might actually include > classes/functions folks would actually want to persist, and in other cases > files include()ed by preload.php may in fact include functions/classes > people do not want to persist. This has to be a manually-applied feature. > > As I mentioned in another reply on this thread, it's also a nice > runtime/configurationless alternative for the blacklist feature, in case > you want to prevent certain files from being cached for whatever reason > (that use case is unrelated to the preload capability). > > I'm personally fine with this being admitted w/o a full fledged RFC if > there are no objections. > > Zeev > It would make more sense to me to not cache anything from the preload script or anything it includes, and only make things for which `opcache_compile_file()` is called part of the preload. As I understand it, right now both included and compiled files will be preloaded. I think that would be a better solution than a declare annotation, because it would allow you to use arbitrary code in your preload script. You could make use of normal library code -- something that would be much harder if you had to hot-patch the library to include no-cache directives first. Nikita --- ## #103508 — Dmitry Stogov — 2018-11-26T12:01:01+00:00 URL: https://externals.io/message/103480#103508 · In reply to: #103505 > > > > […] > It would make more sense to me to not cache anything from the preload > script or anything it includes, and only make things for which > `opcache_compile_file()` is called part of the preload. As I understand it, > right now both included and compiled files will be preloaded. > > I think that would be a better solution than a declare annotation, because > it would allow you to use arbitrary code in your preload script. You could > make use of normal library code -- something that would be much harder if > you had to hot-patch the library to include no-cache directives first. In general, this solution should work. Thanks. Dmitry.