Hi internals,
I would like to propose a small OPcache feature for the master branch and
ask whether it can go through a regular PR or should be an RFC.
Problem
Shared hosts commonly run many copies of the same application from one FPM
master: CMS cores, plugins, vendor/ trees. OPcache keys scripts by path, so
every copy is persisted separately, although most functions compile to the
same opcodes. Sharing whole scripts cannot work, because a compiled script
embeds its path (op_array->filename, FILE/DIR, runtime definition
keys, class entries). This was discussed for APC in bug #63689 and here in
2016 (https://externals.io/message/93024).
Proposal
A new INI_SYSTEM directive, opcache.enable_dedup (default 0). When enabled,
the opcodes and literals of each op_array are looked up by content while
the script is persisted. If an identical block is already in SHM, the
op_array points to it instead of copying it. Everything else stays per
op_array: filename, lines, scope, vars, arg_info, live ranges, try/catch,
static variables. FILE, traces, errors and Reflection are unchanged;
path-dependent blocks simply differ and are not shared.
- A hash match is always confirmed by a full comparison of the normalized
blocks; the key is salted per SHM, probing is bounded (16 slots). - Scripts are still looked up by path; validate_permission/validate_root
are untouched. The table is only used while persisting, under the
existing SHM lock, and nothing is freed before a restart (no refcount). - The calc pass records its decision for each op_array and the persist
pass applies it, so both passes agree on sizes and each block is
serialized once. - Disabled, with a warning, when the JIT is available, when
opcache.file_cache is set, and on 32-bit builds. With the directive off,
nothing is allocated and no code path changes. opcache_get_status()gets a "dedup" key (entries, hits, saved_memory).
The change is one new file (zend_accelerator_dedup.c/.h, about 700 lines)
and about 40 lines of hooks in zend_persist*.c, ZendAccelerator.c and
zend_accelerator_module.c, plus 15 phpt tests.
Results
Load test, 80 WordPress sites (WooCommerce, Yoast, Elementor), one user and
one pool per site, one master, 500,000 requests per run, PHP 8.4 release:
OPcache memory 6,406 MB -> 3,428 MB (-46.5%)
PHP-FPM peak PSS 6,711 MB -> 5,466 MB (-18.6%)
CPU per request 170.3 ms -> 171.7 ms (within run-to-run noise)
compile CPU +2.2% (cold cache only)
errors / crashes 0 / 0
Production, PHP 8.4, 52 pools on one master, 11 days so far:
OPcache memory 5.4 GB of bytecode stored in 3.1 GB (-43%)
shared op_arrays 82.5%
worker crashes 0 (SIGSEGV/SIGBUS/SIGABRT)
Tests: the new tests pass on 8.2 to master; Zend/tests, tests/ and
ext/opcache/tests show no difference with dedup forced on and
protect_memory=1; ASan/valgrind clean; a differential fuzzer (random
programs compiled twice to force sharing, run with and without dedup)
found no divergence.
Security: a PHP-level tenant cannot reach, read or write a shared block
(no API returns opcodes, blocks are immutable). A native attacker can
already rewrite any cached script in SHM; dedup only means one write
reaches every site sharing the block, which argues for restrict_api and
for keeping masters at a reasonable size. Write-up:
https://github.com/studio-vitalyn/php-opcache-dedup/blob/main/docs/SECURITY.md
Open questions
- Is a PR with discussion enough, or do you want an RFC for a new
opt-in INI directive? - Should the dedup table size be configurable? It is 4 slots per
max_accelerated_files, capped at 2^20 slots (32 MB). It is 51% full in
production, but reached 93% on a smaller host. - Follow-ups such as sharing arg_info/vars and, later, the immutable part
of class entries (about 31% of the remaining per-site memory) would
touch the engine; I left them out to keep this change confined to
OPcache persistence.
Patch against master (one commit, tests included):
https://github.com/php/php-src/compare/master...studio-vitalyn:php-src:opcache-dedup
Backports to 8.2-8.6, benchmarks and fuzzer:
https://github.com/studio-vitalyn/php-opcache-dedup
Case study (benchmark and production):
https://vitalyn.com/en/blog/80-wordpress-sites-one-opcache-eleven-days-in-production/
Feedback welcome, especially from OPcache maintainers.
Thanks,
Jean-Christophe Chenailler