I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.
Details, etc... are on the PR.
Regards
-M
Hi Matt,
-----Original Message-----
From: Matt Ficken [mailto:themattficken@gmail.com]
Sent: Wednesday, September 23, 2015 9:10 AM
To: PHP Internals internals@lists.php.net
Cc: dmitry@php.net; laruence@php.net
Subject: [PHP-DEV] Windows OpCache bug fixI want to increase visibility for my PR 1531, https://github.com/php/php-
src/pull/1531, my patch for fixing an intermittent OpCache issue on Windows.Details, etc... are on the PR.
thanks for the patch. It's great to fix that "base address" issue in PHP-7.0 at least. It would probably make sense to backport it to 5.6 later.
One minor comment though - you shouldn't use C++ comments there according to the coding styles.
Regards
Anatol
Hi Matt,
It looks like with your patch, the same file may be mapped to different
virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.
Do I understand properly, that zend_shared_alloc_unlock/lock_win32() around
Sleep() is the main part of the patch?
Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).
Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken themattficken@gmail.com
wrote:
I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
Ok, I have a new PR just to unlock around Sleep():
https://github.com/php/php-src/pull/1536
Just by unlocking around Sleep() the OpenFileMapping loop will work.
Holding the lock in that loop will not only block detach_segments() in
other PHP processes but also reattaching in accel_startup().
Checking the base address used when this issue occurs during some of my
stress testing, most of the time the base address used is the first in
vista_mapping_base_set. Its typically the timing between the locking and
mapped file operations between processes more often than the base address
selection.
Especially ASLR, also PECL DLLs, and other factors will still cause this
issue occasionally, but this fixes a majority of my occurrences.
Regards
-M
Hi Matt,
It looks like with your patch, the same file may be mapped to different
virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.Do I understand properly, that zend_shared_alloc_unlock/lock_win32()
aroundSleep()is the main part of the patch?Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken themattficken@gmail.com
wrote:I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
hi Matt,
Thanks.
I also moved the exit condition to be before Sleep().
This should prevent race condition on last iteration (after unlocked sleep
process should try to reattach).
Committed to PHP-7.0 and master.
http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454
If this really works, we should backport this into php-5.* as well.
Thanks. Dmitry.
On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken themattficken@gmail.com
wrote:
Ok, I have a new PR just to unlock around
Sleep():
https://github.com/php/php-src/pull/1536Just by unlocking around
Sleep()the OpenFileMapping loop will work.
Holding the lock in that loop will not only block detach_segments() in
other PHP processes but also reattaching in accel_startup().Checking the base address used when this issue occurs during some of my
stress testing, most of the time the base address used is the first in
vista_mapping_base_set. Its typically the timing between the locking and
mapped file operations between processes more often than the base address
selection.Especially ASLR, also PECL DLLs, and other factors will still cause this
issue occasionally, but this fixes a majority of my occurrences.Regards
-MHi Matt,
It looks like with your patch, the same file may be mapped to different
virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.Do I understand properly, that zend_shared_alloc_unlock/lock_win32()
aroundSleep()is the main part of the patch?Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken themattficken@gmail.com
wrote:I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
Ops, I forgot to revert the exit condition.
http://git.php.net/?p=php-src.git;a=commitdiff;h=2d55e8c186ef1034c2af64478da8f23dbeb28be9
hi Matt,
Thanks.
I also moved the exit condition to be beforeSleep().
This should prevent race condition on last iteration (after unlocked sleep
process should try to reattach).
Committed to PHP-7.0 and master.http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454
If this really works, we should backport this into php-5.* as well.
Thanks. Dmitry.
On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken themattficken@gmail.com
wrote:Ok, I have a new PR just to unlock around
Sleep():
https://github.com/php/php-src/pull/1536Just by unlocking around
Sleep()the OpenFileMapping loop will work.
Holding the lock in that loop will not only block detach_segments() in
other PHP processes but also reattaching in accel_startup().Checking the base address used when this issue occurs during some of my
stress testing, most of the time the base address used is the first in
vista_mapping_base_set. Its typically the timing between the locking and
mapped file operations between processes more often than the base address
selection.Especially ASLR, also PECL DLLs, and other factors will still cause this
issue occasionally, but this fixes a majority of my occurrences.Regards
-MHi Matt,
It looks like with your patch, the same file may be mapped to different
virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.Do I understand properly, that zend_shared_alloc_unlock/lock_win32()
aroundSleep()is the main part of the patch?Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken themattficken@gmail.com
wrote:I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
Ops, I forgot to revert the exit condition.
http://git.php.net/?p=php-src.git;a=commitdiff;h=2d55e8c186ef1034c2af64478da8f23dbeb28be9
hi Matt,
Thanks.
I also moved the exit condition to be beforeSleep().
This should prevent race condition on last iteration (after unlocked sleep
process should try to reattach).
Committed to PHP-7.0 and master.http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454
If this really works, we should backport this into php-5.* as well.
Thanks. Dmitry.
On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken themattficken@gmail.com
wrote:Ok, I have a new PR just to unlock around
Sleep():
https://github.com/php/php-src/pull/1536Just by unlocking around
Sleep()the OpenFileMapping loop will work.
Holding the lock in that loop will not only block detach_segments() in
other PHP processes but also reattaching in accel_startup().Checking the base address used when this issue occurs during some of my
stress testing, most of the time the base address used is the first in
vista_mapping_base_set. Its typically the timing between the locking and
mapped file operations between processes more often than the base address
selection.Especially ASLR, also PECL DLLs, and other factors will still cause this
issue occasionally, but this fixes a majority of my occurrences.Regards
-MHi Matt,
It looks like with your patch, the same file may be mapped to different
virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.Do I understand properly, that zend_shared_alloc_unlock/lock_win32()
aroundSleep()is the main part of the patch?Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken themattficken@gmail.com
wrote:I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
Great. The locking issue is fixed.
But, it sometimes still fails to reattach to the base address. Because of
ASLR, extra DLLs, etc... the existing base address may not be usable in
other processes that need to reattach. When I see this issue I'm creating a
series of processes (for test runs) and when a process fails to reattach,
it happens to all the following processes for a few minutes. ASLR waits a
few minutes before changing the randomized memory layout when creating a
new process. When this happens to production web sites, it often sounds
like this is the case, as the web site becomes unusable for a while.
Exiting when the base address can't be reattached to, is not a good way to
handle this case, even in a FastCGI scenario.
The contract for create_segments() is to reattach successfully or return
ALLOC_FAILED.
Option 1: change create_segments() contract and zend_shared_alloc_startup()
so that create_segments() can disable OpCache, possibly setting
zend_accel_globals.enable=false and adding additional checks. This
eliminates the benefit of using OpCache, likely for several processes,
causing a major performance drop for a web site.
Option 2: creating a backup, 2nd backup, etc... OpCache in this case. There
may be up to 6 OpCaches(5 backups) with 4 attempts each. This fix is
simpler.This allows processes to share a backup OpCache, a web site will
slow down only to compile the scripts again(at the expense of some
additional memory).
There have been some feature requests to be able to create separate
OpCaches anyway. Shared web hosters would like to have different OpCaches
for different sites, but this would require a new INI directive which is
too late to add for 7, and for machines that run different PHP builds.
This patch passes my functional, stress and performance tests.
I've added it to the PR 1536:
https://github.com/php/php-src/commit/7cbebc747e5c5f824a6e0336cc044ce4dc4c8f9f
Regards
-M
Ops, I forgot to revert the exit condition.
http://git.php.net/?p=php-src.git;a=commitdiff;h=2d55e8c186ef1034c2af64478da8f23dbeb28be9
hi Matt,
Thanks.
I also moved the exit condition to be beforeSleep().
This should prevent race condition on last iteration (after unlocked
sleep process should try to reattach).
Committed to PHP-7.0 and master.http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454
If this really works, we should backport this into php-5.* as well.
Thanks. Dmitry.
On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken themattficken@gmail.com
wrote:Ok, I have a new PR just to unlock around
Sleep():
https://github.com/php/php-src/pull/1536Just by unlocking around
Sleep()the OpenFileMapping loop will work.
Holding the lock in that loop will not only block detach_segments() in
other PHP processes but also reattaching in accel_startup().Checking the base address used when this issue occurs during some of my
stress testing, most of the time the base address used is the first in
vista_mapping_base_set. Its typically the timing between the locking and
mapped file operations between processes more often than the base address
selection.Especially ASLR, also PECL DLLs, and other factors will still cause this
issue occasionally, but this fixes a majority of my occurrences.Regards
-MHi Matt,
It looks like with your patch, the same file may be mapped to different
virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.Do I understand properly, that zend_shared_alloc_unlock/lock_win32()
aroundSleep()is the main part of the patch?Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken themattficken@gmail.com
wrote:I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
Hi Matt,
Thanks for looking into the problem, but I afraid that the proposed
solution won't solve the problem completely, but may introduce new
problems. At first, we might to reserve N times more memory. Then different
SHM segments might contain different data. opcache_reset(),
opcache_invalidate() and others will affect only one segment.
I think the perfect solution would require converting all stored data in
position independent format, but this hardly ever is going to be done even
for php-8.
We may fallback to use file-cache introduced in php-7. (this should work
out of the box or with minimal changes).
Also. if we can't map SHM into desired address space, we may map it in any
other address and copy data into the process memory similar to file-cache.
Finally we may try to use /DYNAMICBASE:NO
Thanks. Dmitry.
On Tue, Sep 29, 2015 at 2:55 AM, Matt Ficken themattficken@gmail.com
wrote:
Great. The locking issue is fixed.
But, it sometimes still fails to reattach to the base address. Because of
ASLR, extra DLLs, etc... the existing base address may not be usable in
other processes that need to reattach. When I see this issue I'm creating a
series of processes (for test runs) and when a process fails to reattach,
it happens to all the following processes for a few minutes. ASLR waits a
few minutes before changing the randomized memory layout when creating a
new process. When this happens to production web sites, it often sounds
like this is the case, as the web site becomes unusable for a while.Exiting when the base address can't be reattached to, is not a good way to
handle this case, even in a FastCGI scenario.The contract for create_segments() is to reattach successfully or return
ALLOC_FAILED.Option 1: change create_segments() contract and
zend_shared_alloc_startup() so that create_segments() candisable
OpCache, possibly setting zend_accel_globals.enable=false and adding
additional checks. This eliminates the benefit of using OpCache, likely for
several processes, causing a major performance drop for a web site.Option 2: creating a backup, 2nd backup, etc... OpCache in this case.
There may be up to 6 OpCaches(5 backups) with 4 attempts each. This fix is
simpler.This allows processes to share a backup OpCache, a web site will
slow down only to compile the scripts again(at the expense of some
additional memory).There have been some feature requests to be able to create separate
OpCaches anyway. Shared web hosters would like to have different OpCaches
for different sites, but this would require a new INI directive which is
too late to add for 7, and for machines that run different PHP builds.This patch passes my functional, stress and performance tests.
I've added it to the PR 1536:
https://github.com/php/php-src/commit/7cbebc747e5c5f824a6e0336cc044ce4dc4c8f9fRegards
-MOps, I forgot to revert the exit condition.
http://git.php.net/?p=php-src.git;a=commitdiff;h=2d55e8c186ef1034c2af64478da8f23dbeb28be9
hi Matt,
Thanks.
I also moved the exit condition to be beforeSleep().
This should prevent race condition on last iteration (after unlocked
sleep process should try to reattach).
Committed to PHP-7.0 and master.http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454
If this really works, we should backport this into php-5.* as well.
Thanks. Dmitry.
On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken themattficken@gmail.com
wrote:Ok, I have a new PR just to unlock around
Sleep():
https://github.com/php/php-src/pull/1536Just by unlocking around
Sleep()the OpenFileMapping loop will work.
Holding the lock in that loop will not only block detach_segments() in
other PHP processes but also reattaching in accel_startup().Checking the base address used when this issue occurs during some of my
stress testing, most of the time the base address used is the first in
vista_mapping_base_set. Its typically the timing between the locking and
mapped file operations between processes more often than the base address
selection.Especially ASLR, also PECL DLLs, and other factors will still cause
this issue occasionally, but this fixes a majority of my occurrences.Regards
-MHi Matt,
It looks like with your patch, the same file may be mapped to
different virtual addresses of different processes and this won't work.
I think usage of vista_mapping_base_set[] in your patch is wrong.Do I understand properly, that zend_shared_alloc_unlock/lock_win32()
aroundSleep()is the main part of the patch?Please, next time, try to make PR diffs as small as possible (without
messing real fix with formatting and rearranging changes).Thanks. Dmitry.
On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken <themattficken@gmail.com
wrote:
I want to increase visibility for my PR 1531,
https://github.com/php/php-src/pull/1531, my patch for fixing an
intermittent OpCache issue on Windows.Details, etc... are on the PR.
Regards
-M
Hi Matt,
Thanks for looking into the problem, but I afraid that the proposed
solution won't solve the problem completely, but may introduce new
problems. At first, we might to reserve N times more memory. Then
different
SHM segments might contain different data.opcache_reset(),
opcache_invalidate()and others will affect only one segment.I think the perfect solution would require converting all stored data in
position independent format, but this hardly ever is going to be done even
for php-8.We may fallback to use file-cache introduced in php-7. (this should work
out of the box or with minimal changes).Also. if we can't map SHM into desired address space, we may map it in any
other address and copy data into the process memory similar to file-cache.
And what wincache does. It is slower but the request is served.
Finally we may try to use /DYNAMICBASE:NO
And what wincache does. It is slower but the request is served.
WinCache (file cache) if it can't reattach, creates a new shared mem file
for just that process: see
http://svn.php.net/viewvc/pecl/wincache/trunk/wincache_filemap.c?revision=336846&view=markup
Line 1122.
Yes, ideally it would be position independent, maybe using based pointers
for the structs in zend_smm_shared_globals, zend_accel_shared_globals,
etc....
I agree my bugfix is suboptimal but its the least complex option to get it
fixed for users in 7, so their requests get served.
My bugfix would increase memory usage, but so does a 2nd WinCache(or 3rd).
But at least if the 2nd OpCache is shared (the same procedure as if no
OpCache existed) its less likely to create a 3rd OpCache. For me at least,
often several processes in a row have this issue, so could have a 3rd or
4th OpCache or WinCache(even more memory), possibly very quickly.
Creating a 2nd OpCache, if its shared or per-process (like WinCache) seems
to be the simpler workaround. The trade-off is memory(especially if
per-process), time to recompile scripts, and sometimes breaking
opcache_reset(), opcache_invalidate() and stat cache pages(shared or
per-process OpCache).
Using the file cache should be better(won't slow down to recompile
scripts). But would still either require changing
zend_shared_alloc_startup() and other places because create_segments()
would still be returning ALLOC_FAILURE, OR creating a 2nd shared or
per-process OpCache anyway. Is this likely to be done? That would be great,
but it seems less likely because of the complexity.
Up to 6 OpCaches(5 backups) may be way too many (based off map_retries, 25
may be too many). My testing only ever gets a 2nd OpCache(never got a 3rd,
etc...).
There should be a workaround/suboptimal fix at least until someday when the
optimal solution is done, so requests get served.
@@ -74,18 +74,18 @@ static void zend_win_error_message(int type, char *msg,
int err)
zend_accel_error(type, msg);
}
-static char *create_name_with_username(char *name)
+static char *create_name_with_username(char *name, int num)
{
static char newname[MAXPATHLEN + UNLEN + 4];
char uname[UNLEN + 1];
DWORD unsize = UNLEN;
GetUserName(uname, &unsize);
- snprintf(newname, sizeof(newname) - 1, "%s@%s", name, uname);
- snprintf(newname, sizeof(newname) - 1, "%s@%s%d", name, uname, num);
return newname;
}
-static char *get_mmap_base_file(void)
+static char *get_mmap_base_file(int num)
{
static char windir[MAXPATHLEN+UNLEN + 3 + sizeof("\\@")];
char uname[UNLEN + 1];
@@ -95,13 +95,13 @@ static char *get_mmap_base_file(void)
GetTempPath(MAXPATHLEN, windir);
GetUserName(uname, &unsize);
l = strlen(windir);
- snprintf(windir + l, sizeof(windir) - l - 1, "\%s@%s",
ACCEL_FILEMAP_BASE, uname);
- snprintf(windir + l, sizeof(windir) - l - 1, "\%s@%s",
ACCEL_FILEMAP_BASE, uname, num);
return windir;
}
void zend_shared_alloc_create_lock(void)
{
- memory_mutex = CreateMutex(NULL, FALSE,
create_name_with_username(ACCEL_MUTEX_NAME));
- memory_mutex = CreateMutex(NULL, FALSE,
create_name_with_username(ACCEL_MUTEX_NAME, 0));
if (!memory_mutex) {
zend_accel_error(ACCEL_LOG_FATAL, "Cannot create mutex");
return;
@@ -123,11 +123,11 @@ void zend_shared_alloc_unlock_win32(void)
ReleaseMutex(memory_mutex);
}
-static int zend_shared_alloc_reattach(size_t requested_size, char
**error_in)
+static int zend_shared_alloc_reattach(size_t requested_size, int num, char
**error_in)
{
int err;
void *wanted_mapping_base;
- char *mmap_base_file = get_mmap_base_file();
- char *mmap_base_file = get_mmap_base_file(num);
FILE *fp = fopen(mmap_base_file, "r");
MEMORY_BASIC_INFORMATION info;
@@ -194,24 +194,18 @@ static int create_segments(size_t requested_size,
zend_shared_segment **shared_
can be called before the child process is killed. In this case, the map
will fail
and we have to sleep some time (until the child releases the mapping
object) and retry./
do {
- memfile = OpenFileMapping(FILE_MAP_WRITE, 0,
create_name_with_username(ACCEL_FILEMAP_NAME));
- memfile = OpenFileMapping(FILE_MAP_WRITE, 0,
create_name_with_username(ACCEL_FILEMAP_NAME, map_retries>>2));
err = GetLastError();
if (memfile == NULL) {
break;
}
- ret = zend_shared_alloc_reattach(requested_size, error_in);
- ret = zend_shared_alloc_reattach(requested_size, map_retries>>2,
error_in);
err = GetLastError();
if (ret == ALLOC_FAIL_MAPPING) {
/* Mapping failed, wait for mapping object to get freed and retry */
@@ -241,7 +235,7 @@ static int create_segments(size_t requested_size,
zend_shared_segment ***shared_
(*shared_segments_p)[0] = shared_segment;
memfile = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0,
requested_size,
- create_name_with_username(ACCEL_FILEMAP_NAME));
- create_name_with_username(ACCEL_FILEMAP_NAME, map_retries>>2));
err = GetLastError();
if (memfile == NULL) {
zend_shared_alloc_unlock_win32();
@@ -306,7 +300,7 @@ static int create_segments(size_t requested_size,
zend_shared_segment ***shared_
*error_in = "MapViewOfFile";
return ALLOC_FAILURE;
} else {
- char *mmap_base_file = get_mmap_base_file();
- char *mmap_base_file = get_mmap_base_file(map_retries>>2);
FILE *fp = fopen(mmap_base_file, "w");
err = GetLastError();
if (!fp) {
On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:
And what wincache does. It is slower but the request is served.
WinCache (file cache) if it can't reattach, creates a new shared mem file
for just that process: see
http://svn.php.net/viewvc/pecl/wincache/trunk/wincache_filemap.c?revision=336846&view=markup
Line 1122.
I'm not sure how WinCache works, but opcache already implements most
necessary functionality. Actually we should just:
- If process can't map SHM - lets map it into different address and
disable default SHM (ZCG(accel_directives.file_cache_only)=1) - extended file_cache may look into improperly mapped segment before
reading it from file.
But ff course this is more difficult.
Yes, ideally it would be position independent, maybe using based pointers
for the structs in zend_smm_shared_globals, zend_accel_shared_globals,
etc....I agree my bugfix is suboptimal but its the least complex option to get it
fixed for users in 7, so their requests get served.My bugfix would increase memory usage, but so does a 2nd WinCache(or 3rd).
But at least if the 2nd OpCache is shared (the same procedure as if no
OpCache existed) its less likely to create a 3rd OpCache. For me at least,
often several processes in a row have this issue, so could have a 3rd or
4th OpCache or WinCache(even more memory), possibly very quickly.
Your proposal is better than WinCache.
Creating a 2nd OpCache, if its shared or per-process (like WinCache) seems
to be the simpler workaround. The trade-off is memory(especially if
per-process), time to recompile scripts, and sometimes breaking
opcache_reset(),opcache_invalidate()and stat cache pages(shared or
per-process OpCache).
This may break application logic. In worst case different SHM may keep
different versons of the same script...
Using the file cache should be better(won't slow down to recompile
scripts). But would still either require changing
zend_shared_alloc_startup() and other places because create_segments()
would still be returning ALLOC_FAILURE, OR creating a 2nd shared or
per-process OpCache anyway. Is this likely to be done? That would be great,
but it seems less likely because of the complexity.
This may be changed in some way.
Up to 6 OpCaches(5 backups) may be way too many (based off map_retries, 25
may be too many). My testing only ever gets a 2nd OpCache(never got a 3rd,
etc...).
6 may be not enough anyway. If we have a limit it's going to be exceeded.
MS will improve their ASLR once again... :)
There should be a workaround/suboptimal fix at least until someday when
the optimal solution is done, so requests get served.
In my experience, temporary incomplete solutions are almost never replaced
with general ones.
New and new incomplete fixes are added on top instead.
Anyway Anatol, if you have time, can you please play with patch and give
your opinion.
Thanks. Dmitry.
@@ -74,18 +74,18 @@ static void zend_win_error_message(int type, char
*msg, int err)
zend_accel_error(type, msg);
}
-static char *create_name_with_username(char *name)
+static char *create_name_with_username(char *name, int num)
{
static char newname[MAXPATHLEN + UNLEN + 4];
char uname[UNLEN + 1];
DWORD unsize = UNLEN;
GetUserName(uname, &unsize);
- snprintf(newname, sizeof(newname) - 1, "%s@%s", name, uname);
- snprintf(newname, sizeof(newname) - 1, "%s@%s%d", name, uname, num);
return newname;
}
-static char *get_mmap_base_file(void)
+static char *get_mmap_base_file(int num)
{
static char windir[MAXPATHLEN+UNLEN + 3 + sizeof("\\@")];
char uname[UNLEN + 1];
@@ -95,13 +95,13 @@ static char *get_mmap_base_file(void)
GetTempPath(MAXPATHLEN, windir);
GetUserName(uname, &unsize);
l = strlen(windir);
- snprintf(windir + l, sizeof(windir) - l - 1, "\%s@%s",
ACCEL_FILEMAP_BASE, uname);
- snprintf(windir + l, sizeof(windir) - l - 1, "\%s@%s",
ACCEL_FILEMAP_BASE, uname, num);
return windir;
}
void zend_shared_alloc_create_lock(void)
{
- memory_mutex = CreateMutex(NULL, FALSE,
create_name_with_username(ACCEL_MUTEX_NAME));
- memory_mutex = CreateMutex(NULL, FALSE,
create_name_with_username(ACCEL_MUTEX_NAME, 0));
if (!memory_mutex) {
zend_accel_error(ACCEL_LOG_FATAL, "Cannot create mutex");
return;
@@ -123,11 +123,11 @@ void zend_shared_alloc_unlock_win32(void)
ReleaseMutex(memory_mutex);
}
-static int zend_shared_alloc_reattach(size_t requested_size, char
**error_in)
+static int zend_shared_alloc_reattach(size_t requested_size, int num,
char **error_in)
{
int err;
void *wanted_mapping_base;
- char *mmap_base_file = get_mmap_base_file();
- char *mmap_base_file = get_mmap_base_file(num);
FILE *fp = fopen(mmap_base_file, "r");
MEMORY_BASIC_INFORMATION info;
@@ -194,24 +194,18 @@ static int create_segments(size_t requested_size,
zend_shared_segment **shared_
can be called before the child process is killed. In this case, the map
will fail
and we have to sleep some time (until the child releases the mapping
object) and retry./
do {
- memfile = OpenFileMapping(FILE_MAP_WRITE, 0,
create_name_with_username(ACCEL_FILEMAP_NAME));
- memfile = OpenFileMapping(FILE_MAP_WRITE, 0,
create_name_with_username(ACCEL_FILEMAP_NAME, map_retries>>2));
err = GetLastError();
if (memfile == NULL) {
break;
}
- ret = zend_shared_alloc_reattach(requested_size, error_in);
- ret = zend_shared_alloc_reattach(requested_size, map_retries>>2,
error_in);
err = GetLastError();
if (ret == ALLOC_FAIL_MAPPING) {
/* Mapping failed, wait for mapping object to get freed and retry */
@@ -241,7 +235,7 @@ static int create_segments(size_t requested_size,
zend_shared_segment ***shared_
(*shared_segments_p)[0] = shared_segment;
memfile = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0,
requested_size,
- create_name_with_username(ACCEL_FILEMAP_NAME));
- create_name_with_username(ACCEL_FILEMAP_NAME, map_retries>>2));
err = GetLastError();
if (memfile == NULL) {
zend_shared_alloc_unlock_win32();
@@ -306,7 +300,7 @@ static int create_segments(size_t requested_size,
zend_shared_segment ***shared_
*error_in = "MapViewOfFile";
return ALLOC_FAILURE;
} else {
- char *mmap_base_file = get_mmap_base_file();
- char *mmap_base_file = get_mmap_base_file(map_retries>>2);
FILE *fp = fopen(mmap_base_file, "w");
err = GetLastError();
if (!fp) {
Hi,
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Thursday, October 1, 2015 12:20 PM
To: Matt Ficken themattficken@gmail.com
Cc: Pierre Joye pierre.php@gmail.com; Anatoliy Belsky ab@php.net;
Laruence laruence@php.net; PHP Internals internals@lists.php.net;
dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixOn Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:And what wincache does. It is slower but the request is served.
WinCache (file cache) if it can't reattach, creates a new shared mem
file
for just that process: see
http://svn.php.net/viewvc/pecl/wincache/trunk/wincache_filemap.c?revis
ion=336846&view=markup
Line 1122.I'm not sure how WinCache works, but opcache already implements most
necessary functionality. Actually we should just:
- If process can't map SHM - lets map it into different address and disable
default SHM (ZCG(accel_directives.file_cache_only)=1)- extended file_cache may look into improperly mapped segment before
reading it from file.
But ff course this is more difficult.Yes, ideally it would be position independent, maybe using based
pointers for the structs in zend_smm_shared_globals,
zend_accel_shared_globals, etc....I agree my bugfix is suboptimal but its the least complex option to
get it fixed for users in 7, so their requests get served.My bugfix would increase memory usage, but so does a 2nd WinCache(or 3rd).
But at least if the 2nd OpCache is shared (the same procedure as if no
OpCache existed) its less likely to create a 3rd OpCache. For me at
least, often several processes in a row have this issue, so could have
a 3rd or 4th OpCache or WinCache(even more memory), possibly very quickly.Your proposal is better than WinCache.
Creating a 2nd OpCache, if its shared or per-process (like WinCache)
seems to be the simpler workaround. The trade-off is memory(especially
if per-process), time to recompile scripts, and sometimes breaking
opcache_reset(),opcache_invalidate()and stat cache pages(shared or
per-process OpCache).This may break application logic. In worst case different SHM may keep different
versons of the same script...Using the file cache should be better(won't slow down to recompile
scripts). But would still either require changing
zend_shared_alloc_startup() and other places because create_segments()
would still be returning ALLOC_FAILURE, OR creating a 2nd shared or
per-process OpCache anyway. Is this likely to be done? That would be
great, but it seems less likely because of the complexity.This may be changed in some way.
Up to 6 OpCaches(5 backups) may be way too many (based off
map_retries, 25 may be too many). My testing only ever gets a 2nd
OpCache(never got a 3rd, etc...).6 may be not enough anyway. If we have a limit it's going to be exceeded.
MS will improve their ASLR once again... :)There should be a workaround/suboptimal fix at least until someday
when the optimal solution is done, so requests get served.In my experience, temporary incomplete solutions are almost never replaced
with general ones.
New and new incomplete fixes are added on top instead.
Anyway Anatol, if you have time, can you please play with patch and give your
opinion.
Yeah, I'll be testing that as well anyway, now that RC4 is out. I was intended also to do some synthetic tests, because seems some (especially 32-bit) environments are affected more than others. Namely, I'm trying hard to get this issue on 64-bit with no chance.
IMHO creating the sidestep caches looks not convincing. From the issues mentioned, like different cached script versions, but also indeed taking away the control over memory limit, etc. Even one would decide for that, the other mentioned parts would need to be fixed. Is that something achievable to fix in the current state?
Dmitry, I'd have a question to this
Also. if we can't map SHM into desired address space, we may map it in any
other address and copy data into the process memory similar to file-cache.
In randomized memory layout, even the base were available and OpenFileMapping were worky, some parts of that memory were already taken. How exactly it could be done, could you please give a couple of pointers to this? Would the file cache be always required then?
Regards
Anatol
Hi,
On Thu, Oct 1, 2015 at 11:38 PM, Anatol Belski anatol.php@belski.net
wrote:
Hi,
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Thursday, October 1, 2015 12:20 PM
To: Matt Ficken themattficken@gmail.com
Cc: Pierre Joye pierre.php@gmail.com; Anatoliy Belsky ab@php.net;
Laruence laruence@php.net; PHP Internals internals@lists.php.net;
dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixOn Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:And what wincache does. It is slower but the request is served.
WinCache (file cache) if it can't reattach, creates a new shared mem
file
for just that process: see
http://svn.php.net/viewvc/pecl/wincache/trunk/wincache_filemap.c?revis
ion=336846&view=markup
Line 1122.I'm not sure how WinCache works, but opcache already implements most
necessary functionality. Actually we should just:
- If process can't map SHM - lets map it into different address and
disable
default SHM (ZCG(accel_directives.file_cache_only)=1)- extended file_cache may look into improperly mapped segment before
reading it from file.
But ff course this is more difficult.Yes, ideally it would be position independent, maybe using based
pointers for the structs in zend_smm_shared_globals,
zend_accel_shared_globals, etc....I agree my bugfix is suboptimal but its the least complex option to
get it fixed for users in 7, so their requests get served.My bugfix would increase memory usage, but so does a 2nd WinCache(or
3rd).
But at least if the 2nd OpCache is shared (the same procedure as if no
OpCache existed) its less likely to create a 3rd OpCache. For me at
least, often several processes in a row have this issue, so could have
a 3rd or 4th OpCache or WinCache(even more memory), possibly very
quickly.Your proposal is better than WinCache.
Creating a 2nd OpCache, if its shared or per-process (like WinCache)
seems to be the simpler workaround. The trade-off is memory(especially
if per-process), time to recompile scripts, and sometimes breaking
opcache_reset(),opcache_invalidate()and stat cache pages(shared or
per-process OpCache).This may break application logic. In worst case different SHM may keep
different
versons of the same script...Using the file cache should be better(won't slow down to recompile
scripts). But would still either require changing
zend_shared_alloc_startup() and other places because create_segments()
would still be returning ALLOC_FAILURE, OR creating a 2nd shared or
per-process OpCache anyway. Is this likely to be done? That would be
great, but it seems less likely because of the complexity.This may be changed in some way.
Up to 6 OpCaches(5 backups) may be way too many (based off
map_retries, 25 may be too many). My testing only ever gets a 2nd
OpCache(never got a 3rd, etc...).6 may be not enough anyway. If we have a limit it's going to be exceeded.
MS will improve their ASLR once again... :)There should be a workaround/suboptimal fix at least until someday
when the optimal solution is done, so requests get served.In my experience, temporary incomplete solutions are almost never
replaced
with general ones.
New and new incomplete fixes are added on top instead.
Anyway Anatol, if you have time, can you please play with patch and give
your
opinion.Yeah, I'll be testing that as well anyway, now that RC4 is out. I was
intended also to do some synthetic tests, because seems some (especially
32-bit) environments are affected more than others. Namely, I'm trying hard
to get this issue on 64-bit with no chance.IMHO creating the sidestep caches looks not convincing. From the issues
mentioned, like different cached script versions, but also indeed taking
away the control over memory limit, etc. Even one would decide for that,
the other mentioned parts would need to be fixed. Is that something
achievable to fix in the current state?Dmitry, I'd have a question to this
Also. if we can't map SHM into desired address space, we may map it in
any
other address and copy data into the process memory similar to
file-cache.
In randomized memory layout, even the base were available and
OpenFileMapping were worky, some parts of that memory were already taken.
How exactly it could be done, could you please give a couple of pointers to
this?
If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
MapViewOfFileEx(..., NULL).
Would the file cache be always required then?
This is not necessary, but depends on implementation of course.
Thanks. Dmitry.
Regards
Anatol
Hi Dmitry,
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Friday, October 2, 2015 9:34 AM
To: Anatol Belski anatol.php@belski.net
Cc: Matt Ficken themattficken@gmail.com; Pierre Joye
pierre.php@gmail.com; Anatoliy Belsky ab@php.net; Laruence
laruence@php.net; PHP Internals internals@lists.php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixDmitry, I'd have a question to this
Also. if we can't map SHM into desired address space, we may map it
in
any
other address and copy data into the process memory similar to
file-cache.
In randomized memory layout, even the base were available and
OpenFileMapping were worky, some parts of that memory were already
taken.
How exactly it could be done, could you please give a couple of
pointers to this?If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
MapViewOfFileEx(..., NULL).Would the file cache be always required then?
This is not necessary, but depends on implementation of course.
Thanks for the advice. I was playing with this over these days. There are two usual cases where it possibly fails when reattaching ATM
- https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L151 - the saved address is available but is not suitable
- https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L159 - the actual MapViewOfFileEx case
An unrelated plain C test shows, that MapViewOfFileEx can possibly fail when called second time, too. Even with NULL or with another address as base. But even if it could map at a different base - the internal structures will probably have invalid addresses. So it looks like there's indeed no profit to do any retry once the actualy base address needed was failed to reattach to.
IMHO the scenario that were usable - in case it fails to reattach to the exact address, it has to switch to heap. Just for one request, it should get a heap allocated segment and then invalidate all the cache. That way we fulfill the following
- the request is served for sure
- the scripts are not cached indeed, so no divergence with the actual real cache
A heap fallback memory handler would be probably quite easy to implement. What do you think?
Apropos with the heap - it also looks like when PHP is used as module under mpm_winnt, all the cache could use heap instead of SHM. In that case, there is only one Apache process serving with many threads. Except one would want to share that cache outside Apache, using heap there could be a much simpler solution.
Regards
Anatol
On Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski anatol.php@belski.net
wrote:
Hi Dmitry,
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Friday, October 2, 2015 9:34 AM
To: Anatol Belski anatol.php@belski.net
Cc: Matt Ficken themattficken@gmail.com; Pierre Joye
pierre.php@gmail.com; Anatoliy Belsky ab@php.net; Laruence
laruence@php.net; PHP Internals internals@lists.php.net;
dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixDmitry, I'd have a question to this
Also. if we can't map SHM into desired address space, we may map it
in
any
other address and copy data into the process memory similar to
file-cache.
In randomized memory layout, even the base were available and
OpenFileMapping were worky, some parts of that memory were already
taken.
How exactly it could be done, could you please give a couple of
pointers to this?If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
MapViewOfFileEx(..., NULL).Would the file cache be always required then?
This is not necessary, but depends on implementation of course.
Thanks for the advice. I was playing with this over these days. There are
two usual cases where it possibly fails when reattaching ATM
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L151
- the saved address is available but is not suitable
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_win32.c#L159
- the actual MapViewOfFileEx case
An unrelated plain C test shows, that MapViewOfFileEx can possibly fail
when called second time, too. Even withNULLor with another address as
base. But even if it could map at a different base - the internal
structures will probably have invalid addresses.
right. we might need different code for zend_accle_hash access or convert
corresponding structures to PIC.
For opcdeos "invalid address" don't matter because we will copy them into
process memory and fix (like in file-cache).
So it looks like there's indeed no profit to do any retry once the actualy
base address needed was failed to reattach to.IMHO the scenario that were usable - in case it fails to reattach to the
exact address, it has to switch to heap. Just for one request, it should
get a heap allocated segment and then invalidate all the cache. That way we
fulfill the following
- the request is served for sure
- the scripts are not cached indeed, so no divergence with the actual real
cacheA heap fallback memory handler would be probably quite easy to implement.
What do you think?Apropos with the heap - it also looks like when PHP is used as module
under mpm_winnt, all the cache could use heap instead of SHM. In that case,
there is only one Apache process serving with many threads. Except one
would want to share that cache outside Apache, using heap there could be a
much simpler solution.
Heap based cache makes the same problems. It increase the memory usage and
doesn't provide cache consistency.
Just fall-back to file-cache may be better.
Actually, when I implemented file-cache I had a though about different
storage back-ends (e.g. sqlite or memcache).
This abstraction might make support for "improperly mapped SHM" quite easy.
Thanks. Dmitry.
Regards
Anatol
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Monday, October 5, 2015 1:08 PM
To: Anatol Belski anatol.php@belski.net
Cc: Matt Ficken themattficken@gmail.com; Pierre Joye
pierre.php@gmail.com; Laruence laruence@php.net; PHP Internals
internals@lists.php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixOn Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski anatol.php@belski.net
wrote:Hi Dmitry,
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Friday, October 2, 2015 9:34 AM
To: Anatol Belski anatol.php@belski.net
Cc: Matt Ficken themattficken@gmail.com; Pierre Joye
pierre.php@gmail.com; Anatoliy Belsky ab@php.net; Laruence
laruence@php.net; PHP Internals internals@lists.php.net;
dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixDmitry, I'd have a question to this
Also. if we can't map SHM into desired address space, we may map
it in
any
other address and copy data into the process memory similar to
file-cache.
In randomized memory layout, even the base were available and
OpenFileMapping were worky, some parts of that memory were already
taken.
How exactly it could be done, could you please give a couple of
pointers to this?If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
MapViewOfFileEx(..., NULL).Would the file cache be always required then?
This is not necessary, but depends on implementation of course.
Thanks for the advice. I was playing with this over these days. There
are two usual cases where it possibly fails when reattaching ATM
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
in32.c#L151
- the saved address is available but is not suitable
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
in32.c#L159
- the actual MapViewOfFileEx case
An unrelated plain C test shows, that MapViewOfFileEx can possibly
fail when called second time, too. Even withNULLor with another
address as base. But even if it could map at a different base - the
internal structures will probably have invalid addresses.right. we might need different code for zend_accle_hash access or convert
corresponding structures to PIC.
For opcdeos "invalid address" don't matter because we will copy them into
process memory and fix (like in file-cache).
Ah, I have to study the file cache code then. But generally it sounds not like something that can be done offhand. I've also thought about other things like interned strings (maybe something else), not sure they're stored with the cache.
So it looks like there's indeed no profit to do any retry once the
actualy base address needed was failed to reattach to.IMHO the scenario that were usable - in case it fails to reattach to
the exact address, it has to switch to heap. Just for one request, it
should get a heap allocated segment and then invalidate all the cache.
That way we fulfill the following
- the request is served for sure
- the scripts are not cached indeed, so no divergence with the actual
real cacheA heap fallback memory handler would be probably quite easy to implement.
What do you think?Apropos with the heap - it also looks like when PHP is used as module
under mpm_winnt, all the cache could use heap instead of SHM. In that
case, there is only one Apache process serving with many threads.
Except one would want to share that cache outside Apache, using heap
there could be a much simpler solution.Heap based cache makes the same problems. It increase the memory usage and
doesn't provide cache consistency.
Just fall-back to file-cache may be better.
Do you think something like this would suffice as file cache fallback https://gist.github.com/weltling/224001a468f04de13693 ? Though it'd still diverge from the "main" cache.
Actually in such case all the processes should switch to file cache? Just not sure how they all would negotiate that when no SHM is available (probably through files, or a separate shared chunk).
Actually, when I implemented file-cache I had a though about different storage
back-ends (e.g. sqlite or memcache).
This abstraction might make support for "improperly mapped SHM" quite easy.
Yeah, that could be useful. Maybe also module based so one could even supply them separately, then an arbitrary storage mechanism could be provided. Fe like I've mentioned for mpm_winnt -if there's no intention to share the cache outside Apache, just one heap for all could be much simpler to avoid all that reattach mechanics.
Thanks.
Anatol
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Monday, October 5, 2015 1:08 PM
To: Anatol Belski anatol.php@belski.net
Cc: Matt Ficken themattficken@gmail.com; Pierre Joye
pierre.php@gmail.com; Laruence laruence@php.net; PHP Internals
internals@lists.php.net; dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixOn Mon, Oct 5, 2015 at 10:29 AM, Anatol Belski anatol.php@belski.net
wrote:Hi Dmitry,
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Friday, October 2, 2015 9:34 AM
To: Anatol Belski anatol.php@belski.net
Cc: Matt Ficken themattficken@gmail.com; Pierre Joye
pierre.php@gmail.com; Anatoliy Belsky ab@php.net; Laruence
laruence@php.net; PHP Internals internals@lists.php.net;
dmitry@php.net
Subject: Re: [PHP-DEV] Re: Windows OpCache bug fixDmitry, I'd have a question to this
Also. if we can't map SHM into desired address space, we may map
it in
any
other address and copy data into the process memory similar to
file-cache.
In randomized memory layout, even the base were available and
OpenFileMapping were worky, some parts of that memory were already
taken.
How exactly it could be done, could you please give a couple of
pointers to this?If MapViewOfFileEx(..., wanted_mapping_base) fails, we do
MapViewOfFileEx(..., NULL).Would the file cache be always required then?
This is not necessary, but depends on implementation of course.
Thanks for the advice. I was playing with this over these days. There
are two usual cases where it possibly fails when reattaching ATM
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
in32.c#L151
- the saved address is available but is not suitable
https://github.com/php/php-src/blob/PHP-7.0/ext/opcache/shared_alloc_w
in32.c#L159
- the actual MapViewOfFileEx case
An unrelated plain C test shows, that MapViewOfFileEx can possibly
fail when called second time, too. Even withNULLor with another
address as base. But even if it could map at a different base - the
internal structures will probably have invalid addresses.right. we might need different code for zend_accle_hash access or convert
corresponding structures to PIC.
For opcdeos "invalid address" don't matter because we will copy them into
process memory and fix (like in file-cache).Ah, I have to study the file cache code then. But generally it sounds not
like something that can be done offhand. I've also thought about other
things like interned strings (maybe something else), not sure they're
stored with the cache.So it looks like there's indeed no profit to do any retry once the
actualy base address needed was failed to reattach to.IMHO the scenario that were usable - in case it fails to reattach to
the exact address, it has to switch to heap. Just for one request, it
should get a heap allocated segment and then invalidate all the cache.
That way we fulfill the following
- the request is served for sure
- the scripts are not cached indeed, so no divergence with the actual
real cacheA heap fallback memory handler would be probably quite easy to
implement.
What do you think?Apropos with the heap - it also looks like when PHP is used as module
under mpm_winnt, all the cache could use heap instead of SHM. In that
case, there is only one Apache process serving with many threads.
Except one would want to share that cache outside Apache, using heap
there could be a much simpler solution.Heap based cache makes the same problems. It increase the memory usage
and
doesn't provide cache consistency.
Just fall-back to file-cache may be better.
Do you think something like this would suffice as file cache fallback
https://gist.github.com/weltling/224001a468f04de13693 ? Though it'd
still diverge from the "main" cache.
I think we should enable file-cache automatically, but we can set
ZCG(accel_directives).file_cache_only
if the file_cache already enabled.
Actually in such case all the processes should switch to file cache?
No. Only the processes that weren't be able to attach to SHM.
Just not sure how they all would negotiate that when no SHM is available
(probably through files, or a separate shared chunk).
yeah. Processes that use file-cache-only won't be able to negotiate SHM
cache. :(
Actually, when I implemented file-cache I had a though about different
storage
back-ends (e.g. sqlite or memcache).
This abstraction might make support for "improperly mapped SHM" quite
easy.Yeah, that could be useful. Maybe also module based so one could even
supply them separately, then an arbitrary storage mechanism could be
provided. Fe like I've mentioned for mpm_winnt -if there's no intention to
share the cache outside Apache, just one heap for all could be much simpler
to avoid all that reattach mechanics.
Does mpm_winnt use pure ZTS without processes?
Thanks. Dmitry.
Thanks.
Anatol
From: Dmitry Stogov [mailto:dmitry@zend.com]
On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:Pierre Joy wrote:
And what wincache does. It is slower but the request is served.WinCache (file cache) if it can't reattach, creates a new shared mem
fileI'm not sure how WinCache works, but opcache already implements most
necessary functionality.
Sorry for getting to the party late, but let me clarify what WinCache does:
- For the opcode cache, if WinCache can't map to the same address, it falls back to using process-local (heap) memory for the opcode arrays. This is because the elements in the opcode arrays contain absolute pointers, and we don't want to spend cycles (a) converting all pointers to offsets when copying opcode arrays into shared memory, and then (b) converting all offsets back to absolute addresses when copying the opcode array back out of shared memory. WinCache simply gets the pointer to the opcode array in shared memory and returns it, unmodified, to the PHP Core for execution.
When WinCache falls back to process-local memory, it does NOT create a second memory mapped segment, and does not chew additional cross-process resources.
-
WinCache's file cache does not map to an identical address in all processes. We simply map it to any address (using
NULLfor the suggested address to MapViewOfFileEx), and then use relative offsets in the cache structures for all entries in shared memory. -
WinCache's user cache and session cache are similar to the file cache, and do not have to map to the same address in all processes. Cache entries use relative offsets to locate cache entries, and the values do not contain absolute addresses.
Finally, For PHP7, WinCache has removed its opcode cache. We assume that Windows consumers of PHP7 will use Zend Opcache for opcode caching.
Thx!
--E.
Ultimately if the 2nd OpCache is shared in a new SHM, in-process in a new
SHM or in-process Heap, memory will increase, scripts have to be recompiled
and there are consistency issues. File-cache will avoid recompiling
scripts, but unless OpCache is disabled, there needs to be a 2nd OpCache
stored somewhere.
For the stat, reset and revalidate consistency issues, to build off
Anatol's idea: If can't reattach to base address, then reattach to any
address and create a 2nd OpCache in Heap, BUT keep the SHM open so that the
stat fields(fe ZCSG(hits)) are still accessible. Those fields aren't
pointers so they should work from whatever base address MapViewOfFileEx()
provides, if not, could fork ZCSG into 2 structs (somehow reuse existing
stat fields on existing SHM).
opcache_get_status() in any/all processes would read those fields from SHM.
Since there would be 2+ processes with an OpCache (the original in SHM, and
sidestep OpCaches on the heaps of those processes), can't use a simple flag
(would be cleared by 2nd process, before 3rd got it). Could add
ZCSG(opcache_count) field and increment every time an OpCache gets created
(and decrement on process exit). Simple, but not using
zend_shared_alloc_lock() avoids creating new locking issues.
Add a new ZCSG(opcache_revalidate_count) in that struct that is set to
ZCSG(opcache_count) anytime opcache_reset() or opcache_invalidate() called
or if persistent_compile_file() finds a changed script.
In process with side-step OpCache, in persistent_compile_file() (assuming
opcache.validate_timestamps && revalidate_freq>0) will check if
ZCSG(opcache_revalidate_count) > 0 and will then decrement the count and
reload scripts from file-cache (where it checks opcache.revalidate_freq).
And then wrap this all in some #ifdef ZEND_WIN32s
Would still have extra memory usage. When I have a process that can't
reattach I usually get several, so having an in-process OpCache(SHM or
Heap) may lead to more memory usage than a shared SHM, but having 2 SHMs in
a process would mess up the nice existing code.
Don't have to recompile scripts(quickly loads from file-cache) and fixes
the 3 consistency issues, and would keep serving up requests without
performance loss.
Yes, temporary incomplete solutions often get forgotten and not replaced
later with a better complete solution, but in this case we're motivated try
to fully fix this whole issue and if that doesn't work, at least build a
more complete solution, but that takes time (known issue for already a few
years), we should do something that works for now. This is more complex
than a bugfix that creates a 2nd SHM OpCache. Can we do this now? What is
still possible for 7.0? 7.0.1?
On Mon, Oct 5, 2015 at 11:49 AM, Eric Stenson ericsten@microsoft.com
wrote:
From: Dmitry Stogov [mailto:dmitry@zend.com]
On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:Pierre Joy wrote:
And what wincache does. It is slower but the request is served.WinCache (file cache) if it can't reattach, creates a new shared mem
fileI'm not sure how WinCache works, but opcache already implements most
necessary functionality.Sorry for getting to the party late, but let me clarify what WinCache does:
- For the opcode cache, if WinCache can't map to the same address, it
falls back to using process-local (heap) memory for the opcode arrays.
This is because the elements in the opcode arrays contain absolute
pointers, and we don't want to spend cycles (a) converting all pointers to
offsets when copying opcode arrays into shared memory, and then (b)
converting all offsets back to absolute addresses when copying the opcode
array back out of shared memory. WinCache simply gets the pointer to the
opcode array in shared memory and returns it, unmodified, to the PHP Core
for execution.When WinCache falls back to process-local memory, it does NOT create a
second memory mapped segment, and does not chew additional cross-process
resources.
WinCache's file cache does not map to an identical address in all
processes. We simply map it to any address (usingNULLfor the suggested
address to MapViewOfFileEx), and then use relative offsets in the cache
structures for all entries in shared memory.WinCache's user cache and session cache are similar to the file cache,
and do not have to map to the same address in all processes. Cache entries
use relative offsets to locate cache entries, and the values do not contain
absolute addresses.Finally, For PHP7, WinCache has removed its opcode cache. We assume that
Windows consumers of PHP7 will use Zend Opcache for opcode caching.Thx!
--E.
Ultimately if the 2nd OpCache is shared in a new SHM, in-process in a new
SHM or in-process Heap, memory will increase, scripts have to be recompiled
and there are consistency issues. File-cache will avoid recompiling
scripts, but unless OpCache is disabled, there needs to be a 2nd OpCache
stored somewhere.For the stat, reset and revalidate consistency issues, to build off
Anatol's idea: If can't reattach to base address, then reattach to any
address and create a 2nd OpCache in Heap, BUT keep the SHM open so that the
stat fields(fe ZCSG(hits)) are still accessible. Those fields aren't
pointers so they should work from whatever base address MapViewOfFileEx()
provides, if not, could fork ZCSG into 2 structs (somehow reuse existing
stat fields on existing SHM).
opcache_get_status()in any/all processes would read those fields from SHM.Since there would be 2+ processes with an OpCache (the original in SHM,
and sidestep OpCaches on the heaps of those processes), can't use a simple
flag (would be cleared by 2nd process, before 3rd got it). Could add
ZCSG(opcache_count) field and increment every time an OpCache gets created
(and decrement on process exit). Simple, but not using
zend_shared_alloc_lock() avoids creating new locking issues.Add a new ZCSG(opcache_revalidate_count) in that struct that is set to
ZCSG(opcache_count) anytimeopcache_reset()oropcache_invalidate()called
or if persistent_compile_file() finds a changed script.
This way work. Actually, it's even possible clean opcoache SHM completely
or invalidate individual files with SHM mapped into different address. Of
course it shpuld be implemented, and I don;t know all the problems yet.
In process with side-step OpCache, in persistent_compile_file()
(assuming opcache.validate_timestamps && revalidate_freq>0) will check if
ZCSG(opcache_revalidate_count) > 0 and will then decrement the count and
reload scripts from file-cache (where it checks opcache.revalidate_freq).
And then wrap this all in some#ifdef ZEND_WIN32sWould still have extra memory usage. When I have a process that can't
reattach I usually get several, so having an in-process OpCache(SHM or
Heap) may lead to more memory usage than a shared SHM, but having 2 SHMs in
a process would mess up the nice existing code.
What extra memory? Do you mean in-process memory for op_arrays during
request processing?
Don't have to recompile scripts(quickly loads from file-cache) and fixes
the 3 consistency issues, and would keep serving up requests without
performance loss.Yes, temporary incomplete solutions often get forgotten and not replaced
later with a better complete solution, but in this case we're motivated try
to fully fix this whole issue and if that doesn't work, at least build a
more complete solution, but that takes time (known issue for already a few
years), we should do something that works for now. This is more complex
than a bugfix that creates a 2nd SHM OpCache. Can we do this now? What is
still possible for 7.0? 7.0.1?
If we understand each other properly, It looks like a more or less
consistent solution now.
This is going to be a self-contained change, so I don't see big problems
releasing some fix for PHP-7.0 branch even after GA 7.0.0 release.
Of course, we may make final decision only after complete implementation.
Thanks. Dmitry.
On Mon, Oct 5, 2015 at 11:49 AM, Eric Stenson ericsten@microsoft.com
wrote:From: Dmitry Stogov [mailto:dmitry@zend.com]
On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:Pierre Joy wrote:
And what wincache does. It is slower but the request is served.WinCache (file cache) if it can't reattach, creates a new shared mem
fileI'm not sure how WinCache works, but opcache already implements most
necessary functionality.Sorry for getting to the party late, but let me clarify what WinCache
does:
- For the opcode cache, if WinCache can't map to the same address, it
falls back to using process-local (heap) memory for the opcode arrays.
This is because the elements in the opcode arrays contain absolute
pointers, and we don't want to spend cycles (a) converting all pointers to
offsets when copying opcode arrays into shared memory, and then (b)
converting all offsets back to absolute addresses when copying the opcode
array back out of shared memory. WinCache simply gets the pointer to the
opcode array in shared memory and returns it, unmodified, to the PHP Core
for execution.When WinCache falls back to process-local memory, it does NOT create a
second memory mapped segment, and does not chew additional cross-process
resources.
WinCache's file cache does not map to an identical address in all
processes. We simply map it to any address (usingNULLfor the suggested
address to MapViewOfFileEx), and then use relative offsets in the cache
structures for all entries in shared memory.WinCache's user cache and session cache are similar to the file cache,
and do not have to map to the same address in all processes. Cache entries
use relative offsets to locate cache entries, and the values do not contain
absolute addresses.Finally, For PHP7, WinCache has removed its opcode cache. We assume that
Windows consumers of PHP7 will use Zend Opcache for opcode caching.Thx!
--E.
Hi Eric,
From: Dmitry Stogov [mailto:dmitry@zend.com]
On Thu, Oct 1, 2015 at 11:54 AM, Matt Ficken themattficken@gmail.com
wrote:Pierre Joy wrote:
And what wincache does. It is slower but the request is served.WinCache (file cache) if it can't reattach, creates a new shared mem
fileI'm not sure how WinCache works, but opcache already implements most
necessary functionality.Sorry for getting to the party late, but let me clarify what WinCache does:
- For the opcode cache, if WinCache can't map to the same address, it
falls back to using process-local (heap) memory for the opcode arrays.
This is because the elements in the opcode arrays contain absolute
pointers, and we don't want to spend cycles (a) converting all pointers to
offsets when copying opcode arrays into shared memory, and then (b)
converting all offsets back to absolute addresses when copying the opcode
array back out of shared memory. WinCache simply gets the pointer to the
opcode array in shared memory and returns it, unmodified, to the PHP Core
for execution.
Opcache uses the same strategy,.
When WinCache falls back to process-local memory, it does NOT create a
second memory mapped segment, and does not chew additional cross-process
resources.
Opcahce but don't have a fall-back mechanism in case of inability to remap
SHM into the same address space
- WinCache's file cache does not map to an identical address in all
processes. We simply map it to any address (usingNULLfor the suggested
address to MapViewOfFileEx), and then use relative offsets in the cache
structures for all entries in shared memory.
File-cache in opcache is a second level opcode cache. Instead of SHM (or in
addition to SHM) we may store precompiled php scripts on disk.
Thanks. Dmitry.
- WinCache's user cache and session cache are similar to the file cache,
and do not have to map to the same address in all processes. Cache entries
use relative offsets to locate cache entries, and the values do not contain
absolute addresses.Finally, For PHP7, WinCache has removed its opcode cache. We assume that
Windows consumers of PHP7 will use Zend Opcache for opcode caching.
Thx!
--E.