Newsgroups: php.internals Path: news.php.net Xref: news.php.net php.internals:88586 Return-Path: Mailing-List: contact internals-help@lists.php.net; run by ezmlm Delivered-To: mailing list internals@lists.php.net Received: (qmail 46671 invoked from network); 30 Sep 2015 20:59:17 -0000 Received: from unknown (HELO lists.php.net) (127.0.0.1) by localhost with SMTP; 30 Sep 2015 20:59:17 -0000 Authentication-Results: pb1.pair.com header.from=dmitry@zend.com; sender-id=pass Authentication-Results: pb1.pair.com smtp.mail=dmitry@zend.com; spf=pass; sender-id=pass Received-SPF: pass (pb1.pair.com: domain zend.com designates 209.85.160.182 as permitted sender) X-PHP-List-Original-Sender: dmitry@zend.com X-Host-Fingerprint: 209.85.160.182 mail-yk0-f182.google.com Received: from [209.85.160.182] ([209.85.160.182:36379] helo=mail-yk0-f182.google.com) by pb1.pair.com (ecelerity 2.1.1.9-wez r(12769M)) with ESMTP id F1/F0-38941-42D4C065 for ; Wed, 30 Sep 2015 16:59:16 -0400 Received: by ykdt18 with SMTP id t18so58075971ykd.3 for ; Wed, 30 Sep 2015 13:59:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=0V43Qnw4yQxOi8eHvOz5zaQnZ8MXBRPCAvaP/baKmNI=; b=HId9aKbTA0XL39TEpoBwl0QTyy+p/GWdYlTRebXJuQUXJswU3/yY56LXyzftGlx9Jr ml3UpZszkLeQKdSJEZDhwakDDhxDpYaXRaykgTxwnG4KO5OlwqpGkjBvAG1lKmsJbR5a BzNBXzCP+H7DVF6S0QwB7dhFa1upHZ48X2fSYtC2vknnT3Fv5iFzqg3FgaQsCrGngHNg yyZfdIZ4euCPn9VLK/kR3w4rQrb8IA8cUqSz8c1QyM00h7QflUWEw+f5l11podsceCiK iyJTEswmr40sQsWp18Z15eInYwbh2MGbLp5RoZN3jwqG6pYokb6frlAHW3CU/dtdEpl5 sQ3w== X-Gm-Message-State: ALoCoQncEcvt+DoqEHhs93Ffj1je3lj2DWamDZ65FryTHDpsezLtIrzJ63y3G0r7EyiKfiREr25DAwaEkQTqxf8+0ocQXgvSQazVJYfgsuqOjTnlWe9KXv5Je9+iwfNUImC1QAp7NeCywFc1WNWvsZoBoOZkN7XhCJrDzNFGQzAwcgEB1AX2T0U= MIME-Version: 1.0 X-Received: by 10.31.54.194 with SMTP id d185mr4521492vka.131.1443646753600; Wed, 30 Sep 2015 13:59:13 -0700 (PDT) Received: by 10.103.24.5 with HTTP; Wed, 30 Sep 2015 13:59:13 -0700 (PDT) In-Reply-To: References: Date: Wed, 30 Sep 2015 23:59:13 +0300 Message-ID: To: Matt Ficken Cc: Anatol Belski , PHP Internals , "dmitry@php.net" , Xinchen Hui Content-Type: multipart/alternative; boundary=001a11438b28ee17800520fd33d7 Subject: Re: Windows OpCache bug fix From: dmitry@zend.com (Dmitry Stogov) --001a11438b28ee17800520fd33d7 Content-Type: text/plain; charset=UTF-8 Hi Matt, Thanks for looking into the problem, but I afraid that the proposed solution won't solve the problem completely, but may introduce new problems. At first, we might to reserve N times more memory. Then different SHM segments might contain different data. opcache_reset(), opcache_invalidate() and others will affect only one segment. I think the perfect solution would require converting all stored data in position independent format, but this hardly ever is going to be done even for php-8. We may fallback to use file-cache introduced in php-7. (this should work out of the box or with minimal changes). Also. if we can't map SHM into desired address space, we may map it in any other address and copy data into the process memory similar to file-cache. Finally we may try to use /DYNAMICBASE:NO Thanks. Dmitry. On Tue, Sep 29, 2015 at 2:55 AM, Matt Ficken wrote: > Great. The locking issue is fixed. > > But, it sometimes still fails to reattach to the base address. Because of > ASLR, extra DLLs, etc... the existing base address may not be usable in > other processes that need to reattach. When I see this issue I'm creating a > series of processes (for test runs) and when a process fails to reattach, > it happens to all the following processes for a few minutes. ASLR waits a > few minutes before changing the randomized memory layout when creating a > new process. When this happens to production web sites, it often sounds > like this is the case, as the web site becomes unusable for a while. > > Exiting when the base address can't be reattached to, is not a good way to > handle this case, even in a FastCGI scenario. > > The contract for create_segments() is to reattach successfully or return > ALLOC_FAILED. > > > Option 1: change create_segments() contract and > zend_shared_alloc_startup() so that create_segments() can `disable` > OpCache, possibly setting zend_accel_globals.enable=false and adding > additional checks. This eliminates the benefit of using OpCache, likely for > several processes, causing a major performance drop for a web site. > > Option 2: creating a backup, 2nd backup, etc... OpCache in this case. > There may be up to 6 OpCaches(5 backups) with 4 attempts each. This fix is > simpler.This allows processes to share a backup OpCache, a web site will > slow down only to compile the scripts again(at the expense of some > additional memory). > > There have been some feature requests to be able to create separate > OpCaches anyway. Shared web hosters would like to have different OpCaches > for different sites, but this would require a new INI directive which is > too late to add for 7, and for machines that run different PHP builds. > > This patch passes my functional, stress and performance tests. > > I've added it to the PR 1536: > https://github.com/php/php-src/commit/7cbebc747e5c5f824a6e0336cc044ce4dc4c8f9f > > > Regards > -M > > > On Thu, Sep 24, 2015 at 1:29 AM, Dmitry Stogov wrote: > >> Ops, I forgot to revert the exit condition. >> >> >> http://git.php.net/?p=php-src.git;a=commitdiff;h=2d55e8c186ef1034c2af64478da8f23dbeb28be9 >> >> >> >> On Thu, Sep 24, 2015 at 11:00 AM, Dmitry Stogov wrote: >> >>> hi Matt, >>> >>> Thanks. >>> I also moved the exit condition to be before Sleep(). >>> This should prevent race condition on last iteration (after unlocked >>> sleep process should try to reattach). >>> Committed to PHP-7.0 and master. >>> >>> >>> http://git.php.net/?p=php-src.git;a=commitdiff;h=262160e0e9919dce914df2f0643c4b16ca137454 >>> >>> If this really works, we should backport this into php-5.* as well. >>> >>> Thanks. Dmitry. >>> >>> On Thu, Sep 24, 2015 at 8:34 AM, Matt Ficken >>> wrote: >>> >>>> Ok, I have a new PR just to unlock around Sleep(): >>>> https://github.com/php/php-src/pull/1536 >>>> >>>> Just by unlocking around Sleep() the OpenFileMapping loop will work. >>>> Holding the lock in that loop will not only block detach_segments() in >>>> other PHP processes but also reattaching in accel_startup(). >>>> >>>> Checking the base address used when this issue occurs during some of my >>>> stress testing, most of the time the base address used is the first in >>>> vista_mapping_base_set. Its typically the timing between the locking and >>>> mapped file operations between processes more often than the base address >>>> selection. >>>> >>>> Especially ASLR, also PECL DLLs, and other factors will still cause >>>> this issue occasionally, but this fixes a majority of my occurrences. >>>> >>>> Regards >>>> -M >>>> >>>> >>>> On Wed, Sep 23, 2015 at 2:04 AM, Dmitry Stogov wrote: >>>> >>>>> Hi Matt, >>>>> >>>>> It looks like with your patch, the same file may be mapped to >>>>> different virtual addresses of different processes and this won't work. >>>>> I think usage of vista_mapping_base_set[] in your patch is wrong. >>>>> >>>>> Do I understand properly, that zend_shared_alloc_unlock/lock_win32() >>>>> around Sleep() is the main part of the patch? >>>>> >>>>> Please, next time, try to make PR diffs as small as possible (without >>>>> messing real fix with formatting and rearranging changes). >>>>> >>>>> Thanks. Dmitry. >>>>> >>>>> >>>>> >>>>> On Wed, Sep 23, 2015 at 10:10 AM, Matt Ficken >>>> > wrote: >>>>> >>>>>> I want to increase visibility for my PR 1531, >>>>>> https://github.com/php/php-src/pull/1531, my patch for fixing an >>>>>> intermittent OpCache issue on Windows. >>>>>> >>>>>> Details, etc... are on the PR. >>>>>> >>>>>> Regards >>>>>> -M >>>>>> >>>>> >>>>> >>>> >>> >> > --001a11438b28ee17800520fd33d7--