Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:66139
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain newgrounds.com designates 209.85.214.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <5122E451.1040308@lerdorf.com>
References: <CAHHbRnEUVcGqH6+++CWXM6XWobJn5MzKxLdfX9RHxoJ0uXaZdg@mail.gmail.com>
	<51229088.90306@lerdorf.com>
	<5122DA51.6090606@sugarcrm.com>
	<5122DBA9.2010004@lerdorf.com>
	<5122E00F.80409@sugarcrm.com>
	<5122E451.1040308@lerdorf.com>
Date: Thu, 21 Feb 2013 18:15:44 -0500
Message-ID: <CAHHbRnEZo9nmon+yhvnEW0t2FFV_veBXzkSdDjDXEvpfet141A@mail.gmail.com>
To: Rasmus Lerdorf <rasmus@lerdorf.com>
Cc: "internals@lists.php.net" <internals@lists.php.net>
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: [PHP-DEV] PHP causing high number of NFS getattr operations?
From: brendon@newgrounds.com (Brendon Colby)

On Mon, Feb 18, 2013 at 3:35 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

> Wow, people are still serving web files over NFS? Sounds painful.

On Mon, Feb 18, 2013 at 9:32 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:

> So they are serving up all their PHP over NFS for some reason.

I've been thinking about this and wanted to make a few final points. I
think this is potentially a pretty major issue and not something to be
dismissed.

We've invested a great deal of money in a high performance shared
storage system, so of course we want to use this system for absolutely
everything we can within reason, including sharing 1500+ PHP files to
our many web servers. I don't think most systems administrators would
find this unreasonable.

Why would we want to split our architecture up so just our PHP files
are being synced from local disk and everything else is shared via
NFS? Doing this seems far more unreasonable to me and just doesn't
make any sense.

NFS is so common for sharing files that saying "Wow, people are still
serving web files over NFS?" is like saying "Wow, people are still
using the ls command to list directory contents on Linux?" I think NFS
is still very widely used, even for sharing web files.

This being said, you stated in bug #59372 that "PHP by default does
open+fstat for performance reasons" and while doing an open+fstat
might result in better performance on local file systems, doing so is
not optimal when using NFS. NFS "close-to-open cache consistency" is
designed such that opening a file stored on an NFS server causes an
automatic getattr or access request. This could also be true for other
shared file systems as others have mentioned.

All of these open+fstats are definitely causing a measurable impact to
our systems. Right now, for example, 50% of the 6-8K+ ops/s being
processed by our filer are getattr operations, and a majority of these
getattr calls are coming from our Apache/PHP servers. Doing open+fstat
bypasses the NFS attribute cache similar to disabling it altogether,
where a simple stat would likely utilize the attribute cache.
Disabling the attribute cache is generally not recommended due to the
performance impact this has on the filer (and the application).

I don't think the appropriate answer is "don't use NFS" because this
is ridiculous as a long term solution (NFS is common, and people are
going to use it or something similar). I think the appropriate answer
is to update PHP to use stat vs. open+fstat or doing something similar
that would be optimized for both local AND shared file systems (I
would be writing a patch instead of this email if I could).

Until then, I think it's important that people know that PHP is NOT
optimized for use with NFS (and possilby other shared file systems)
even when using an opcode cache like APC. I think this is a very
important piece of information that people need to know. Up until this
week, I wouldn't have thought twice about sharing PHP files over NFS,
and I've been a Unix admin for over twelve years (only new to this
environment).

Brendon