Newsgroups: php.apc.dev,php.internals
Path: news.php.net
Xref: news.php.net php.apc.dev:188 php.internals:43161
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain tekrat.com from 208.43.138.18 cause and error)
Message-ID: <49A2F6DB.6010202@tekrat.com>
Date: Mon, 23 Feb 2009 11:19:55 -0800
User-Agent: Postbox 1.0b7 (Macintosh/2009020805)
MIME-Version: 1.0
To: Ronald Chmara <ron@Opus1.COM>
CC: PHP Internals List <internals@lists.php.net>, apc-dev@lists.php.net
References: <49A097FE.10205@tekrat.com> <633918AD-16DE-4B34-A76E-E90A21E325B3@opus1.com> <49A0F6D6.3080605@tekrat.com> <6A41B9D3-7312-49F3-A4B0-C5D6DC23AF9D@Opus1.COM>
In-Reply-To: <6A41B9D3-7312-49F3-A4B0-C5D6DC23AF9D@Opus1.COM>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
From: shire@tekrat.com (shire)



Ronald Chmara wrote:
>
> On Feb 21, 2009, at 10:55 PM, shire wrote:
>> Hi Ronald,
>> Ronald Chmara wrote:
>>> Wait... so if I understand this right, let's envision a code base where,
>>> per some random page load, 70 functions are actually called, but, oh,
>>> 7,000, or even 700,000, are being included for whatever reason?
>>> The speed optimization is in *not* copying a massive amount of things
>>> that weren't even needed, or used, in the first place?
>> Essentially, yes, this is probably best summed up by the 80/20 rule
>> where we only use 20% of the code etc...
>
> Well, I can see 80% actually *used* code, with 20% in there by
> accident.... but 80% unused code? eep! ack! Call the villagers and get
> the torches and pitchforks!...

I should probably be more specific, it's not that you *don't* use the other 80% it's that you use 20% of the code 80% of the time: http://en.wikipedia.org/wiki/Pareto_principle

Of course I'm abusing this slightly, but this is the *basic* idea, and it's not based on actual usage statistics.  In reality you probably use all 100% of your code in smaller chunks depending on the request.  As an illustrative example, if you have HTTP requests for setting, retrieving, deleting items in a database which is supported by corresponding functions.  In each request you'll only use one of these functions (30%), but you'll still need to load the entire file/class.  The goal of lazy loading is to optimize this and similar situations so you don't have to re-organize your code into unfeasibly small files, or in other ways that might inhibit productivity.

  
> One thing I see as quite a beneficial future outcome of your work is the
> ability to further profile code, and be able to seek out code that marks
> massive amounts of functions as "available".... without actually ever
> using them.

I think this would be better instrumented through tools actually designed to do this sort of profiling, specifically XDebug, or tools like inclued are very useful.

http://pecl.php.net/package/xdebug
http://pecl.php.net/package/inclued


>> It's actually likely that only a fraction of the code at Facebook will
>> be used in a request, hence the need for lazy loading.
>
> Ouch. Seriously.
>
> I can't tell you how to build your code, but I think you might seriously
> benefit from:
>....
> Does that make sense? Or did you try it already? :)


I'd rather not enter into a public discussion of Facebook's current optimization tactics outside of this patch, but suffice to say we have implemented many optimization techniques, both in PHP code and Internals, many of which we have and will continue to contribute back to the community.   Lazy Loading has been a very necessary and optimal solution to this one aspect of our scalability, thus allowing our engineers to focus more of their time and energy on creating new and exciting features for Facebook.


Thanks,

-shire