Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:45329
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: pass (pb1.pair.com: domain joeysmith.com designates 209.90.98.146 as permitted sender)
Date: Fri, 21 Aug 2009 13:11:45 -0600
To: internals@lists.php.net
Cc: Chris Smith <php.net@chris.cs278.org>
Message-ID: <20090821191145.GA9393@joeysmith.com>
References: <4A8EC373.8040002@chris.cs278.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4A8EC373.8040002@chris.cs278.org>
User-Agent: Mutt/1.5.18 (2008-05-17)
Subject: Re: [PHP-DEV] mail() and header folding/line endings
From: joey@joeysmith.com (Joey Smith)

On Fri, Aug 21, 2009 at 04:55:31PM +0100, Chris Smith wrote:
> 
> I've encountered difficulties utilising the mail() function properly
> under a NIX environments while conforming to RFC 2822. There two
> specific issues, one is a code problem the other a documentation
> issue they are intertwined so I thought best run it by here and see
> that people understand the issue before I head to the bug tracker.
> 
> As you are no doubt aware mail() operates using the system sendmail
> interface under NIX environments, and under Windows it either utilises a
> sendmail binary or more commonly the SMTP feature. The problem stems
> from confusion and inappropriate application of the SMTP standards to
> the sendmail interface.
> 
> Under a NIX environment the sendmail interface operates taking an e-mail
> message constructed using the system line ending, LF. I've confirmed
> this applies to the Postfix sendmail interface [1], and could ask other
> vendors.

1) Maybe you could go back to Wietse and ask him to justify the
seemingly contradictory assertions that "text is expected to be
in native UNIX stream-LF format" [1] but "Postfix receives local
submissions in (LF or CRLF) format" [2]  and "Postfix looks at 
the first input line [to determine what format you are using]" [3]?
It's hard to know what to tell PHP developers when we get mixed
messages from someone like Wietse...

As for "other vendors", I can tell you first-hand that sendmail and
exim both handle "mixed line-endings" just fine. However, I agree that
not sending them in the first place is the ideal scenario...

> Under Windows either the sendmail binary or SMTP is utilised. The
> correct line endings for SMTP are certainly CR-LF, sendmail I guess
> could be either.

2) Perhaps this has changed, but last time I looked, the built-in
SMTP client is the ONLY method available for mail() on Win32.

> The documentation mentions to use CR-LF as the line endings for the
> $additional_headers parameters whilst the function implementation
> utilises LFs [2] which results in a mix of line endings, which is worse
> than incorrect line endings. There is a 7 year old bug open about this
> inconsistency [3] and as of yet nobody has fixed it, I hope this can be
> rectified.

3) I don't have an Apple platform for testing, what will happen on
Mac if PHP_EOL is used as the separator for $additional_headers? I
would like to change the documentation to say "Multiple extra
headers should be separated with the PHP_EOL constant", but I'm not
the least bit certain this is going to work correctly on Mac. I can
tell you that on my machines (Linux, using a mix exim and sendmail
as MTAs), it will not see the \r as a separator, but mixing \r\n and
\n within the same message works just fine (another case of the
ever-prevalent SMTP mantra of "Be permissive in what you accept, and
strict in what you send"). If PHP_EOL can't be safely used, I imagine
we'll have to document it as 'Use "\r\n" on Win32, and "\n"
everywhere else', which I'd really rather not do - it seems hackish.


> The second issue I have is quite a show stopper with regards to properly
> formatted emails under the NIX environment; email headers maybe no
> longer than 998 characters but the advised cut off is 78, a long
> header should be folded over two lines with some white space
> indentation. [4] As the To and Subject headers are populated from the
> function arguments of the same name they pass through additional checks
> over the other headers. The problem arises with the SKIP_LONG_HEADER_SEP
> macro [5] which only skips over the SMTP standard of CR-LF-WSP and not
> LF-WSP required for NIX sendmail as a consequence the LF is replace with
> a space effectively unfolding the folded line. It is therefore
> impossible to correctly create an email using mail() with a large number
> of recipients or a long subject, the effective limit of text shrinks
> when encoding is used for non ASCII characters as well.

4) I don't write/maintain an MTA, but it seems like you're conflating
SMTP and the "local pipe to a sendmail binary" conversation where it
should not be. If you're sending your emails to a local pipe, I don't
think you should be wrapping your headers in the PHP code with a LF-WSP.
The section of RFC2822 on folding long headers quite clearly states [4]:

       The general rule is that wherever this standard allows for 
       folding white space (not simply WSP characters), a *CRLF* may
       be inserted before any WSP. [emphasis mine]

Since PHP's mail() doesn't know if it's sending over SMTP or a local
pipe, I honestly feel that SKIP_LONG_HEADER_SEP is doing the correct
thing here - your MTA should be able to accept long headers via a
local pipe and make sure it formats them appropriately before sending
them on to their SMTP destination - PHP should only be concerned with
SKIP_LONG_HEADER_SEP in a *pure* SMTP situation.

Note: I just fed exim a 12000 character long subject from PHP and exim
wrapped it before sending it to the other end of the SMTP conversation.
Are you saying that Postfix does not do this?


-- 

[1] http://article.gmane.org/gmane.mail.postfix.user/200795
[2] http://article.gmane.org/gmane.mail.postfix.user/188716
[3] http://article.gmane.org/gmane.mail.postfix.user/188756
[4] http://tools.ietf.org/html/rfc2822#section-2.2.3