Hi,
I've encountered difficulties utilising the mail() function properly
under a NIX environments while conforming to RFC 2822. There two
specific issues, one is a code problem the other a documentation
issue they are intertwined so I thought best run it by here and see
that people understand the issue before I head to the bug tracker.
As you are no doubt aware mail() operates using the system sendmail
interface under NIX environments, and under Windows it either utilises a
sendmail binary or more commonly the SMTP feature. The problem stems
from confusion and inappropriate application of the SMTP standards to
the sendmail interface.
Under a NIX environment the sendmail interface operates taking an e-mail
message constructed using the system line ending, LF. I've confirmed
this applies to the Postfix sendmail interface [1], and could ask other
vendors.
Under Windows either the sendmail binary or SMTP is utilised. The
correct line endings for SMTP are certainly CR-LF, sendmail I guess
could be either.
The documentation mentions to use CR-LF as the line endings for the
$additional_headers parameters whilst the function implementation
utilises LFs [2] which results in a mix of line endings, which is worse
than incorrect line endings. There is a 7 year old bug open about this
inconsistency [3] and as of yet nobody has fixed it, I hope this can be
rectified.
The second issue I have is quite a show stopper with regards to properly
formatted emails under the NIX environment; email headers maybe no
longer than 998 characters but the advised cut off is 78, a long
header should be folded over two lines with some white space
indentation. [4] As the To and Subject headers are populated from the
function arguments of the same name they pass through additional checks
over the other headers. The problem arises with the SKIP_LONG_HEADER_SEP
macro [5] which only skips over the SMTP standard of CR-LF-WSP and not
LF-WSP required for NIX sendmail as a consequence the LF is replace with
a space effectively unfolding the folded line. It is therefore
impossible to correctly create an email using mail() with a large number
of recipients or a long subject, the effective limit of text shrinks
when encoding is used for non ASCII characters as well.
Can anyone provide me with any insight?
Regards,
Chris
[1] http://thread.gmane.org/gmane.mail.postfix.user/200784
[2]
http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3/ext/standard/mail.c?view=markup#l312
[3] http://bugs.php.net/bug.php?id=15841
http://tools.ietf.org/html/rfc2822#section-2.2.3
[4] http://tools.ietf.org/html/rfc2822#section-2.2.3
[5]
http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3/ext/standard/mail.c?view=markup#l56
--
Chris Smith
I've encountered difficulties utilising the
mail()function properly
under a NIX environments while conforming to RFC 2822. There two
specific issues, one is a code problem the other a documentation
issue they are intertwined so I thought best run it by here and see
that people understand the issue before I head to the bug tracker.As you are no doubt aware
mail()operates using the system sendmail
interface under NIX environments, and under Windows it either utilises a
sendmail binary or more commonly the SMTP feature. The problem stems
from confusion and inappropriate application of the SMTP standards to
the sendmail interface.Under a NIX environment the sendmail interface operates taking an e-mail
message constructed using the system line ending, LF. I've confirmed
this applies to the Postfix sendmail interface [1], and could ask other
vendors.
- Maybe you could go back to Wietse and ask him to justify the
seemingly contradictory assertions that "text is expected to be
in native UNIX stream-LF format" [1] but "Postfix receives local
submissions in (LF or CRLF) format" [2] and "Postfix looks at
the first input line [to determine what format you are using]" [3]?
It's hard to know what to tell PHP developers when we get mixed
messages from someone like Wietse...
As for "other vendors", I can tell you first-hand that sendmail and
exim both handle "mixed line-endings" just fine. However, I agree that
not sending them in the first place is the ideal scenario...
Under Windows either the sendmail binary or SMTP is utilised. The
correct line endings for SMTP are certainly CR-LF, sendmail I guess
could be either.
- Perhaps this has changed, but last time I looked, the built-in
SMTP client is the ONLY method available formail()on Win32.
The documentation mentions to use CR-LF as the line endings for the
$additional_headers parameters whilst the function implementation
utilises LFs [2] which results in a mix of line endings, which is worse
than incorrect line endings. There is a 7 year old bug open about this
inconsistency [3] and as of yet nobody has fixed it, I hope this can be
rectified.
- I don't have an Apple platform for testing, what will happen on
Mac ifPHP_EOLis used as the separator for $additional_headers? I
would like to change the documentation to say "Multiple extra
headers should be separated with thePHP_EOLconstant", but I'm not
the least bit certain this is going to work correctly on Mac. I can
tell you that on my machines (Linux, using a mix exim and sendmail
as MTAs), it will not see the \r as a separator, but mixing \r\n and
\n within the same message works just fine (another case of the
ever-prevalent SMTP mantra of "Be permissive in what you accept, and
strict in what you send"). IfPHP_EOLcan't be safely used, I imagine
we'll have to document it as 'Use "\r\n" on Win32, and "\n"
everywhere else', which I'd really rather not do - it seems hackish.
The second issue I have is quite a show stopper with regards to properly
formatted emails under the NIX environment; email headers maybe no
longer than 998 characters but the advised cut off is 78, a long
header should be folded over two lines with some white space
indentation. [4] As the To and Subject headers are populated from the
function arguments of the same name they pass through additional checks
over the other headers. The problem arises with the SKIP_LONG_HEADER_SEP
macro [5] which only skips over the SMTP standard of CR-LF-WSP and not
LF-WSP required for NIX sendmail as a consequence the LF is replace with
a space effectively unfolding the folded line. It is therefore
impossible to correctly create an email usingmail()with a large number
of recipients or a long subject, the effective limit of text shrinks
when encoding is used for non ASCII characters as well.
-
I don't write/maintain an MTA, but it seems like you're conflating
SMTP and the "local pipe to a sendmail binary" conversation where it
should not be. If you're sending your emails to a local pipe, I don't
think you should be wrapping your headers in the PHP code with a LF-WSP.
The section of RFC2822 on folding long headers quite clearly states [4]:The general rule is that wherever this standard allows for folding white space (not simply WSP characters), a *CRLF* may be inserted before any WSP. [emphasis mine]
Since PHP's mail() doesn't know if it's sending over SMTP or a local
pipe, I honestly feel that SKIP_LONG_HEADER_SEP is doing the correct
thing here - your MTA should be able to accept long headers via a
local pipe and make sure it formats them appropriately before sending
them on to their SMTP destination - PHP should only be concerned with
SKIP_LONG_HEADER_SEP in a pure SMTP situation.
Note: I just fed exim a 12000 character long subject from PHP and exim
wrapped it before sending it to the other end of the SMTP conversation.
Are you saying that Postfix does not do this?
--
[1] http://article.gmane.org/gmane.mail.postfix.user/200795
[2] http://article.gmane.org/gmane.mail.postfix.user/188716
[3] http://article.gmane.org/gmane.mail.postfix.user/188756
[4] http://tools.ietf.org/html/rfc2822#section-2.2.3
PHP_EOL is \n on OS X. So the \r worries are not a
concern. PHP_EOL would be fine in this case, assuming
the OSX sendmail is fine with it.
- Davey
- I don't have an Apple platform for testing, what will happen on
Mac ifPHP_EOLis used as the separator for $additional_headers? I
would like to change the documentation to say "Multiple extra
headers should be separated with thePHP_EOLconstant", but I'm not
the least bit certain this is going to work correctly on Mac. I can
tell you that on my machines (Linux, using a mix exim and sendmail
as MTAs), it will not see the \r as a separator, but mixing \r\n and
\n within the same message works just fine (another case of the
ever-prevalent SMTP mantra of "Be permissive in what you accept, and
strict in what you send"). IfPHP_EOLcan't be safely used, I imagine
we'll have to document it as 'Use "\r\n" on Win32, and "\n"
everywhere else', which I'd really rather not do - it seems hackish.
Joey Smith wrote:
- Maybe you could go back to Wietse and ask him to justify the
seemingly contradictory assertions that "text is expected to be
in native UNIX stream-LF format" [1] but "Postfix receives local
submissions in (LF or CRLF) format" [2] and "Postfix looks at
the first input line [to determine what format you are using]" [3]?
It's hard to know what to tell PHP developers when we get mixed
messages from someone like Wietse...
I think from his previous replies on the subject Postfix tolerates CRLF
formatted submissions and prefers LF, which is understandable as it is a
NIX program.
As for "other vendors", I can tell you first-hand that sendmail and
exim both handle "mixed line-endings" just fine. However, I agree that
not sending them in the first place is the ideal scenario...
Mixed line endings should never be passed to the mailer.
- Perhaps this has changed, but last time I looked, the built-in
SMTP client is the ONLY method available formail()on Win32.
It is perfectly possible to use sendmail under Windows it's not really
advertised though, I've not tested it either.
- I don't have an Apple platform for testing, what will happen on
Mac ifPHP_EOLis used as the separator for $additional_headers?
Well you are entering the realms of pre-OSX systems there, not really a
valid concern.
- I don't write/maintain an MTA, but it seems like you're conflating
SMTP and the "local pipe to a sendmail binary" conversation where it
should not be. If you're sending your emails to a local pipe, I don't
think you should be wrapping your headers in the PHP code with a LF-WSP.
I have to disagree RFC 2822 specifies the message format. While I agree
it does specify that the end of line should be CRLF and that folding is
conducted using the CRLF followed by WSP, this is not really a point of
discussion suited to PHP it is for the MTA authors. The derivation from
the RFC 2822 specification as far as I am aware is just line endings,
which ever the MTA expects should be used. I think its a fair assumption
to expect nearly all mailers to permit both CRLF and LF separated
emails, however the mail() function only permits talking LF. As stated
previously sending mixed line endings should be a total no go, as it
stands then only LF can be used by headers passed to mail().
Since PHP's
mail()doesn't know if it's sending over SMTP or a local
pipe, I honestly feel that SKIP_LONG_HEADER_SEP is doing the correct
thing here - your MTA should be able to accept long headers via a
local pipe and make sure it formats them appropriately before sending
them on to their SMTP destination - PHP should only be concerned with
SKIP_LONG_HEADER_SEP in a pure SMTP situation.
It is simple to detect the method used, SMTP is used if sendmail_path is
empty and the system is Windows, otherwise sendmail is used. The
SKIP_LONG_HEADER_SEP was added for a reason back in 2003 in response to
bug #22355. [1]
Note: I just fed exim a 12000 character long subject from PHP and exim
wrapped it before sending it to the other end of the SMTP conversation.
Are you saying that Postfix does not do this?
Did you try encoding a 12000 UTF-8 string using RFC 2047 [2], this is
where the real problems emerge. Postfix will handle a really long
subject just fine, but an encoded header reaching that long gets folded
rendering it invalid. This is why it is imperative to be able to fold
the Subject and To headers in PHP, which is where it should be done.
Regards,
Chris
[1] http://bugs.php.net/bug.php?id=22355
[2] http://tools.ietf.org/html/rfc2047
--
Chris Smith
Joey Smith:
It's hard to know what to tell PHP developers when we get mixed
messages from someone like Wietse...
No, it's not so hard.
PHP developers must follow the standard:
- use "network EOL" (CRNL) when speaking SMTP;
- use "system EOL" (CRNL or NL or CR) when speaking with local MTA.
Eventuallymail()should convert "mixed line-endings" to "right
line-endings", and "right line-endings" may be selected by the
system administrator (a php.ini "mail_line_ending = system|network"
switch?).
As for "other vendors", I can tell you first-hand that sendmail and
exim both handle "mixed line-endings" just fine.
Actually a "real" PHP programmer is not using mail() at all, because
nobody know where the application will be installed.
--
Tullio Andreatta
Disclaimer: "Please treat this email message in a reasonable way, or we
might get angry" ( http://www.goldmark.org/jeff/stupid-disclaimers )
- Perhaps this has changed, but last time I looked, the built-in
SMTP client is the ONLY method available formail()on Win32.
I believe some people have had success in installing sendmail-like
software on Windows and using the 'linux' sendmail version.
But I could be mis-remembering...
I have no intention of commenting on the actual thrust of this \n \r\n
\r mess that has been stinking up not only php mail() but the entire
software industry for decades...
Then Chris Smith wrote:
It is simple to detect the method used, SMTP is used if sendmail_path
is empty and the system is Windows, otherwise sendmail is used.
I think you should revise this to be "if sendmail_path is empty".
Many *nix folks use SMTP as well...
--
Some people ask for gifts here.
I just want you to buy an Indie CD for yourself:
http://cdbaby.com/search/from/lynch
Richard Lynch wrote:
Then Chris Smith wrote:
It is simple to detect the method used, SMTP is used if sendmail_path
is empty and the system is Windows, otherwise sendmail is used.I think you should revise this to be "if sendmail_path is empty".
Many *nix folks use SMTP as well...
The SMTP function of mail() only works under Windows. [1]
Regards,
Chris
[1]
http://svn.php.net/viewvc/php/php-src/tags/php_5_3_0/ext/standard/mail.c?view=markup#l252
--
Chris Smith
Richard Lynch wrote:
Then Chris Smith wrote:
It is simple to detect the method used, SMTP is used if
sendmail_path
is empty and the system is Windows, otherwise sendmail is used.I think you should revise this to be "if sendmail_path is empty".
Many *nix folks use SMTP as well...
The SMTP function of
mail()only works under Windows. [1]Regards,
Chris
[1]
http://svn.php.net/viewvc/php/php-src/tags/php_5_3_0/ext/standard/mail.c?view=markup#l252
Sorry. I was thinking of Manuel Lemos' SMTP class oft-used on Un*x to
provide SMTP support, rather than the built-in mail() function.
--
Some people ask for gifts here.
I just want you to buy an Indie CD for yourself:
http://cdbaby.com/search/from/lynch