# What's up with emails? - Thread: https://externals.io/message/122454 - Original archive: https://news-web.php.net/php.internals/122454 - Started by Robert Landers on 2024-02-22T13:52:49+00:00 - Messages: 20 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #122454 — Robert Landers — 2024-02-22T13:52:49+00:00 URL: https://externals.io/message/122454#122454 I have noticed quite a number of emails appearing on externals but not appearing in my inbox. I randomly get an email saying: Hi, this is the Mlmmj program managing the mailing list. Some messages to you could not be delivered. If you're seeing this message it means things are back to normal, and it's merely for your information. Here is the list of the bounced messages: - 122446 - 122448 But I can clearly see far more than that in externals that are never received. IIRC from the email spec and internet customs, soft-bounces should be retried for a number of days (though I have no idea why gmail soft-bounced the messages, maybe rate-limiting?). If I'm using gmail and not getting emails, how many other people are also not getting emails on this list? Robert Landers Software Engineer Utrecht NL --- ## #122455 — Deleu — 2024-02-22T14:01:58+00:00 URL: https://externals.io/message/122454#122455 · In reply to: #122454 On Thu, Feb 22, 2024 at 10:55 AM Robert Landers wrote: > I have noticed quite a number of emails appearing on externals but not > appearing in my inbox. I randomly get an email saying: > > Hi, this is the Mlmmj program managing the > mailing list. > > Some messages to you could not be delivered. If you're seeing this > message it means things are back to normal, and it's merely for your > information. > > Here is the list of the bounced messages: > - 122446 > - 122448 > > But I can clearly see far more than that in externals that are never > received. > > IIRC from the email spec and internet customs, soft-bounces should be > retried for a number of days (though I have no idea why gmail > soft-bounced the messages, maybe rate-limiting?). > > If I'm using gmail and not getting emails, how many other people are > also not getting emails on this list? > > Robert Landers > Software Engineer > Utrecht NL > I'm also affected by this exact same thing. [image: image.png] -- Marco Deleu --- ## #122456 — Erick de Azevedo Lima — 2024-02-22T14:03:55+00:00 URL: https://externals.io/message/122454#122456 · In reply to: #122454 > If I'm using gmail and not getting emails, how many other people are > also not getting emails on this list? Same here. --- ## #122457 — Derick Rethans — 2024-02-22T14:05:22+00:00 URL: https://externals.io/message/122454#122457 · In reply to: #122454 >I have noticed quite a number of emails appearing on externals but not >appearing in my inbox. I randomly get an email saying: > >Hi, this is the Mlmmj program managing the >mailing list. > >Some messages to you could not be delivered. If you're seeing this >message it means things are back to normal, and it's merely for your >information. > >Here is the list of the bounced messages: >- 122446 >- 122448 > >But I can clearly see far more than that in externals that are never received. > >IIRC from the email spec and internet customs, soft-bounces should be >retried for a number of days (though I have no idea why gmail >soft-bounced the messages, maybe rate-limiting?). > >If I'm using gmail and not getting emails, how many other people are >also not getting emails on this list? Lots. Gmail is rejecting emails after we moved the servers without telling us why, in enough detail to do anything about it. We can put a delay in delivering to Gmail, but that means mails get delayed for days. So my advice would for now to not rely on Gmail while we figure out how to improve this. cheers Derick --- ## #122458 — Jeffrey Dafoe — 2024-02-22T14:13:32+00:00 URL: https://externals.io/message/122454#122458 · In reply to: #122454 I had an email to the list bounce due to SpamCop adding a sizeable portion of Outlook's IP space into a block (as they've been erroneously doing for 25 years now). Emailed systems@php.net about it, never heard back. I'm also receiving next-to-no list traffic and receiving similar bounce messages as what is shown below. -Jeff > -----Original Message----- > From: Robert Landers > Sent: Thursday, February 22, 2024 8:53 AM > To: internals > Subject: External Message: [PHP-DEV] What's up with emails? > > I have noticed quite a number of emails appearing on externals but not > appearing in my inbox. I randomly get an email saying: > > Hi, this is the Mlmmj program managing the mailing > list. > > Some messages to you could not be delivered. If you're seeing this message it > means things are back to normal, and it's merely for your information. > > Here is the list of the bounced messages: > - 122446 > - 122448 > > But I can clearly see far more than that in externals that are never received. > > IIRC from the email spec and internet customs, soft-bounces should be retried > for a number of days (though I have no idea why gmail soft-bounced the > messages, maybe rate-limiting?). > > If I'm using gmail and not getting emails, how many other people are also not > getting emails on this list? > > Robert Landers > Software Engineer > Utrecht NL --- ## #122459 — Jeffrey Dafoe — 2024-02-22T14:14:24+00:00 URL: https://externals.io/message/122454#122459 · In reply to: #122457 > Gmail is rejecting emails after we moved the servers without telling us why, in > enough detail to do anything about it. It's not just gmail. -Jeff --- ## #122460 — Daniil Gentili — 2024-02-22T14:35:56+00:00 URL: https://externals.io/message/122454#122460 · In reply to: #122454 > If I'm using gmail and not getting emails, how many other people are > also not getting emails on this list? > Same here, with both gmail and zoho mail. Personally I would switch to github issues and discussions, even ignoring all the deliverability issues which I had even before the switch to the new mailing list server on my zoho mail daniil@daniil.it account, I don't think using a mailing list is a good idea for a modern programming language, seeking new contributors and new ideas in 2024. VCS was already moved to github after the recent hack of the php VCS, a lot of technical internals-related discussion is already using exclusively github issues and pull request discussions, I believe that the mailing list is nothing more than a redundant relic of the past. Regards, Daniil Gentili --- ## #122461 — Michael Kliewe — 2024-02-22T14:44:13+00:00 URL: https://externals.io/message/122454#122461 · In reply to: #122459 Am 22.02.2024 um 15:14 schrieb Jeffrey Dafoe: > > […] > It's not just gmail. I guess it's because of a wrong setting in the mailinglist server. The content of the original email is changed (the subject is prepended by "[PHP-DEV]"), which breaks the original DKIM signature by the sender. A new DKIM-signature is added by php.net, but it's not aligned to the From:-header domain (which is still the original sender). So the email doesn't have a valid ALIGNED DKIM signature. SPF is valid for the envelope from, but is not aligned to the From:-header. The result is: There is no aligned SPF nor aligned DKIM, which results in a dmarc=fail. Providers which honor DMARC will quarantine or reject these unauthorized/forged emails. There are 2 solutions: 1. Don't change the content or the DKIM-signed headers of the email (do not prepend something in the subject). Then the original DKIM signature stays valid and the From:-header can stay untouched 2. If you change the the email and break the original DKIM signature, also change the From:-header to a domain which matches the new DKIM signature, in this case set the From:-header to internals@lists.php.net. This is called "munging" in mailinglist software. Michael --- ## #122462 — tag Knife — 2024-02-22T14:54:48+00:00 URL: https://externals.io/message/122454#122462 · In reply to: #122457 > On 22 February 2024 08:52:49 GMT-05:00, Robert Landers < > landers.robert@gmail.com> wrote: > > […] > received. > > […] > > > Lots. > > Gmail is rejecting emails after we moved the servers without telling us > why, in enough detail to do anything about it. > > We can put a delay in delivering to Gmail, but that means mails get > delayed for days. > > So my advice would for now to not rely on Gmail while we figure out how to > improve this. > > cheers > Derick > This could be related to the new servers new ip not being trusted. Did you do any sort of IP Warming for the new server? As without this, Google sees a new IP sending thousands of emails out of nowhere and it triggers spam filters. We have to do this at work when we onboard a new client, since we give all clients apps there own IP, we have to run a 14 day warming system before we can let the client go live, since without it, their emails will most likely be blocked for spam from various inbox providers. --- ## #122463 — Alain D D Williams — 2024-02-22T15:29:06+00:00 URL: https://externals.io/message/122454#122463 · In reply to: #122457 > Gmail is rejecting emails after we moved the servers without telling us why, in enough detail to do anything about it. I run other mail lists and have had similar problems with gmail recently. The problem seems to be SPF. This says which IP addresses can be used for sending email for particular domains. gmail has recently become much stricter. List emails are sent from php-smtp4.php.net (45.112.84.5). However the 'From' address is still that of the original sender -- which is OK for the email that I am replying to as it is From: Derick Rethans and the MX for php.net is php-smtp4-ip4.php.net. which has address 45.112.84.5 However I suspect that *this* email (the one that I am typing now) will get bounced by gmail as my address is: addw@phcomp.co.uk and 45.112.84.5 is not allowed (by SPF) to send email for phcomp.co.uk. So: how to fix ? Make the From: address internals@lists.php.net The envelope address is already, something like internals+bounces-122457-addw=phcomp.co.uk@lists.php.net This means a fix to whatever list processor that you are using. Other recommendations: • MX for lists.php.net is php-smtp4-ip4.php.net. Change to php-smtp4.php.net. • SPF for lists.php.net is "v=spf1 a mx a:osu1php.osuosl.org. ip4:45.112.84.5 ip6:2a02:cb43:8000::1102" Change to "v=spf1 a mx -all" •• osu1php.osuosl.org. does not seem to exist - so zap it •• If you make the MX change that I suggest then the following are redundant: ip4:45.112.84.5 ip6:2a02:cb43:8000::1102 •• -all means no other addresses accepted. gmail started making that implicit in the last few months, it is this that is causing the current problems. Once that has settled down you should enable sending email from (& thus to) IPv6 addresses. This is the way that the world is moving. It is not hard. Regards -- Alain Williams Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer. +44 (0) 787 668 0256 https://www.phcomp.co.uk/ Parliament Hill Computers. Registration Information: https://www.phcomp.co.uk/Contact.html #include > > > […] > > I run other mail lists and have had similar problems with gmail recently. Gmail explicitly tightened their requirements for senders this month, so while it's likely the server change is related, it may not be the entire cause. Specifically, they require: - valid spf, with specific rules for quarantining email that has @gmail.com From but doesn't pass spf - valid dkim - dmarc configured if the server sends more than 5k emails to Gmail They also equivocally indicate a requirement for ARC and a List-Id header. This is all at https://support.google.com/a/answer/81126. I think ARC is a must; without it we'll never get to passing dkim at least, even if we rewrite from so spf passes. I'm happy to pitch in to help getting this configured if that's helpful, though I'm also very conscious that too many cooks is often a greater harm than good when it comes to administration. But reach out if you think I can be helpful. mjec --- ## #122465 — Derick Rethans — 2024-02-22T16:46:04+00:00 URL: https://externals.io/message/122454#122465 · In reply to: #122463 > > > > […] > > I run other mail lists and have had similar problems with gmail recently. > > The problem seems to be SPF. This says which IP addresses can be used for > sending email for particular domains. gmail has recently become much stricter. > > List emails are sent from php-smtp4.php.net (45.112.84.5). > > However the 'From' address is still that of the original sender -- which is OK > for the email that I am replying to as it is > From: Derick Rethans > > and the MX for php.net is php-smtp4-ip4.php.net. which has address 45.112.84.5 > > However I suspect that *this* email (the one that I am typing now) will get > bounced by gmail as my address is: addw@phcomp.co.uk and 45.112.84.5 is not > allowed (by SPF) to send email for phcomp.co.uk. > > So: how to fix ? > > Make the From: address internals@lists.php.net We don't want to change the From address, as that means that replies go to the list, rather than the original sender. We never had to do any of that before. > The envelope address is already, something like > internals+bounces-122457-addw=phcomp.co.uk@lists.php.net > > This means a fix to whatever list processor that you are using. > > Other recommendations: > > • MX for lists.php.net is php-smtp4-ip4.php.net. > Change to php-smtp4.php.net. Done > • SPF for lists.php.net is "v=spf1 a mx a:osu1php.osuosl.org. ip4:45.112.84.5 ip6:2a02:cb43:8000::1102" > Change to "v=spf1 a mx -all" Done, but I kept it as ~all. > •• osu1php.osuosl.org. does not seem to exist - so zap it Done > •• If you make the MX change that I suggest then the > following are redundant: ip4:45.112.84.5 ip6:2a02:cb43:8000::1102 Done > •• -all means no other addresses accepted. gmail started making that implicit in > the last few months, it is this that is causing the current problems. I've added ~all instead. > Once that has settled down you should enable sending email from (& > thus to) IPv6 addresses. This is the way that the world is moving. It > is not hard. Done that now too. Fingers crossed to see if any of this is helping. cheers, Derick --- ## #122466 — Derick Rethans — 2024-02-22T16:49:10+00:00 URL: https://externals.io/message/122454#122466 · In reply to: #122464 > > > […] > > Gmail explicitly tightened their requirements for senders this month, > so while it's likely the server change is related, it may not be the > entire cause. > > Specifically, they require: > > - valid spf, with specific rules for quarantining email that has @gmail.com From but doesn't pass spf > - valid dkim > - dmarc configured if the server sends more than 5k emails to Gmail We should have all of that though? > > They also equivocally indicate a requirement for ARC and a List-Id header. We have a List-Id header, but I've not even heard of ARC. It is on our list to investigate though. Our SMTP logs definitely don't say anything about this though. > This is all at https://support.google.com/a/answer/81126. > > I think ARC is a must; without it we'll never get to passing dkim at > least, even if we rewrite from so spf passes. But SPF is now passing? The postmaster tools show this: Date DKIM success rate SPF success rate DMARC success rate 15 Feb 2024 100.0% 100.0% 0.0% 16 Feb 2024 100.0% 100.0% 0.0% 17 Feb 2024 100.0% 100.0% 0.0% 18 Feb 2024 100.0% 100.0% 0.0% 19 Feb 2024 100.0% 100.0% 100.0% 20 Feb 2024 100.0% 100.0% 100.0% > I'm happy to pitch in to help getting this configured if that's > helpful, though I'm also very conscious that too many cooks is often a > greater harm than good when it comes to administration. But reach out > if you think I can be helpful. We could definitely use some people that now email delivery. cheers, Derick --- ## #122467 — Alain D D Williams — 2024-02-22T17:00:35+00:00 URL: https://externals.io/message/122454#122467 · In reply to: #122465 > > […] > > We don't want to change the From address, as that means that replies go to > the list, rather than the original sender. We never had to do any of that > before. I understand that, it also means that someone looking at a list of emails in a mail box sees who (ie individuals) they are from rather than an email list. > Fingers crossed to see if any of this is helping. If things do not improve - change the From address - just for a day to see if that makes things better. -- Alain Williams Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer. +44 (0) 787 668 0256 https://www.phcomp.co.uk/ Parliament Hill Computers. Registration Information: https://www.phcomp.co.uk/Contact.html #include > > > […] > > We should have all of that though? > > > […] > > We have a List-Id header, but I've not even heard of ARC. It is on our list to investigate though. Our SMTP logs definitely don't say anything about this though. > > > […] > > But SPF is now passing? The postmaster tools show this: > > Date DKIM success rate SPF success rate DMARC success rate > 15 Feb 2024 100.0% 100.0% 0.0% > 16 Feb 2024 100.0% 100.0% 0.0% > 17 Feb 2024 100.0% 100.0% 0.0% > 18 Feb 2024 100.0% 100.0% 0.0% > 19 Feb 2024 100.0% 100.0% 100.0% > 20 Feb 2024 100.0% 100.0% 100.0% > > > […] > > We could definitely use some people that now email delivery. The main issue is that the things came together: Us moving to a new listserver and Gmail introducing new ways of fighting spam. The later is what makes life hars now and would also have caused trouble on the old server. We are currently in the process of implementig all of the stuff that gmail expects us to have. The last ones are ARC and one-click unsubscribe. If you want to know more about the hoops we have to jump in addition to moving the list to a new server check out https://blog.google/products/gmail/gmail-security-authentication-spam-protection/ Cheers Andreas -- Andreas Heigl --- ## #122469 — timwolla@bastelstu.be — 2024-02-22T18:45:41+00:00 URL: https://externals.io/message/122454#122469 · In reply to: #122463 Hi > • SPF for lists.php.net is "v=spf1 a mx a:osu1php.osuosl.org. ip4:45.112.84.5 ip6:2a02:cb43:8000::1102" > Change to "v=spf1 a mx -all" > > •• If you make the MX change that I suggest then the > following are redundant: ip4:45.112.84.5 ip6:2a02:cb43:8000::1102 I disagree with that. An MX is the receiver for inbound mail. It does not follow that emails are sent out via the MX. Removing the explicit listing of the IP addresses will just cause accidents in the future, if the MX no longer happens to also send email. If anything the 'a' within the SPF record could be removed, because that's the HTTP box on DigitalOcean. Best regards Tim Düsterhus --- ## #122470 — timwolla@bastelstu.be — 2024-02-22T18:51:11+00:00 URL: https://externals.io/message/122454#122470 · In reply to: #122461 Hi > 1. Don't change the content or the DKIM-signed headers of the email (do > not prepend something in the subject). Then the original DKIM signature > stays valid and the From:-header can stay untouched > Strong +1 for that one. I've suggested the same in the past, unfortunately without much success so far. Best regards Tim Düsterhus --- ## #122471 — Alain D D Williams — 2024-02-22T18:54:01+00:00 URL: https://externals.io/message/122454#122471 · In reply to: #122469 > Hi > > > > […] > > I disagree with that. An MX is the receiver for inbound mail. It does not > follow that emails are sent out via the MX. Removing the explicit listing of > the IP addresses will just cause accidents in the future, if the MX no > longer happens to also send email. If the setup changes and the MX no longer send email then the SPF record can be updated. For now it is a quick way of saying a:php-smtp4.php.net. > If anything the 'a' within the SPF record could be removed, because that's > the HTTP box on DigitalOcean. "a" allows lists.php.net. to send email for the domain. I do not know if it does but could easily believe that administrative/error/... type messages come from there. Regards -- Alain Williams Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer. +44 (0) 787 668 0256 https://www.phcomp.co.uk/ Parliament Hill Computers. Registration Information: https://www.phcomp.co.uk/Contact.html #include This is a test e-mail from a subscribed GMail address, to see if the > "451: Temporary lookup failure" errors are now resolved. > Email successfully received, let's see whether my response is delivered back to you via the list. Best regards Tim Düsterhus