Skip to content

Testing mail.smtp.ssl.trust seems to be broken #187#188

Open
jbescos wants to merge 3 commits into
eclipse-ee4j:masterfrom
jbescos:issue187
Open

Testing mail.smtp.ssl.trust seems to be broken #187#188
jbescos wants to merge 3 commits into
eclipse-ee4j:masterfrom
jbescos:issue187

Conversation

@jbescos

@jbescos jbescos commented Nov 7, 2025

Copy link
Copy Markdown
Member

It does not solve #187 but it adds some tests nice to have, a reproducer for that issue (test is disabled to avoid failure), and it has a comment where the issue happens.

I tried to fix it in this way:

                boolean endpointIdentificationAlgorithm = true;
                if (sf instanceof MailSSLSocketFactory) {
                    MailSSLSocketFactory msf = (MailSSLSocketFactory) sf;
                    endpointIdentificationAlgorithm = !(msf.isTrustAllHosts() || msf.getTrustedHosts() != null);
                }
                if (endpointIdentificationAlgorithm) {
                    params.setEndpointIdentificationAlgorithm(eia);
                }

But there are other tests failing.

The root issue is that when setEndpointIdentificationAlgorithm is invoked with a not null argument, MailTrustManager is replaced, and this method we have there is not invoked:

        @Override
        public void checkServerTrusted(X509Certificate[] certs, String authType)
                throws CertificateException {

            if (!(isTrustAllHosts() || getTrustedHosts() != null))
                adapteeTrustManager.checkServerTrusted(certs, authType);
        }

I am not brave enough to change this behavior because there are tests failing, and it has been working in this way forever. It just started to be more visible since this commit.

Signed-off-by: Jorge Bescos Gascon <jorge.bescos.gascon@oracle.com>
@jmehrens

jmehrens commented Nov 8, 2025

Copy link
Copy Markdown
Contributor

Another thought was I wrote some notes in the COMPAT.TXT

In previous releases of Angus Mail, a custom X509ExtendedTrustManager or a
custom SSLSocket was able to disable verification at the socket level but
could not disable hostname verification in Angus Mail provided by
mail..ssl.checkserveridentity session property. This behavior has
been relaxed in Angus Mail 2.0.3, where even though the
mail..ssl.checkserveridentity session property was set to true the
hostname verification may be disabled by a X509ExtendedTrustManager
implementation, SSLSocketFactory implementation, or a SSLSocket
implementation.

I wonder if the TrustManager in MailSSLSocketFactory should extend/implement X509ExtendedTrustManager. Then disable EIA as needed.

@jmehrens

jmehrens commented Nov 9, 2025

Copy link
Copy Markdown
Contributor

@jbescos

I tried to fix it in this way:
...
But there are other tests failing.

I think that is right fix. However, it should only work that way when checkserveridentity has not been explicitly set. That might fix failing tests.

@jbescos

jbescos commented Nov 11, 2025

Copy link
Copy Markdown
Member Author

@jbescos

I tried to fix it in this way:
...
But there are other tests failing.

I think that is right fix. However, it should only work that way when checkserveridentity has not been explicitly set. That might fix failing tests.

I will insist more. I have a new laptop and I am configuring everything, but I hope I will be able to comeback to this during the week.

@jmehrens

Copy link
Copy Markdown
Contributor

Additionally we might have to implement X509ExtendedTrustManager and conditionally set EIA when not a trusted host and checkserveridentity was not explicitly set.

@jbescos
jbescos force-pushed the issue187 branch 2 times, most recently from 4626c42 to 942f6bd Compare November 19, 2025 08:57
Signed-off-by: Jorge Bescos Gascon <jorge.bescos.gascon@oracle.com>
fail("Expects exception");
} catch (MessagingException e) {
e.printStackTrace();
assertEquals("Server is not trusted: mailtest.local", e.getCause().getMessage());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need another test that is a clone of this test but

  1. mail.smtps.ssl.checkserveridentity=false
  2. Expected result is pass because other domain is trusted and not subject to EIA.

@jbescos jbescos Nov 25, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently this test would fail, because this line does not check the value of ssl.checkserveridentity.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jmehrens jmehrens linked an issue Nov 19, 2025 that may be closed by this pull request
Comment thread doc/src/main/resources/docs/COMPAT.txt Outdated
Prior to this change, MailTrustManager only received the certificate chain
and authentication type. As a result, it could not participate in hostname
verification triggered by the SSLParameters endpoint identification
algorithm.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe EIA worked as long as MailSSLSocketFactory wasn't installed explicitly or implicitly as it would have used default SSL classes which extend X509ExtendedTrustManager

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

throw new AssertionError(t);
}
public void testSSLCheckServerIdentityNull() throws Throwable {
// localhost should fail? See TestSSLSocketFactory#84 -> ((MailSSLSocketFactory) defaultFactory).setTrustedHosts("localhost");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of the test was assert that the EIA was being enforced by the JDK. I think only change needed for original code is to switch paramter from "localhost" to "mailtest.local" in each test.

@jbescos jbescos Nov 25, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying this and it is not working as expected because we are missing to implement the validation of real values of trusted hosts. Here we only check that trusted hosts are not null and we should check values of it with the real host of the request.

Then, what is happening is that TestSSLSocketFactory sets localhost as trusted hosts. This has the same effect than trusting all hosts.

I am not sure why this was not implemented and why nobody reported this before.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I read that you were talking about this in the next comment.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I think it broke when I implemented EIA in SocketFetcher. I think the checks have to moved to trustmanager. All hostname checks happened in SocketFetcher and I only moved the ones covered by EIA which is incorrect.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why this was not implemented and why nobody reported this before.

I would assume it is because checkserveridentity defaulted to false as you pointed out.

The old flow, prior to EIA validation was to suppress forwarding in trust manager and optional perform hostname validation and isServerTrusted in SocketFetcher.

This legacy behavior can be simulated by setting

mail.<protocol>.ssl.hostnameverifier.class=legacy
mail.<protocol>.ssl.checkserveridentity=false

I think even in older versions of JakartaMail/JavaMail there wouldn't have been a way to trust a host and check not trusted hosts. Which is why hostnameverifier is invoked unconditionally to MailSSLSocketFactory.

}

private boolean shouldDelegate() {
return !(isTrustAllHosts() || getTrustedHosts() != null);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double check me on this, but i think for X509ExtendedTrustManager methods this is not correct when EIA is enabled and we are trusting a list of hosts. In that case we should short circuit if matching, otherwise delegate.

Essentially we should call MailSSLSocketFactory::isServerTrusted to determine if we delegate. Or rather update this private method to take a host name.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might have to remove MailSSLSocketFactory::isServerTrusted call from SocketFetcher and just have the call happen in the trustmanager. The reason I think we have to do that is because we don't reflective invoke EIA from SocketFetcher.

Signed-off-by: Jorge Bescos Gascon <jorge.bescos.gascon@oracle.com>
if (sf instanceof MailSSLSocketFactory) {
MailSSLSocketFactory msf = (MailSSLSocketFactory) sf;
if (!msf.isServerTrusted(host, sslsocket)) {
if (checkServerIdentity && !msf.isServerTrusted(host, sslsocket)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe all of this code for MailSSLSocketFactory should be removed from SocketFetcher. The isServerTrusted should be invoked in all TrustManager methods and the result should be the decision to forward the call.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I get your point and I am working on it.

@jbescos

jbescos commented Nov 27, 2025

Copy link
Copy Markdown
Member Author

I was implementing it with the suggested changes, and I found out that we don't support to specify custom trust managers:

So I am getting the next error in one test that is supposed to fail with an error Server is not trusted: mailtest.local:

[INFO] Running org.eclipse.angus.mail.smtps.SmtpStartTlsTrustTest
javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:365)
	at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:287)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:204)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
	at org.eclipse.angus.mail.smtps.SmtpStartTlsTrustTest.lambda$before$0(SmtpStartTlsTrustTest.java:101)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
jakarta.mail.MessagingException: Could not connect to SMTP host: mailtest.local, port: 16025;
  nested exception is:
	javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at org.eclipse.angus.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2245)
	at org.eclipse.angus.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:729)
	at jakarta.mail.Service.connect(Service.java:335)
	at jakarta.mail.Service.connect(Service.java:220)
	at jakarta.mail.Service.connect(Service.java:169)
	at org.eclipse.angus.mail.smtps.SmtpStartTlsTrustTest.testHostnameVerificationFails(SmtpStartTlsTrustTest.java:181)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:316)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:240)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:214)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:155)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:378)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:316)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1318)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1195)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1138)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:447)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:201)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
	at org.eclipse.angus.mail.util.SocketFetcher.configureSSLSocket(SocketFetcher.java:665)
	at org.eclipse.angus.mail.util.SocketFetcher.createSocket(SocketFetcher.java:409)
	at org.eclipse.angus.mail.util.SocketFetcher.getSocket(SocketFetcher.java:243)
	at org.eclipse.angus.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2193)
	... 35 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:388)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:271)
	at java.base/sun.security.validator.Validator.validate(Validator.java:256)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:230)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
	at org.eclipse.angus.mail.util.MailSSLSocketFactory$MailTrustManager.checkServerTrusted(MailSSLSocketFactory.java:460)
	at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1302)
	... 50 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:148)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:129)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:383)

Do we create new properties to specify custom trust stores?. Or at least being able to inject it for testing purposes.

A part of this, to be honest, I am getting puzzled and confused with the checkServerIdentity property. Maybe it is better to document that: mail.smtp.ssl.trust works in combination with mail.smtp.ssl.checkserveridentity set to false.

@jmehrens

Copy link
Copy Markdown
Contributor

Do we create new properties to specify custom trust stores?. Or at least being able to inject it for testing purposes.

A part of this, to be honest, I am getting puzzled and confused with the checkServerIdentity property. Maybe it is better to document that: `mail.smtp.ssl.trust works in combination with mail.smtp.ssl.checkserveridentity set to false

@jbescos
In the test, you should be able to create a MailSSLSocketFactory object and put it in properties (no dot class suffix on key) then call

https://eclipse-ee4j.github.io/angus-mail/docs/api/org.eclipse.angus.mail/org/eclipse/angus/mail/util/MailSSLSocketFactory.html#setTrustManagers(javax.net.ssl.TrustManager...) with mock trust manager.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"mail.smtp.ssl.trust" seems to be broken

2 participants