Uploaded image for project: 'mod-pubsub'
  1. mod-pubsub
  2. MODPUBSUB-186

SSL connection using DNS CNAME doesn't work

    XMLWordPrintable

Details

    • 1
    • Folijet Support
    • R2 2021 Bugfix

    Description

      For EBSCO hosted accounts, we use DNS CNAME records for Kafka endpoints:

      kafka.igp4.folio-eis.us-east-1 CNAME b-1.tenant-ssl.qg3qvo.c3.kafka.us-east-1.amazonaws.com
      

       

      Container start up is failed:

      #LOG_RECORD10:40:48 [] [] [] [] INFO  afkaTopicServiceImpl Some of the topics [QM_RECORD_UPDATED] were not created. Cause: Call(callName=createTopics, deadlineMs=1626172848515) timed out at 1626172848516 after 1 attempt(s)
      10:40:48 [] [] [] [] INFO  aConsumerServiceImpl Subscribed to topic {igp4.pub-sub.fs00001007.QM_RECORD_UPDATED.mod-pubsub-2.0.8}
      10:40:49 [] [] [] [] INFO  afkaTopicServiceImpl Some of the topics [QM_SRS_MARC_BIB_RECORD_UPDATED] were not created. Cause: Call(callName=createTopics, deadlineMs=1626172848974) timed out at 1626172848975 after 1 attempt(s)
      10:40:49 [] [] [] [] INFO  afkaTopicServiceImpl Some of the topics [FEE_FINE_BALANCE_CHANGED] were not created. Cause: Call(callName=createTopics, deadlineMs=1626172848974) timed out at 1626172848975 after 1 attempt(s)
      10:40:49 [] [] [] [] INFO  afkaTopicServiceImpl Some of the topics [ITEM_CHECKED_OUT] were not created. Cause: Call(callName=createTopics, deadlineMs=1626172848974) timed out at 1626172848975 after 1 attempt(s)
      java.lang.OutOfMemoryError: Java heap space
      Dumping heap to /usr/ms/mod-pubsub.hprof ...
      10:40:51 [] [] [] [] INFO  afkaTopicServiceImpl Some of the topics [ITEM_CHECKED_IN] were not created. Cause: Call(callName=createTopics, deadlineMs=1626172848974) timed out at 1626172848975 after 1 attempt(s)
      Heap dump file created [409011493 bytes in 1.811 secs]
      10:40:51 [] [] [] [] INFO  afkaTopicServiceImpl Some of the topics [ITEM_DECLARED_LOST] were not created. Cause: Call(callName=createTopics, deadlineMs=1626172848974) timed out at 1626172848975 after 1 attempt(s)
      #
      # java.lang.OutOfMemoryError: Java heap space
      # 
      

       

      Checking SSL in container:

      # cat client.properties
      security.protocol=SSL
      ssl.truststore.location=/usr/ms/kafka.client.truststore.jks
      ssl.keystore.location=/usr/ms/kafka.client.keystore.jks
      ssl.keystore.password=Yah...
      ssl.truststore.password=changeit
      ssl.key.password=Yah...
      
      # ./bin/kafka-topics.sh --bootstrap-server b-1.tenant-ssl.qg3qvo.c3.kafka.us-east-1.amazonaws.com:9094 --command-config client.properties --list
      __amazon_msk_canary
      __amazon_msk_canary_state
      __consumer_offsets
      igp4.pub-sub.fs00001007.FEE_FINE_BALANCE_CHANGED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.FEE_FINE_BALANCE_CHANGED.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.ITEM_AGED_TO_LOST.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.ITEM_AGED_TO_LOST.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.ITEM_CHECKED_IN.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.ITEM_CHECKED_IN.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.ITEM_CHECKED_OUT.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.ITEM_CHECKED_OUT.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.ITEM_CLAIMED_RETURNED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.ITEM_CLAIMED_RETURNED.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.ITEM_DECLARED_LOST.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.ITEM_DECLARED_LOST.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.LOAN_DUE_DATE_CHANGED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.LOAN_DUE_DATE_CHANGED.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.LOAN_RELATED_FEE_FINE_CLOSED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.LOAN_RELATED_FEE_FINE_CLOSED.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.LOG_RECORD.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.LOG_RECORD.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.QM_ERROR.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.QM_ERROR.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.QM_INVENTORY_INSTANCE_UPDATED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.QM_INVENTORY_INSTANCE_UPDATED.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.QM_RECORD_UPDATED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.QM_RECORD_UPDATED.mod-pubsub-2.0.8
      igp4.pub-sub.fs00001007.QM_SRS_MARC_BIB_RECORD_UPDATED.mod-pubsub-2.0.5
      igp4.pub-sub.fs00001007.QM_SRS_MARC_BIB_RECORD_UPDATED.mod-pubsub-2.0.8
      

      But CNAME records doesn't work:

      # ./bin/kafka-topics.sh --bootstrap-server $KAFKA_HOST:9094 --command-config client.properties --list
      [2021-07-13 11:03:42,571] ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (kafka.igp4.folio-eis.us-east-1/10.23.10.243:9094) failed authentication due to: SSL handshake failed (org.apache.kafka
      .clients.NetworkClient)
      [2021-07-13 11:03:42,579] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
      org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
      Caused by: javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching kafka.igp4.folio-eis.us-east-1 found.
              at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
              at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:349)
              at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
              at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287)
              at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
              at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
              at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
              at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
              at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
              at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)
              at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)
              at java.base/java.security.AccessController.doPrivileged(Native Method)
              at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)
              at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:430)
              at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:514)
              at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:368)
              at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:291)
              at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178)
              at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543)
              at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
              at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:561)
              at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1333)
              at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1264)
              at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching kafka.igp4.folio-eis.us-east-1 found.
      

      Here are envs. for mod-pubsub container:

      # env
      KAFKA_SSL_TRUSTSTORE_PASSWORD=changeit
      KAFKA_HOST=kafka.igp4.folio-eis.us-east-1
      KAFKA_SSL_KEYSTORE_PASSWORD=Yah...
      KAFKA_SSL_KEY_PASSWORD=Yah...
      NUMBER_OF_PARTITIONS=1
      REPLICATION_FACTOR=2
      KAFKA_PORT=9094
      KAFKA_SSL_KEYSTORE_LOCATION=/usr/ms/kafka.client.keystore.jks
      KAFKA_SSL_TRUSTSTORE_LOCATION=/usr/ms/kafka.client.truststore.jks
      KAFKA_SECURITY_PROTOCOL=SSL
      ...
      

      TestRail: Results

        Attachments

          Issue Links

            Activity

              People

                Kateryna Senchenko Kateryna Senchenko
                Miroshnichenko Stanislav Miroshnichenko
                Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases