[BUG] Send message to service bus using ServiceBusSenderClient get timeout

See original GitHub issue

Describe the bug Send message to service bus using ServiceBusSenderClient get timeout on databricks 10.4 LTS (spark 3.2.1). Works fine with DB 9.1 LTS (spark 3.1.2)

Exception or Stack Trace IllegalStateException: Timeout on blocking read for 245600000000 NANOSECONDS at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:123) at reactor.core.publisher.Mono.block(Mono.java:1731)

To Reproduce Steps to reproduce the behavior (HAPPY PATH):

build a sample scala (using java 8) client using official samples e.g. https://docs.microsoft.com/en-us/java/api/overview/azure/messaging-servicebus-readme?view=azure-java-stable#send-messages
deploy and run the jar on azure databricks cluster ver. 9.1 LTS
messages are correctly send and received by the configured service bus instance

Steps to reproduce the behavior (ERROR): 4) change databricks cluster to ver 10.4 LTS 5) run the job and get the error

Code Snippet Add the code snippet that causes the issue.

val senderClient = builder.sender().topicName(serviceBusChannelName).buildClient() 
// prepare messages
val jsonMessages = Seq(
  new ServiceBusMessage(BinaryData.fromString("{\"color\":\"red\"}")),
  new ServiceBusMessage(BinaryData.fromString("{\"color\":\"blue\"}")),
  new ServiceBusMessage(BinaryData.fromString("{\"color\":\"orange\"}")),
  new ServiceBusMessage(BinaryData.fromString("{\"color\":\"green\"}")))

senderClient.sendMessages(jsonMessages.asJava)

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

OS: azure-cloud
IDE: IntelliJ (with scala 2.12 + Java 8)
Library/Libraries: com.azure:azure-messaging-servicebus:7.8.0
Java version: 8
App Server/Environment: Databricks 9.1/10.4 LTS (with spark+scala)
Frameworks: N.A.

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn’t provide solution for the problem, please provide:

verbose dependency tree (mvn dependency:tree -Dverbose)
exception message, full stack trace, and any available logs

Additional context Add any other context about the problem here.

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

Bug Description Added
Repro Steps Added
Setup information Added

Issue Analytics

State:
Created a year ago
Comments:5 (2 by maintainers)

Top GitHub Comments

1reaction

anuchandycommented, May 16, 2022

Good to hear! Shading is the only solution I’m aware of if the pinned dependencies (e.g., Reactor here) in a hosted environment (e.g., DataBricks here) cause conflicts. It worked for you because the shading had relocation defined for Reactor

<relocation>
  <pattern>reactor</pattern>
  <shadedPattern>ms.shaded.reactor</shadedPattern>
</relocation>

I’m not sure about jackson-dataformat-xml though; probably some other parts of your project needed it.

Closing this as you’re unblocked.

1reaction

gioma68commented, May 16, 2022

Hi @anuchandy the solution was to implement shaded jar as in your referenced document. I don’t know if the problem is specific for Databricks 10.4 LTS but I need to remove fasterxml from shading and to add a specific import version in POM.XML for com.fasterxml.jackson.dataformat now sending to ServiceBus with sdk work well. Thanks for your kindly support.

    <dependency>            
          <groupId>com.fasterxml.jackson.dataformat</groupId>
         <artifactId>jackson-dataformat-xml</artifactId>
         <version>2.10.1</version>
     </dependency>