There is no leader for this topic-partition as we are in the middle of a leadership election

Describe the bug With a kafka started with KAFKA_AUTO_CREATE_TOPICS_ENABLE, I get “There is no leader for this topic-partition as we are in the middle of a leadership election” errors when sending a message on a non existing topic.

Also described here

I use the admin tool to create non existing topic as a workaround:

await this.admin.connect()
const existingTopics = await this.admin.listTopics()
if (!existingTopics.some((topic) => topic === topics.create.event)) {
  await this.admin.createTopics({
    waitForLeaders: true,
    topics: [
      { topic: topics.create.event }
    ]
  })
}
await this.admin.disconnect()

But since I have auto create topics on it does not really make sense to create them upfront.

To Reproduce Start a kafka with no topics send a message with a non-existing topic, and get the error.

Expected behavior No error produced

Observed behavior The error is produced

Feature request Make producer.send() waitForLeaders like admin.createTopics() does

Environment:

• KafkaJS version 1.15.0
• Kafka version latest
• NodeJS version 15.5.1