Enabling @KafkaListener to take in variable topic names from application.yml file

See original GitHub issue

I am attempting to load in multiple topics to a single @KafkaListener but am running into trouble as I believe it is looking for a constant value, but initializing the topics variable from the application.yml file causing something issues, I was wondering if someone could help me troubleshoot this issue, or provide me with direction into how to load multiple Kafka topics into a single KafkaListener. I realize I could do an object with comma delimited values representing the topics, but I want to be able to add topics through a config file, rather than changing code in the code base.

KafkaWebSocketConnector.java

@Component
public class KafkaWebSocketConnector
{


   @Value("${spring.kafka.topics}")
   private String[] topics;

   @KafkaListener(topics = topics)
   public void listen(ConsumerRecord<?, Map<String, String>> message)
   {
      log.info("Received messages on topic [{}]: [{}]", message.topic(), message.value());
      String dest = "/" + message.topic();
      log.info("destination = {}", dest);
      log.info("msg: {}", message);
      messageTemplate.convertAndSend(dest, message.value());
   }
}

application.yml

spring:
  kafka:
    consumer:
      auto-offset-reset: earliest
      group-id: kafka-websocket-connector
    topics: flight-events,
      flight-time-events,
      canceled-events,
      pax-events,
      flight-delay-events

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:1
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

30reactions
garyrussellcommented, Jul 5, 2017

You can use a SpEL expression; there’s an example in EnableKafkaIntegrationTests

@KafkaListener(id = "foo", topics = "#{'${topicOne:annotated1,foo}'.split(',')}")

In your case "#{'${spring.kafka.topics}'.split(',')}"

1reaction
garyrussellcommented, Sep 13, 2019

Please don’t comment on old closed issues. You should ask a question on Stack Overflow (tagged with spring-kafka) and show your code and configuration.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Enabling @KafkaListener to take in variable topic names from ...
Enabling @KafkaListener to take in variable topic names from application.yml file ... I am attempting to load in multiple topics to a single...
Read more >
Enabling @KafkaListener to take in variable topic names from ...
yml file. I am attempting to load in multiple topics to a single @KafkaListener but am running into trouble as I believe it...
Read more >
Spring Boot and Kafka – Practical Example
A practical example project using Spring Boot and Kafka with multiple consumers and different serialization methods.
Read more >
Spring for Apache Kafka
The following simple Spring Boot application provides an example of how to use the same template to send to different topics, each using...
Read more >
Dynamic Spring Boot Kafka Consumer | Blibli.com Tech Blog
Then i started to think, what if I have a logic process that I want to apply for some kafka topics? Do I...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found