Randomly get topic ... whcih is not assigned to any partitions exception
See original GitHub issueBug Report
Current behavior
I am using NestJs with Kafka and the error The client consumer subscribed to the topic (${topic}) whcih is not assigned to any partitions., is often thrown with failed code execution
Some other time I get The client consumer did not subscribe to the corresponding reply topic (${topic}).,
The setup is very basic
Input Code
@Client(serviceConfig)
streamClient;
onModuleInit() {
this.streamClient.subscribeToResponseOf(...);
};
}
onModuleDestroy() {
this.streamClient.close();
}
export const serviceConfig = {
transport: Transport.KAFKA,
options: {
client: {
clientId: '...',
brokers: [`...`],
},
consumer: {
groupId: '...',
},
},
};
Expected behavior
The code often runs fine but sometimes the error just happens, specifically around the time of rebalancing
My configuration consists of 3 kafka broker and 2 NestJS instances running in paralell with same source code / same setting (kubernete setup)
I put also my situation here
Possible Solution
I see that my NestJS version is before this commit https://github.com/nestjs/nest/commit/b0a0623cf8374c2dca19a5aaff1ded19ebda5926#diff-72a9f557d4f070b2a084eaf4ba5d7e3ad02a496f670958e6efee128624e6b097
I am not sure if the behaviour i am facing is related to what the commit is fixxing?
Environment
"@nestjs/common": "^7.0.0",
"@nestjs/core": "^7.0.0",
"@nestjs/microservices": "^7.4.2",
Thanks for the help!Issue Analytics
- State:
- Created 3 years ago
- Comments:6 (2 by maintainers)
Top Related StackOverflow Question
Why was this closed? Having a unique consumer group per instance is not really a solution because while you want all instances to consume the reply message you do not want that behaviour for regular messages which should be processed only once.
I am afraid this happens with specific HA configuration so it would be kind of hard to provide a minimal repo. I will still try to create a minimal reproduction and update when possible Thanks a lot @Tony133