Error: Local: Broker transport failure every 10~15 minutes

See original GitHub issue
[2017-11-25 09:34:41.065] [WARN] info - { Error: Local: Broker transport failure
  origin: 'local',
  message: 'broker transport failure',
  code: -1,
  errno: -1,
  stack: 'Error: Local: Broker transport failure' }
[2017-11-25 09:34:42.355] [WARN] info - { Error: Local: Broker transport failure
  origin: 'local',
  message: 'broker transport failure',
  code: -1,
  errno: -1,
  stack: 'Error: Local: Broker transport failure' }

My node-rdkafka consuming code:

const Kafka = require('node-rdkafka')
const log = require('./log')

let env = process.env.NODE_ENV || 'development'

let stream = Kafka.KafkaConsumer.createReadStream(
  {
    'metadata.broker.list': 'logkfk1:19092,logkfk2:19092',
    'group.id': 'node-consumer-production2',
    'socket.keepalive.enable': true,
    'enable.auto.commit': false
  },  
  {
    'auto.offset.reset': 'earliest'
  },  
  {
    'topics': ['userlog']
  }
)

let counter = 0 
let numMessages = 10

stream.on('data', async function (m) {
  stream.pause()
  counter++
  try {
    let userLog = JSON.parse(m.value)
    let actType = userLog.ul_actType
    let targetId = userLog.ul_targetId
    let addTime = userLog.ul_addTime
    let userId = userLog.ul_userId
    if (actType === 1000 || actType === 1010 || actType === 1030 || actType === 2000) {
      await log.logArticle(targetId, actType, addTime, userId)
    }   

    if (counter % numMessages === 0) {
      stream.consumer.commit()
    }   
  } catch (error) {
    __logger.error(error.message)
  } finally {
    stream.resume()
  }
})

stream.on('error', function (err) {
  __logger.error(err)
  process.exit(1)
})

stream.consumer.on('event.error', function (err) {
  __logger.warn(err) // Here logs the error every 10~15 minutes
})


The kafka cluster info:

$ ./bin/kafka-topics.sh --describe --zookeeper localhost:12181 --topic userlog
Topic:userlog	PartitionCount:4	ReplicationFactor:2	Configs:
	Topic: userlog	Partition: 0	Leader: 1	Replicas: 1,2	Isr: 1,2
	Topic: userlog	Partition: 1	Leader: 2	Replicas: 2,1	Isr: 1,2
	Topic: userlog	Partition: 2	Leader: 1	Replicas: 1,2	Isr: 1,2
	Topic: userlog	Partition: 3	Leader: 2	Replicas: 2,1	Isr: 1,2

Did I miss some config options?

Thanks.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:4
  • Comments:6 (1 by maintainers)

github_iconTop GitHub Comments

7reactions
erik-stephenscommented, Nov 27, 2017

@kidlj I’ve seen that as well due to the broker reaping idle connections. Would be nice if we could see the underlying reason for the disconnect but I think this library & librdkafka are simply relaying what the broker is telling them. Maybe try log.connection.close=false if too noisy.

0reactions
webmakerstevecommented, Feb 10, 2018

Going to close this issue. I believe it is pretty safe to ignore the connection close messages the vast majority of the time. They will still manifest as broker errors when transactions irreparably fail, but librdkafka deals with reconnection internally.

Read more comments on GitHub >

github_iconTop Results From Across the Web

node-rdkafka - debug set to all but I only see broker transport ...
I am trying to connect to kafka server. Authentication is based on GSSAPI. /opt/app-root/src/server/node_modules/node-rdkafka/lib/error.js ...
Read more >
edenhill/librdkafka - Gitter
@edenhill Any tips on how to debug Broker transport failure errors? this is running on kubernetes and it ... I tried to replicate...
Read more >
After the sentry upgrade no more catch errors - On-Premise
I suspect it is a relay problem, This is the error log error logs Part ... error: BrokerTransportFailure (Local: Broker transport failure): ...
Read more >
S02E08 - Checking Kafka advertised.listeners with Go
At the beginning of all this my aim was to learn something new (Go), ... Error (Local: Broker transport failure) getting cluster Metadata....
Read more >
Troubleshooting your Amazon MSK cluster
The Failed authentication ... Too many connects error indicates that a broker is protecting itself because one or more IAM clients are trying...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found