group corodinate offset commit failure with same offsets

[bt-wil]

[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=67) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError
[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=7) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError
[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=87) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError
[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=27) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError
[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=107) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError
[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=47) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError
[2018-04-20 03:08:31+0000] ERROR    aiokafka.consumer.group_coordinator: OffsetCommit failed for group <GROUP-ID> on partition TopicPartition(topic='<TOPIC>', partition=127) with offset OffsetAndMetadata(offset=19983421, metadata=''): RequestTimedOutError

My first question is should I be worried about this error since it is a error?

The second thing is that I notice that all the topic offset are the same, which seems odd. Looking at the code, it seem to be a reporting error: https://github.com/aio-libs/aiokafka/blob/v0.4.0/aiokafka/consumer/group_coordinator.py#L930-L933, with the last offset was assigned here: https://github.com/aio-libs/aiokafka/blob/v0.4.0/aiokafka/consumer/group_coordinator.py#L850.

The offset might also be reported incorrectly here.
https://github.com/aio-libs/aiokafka/blob/v0.4.0/aiokafka/consumer/group_coordinator.py#L876-L880

[tvoinarovskyi]

My first question is should I be worried about this error since it is a error?

Depends on how many of those you have. If you use autocommit and next commit was successful it will have no impact. Probably the issue was in a connection loss, as long as those don't repeat frequently you should be ok.

The second thing is that I notice that all the topic offset are the same, which seems odd. Looking at the code, it seem to be a reporting error: https://github.com/aio-libs/aiokafka/blob/v0.4.0/aiokafka/consumer/group_coordinator.py#L930-L933, with the last offset was assigned here: https://github.com/aio-libs/aiokafka/blob/v0.4.0/aiokafka/consumer/group_coordinator.py#L850.

The offset might also be reported incorrectly here.
https://github.com/aio-libs/aiokafka/blob/v0.4.0/aiokafka/consumer/group_coordinator.py#L876-L880

Yes, it is an error, sorry about that.

I will look into the log levels. Probably this case should not produce an error.

[tvoinarovskyi]

Checked with Java client. The error is logged in the same ERROR level, https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L748. I will leave the behaviour in aiokafka the same. But it bugs me, that we issue an error per partition. Probably it's better to produce only a single error.