kafka message size

一：异常信息

21/09/23 10:39:46 ERROR internals.ErrorLoggingCallback: Error when sending message to topic ad_source_mob_prtsc with key: null, value: 5242233 bytes with error:
org.apache.kafka.common.errors.RecordTooLargeException: The message is 5242321 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.
>21/09/23 10:39:46 ERROR internals.ErrorLoggingCallback: Error when sending message to topic ad_source_mob_prtsc with key: null, value: 6376666 bytes with error:
org.apache.kafka.common.errors.RecordTooLargeException: The message is 6376754 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration.

二：处理办法

都知道是配置***max***size的配置项，但细节往往容易忽略：

1. server、producer、consumer都需要配置
2. server中有两个，其中【message.max.bytes】是个临界，设为aMb，后面的配置需要根据临界调整
3. producer中的【max.request.size】要小于等于临界a
4. consumer中的【fetch.message.max.bytes】要大于等于临界a
5. server中还有另一个【replica.fetch.max.bytes】，这个是每个分区尝试获取的字节数

 总结一下就是小心使得万年船。

对了，最后还要记得重启kafka才能生效。