Kafka 실행 안됨 (java.lang.OutOfMemoryError: Java heap space)

Kafka를 재시작하는데 아래 오류가 발생하면서 실행되지가 않았다.

[2024-01-08 20:28:55,718] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.kafka.storage.internals.log.SkimpyOffsetMap.<init>(SkimpyOffsetMap.java:84)
	at kafka.log.LogCleaner$CleanerThread.<init>(LogCleaner.scala:354)
	at kafka.log.LogCleaner.$anonfun$startup$2(LogCleaner.scala:158)
	at kafka.log.LogCleaner$$Lambda$338/1171434979.apply$mcVI$sp(Unknown Source)
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
	at kafka.log.LogCleaner.startup(LogCleaner.scala:157)
	at kafka.log.LogManager.startupWithConfigOverrides(LogManager.scala:584)
	at kafka.log.LogManager.startup(LogManager.scala:516)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:282)
	at kafka.Kafka$.main(Kafka.scala:113)
	at kafka.Kafka.main(Kafka.scala)
[2024-01-08 20:28:55,723] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)
[2024-01-08 20:28:55,729] INFO Shutting down. (kafka.log.LogManager)
[2024-01-08 20:28:56,346] INFO Shutdown complete. (kafka.log.LogManager)
[2024-01-08 20:28:56,350] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2024-01-08 20:28:56,462] INFO Session: 0x102578eeef60000 closed (org.apache.zookeeper.ZooKeeper)
[2024-01-08 20:28:56,462] INFO EventThread shut down for session: 0x102578eeef60000 (org.apache.zookeeper.ClientCnxn)
[2024-01-08 20:28:56,463] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2024-01-08 20:28:56,464] INFO [ThrottledChannelReaper-Fetch]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Fetch]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Fetch]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Produce]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Produce]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Produce]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Request]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Request]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-Request]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,467] INFO [ThrottledChannelReaper-ControllerMutation]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,468] INFO [ThrottledChannelReaper-ControllerMutation]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,468] INFO [ThrottledChannelReaper-ControllerMutation]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-01-08 20:28:56,469] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
[2024-01-08 20:28:56,469] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
[2024-01-08 20:28:56,471] INFO Broker and topic stats closed (kafka.server.BrokerTopicStats)
[2024-01-08 20:28:56,475] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2024-01-08 20:28:56,475] INFO [KafkaServer id=0] shut down completed (kafka.server.KafkaServer)
[2024-01-08 20:28:56,475] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.kafka.storage.internals.log.SkimpyOffsetMap.<init>(SkimpyOffsetMap.java:84)
	at kafka.log.LogCleaner$CleanerThread.<init>(LogCleaner.scala:354)
	at kafka.log.LogCleaner.$anonfun$startup$2(LogCleaner.scala:158)
	at kafka.log.LogCleaner$$Lambda$338/1171434979.apply$mcVI$sp(Unknown Source)
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
	at kafka.log.LogCleaner.startup(LogCleaner.scala:157)
	at kafka.log.LogManager.startupWithConfigOverrides(LogManager.scala:584)
	at kafka.log.LogManager.startup(LogManager.scala:516)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:282)
	at kafka.Kafka$.main(Kafka.scala:113)
	at kafka.Kafka.main(Kafka.scala)
[2024-01-08 20:28:56,476] INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer)

 

이런 저런 시도를 하다보니 아래 디렉토리의 용량이 55Gb나 되는 것을 확인했다.

/var/log/kafka

 

해당 디렉토리명 변경 후 새로 생성해주었더니 Kafka가 정상 기동되었다.

평소 kill -9 명령어로 Kafka를 종료했던 것이 누적되어서 문제가 되었던걸까?

좀 더 공부가 필요할 것 같다.