hadoop集成kerberos错误排查-core dump

最新推荐文章于 2024-02-20 11:00:13 发布

原创最新推荐文章于 2024-02-20 11:00:13 发布 · 2.8k 阅读

3 ·

CC 4.0 BY-SA版权

文章标签：

#kerberos #coredump

大数据专栏收录该内容

22 篇文章

订阅专栏

本文记录了一次安装配置 Kerberos 过程中遇到的 KDC coredump 问题及其解决过程。作者通过排查日志发现磁盘空间不足导致数据库异常，清理磁盘并重建数据库后成功解决问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

最近研究kerberos，刚开始安装就遇到了个棘手的问题，core dump了。
好些年不写c程序，这东西还真有点陌生。

[root@hadoop165 ~]# service krb5kdc start
Redirecting to /bin/systemctl start  krb5kdc.service
Job for krb5kdc.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status krb5kdc.service" and "journalctl -xe" for details.

[root@hadoop165 ~]# systemctl status krb5kdc.service
● krb5kdc.service - Kerberos 5 KDC
   Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service; enabled; vendor preset: disabled)
   Active: failed (Result: core-dump) since 一 2017-08-28 16:31:20 CST; 4s ago
  Process: 792 ExecStart=/usr/sbin/krb5kdc -P /var/run/krb5kdc.pid $KRB5KDC_ARGS (code=dumped, signal=SEGV)
 Main PID: 9146 (code=dumped, signal=SEGV)

8月 28 16:31:08 hadoop165 systemd[1]: Starting Kerberos 5 KDC...
8月 28 16:31:20 hadoop165 systemd[1]: krb5kdc.service: control process exited, code=dumped status=11
8月 28 16:31:20 hadoop165 systemd[1]: Failed to start Kerberos 5 KDC.
8月 28 16:31:20 hadoop165 systemd[1]: Unit krb5kdc.service entered failed state.
8月 28 16:31:20 hadoop165 systemd[1]: krb5kdc.service failed.

根据提示又查看了下journalctl -xe。
发现一连串磁盘空间满的报错。
df查看还真是满了。

[root@hadoop165 sbin]# df
文件系统                   1K-块     已用     可用 已用% 挂载点
/dev/mapper/centos-root 52464612 52464592       20  100% /
devtmpfs                 3995156        0  3995156    0% /dev
tmpfs                    4005608        0  4005608    0% /dev/shm
tmpfs                    4005608   410992  3594616   11% /run
tmpfs                    4005608        0  4005608    0% /sys/fs/cgroup
/dev/mapper/centos-home 94919392   247052 94672340    1% /home
/dev/sda1                 508588   128924   379664   26% /boot
tmpfs                     801124        0   801124    0% /run/user/0
tmpfs                     801124        0   801124    0% /run/user/1000

清理磁盘，空间剩余42%。
重新测试，还是报原来的错误。
systemctl status的信息没有说明。
继续查看journalctl -xe的报错，根据时间点发现了具体的报错。
这里发现核心报错为db2.so的segment，怀疑是存储出了问题，结合上面的磁盘空间满了的情况怀疑数据库处于一种中间状态导致程序异常。

[root@hadoop165 logs]# journalctl -xe --no-pager

8月 29 09:45:09 hadoop165 polkitd[931]: Registered Authentication Agent for unix-process:12871:411771162 (system bus name :1.193646 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale zh_CN.UTF-8)
8月 29 09:45:09 hadoop165 systemd[1]: Starting Kerberos 5 KDC...
-- Subject: Unit krb5kdc.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit krb5kdc.service has begun starting up.
8月 29 09:45:10 hadoop165 kernel: krb5kdc[12917]: segfault at 7fd000053166 ip 00007fd0694cafa6 sp 00007ffd92e568e0 error 6 in db2.so[7fd0694bc000+18000]
8月 29 09:45:58 hadoop165 abrt-hook-ccpp[12919]: /var/spool/abrt is 3445558898 bytes (more than 1279MiB), deleting 'ccpp-2017-08-29-09:41:58-12318'
8月 29 09:45:59 hadoop165 systemd[1]: krb5kdc.service: control process exited, code=dumped status=11
8月 29 09:45:59 hadoop165 systemd[1]: Failed to start Kerberos 5 KDC.
-- Subject: Unit krb5kdc.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit krb5kdc.service has failed.
-- 
-- The result is failed.
8月 29 09:45:59 hadoop165 systemd[1]: Unit krb5kdc.service entered failed state.
8月 29 09:45:59 hadoop165 systemd[1]: krb5kdc.service failed.
8月 29 09:45:59 hadoop165 polkitd[931]: Unregistered Authentication Agent for unix-process:12871:411771162 (system bus name :1.193646, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale zh_CN.UTF-8) (disconnected from bus)
8月 29 09:46:00 hadoop165 abrt-server[12991]: 正在生成核心转储

将数据库数据清空，并重新生成，测试问题解决。

[root@hadoop165 ~]# cd /var/kerberos/krb5kdc/
[root@hadoop165 krb5kdc]# ls
kadm5.acl  kdc.conf  principal  principal.kadm5  principal.kadm5.lock  principal.ok
[root@hadoop165 krb5kdc]# rm principal* -rf
[root@hadoop165 krb5kdc]# kdb5_util create -r BONC -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'BONC',
master key name 'K/M@BONC'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key: 
Re-enter KDC database master key to verify: 
[root@hadoop165 krb5kdc]#