Debugging Kernel Crash using Kernel Crash Dump tool
1. Install the kernel crash dump utility with the following command
$ sudo apt install linux-crashdump
Follow the steps mentioned in this link Kernel Crash Dump
2. Follow the steps to setup linux-crashdump and test a sample crash:
# echo c > /proc/sysrq-trigger
Above test generates/saves the kernel core dump file, dmesg under /var/crash which we can analyze using linux crash tool.
linux@linux:/var/crash/201706071402$ sudo crash /home/test/vmlinux /var/crash/201706071402/dump.201706071402
crash 7.0.3
Copyright (C) 2002-2013 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
WARNING: kernel version inconsistency between vmlinux and dumpfile
KERNEL: /home/test/vmlinux
DUMPFILE: /var/crash/201706071402/dump.201706071402 [PARTIAL DUMP]
CPUS: 4
DATE: Wed Jun 7 14:02:11 2017
UPTIME: 00:02:16
LOAD AVERAGE: 1.57, 0.85, 0.33
TASKS: 369
NODENAME: cae
RELEASE: 3.16_kernel+
VERSION: #2 SMP Mon Jun 5 16:31:44 IST 2017
MACHINE: x86_64 (1799 Mhz)
MEMORY: 3 GB
PANIC: "Oops: 0002 [#1] SMP " (check log for details)
PID: 2103
COMMAND: "bash"
TASK: ffff880091645d40 [THREAD_INFO: ffff88008f202000]
CPU: 1
STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 2103 TASK: ffff880091645d40 CPU: 1 COMMAND: "bash"
#0 [ffff88008f203aa8] machine_kexec at ffffffff8103cf82
#1 [ffff88008f203b00] crash_kexec at ffffffff810c3a93
#2 [ffff88008f203bc8] __die at ffffffff816f16e8
#3 [ffff88008f203bf0] __bad_area_nosemaphore at ffffffff816e47e5
#4 [ffff88008f203c38] __bad_area_nosemaphore at ffffffff816e4865
#5 [ffff88008f203c80] __flush_tlb_all at ffffffff816e4bde
#6 [ffff88008f203ca8] __do_page_fault at ffffffff816f43d2
#7 [ffff88008f203da8] notifier_call_chain at ffffffff816f44ec
#8 [ffff88008f203dd0] machine_check at ffffffff816f0a38
#9 [ffff88008f203e88] __handle_sysrq at ffffffff8141b9f2
#10 [ffff88008f203ec0] write_sysrq_trigger at ffffffff8141beea
#11 [ffff88008f203ee0] proc_reg_write at ffffffff811f105d
#12 [ffff88008f203f00] vfs_write at ffffffff8118e89d
#13 [ffff88008f203f40] sys_write at ffffffff8118f269
#14 [ffff88008f203f80] int_careful at ffffffff816f8e2f
RIP: 00007f2c145e6f80 RSP: 00007fff7152b208 RFLAGS: 00000246
RAX: ffffffffffffffda RBX: ffffffff816f8e2f RCX: ffffffffffffffff
RDX: 0000000000000002 RSI: 000000000203e408 RDI: 0000000000000001
RBP: 000000000203e408 R8: 000000000000000a R9: 00007f2c14f1e740
R10: 00007f2c148b86a0 R11: 0000000000000246 R12: 0000000000000001
R13: 0000000000000002 R14: 00007f2c148ba400 R15: 0000000000000002
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
crash>
crash>
Hope this helps in debugging the kernel crashes. Now some deep house music Frits Wentink - Deep House Amsterdam
Comments
Post a Comment