Sometimes we may want to investigate why a linux system was shutdown unexpectedly .
As you know already , the only place we get this information from is /var/log directory .
last -x command will display some information about shutdown/reboot like below .
runlevel (to lvl 3) 2.6.32-754.17.1. Sat Sep 7 16:24 – 18:41 (02:17)
reboot system boot 2.6.32-754.17.1. Sat Sep 7 16:24 – 18:41 (02:17)
runlevel (to lvl 0) 2.6.32-754.17.1. Sat Sep 7 14:58 – 16:24 (01:26)
If it is an unexpected shutdown due to power failure or something then you will see just something runlevel (to lvl 0) . This could be a forceful shutdown . If it is in a datacenter we can ask support team for an explanation .
This command may display some useful information .
grep -iv ‘: starting\|kernel: .*: Power Button\|watching system buttons\|Stopped Cleaning Up\|Started Crash recovery kernel’ /var/log/messages /var/log/messages /var/log/apcupsd* | grep -iw ‘recover[a-z]*\|power[a-z]*\|shut[a-z ]*down\|rsyslogd\|ups’
/var/log/messages:Sep 7 16:24:40 localhost kernel: EXT4-fs (md2): INFO: recovery required on readonly filesystem
/var/log/messages:Sep 7 16:24:40 localhost kernel: EXT4-fs (md2): write access will be enabled during recovery
/var/log/messages:Sep 7 16:24:40 localhost kernel: EXT4-fs (md2): recovery complete
If the last shutdown was due to crash or power failure then at next successful boot system logs will write something like “recovery required on readonly filesystem” followed by “recovery complete” .
We can know more about power or system health If we have KVM access to server and event log is enabled for that .