kernel lockup on machine that used to work fine only yesterday

[JackC]

Hi everyone,

seems like I got a serious kernel problem on my Fedora 16, 64bit.
Only yesterday everything worked just perfect. Today, after booting my
whole desktop froze a few seconds after the OS has been loaded.

After setting the runlevel to 3 (setting that symlink to multi-user.target that
is), I get the following:

[ 104.163.988] BUG: soft lockup - CPU#2 stuck for 22s! [kworker/2.2.81]

followed by a lot of traces and the same message in variations again.

I had a similar problem with kernel lockups a few weeks ago when trying
to install adobe reader via Adobe's yum repository, which I manged to solve back
then by removing the newly installed packages again. However, this time
I did not install anything. The last entry in /var/log/yum.log is from Feb 9th.
The last thing I did, was to configure some test shares with samba.
I already removed all the samba packages, to no avail. It's not tied to
a specific kernel ... the latest one I have installed (3.2.3-2.fc16.x86_64)
produces the same problem as any earlier version and it's not the X server
(since the problem stays in pure text mode "runlevel 3" as well).

There's nothing in /var/log messages either. The system just works for about
half a minute as usual then hangs with this error. I can only assume that
something get's loaded after that time, that causes the system to freeze. Selecting
the rescue mode kernel in grub works but doesn't give me any more clues
either. CPU is an AMD FX-4100, with 8GBs of memory installed. Any hints
or suggestions on how I could trace this problem or provide more information
would be greatly appreciated, as I'm close to a nervous breakdown.

[EvilRedHorse]

# I have had this problem while overclocking. Are you using your default BIOS settings or have you made modifications?

[PatMcLJr]

when this type of thing has happened to me it was related to a hardware issue, in my cases hard drive that got flaky, and a stick of memory that didn't want to work any more

[JackC]

Hi,

thanks for the replies. Meanwhile I've reinstalled the whole Fedora 16 system, only keeping the /home filesystem. It worked fine for a fewl days and now I've got the same error again.
I haven't overclocked the system and I'm using the deault BIOS with only one modification:
I've enabled hardware support for virtualization which is disabled by default. Again,
these settings worked fine for weeks. Windooze still works fine. it's just Fedora that freezes.
Wish it was the other way round - I rely on linux a lot.
One thing I noticed, right after logging in, everything seems to work as usual, for some reason the network doesn't seem to provide any reaction, maybe it's related to the network
somehow? Any idea on where I could get more information?
The last thing that happens before the whole thing finally freezes is that abrt spams
a dozen times "your kernel is tainted by flags". I guess that one is related to the nvidia
propriety driver, which is absoltely necessary, because otherwise the video card fan produces so much noise that it doesnt really matter any more if it works or not.

[marko]

Originally Posted by JackC

Hi,

thanks for the replies. Meanwhile I've reinstalled the whole Fedora 16 system, only keeping the /home filesystem. It worked fine for a fewl days and now I've got the same error again.
I haven't overclocked the system and I'm using the deault BIOS with only one modification:
I've enabled hardware support for virtualization which is disabled by default. Again,
these settings worked fine for weeks. Windooze still works fine. it's just Fedora that freezes.
Wish it was the other way round - I rely on linux a lot.
One thing I noticed, right after logging in, everything seems to work as usual, for some reason the network doesn't seem to provide any reaction, maybe it's related to the network
somehow? Any idea on where I could get more information?
The last thing that happens before the whole thing finally freezes is that abrt spams
a dozen times "your kernel is tainted by flags". I guess that one is related to the nvidia
propriety driver, which is absoltely necessary, because otherwise the video card fan produces so much noise that it doesnt really matter any more if it works or not.

The abrt stuff about "tainted flags" is irrelevant, it would always do that if you're running the nvidia closed driver and had some kind of problem that triggers abrt processing.

I would try running memtest86 or examine /var/log/messages or /var/log/boot.log.
A quick first try memory test would be to activate the basic memory test in the bios (sometimes you'd get that by disabling "fast boot")

also try running smartctl -a <device>
where <device> would be the hard drive you use (for example /dev/sda )
Look for lots of unrecoverable errors or the disk generally flagged as not PASSING

[JackC]

Ok, thanks for your answer. I did as you suggested, to no avail. Memtest86+ passed on all tests. Smartctl reported no errors. On a sidenote, I had to turn smartctl on for my harddrive first - is this anything unusual that I should worry about it?
Now, the strangest part: it works again. After accessing the system via the "recovery mode" in grub, and using
"exit" on the root shell from there, I got back to full multiuser and network mode - without the kernel stalls
after a short time. I then did "yum update" which installed a new kernel update and now everything works
again, even the old "Linux 3.2.6-3.fc16.x86_64" entry, where I experienced the problem. The question is just what might have caused the problem how I can prevent it from happening again.

I can't find anything in /var/log/messages and boot.log has no erros either.

dmesg | grep failed gives me the following:

[ 0.295579] pci0000:00: ACPI _OSC request failed (AE_NOT_FOUND), returned control mask: 0x1d
[ 17.839627] microcode: failed to load file amd-ucode/microcode_amd_fam15h.bin
[ 17.842972] microcode: failed to load file amd-ucode/microcode_amd_fam15h.bin
[ 17.846382] microcode: failed to load file amd-ucode/microcode_amd_fam15h.bin
[ 17.849773] microcode: failed to load file amd-ucode/microcode_amd_fam15h.bin
[ 19.390093] udev-configure-printer[757]: failed to connect to CUPS server; giving up
[ 20.395831] udev-configure-printer[760]: failed to connect to CUPS server; giving up

I guess the first one can be ignored. What about that mircrocode error? Also what's that CUPS message, priinting works fine.

[JackC]

Ok, it happened again. My guess now, that it's somehow related to smartctl.
After selecting the recovery mode entry in grub, I noticed that smart was turned
off yet again. I ran "smartctl -s on /dev/sda" and the kernel stalls were gone and I could
boot up my system as usual. This may be coincidence, but none of the suggested hardware
tests provided any errors. What do you think?