Fedora Linux Support Community & Resources Center
  #1  
Old 27th November 2010, 01:39 PM
Goulou Offline
Registered User
 
Join Date: Nov 2010
Posts: 12
linuxubuntufirefox
F13-14 Hangs for few minutes and clock drifts

Hi All,

I've been running a fedora server for over a year now, and has seen some very strange issue, that really make it uncomfortable to use, and that I cannot manage to solve easily.
The problem is as follow : sometimes (every 20-30 minutes or so : this time is quite random), the server completely hangs.
I'm using it mainly remotely, through ssh and nfs. What I see when it "hangs" is the following
-no ping response
-nfs stalled
-ssh sessions hangs (for example, if I run a "top" command", it just isn't updated anymore
-disk activity led stays completely off (in normal activity, it is almost always blinking, even due to "internal" server activities, so network disconnection doesn't explain that either)

The most strange thing is that, to resume it, I basically have 3 options : wait (sometimes few tens of seconds, usually between 2 and 4 minutes!), just hit a key on the keyboard!, or do something like un/plugging any king of usb peripheral (which makes me think of some interruption mechanism that is stalled).

When it "wakes up", I see my "top" session over ssh suddenly being quickly updated hundreds of times (for all it has not received during the "pause"), ping says that the packets have actually all been received (with long times, for example, packet 1 : 80xxxms, packet 2 : 79xxxms, packet 3 : 78xxxms..... packet 79 : 1xxxms, packet 80 : 0.xxxms), the disk is quite overloaded for a few seconds, and everything is back at normal!!
Furthermore, I think this i related, but my clock drifts for few hours per day (3 holding minutes every 20-30 minutes makes me think there is some relation!)
I tried to set up ntpd to compensate it, but sometimes the suspensions are just too long, and I ended up with
Code:
Nov 27 01:22:36 server ntpd[699]: 0.0.0.0 0617 07 panic_stop +1203 s; set clock manually within 1000 s
and ntpd dies...

You'll ask me to have a look at the log, which I did by
Code:
tail -f /var/log/* /var/log/*/*
, but when the suspension happens, there is absolutely nothing new in all those files!

Version information :
Code:
uname -a
Linux vmserver.grelot.net 2.6.35.6-48.fc14.x86_64 #1 SMP Fri Oct 22 15:36:08 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux
I said in the title "F13-F14", because I already had the problem with F13, but after some time it disappeared (I may have changed some configuration, I have to admit...). I still have a backup of the "/etc" tree of that "working" F13.
The hardware for F13[working] and F14[not_working] is the same : Phenom II X4 on an ASUS M3N78 PRO, data on raid 10, system on separated SATA disk, 8Gb RAM, some qemu vms running (between 4 and 6).

It would be really great if someone had an idea about this problem : it annoyed me for several months now, and I cannot stay calm anymore when I suddenly cannot do anything but wait, since I'm not near enough to press the "Ctrl" key on the keyboard!!

Thanks a lot in advance,

Goulou.

Last edited by Goulou; 27th November 2010 at 01:45 PM.
Reply With Quote
  #2  
Old 27th November 2010, 01:58 PM
jpollard Offline
Registered User
 
Join Date: Aug 2009
Location: Waldorf, Maryland
Posts: 6,898
linuxfedorafirefox
Re: F13-14 Hangs and clock drifts

It sort of sounds like a hardware problem with your clock.

You might try booting the system with nohz=off for testing. In one system
I have (an old Sony laptop) when the system warms up the clock starts
acting strange (not the time clock, but an interrupt clock). In this system
it would hang the system with a burst of interrupts - would lock the system
up (or severely slow it down) for 10-20 seconds. nohz=off seemed to help
(but the hardware problem remains).
Reply With Quote
  #3  
Old 27th November 2010, 06:18 PM
Goulou Offline
Registered User
 
Join Date: Nov 2010
Posts: 12
linuxubuntufirefox
Re: F13-14 Hangs and clock drifts

Unfortunately I don't have any screen plugged onto the server, so I'm afraid I'll have to wait for few days before any attempts at modifying boot parameters.
However, you can be sure I'll tell you the result.
Meanwhile, I'll try to have a look at the bios, see if anything looks wrong.

By the way, are there any drawbacks with putting the "nohz=off" option? Does it disable the tickless capability of the kernel? And in that case, won't that increase cpu overall usage?

Thanks for your help!

Goulou
Reply With Quote
  #4  
Old 27th November 2010, 08:22 PM
jpollard Offline
Registered User
 
Join Date: Aug 2009
Location: Waldorf, Maryland
Posts: 6,898
linuxfedorafirefox
Re: F13-14 Hangs and clock drifts

Yes, it does increase CPU usage a bit - but if it works smoother, then it
will help usage, especially if it turns out to be a hardware problem.
Reply With Quote
  #5  
Old 30th November 2010, 08:00 PM
Goulou Offline
Registered User
 
Join Date: Nov 2010
Posts: 12
linuxubuntufirefox
Re: F13-14 Hangs and clock drifts

I found a display to plug, and did test the nohz=off option... Without success

For information :
Code:
 cat /proc/cmdline 
ro root=UUID=e7cc31ce-8afd-4157-a8f1-bc3ce70d8f24 rd_MD_UUID=bf9db417:dc08c1db:b7c728be:f4ed51e6 rd_LVM_LV=vg_raid10/lv_swap rd_NO_LUKS rd_NO_DM LANG=fr_FR.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=fr-latin9 rhgb nohz=off
I really don't know what else to try... Any idea?
Reply With Quote
  #6  
Old 30th November 2010, 08:14 PM
vallimar Offline
Registered User
 
Join Date: Jul 2008
Posts: 1,067
windows_xp_2003chrome
Re: F13-14 Hangs and clock drifts

Did you try disabling HPET? Add the "hpet=disable" to the kernel line when booting to see if that helps.
Something else to try is "clocksource=pit". Your problem could be apic or acpi related as well so you could try altering some of those settings if problems persist.
Reply With Quote
  #7  
Old 1st December 2010, 01:12 AM
sidebrnz Offline
Registered User
 
Join Date: Oct 2007
Location: Freedonia
Age: 65
Posts: 2,657
linuxfedorafirefox
Re: F13-14 Hangs and clock drifts

You didn't say (or I didn't see) which way your clock's drifting. If it's running slow, it may well be nothing more than your CMOS battery needs replacing. The BIOS is designed to make the clock run slow as a warning that you need to change the battery.
__________________
Registered Linux user #470359 and permanently recovered BOFH.

Any advice in this post is worth exactly what you paid for it.
Reply With Quote
  #8  
Old 1st December 2010, 03:03 AM
marko Offline
Registered User
 
Join Date: Jun 2004
Location: Laurel, MD USA
Posts: 6,121
windows_xp_2003firefox
Re: F13-14 Hangs and clock drifts

Quote:
Originally Posted by sidebrnz View Post
You didn't say (or I didn't see) which way your clock's drifting. If it's running slow, it may well be nothing more than your CMOS battery needs replacing. The BIOS is designed to make the clock run slow as a warning that you need to change the battery.

Checkout the BIOS update listing for that board, I don't know if I've ever seen a board
with so many updates:

http://support.asus.com/download/dow...e=map&f_type=3


Especially look at
revision 0603 -> "2. Resolve system freezing problem when using certain memory. "

and
revision 1003 -> "Improve stability and performance of LAN function. "

You might want to update the bios ?
Reply With Quote
  #9  
Old 11th December 2010, 02:27 PM
Goulou Offline
Registered User
 
Join Date: Nov 2010
Posts: 12
linuxubuntufirefox
Re: F13-14 Hangs and clock drifts

The bios may have been the problem : my cpu model was not "officially" recognised by the hardware (it said "model unknown" at startup and in lshw), while it is now.

I updated the bios to the latest version (I have to admit that they have really improved the process!! : Put it on a usb stick, go to the bios, launch the built-in utility, browse the stick and update!). So far (2 hours) I didn't notice any problem.
Last week was horrible thought : after 1 week, the delay was of 32 hours!!

I also removed the nohz=off option, since it didn't seem to do anything.

I'll update the post if it drifts again, otherwise it means that bios update solved it! (it may be of interest for future users).

Thanks a lot for your help.

Goulou
Reply With Quote
Reply

Tags
clock, hangs, interrupt

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
F8 very slow to boot, hangs; Hangs on reinstall, too! juanzo007 Installation, Upgrades and Live Media 13 19th January 2008 09:17 AM
Shutdown Hangs at clock sync x6vhcb5 Hardware & Laptops 2 21st December 2007 02:40 PM
hardware clock system clock vanderkerkoff Using Fedora 2 8th December 2007 01:50 PM
boot hangs after fstab, shutdown hangs too (FC6 only) Justincase Using Fedora 5 25th September 2006 11:35 PM
System clock slow; Hardware clock fine voxelbox Using Fedora 8 23rd September 2004 04:43 AM


Current GMT-time: 09:15 (Monday, 22-12-2014)

TopSubscribe to XML RSS for all Threads in all ForumsFedoraForumDotOrg Archive
logo

All trademarks, and forum posts in this site are property of their respective owner(s).
FedoraForum.org is privately owned and is not directly sponsored by the Fedora Project or Red Hat, Inc.

Privacy Policy | Term of Use | Posting Guidelines | Archive | Contact Us | Founding Members

Powered by vBulletin® Copyright ©2000 - 2012, vBulletin Solutions, Inc.

FedoraForum is Powered by RedHat
...Chateauneuf-sur-Charentes Photos - WOK Christianshavn Photos on Instagram - Witbank. Photos on Instagram