rpmdb broken yum panic messed up system

geomar · #1 24th September 2007, 10:02 PM

Dear All,

First of all let me tell you where I stand now.
Right now I have a system that runs, but it is un-updateable(sic!). Before trying a reinstall, i reach out for your help. But how, did I end up here? This evening i did a regular

Code:

yum update

Which crashed yum with a 'segmentation fault'.

Then I did a

Code:

yum cleanall

But neither did that help a thing. After these two steps I turned to google, which came up with a seemingly plausible solution.

Code:

rm -f /var/lib/rpm/__db*

Here, when I tried to install a package it did complain about missing dependencies first, but after another yum cleanall and rpm -rebuilddb my computer produced a new way of speeding up my aging process.

Code:

rpmdb: PANIC: fatal region error detected; run recovery error:

Then I read on google that erasing /etc/yum.repos.d 's contents can solve this. Do I need to tell you it didnt? ;-)

So as I see now; I lost my repos, which is okay I can write or download them back.
But the major problem is, that I lost all data to my installed rpm packages as well. The system does not know it has anything installed.

For instance, when doing a repo install i get the following:

Code:

[root@localhost log]# rpm -Uhv http://rpm.livna.org/livna-release-7.rpm
Retrieving http://rpm.livna.org/livna-release-7.rpm
warning: /var/tmp/rpm-xfer.TWSFvS: Header V3 DSA signature: NOKEY, key ID a109b1ec
error: Failed dependencies:
        /bin/bash is needed by livna-release-7-2.noarch
        fedora-release >= 7 is needed by livna-release-7-2.noarch

This was ...well .... sort of funny, because I typed in everything in the missing shell. But I understand that yum having lost its database didn't realize this. Anyway I installed it with --nodeps.

An update test produces:

Code:

yum update
http://rpm.livna.org/fedora/%24releasever/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Content-Length: 345
Date: Mon, 24 Sep 2007 19:59:22 GMT
Accept-Ranges: bytes
Content-Type: text/html
Server: lighttpd/1.3.16

Trying other mirror.
http://livna.cat.pdx.edu/fedora/%24releasever/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Server: nginx/0.4.13
Date: Mon, 24 Sep 2007 20:00:26 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive

Trying other mirror.
http://wftp.tu-chemnitz.de/pub/linux/livna/fedora/%24releasever/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Content-Length: 345
Date: Mon, 24 Sep 2007 19:59:23 GMT
Accept-Ranges: bytes
Content-Type: text/html
Server: lighttpd/1.3.16

Trying other mirror.
http://ftp-stud.fht-esslingen.de/pub/Mirrors/rpm.livna.org/fedora/%24releasever/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Content-Length: 345
Date: Mon, 24 Sep 2007 19:59:23 GMT
Accept-Ranges: bytes
Content-Type: text/html
Server: lighttpd/1.3.16

Trying other mirror.
http://mirror.atrpms.net/livna/fedora/%24releasever/i386/repodata/repomd.xml: [Errno 14] HTTP Error 404: Date: Mon, 24 Sep 2007 20:00:27 GMT
Server: Apache/2.2.3 (Red Hat)
Content-Length: 330
Content-Type: text/html; charset=iso-8859-1

Trying other mirror.
ftp://mirrors.tummy.com/pub/rpm.livna.org/fedora/%24releasever/i386/repodata/repomd.xml: [Errno 4] IOError: [Errno ftp error] 550 Failed to change directory.
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: livna. Please verify its path and try again

Do I have hope, or shall I use my livecd again? I hope someone is able to give me guidance on how to resolve this.

tho.mei · #2 24th September 2007, 10:56 PM

Hallo geomar

help, help.... it look realy not good!

First of all I need to tell you a bit about package managmant in Fedora and RedHat:

rpm = (RedHat Package Mangaer)
rpm is a tool for installaing, removing and creating rpm-Packages. On the other hand rpm does look for dependency between packages. Therefor it builds a data base. rpm needs allwasy to know exactly what's installed and way. This data base is /var/lib/rpm/__db* (do you remeber?)

yum (= yellowdog upgate manager)
yum is a download application for rpm-repositorys. It is able to sinc repositorys with rpm-querys. So it can update the system. All the installation and remove of package are only passed to the rpm (-app). Yum does only download and sinc the packages.

OK, now we will face your problem.

you deleted the rpm database. The rpm db can crash. Therfor does rpm rwite a jurnal. Out of the journal, rpm should be able to rebuilt it's db.

To rebuild the db, first delet it again. (The rebilding will take a lot of time!)

1. reboot the system
2. rm -f /var/lib/rpm/__db*
3. rpm -vv --rebuilddb

I'm not sure if it works, because you tryed to install apps after the crash.
If the rebulid won't works, is your sytem fine, but rpm and yum are useles. It means you will not be able install, remove or update any app. In this case I would install fedora again. It's the fastes and most easy way.

Sorry for the band news.

ppesci · #3 25th September 2007, 12:24 AM

May be in the install Fedora 7 process, you can select upgrade and not a clean install. This will maintain your data files intact and create the database repository using the cd/dvd packages.
Later you can run:

yum update

You will have a problem overwriting the package files but will have consistent database.

HTH

geomar · #4 25th September 2007, 07:15 AM

Sound advice Pietro, this is what I had in mind as well. Reinstall with the [x] preserve data option and see what happens.

tho.mei:
rpm -vv -rebuilddb produces and empty db, although i have found the list of installed packages in a file (cant remember the name and have no chance to look it up, as i am work now. it was something like rpmpkgs or similar)

ppesci · #5 26th September 2007, 01:08 AM

If you try the reinstall preserving data, put the results here. This will be helpful for others because the database package corruption may be more frequent and generate heavy headaches.

geomar · #6 28th September 2007, 09:53 AM

I shall do it tonight and report on the result.

My assumption is, that it will generate a db with the first yum update execution.

More tomorrow.

Coarch · #7 27th November 2007, 05:41 PM

I had a similar issue.

The RPM task in /etc/cron.daily wasn't finishing. This left a processes in a hung state that was visible when I ran 'ps ax | grep rpm'. There was a long list of processes. Yum would not respond and rpm hung at the prompt whenever I tried to install something.

To resolve the issue I deleted the db files from the '/var/lib/rpm' directory.

rm -rf __db*

I then rebuilt the database and updated yum
rpm -vv --rebuilddb
yum clean all
yum update

This has fixed both my yum and rpm issues. Thanks for the info and I hope this helps someone.