The only way I know is to groupremove kde
and re-install those libraries you do need but have been removed by yum. I faced the same problem when I tried to remove xfce from my system and it removed a bunch of libraries.
It was discussed recently in this thread.
Based on this thread, re-installing missing dependencies seems to be the only way out.