As Bob says, and for those reasons, Fedora isn't designed to be a server. (Though it can successfully be used as one.)
As for RHEL (or CentOS, SL, and Oracle), they install without GUI by default. (Using the "Minimal" package group.) Choosing a few others (I haven't tested them all, only web server) will also not install a GUI.
And here, ironically, Fedora offers a better choice of GUIs. If you pick one with RHEL, I think the only choices are Gnome, KDE, and Matchbox. (and probably twm). Gnome and KDE are full blown desktop environments, carrying all the things you mention you don't want. Matchbox might fit the bill. Fedora, on the other hand, has a bunch of less full featured window managers.
Personally, if for whatever reason, I need a GUI, I'll use one of the smaller ones, such as dwm, which needs almost nothing as far as dependencies. I have a little page on it at
(However, as far as I know, it's not in the CentOS repos. There are extra repos individuals or groups do for RHEL, that aren't officially connected, and you can get fluxbox through one of those and openbox through another.)
There is also ClearOS, RHEL based, that has some GUI configuration tools.
(Link is for the screenshots page.)