If you installed from the repo using yum or PackageKit (which is the recommended way), just use the same method to uninstall, e.g.:
yum erase kmod-nvidia-173xx
or Add/Remove Software from the Administration menu and uncheck the package.
If you installed from another source such as a .run package downloaded from NVidia, use the --uninstall option on the command line for the package.