Already, the rumors are severely hurting his approval ratings and providing political munition to his political rivals.