Praise be to Allah, the Lord of the Worlds; and may His blessings and peace be upon our Prophet Muhammad and upon all his Family and Companions.
It is not permissible for your wife to let this man enter her home, as he is alien to her. You have to forbid her to do so and remind her about Allah and tell her that this is forbidden in Shariah.
The coming of this man to visit his child is not an excuse for that. It is not permissible for him to see the child if he transgresses the limits of Shariah. For instance if the child is old enough, he can get out of the house to see his father. Or that can be done in your presence or one of your wife's Mahrams, if the child is still too young.
As regards teaching him the Islamic rulings and bringing him up according to the teachings of Islam, then this can be done by the mother as the child is with her most of the time.
However, if you do your best and he turns out to be non-Muslim, then you are not sinful. But you have to help your wife give him a good education, may Allah enable you to do so, and you will get a great reward from Allah for doing so.
Allah knows best.