All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
First, you should know that it is unlawful according to Islam for a man to befriend a woman who is not a Mahram to him or stay alone with her, as doing so might lead to falling into vice and forbidden acts. The Prophet warned against looking at women who are not Mahram or staying alone with them. He said: “No man ever stays alone with a woman but Satan becomes their third (companion).” [At-Tirmithi]
The Prophet also said: “Beware of entering a place where there are women.” [Al-Bukhari and Muslim]
Indeed, the fact that you and this man committed Zina (fornication or adultery) is but an example of this danger. In fact, it is a result of your failure to obey the Sharee’ah rulings.
Now, the appropriate solution for you is to get married. But this marriage has to fulfill all the conditions of a valid marriage in Islam; viz., there should be a Wali (guardian for the woman) and two trustworthy witnesses. But you are not required to make your marriage public. If the Wali is present that is good; otherwise he may appoint someone else to act on his behalf. The Prophet said: “Any woman who gets married without the consent of her Wali, her marriage is null and void.”
Know also that you cannot marry this man before you both repent to Allah from what you have committed and commit yourself never to fall in this sin again. On the other hand, if you can wait until you make your marriage public, then that would be better.
Allah knows best.