All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger. We ask Allaah to exalt his mention as well as that of his family and all his companions.
Dear sister, may Allaah bless you and grant you a happy life.
If the reality is as you mentioned in your question, your husband is mistaken and is committing a great sin by preventing you from contact with your son. You have all the rights of a mother on your child even if you are divorced.
A mother has more right than the father to keep her small children unless she marries another person. The father has to bear the expenditure of his children even if they are with their mother. We have said about the matter of custody of children after the divorce in a previous Fatwa: 83052. [You are more entitled to take custody and care of your child as long as you do not get married, for the Hadeeth narrated in al-Musnad, Sunan Abu Daawood and al-Mustadrak that a woman complained to the Prophet that her husband who divorced her wanted to take their son with him. The Prophet said: "You have more rights over him (the son) than the son's father as long as you have not married." This rule applies to every divorced woman who is capable and suitable to fulfill the religious duties and other needs of a child. She deserves to look after her child till the age of discernment, i.e. the stage when a child can distinguish between good and bad, and can take care of himself. When a child reaches this stage, if his parents dispute over taking care of the him, and both are equal in ability to take care of the child as far as religious matters and other matters are concerned, then one of them becomes liable to care for him following the results of casting lots between them. This rule is from the Hadeeth reporting that a woman came to the Prophet and said: "My husband wants to take away my son now, while he brings me water from the Inabah well (a well in Madeenah) and he is useful for me. The Prophet said: "Cast lot between both of you. Then the Prophet said to the child: "This is your father and this is your mother, choose one of them. He chose his mother and she took him." [Abu Daawood]].
For other details read Fatwa: 82128.
Allaah knows best.