All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger. We ask Allaah to exalt his mention as well as that of his family and all his companions.
If this man does not perform prayers at all, then some scholars are of the view that a person who completely abandons the prayers is a Kaafir (a non believer). The Prophet said: "Between a person and polytheism (or disbelief) is the abandonment of prayers." [Muslim] The Prophet further said: "The pact between us and them is prayer, so whoever abandons it is a disbeliever." [At-Tirmithi and An-Nasaa'ee]
So if this man is such [that he abandons the prayer and is a disbeliever] then it is not permissible to give Zakat money to him and whoever gives the Zakat to him did not fulfil his obligation and still has to pay the Zakat.
Shaykh Al-Islam Ibn Taymiyyah said: 'It is not permissible to give anything to a needy person who does not perform the prayer until he repents and performs the prayer regularly.'
However, if this man performs the prayer but sometimes he is lazy and does not perform it, and he does not completely abandon it, then there is no harm to give Zakat to him, especially if this is done with the intention of calling him to abide by the Commandments of Allaah and influence him to do so.
Finally, we advise you to give Zakat to the poor righteous and pious people and other people who are eligible for Zakat, and they are so many these days. So it is more proper to give Zakat to these people rather than giving it to those who do not give any consideration to the religion of Allaah. For more benefit about the appropriate recipients of Zakat, please refer to Fataawa: 88368 and 86546.
Allaah knows best.