All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
The money which you mentioned in the question belongs to you and you are the one who is responsible for paying its Zakaah and for distributing it among its legitimate recipients. As regards the person who holds it as a loan or a deposit, then he is not responsible for paying its Zakaah unless you entrusted him to pay its Zakaah on your behalf.
If a trustee who is entrusted with the money is absent and disposed of the money as he wished, then this money is considered as a debt and the ruling of Zakaah on debts apply to it.
If the indebted person is wealthy and able to repay his loan then the owner is responsible to pay the Zakaah of that loan every year.
However, if the indebted person is poor or he is a procrastinator, then the owner should pay the Zakaah of that money for only one year after the loan is repaid.
Allah knows best.