All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad, sallallaahu 'alayhi wa sallam, is His slave and Messenger.
Inner thoughts about the faults and shortcomings of a certain person does not fall into the category of forbidden backbiting. Allah, The Exalted, has pardoned for the Muslim nation their unspoken inner thoughts as long as they do not act upon them or verbalize them. However, one should not excessively think about people’s conditions and flaws. It is better for a person to concern himself with his own flaws, strive to rectify them, and think about what benefits him in this worldly life and the Hereafter instead.
Allah knows best.