All perfect praise be to Allah, The Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallahu ‘alayhi wa sallam, is His slave and Messenger.
Harming one's neighbors, regardless of whether they are Muslim or not, is a serious matter, as you mentioned. Some scholars considered it a major sin. Ibn Hajar Al-Haytami wrote, "Major sin no. 210: Harming the neighbor even if he is a Thimmi (non-Muslim living in the Muslim state), such as spying on the women of his household or building what harms him (by blocking his view or ventilation for instance) without a valid reason." [Az-Zawaajir] Then he mentioned some texts that urge good treatment of the neighbor and forbid harming him.
We believe that your situation with your neighbor is a simple matter. If you recite the Quran at a normal volume (without raising your voice), this is not considered harming him, especially given that he has not expressed any objection nor has complained about it. It is easy to lower your voice during the recitation in order to attain both benefits, reciting the Quran and avoiding harming others.
Allah knows best.