"Temp tables impact database performance?"
Yes. By dropping all your temporary tables before you query them, the query ends very quickly. The only drawback is that is ends with an error saying the table does not exist.
Seriously, temporary table do a job. You use them when they are appropriate, and don't use them when they are not appropriate. What performance problems are you having, and why do you think temporary tables are causing them ?