Is it time to call it quits?

You need to run, not walk away from this guy. He will never be the one you need in your future.He has problems deeper then you need to deal with. Let someone else adopt him to raise.