This means to stop bothering someone. With friends and family, it may mean to stop asking questions, or to just give them some quiet time. You may ask a stranger to leave you alone if they are making you uncomfortable.
This guy at the bar would not leave me alone.
Sometimes I just leave him alone ‘til he’s in a better mood.
We can also say “leave (someone) alone about (something).”
My mother won’t leave me alone about the Christmas party!
This probably means the person’s mother is asking a lot of questions.