The phrase "Let's ___" is well-known to English learners. However, learners often over-use "Let's ___" and use it in situations where it isn't appropriate.
When can you say "Let's ___"? You usually use it when you're suggesting something that you're sure the listener will accept. If you're meeting a friend for lunch, then you're sure that they will agree to eat with you. So you can say:
In situations where you're not sure how the listener will respond to your suggestion, you can use other phrases like "Would you like to ___?", "Why don't we ___?" or "We should ___":
Would you like to go out to dinner with me some time?
Why don't we meet at 7:30?
We should get together some time and have a drink.