“By the way, how much did you leave him?”
Your friend took you out for dinner, but the service was terrible. You're talking about it on the way home, and you ask him how much he tipped the waiter.
By the way, how much did you leave him?
The most common verb to use with "tip" when you're at a restaurant is "leave", not "pay":
Did you leave her a tip?
How much did you leave?
You use "leave" because the normal way of giving a waiter a tip is to leave the money on the table when you leave the restaurant. In other situations, like when you are tipping a taxi driver, you "give" instead:
I gave him a five dollar tip.
"By the way..." is a phrase that you can use when you want to change the topic of conversation. You use it when you want to talk about something that's unrelated to what you were just saying.
For example, imagine that you were talking with a coworker about your vacation plans, but you suddenly remember something about her husband:
We're going to spend a day in Paris and then, I think, a day or two in Rome. Oh, by the way, how did Randy's interview go?