This is one way to end a conversation:

I'll let you get some rest.

I'll let you go.

This phrase makes it seem like you don't want to end the converstion, but you're being considerate of what the other person wants. For example, if you're at a party, you can end a conversation with someone by saying:

Listen, I'll let you mingle. It was great talking to you!

This means "I know that you want to mingle (walk around and talk to a variety of people), so I won't force you to continue this conversation with me."