You say "It's up to you" when someone asks you to make a decision but you want them to decide:
A: Which one should we get?
B: It's up to you.
When a decision is "up to" someone, that means that they are the one who can decide. You can talk about who has the power to make a decision:
I'd like to help you, but it's not up to me. You'll have to ask Karen.
"It's up to you" is a pretty polite way to let someone else make a decision. A more direct and less polite way is:
I don't care.
And a more formal and polite way is:
Whatever you prefer.