You say "It's up to you" when someone asks you to make a decision but you want them to decide:

A: Which one should we get?

B: It's up to you.

When a decision is "up to" someone, that means that they are the one who can decide. You can talk about who has the power to make a decision:

I'd like to help you, but it's not up to me. You'll have to ask Karen.

"It's up to you" is a pretty polite way to let someone else make a decision. A more direct and less polite way is:

I don't care.

And a more formal and polite way is:

Whatever you prefer.