This is a set expression that people use when asking to use a chair in a public place that's close to where another person is sitting.
You will often hear "Is this seat taken?" in movies and on TV shows when a person sits with someone who they don't know well. Usually in those scenes, the characters have a conversation with each other. So if you ask "Is this seat taken?" people might think that you're going to talk to them. If you want to make sure that they don't think this, you can ask for the chair in a less common way like:
Is anyone using this?
Do you mind if I sit here?