"Start something" is a slang phrase for starting an argument or a fight:
He came up to me and tried to start something. I was like, "Whatever!". I just ignored him.
If someone is acting in a way that seems like it might cause an argument, then you can say that that person is "trying to start something". For example, family members might have a conversation like this:
A: What are you doing? You're doing that all wrong.
B: Are you trying to start something?
A: No, I'm just saying, that's not the right way to do it.
B: I don't want to hear it!