“I can never tell whether he's joking.”
You have an uncle who has a strange sense of humor. He tells jokes, but his face always stays serious. You get confused by this. Now you're talking about your uncle's personality with your mother. This is how you explain your confusion.
I can never tell whether he's joking.
In this case, "tell" doesn't mean "say". It means "decide" or "figure out". The phrase "I can't tell ___" is used to talk about something you're not sure about:
I can't tell whether it's finished or not. I can't tell whether she's being shy or just doesn't want to talk to me.
You can use "can't tell" with any question word like "how", "what", "who", and so on:
I couldn't tell what they were saying.
For the life of me, I can not tell how he did it.
When someone "is joking", it means that they're saying something that's not true, in order to be funny:
A: You're quitting? Are you serious? B: No, I was just joking.
You probably also know that "jokes" are little funny stories or sayings that you tell people. When you're doing this, it's called "telling" a joke:
He always tries to tell jokes, but he usually messes them up.