Say this when your child, your employee, or someone else who you have authority over makes a stupid or irresponsible decision.
An example of another situation in which you can use "What were you thinking?" is when your husband does the laundry and uses bleach, making all of the clothes white and unwearable. In this situation, you can ask "What were you thinking?"
One reason that people ask this when they're angry is that, in Western culture, when you make a mistake people expect you to explain why you did it. Only saying "sorry" for your mistakes isn't usually enough - you're also expected to explain what you were thinking.