Use this phrase when a pet, or a very young child, does something wrong. It's a simple, direct way of scolding them for misbehaving.
It's strange to use this with any people who are over about 3 years old. Once children are able to talk, you should use longer sentences to scold them.
If you do say "Bad boy!" or "Bad girl!" to an adult, it's usually within a joke, where you're pretending that the person is a baby or a pet.