The word "scruffy" is funny because it's almost always used to describe either men, or dogs.
A "scruffy" man has messy hair, hasn't shaved, or has wrinkled clothes. But you wouldn't call someone who was really messy-looking, like a homeless man. A "scruffy" guy is just a little bit messy-looking.
A scruffy dog might look like it hasn't taken a bath in a long time. Its hair is long and probably hasn't been cut recently.
It's possible to call a woman "scruffy", but it's very rare.