A "well-to-do family" is a family that has a lot of money. They might not be extremely rich, but they're pretty rich.

This phrase sounds pretty positive. It's not rude to call someone "well-to-do".