“How long have you been doing that?”
You're chatting with someone at a party who you haven't spoken with very much. He just told you about his job. You ask for more information about it.
How long have you been doing that?
Use "that" to refer to something that a person just said:
A: I'm going to try to start a career as an actor.
B: So does that mean you're moving to L.A.?
Or you can talk about something that you heard on T.V. or radio using "that".
This is a question that you can ask someone about an activity that they still do:
How long have you been practicing Judo?
How long have you been waiting here?
If someone no longer does something, you should ask a different question:
How long did you work at IBM?
By the way, if you want to be extra formal and grammatically correct, you can as "For how long...?" For example, you might hear this in a legal court:
For how long were you and the defendant involved romantically?