When you're talking to someone who's part of a group of people, you can talk about the group like this:
Where did the two of you meet?
I've known the three of them since they were little boys.
You can also talk about a group that you are part of with the phrase "the ___ of us":
The three of us went out that night to an art gallery.