You start a sentence with "I must admit..." when you're telling people something that you feel just a little bit guilty or embarrassed about:
I must admit, I'm not very good at this sort of thing.
You can also say "I must admit..." when someone else impresses you more than you expected:
I must admit, you're quite a singer.
You don't use this for things that you feel really guilty about, like making a big mistake at work.
You can also use the phrase "I have to admit" in a similar way.