This is a phrase that people say when talking about a great book. It means that, while you were reading it, you didn't want to stop reading. Or you may not have even been able to make yourself stop reading.
People usually use this phrase to describe books with really addictive stories, like mysteries, really interesting biographies or memoirs, or romance novels. You don't usually say "I couldn't put it down" about a textbook or a work of serious literature.
Another way to describe a book like this is to call it "a page turner". This means that the book makes you want to turn to the next page quickly to find out what happens next.