The "foundation" of a building is the part of the building at the very bottom that's connected to the ground. For a building to be strong, the foundation needs to be strong as well.

People often compare education to a building: in order to learn complicated things, you need to understand the basic ideas really well first. To explain this, people say that you need "a strong foundation in ___" or "a solid foundation in ___".