"Defense" is a a category of government which includes the military. Saying that you agree with the Prime Minister on defense means that you agree with his plans for wars, the way he spends money on the military, and so on.