Summary: The psalmist praises God as Protector of Jerusalem, and therefore his home and people. God inhabits every nook and cranny of the city and pushes back any attempt at capture.
Response: So the problem of evil (how did I get there? Gimme a sec): why does God allow evil things to happen when He's all-powerful? I believe in free will, that we can choose to go with God's will or on our own. Through the biblical kingdom of Israel's history, its people would turn away from God over and over. "God, we don't need your help. We have this on our own." And so God backs off. He could have prevented the siege of Babylon or Roman rule, but Israel asked Him not to interfere, so he respected their wishes.
Lord, I do this too often myself. Keep me from this sin (I tried folly here, but it's too nice a word) and keep me company reliant on You.