Why shouldn't I murder all the things?
Mainly because it will increase Chaos levels.
The Dishonored wiki, on the Chaos system:
Changes in the Chaos system may result in subtle in-game consequences, such as more rats appearing in an area or slight differences in dialogue, to more noticable consequences, such as more guards appearing on patrols or characters refusing to interact with Corvo or rewarding him, depending on what they think of his actions.
If high enough Chaos is created, bedlam ensues: NPCs begin to attack each other, rats swarm the area, and the world through which Corvo moves becomes an altogether nightmarish place.
The developers emphasise that the Chaos system is not a morality metre, but a stability metre. It is meant to highlight the plight of Dunwall: its people are no longer concerned with right and wrong, but merely with holding on to the little certainty that still exists.
I noticed that on a high chaos playthrough, NPCs like some Survivors will alert hostile, armed NPCs, if they detect your presence. While on a low chaos playthrough, some Survivors will be friendly to you or won't mind your presence (no detection meter), and won't alert anyone.
Also, there is actually a significant difference between high chaos and very high chaos (not shown in the ending):
I just finished a 3rd playthrough where I went lethal+stealthy. As a
result, I had a moderate number of kills - the first few missions said
I had Low chaos, but it had changed to High by the time I got to the
last mission, since I went on a bit of a rampage with the assassins.
On "very high" chaos, Samuel is a total dick to you and warns guards
by firing into the air. On my playthrough, he simply said he was
disappointed by how "brutal" I had become, and drove the boat away in
Source: Reddit /r/dishonored: "There IS a difference between "high" chaos and "very high" chaos, just not in the ending"
Also, see this answer on another question about how the Chaos system works.