Responsibility and circumstances

To what extent should individuals be held responsible for what they do, and how much allowance should be made for mitigating circumstances such as poverty and troubled backgrounds?