But is it not contrary to the justice of God to punish someone for others?

If we consider the condition of the human race, the question is answered. For by nature we are all cursed, and we cannot complain of God when He leaves us in this condition. Moreover as He manifests His grace and love toward His servants in blessing their children, so this is a testimony to His punishment of the wicked, when He leaves their seed accursed.