I looked around for a plugin that would automatically set comments to closed after a set number of days. I didn't find any, but I did find some code at the WordPress wiki that was useful, so I turned it into my own plugin.
It checks for posts to be closed every time the database is updated, so it doesn't require a cronjob.