We don't always want to immediately remove them, as we use these configs to pull them off of other hosts.
