If something goes wrong with a pip installation, we should know about it.