When celery runs in eager mode, there is a bug and an exception
is not thrown when using retry logic. A PR to fix the bug can
be found here: https://github.com/celery/celery/pull/2607

In the mean time, we have to deal with the fact that in eager mode
celery will not throw the exception but simply return it. So we
raise it ourselves.