Vfrdtyky

I use celery to process some long running tasks and locally everything works fine.

Now I uploaded everything on heroku and it doesn't work. I see no errors or warnings. I see that the tasks are initiated and I also see some activity in my redis add-on (Heroku Redis :: Redis) but thats it, the tasks never finish, it looks like an endless loop.

Here is my procfile:

web: gunicorn stats_main:app

worker: celery -A stats_main.celery --loglevel=info

I added the heroku redis add-on and started it with:

heroku ps:scale worker=1

It was successful and under ressources in heroku I see now 2 dyno types, a web and a worker dyno.

Here is my configuration for the redis server:

if 'DYNO' in os.environ:

    app.config['CELERY_BROKER_URL'] = 'redis://[the full URL from the redis add-on]'

    app.config['CELERY_RESULT_BACKEND'] = 'redis://[the full URL from the redis add-on]'

else:

    app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'

    app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost'

Here is an example of a task:

@celery.task(bind=True)

@basic_auth.required

@check_verified

def get_users_deregistrations_task(self, g_start_date, g_end_date):



    start_date = datetime.strptime(g_start_date, '%d-%m-%Y')

    end_date = datetime.strptime(g_end_date, '%d-%m-%Y')



    # A long task



    return {'some_value' : 'some_value'}



# Selects user deregistrations between selected dates

@app.route('/get-users-deregistration', methods=["POST"])

@basic_auth.required

@check_verified

def get_users_deregistrations():

    if request.method == "POST":



        # init task

        task = get_users_deregistrations_task.apply_async([session['g_start_date'], session['g_end_date']])

        return json.dumps({}), 202, {'Location': url_for('taskstatus_get_users_deregistrations', task_id=task.id)}



@app.route('/status/<task_id>')

@basic_auth.required

@check_verified

def taskstatus_get_users_deregistrations(task_id):

    task = get_users_deregistrations_task.AsyncResult(task_id)

    if task.state == 'PENDING':

        response = {

            'state': task.state,

            'current': 0,

            'total': 1,

            'status': 'Pending...'

        }

    elif task.state != 'FAILURE':

        response = {

            'state': task.state,

            'current': task.info['current'],

            'total': task.info['total'],

            'status': 'Finished',

            'statistic': task.info['statistic'],

            'final_dataset': task.info     

        }

        if 'result' in task.info:

            response['result'] = task.info['result']

    else:

        print ('in else')

        # something went wrong in the background job

        response = {

            'state': task.state,

            'current': 1,

            'total': 1,

            'status': str(task.info),  # this is the exception raised

        }

    return json.dumps(response)