AWS Elastic Beanstalk – La secuencia de comandos se agotó antes de devolver los encabezados: application.py

Tengo una aplicación de matraz Elastic Beanstalk en AWS que ocasionalmente no se inicializa y da el siguiente error:

[Mon Jan 23 10:06:51.550205 2017] [core:error] [pid 7331] [client 127.0.0.1:43790] script timed out before returning headers: application.py [Mon Jan 23 10:10:43.910014 2017] [core:error] [pid 7329] [client 127.0.0.1:43782] End of script output before headers: application.py 

¿Alguna idea de por qué esto podría ser? Más recientemente cambié los requirements.txt del proyecto.txt para incluir pandas==0.19.2 . Antes de ese cambio, el progtwig funcionaría durante varios días antes de devolver el mismo error. Más registros / detalles del progtwig:

 [Mon Jan 23 10:05:36.877664 2017] [suexec:notice] [pid 7323] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Mon Jan 23 10:05:36.886151 2017] [so:warn] [pid 7323] AH01574: module wsgi_module is already loaded, skipping AH00557: httpd: apr_sockaddr_info_get() failed for ip-10-55-254-33 AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message [Mon Jan 23 10:05:36.887302 2017] [auth_digest:notice] [pid 7323] AH01757: generating secret for digest authentication ... [Mon Jan 23 10:05:36.887797 2017] [lbmethod_heartbeat:notice] [pid 7323] AH02282: No slotmem from mod_heartmonitor [Mon Jan 23 10:05:36.887828 2017] [:warn] [pid 7323] mod_wsgi: Compiled for Python/2.7.10. [Mon Jan 23 10:05:36.887832 2017] [:warn] [pid 7323] mod_wsgi: Runtime using Python/2.7.12. [Mon Jan 23 10:05:36.889729 2017] [mpm_prefork:notice] [pid 7323] AH00163: Apache/2.4.23 (Amazon) mod_wsgi/3.5 Python/2.7.12 configured -- resuming normal operations [Mon Jan 23 10:05:36.889744 2017] [core:notice] [pid 7323] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' [Mon Jan 23 10:06:43.542607 2017] [core:error] [pid 7328] [client 127.0.0.1:43786] Script timed out before returning headers: application.py [Mon Jan 23 10:06:47.548360 2017] [core:error] [pid 7330] [client 127.0.0.1:43788] Script timed out before returning headers: application.py [Mon Jan 23 10:06:51.550205 2017] [core:error] [pid 7331] [client 127.0.0.1:43790] Script timed out before returning headers: application.py [Mon Jan 23 10:10:43.910014 2017] [core:error] [pid 7329] [client 127.0.0.1:43782] End of script output before headers: application.py 

aplicacion.py

 import flask from flask import request, Response import logging import json import JobType1 import JobType2 import sys application = flask.Flask(__name__) application.config.from_object('default_config') application.debug = application.config['FLASK_DEBUG'] in ['true', 'True']` logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) @application.route('/', methods=['GET']) def index(): logger.info("The message received was '/', no action taken") response = Response("Success", status=200) return response @application.route('/StartJob', methods=['POST']) def StartJob(): logger.info("!!start_job message received! This is the start job logger") print("!!start_job message received! This is the start job printer") response = None if request.json is None: response = Response("Error, no job specified.", status=400) else: message = dict() try: if request.json.has_key('TopicArn') and request.json.has_key('Message'): message = json.loads(request.json['Message']) job = message['job'] else: message = request.json job = message['job'] date_key = None try: date_key = message['runkey'] except Exception as e: print "printing Exception:" print e pass start_job(job, date_key) response = Response("Called Job", status=200) except Exception as ex: logging.exception("There was an error processing this message: %s" % request.json) response = Response("Error processing message", status=400) return response @application.route('/CronMessage', methods=['POST']) def cron_message(): logger.info("!!Cron message received! This is the Cron Start Logger") response = None logger.info("About to print headers of CRON***") job = request.headers.get('X-Aws-Sqsd-Taskname') start_job(job, None) response = Response("Called Job", status=200) return response def start_job(job_name, date_key): logger.info("JOB NAME SUBMITTED IS:") logger.info(job_name) if job_name == 'JobType1': start_job_if_not_running(job_name, JobType1.main, True, date_key) if job_name == 'JobType2': start_job_if_not_running(job_name, JobType2.main, True, date_key) else: print "Submitted job nome is invalid, no job started. The invalid submitted job name was %s" % job_name def start_job_if_not_running(job_name, program_to_execute, uses_date_key_flag, date_key): global running_jobs logger.info("running_jobs are:") logger.info(running_jobs) if job_name in running_jobs: logger.info("Currently running job " + job_name + ", will not start again.") return False else: try: running_jobs.append(job_name) if uses_date_key_flag: logger.info("") program_to_execute(date_key) else: program_to_execute() except Exception as e: handle_job_end(job_name) print "Error in " + job_name error_message = str(e) + "-" + str(sys.exc_info()[0]) print error_message EmailUsers.main(subject="Error in " + job_name, message=error_message, message_type='error', date_key=date_key, job_name=job_name, process_name='application.py', notification_group='bp_only') handle_job_end(job_name) def handle_job_end(job_name): while job_name in running_jobs: running_jobs.remove(job_name) logger.info("Process Complete") if __name__ == '__main__': application.run(host='0.0.0.0', threaded=True) 

Cualquier ayuda es apreciada, puedo compartir más código de los otros archivos según sea necesario.

Además, si /etc/httpd/conf.d/wsgi.conf a /etc/httpd/conf.d/wsgi.conf , veo:

 LoadModule wsgi_module modules/mod_wsgi.so WSGIPythonHome /opt/python/run/baselinenv WSGISocketPrefix run/wsgi WSGIRestrictEmbedded On  Alias /static/ /opt/python/current/app/static/  Order allow,deny Allow from all  WSGIScriptAlias / /opt/python/current/app/application.py  Require all granted  WSGIDaemonProcess wsgi processes=1 threads=15 display-name=%{GROUP} \ python-path=/opt/python/current/app:/opt/python/run/venv/lib64/python2.7/site-packages:/opt/python/run/venv/lib/python2.7/site-packages user=wsgi group=wsgi \ home=/opt/python/current/app WSGIProcessGroup wsgi  LogFormat "%h (%{X-Forwarded-For}i) %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined 

La respuesta de @ user2752159 resalta el problema, sin embargo, voy a agregar esto para mostrar cómo superar este problema en el contexto de AWS Beanstalk (es decir, si una nueva instancia o usted implementa más código, entonces el problema permanecerá resuelto, en lugar de tener que hacerlo). ssh en el cuadro cada vez para modificar wsgi.conf ).

 nano .ebextensions/.config #add the following to .config files: "/etc/httpd/conf.d/wsgi_custom.conf": mode: "000644" owner: root group: root content: | WSGIApplicationGroup %{GLOBAL} #add to git git add .ebextensions/.config git commit -m 'message here' #deploy to beanstalk eb deploy 

Ahora, cada vez que implementes, WSGIApplicationGroup %{GLOBAL} se agregará a wsgi_custom.conf , solucionando el problema.

Muchas gracias a @GrahamDumpleton por su ayuda. La solución que utilicé fue:

-Editar el archivo wsgi.conf que se encuentra en /etc/httpd/conf.d/wsgi.conf en la instancia de Elastic Beanstalk EC2.

Para hacer esto, utilicé el comando sudo -e /etc/httpd/conf.d/wsgi.conf para abrir el editor, WSGIApplicationGroup %{GLOBAL} INSERT para comenzar la edición y agregué WSGIApplicationGroup %{GLOBAL} en cualquier parte del archivo. Entonces su ESCAPE y usé el comando :wq para guardar los cambios.

Después de esto, seleccioné Reiniciar servidores de aplicaciones del menú desplegable Acción de la consola de Elastic Beanstalk. Después de esto, el progtwig cargaría y daría el AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' , pero no los mensajes de error posteriores. Además, la aplicación recibiría mensajes SQS y se ejecutaría como se esperaba.

Una cosa a tener en cuenta es que parece que el archivo wsgi.conf se revertirá si se realizan cambios en la configuración de Elastic Beanstalk. No estoy seguro de una forma de evitar esto, pero si encuentro algo, lo publicaré aquí.

Gracias nuevamente a @GrahamDumpleton por su pronta respuesta y ayuda para resolver este problema.