Luigi framework crash

Question

luigi 작업을 실행할 때 가끔 프레임 워크 충돌이 발생하여 다음 작업이 모두 실패합니다. 다음은 오류 로그 정보는 :

2017-10-05 22:02:02,564 luigi-interface WARNING Failed pinging scheduler 
2017-10-05 22:02:03,129 requests.packages.urllib3.connectionpool INFO  Starting new HTTP connection (126): localhost 
2017-10-05 22:02:03,130 luigi-interface ERROR Failed connecting to remote scheduler 'http://localhost:8082' 
Traceback (most recent call last): 
    ... 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send 
    r = adapter.send(request, **kwargs) 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 467, in send 
    raise ConnectionError(e, request=request) 
    ConnectionError: HTTPConnectionPool(host='localhost', port=8082): Max retries exceeded with url: /api/add_worker (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f15128cb3d0>: Failed to establish a new connection: [Errno 111] Connection refused',)) 
2017-10-05 22:02:03,180 luigi-interface INFO  Worker Worker(salt=150908931, workers=3, host=etl2, username=develop, pid=18019) was stopped. Shutting down Keep-Alive thread 
Traceback (most recent call last): 
    File "app_metadata.py", line 1567, in <module> 
    luigi.run() 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/interface.py", line 210, in run 
    return _run(*args, **kwargs)['success'] 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/interface.py", line 238, in _run 
    return _schedule_and_run([cp.get_task_obj()], worker_scheduler_factory) 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/interface.py", line 197, in _schedule_and_run 
    success &= worker.run() 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/worker.py", line 867, in run 
    self._add_worker() 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/worker.py", line 652, in _add_worker 
    self._scheduler.add_worker(self._id, self._worker_info) 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/rpc.py", line 219, in add_worker 
    return self._request('/api/add_worker', {'worker': worker, 'info': info}) 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/rpc.py", line 146, in _request 
    page = self._fetch(url, body, log_exceptions, attempts) 
    File "/home/develop/data_warehouse/venv/local/lib/python2.7/site-packages/luigi/rpc.py", line 138, in _fetch 
    last_exception 
    luigi.rpc.RPCError: Errors (3 attempts) when connecting to remote scheduler 'http://localhost:8082' 

중앙 일정에 ping을 시도처럼 보이지만 실패 할 후 추락, 모두 차단 후 작업을 성공적으로 실행할 수 없습니다.

과 비슷한 오류를 만나는 사람도 있지만 해결 방법이 없습니다.

Answer 1

중앙 스케줄러에 과부하가 걸리는 경우 시간 초과를 조금 더 오래 설정하려고합니다. 재 시도를 늘리고 대기 시간을 재 시도 할 수도 있습니다. luigi.cfg

또한 시계를 추가 할 수 있습니다

[core] 
rpc-connect-timeout=60.0 #default is 10.0 
rpc-retry-attempts=10 #default is 3 
rpc-retry-wait=60  #default is 30 

에서

는 스케줄러 프로세스가 자동으로 충돌에 다시 시작해야합니다.