파이썬 다중 처리로 서브 프로세스에 LDAP 객체를 공유 할 수 없습니다.

Question

응답을 검색하기 위해 LDAP 쿼리와 다른 하위 프로세스를 보낼 하나의 하위 프로세스가 있습니다. 두 프로세스간에 LDAP 개체를 공유하는 데 문제가 있습니다. 누군가 해결책을 가르쳐 줄 수 있습니까?

import ldap 
from multiprocessing import Process 


def send_ldap(ldap_conn): 
    print ldap_conn 

def receive_ldap(ldap_conn): 
    print ldap_conn 

def main(): 
    ldap_conn = ldap.initialize('ldap://abc:12345') 
    ldap_sender = Process(target=send_ldap, args=(ldap_conn,)) 
    ldap_receiver = Process(target=receive_ldap, args=(ldap_conn,)) 
    ldap_sender.start() 


if __name__ == '__main__': 
    main() 

오류는 피클 모듈입니다 :

Traceback (most recent call last): 
    File "t.py", line 22, in <module> 
    main() 
    File "t.py", line 16, in main 
    ldap_sender.start() 
    File "c:\python27\lib\multiprocessing\process.py", line 130, in start 
    self._popen = Popen(self) 
    File "c:\python27\lib\multiprocessing\forking.py", line 277, in __init__ 
    dump(process_obj, to_child, HIGHEST_PROTOCOL) 
    File "c:\python27\lib\multiprocessing\forking.py", line 199, in dump 
    ForkingPickler(file, protocol).dump(obj) 
    File "c:\python27\lib\pickle.py", line 224, in dump 
    self.save(obj) 
    File "c:\python27\lib\pickle.py", line 331, in save 
    self.save_reduce(obj=obj, *rv) 
    File "c:\python27\lib\pickle.py", line 419, in save_reduce 
    save(state) 
    File "c:\python27\lib\pickle.py", line 286, in save 
    f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 649, in save_dict 
    self._batch_setitems(obj.iteritems()) 
    File "c:\python27\lib\pickle.py", line 681, in _batch_setitems 
    save(v) 
    File "c:\python27\lib\pickle.py", line 286, in save 
    f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 548, in save_tuple 
    save(element) 
    File "c:\python27\lib\pickle.py", line 286, in save 
    f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 725, in save_inst 
    save(stuff) 
    File "c:\python27\lib\pickle.py", line 286, in save 
    f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 649, in save_dict 
    self._batch_setitems(obj.iteritems()) 
    File "c:\python27\lib\pickle.py", line 681, in _batch_setitems 
    save(v) 
    File "c:\python27\lib\pickle.py", line 286, in save 
    f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 725, in save_inst 
    save(stuff) 
    File "c:\python27\lib\pickle.py", line 286, in save 
    f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 649, in save_dict 
    self._batch_setitems(obj.iteritems()) 
    File "c:\python27\lib\pickle.py", line 681, in _batch_setitems 
    save(v) 
    File "c:\python27\lib\pickle.py", line 331, in save 
    self.save_reduce(obj=obj, *rv) 
Traceback (most recent call last): 
    File "c:\python27\lib\pickle.py", line 396, in save_reduce 
    File "<string>", line 1, in <module> 
     File "c:\python27\lib\multiprocessing\forking.py", line 381, in main 
save(cls) 
     File "c:\python27\lib\pickle.py", line 286, in save 
self = load(from_parent) 
     File "c:\python27\lib\pickle.py", line 1378, in load 
f(self, obj) # Call unbound method with explicit self 
    File "c:\python27\lib\pickle.py", line 748, in save_global 
    return Unpickler(file).load() 
    File "c:\python27\lib\pickle.py", line 858, in load 
    (obj, module, name)) 
pickle.PicklingError: Can't pickle <type 'thread.lock'>: it's not found as thread.lock 
dispatch[key](self) 
    File "c:\python27\lib\pickle.py", line 880, in load_eof 
    raise EOFError 
EOFError 

Answer 1

문제는 ldap_conn 개체가 Windows에서 프로세스 사이를 보낼 필요로하는, picklable되지 않는 것입니다. 일부 비 픽업 가능 threading.Lock 오브젝트를 내부적으로 사용하기 때문에 picklable이 아닙니다. 도서관은 실제로 picklable로되어있는 ReconnectLDAPObject을 제공한다고 주장하지만, 고장 났고 picklable도 아닙니다. 우리는 서브 그것에 의해,하지만,이 문제를 해결할 수 버그 (이것은 세척하기 전에 내부 잠금 한 제거 미스) 고정 : 그것은 multiprocessing위한 유용한되어야 즉, 지금 대상 미세 피클 것이다

from ldap.ldapobject import ReconnectLDAPObject 
from multiprocessing import Process 
import ldap 

class PicklableLDAPObject(ReconnectLDAPObject): 
    def __getstate__(self): 
     d = ReconnectLDAPObject.__getstate__(self) 
     del d['_reconnect_lock'] 
     return d 

    def __setstate__(self, d): 
     self._reconnect_lock = ldap.LDAPLock(desc='reconnect lock within %s' % (repr(self))) 
     ReconnectLDAPObject.__setstate__(self, d) 

def send_ldap(ldap_conn): 
    print ldap_conn 

def receive_ldap(ldap_conn): 
    print ldap_conn 

def main(): 
    #ldap_conn = ldap.initialize('ldap://abc:12345') 
    ldap_conn = PicklableLDAPObject('ldap://abc:12345') 
    ldap_sender = Process(target=send_ldap, args=(ldap_conn,)) 
    ldap_receiver = Process(target=receive_ldap, args=(ldap_conn,)) 
    ldap_sender.start() 

을 목적.