'ForkAwareLocal' object has no attribute 'connection' for multithreaded cp

See original GitHub issue

When running gsutil -m cp -r gs://example/ ./ on a fairly large folder on macOS with the system Python 3.7 (to prevent the issues from #961), I see many instances of the following error:

Exception in thread Thread-4:
Traceback (most recent call last):
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 788, in _callmethod
    conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2348, in run
    cls = copy.copy(class_map[caller_id])
  File "<string>", line 2, in __getitem__
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 792, in _callmethod
    self._connect()
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 779, in _connect
    conn = self._Client(self._token.address, authkey=self._authkey)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 492, in Client
    c = SocketClient(address)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
    s.connect(address)
ConnectionRefusedError: [Errno 61] Connection refused

As a result the command seems to hang after starting a first batch of downloads.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:35
  • Comments:8 (1 by maintainers)

github_iconTop GitHub Comments

11reactions
dweeklycommented, Feb 10, 2021

The -o “GSUtil:parallel_process_count=1” workaround works for me on Big Sur (macOS 11.2) but it’s frustrating to me that this continues to persist as an issue on the Mac platform.

10reactions
alamothecommented, Dec 1, 2020

Thank you for providing a workaround, it works for me.

This tool is a joke though. Not a single version since 297 works out of the box without some kind of patching.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python multiprocessing error 'ForkAwareLocal' object has no ...
The root cause of this problem is the main thread is done so crash the connection between two processes. However, ...
Read more >
Developers - 'ForkAwareLocal' object has no attribute 'connection ...
When running gsutil -m cp -r gs://example/ ./ on a fairly large folder on macOS with the system Python 3.7 (to prevent the...
Read more >
Python Multiprocessing Manager Error-'ForkAwareLocal ...
Queue in multiprocessing module is used to communicate with the main process. ... AttributeError: 'ForkAwareLocal' object has no attribute 'connection'.
Read more >
1184108 - Upgrade gsutil pin to 4.6x - chromium - Monorail
AttributeError : 'ForkAwareLocal' object has no attribute 'connection' During handling of the above exception, another exception occurred:
Read more >
Issue 27749: multiprocessing Manager error: send() called for ...
After you acquire the lock, try logging the connection information. ... is closed 'ForkAwareLocal' object has no attribute 'connection' ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found