Issue: Running into `URLError <urlopen error The write operation timed out>` when uploading CSV file through API
See original GitHub issueDescription
Hi Folks, I am running into URLError <urlopen error The write operation timed out> when uploading CSV file of large size (~200MB). The file is eventually uploaded into slack, yet the code breaks with that error. Can anyone explain why that’s happening?
What type of issue is this? (place an x in one of the [ ])
- bug
- enhancement (feature request)
- question
- documentation related
- testing related
- discussion
Requirements (place an x in each of the [ ])
- I’ve read and understood the Contributing guidelines and have done my best effort to follow them.
- I’ve read and agree to the Code of Conduct.
- I’ve searched for any related issues and avoided creating a duplicate issue.
Bug Report
Filling out the following details about bugs will help us solve your issue sooner.
Reproducible in:
slackclient version: 2.7.1
python version: 3.6.5
OS version(s): macOS 10.14.6
Steps to reproduce:
- Create a webclient with the necessary permissions
- Create/Load a dataframe with size ~200MB as df (the one I tested with had a shape of (5400000, 6))
- use the files_upload method to load files using the command as shown
web_client.files_upload(channels = 'CXXXXXXXXXX', initial_comment="something", content=df.to_csv(index=False), filename="test.csv")
Expected result:
What you expected to happen: File sent with response ‘ok’:true
Actual result:
What actually happened:
---------------------------------------------------------------------------
timeout Traceback (most recent call last)
~/anaconda3/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1317 h.request(req.get_method(), req.selector, req.data, headers,
-> 1318 encode_chunked=req.has_header('Transfer-encoding'))
1319 except OSError as err: # timeout error
~/anaconda3/lib/python3.6/http/client.py in request(self, method, url, body, headers, encode_chunked)
1238 """Send a complete request to the server."""
-> 1239 self._send_request(method, url, body, headers, encode_chunked)
1240
~/anaconda3/lib/python3.6/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
1284 body = _encode(body, 'body')
-> 1285 self.endheaders(body, encode_chunked=encode_chunked)
1286
~/anaconda3/lib/python3.6/http/client.py in endheaders(self, message_body, encode_chunked)
1233 raise CannotSendHeader()
-> 1234 self._send_output(message_body, encode_chunked=encode_chunked)
1235
~/anaconda3/lib/python3.6/http/client.py in _send_output(self, message_body, encode_chunked)
1064 + b'\r\n'
-> 1065 self.send(chunk)
1066
~/anaconda3/lib/python3.6/http/client.py in send(self, data)
985 try:
--> 986 self.sock.sendall(data)
987 except TypeError:
~/anaconda3/lib/python3.6/ssl.py in sendall(self, data, flags)
971 while count < amount:
--> 972 v = self.send(byte_view[count:])
973 count += v
~/anaconda3/lib/python3.6/ssl.py in send(self, data, flags)
940 self.__class__)
--> 941 return self._sslobj.write(data)
942 else:
~/anaconda3/lib/python3.6/ssl.py in write(self, data)
641 """
--> 642 return self._sslobj.write(data)
643
timeout: The write operation timed out
During handling of the above exception, another exception occurred: [74/639]
URLError Traceback (most recent call last)
<ipython-input-16-e4c541ebf2f5> in <module>
----> 1 web_client.files_upload(channels = 'CXXXXXXXXXX', initial_comment="something", content=df.to_csv(index=False), filename="test.csv")
~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/client.py in files_upload(self, file, content, **kwargs)
1217 data = kwargs.copy()
1218 data.update({"content": content})
-> 1219 return self.api_call("files.upload", data=data)
1220
1221 def groups_archive(self, *, channel: str, **kwargs) -> Union[Future, SlackResponse]:
~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in api_call(self, api_method, http_verb, files, data, params, json, headers, auth)
213 return self._event_loop.run_until_complete(future)
214 else:
--> 215 return self._sync_send(api_url=api_url, req_args=req_args)
216
217 def _get_url(self, api_method):
~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _sync_send(self, api_url, req_args)
346 files=files,
347 json_body=_json,
--> 348 additional_headers=headers,
349 )
350
~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _urllib_api_call(self, token, url, query_params, json_body, body_params, files, additional_headers)
448 url = f"{url}&{q}" if "?" in url else f"{url}?{q}"
449
--> 450 response = self._perform_urllib_http_request(url=url, args=request_args)
451 if response.get("body", None):
452 response_body_data: dict = json.loads(response["body"])
~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _perform_urllib_http_request(self, url, args)
579 except Exception as err:
580 self._logger.error(f"Failed to send a request to Slack API server: {err}")
--> 581 raise err
582
583 def _build_urllib_request_headers(
~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _perform_urllib_http_request(self, url, args)
560
561 resp: HTTPResponse = urlopen(
--> 562 req, context=self.ssl, timeout=self.timeout
563 )
564 charset = resp.headers.get_content_charset()
~/anaconda3/lib/python3.6/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
221 else:
222 opener = _opener
--> 223 return opener.open(url, data, timeout)
224
225 def install_opener(opener):
~/anaconda3/lib/python3.6/urllib/request.py in open(self, fullurl, data, timeout)
524 req = meth(req)
525
--> 526 response = self._open(req, data)
527
528 # post-process response
~/anaconda3/lib/python3.6/urllib/request.py in _open(self, req, data)
542 protocol = req.type
543 result = self._call_chain(self.handle_open, protocol, protocol +
--> 544 '_open', req)
545 if result:
546 return result
~/anaconda3/lib/python3.6/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
502 for handler in handlers:
503 func = getattr(handler, meth_name)
--> 504 result = func(*args)
505 if result is not None:
506 return result
~/anaconda3/lib/python3.6/urllib/request.py in https_open(self, req)
1359 def https_open(self, req):
1360 return self.do_open(http.client.HTTPSConnection, req,
-> 1361 context=self._context, check_hostname=self._check_hostname)
1362
1363 https_request = AbstractHTTPHandler.do_request_
~/anaconda3/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
1318 encode_chunked=req.has_header('Transfer-encoding'))
1319 except OSError as err: # timeout error
-> 1320 raise URLError(err)
1321 r = h.getresponse()
1322 except:
URLError: <urlopen error The write operation timed out>
Issue Analytics
- State:
- Created 3 years ago
- Comments:10 (5 by maintainers)
Top Results From Across the Web
python - urllib.error.URLError: <urlopen error timed out>
To handle the timeout case: try: response = urllib.request.urlopen(request, timeout=10) return response except urllib.error.URLError as e: ...
Read more >socket.timeout: The read operation timed out - Kite Connect
I belive it has something to do with the websocket read time out. I am running the script on google cloud. Error log...
Read more >Import a CSV file for this view - AppSheet Help - Google Support
The App: import a CSV file for this view action uploads the rows in a CSV file to the specified view. You can...
Read more >urllib.error.urlerror: <urlopen error [ssl: certificate_verify_failed ...
Here are the steps for macOS: Open the folder /Applications/Python 3.x ( x is the version you are running). Double click the Install...
Read more >Importing User definitions into Sophos Firewall after v18.0 ...
Using the XML API is a great alternative to the old CSV import feature for ... from time to time, you create a...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@seratch Thanks. We can close it. I’m good on my end!! 👍
@Arszilla We’re not planning to implement such. Please set an extremely large timeout and implement your own retry logic.