Issue: Running into `URLError <urlopen error The write operation timed out>` when uploading CSV file through API

See original GitHub issue

Description

Hi Folks, I am running into URLError <urlopen error The write operation timed out> when uploading CSV file of large size (~200MB). The file is eventually uploaded into slack, yet the code breaks with that error. Can anyone explain why that’s happening?

What type of issue is this? (place an x in one of the [ ])

  • bug
  • enhancement (feature request)
  • question
  • documentation related
  • testing related
  • discussion

Requirements (place an x in each of the [ ])

  • I’ve read and understood the Contributing guidelines and have done my best effort to follow them.
  • I’ve read and agree to the Code of Conduct.
  • I’ve searched for any related issues and avoided creating a duplicate issue.

Bug Report

Filling out the following details about bugs will help us solve your issue sooner.

Reproducible in:

slackclient version: 2.7.1

python version: 3.6.5

OS version(s): macOS 10.14.6

Steps to reproduce:

  1. Create a webclient with the necessary permissions
  2. Create/Load a dataframe with size ~200MB as df (the one I tested with had a shape of (5400000, 6))
  3. use the files_upload method to load files using the command as shown web_client.files_upload(channels = 'CXXXXXXXXXX', initial_comment="something", content=df.to_csv(index=False), filename="test.csv")

Expected result:

What you expected to happen: File sent with response ‘ok’:true

Actual result:

What actually happened:

---------------------------------------------------------------------------
timeout                                   Traceback (most recent call last)
~/anaconda3/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1317                 h.request(req.get_method(), req.selector, req.data, headers,
-> 1318                           encode_chunked=req.has_header('Transfer-encoding'))
   1319             except OSError as err: # timeout error

~/anaconda3/lib/python3.6/http/client.py in request(self, method, url, body, headers, encode_chunked)
   1238         """Send a complete request to the server."""
-> 1239         self._send_request(method, url, body, headers, encode_chunked)
   1240

~/anaconda3/lib/python3.6/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
   1284             body = _encode(body, 'body')
-> 1285         self.endheaders(body, encode_chunked=encode_chunked)
   1286

~/anaconda3/lib/python3.6/http/client.py in endheaders(self, message_body, encode_chunked)
   1233             raise CannotSendHeader()
-> 1234         self._send_output(message_body, encode_chunked=encode_chunked)
   1235

~/anaconda3/lib/python3.6/http/client.py in _send_output(self, message_body, encode_chunked)
   1064                         + b'\r\n'
-> 1065                 self.send(chunk)
   1066

~/anaconda3/lib/python3.6/http/client.py in send(self, data)
    985         try:
--> 986             self.sock.sendall(data)
    987         except TypeError:

~/anaconda3/lib/python3.6/ssl.py in sendall(self, data, flags)
    971                 while count < amount:
--> 972                     v = self.send(byte_view[count:])
    973                     count += v

~/anaconda3/lib/python3.6/ssl.py in send(self, data, flags)
    940                     self.__class__)
--> 941             return self._sslobj.write(data)
    942         else:

~/anaconda3/lib/python3.6/ssl.py in write(self, data)
    641         """
--> 642         return self._sslobj.write(data)
    643

timeout: The write operation timed out
During handling of the above exception, another exception occurred:                                                                                                                                                                                                     [74/639]

URLError                                  Traceback (most recent call last)
<ipython-input-16-e4c541ebf2f5> in <module>
----> 1 web_client.files_upload(channels = 'CXXXXXXXXXX', initial_comment="something", content=df.to_csv(index=False), filename="test.csv")

~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/client.py in files_upload(self, file, content, **kwargs)
   1217         data = kwargs.copy()
   1218         data.update({"content": content})
-> 1219         return self.api_call("files.upload", data=data)
   1220
   1221     def groups_archive(self, *, channel: str, **kwargs) -> Union[Future, SlackResponse]:

~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in api_call(self, api_method, http_verb, files, data, params, json, headers, auth)
    213                 return self._event_loop.run_until_complete(future)
    214         else:
--> 215             return self._sync_send(api_url=api_url, req_args=req_args)
    216
    217     def _get_url(self, api_method):

~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _sync_send(self, api_url, req_args)
    346             files=files,
    347             json_body=_json,
--> 348             additional_headers=headers,
    349         )
    350

~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _urllib_api_call(self, token, url, query_params, json_body, body_params, files, additional_headers)
    448                 url = f"{url}&{q}" if "?" in url else f"{url}?{q}"
    449
--> 450             response = self._perform_urllib_http_request(url=url, args=request_args)
    451             if response.get("body", None):
    452                 response_body_data: dict = json.loads(response["body"])

~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _perform_urllib_http_request(self, url, args)
    579         except Exception as err:
    580             self._logger.error(f"Failed to send a request to Slack API server: {err}")
--> 581             raise err
    582
    583     def _build_urllib_request_headers(

~/.virtualenvs/scratch/lib/python3.6/site-packages/slack/web/base_client.py in _perform_urllib_http_request(self, url, args)
    560
    561                 resp: HTTPResponse = urlopen(
--> 562                     req, context=self.ssl, timeout=self.timeout
    563                 )
    564                 charset = resp.headers.get_content_charset()

~/anaconda3/lib/python3.6/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    221     else:
    222         opener = _opener
--> 223     return opener.open(url, data, timeout)
    224
    225 def install_opener(opener):

~/anaconda3/lib/python3.6/urllib/request.py in open(self, fullurl, data, timeout)
    524             req = meth(req)
    525
--> 526         response = self._open(req, data)
    527
    528         # post-process response
~/anaconda3/lib/python3.6/urllib/request.py in _open(self, req, data)
    542         protocol = req.type
    543         result = self._call_chain(self.handle_open, protocol, protocol +
--> 544                                   '_open', req)
    545         if result:
    546             return result

~/anaconda3/lib/python3.6/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    502         for handler in handlers:
    503             func = getattr(handler, meth_name)
--> 504             result = func(*args)
    505             if result is not None:
    506                 return result

~/anaconda3/lib/python3.6/urllib/request.py in https_open(self, req)
   1359         def https_open(self, req):
   1360             return self.do_open(http.client.HTTPSConnection, req,
-> 1361                 context=self._context, check_hostname=self._check_hostname)
   1362
   1363         https_request = AbstractHTTPHandler.do_request_

~/anaconda3/lib/python3.6/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1318                           encode_chunked=req.has_header('Transfer-encoding'))
   1319             except OSError as err: # timeout error
-> 1320                 raise URLError(err)
   1321             r = h.getresponse()
   1322         except:

URLError: <urlopen error The write operation timed out>

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:10 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
praveen2593commented, Jul 13, 2020

@seratch Thanks. We can close it. I’m good on my end!! 👍

1reaction
seratchcommented, Jul 9, 2020

@Arszilla We’re not planning to implement such. Please set an extremely large timeout and implement your own retry logic.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python - urllib.error.URLError: <urlopen error timed out>
To handle the timeout case: try: response = urllib.request.urlopen(request, timeout=10) return response except urllib.error.URLError as e: ...
Read more >
socket.timeout: The read operation timed out - Kite Connect
I belive it has something to do with the websocket read time out. I am running the script on google cloud. Error log...
Read more >
Import a CSV file for this view - AppSheet Help - Google Support
The App: import a CSV file for this view action uploads the rows in a CSV file to the specified view. You can...
Read more >
urllib.error.urlerror: <urlopen error [ssl: certificate_verify_failed ...
Here are the steps for macOS: Open the folder /Applications/Python 3.x ( x is the version you are running). Double click the Install...
Read more >
Importing User definitions into Sophos Firewall after v18.0 ...
Using the XML API is a great alternative to the old CSV import feature for ... from time to time, you create a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found