Reverse Proxy with MinIO Server availability problem

See original GitHub issue

Hey, Proxy Manager is really really awesome, it saves me a lot of time but I am facing a little issue with my personal single mode deployment MinIO server.

I have deployed on my Synology NAS (DS415+) a MinIO server though Docker, here is the command (these are not the real root user credentials obviously):

docker run -d --name minio  \
    -p 9100:9000 -p 9101:9001 \
    -e "MINIO_ROOT_USER=root" \
    -e "MINIO_ROOT_PASSWORD=pass" \
    -e "MINIO_DOMAIN=s3.chainyo.tech" \
    -v /volume1/docker/minio/data:/data \
    -v /volume1/docker/minio/config:/root/.minio \
    quay.io/minio/minio server /data --console-address ":9101"

Everything is running well in local, I can access the console (192.168.1.x:9101) and the API (192.168.1.x:9100) though Python script.

But I am using Ngninx Proxy Manager to make this service available outside my local network with my own domain name which is chainyo.tech.

So I added the minio container in the proxy network to make them communicate.

I have setup two redirections:

  • Console: on s3.chainyo.tech I can access to the console on port 9101, everything is running well, it’s working I can access to the console, I can login and manage the server without any restrictions.
  • API: on s3.api.chainyo.tech I can’t use this endpoint in my Python script, the client seems to connect but finally timeout after a long long time.

Here is my simple test script, which timeout after the “client connected” print:

from minio import Minio

def connect(endpoint, access_key, secret_key):
    minioClient = Minio(endpoint, access_key=access_key, secret_key=secret_key, secure=False)
    
    return minioClient

client = connect("s3.api.chainyo.tech:9100", "root", "pass")
print("client connected")

buckets = client.list_buckets()
for bucket in buckets:
    print(bucket.name, bucket.creation_date)

bucket = "public-bucket"
object_to_push = "test.txt"

client.fput_object(bucket, "test.txt", object_to_push)
print("Uploaded object to minio")

And finally this is my Nginx conf file inspired from the MinIO documentation I have found here:

    server {
        set $forward_scheme http;
        set $server         "minio";
        set $port           9100;

        listen 80;
        listen [::]:80;

        server_name s3.api.chainyo.tech;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;

		# Block Exploits
        include conf.d/include/block-exploits.conf;

        access_log /data/logs/proxy-host-15_access.log proxy;
        error_log /data/logs/proxy-host-15_error.log warn;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;

            proxy_pass http://minio:9100
        }
    }

This is the error trace while trying to interact with the api in Python:

Traceback (most recent call last):
  File "/home/chainyo/code/minio/upload_to_minio.py", line 12, in <module>
    buckets = client.list_buckets()
  File "/home/chainyo/.local/lib/python3.8/site-packages/minio/api.py", line 633, in list_buckets
    response = self._execute("GET")
  File "/home/chainyo/.local/lib/python3.8/site-packages/minio/api.py", line 397, in _execute
    return self._url_open(
  File "/home/chainyo/.local/lib/python3.8/site-packages/minio/api.py", line 266, in _url_open
    response = self._http.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/poolmanager.py", line 330, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 747, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 747, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 747, in urlopen
    return self.urlopen(
  [Previous line repeated 2 more times]
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 436, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='s3.api.chainyo.tech', port=9100): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd11dc65e20>: Failed to establish a new connection: [Errno 110] Connection timed out'))

Is there anything I am missing ? Anyone have already achieved that kind of stuff ?

P.S. I used ports 9100 and 9101 because I’m already using port 9000 for portainer service.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:11

github_iconTop GitHub Comments

1reaction
Adigezalovcommented, Sep 25, 2022

@ChainYo Hello. Did you manage to solve this problem? I ran into exactly the same problem and can’t seem to find a solution to the problem.

1reaction
samgaw58commented, Jul 20, 2022

OK thanks, someone somewhere must have gotten this to work!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Set up Nginx proxy with MinIO Server
Nginx is an open source Web server and a reverse proxy server. In this recipe we will learn how to set up Nginx...
Read more >
Minio console not accessible behind nginx reverse proxy
My problem is that, when I'm trying to reverse proxy the minio endpoint to a path, like /minio it does not work, but...
Read more >
Nginx reverse proxy the Minio - Server Fault
There is a method explained by:tapos-ghosh Where you will be making basically a bucket public through backend calls from Nginx..it works ...
Read more >
NGINX settings - GitLab Docs
If your GitLab is behind a reverse proxy, you may not want the IP address of the proxy to show up as the...
Read more >
Enterprise-Grade Cloud Storage with NGINX Plus and Minio
Still, the question remains: How do you build an object storage setup that is ... With an NGINX Plus reverse proxy in front...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found