Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Self-host: Storage API is not working (Docker container unhealthy) #27312

Open
2 tasks done
RoadToDev101 opened this issue Jun 17, 2024 · 12 comments · May be fixed by #29805
Open
2 tasks done

Self-host: Storage API is not working (Docker container unhealthy) #27312

RoadToDev101 opened this issue Jun 17, 2024 · 12 comments · May be fixed by #29805
Labels
bug Something isn't working self-hosted Issues related to self hosting

Comments

@RoadToDev101
Copy link

Bug report

  • I confirm this is a bug with Supabase, not with my own application.
  • I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

I self-host Supabase with Docker on my machine as per the instructions on Supabase Docs. After successfully composing, the containers supabase/storage-api:v1.0.6 and supabase/studio are in an unhealthy status. I can create buckets in the dashboard but fail to create folders in the bucket or upload files.

To Reproduce

Steps to reproduce the behavior, please provide code snippets or a repository:

  1. Follow the instructions on the Supabase self-hosting Docker guide.
  2. Run docker-compose up.
  3. Check the status of the containers supabase/storage-api and supabase/studio.
  4. Try creating a bucket, then attempt to create a folder in the bucket or upload a file.

Expected behavior

The supabase/storage-api and supabase/studio containers should be in a healthy status, allowing the creation of folders and uploading of files within buckets.

Screenshots

image

System information

  • Operating System: Ubuntu 22.04.4 LTS
  • Kernel: Linux 5.15.0-107-generic
  • Browser (if applies) Chrome
  • Version of Node.js: v20.11.1

image

@RoadToDev101 RoadToDev101 added the bug Something isn't working label Jun 17, 2024
@encima encima added the self-hosted Issues related to self hosting label Jun 17, 2024
@encima
Copy link
Member

encima commented Jun 17, 2024

Thanks for opening @RoadToDev101
This is a bug in the self-hosted docker-compose as the storage api seems not to listen on localhost, so it reports as unhealthy when it is fine.
This can be confirmed with docker logs $STORAGE_CONTAINER_ID

We will make a PR to address this. That said, the issue of not being able to create folders cannot be reproduced, can you provide logs from the storage container, please?

@RoadToDev101
Copy link
Author

RoadToDev101 commented Jun 18, 2024

Hi @encima,

This is the log I get from the storage container when render the storage page on dashboard and create a new bucket name 'design_views'

{"level":30,"time":"2024-06-18T01:26:18.129Z","pid":1,"hostname":"a122cace4088","msg":"Server listening at http://0.0.0.0:5000"} {"level":30,"time":"2024-06-18T01:28:10.474Z","pid":1,"hostname":"a122cace4088","reqId":"req-1","tenantId":"stub","project":"stub","reqId":"req-1","type":"request","req":{"traceId":"req-1","method":"GET","url":"/bucket","headers":{"host":"storage:5000","x_forwarded_proto":"http","x_forwarded_host":"kong","x_forwarded_port":"8000","x_real_ip":"192.168.48.11","x_client_info":"supabase-js-node/2.41.1","accept":"*/*","user_agent":"node"},"hostname":"storage:5000","remoteAddress":"192.168.48.12","remotePort":43888},"res":{"statusCode":200,"headers":{"content_type":"application/json; charset=utf-8","content_length":"2"}},"responseTime":40.07513475418091,"msg":"stub | GET | 200 | 192.168.48.12 | req-1 | /bucket | node"} {"level":30,"time":"2024-06-18T01:28:26.236Z","pid":1,"hostname":"a122cace4088","reqId":"req-2","tenantId":"stub","project":"stub","reqId":"req-2","type":"request","req":{"traceId":"req-2","method":"POST","url":"/bucket","headers":{"host":"storage:5000","x_forwarded_proto":"http","x_forwarded_host":"kong","x_forwarded_port":"8000","x_real_ip":"192.168.48.11","content_length":"107","content_type":"application/json","x_client_info":"supabase-js-node/2.41.1","accept":"*/*","user_agent":"node"},"hostname":"storage:5000","remoteAddress":"192.168.48.12","remotePort":53754},"res":{"statusCode":200,"headers":{"content_type":"application/json; charset=utf-8","content_length":"23"}},"responseTime":131.50732326507568,"msg":"stub | POST | 200 | 192.168.48.12 | req-2 | /bucket | node"} {"level":30,"time":"2024-06-18T01:28:26.252Z","pid":1,"hostname":"a122cace4088","reqId":"req-3","tenantId":"stub","project":"stub","reqId":"req-3","type":"request","req":{"traceId":"req-3","method":"GET","url":"/bucket","headers":{"host":"storage:5000","x_forwarded_proto":"http","x_forwarded_host":"kong","x_forwarded_port":"8000","x_real_ip":"192.168.48.11","x_client_info":"supabase-js-node/2.41.1","accept":"*/*","user_agent":"node"},"hostname":"storage:5000","remoteAddress":"192.168.48.12","remotePort":53754},"res":{"statusCode":200,"headers":{"content_type":"application/json; charset=utf-8","content_length":"200"}},"responseTime":5.60968017578125,"msg":"stub | GET | 200 | 192.168.48.12 | req-3 | /bucket | node"} {"level":30,"time":"2024-06-18T01:28:26.349Z","pid":1,"hostname":"a122cace4088","reqId":"req-4","tenantId":"stub","project":"stub","reqId":"req-4","type":"request","req":{"traceId":"req-4","method":"POST","url":"/object/list/design_views","headers":{"host":"storage:5000","x_forwarded_proto":"http","x_forwarded_host":"kong","x_forwarded_port":"8000","x_real_ip":"192.168.48.11","content_length":"89","content_type":"application/json","x_client_info":"supabase-js-node/2.41.1","accept":"*/*","user_agent":"node"},"hostname":"storage:5000","remoteAddress":"192.168.48.12","remotePort":53754},"res":{"statusCode":200,"headers":{"content_type":"application/json; charset=utf-8","content_length":"2"}},"responseTime":7.82005500793457,"msg":"stub | POST | 200 | 192.168.48.12 | req-4 | /object/list/design_views | node"}

After that, I try to create a new folder in this bucket, the frontend shows that it created but when I reload, the folder is not there (I think it is not actually create the folder). And when I try uploading the file, it just spinning forever. And in these 2 actions, there is no log in the storage container.

image

Plus the studio container also unhealthy, but when check the log, it seems there is no problem. I hope this help.

image

@encima
Copy link
Member

encima commented Jun 18, 2024

thanks @RoadToDev101 , those logs show success for the storage requests so that certainly is weird!
Is there some issue with the volume mounting? Are you able to create a folder when you call the storage API directly?

@RoadToDev101
Copy link
Author

Hi @encima,

I try to fetch the bucket api on Postman but I cant due to "Invalid authentication credentials". I try both Anon and Service Key for authorization. See the image below

image

The jwt in .env :
JWT_SECRET=girSzUKW1c937xjnJhKhuqLLM4WPIjNXrV8Odvra ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.ewogICJyb2xlIjogImFub24iLAogICJpc3MiOiAic3VwYWJhc2UiLAogICJpYXQiOiAxNzE4NjQzNjAwLAogICJleHAiOiAxODc2NDEwMDAwCn0.STjbgWo03Cn8SRaRAn2bqw0M1io-k6g8GJkHAeUwXY4 SERVICE_ROLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.ewogICJyb2xlIjogInNlcnZpY2Vfcm9sZSIsCiAgImlzcyI6ICJzdXBhYmFzZSIsCiAgImlhdCI6IDE3MTg2NDM2MDAsCiAgImV4cCI6IDE4NzY0MTAwMDAKfQ.aq4Nv_JAsUJAJxwvr5Ro6GDyXVbyoGb9ij2UN40kJ_Y

Am I do it wrong?

image

@RoadToDev101
Copy link
Author

@encima, so strange... I can upload the image through my NodeJS app.

image

@encima
Copy link
Member

encima commented Jun 22, 2024

@RoadToDev101 thanks for the update. so it works using the supabase client is that right? does it still fail with the dashboard?

from Postman, it looks like the credentials are probably failing due to them being incorrectly encoded or not picked up properly.

@bachkukkik
Copy link

bachkukkik commented Jun 23, 2024

for supabase/studio, I think it is because of the starting period

I went inside the docker container and command

$ node -e "require('http').get('http://localhost:3000/api/profile', (r) => {if (r.statusCode !== 200) throw new Error(r.statusCode)})"

it gave me no error so i increased timeout in health check condition as below to fix unhealthy

supabase-studio:
  image: 'supabase/studio:20240422-5cf8f30'
  healthcheck:
    test:
      - CMD
      - node
      - '-e'
      - "require('http').get('http://localhost:3000/api/profile', (r) => {if (r.statusCode !== 200) throw new Error(r.statusCode)})"
    timeout: 15s
    interval: 5s
    retries: 5
  depends_on:
    supabase-analytics:
      condition: service_healthy

However it doesn't work with storage-api. I tried to debug the same way but got below result

/app # wget --no-verbose --tries=1 --spider http://localhost:5000/status
Connecting to localhost:5000 ([::1]:5000)
wget: can't connect to remote host: Connection refused

@encima
Copy link
Member

encima commented Jun 23, 2024

for supabase/studio, I think it is because of the starting period

I went inside the docker container and command

$ node -e "require('http').get('http://localhost:3000/api/profile', (r) => {if (r.statusCode !== 200) throw new Error(r.statusCode)})"

it gave me no error so i increased timeout in health check condition as below to fix unhealthy

supabase-studio:
  image: 'supabase/studio:20240422-5cf8f30'
  healthcheck:
    test:
      - CMD
      - node
      - '-e'
      - "require('http').get('http://localhost:3000/api/profile', (r) => {if (r.statusCode !== 200) throw new Error(r.statusCode)})"
    timeout: 15s
    interval: 5s
    retries: 5
  depends_on:
    supabase-analytics:
      condition: service_healthy

However it doesn't work with storage-api. I tried to debug the same way but got below result

/app # wget --no-verbose --tries=1 --spider http://localhost:5000/status
Connecting to localhost:5000 ([::1]:5000)
wget: can't connect to remote host: Connection refused

Are you connecting to storage from within the studio container? Either way, localhost will not work as it is not exposed there (unless on the storage container). You can use the same command but use studio instead of localhost

@RoadToDev101
Copy link
Author

@encima I use the healthcheck command in the docker compose file for studio container and get this error:
image

@encima
Copy link
Member

encima commented Jun 24, 2024

@encima I use the healthcheck command in the docker compose file for studio container and get this error:

image

@RoadToDev101 that is expected yes. the studio is accessible through "studio" but not localhost.

RoadToDev101 pushed a commit to RoadToDev101/supabase that referenced this issue Jun 24, 2024
@major-winter
Copy link

I'm also facing the unhealthy issues on those two services. In addition, I also got the unauthorized issue when uploading files that are more than 6 MB; though, it works fine for those that are less than that.

@BierDav
Copy link

BierDav commented Sep 24, 2024

Same issue here, the nginx server doesn't serve on localhost which makes the healthcheck provided in the github repo not working. Is there a workaround? Another healthcheck command. I tried to comeup with one, but without the ability to fetch from localhost it is kinda difficult to not hard code an IP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working self-hosted Issues related to self hosting
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

5 participants