-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Supabase/Supavisor #238
Comments
Happy to help with testing. But just wanted to let you know I'm not actually using Supabase in production (I'm using crunchy data which seems fine). |
Currently, I'm thinking that River should support a "poll only" mode for listening for new jobs and leader election in order to work around systems where nothing like A separate problem is that the errors above appear to be from prepared statement problems, although theoretically Hypavisor supports prepared statements now. More investigation needed to detangle what's happening there. |
Thanks @andreirtaylor.
Good choice! |
We also run into issues with supabase. End of January supabase switched all users over to Supavisor. So we connect to Supavisor in session mode and run into the issue that too many connections are opened and eventually we run out of free connections. But everything works for a while. We can start the river queue and runs fine for a couple of minutes. I am fairly new to Supavisor River. So far I tried the following to debug the issue: By looking at the queries that open the new connections with Is there a workaround for that issue? In the comment #205 (comment) a potential fix (b2cb142) was mentioned. To me it looks like it addresses a different issue. Is that correct? We are not using any supabase specific features (except auth). So we might just migrate away from them to a different managed Postgres provider if the issue can't be fixed easily. |
@JanRuettinger It is possible there's a bug in there somewhere, I'll try to repro it. Can you give us a few more details about what your code looks like to help with the repro? i.e. Client configuration, what your worker configuration roughly looks like, etc.
Yeah, I don't think that's related to what you're encoutering. |
I have created a minimal example to reproduce the issue: EDIT: connecting directly to the db works without issues. So the bug seems to be related to supavisor. |
@JanRuettinger Could you try adding the following to https://github.com/JanRuettinger/river_issue/blob/main/jobqueue/river.go#L39: config.ConnConfig.DefaultQueryExecMode = pgx.QueryExecModeSimpleProtocol You need to also import the https://github.com/riverqueue/river/tree/brandur-simple-protocol branch for it to work with the above change. |
Not really proprietary . But they do advertise it as an: Easy drop-in replacement for PgBouncer. I think this should be a low priority and if possible, be pushed to the Supabase team to properly support binary protocol because river now works with pgbouncer. |
Any updates on this? I am using supabase and sometimes, I do get these errors. Would like to know what's the current situation, what's getting affected and if there are any recommended changes on my end. Thanks! |
@brycegoh Can you elaborate which errors you're getting? I would think that you'd either see a total failure if using Supavisor, or reasonable success if using PgBouncer. |
Thanks for the quick reply. I am getting these kind of errors: I use both normal jobs and periodic jobs with only 1 worker, both seems to be working even though I get those errors. Therefore, I am trying to understand the degree of the problem and if this is an urgent bug on my end. Apologies as I don't 100% understand the inner workings of the prepared-statements and river etc, would appreciate some assistance as I wrap my head around this. |
This means you're using transaction mode. You can use session mode on Supabase with Supavisor using port You can use two separate pool types this way (seems may be useful for this project from reading another thread). re: listen/notify on Supavisor Should work with session mode. re: binary protocol We proxy the binary protocol like pgbouncer does. Extended queries work. re: @JanRuettinger your repro repo This is amazing! Thanks so much. Was able to reproduce. Seems we're getting:
Which I suspect is leaving stray connections open eventually eating up your connection pool. I will open a Supavisor issue. |
River picked up a poll-only mode in version 0.3.0 that doesn't use listen/notify. It may help with some of the Supabase cases, if anyone wants to try it. |
Hi, I am using supabase with riverqueue. I have 2 river clients using river v0.0.15:
Both insert only and worker river client are connected to supabase using the I am getting May I ask for some advice on this issue please. Thanks! |
@brycegoh I'm afraid near myself nor Blake are all that familiar with Supabase, especially with regards to their connection pooler. You're saying though that if you switch it to use the more strict transaction mode, it actually starts working? That seems odd. Is there any other detail in the error message to indicate why there was an error beginning the transaction? |
Extracted from @andreirtaylor in #205:
The text was updated successfully, but these errors were encountered: