Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use AWS Redis Primary Endpoint & Reader Endpoint #2503

Open
2 tasks done
Nyholm opened this issue Jun 9, 2024 · 9 comments
Open
2 tasks done

Use AWS Redis Primary Endpoint & Reader Endpoint #2503

Nyholm opened this issue Jun 9, 2024 · 9 comments
Assignees

Comments

@Nyholm
Copy link

Nyholm commented Jun 9, 2024

Using AWS, you may setup multiple Redis nodes and they run with cluster_mode=disabled.
AWS will give you a write endpoint and a read endpoint. AWS will automatically load balance the read requests between nodes.

Expected behaviour

$nodes = [
  'cluster-with-no-cluster-mode.xxx.ng.0001.euw1.cache.amazonaws.com:6379',
  'cluster-with-no-cluster-mode-ro.xxx.ng.0001.euw1.cache.amazonaws.com:6379'
];
$redis = new \RedisCluster(null, $nodes);
$redis->setOption(\RedisCluster::OPT_SLAVE_FAILOVER, \RedisCluster::FAILOVER_DISTRIBUTE_SLAVES);

Actual behaviour

When instantiating RedisCluster a RedisClusterException is thrown with:

Couldn't map cluster keyspace using any provided seed"

I assume it is because I am trying to use the RedisCluster class when my nodes has disabled cluster mode. I am curious how I could separate my reads from my writes easily with this setup.

I'm seeing this behaviour on

  • OS: Linux
  • Redis: 7.1.0
  • PHP: 8.3.7
  • phpredis: 6.0.2

Steps to reproduce, backtrace or example script

I've checked

  • There is no similar issue from other users
  • Issue isn't fixed in develop branch

I've seen related issues like: #1899

@michael-grunder michael-grunder self-assigned this Jun 10, 2024
@michael-grunder
Copy link
Member

Hi, I'm honestly not sure about this one since I've never tried it.

My instinct is that if AWS is doing the load balancing you probably don't want to set OPT_SLAVE_FAILOVER in PhpRedis but that's just a guess.

Also that shouldn't be why we can't map the cluster. I probably need to play around with this in a small test cluster to figure out what's going on.

Is it possible that AWS is doing the "cluster" work for you and you want to use the non-cluster Redis class instead?

You can also try running this utility script I created a while back that just tries to map out cluster topology with the Redis class to try and diagnose where there might be issues.

@Nyholm
Copy link
Author

Nyholm commented Jun 12, 2024

Thank you for your reply.

Is it possible that AWS is doing the "cluster" work for you and you want to use the non-cluster Redis class instead?

Yes, that is very possible. Using Redis with my "Primary endpoint" works as expected. But I (naively) would like the extension to separate the my reads from writes. Is that something I can do with the Redis class?

You can also try running this utility script

Yes, the Redis cluster mode is not active. AWS is doing the "cluster" work for me.

--------------------------------------------------------------------
START
Checking general cluster INFO: --- CLUSTER INFO ---
Error: Cluster is not properly up!
--- BACKTRACE ---
[0] script.php:172 panicAbort()
[1] script.php:38 checkClusterState()
[2] script.php:298 run()

@michael-grunder
Copy link
Member

But I (naively) would like the extension to separate the my reads from writes. Is that something I can do with the Redis class

We'd have to add that feature to the extension to do that transparently. It could also be done purely in userland PHP by proxying two Redis objects (one read, and one write) although that probably incurs a significant performance hit.

If this is a common setup in AWS it might be worth adding to PhpRedis.

@Nyholm
Copy link
Author

Nyholm commented Jun 13, 2024

It could also be done purely in userland PHP by proxying two Redis objects (one read, and one write) although that probably incurs a significant performance hit.

That is where I started =)
Predis does something like that, so I would write a small library around two Redis classes. But that will cause issues if you use a higher level abstraction like psr-6 or Symfony cache. Ie, they require Redis | Predis | RedisCluster as argument.

If this is a common setup in AWS it might be worth adding to PhpRedis.

I cannot say how popular it is. But I did some research on Google Cloud. Both AWS and Google cloud have 3 options. (A) One node, (B) cluster mode or (C) one write node with many read replicas. Note that all read replicas are sharing one endpoint.

I would be interested to see this feature in PhpRedis.

@dkarlovi
Copy link

Note that all read replicas are sharing one endpoint.

So this is sort of:

  1. one (logical and actual) write node
  2. one (logical) read node (backed by one or more actual read nodes)

This wouldn't be a cluster then, it would be the plain Redis with an additional connection to the logical read node, with the extension automatically using that for any reads?

@Nyholm
Copy link
Author

Nyholm commented Jun 18, 2024

This wouldn't be a cluster then, it would be the plain Redis with an additional connection to the logical read node, with the extension automatically using that for any reads?

Yes, that is very much the idea.

@dkarlovi
Copy link

dkarlovi commented Jun 18, 2024

Who's in charge of solving eventual consistency?

If you send a write and then immediately a read for the same key, you're bound to get inconsistencies with the most basic approach, right? Or do they fix it for you in their proxy somehow?

@Nyholm
Copy link
Author

Nyholm commented Jun 18, 2024

Who's in charge of solving eventual consistency?

The platform (AWS/Goggle) is. And the application. We would never want the PHP extension to keep state.

For the application: A common setup is to use chain cache. Ie you have an in-memory (or APCu) cache in front of Redis.

For the platform: The smart people att AWS/Google are doing their best to reduce the time window for inconsistencies.

@Nyholm
Copy link
Author

Nyholm commented Jul 16, 2024

How big of a task is this to implement? Ie, is this a massive job or is it something I should try myself over a few weekends?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants