Summary

Adding to read only command preference option to rueidis. read only command preference can distributes read only commands to primary and replicas evenly.

Problem Definition

Background

Handling read only command via replica makes valkey/redis cluster scalable. When using replica, it is important to distribute loads in a each node. Because uneven load balancing makes cluster less-scalable and inefficient in terms of resource usage especially read-heavy workload.

Current rueidis can distribute command using “SendToReplicas”. But problem is “SendToReplicas” doesn’t consider how many nodes in a each shard. If a valkey/redis cluster uses auto scaling, “SendToReplicas” shows less efficient load balancing.

Goals

• Load balancing strategy to read only command.
• Dynamic Load Balancing Strategy to read only command.

Keywords Description

• Read only command preference: Load Balancing strategy applied to only read only command. This preference determines which node should be handle read only command in a slot range.

Design

Load Balancing Strategy

Reader node selection.

One of important thing in load balancing is readerNodeSelector . readerNodeSelector chooses which node is reader each slot. readerNodeSelector runs in _refresh function, when setting conn per each slot.

// readerNodeSelector selects a reader node from a list of connections.
type readerNodeSelector interface {
	// Select selects a reader node from a list of connections based on the read preference.
	Select(primaryConn connrole, replicaConns []connrole) (connrole, error)
}

User can configure readerNodeSelector via ReadPreference option in ClientOption .

	// ReadPreference is the read preference for the cluster client.
	// NOTE: This function can't be used with ReplicaOnly or SendToReplicas.
	ReadPreference ReadPreference

// ReadPreference defines which node to read from in a cluster
type ReadPreference int

Using factory method, readerNodeSelector implementation is returned internally.

func newReaderNodeSelector(preference ReadPreference) (readerNodeSelector, error)

Routing to reader node.

If ReadPreference is configured in ClientOption , then sendToReadershould be set to SendReplicas function. sendToReader sends read only command to selected reader node.

var sendToReader = func(cmd Completed) bool {
	if cmd.IsReadOnly() {
		return true
	}
	return false
}

Dynamic Load Balancing

We can achieve Any read preference and ShardsRefreshInterval .

If Any is set, then newReaderNodeSelector returns anyNodeSelector . anyNodeSelector chooses connection randomly. So If replica is added because of auto scaling, then after refreshed cluster connection cache, we can distribute read requests to each node evenly.

const (
	PreferenceUndefined ReadPreference = iota // PreferenceUndefined is the default read preference
	Any                 ReadPreference = iota // Any reads from any node
)

// anyNodeSelector selects any node from a list of connections.
type anyNodeSelector struct{}

// Select selects any node from a list of connections.
func (s *anyNodeSelector) Select(primaryConn connrole, replicaConns []connrole) (connrole, error) {
	i := util.FastRand(len(replicaConns)   1)
	if i == 0 {
		return primaryConn, nil
	}
	return replicaConns[i-1], nil
}

Alternatives Considered

I’ve considered making custom SendReplicas . But it is hard. Because In a SendReplicas we can’t know how many nodes in each shard.