Cluster bootstrap fails ("too many colons in address") when IPv6 addresses are returned by DNS discovery #1500
Comments
|
Thanks -- I've just released 8.12.2 which should fix your immediate issue. That said, while I want to support IPv6, I suspect you're going to hit other issues like this since I've never actually tested IPv6. I'll fix them as you hit them. As for the other issues, that was due to out-of-date docs, which I've fixed now (though let me know if needs to be clearer). In summary rqlite 8.x switched to using the Raft port of other nodes for Join operations. Releases before that used the HTTP port of other nodes. Perhaps this means you will need two versions of your charts -- one for 7.x and one for 8.x (or just support 8.x -- there will be no more 7.x releases). |
|
Anyway, if you confirm this specific issue is addressed in the latest release, perhaps you can close. |
|
All my testing so far shows that using IPv6 addresses works. |
|
Yep, looks good here with v8.12.3, @otoolep. Cluster is now bootstrapping with v6 addresses from DNS discovery. Also thanks for fixing the port details in the docs. I'm getting the impression that you don't sleep. :) |
The language is clear enough now. The only tweak I might be inclined to make is in the code block (and the corresponding DNS name in the paragraph below), changing the service name from
I think I'll just target 8.x for the chart, unless there's a compelling use case for new deployments running 7.x. |
Good idea, done. See rqlite/rqlite.io@186d936 |
Given that, I believe https://rqlite.io/docs/guides/security/#secure-cluster-example also needs updating? The section has examples that use HTTP URLs with |
|
Jeez -- yeah. Thanks. |
What version are you running?
v8.12.1
Are you using Docker or Kubernetes to run your system?
Yes, Microk8s configured with IPv6 support (dual stack)
Are you running a single node or a cluster?
3 node cluster
What did you do?
In developing the Helm chart, I tested out enabling dual stack mode on the K8s service that rqlite uses for service discovery. So rqlite will receive IPv6 addresses from DNS in discovering its peers. With both
dnsanddns-srv, I observed the cluster failing to bootstrap with the following error:I haven't looked at the code, but my complete guess is that rqlite is handling the DNS lookup, receiving IPv6 addresses, appending
:<port>to each address, and passing that up to the raft library. I suspect the raft library requires IPv6 addresses to be represented in the canonical form[<addr>]:<port>, for example[2001:470:b0d6:fd:dfe:afeb:1d8a:32bb]:4001, and it's rqlite's responsibility to format it appropriately?A bit of a tangent here, but note port 4001 above. As you can tell from the diagnostic info below, I started rqlited with
-disco-mode=dns-srv -disco-config={"name":"rqlite-headless","service":"http"}. Even once I reverted therqlite-headlessservice to IPv4-only, I still had problems withdns-srv. I specifiedhttpas the service name because the docs references the HTTP port. But in practice this seems to require the raft port (4002), not http. Changing disco-config to{"name":"rqlite-headless","service":"raft"}which gets back port 4002 fixed that additional issue.Please include the Status, Nodes, and Expvar output from each node (or at least the Leader!)
Run from one of the nodes. Cluster didn't bootstrap so technically no leader.
The text was updated successfully, but these errors were encountered: