[BUG] K3d is not working with different docker context #593

developer-guy · 2021-05-09T13:38:29Z

What did you do

I exited from running Docker daemon on my local machine, and I used a different context for it.

To get more about Docker Context, you can follow the link.

What did you expect to happen

It should create the cluster properly.

Screenshots or terminal output

I changed the Docker Context with the different daemon, then tried to create a k3d cluster.

$ docker context use google-cloud-ubuntu1604-xenial-vm
$ k3d cluster create
ERRO[0000] Failed to get nodes for cluster 'k3s-default'
INFO[0000] Prep: Network
ERRO[0000] Failed to list docker networks
ERRO[0000] Failed to check for duplicate networks
ERRO[0000] Failed to create cluster network
ERRO[0000] Failed Cluster Preparation: Failed Network Preparation: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
ERRO[0000] Failed to create cluster >>> Rolling Back
INFO[0000] Deleting cluster 'k3s-default'
ERRO[0000] Failed to get nodes for cluster 'k3s-default'
ERRO[0000] No nodes found for given cluster
FATA[0000] Cluster creation FAILED, also FAILED to rollback changes!

Which OS & Architecture

$ uname -a
Darwin xxxx 19.6.0 Darwin Kernel Version 19.6.0: Mon Apr 12 20:57:45 PDT 2021; root:xnu-6153.141.28.1~1/RELEASE_X86_64 x86_64

Which version of `k3d`

$ k3d version
k3d version v4.4.3
k3s version latest (default)

Which version of docker

$ docker version
Client: Docker Engine - Community
 Cloud integration: 1.0.12
 Version:           20.10.5
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        55c4c88
 Built:             Tue Mar  2 20:13:00 2021
 OS/Arch:           darwin/amd64
 Context:           google-cloud-ubuntu1604-xenial-vm
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.9
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       9d988398e7
  Built:            Fri May 15 00:24:07 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

$ docker info
Client:
 Context:    google-cloud-ubuntu1604-xenial-vm
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.6.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.10.0-27-generic
 Operating System: Ubuntu 16.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 3.611GiB
 Name: ubuntu-1804-bionic-vm1
 ID: 36FZ:RZFN:6TE3:F37P:NLHW:FAXC:HH3X:3QXK:ANWJ:OIAK:A647:AAML
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
  provider=google
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

The text was updated successfully, but these errors were encountered:

developer-guy · 2021-05-09T14:02:37Z

I would love to fix this issue, if you agree with me on this.

iwilltry42 · 2021-05-10T15:06:50Z

Hi @developer-guy , thanks for opening this issue!
We're using Docker CLI's connection helper for remote connections via SSH.
What connection mode are you using for the different context?

I'd love to accept a PR for this in any case :)

developer-guy · 2021-05-10T15:09:06Z

Hi @developer-guy , thanks for opening this issue!
We're using Docker CLI's connection helper for remote connections via SSH.
What connection mode are you using for the different context?

I'd love to accept a PR for this in any case :)

where can I find the connection mode for my context?

developer-guy · 2021-05-10T19:21:23Z

@iwilltry42 I did some experiment about the Docker Context, and updated the GetDockerClient method like the following:

func GetDockerClient() (*client.Client, error) {
	dockerCli, err := command.NewDockerCli(command.WithStandardStreams())
	if err != nil {
		return nil, err
	}

	err = dockerCli.Initialize(flags.NewClientOptions())
	if err != nil {
		return nil, err
	}

	currentContext := dockerCli.CurrentContext()

	var cli *client.Client

	dockerHost := os.Getenv("DOCKER_HOST")

	if strings.HasPrefix(dockerHost, "ssh://") {
		var helper *connhelper.ConnectionHelper

		helper, err = connhelper.GetConnectionHelper(dockerHost)
		if err != nil {
			return nil, err
		}
		cli, err = client.NewClientWithOpts(
			client.WithHost(helper.Host),
			client.WithDialContext(helper.Dialer),
			client.WithAPIVersionNegotiation(),
		)
	} else if currentContext != "default" {
		c, err := dockerCli.ContextStore().GetMetadata(currentContext)
		if err != nil {
			return nil, err
		}

		storageInfo := dockerCli.ContextStore().GetStorageInfo(currentContext)

		tlsFilesMap, err := dockerCli.ContextStore().ListTLSFiles(currentContext)
		if err != nil {
			return nil, err
		}

		var tlsFiles store.EndpointFiles
		var endpointDriver string
		if len(tlsFilesMap["docker"]) > 0 {
			endpointDriver = "docker"
		} else if len(tlsFilesMap["kubernetes"]) > 0 {
			endpointDriver = "kubernetes"
		}

		tlsFiles = tlsFilesMap[endpointDriver]

		endpointMetaBase := c.Endpoints[endpointDriver].(dockercontext.EndpointMetaBase)

		cli, err = client.NewClientWithOpts(
			client.FromEnv,
			client.WithAPIVersionNegotiation(),
			client.WithTLSClientConfig(
				path.Join(storageInfo.TLSPath, endpointDriver, tlsFiles[0]),
				path.Join(storageInfo.TLSPath, endpointDriver, tlsFiles[1]),
				path.Join(storageInfo.TLSPath, endpointDriver, tlsFiles[2])),
			client.WithHost(endpointMetaBase.Host),
		)
	} else {
		cli, err = client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
	}
	if err != nil {
		return nil, errors.WithStack(err)
	}

	return cli, err
}

it worked but it couldn't get the right API Server IP, so, I manually updated the kubeconfig file, maybe you can help me to resolve this problem, or you can give me a suggestion about what I need to do?

iwilltry42 · 2021-05-11T06:20:43Z

@developer-guy , can you create a Draft PR with your changes, please?
That'd make it easier to review, test and collaborate! 👍

developer-guy · 2021-05-11T08:16:02Z

@iwilltry42 there is a lots of updated packages in PR in vendor , so I don't want to create it, so I just wanted to share the code that I edited.

iwilltry42 · 2021-05-11T09:25:03Z

@developer-guy then I'd at least need the missing imports to reproduce what you did there.
Also, vendor changes shouldn't block you from creating a PR 👍

developer-guy · 2021-05-11T09:45:17Z

@developer-guy then I'd at least need the missing imports to reproduce what you did there.
Also, vendor changes shouldn't block you from creating a PR 👍

Thanks, @iwilltry42 for your understanding, I opened the PR btw.

developer-guy added the bug Something isn't working label May 9, 2021

iwilltry42 added this to the Backlog milestone May 10, 2021

This was referenced May 11, 2021

WIP: enable docker context support #600

Closed

[FEATURE] docker context support #601

Merged

allanger mentioned this issue Feb 23, 2022

METAL-3331 - Add database URL feature with template kloeckner-i/db-operator#131

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] K3d is not working with different docker context #593

[BUG] K3d is not working with different docker context #593

developer-guy commented May 9, 2021 •

edited

Loading

developer-guy commented May 9, 2021

iwilltry42 commented May 10, 2021

developer-guy commented May 10, 2021

developer-guy commented May 10, 2021 •

edited

Loading

iwilltry42 commented May 11, 2021 •

edited

Loading

developer-guy commented May 11, 2021

iwilltry42 commented May 11, 2021

developer-guy commented May 11, 2021 •

edited

Loading

[BUG] K3d is not working with different docker context #593

[BUG] K3d is not working with different docker context #593

Comments

developer-guy commented May 9, 2021 • edited Loading

What did you do

What did you expect to happen

Screenshots or terminal output

Which OS & Architecture

Which version of k3d

Which version of docker

developer-guy commented May 9, 2021

iwilltry42 commented May 10, 2021

developer-guy commented May 10, 2021

developer-guy commented May 10, 2021 • edited Loading

iwilltry42 commented May 11, 2021 • edited Loading

developer-guy commented May 11, 2021

iwilltry42 commented May 11, 2021

developer-guy commented May 11, 2021 • edited Loading

developer-guy commented May 9, 2021 •

edited

Loading

Which version of `k3d`

developer-guy commented May 10, 2021 •

edited

Loading

iwilltry42 commented May 11, 2021 •

edited

Loading

developer-guy commented May 11, 2021 •

edited

Loading