Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Per namespace rules for CoreDNS in Kubernetes? #6919

Open
rayjlinden opened this issue Oct 10, 2024 · 2 comments
Open

Per namespace rules for CoreDNS in Kubernetes? #6919

rayjlinden opened this issue Oct 10, 2024 · 2 comments
Labels

Comments

@rayjlinden
Copy link

We set up our testing environment in Kubernetes. But we wanted to have each "service" talk to one another via SSL inside the cluster. We got that all working using caddy - but we needed a FQDN in the cluster to make things work. I was able to do this with a rewrite rule:

rewrite stop name regex (.*).dev.mycompany.com.$ caddy.default.svc.cluster.local

This worked great!!!!

However, we want to allow each developer to have their own version of this in their own Kubernetes namespace in the cluster.

Which means in a different namespace I need the rewrite rule to be:
rewrite stop name regex (.*).dev.mycompany.com.$ caddy.mynamespace.svc.cluster.local

But there is only one CoreDNS for the entire cluster - so how the heck can I do this?

@akhilsingh-git
Copy link

You’ve discovered that using a rewrite rule in CoreDNS to map your external FQDNs to Kubernetes service hostnames works well. The challenge now is that you want to replicate this pattern for multiple developers, each with their own namespace, but still use only one CoreDNS instance for the entire cluster.

Key Points:
1. Problem Setup:
• Originally, you had a rewrite rule like:

rewrite stop name regex (.*).dev.mycompany.com. caddy.default.svc.cluster.local

This takes any query ending in .dev.mycompany.com and rewrites it to caddy.default.svc.cluster.local.

•	Now you want each developer to have their own namespace, e.g. mynamespace. The target should become:

rewrite stop name regex (.*).dev.mycompany.com. caddy.mynamespace.svc.cluster.local

•	Doing this for every developer namespace individually doesn’t scale, and you only have one CoreDNS instance for the cluster.

CoreDNS’s rewrite plugin supports using capture groups from the regex in the replacement string.
Let’s assume each developer’s namespace is represented by the subdomain. For example:
• alice.dev.mycompany.com should map to caddy.alice.svc.cluster.local
• bob.dev.mycompany.com should map to caddy.bob.svc.cluster.local
You can achieve this with:

rewrite stop name regex ([^.] ).dev.mycompany.com. caddy.{1}.svc.cluster.local

Explanation of the regex:
• ([^.] ) matches one or more characters that are not a dot. This captures the subdomain part before .dev.mycompany.com.
• .dev.mycompany.com. matches the literal .dev.mycompany.com. suffix.
The {1} in the replacement string refers to the first capturing group (the subdomain), allowing CoreDNS to rewrite the query dynamically based on the captured text.
For example, if the query is bob.dev.mycompany.com., then {1} = bob, resulting in caddy.bob.svc.cluster.local.

With this single rewrite directive, you can handle all developers’ namespaces. Any xxxxx.dev.mycompany.com. query is automatically rewritten to caddy.xxxxx.svc.cluster.local, where xxxxx corresponds to the namespace name. There’s no need to run multiple CoreDNS instances or maintain separate rewrite rules for each developer.

In Summary:
By using a capture group and referencing it in the rewritten name, you can keep a single, cluster-wide CoreDNS configuration that dynamically maps the requested subdomain to a corresponding namespace’s caddy service address.

@rayjlinden
Copy link
Author

rayjlinden commented Dec 17, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants