Skip to content

Scripts to create cluster and install kubernetes cluster.

Notifications You must be signed in to change notification settings

sumitmaji/kubernetes

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

kubernetes

Installation of kubernetes cluster in private cloud using kubeadm.

Configuration

The following table lists the configurable parameters of the kubernetes cluster install and their default values.

Parameter Description Default
CLUSTER_NAME Kubernetes cluster name cloud.com
MOUNT_PATH The mount path where kubernetes scripts and certificates are available. /root
DNS_DOMAIN The domains where vms are hosted. cloud.uat
CERTIFICATE_PATH The path where certificates are present. /etc/kubernetes/pki
SERVER_DNS The comma separated dns names where kubernetes master would be running, this also includes name of the kubernetes api service dns names, the dns name where ha proxy is running master.cloud.com..
SERVER_IP The comma separated list of all the ip addresses where master and ha proxy would be running (actual & virtual). This should also include the ip addess of kubernetes cluster api service ip. 11.0.0.1,..
HA_PROXY_PORT Port of HA Proxy. 6443
HA_PROXY_HOSTNAME IP/Hostname where HA Proxy running. 11.0.0.1
LOAD_BALANCER_URL Endpoint of HA Proxy. 11.0.0.1:6443
APP_HOST Hostname that would be put in ingress. master.cloud.com
API_SERVERS List of api servers which are used in creating certificates for ha proxy. 11.0.0.1:master.clud.com,..
OIDC_ISSUE_URL OpenID Connect issuer url. https://skmaji.auth0.com/
OIDC_CLIENT_ID OpenID Connect Application ID. Client ID
OIDC_USERNAME_CLAIM Field name in the ID Token for username claim. sub
OIDC_GROUPS_CLAIM Field name in the ID Token for group claim. http://localhost:8080/claims/groups

Installation steps:

  • Install Cluster in Digital Ocean
cd /root/kubernetes/install_cluster
./install_master_node.sh -e CLOUD
  • Install Cluster in Local Ubuntu Cluster
cd /root/kubernetes/install_cluster
./install_master_node.sh -e LOCAL
  • Install kubernetes:

    • Master:
    cd /root/kubernetes/install_k8s
    ./gok install kubernetes
    • Worker:
    cd /root/kubernetes/install_k8s
    ./gok install kubernetes-worker
    • To join a worker node:
    join
    • To remove a kubernetes setup:
    ./gok reset kubernetes
    • To create a new join token
    kubeadm token create print-join-command

When installation of master is completed you should see below components in kube-system namespace alt text

When you reboot the vms, the kubelet service may not run, you need to restart the kubelet.

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a

systemctl stop kubelet
syatemctl start kubelet

Or Execute below command

./gok start proxy

Other debugging steps

systemctl status kubelet
journalctl -u kubelet

Useful commands:

  • To change namespace
kcd name_space
  • Inorder to login via OpenId Connect username/password
alias kctl='kubectl --kubeconfig=/root/oauth.conf --token=$(python3 /root/kubernetes/install_k8s/kube-login/cli-auth.py)'

In order to use the above approach, you must install and run

  1. Ingress Instress ReadME
  2. Kubeauthentication service KubeAuth ReadME
  • To enable verbose(logging) of kubectl command
kubectl get pods --v 6

Useful commands

kubectl top nodes --v 6
kubectl get pod, svc, hpa -owide
watch -n1 !!
kubectl get componentstatus

Automation of build and deployment

Note: Ingress should be installed before this can begin.

  • Execute below command to install ci/cd pipeline
cd /root/kubernetes/install_k8s/
./setuUp-devops.sh 

This would install local docker registry, Git webhook, Docker webhook and Helm webhook

Architecture of the pipeline alt text

In order to test the pipeline use hlw repository, make a sample change and commit.

Application would be accessible in https://master.cloud.com:32028/hlw

Installing components

  1. Ingress Controller
  2. Cert Manager
  3. Keycloak
  4. Kube Login Webhook
  5. Devops: Githook, Dockerhook,Reghook
  6. Kubernetes Dashboard
  7. Kube Authentication
  8. Ldap: Link
  9. Kerberos: Link
  10. Ldap Client: Link
  11. Ldap Client2: Link
  12. Spinnaer
  13. Opensearch
  14. Prometheus-Grafana
  15. Oauth2 Proxy
  16. Fluentd
  • Add ca.crt and server.crt file in chrome browser, please refer link on how to add certificate. Add server.crt in Other People tab and ca.crt in Trusted Root Certificate Authority tab.
  • Add ip_address master.cloud.com to windows host file located in C:\Windows\System32\drivers\etc. e.g. 192.168.1.5 master.cloud.com >> host file.