![\"kubernetes-logo\"](\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/kubernetes-logo.png\"){kind=logo}
<\/span><\/p>\nIn this blog, we will learn how to setup Kubernetes cluster on servers running on CentOS (Bare-metal installation) as well as deploy add-on services such as DNS and Kubernetes Dashboard.<\/span><\/p>\n If you are new to Kubernetes cluster\u00a0and want to understand its architecture then you can through the blog on the Introduction on Kubernetes<\/a>. \u00a0So, Let\u2019s get started with the installation.\u00a0<\/span><\/p>\n Prerequisites:<\/strong><\/p>\n You need at least 2 servers for setting\u00a0Kubernetes<\/span>\u00a0cluster. For this blog, we are using three servers to form\u00a0Kubernetes<\/span>\u00a0cluster. Make sure that each of these servers has at least 1 core and 2 GB memory.<\/span><\/p>\n Master \u00a0 172.16.0.1 Cluster Configuration:<\/strong><\/p>\n Step 1: Create a Repo on all Host i.e. Master, Minion1, and Minion2.<\/b><\/p>\n [js]<\/p>\n vim \/etc\/yum.repos.d\/virt7-docker-common-release.repo<\/p>\n [virt7-docker-common-release] [\/js]<\/p>\n Step 2: Installing Kubernetes, etcd and flannel. <\/b><\/p>\n [js]<\/p>\n yum -y install –enablerepo=virt7-docker-common-release kubernetes etcd flannel<\/p>\n [\/js]<\/p>\n The above command also installs Docker and cadvisor.<\/span>\u00a0<\/strong><\/strong><\/p>\n Step 3: Next Configure\u00a0<\/b>Kubernetes<\/b><\/span>\u00a0Components.<\/b><\/p>\n Kubernetes Common Configuration (On All Nodes)<\/b><\/p>\n Let\u2019s get started with the common configuration for\u00a0Kubernetes<\/span>\u00a0cluster. This configuration should be done on all the host i.e. Master and Minions.\u00a0<\/span><\/p>\n [js]<\/p>\n vi \/etc\/kubernetes\/config<\/p>\n # Comma separated list of nodes running etcd cluster [\/js]<\/p>\n ETCD Configuration (On Master)<\/b><\/p>\n Next, we need to configure etcd for\u00a0Kubernetes<\/span>\u00a0cluster. Etcd configuration is stored in \/etc\/etcd\/etcd.conf<\/span><\/p>\n [js]<\/p>\n vi \/etc\/etcd\/etcd.conf<\/p>\n #[member] ETCD_LISTEN_CLIENT_URLS="http:\/\/0.0.0.0:2379" [\/js]<\/p>\n All the configuration data of\u00a0Kubernetes<\/span>\u00a0is stored in etcd. To increase security, etcd\u00a0can be bind to the private IP address of the master node. Now,\u00a0etcd endpoint can only be accessed from the private Subnet.<\/span><\/p>\n API Server Configuration (On Master)<\/b><\/p>\n API Server handles the REST operations and acts as a front-end to the cluster\u2019s shared state. API Server Configuration is stored at \/etc\/kubernetes\/apiserver.<\/span><\/p>\n Kubernetes uses certificates to authenticate API request. Before configuring API server, we need to generate certificates that can be used for authentication. Kubernetes provides ready made scripts for generating these certificates which can be found <\/span>here<\/span><\/a>.<\/span><\/p>\n Download this script and update line number 30 in the file.<\/span><\/p>\n [js]<\/p>\n # update the below line with the group that exists on Kubernetes Master. [\/js]<\/p>\n Now, run the script with following parameters to create certificates:<\/span><\/p>\n [js]<\/p>\n bash make-ca-cert.sh "172.16.0.1" "IP:172.16.0.1,IP:10.254.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local"<\/p>\n [\/js]<\/p>\n Where 172.16.0.1 is the IP of the master server, 10.254.0.1 is the IP of\u00a0Kubernetes<\/span>\u00a0service. Now, we can configure API server:\u00a0<\/span><\/p>\n [js]<\/p>\n vi \/etc\/kubernetes\/apiserver<\/p>\n # Bind kube API server to this IP [\/js]<\/p>\n Note: Please make sure service cluster IP range doesn\u2019t overlap with infrastructure Subnet IP range.<\/b><\/p>\n Controller Manager Configuration (On Master)<\/b><\/p>\n [js]<\/p>\n vi \/etc\/kubernetes\/controller-manager<\/p>\n # Add your own! [\/js]<\/p>\n Kubelet Configuration (On Minions)<\/b><\/p>\n Kubelet is a node\/minion agent that runs pods and make sure that it is healthy. It also communicates pod details to Kubernetes Master. Kubelet configuration is stored in \/etc\/kubernetes\/kubelet<\/span><\/p>\n [On Minion1]<\/span><\/p>\n [js] # kubelet bind ip address(Provide private ip of minion) [\/js]<\/p>\n [On Minion2]<\/span><\/p>\n [js] # kubelet bind ip address(Provide private ip of minion) [\/js]<\/p>\n Before Configuring Flannel for\u00a0Kubernetes<\/span>\u00a0cluster, we need to create network configuration for Flannel in etcd. <\/span><\/p>\n So start the etcd node on the\u00a0master using the\u00a0following command:<\/span><\/p>\n [js] Create a new key in etcd to store Flannel configuration using the\u00a0following command:<\/span><\/p>\n [js] Next, we need to define the network configuration for Flannel:<\/span><\/p>\n [js]<\/p>\n etcdctl mk \/kube-centos\/network\/config "{ \\"Network\\": \\"172.30.0.0\/16\\", \\"SubnetLen\\": 24, \\"Backend\\": { \\"Type\\": \\"vxlan\\" } }"<\/p>\n [\/js]<\/p>\n The above command allocates the 172.30.0.0\/16 subnet to the Flannel network. A flannel subnet of CIDR 24 is allocated to each server in\u00a0Kubernetes<\/span>\u00a0cluster. <\/span><\/p>\n Note: Please make sure Flannel subnet doesn\u2019t overlap with infrastructure subnet or service cluster IP range.<\/b><\/p>\n Flannel Configuration (On All Nodes)<\/b><\/p>\n Kubernetes uses Flannel to build an overlay network for inter-pod communication. Flannel configuration is stored in \/etc\/sysconfig\/flanneld<\/span><\/p>\n [js]<\/p>\n vi \/etc\/sysconfig\/flanneld<\/p>\n # etcd URL location. \u00a0Point this to the server where etcd runs [\/js]<\/p>\n Step 4: Start services on Master and Minion<\/b><\/p>\n On Master<\/strong><\/p>\n [js]<\/p>\n systemctl enable kube-apiserver [\/js]<\/p>\n On Minions<\/strong><\/p>\n [js]<\/p>\n systemctl enable kube-proxy [\/js]<\/p>\n Note: <\/strong>In each host, make sure that IP address allocated to Docker0 is the first IP address in the Flannel subnet, otherwise your cluster won’t work properly. To check this, use “ifconfig” command. \u00a0<\/strong><\/p>\n Step 5: Check Status of all Services<\/b><\/p>\n Make sure etcd, kube-apiserver, kube-controller-manager, kube-scheduler, and flanneld is running on Master and kube-proxy, kubelet, flanneld, and docker is running on the slave.<\/span>\u00a0<\/strong><\/strong><\/p>\n Deploying Addons in Kubernetes<\/b><\/p>\n Configuring DNS for Kubernetes Cluster<\/b><\/p>\n To enable service name discovery in our\u00a0Kubernetes\u00a0cluster, we need to configure DNS for our Kubernetes cluster. To do so, we need to deploy DNS pod and service in our cluster and configure Kubelet to resolve all DNS queries from this DNS service (local DNS).<\/span><\/p>\n You can download DNS Replication Controller and Service YAML from my repository<\/a>. You can also download the latest version of DNS from official Kubernetes repository (kubernetes\/cluster\/addons\/dns).<\/span><\/p>\n Next, use the following command to create a replication controller and service:\u00a0<\/span><\/p>\n [js]<\/p>\n Kubectl create -f DNS\/skydns-rc.yaml [\/js]<\/p>\n Note: Make sure you have entered correct cluster IP for DNS service in skydns-svc.yaml<\/strong><\/p>\n For this blog, we have used 10.254.3.100 as DNS service IP.<\/p>\n Now, configure Kubelet in all Minion to resolve all DNS queries from our local DNS service.<\/span><\/p>\n [js]<\/p>\n vim \/etc\/kubernetes\/kubelet<\/p>\n # Add your own! [\/js]<\/p>\n Restart kubelet on all Minions to load the new kubelet configuration.<\/span><\/p>\n [js] Configuring Dashboard for Kubernetes Cluster<\/b><\/p>\n Kubernetes Dashboard provides User Interface through which we can manage Kubernetes work units. We can create, delete or edit all work unit from Dashboard. <\/span><\/p>\n
\nMinion1 172.16.0.2
\nMinion2 172.16.0.3<\/p>\n\n
\nname=virt7-docker-common-release
\nbaseurl=http:\/\/cbs.centos.org\/repos\/virt7-docker-common-release\/x86_64\/os\/
\ngpgcheck=0<\/p>\n
\nKUBE_ETCD_SERVERS="–etcd-servers=http:\/\/172.16.0.1:2379"
\n# Logging will be stored in system journal
\nKUBE_LOGTOSTDERR="–logtostderr=true"
\n# Journal message level, 0 is debug
\nKUBE_LOG_LEVEL="–v=0"
\n# Should this cluster be allowed to run privileged docker containers
\nKUBE_ALLOW_PRIV="–allow-privileged=false"
\n# Api-server endpoint used in scheduler and controller-manager
\nKUBE_MASTER="–master=http:\/\/172.16.0.1:8080"<\/p>\n
\nETCD_NAME=default
\nETCD_DATA_DIR="\/var\/lib\/etcd\/default.etcd"<\/p>\n
\n#[cluster]
\nETCD_ADVERTISE_CLIENT_URLS="http:\/\/0.0.0.0:2379"<\/p>\n
\n\/* Use the user group with which you are planning to run kubernetes services *\/
\ncert_group=${CERT_GROUP:-kube}<\/p>\n
\nKUBE_API_ADDRESS="–address=0.0.0.0"
\n# Port that kube api server listens to.
\nKUBE_API_PORT="–port=8080"
\n# Port kubelet listen on
\nKUBELET_PORT="–kubelet-port=10250"
\n# Address range to use for services(Work unit of Kubernetes)
\nKUBE_SERVICE_ADDRESSES="–service-cluster-ip-range=10.254.0.0\/16"
\n# default admission control policies
\nKUBE_ADMISSION_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
\n# Add your own!
\nKUBE_API_ARGS="–client-ca-file=\/srv\/kubernetes\/ca.crt –tls-cert-file=\/srv\/kubernetes\/server.cert –tls-private-key-file=\/srv\/kubernetes\/server.key"<\/p>\n
\nKUBE_CONTROLLER_MANAGER_ARGS="–root-ca-file=\/srv\/kubernetes\/ca.crt –service-account-private-key-file=\/srv\/kubernetes\/server.key"<\/p>\n
\nvi \/etc\/kubernetes\/kubelet<\/p>\n
\nKUBELET_ADDRESS="–address=0.0.0.0"
\n# port on which kubelet listen
\nKUBELET_PORT="–port=10250"
\n# leave this blank to use the hostname of server
\nKUBELET_HOSTNAME="–hostname-override=172.16.0.2"
\n# Location of the api-server
\nKUBELET_API_SERVER="–api-servers=http:\/\/172.16.0.1:8080"
\n# Add your own!
\nKUBELET_ARGS=""<\/p>\n
\nvi \/etc\/kubernetes\/kubelet<\/p>\n
\nKUBELET_ADDRESS="–address=0.0.0.0"
\n# port on which kubelet listen
\nKUBELET_PORT="–port=10250"
\n# leave this blank to use the hostname of server
\nKUBELET_HOSTNAME="–hostname-override=172.16.0.3"
\n# Location of the api-server
\nKUBELET_API_SERVER="–api-servers=http:\/\/172.16.0.1:8080"
\n# Add your own!
\nKUBELET_ARGS=""<\/p>\n
\nsystemctl start etcd
\n[\/js]<\/p>\n
\netcdctl mkdir \/kube-centos\/network
\n[\/js]<\/p>\n
\nFLANNEL_ETCD="http:\/\/172.16.0.1:2379"
\n# etcd config key. \u00a0This is the configuration key that flannel queries
\n# For address range assignment
\nFLANNEL_ETCD_PREFIX="\/kube-centos\/network"
\n# Any additional options that you want to pass
\nFLANNEL_OPTIONS=""<\/p>\n
\nsystemctl start kube-apiserver
\nsystemctl enable kube-controller-manager
\nsystemctl start kube-controller-manager
\nsystemctl start kube-scheduler
\nsystemctl start kube-scheduler
\nsystemctl enable flanneld
\nsystemctl start flanneld<\/p>\n
\nsystemctl start kube-proxy
\nsystemctl enable kubelet
\nsystemctl start kubelet
\nsystemctl enable flanneld
\nsystemctl start flanneld
\nsystemctl enable docker
\nsystemctl start docker<\/p>\n![\"blog_random\"](\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/blog_random.png\")
<\/p>\n
\nKubectl create -f DNS\/skydns-svc.yaml<\/p>\n
\nKUBELET_ARGS="–cluster-dns=10.254.3.100 –cluster-domain=cluster.local"<\/p>\n
\nsystemctl restart kubelet
\n[\/js]<\/p>\n
![\"kube-dash\"](\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/kube-dash1.png\")
<\/p>\n![\"kube1\"](\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/kube1.png\")
<\/p>\n