Default Kubernetes cluster templates in 3Engines Cloud Cloud[πŸ”—](#default-kubernetes-cluster-templates-in-brand-name-cloud "Permalink to this headline") ========================================================================================================================================================= In this article we shall list Kubernetes cluster templates available on 3Engines Cloud and explain the differences among them. What We Are Going To Cover[πŸ”—](#what-we-are-going-to-cover "Permalink to this headline") --------------------------------------------------------------------------------------- > * List available templates on your cloud > * Explain the difference between *calico* and *cilium* network drivers > * How to choose proper template > * Overview and benefits of *localstorage* templates > * Example of creating *localstorage* template using HMD and HMAD flavors Prerequisites[πŸ”—](#prerequisites "Permalink to this headline") ------------------------------------------------------------- No. 1 **Account** You need a 3Engines Cloud hosting account with access to the Horizon interface: . No. 2 **Private and public keys** To create a cluster, you will need an available SSH key pair. If you do not have one already, follow this article to create it in the 3Engines dashboard: [How to create key pair in 3Engines Dashboard on 3Engines Cloud](../cloud/How-to-create-key-pair-in-3Engines-Dashboard-on-3Engines-Cloud.html.md). No. 3 **Documentation for standard templates** Documentation for all **1.23.16** drivers is [here](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#v12316). Documentation for *localstorage* templates: > | | | > | --- | --- | > | k8s-stable-localstorage-1.21.5 | [Kubernetes release 1.21](https://kubernetes.io/blog/2021/04/08/kubernetes-1-21-release-announcement/) | > | k8s-stable-localstorage-1.22.5 | [Kubernetes release 1.22](https://kubernetes.io/blog/2021/08/04/kubernetes-1-22-release-announcement/) | > | k8s-stable-localstorage-1.23.5 | [Kubernetes release 1.23](https://kubernetes.io/blog/2021/12/07/kubernetes-1-23-release-announcement/) | No. 4 **How to create Kubernetes clusters** The general procedure is explained in [How to Create a Kubernetes Cluster Using 3Engines Cloud 3Engines Magnum](How-to-Create-a-Kubernetes-Cluster-Using-3Engines-Cloud-3Engines-Magnum.html.md). No. 5 **Using vGPU in Kubernetes clusters** If template name contains β€œvgpu”, this template can be used to create so-called β€œvGPU-first” clusters. To learn how to set up vGPU in Kubernetes clusters on 3Engines Cloud cloud, see [Deploying vGPU workloads on 3Engines Cloud Kubernetes](Deploying-vGPU-workloads-on-3Engines-Cloud-Kubernetes.html.md). Templates available on your cloud[πŸ”—](#templates-available-on-your-cloud "Permalink to this headline") ----------------------------------------------------------------------------------------------------- The exact number of available default Kubernetes cluster templates depends on the cloud you choose to work with. WAW4-1 : These are the default Kubernetes cluster templates on WAW4-1 cloud: ![waw4-1-cluster-templates-2.png](../_images/waw4-1-cluster-templates-2.png) WAW3-1 : These are the default Kubernetes cluster templates on WAW3-1 cloud: ![waw3-1-default.png](../_images/waw3-1-default.png) WAW3-2 : Default templates for WAW3-2 cloud: ![waw3-2-default_templates.png](../_images/waw3-2-default_templates.png) FRA1-2 : Default templates for FRA1-2 cloud: ![fra1-2-default-template.png](../_images/fra1-2-default-template.png) The converse is also true, you may want to select the cloud that you want to use according to the type of cluster that you would want to use. For instance, you would have to select WAW3-1 cloud if you wanted to use vGPU on your cluster. How to choose a proper template[πŸ”—](#how-to-choose-a-proper-template "Permalink to this headline") ------------------------------------------------------------------------------------------------- **Standard templates** Standard templates are general in nature and you can use them for any type of Kubernetes cluster. Each will produce a working Kubernetes cluster on 3Engines Cloud 3Engines Magnum hosting. The default network driver is *calico*. Template that does not specify calico, k8s-1.23.16-v1.0.3, and is identical to the template that does specify *calico* in its name. Both are placed in the left column in the following table: | calico | cilium | | --- | --- | | k8s-1.23.16-v1.0.3 | k8s-1.23.16-cilium-v1.0.3 | | k8s-1.23.16-calico-v1.0.3 | | Standard templates can also use vGPU hardware if available in the cloud. Using vGPU with Kubernetes clusters is explained in Prerequisite No. 5. **Templates with vGPU** | calico vGPU | cilium vGPU | | --- | --- | | k8s-1.23.16-vgpu-v1.0.0 | k8s-1.23.16-cilium-vgpu-v1.0.0 | | k8s-1.23.16-calico-vgpu-v1.0.0 | | Again, the templates in the left column are identical. If the application does not require a great many operations, then a standard template should be sufficient. You can also dig deeper and choose the template according to the the network plugin used. ### Network plugins for Kubernetes clusters[πŸ”—](#network-plugins-for-kubernetes-clusters "Permalink to this headline") Kubernetes cluster templates at 3Engines Cloud cloud use *calico* or *cilium* plugins for controlling network traffic. Both are [CNI](https://www.cncf.io/projects/kubernetes/) compliant. *Calico* is the default plugin, meaning that if the template name does not specify the plugin, the *calico* driver is used. If the template name specifies *cilium* then, of course, the *cilium* driver is used. ### Calico (the default)[πŸ”—](#calico-the-default "Permalink to this headline") [Calico](https://projectcalico.docs.tigera.io/about/about-calico) uses BGP protocol to move network packets towards IP addresses of the pods. *Calico* can be faster then its competitors but its most remarkable feature is support for *network policies*. With those, you can define which pods can send and receive traffic and also manage the security of the network. *Calico* can apply policies to multiple types of endpoints such as pods, virtual machines and host interfaces. It also supports cryptographics identity. *Calico* policies can be used on its own or together with the Kubernetes network policies. ### Cilium[πŸ”—](#cilium "Permalink to this headline") [Cilium](https://cilium.io/) is drawing its power from a technology called *eBPF*. It exposes programmable hooks to the network stack in Linux kernel. *eBPF* uses those hooks to reprogram Linux runtime behaviour without any loss of speed or safety. There also is no need to recompile Linux kernel in order to become aware of events in Kubernetes clusters. In essence, *eBPF* enables Linux to watch over Kubernetes and react appropriately. With *Cilium*, the relationships amongst various cluster parts are as follows: > * pods in the cluster (as well as the *Cilium* driver itself) are using *eBPF* instead of using Linux kernel directly, > * kubelet uses *Cilium* driver through the CNI compliance and > * the *Cilium* driver implements network policy, services and load balancing, flow and policy logging, as well as computing various metrics. Using *Cilium* especially makes sense if you require fine-grained security controls or need to reduce latency in large Kubernetes clusters. Overview and benefits of *localstorage* templates[πŸ”—](#overview-and-benefits-of-localstorage-templates "Permalink to this headline") ----------------------------------------------------------------------------------------------------------------------------------- Compared to standard templates, the *localstorage* templates may be a better fit for *resources intensive* apps. NVMe stands for *Nonvolatile Memory Express* and is a newer storage access and transport protocol for flash and solid-state drives (SSDs). *localstorage* templates provision the cluster with Virtual Machine flavors which have NVMe storage available. Each cluster contains an instance of **etcd** volume, which serves as its external database. Using NVMe storage will speed up access to **etcd** and it will, by definition, speed up cluster operations. Applications such as day trading, personal finances, AI and the similar, may have so many transactions that using *localstorage* templates may become a viable option. In WAW3-1 cloud, virtual machine flavors with NVMe have the prefix of HMD and they are resource-intensive: ``` 3Engines flavor list +--------------+--------+------+-----------+-------+ | Name | RAM | Disk | Ephemeral | VCPUs | +--------------+--------+------+-----------+-------+ | hmd.xlarge | 65536 | 200 | 0 | 8 | | hmd.medium | 16384 | 50 | 0 | 2 | | hmd.large | 32768 | 100 | 0 | 4 | ``` You would use an HMD flavor mainly for the master node(s) in the cluster. In WAW3-2 cloud, you would use flavors starting with HMAD instead of HMD. Example parameters to create a new cluster with localstorage and NVMe[πŸ”—](#example-parameters-to-create-a-new-cluster-with-localstorage-and-nvme "Permalink to this headline") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- For general discussion of parameters, see Prerequisite No. 4. What follows is a simplified example, geared to creation of cluster using *localstorage*. We shall use WAW3-1 with HMD flavors in the example but you can, of course, supply HMAD flavors for WAW3-2 and so on. The only deviation from the usual procedure is that it is mandatory to add label **etcd\_volume\_size=0** in the **Advanced** window. Without it, *localstorage* template won’t work. Start creating a cluster with the usual chain of commands **Container Infra** -> **Clusters** -> **+ Create New Cluster**. In the screenshot below, we selected *k8s-stable-localstorage-1.23.5* as our local storage template of choice, in mandatory field **Cluster Template**. For field **Keypair** use SSH key that you already have and if you do not have it yet, use Prerequisite No. 2 to obtain it. ![create_cluster_details.png](../_images/create_cluster_details.png) Let master nodes use one of the HMD flavors: ![create_cluster_size.png](../_images/create_cluster_size.png) Proceed to enter the usual parameters into the Network and Management windows. The last window, **Advanced**, is the place to add label **etcd\_volume\_size=0**. ![create_cluster_advanced.png](../_images/create_cluster_advanced.png) The result will be a formed cluster NVMe: ![create_cluster_working.png](../_images/create_cluster_working.png)