Support for Kubernetes via Terraform and Manifests (#5721)

# Support for Kubernetes via Terraform and Manifests

Adding basic support for the following Kubernetes resources

- persistent volume
  - server
  - database
- persistent volume claim
  - server
  - database
- deployment
  - server
  - database
- ingress
  - server
- service
  - server
  - database
- secret
  - server

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
This commit is contained in:
Ciara Hatcher
2024-07-10 06:54:27 -05:00
committed by GitHub
parent ef5657c353
commit 43016db801
24 changed files with 849 additions and 2 deletions

View File

@ -0,0 +1,113 @@
# README
## Overview
This repository contains Kubernetes manifests and Terraform files to help you deploy and manage the TwentyCRM application. The files are located in the `packages/twenty-docker/k8s` directory.
## Prerequisites
Before using these files, ensure you have the following installed and configured on your system:
- Kubernetes cluster (e.g., Minikube, EKS, GKE)
- kubectl
- Terraform
- Docker
## Setup Instructions
### Step 1: Clone the Repository
Clone the repository to your local machine:
``` bash
git clone https://github.com/twentyhq/twenty.git
cd twentycrm/packages/twenty-docker/k8s
```
### Step 2: Customize the Manifests and Terraform Files
**Important:** These files require customization for your specific implementation. Update the placeholders and configurations according to your environment and requirements.
### Step 3: Deploy with Terraform
1. Navigate to the Terraform directory:
```bash
cd terraform
```
2. Initialize Terraform:
```bash
terraform init
```
3. Plan the deployment:
```bash
terraform plan
```
4. Apply the deployment:
```bash
terraform apply
```
## OR
### Step 3: Deploy with Kubernetes Manifests
1. Navigate to the Kubernetes manifests directory:
```bash
cd ../k8s
```
2. Create Server Secret
``` bash
kubectl create secret generic -n twentycrm tokens --from-literal accessToken=changeme --from-literal loginToken="changeme" --from-literal refreshToken="changeme" --from-literal fileToken="changeme"
```
3. Apply the manifests:
```bash
kubectl apply -f .
```
## Customization
### Kubernetes Manifests
- **Namespace:** Update the `namespace` in the manifests as needed.
- **Resource Limits:** Adjust the resource limits and requests according to your application's requirements.
- **Environment Variables:** Configure server tokens in the `Secret` command above.
### Terraform Files
- **Variables:** Update the variables in the `variables.tf` file to match your environment.
- **Locals:** Update the locals in the `main.tf` file to match your environment.
- **Providers:** Ensure the provider configurations (e.g., AWS, GCP) are correct for your setup.
- **Resources:** Modify the resource definitions as needed to fit your infrastructure.
## Troubleshooting
### Common Issues
- **Connectivity:** Ensure your Kubernetes cluster is accessible and configured correctly.
- **Permissions:** Verify that you have the necessary permissions to deploy resources in your cloud provider.
- **Resource Limits:** Adjust resource limits if you encounter issues related to insufficient resources.
### Logs and Debugging
- Use `kubectl logs` to check the logs of your Kubernetes pods.
- Use `terraform show` and `terraform state` to inspect your Terraform state and configurations.
## Conclusion
This setup provides a basic structure for deploying the TwentyCRM application using Kubernetes and Terraform. Ensure you thoroughly customize the manifests and Terraform files to suit your specific needs. For any issues or questions, please refer to the official documentation of Kubernetes and Terraform or seek support from your cloud provider.
---
Feel free to contribute and improve this repository by submitting pull requests or opening issues. Happy deploying!

View File

@ -0,0 +1,54 @@
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: twentycrm-db
name: twentycrm-db
namespace: twentycrm
spec:
progressDeadlineSeconds: 600
replicas: 1
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
selector:
matchLabels:
app: twentycrm-db
template:
metadata:
labels:
app: twentycrm-db
spec:
volumes:
- name: twentycrm-db-data
persistentVolumeClaim:
claimName: twentycrm-db-pvc
containers:
- env:
- name: POSTGRES_PASSWORD
value: "twenty"
- name: BITNAMI_DEBUG
value: "true"
- image: twentycrm/twenty-postgres:latest
imagePullPolicy: Always
name: twentycrm
ports:
- containerPort: 5432
name: tcp
protocol: TCP
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "1024Mi"
cpu: "1000m"
stdin: true
tty: true
volumeMounts:
- mountPath: /bitnami/postgresql
name: twentycrm-db-data
dnsPolicy: ClusterFirst
restartPolicy: Always

View File

@ -0,0 +1,82 @@
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: twentycrm-server
name: twentycrm-server
namespace: twentycrm
spec:
progressDeadlineSeconds: 600
replicas: 1
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
selector:
matchLabels:
app: twentycrm-server
template:
metadata:
labels:
app: twentycrm-server
spec:
volumes:
- name: twentycrm-server-data
persistentVolumeClaim:
claimName: twentycrm-server-pvc
containers:
- env:
- name: PORT
value: 3000
- name: SERVER_URL
value: "https://crm.example.com:443"
- name: PG_DATABASE_URL
value: "postgres://twenty:twenty@twenty-db.twentycrm.svc.cluster.local/default"
- name: ENABLE_DB_MIGRATIONS
value: "true"
- name: SIGN_IN_PREFILLED
value: "true"
- name: STORAGE_TYPE
value: "local"
- name: ACCESS_TOKEN_SECRET
valueFrom:
secretKeyRef:
name: tokens
key: accessToken
- name: LOGIN_TOKEN_SECRET
valueFrom:
secretKeyRef:
name: tokens
key: loginToken
- name: REFRESH_TOKEN_SECRET
valueFrom:
secretKeyRef:
name: tokens
key: refreshToken
- name: FILE_TOKEN_SECRET
valueFrom:
secretKeyRef:
name: tokens
key: fileToken
- image: twentycrm/twenty:latest
imagePullPolicy: Always
name: twentycrm
ports:
- containerPort: 3000
name: http-tcp
protocol: TCP
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "1024Mi"
cpu: "1000m"
stdin: true
tty: true
volumeMounts:
- mountPath: /app/.local-storage
name: twentycrm-server-data
dnsPolicy: ClusterFirst
restartPolicy: Always

View File

@ -0,0 +1,24 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: twentycrm
namespace: twentycrm
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "X-Forwarded-For $http_x_forwarded_for";
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
spec:
ingressClassName: nginx
rules:
- host: crm.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: twentycrm-server
port:
name: http-tcp

View File

@ -0,0 +1,11 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: twentycrm-db-pv
spec:
storageClassName: default
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain

View File

@ -0,0 +1,12 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: twentycrm-server-pv
namespace: twentycrm
spec:
storageClassName: default
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain

View File

@ -0,0 +1,13 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: twentycrm-db-pvc
namespace: twentycrm
spec:
storageClassName: default
volumeName: twentycrm-db-pv
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi

View File

@ -0,0 +1,13 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: twentycrm-server-pvc
namespace: twentycrm
spec:
storageClassName: default
volumeName: twentycrm-server-pv
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi

View File

@ -0,0 +1,18 @@
apiVersion: v1
kind: Service
metadata:
name: twentycrm-db
namespace: twentycrm
spec:
internalTrafficPolicy: Cluster
ports:
- port: 5432
protocol: TCP
targetPort: 5432
selector:
app: twentycrm-db
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
type: ClusterIP

View File

@ -0,0 +1,19 @@
apiVersion: v1
kind: Service
metadata:
name: twentycrm-server
namespace: twentycrm
spec:
internalTrafficPolicy: Cluster
ports:
- name: http-tcp
port: 3000
protocol: TCP
targetPort: 3000
selector:
app: twentycrm-server
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 10800
type: ClusterIP

View File

@ -0,0 +1,90 @@
resource "kubernetes_deployment" "twentycrm_db" {
metadata {
name = "${local.twentycrm_app_name}-db"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
labels = {
app = "${local.twentycrm_app_name}-db"
}
}
spec {
replicas = 1
selector {
match_labels = {
app = "${local.twentycrm_app_name}-db"
}
}
strategy {
type = "RollingUpdate"
rolling_update {
max_surge = "1"
max_unavailable = "1"
}
}
template {
metadata {
labels = {
app = "${local.twentycrm_app_name}-db"
}
}
spec {
# security_context {
# fs_group = 0
# }
container {
image = local.twentycrm_db_image
name = local.twentycrm_app_name
stdin = true
tty = true
security_context {
allow_privilege_escalation = true
}
env {
name = "POSTGRES_PASSWORD"
value = "twenty"
}
env {
name = "BITNAMI_DEBUG"
value = true
}
port {
container_port = 5432
protocol = "TCP"
}
resources {
requests = {
cpu = "250m"
memory = "256Mi"
}
limits = {
cpu = "1000m"
memory = "1024Mi"
}
}
volume_mount {
name = "nfs-twentycrm-db-data"
mount_path = "/bitnami/postgresql"
}
}
volume {
name = "nfs-twentycrm-db-data"
persistent_volume_claim {
claim_name = "nfs-twentycrm-db-data-pvc"
}
}
dns_policy = "ClusterFirst"
restart_policy = "Always"
}
}
}
}

View File

@ -0,0 +1,169 @@
resource "kubernetes_deployment" "twentycrm_server" {
metadata {
name = "${local.twentycrm_app_name}-server"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
labels = {
app = "${local.twentycrm_app_name}-server"
}
}
spec {
replicas = 1
selector {
match_labels = {
app = "${local.twentycrm_app_name}-server"
}
}
strategy {
type = "RollingUpdate"
rolling_update {
max_surge = "1"
max_unavailable = "1"
}
}
template {
metadata {
labels = {
app = "${local.twentycrm_app_name}-server"
}
}
spec {
container {
image = local.twentycrm_server_image
name = local.twentycrm_app_name
stdin = true
tty = true
security_context {
allow_privilege_escalation = true
privileged = true
run_as_user = 1000
}
env {
name = "PORT"
value = "3000"
}
env {
name = "DEBUG_MODE"
value = false
}
env {
name = "SERVER_URL"
value = "https://crm.example.com:443"
}
env {
name = "FRONT_BASE_URL"
value = "https://crm.example.com:443"
}
env {
name = "BACKEND_SERVER_URL"
value = "https://crm.example.com:443"
}
env {
name = "PG_DATABASE_URL"
value = "postgres://twenty:twenty@twentycrm-db.twentycrm.svc.cluster.local/default"
}
env {
name = "ENABLE_DB_MIGRATIONS"
value = "true"
}
env {
name = "SIGN_IN_PREFILLED"
value = "true"
}
env {
name = "STORAGE_TYPE"
value = "local"
}
env {
name = "ACCESS_TOKEN_SECRET"
value_from {
secret_key_ref {
name = "tokens"
key = "accessToken"
}
}
}
env {
name = "LOGIN_TOKEN_SECRET"
value_from {
secret_key_ref {
name = "tokens"
key = "loginToken"
}
}
}
env {
name = "REFRESH_TOKEN_SECRET"
value_from {
secret_key_ref {
name = "tokens"
key = "refreshToken"
}
}
}
env {
name = "FILE_TOKEN_SECRET"
value_from {
secret_key_ref {
name = "tokens"
key = "fileToken"
}
}
}
port {
container_port = 3000
protocol = "TCP"
}
resources {
requests = {
cpu = "250m"
memory = "256Mi"
}
limits = {
cpu = "1000m"
memory = "1024Mi"
}
}
volume_mount {
name = "nfs-twentycrm-server-data"
mount_path = "/app/.local-storage"
}
}
volume {
name = "nfs-twentycrm-server-data"
persistent_volume_claim {
claim_name = "nfs-twentycrm-server-data-pvc"
}
}
dns_policy = "ClusterFirst"
restart_policy = "Always"
}
}
}
depends_on = [
kubernetes_deployment.twentycrm_db,
kubernetes_secret.twentycrm_tokens
]
}

View File

@ -0,0 +1,30 @@
resource "kubernetes_ingress" "twentycrm" {
wait_for_load_balancer = true
metadata {
name = "${local.twentycrm_app_name}-ingress"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
annotations = {
"kubernetes.io/ingress.class" = "nginx"
"nginx.ingress.kubernetes.io/configuration-snippet" = <<EOF
more_set_headers "X-Forwarded-For $http_x_forwarded_for";
EOF
"nginx.ingress.kubernetes.io/force-ssl-redirect" = "false"
"nginx.ingress.kubernetes.io/backend-protocol" = "HTTP"
}
}
spec {
ingress_class_name = "nginx"
rule {
host = local.twentycrm_app_hostname
http {
path {
path = "/*"
backend {
service_name = kubernetes_service.twentycrm_server.metadata.0.name
service_port = kubernetes_service.twentycrm_server.spec.0.port.0.port
}
}
}
}
}
}

View File

@ -0,0 +1,36 @@
#############
# Providers #
#############
provider "kubernetes" {
config_path = "~/.kube/config"
}
#################
# Global Locals #
#################
locals {
twentycrm_app_name = "twentycrm"
twentycrm_app_hostname = "crm.example.com"
twentycrm_server_image = "twentycrm/twenty:v0.10.4"
twentycrm_db_image = "twentycrm/twenty-postgres:v0.10.4"
twentycrm_db_pv_path = "/path/to/mystorage"
twentycrm_db_pv_capacity = "10Gi"
twentycrm_db_pvc_requests = "10Gi"
twentycrm_server_pv_path = "/path/to/mystorage"
twentycrm_server_pv_capacity = "10Gi"
twentycrm_server_pvc_requests = "10Gi"
}
####################
# Terraform Config #
####################
terraform {
required_version = ">= 1.7.4"
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
version = ">= 2.23.0"
}
}
}

View File

@ -0,0 +1,9 @@
resource "kubernetes_namespace" "twentycrm" {
metadata {
annotations = {
name = "twentycrm"
}
name = "twentycrm"
}
}

View File

@ -0,0 +1,19 @@
resource "kubernetes_persistent_volume" "db" {
metadata {
name = "${local.twentycrm_app_name}-db-pv"
}
spec {
storage_class_name = "default"
capacity = {
storage = local.twentycrm_db_pv_capacity
}
access_modes = ["ReadWriteOnce"]
# refer to Terraform Docs for your specific implementation requirements
# https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/persistent_volume
persistent_volume_source {
local {
path = local.twentycrm_db_pv_path
}
}
}
}

View File

@ -0,0 +1,19 @@
resource "kubernetes_persistent_volume" "server" {
metadata {
name = "${local.twentycrm_app_name}-server-pv"
}
spec {
storage_class_name = "default"
capacity = {
storage = local.twentycrm_server_pv_capacity
}
access_modes = ["ReadWriteOnce"]
# refer to Terraform Docs for your specific implementation requirements
# https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/persistent_volume
persistent_volume_source {
local {
path = local.twentycrm_server_pv_path
}
}
}
}

View File

@ -0,0 +1,15 @@
resource "kubernetes_persistent_volume_claim" "db" {
metadata {
name = "${local.twentycrm_app_name}-db-pvc"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
}
spec {
access_modes = ["ReadWriteOnce"]
resources {
requests = {
storage = local.twentycrm_db_pvc_requests
}
}
volume_name = kubernetes_persistent_volume.db.metadata.0.name
}
}

View File

@ -0,0 +1,15 @@
resource "kubernetes_persistent_volume_claim" "server" {
metadata {
name = "${local.twentycrm_app_name}-server-pvc"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
}
spec {
access_modes = ["ReadWriteOnce"]
resources {
requests = {
storage = local.twentycrm_server_pvc_requests
}
}
volume_name = kubernetes_persistent_volume.server.metadata.0.name
}
}

View File

@ -0,0 +1,15 @@
resource "kubernetes_secret" "twentycrm_tokens" {
metadata {
name = "tokens"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
}
data = {
accessToken = var.twentycrm_token_accessToken
loginToken = var.twentycrm_token_loginToken
refreshToken = var.twentycrm_token_refreshToken
fileToken = var.twentycrm_token_fileToken
}
# type = "kubernetes.io/basic-auth"
}

View File

@ -0,0 +1,18 @@
resource "kubernetes_service" "twentycrm_db" {
metadata {
name = "${local.twentycrm_app_name}-db"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
}
spec {
selector = {
app = "${local.twentycrm_app_name}-db"
}
session_affinity = "ClientIP"
port {
port = 5432
target_port = 5432
}
type = "ClusterIP"
}
}

View File

@ -0,0 +1,19 @@
resource "kubernetes_service" "twentycrm_server" {
metadata {
name = "${local.twentycrm_app_name}-server"
namespace = kubernetes_namespace.twentycrm.metadata.0.name
}
spec {
selector = {
app = "${local.twentycrm_app_name}-server"
}
session_affinity = "ClientIP"
port {
name = "http-tcp"
port = 3000
target_port = 3000
}
type = "ClusterIP"
}
}

View File

@ -0,0 +1,24 @@
variable "twentycrm_token_accessToken" {
type = string
description = "TwentyCRM access Token"
}
variable "twentycrm_token_loginToken" {
type = string
description = "TwentyCRM login Token"
}
variable "twentycrm_token_refreshToken" {
type = string
description = "TwentyCRM refresh Token"
}
variable "twentycrm_token_fileToken" {
type = string
description = "TwentyCRM file Token"
}
variable "twentycrm_pgdb_admin_password" {
type = string
description = "TwentyCRM password for postgres database"
}

View File

@ -6,21 +6,31 @@ image: /images/user-guide/notes/notes_header.png
<ArticleWarning>
This document is maintained by the community. It might contain issues.
Feel free to join our discord if you need assistance.
</ArticleWarning>
## Kubernetes via Terraform and Manifests
Community-led documentation for Kubernetes deployment is available (here)[https://github.com/twentyhq/twenty/tree/main/packages/twenty-docker/k8s]
## Render
Community-led, might not be up to date
[![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/twentyhq/twenty)
## RepoCloud
## RepoCloud
Community-led, might not be up to date
[![Deploy on RepoCloud](https://d16t0pc4846x52.cloudfront.net/deploy.png)](https://repocloud.io/details/?app_id=259)
## Azure Container Apps
Community-led, might not be up to date
### About
Hosts Twenty CRM using Azure Container Apps.