Migrate to a monorepo structure (#2909)

This commit is contained in:
Charles Bochet
2023-12-10 18:10:54 +01:00
committed by GitHub
parent a70a9281eb
commit 5bdca9de6c
2304 changed files with 37152 additions and 25869 deletions

View File

@ -0,0 +1,3 @@
{
"position": 4
}

View File

@ -0,0 +1,4 @@
{
"label": "Basics",
"position": 1
}

View File

@ -0,0 +1,41 @@
---
title: Custom Objects
sidebar_position: 4
sidebar_custom_props:
icon: TbAugmentedReality
---
Objects are structures that allow you to store data (records, attributes, and values) specific to an organization. Twenty provides both standard and custom objects.
Standard objects are in-built objects with a set of attributes available for all users. Examples of standard objects in Twenty include Company and Person. Standard objects have standard fields that are also available for all Twenty users, like Company.displayName.
Custom objects are objects that you can create to store information that is unique to your organization. They are not built-in; members of your workspace can create and customize custom objects to hold information that standard objects aren't suitable for.
## High-level schema
<div style={{textAlign: 'center'}}>
<img src="/img/contributor/custom-object-schema.jpeg" alt="High level schema" />
</div>
<br/>
## How it works
Custom objects come from metadata tables that determine the shape, name, and type of the objects. All this information is present in the metadata schema database, consisting of tables:
- **DataSource**: Details where the data is present.
- **Object**: Describes the object and links to a DataSource.
- **Field**: Outlines an Object's fields and connects to the Object.
To add a custom object, the workspaceMember will query the /metadata API. This updates the metadata accordingly and computes a GraphQL schema based on the metadata, storing it in a GQL cache for later use.
<div style={{textAlign: 'center'}}>
<img src="/img/contributor/add-custom-objects.jpeg" alt="Query the /metadata API to add custom objects" />
</div>
<br/>
To fetch data, the process involves making queries through the /graphql endpoint and passing them through the Query Resolver.
<div style={{textAlign: 'center'}}>
<img src="/img/contributor/custom-objects-schema.png" alt="Query the /graphql endpoint to fetch data" />
</div>

View File

@ -0,0 +1,132 @@
---
title: Folder Architecture
sidebar_position: 2
description: A detailed look into our server folder architecture
sidebar_custom_props:
icon: TbFolder
---
The backend directory structure is as follows:
```
server
└───ability
└───constants
└───core
└───database
└───decorators
└───filters
└───guards
└───health
└───integrations
└───metadata
└───workspace
└───utils
```
## Ability
Defines permissions and includes handlers for each entity.
## Decorators
Defines custom decorators in NestJS for added functionality.
See [custom decorators](https://docs.nestjs.com/custom-decorators) for more details.
## Filters
Includes exception filters to handle exceptions that might occur in GraphQL endpoints.
## Guards
See [guards](https://docs.nestjs.com/guards) for more details.
## Health
Includes a publicly available REST API (healthz) that returns a JSON to confirm whether the database is working as expected.
## Metadata
Defines custom objects and makes available a GraphQL API (graphql/metadata).
## Workspace
Generates and serves custom GraphQL schema based on the metadata.
### Workspace Directory Structure
```
workspace
└───workspace-schema-builder
└───factories
└───graphql-types
└───database
└───interfaces
└───object-definitions
└───services
└───storage
└───utils
└───workspace-resolver-builder
└───factories
└───interfaces
└───workspace-query-builder
└───factories
└───interfaces
└───workspace-query-runner
└───interfaces
└───utils
└───workspace-datasource
└───workspace-manager
└───workspace-migration-runner
└───utils
└───workspace.module.ts
└───workspace.factory.spec.ts
└───workspace.factory.ts
```
The root of the workspace directory includes the `workspace.factory.ts`, a file containing the `createGraphQLSchema` function. This function generates workspace-specific schema by using the metadata to tailor a schema for individual workspaces. By separating the schema and resolver construction, we use the `makeExecutableSchema` function, which combines these discrete elements.
This strategy is not just about organization, but also helps with optimization, such as caching generated type definitions to enhance performance and scalability.
### Workspace Schema builder
Generates the GraphQL schema, and includes:
#### Factories:
Specialised constructors to generate GraphQL-related constructs.
- The type.factory translates field metadata into GraphQL types using `TypeMapperService`.
- The type-definition.factory creates GraphQL input or output objects derived from `objectMetadata`.
#### GraphQL Types
Includes enumerations, inputs, objects, and scalars, and serves as the building blocks for the schema construction.
#### Interfaces and Object Definitions
Contains the blueprints for GraphQL entities, and includes both predefined and custom types like `MONEY` or `URL`.
#### Services
Contains the service responsible for associating FieldMetadataType with its appropriate GraphQL scalar or query modifiers.
#### Storage
Includes the `TypeDefinitionsStorage` class that contains reusable type definitions, preventing duplication of GraphQL types.
### Workspace Resolver Builder
Creates resolver functions for querying and mutatating the GraphQL schema.
Each factory in this directory is responsible for producing a distinct resolver type, such as the `FindManyResolverFactory`, designed for adaptable application across various tables.
### Workspace Query Builder
Includes factories that generate `pg_graphql` queries.
### Workspace Query Runner
Runs the generated queries on the database and parses the result.

View File

@ -0,0 +1,35 @@
---
title: Overview
sidebar_position: 0
sidebar_custom_props:
icon: TbEyeglass
---
Twenty primarily uses NestJS for the backend.
Prisma was the first choice as the ORM with a lot of auto-generated code under the hood. But to offer users flexibility and allow them to create custom fields and custom objects, something more low-level than Prisma made more sense to have more fine-grained control. This is why the project now uses TypeORM.
Here's what the tech stack now looks like.
## Tech Stack
**Core**
- [NestJS](https://nestjs.com/)
- [TypeORM](https://typeorm.io/)
- [GraphQL Yoga](https://the-guild.dev/graphql/yoga-server)
**Database**
- [Postgres](https://www.postgresql.org/)
**Third-party integrations**
- [Sentry](https://sentry.io/welcome/) for tracking bugs
**Testing**
- [Jest](https://jestjs.io/)
**Tooling**
- [Yarn](https://yarnpkg.com/)
- [ESLint](https://eslint.org/)
**Development**
- [AWS EKS](https://aws.amazon.com/eks/)

View File

@ -0,0 +1,47 @@
---
title: Message Queue
sidebar_position: 5
sidebar_custom_props:
icon: TbSchema
---
Queues facilitate async operations to be performed. They can be used for performing background tasks such as sending a welcome email on register.
Each use case will have its own queue class extended from `MessageQueueServiceBase`.
Currently, queue supports two drivers which can be configurred by env variable `MESSAGE_QUEUE_TYPE`.
1. `pg-boss`: this is the default driver, which uses [pg-boss](https://github.com/timgit/pg-boss) under the hood.
2. `bull-mq`: this uses [bull-mq](https://bullmq.io/) under the hood.
## Steps to create and use a new queue
1. Add a queue name for your new queue under enum `MESSAGE_QUEUES`.
2. Provide the factory implementation of the queue with the queue name as the dependency token.
3. Inject the queue that you created in the required module/service with the queue name as the dependency token.
4. Add worker class with token based injection just like producer.
### Example usage
```ts
class Resolver {
constructor(@Inject(MESSAGE_QUEUES.custom) private queue: MessageQueueService) {}
async onSomeAction() {
//business logic
await this.queue.add(someData);
}
}
//async worker
class CustomWorker {
constructor(@Inject(MESSAGE_QUEUES.custom) private queue: MessageQueueService) {
this.initWorker();
}
async initWorker() {
await this.queue.work(async ({ id, data }) => {
//worker logic
});
}
}
```

View File

@ -0,0 +1,61 @@
---
title: Development workflow
sidebar_position: 3
sidebar_custom_props:
icon: TbTopologyStar
---
## First time setup
```
cd server
yarn # install dependencies
yarn prisma:migrate # run migrations
yarn prisma:generate # generate prisma and nestjs-graphql schemas
yarn prisma:seed # provision database with seeds
# alternatively, you can run
yarn prisma:reset # all-in-one command to reset, migrate, seed and generate schemas
```
## Starting a new feature
Make sure your database is running on the URL provided in your `server/.env` file.
```
cd server
yarn
yarn prisma:migrate && yarn prisma:generate
yarn start:dev
```
## Lint
```
yarn lint
```
## Test
```
yarn test
```
## Resetting the database
If you want to reset the database, you can run the following command:
```bash
cd server
yarn database:reset
```
:::warning
This will drop the database and re-run the migrations and seed.
Make sure to back up any data you want to keep before running this command.
:::

View File

@ -0,0 +1,4 @@
{
"label": "Others",
"position": 2
}

View File

@ -0,0 +1,25 @@
---
title: Best Practices
sidebar_position: 3
sidebar_custom_props:
icon: TbChecklist
---
This document outlines the best practices you should follow when working on the backend.
## Follow a modular approach
The backend follows a modular approach, which is a fundamental principle when working with NestJS. Make sure you break down your code into reusable modules to maintain a clean and organized codebase.
Each module should encapsulate a particular feature or functionality and have a well-defined scope. This modular approach enables clear separation of concerns and removes unnecessary complexities.
## Expose services to use in modules
Always create services that have a clear and single responsibility, which enhances code readability and maintainability. Name the services descriptively and consistently.
You should also expose services that you want to use in other modules. Exposing services to other modules is possible through NestJS's powerful dependency injection system, and promotes loose coupling between components.
## Avoid using `any` type
When you declare a variable as `any`, TypeScript's type checker doesn't perform any type checking, making it possible to assign any type of values to the variable. TypeScript uses type inference to determine the type of variable based on the value. By declaring it as `any`, TypeScript can no longer infer the type. This makes it hard to catch type-related errors during development, leading to runtime errors and makes the code less maintainable, less reliable, and harder to understand for others.
This is why everything should have a type. So if you create a new object with a first name and last name, you should create an interface or type that contains a first name and last name that defines the shape of the object you are manipulating.

View File

@ -0,0 +1,75 @@
---
title: Zapier App
sidebar_position: 1
sidebar_custom_props:
icon: TbBrandZapier
---
Effortlessly sync Twenty with 3000+ apps using [Zapier](https://zapier.com/). Automate tasks, boost productivity, and supercharge your customer relationships!
## About Zapier
Zapier is a tool that allows you automate workflows by connecting the apps that your team uses everyday. The fundamental concept of Zapier is automation workflows, called Zaps, and include triggers and actions.
You can learn more about how Zapier works [here](https://zapier.com/how-it-works).
## Setup
### Step 1: Install Zapier packages
```bash
cd packages/twenty-zapier
yarn
```
### Step 2: Login with the CLI
Use your Zapier credentials to log in using the CLI:
```bash
zapier login
```
### Step 3: Set environment variables
From the `packages/twenty-zapier` folder, run:
```bash
cp .env.example .env
```
Run the application locally, go to [http://localhost:3000/settings/developers/api-keys](http://localhost:3000/settings/developers/api-keys), and generate an API key.
Replace the **YOUR_API_KEY** value in the `.env` file with the API key you just generated.
## Development
:::caution Note
Make sure to run `yarn build` before any `zapier` command.
:::
### Test
```bash
yarn test
```
### Lint
```bash
yarn format
```
### Watch and compile as you edit code
```bash
yarn watch
```
### Validate your Zapier app
```bash
yarn validate
```
### Deploy your Zapier app
```bash
yarn deploy
```
### List all Zapier CLI commands
```bash
zapier
```

View File

@ -0,0 +1,12 @@
---
title: Backend Development
displayed_sidebar: backendSidebar
sidebar_position: 0
sidebar_custom_props:
icon: TbTerminal
isSidebarRoot: true
---
Welcome to the Backend Development section of the documentation.
Here you will find information about the development process, the recommended tools, and the best practices you should follow.