Migrate to a monorepo structure (#2909)
This commit is contained in:
Charles Bochet
@ -0,0 +1,4 @@
|
||||
{
|
||||
"label": "Basics",
|
||||
"position": 1
|
||||
}
|
||||
@ -0,0 +1,41 @@
|
||||
---
|
||||
title: Custom Objects
|
||||
sidebar_position: 4
|
||||
sidebar_custom_props:
|
||||
icon: TbAugmentedReality
|
||||
---
|
||||
|
||||
Objects are structures that allow you to store data (records, attributes, and values) specific to an organization. Twenty provides both standard and custom objects.
|
||||
|
||||
Standard objects are in-built objects with a set of attributes available for all users. Examples of standard objects in Twenty include Company and Person. Standard objects have standard fields that are also available for all Twenty users, like Company.displayName.
|
||||
|
||||
Custom objects are objects that you can create to store information that is unique to your organization. They are not built-in; members of your workspace can create and customize custom objects to hold information that standard objects aren't suitable for.
|
||||
|
||||
|
||||
## High-level schema
|
||||
|
||||
<div style={{textAlign: 'center'}}>
|
||||
<img src="/img/contributor/custom-object-schema.jpeg" alt="High level schema" />
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
|
||||
## How it works
|
||||
|
||||
Custom objects come from metadata tables that determine the shape, name, and type of the objects. All this information is present in the metadata schema database, consisting of tables:
|
||||
|
||||
- **DataSource**: Details where the data is present.
|
||||
- **Object**: Describes the object and links to a DataSource.
|
||||
- **Field**: Outlines an Object's fields and connects to the Object.
|
||||
|
||||
To add a custom object, the workspaceMember will query the /metadata API. This updates the metadata accordingly and computes a GraphQL schema based on the metadata, storing it in a GQL cache for later use.
|
||||
<div style={{textAlign: 'center'}}>
|
||||
<img src="/img/contributor/add-custom-objects.jpeg" alt="Query the /metadata API to add custom objects" />
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
|
||||
To fetch data, the process involves making queries through the /graphql endpoint and passing them through the Query Resolver.
|
||||
<div style={{textAlign: 'center'}}>
|
||||
<img src="/img/contributor/custom-objects-schema.png" alt="Query the /graphql endpoint to fetch data" />
|
||||
</div>
|
||||
@ -0,0 +1,132 @@
|
||||
---
|
||||
title: Folder Architecture
|
||||
sidebar_position: 2
|
||||
description: A detailed look into our server folder architecture
|
||||
sidebar_custom_props:
|
||||
icon: TbFolder
|
||||
---
|
||||
|
||||
|
||||
The backend directory structure is as follows:
|
||||
|
||||
```
|
||||
server
|
||||
└───ability
|
||||
└───constants
|
||||
└───core
|
||||
└───database
|
||||
└───decorators
|
||||
└───filters
|
||||
└───guards
|
||||
└───health
|
||||
└───integrations
|
||||
└───metadata
|
||||
└───workspace
|
||||
└───utils
|
||||
```
|
||||
|
||||
## Ability
|
||||
|
||||
Defines permissions and includes handlers for each entity.
|
||||
|
||||
## Decorators
|
||||
|
||||
Defines custom decorators in NestJS for added functionality.
|
||||
|
||||
See [custom decorators](https://docs.nestjs.com/custom-decorators) for more details.
|
||||
|
||||
## Filters
|
||||
|
||||
Includes exception filters to handle exceptions that might occur in GraphQL endpoints.
|
||||
|
||||
## Guards
|
||||
|
||||
See [guards](https://docs.nestjs.com/guards) for more details.
|
||||
|
||||
## Health
|
||||
|
||||
Includes a publicly available REST API (healthz) that returns a JSON to confirm whether the database is working as expected.
|
||||
|
||||
## Metadata
|
||||
|
||||
Defines custom objects and makes available a GraphQL API (graphql/metadata).
|
||||
|
||||
## Workspace
|
||||
|
||||
Generates and serves custom GraphQL schema based on the metadata.
|
||||
|
||||
### Workspace Directory Structure
|
||||
|
||||
```
|
||||
workspace
|
||||
└───workspace-schema-builder
|
||||
└───factories
|
||||
└───graphql-types
|
||||
└───database
|
||||
└───interfaces
|
||||
└───object-definitions
|
||||
└───services
|
||||
└───storage
|
||||
└───utils
|
||||
└───workspace-resolver-builder
|
||||
└───factories
|
||||
└───interfaces
|
||||
└───workspace-query-builder
|
||||
└───factories
|
||||
└───interfaces
|
||||
└───workspace-query-runner
|
||||
└───interfaces
|
||||
└───utils
|
||||
└───workspace-datasource
|
||||
└───workspace-manager
|
||||
└───workspace-migration-runner
|
||||
└───utils
|
||||
└───workspace.module.ts
|
||||
└───workspace.factory.spec.ts
|
||||
└───workspace.factory.ts
|
||||
```
|
||||
|
||||
|
||||
The root of the workspace directory includes the `workspace.factory.ts`, a file containing the `createGraphQLSchema` function. This function generates workspace-specific schema by using the metadata to tailor a schema for individual workspaces. By separating the schema and resolver construction, we use the `makeExecutableSchema` function, which combines these discrete elements.
|
||||
|
||||
This strategy is not just about organization, but also helps with optimization, such as caching generated type definitions to enhance performance and scalability.
|
||||
|
||||
### Workspace Schema builder
|
||||
|
||||
Generates the GraphQL schema, and includes:
|
||||
|
||||
#### Factories:
|
||||
|
||||
Specialised constructors to generate GraphQL-related constructs.
|
||||
- The type.factory translates field metadata into GraphQL types using `TypeMapperService`.
|
||||
- The type-definition.factory creates GraphQL input or output objects derived from `objectMetadata`.
|
||||
|
||||
#### GraphQL Types
|
||||
|
||||
Includes enumerations, inputs, objects, and scalars, and serves as the building blocks for the schema construction.
|
||||
|
||||
#### Interfaces and Object Definitions
|
||||
|
||||
Contains the blueprints for GraphQL entities, and includes both predefined and custom types like `MONEY` or `URL`.
|
||||
|
||||
#### Services
|
||||
|
||||
Contains the service responsible for associating FieldMetadataType with its appropriate GraphQL scalar or query modifiers.
|
||||
|
||||
#### Storage
|
||||
|
||||
Includes the `TypeDefinitionsStorage` class that contains reusable type definitions, preventing duplication of GraphQL types.
|
||||
|
||||
### Workspace Resolver Builder
|
||||
|
||||
Creates resolver functions for querying and mutatating the GraphQL schema.
|
||||
|
||||
Each factory in this directory is responsible for producing a distinct resolver type, such as the `FindManyResolverFactory`, designed for adaptable application across various tables.
|
||||
|
||||
### Workspace Query Builder
|
||||
|
||||
Includes factories that generate `pg_graphql` queries.
|
||||
|
||||
### Workspace Query Runner
|
||||
|
||||
Runs the generated queries on the database and parses the result.
|
||||
@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Overview
|
||||
sidebar_position: 0
|
||||
sidebar_custom_props:
|
||||
icon: TbEyeglass
|
||||
---
|
||||
|
||||
Twenty primarily uses NestJS for the backend.
|
||||
|
||||
Prisma was the first choice as the ORM with a lot of auto-generated code under the hood. But to offer users flexibility and allow them to create custom fields and custom objects, something more low-level than Prisma made more sense to have more fine-grained control. This is why the project now uses TypeORM.
|
||||
|
||||
Here's what the tech stack now looks like.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
**Core**
|
||||
- [NestJS](https://nestjs.com/)
|
||||
- [TypeORM](https://typeorm.io/)
|
||||
- [GraphQL Yoga](https://the-guild.dev/graphql/yoga-server)
|
||||
|
||||
**Database**
|
||||
- [Postgres](https://www.postgresql.org/)
|
||||
|
||||
**Third-party integrations**
|
||||
- [Sentry](https://sentry.io/welcome/) for tracking bugs
|
||||
|
||||
**Testing**
|
||||
- [Jest](https://jestjs.io/)
|
||||
|
||||
**Tooling**
|
||||
- [Yarn](https://yarnpkg.com/)
|
||||
- [ESLint](https://eslint.org/)
|
||||
|
||||
**Development**
|
||||
- [AWS EKS](https://aws.amazon.com/eks/)
|
||||
@ -0,0 +1,47 @@
|
||||
---
|
||||
title: Message Queue
|
||||
sidebar_position: 5
|
||||
sidebar_custom_props:
|
||||
icon: TbSchema
|
||||
---
|
||||
|
||||
Queues facilitate async operations to be performed. They can be used for performing background tasks such as sending a welcome email on register.
|
||||
Each use case will have its own queue class extended from `MessageQueueServiceBase`.
|
||||
|
||||
Currently, queue supports two drivers which can be configurred by env variable `MESSAGE_QUEUE_TYPE`.
|
||||
1. `pg-boss`: this is the default driver, which uses [pg-boss](https://github.com/timgit/pg-boss) under the hood.
|
||||
2. `bull-mq`: this uses [bull-mq](https://bullmq.io/) under the hood.
|
||||
|
||||
## Steps to create and use a new queue
|
||||
|
||||
1. Add a queue name for your new queue under enum `MESSAGE_QUEUES`.
|
||||
2. Provide the factory implementation of the queue with the queue name as the dependency token.
|
||||
3. Inject the queue that you created in the required module/service with the queue name as the dependency token.
|
||||
4. Add worker class with token based injection just like producer.
|
||||
|
||||
### Example usage
|
||||
```ts
|
||||
class Resolver {
|
||||
constructor(@Inject(MESSAGE_QUEUES.custom) private queue: MessageQueueService) {}
|
||||
|
||||
async onSomeAction() {
|
||||
//business logic
|
||||
await this.queue.add(someData);
|
||||
}
|
||||
}
|
||||
|
||||
//async worker
|
||||
class CustomWorker {
|
||||
constructor(@Inject(MESSAGE_QUEUES.custom) private queue: MessageQueueService) {
|
||||
this.initWorker();
|
||||
}
|
||||
|
||||
async initWorker() {
|
||||
await this.queue.work(async ({ id, data }) => {
|
||||
//worker logic
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
@ -0,0 +1,61 @@
|
||||
---
|
||||
title: Development workflow
|
||||
sidebar_position: 3
|
||||
sidebar_custom_props:
|
||||
icon: TbTopologyStar
|
||||
---
|
||||
|
||||
## First time setup
|
||||
|
||||
```
|
||||
cd server
|
||||
yarn # install dependencies
|
||||
|
||||
yarn prisma:migrate # run migrations
|
||||
yarn prisma:generate # generate prisma and nestjs-graphql schemas
|
||||
yarn prisma:seed # provision database with seeds
|
||||
|
||||
# alternatively, you can run
|
||||
yarn prisma:reset # all-in-one command to reset, migrate, seed and generate schemas
|
||||
```
|
||||
|
||||
## Starting a new feature
|
||||
|
||||
Make sure your database is running on the URL provided in your `server/.env` file.
|
||||
|
||||
```
|
||||
cd server
|
||||
yarn
|
||||
yarn prisma:migrate && yarn prisma:generate
|
||||
|
||||
yarn start:dev
|
||||
```
|
||||
|
||||
## Lint
|
||||
|
||||
```
|
||||
yarn lint
|
||||
```
|
||||
|
||||
## Test
|
||||
|
||||
```
|
||||
yarn test
|
||||
```
|
||||
|
||||
## Resetting the database
|
||||
|
||||
If you want to reset the database, you can run the following command:
|
||||
|
||||
```bash
|
||||
cd server
|
||||
yarn database:reset
|
||||
```
|
||||
|
||||
:::warning
|
||||
|
||||
This will drop the database and re-run the migrations and seed.
|
||||
|
||||
Make sure to back up any data you want to keep before running this command.
|
||||
|
||||
:::
|
||||
Reference in New Issue
Block a user