admin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Improve sync performances (#12639)

Browse Source 
## Goal

We have identified that sync-metadata (which is called during new
workspace initialization) is slow mainly because of workspaceMigration
application (migration-runner module). This is due to the fact that we
use typeORM API to perform schema changes, which often query the
existing schema. As querying the existing schema is costly (especially
with ~1M existing columns) and as we already have what we need described
as metadata, we will use raw SQL directly. This should divide the
workspace initialization time by x2.

## How

This PR can be read in two commits:

1. Extract functions tied to column migrations in a separate service
(`workspace-migration-column.service`) + deprecate COMMENT column
migration type which is not useful since we are not using pg-graphql
anymore
2. Re-work `workspace-migration-column.service` to make it clearer + use
raw SQL

## Result

Before:
<img width="1367" alt="image"
src="https://github.com/user-attachments/assets/e730df7b-db7f-4433-9ce5-52841b010990"
/>

After:
<img width="1367" alt="image"
src="https://github.com/user-attachments/assets/72d2c2b1-2475-4541-a3d5-50b70824a2e4"
/>



## Manual Testing

- Sync-metadata OK
- Workspace init OK
This commit is contained in:
Charles Bochet
2025-06-16 23:53:42 +02:00
committed by GitHub
parent 497faca574
commit d1e0af7f38
9 changed files with 460 additions and 483 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
packages/twenty-server/src/engine/metadata-modules/workspace-migration/workspace-migration.entity.ts
View File

@ -14,7 +14,6 @@ export enum WorkspaceMigrationColumnActionType {
CREATE_FOREIGN_KEY = 'CREATE_FOREIGN_KEY',
DROP_FOREIGN_KEY = 'DROP_FOREIGN_KEY',
DROP = 'DROP',
CREATE_COMMENT = 'CREATE_COMMENT',
}
export type WorkspaceMigrationRenamedEnum = { from: string; to: string };
export type WorkspaceMigrationEnum = string | WorkspaceMigrationRenamedEnum;
@ -57,16 +56,15 @@ export type WorkspaceMigrationColumnAlter = {
alteredColumnDefinition: WorkspaceMigrationColumnDefinition;
};
export type WorkspaceMigrationColumnCreateRelation = {
export type WorkspaceMigrationColumnCreateForeignKey = {
action: WorkspaceMigrationColumnActionType.CREATE_FOREIGN_KEY;
columnName: string;
referencedTableName: string;
referencedTableColumnName: string;
isUnique?: boolean;
onDelete?: RelationOnDeleteAction;
};
export type WorkspaceMigrationColumnDropRelation = {
export type WorkspaceMigrationColumnDropForeignKey = {
action: WorkspaceMigrationColumnActionType.DROP_FOREIGN_KEY;
columnName: string;
};
@ -76,11 +74,6 @@ export type WorkspaceMigrationColumnDrop = {
columnName: string;
};
export type WorkspaceMigrationCreateComment = {
action: WorkspaceMigrationColumnActionType.CREATE_COMMENT;
comment: string;
};
export type WorkspaceMigrationForeignColumnDefinition =
WorkspaceMigrationColumnDefinition & {
distantColumnName: string;
@ -108,10 +101,9 @@ export type WorkspaceMigrationColumnAction = {
} & (
| WorkspaceMigrationColumnCreate
| WorkspaceMigrationColumnAlter
| WorkspaceMigrationColumnCreateRelation
| WorkspaceMigrationColumnDropRelation
| WorkspaceMigrationColumnCreateForeignKey
| WorkspaceMigrationColumnDropForeignKey
| WorkspaceMigrationColumnDrop
| WorkspaceMigrationCreateComment
);
/**