admin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Improve performance on metadata computation (#12785)

Browse Source 
In this PR:

## Improve recompute metadata cache performance. We are aiming for
~100ms

Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing

## Remove caching lock

As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces

## Introduce self recovery mecanisms to recompute cache automatically if
corrupted

Aka getFreshObjectMetadataMaps

## custom object resolver performance improvement:  1sec to 200ms

Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap

## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>

We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge

## Add dataloader on IndexMetadata (aka indexMetadataList in the API)

## Improve field resolver performances too

## Deprecate ClientConfig
This commit is contained in:
Charles Bochet
2025-06-23 21:06:17 +02:00
committed by GitHub
parent 6aee42ab22
commit d5c974054d
145 changed files with 1485 additions and 2245 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
packages/twenty-server/src/database/commands/command-runners/active-or-suspended-workspaces-migration.command-runner.ts
View File

@ -140,7 +140,6 @@ export abstract class ActiveOrSuspendedWorkspacesMigrationCommandRunner<
const dataSource =
await this.twentyORMGlobalManager.getDataSourceForWorkspace({
workspaceId,
shouldFailIfMetadataNotFound: false,
});
await this.runOnWorkspace({

30
packages/twenty-server/src/database/typeorm/core/migrations/common/1750673748111-remove-relation-metadata.ts Normal file
View File

@ -0,0 +1,30 @@
import { MigrationInterface, QueryRunner } from 'typeorm';
export class RemoveRelationMetadata1750673748111 implements MigrationInterface {
name = 'RemoveRelationMetadata1750673748111';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE INDEX "IDX_DATA_SOURCE_WORKSPACE_ID_CREATED_AT" ON "core"."dataSource" ("workspaceId", "createdAt") `,
);
await queryRunner.query(
`ALTER TABLE "core"."relationMetadata" DROP CONSTRAINT IF EXISTS "FK_9dea8f90d04edbbf9c541a95c3b"`,
);
await queryRunner.query(
`ALTER TABLE "core"."relationMetadata" DROP CONSTRAINT IF EXISTS "FK_3deb257254145a3bdde9575e7d6"`,
);
await queryRunner.query(
`ALTER TABLE "core"."relationMetadata" DROP CONSTRAINT IF EXISTS "FK_0f781f589e5a527b8f3d3a4b824"`,
);
await queryRunner.query(
`ALTER TABLE "core"."relationMetadata" DROP CONSTRAINT IF EXISTS "FK_f2a0acd3a548ee446a1a35df44d"`,
);
await queryRunner.query(`DROP TABLE IF EXISTS "core"."relationMetadata"`);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`DROP INDEX "core"."IDX_DATA_SOURCE_WORKSPACE_ID_CREATED_AT"`,
);
}
}