admin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Improve migration runner performances (#10572)

Browse Source 
## Context
Workspace creation and more specifically sync-metadata performances are
bad at the moment. We are trying to identify bottlenecks and one of the
root causes is the migration runner that can take up to 10s when setting
up a new workspaces with all its tables.
First observation is we do a lot of things sequentially, mostly to make
the code easier to read and debug but it impacts performances. For
example, a table creation is done in two steps, we first ask typeorm to
create the table then ask typeorm to create columns (and sometimes
columns one by one), each instruction can take time because typeorm
seems to do some checks internally.

The proposition here is to try to merge migrations when possible, for
example when we create a table we want the migration to also contain the
columns it will contain so we can ask typeorm to add the columns at the
same time. We are also using batch operations when possible (addColumns
instead of addColumn, dropColumns instead of dropColumn)
Still, we could go further with foreign keys creations or/and try with
raw query directly.

## Test
New workspace creation:
See screenshot, 9865.40233296156ms is on main, the rest is after the
changes:

<img width="610" alt="Screenshot 2025-02-28 at 09 27 21"
src="https://github.com/user-attachments/assets/42e880ff-279e-4170-b705-009e4b72045c"
/>

ResetDB and Sync-metadata on an existing workspace commands still work
This commit is contained in:
Weiko
2025-02-28 19:51:45 +01:00
committed by GitHub
parent baa3043954
commit d19144208b
4 changed files with 385 additions and 158 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
packages/twenty-server/src/database/typeorm/typeorm.service.ts
View File

@ -5,6 +5,7 @@ import { DataSource } from 'typeorm';
import { NodeEnvironment } from 'src/engine/core-modules/environment/interfaces/node-environment.interface';
import { AppToken } from 'src/engine/core-modules/app-token/app-token.entity';
import { ApprovedAccessDomain } from 'src/engine/core-modules/approved-access-domain/approved-access-domain.entity';
import { BillingCustomer } from 'src/engine/core-modules/billing/entities/billing-customer.entity';
import { BillingEntitlement } from 'src/engine/core-modules/billing/entities/billing-entitlement.entity';
import { BillingMeter } from 'src/engine/core-modules/billing/entities/billing-meter.entity';
@ -22,7 +23,6 @@ import { UserWorkspace } from 'src/engine/core-modules/user-workspace/user-works
import { User } from 'src/engine/core-modules/user/user.entity';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { DataSourceEntity } from 'src/engine/metadata-modules/data-source/data-source.entity';
import { ApprovedAccessDomain } from 'src/engine/core-modules/approved-access-domain/approved-access-domain.entity';
@Injectable()
export class TypeORMService implements OnModuleInit, OnModuleDestroy {
private mainDataSource: DataSource;
@ -33,7 +33,10 @@ export class TypeORMService implements OnModuleInit, OnModuleDestroy {
this.mainDataSource = new DataSource({
url: environmentService.get('PG_DATABASE_URL'),
type: 'postgres',
logging: false,
logging:
environmentService.get('NODE_ENV') === NodeEnvironment.development
? ['query', 'error']
: ['error'],
schema: 'core',
entities: [
User,