admin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Add unique indexes and indexes for composite types (#7162)

Browse Source 
Add support for indexes on composite fields and unicity constraint on
indexes

This pull request includes several changes across multiple files to
improve error handling, enforce unique constraints, and update database
migrations. The most important changes include updating error messages
for snack bars, adding a new command to enforce unique constraints, and
updating database migrations to include new fields and constraints.

### Error Handling Improvements:
*
[`packages/twenty-front/src/modules/error-handler/components/PromiseRejectionEffect.tsx`](diffhunk://#diff-e7dc05ced8e4730430f5c7fcd0c75b3aa723da438c26e0bef8130b614427dd9aL23-R23):
Updated error messages in `enqueueSnackBar` to use `error.message`
directly.
*
[`packages/twenty-front/src/modules/object-metadata/hooks/useFindManyObjectMetadataItems.ts`](diffhunk://#diff-74c126d6bc7a5ed6b63be994d298df6669058034bfbc367b11045f9f31a3abe6L44-R46):
Simplified error messages in `enqueueSnackBar`.
*
[`packages/twenty-front/src/modules/object-record/hooks/useFindDuplicateRecords.ts`](diffhunk://#diff-af23a1d99639a66c251f87473e63e2b7bceaa4ee4f70fedfa0fcffe5c7d79181L56-R58):
Simplified error messages in `enqueueSnackBar`.
*
[`packages/twenty-front/src/modules/object-record/hooks/useHandleFindManyRecordsError.ts`](diffhunk://#diff-da04296cbe280202a1eaf6b1244a30490d4f400411bee139651172c59719088eL22-R24):
Simplified error messages in `enqueueSnackBar`.

### New Command for Unique Constraints:
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-enforce-unique-constraints.command.ts`](diffhunk://#diff-8337096c8c80dd2619a5ba691ae5145101f8ae0368a75192a050047e8c6ab7cbR1-R159):
Added a new command to enforce unique constraints on company domain
names and person emails.
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.command.ts`](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR13-R14):
Integrated the new `EnforceUniqueConstraintsCommand` into the upgrade
process.
[[1]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR13-R14)
[[2]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR31)
[[3]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR64-R68)
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.module.ts`](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R7):
Registered the new `EnforceUniqueConstraintsCommand` in the module.
[[1]](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R7)
[[2]](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R24)

### Database Migrations:
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368824-migrationDebt.ts`](diffhunk://#diff-c450aeae7bc0ef4416a0ade2dc613ca3f688629f35d2a32f90a09c3f494febdcR1-R53):
Added a migration to update the `relationMetadata_ondeleteaction_enum`
and set default values.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368825-addIsUniqueToIndexMetadata.ts`](diffhunk://#diff-8f1e14bd7f6835ec2c3bb39bcc51e3c318a3008d576a981e682f4c985e746fbfR1-R19):
Added a migration to include the `isUnique` field in `indexMetadata`.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726762935841-addCompostiveColumnToIndexFieldMetadata.ts`](diffhunk://#diff-7c96b7276c7722d41ff31de23b2de4d6e09adfdc74815356ba63bc96a2669440R1-R19):
Added a migration to include the `compositeColumn` field in
`indexFieldMetadata`.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726766871572-addWhereToIndexMetadata.ts`](diffhunk://#diff-26651295a975eb50e672dce0e4e274e861f66feb1b68105eee5a04df32796190R1-R14):
Added a migration to include the `indexWhereClause` field in
`indexMetadata`.

### GraphQL Exception Handling:
*
[`packages/twenty-server/src/engine/api/graphql/workspace-query-runner/utils/workspace-query-runner-graphql-api-exception-handler.util.ts`](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R1-R4):
Enhanced exception handling for `QueryFailedError` to provide more
specific error messages for unique constraint violations.
[[1]](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R1-R4)
[[2]](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R23-R59)
*
[`packages/twenty-server/src/engine/api/graphql/workspace-resolver-builder/factories/create-many-resolver.factory.ts`](diffhunk://#diff-233d58ab2333586dd45e46e33d4f07e04a4b8adde4a11a48e25d86985e5a7943L58-R58):
Updated the `workspaceQueryRunnerGraphqlApiExceptionHandler` call to
include context.
*
[`packages/twenty-server/src/engine/api/graphql/workspace-resolver-builder/factories/create-one-resolver.factory.ts`](diffhunk://#diff-68b803f0762c407f5d2d1f5f8d389655a60654a2dd2394a81318655dcd44dc43L58-R58):
Updated the `workspaceQueryRunnerGraphqlApiExceptionHandler` call to
include context.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
This commit is contained in:
Félix Malfait
2024-10-13 10:21:03 +02:00
committed by GitHub
parent d1d4af0c63
commit b792d2a4d3
137 changed files with 22351 additions and 17974 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
packages/twenty-server/src/database/commands/database-command.module.ts
View File

@ -7,7 +7,7 @@ import { DataSeedDemoWorkspaceCommand } from 'src/database/commands/data-seed-de
import { DataSeedDemoWorkspaceModule } from 'src/database/commands/data-seed-demo-workspace/data-seed-demo-workspace.module';
import { DataSeedWorkspaceCommand } from 'src/database/commands/data-seed-dev-workspace.command';
import { ConfirmationQuestion } from 'src/database/commands/questions/confirmation.question';
import { UpgradeTo0_31CommandModule } from 'src/database/commands/upgrade-version/0-31/0-31-upgrade-version.module';
import { UpgradeTo0_32CommandModule } from 'src/database/commands/upgrade-version/0-32/0-32-upgrade-version.module';
import { TypeORMModule } from 'src/database/typeorm/typeorm.module';
import { BillingSubscription } from 'src/engine/core-modules/billing/entities/billing-subscription.entity';
import { FeatureFlagEntity } from 'src/engine/core-modules/feature-flag/feature-flag.entity';
@ -46,7 +46,7 @@ import { WorkspaceSyncMetadataModule } from 'src/engine/workspace-manager/worksp
DataSeedDemoWorkspaceModule,
WorkspaceCacheStorageModule,
WorkspaceMetadataVersionModule,
UpgradeTo0_31CommandModule,
UpgradeTo0_32CommandModule,
FeatureFlagModule,
],
providers: [

128
packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-add-index-key-to-tasks-and-notes-views.command.ts
View File

@ -1,128 +0,0 @@
import { InjectRepository } from '@nestjs/typeorm';
import chalk from 'chalk';
import { Command } from 'nest-commander';
import { In, Repository } from 'typeorm';
import {
ActiveWorkspacesCommandOptions,
ActiveWorkspacesCommandRunner,
} from 'src/database/commands/active-workspaces.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';
import { STANDARD_OBJECT_IDS } from 'src/engine/workspace-manager/workspace-sync-metadata/constants/standard-object-ids';
import { ViewWorkspaceEntity } from 'src/modules/view/standard-objects/view.workspace-entity';
@Command({
name: 'upgrade-0.31:add-index-key-to-tasks-and-notes-views',
description: 'Add index key to tasks and notes views',
})
export class AddIndexKeyToTasksAndNotesViewsCommand extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
@InjectRepository(ObjectMetadataEntity, 'metadata')
private readonly objectMetadataRepository: Repository<ObjectMetadataEntity>,
private readonly twentyORMGlobalManager: TwentyORMGlobalManager,
) {
super(workspaceRepository);
}
async executeActiveWorkspacesCommand(
_passedParam: string[],
_options: ActiveWorkspacesCommandOptions,
workspaceIds: string[],
): Promise<void> {
this.logger.log('Running command to fix migration');
for (const workspaceId of workspaceIds) {
this.logger.log(`Running command for workspace ${workspaceId}`);
try {
this.logger.log(chalk.green(`Cleaning views of ${workspaceId}.`));
await this.addIndexKeyToTasksAndNotesViews(
workspaceId,
_options.dryRun ?? false,
);
await this.twentyORMGlobalManager.destroyDataSourceForWorkspace(
workspaceId,
);
} catch (error) {
this.logger.log(
chalk.red(
`Running command on workspace ${workspaceId} failed with error: ${error}`,
),
);
continue;
} finally {
this.logger.log(
chalk.green(`Finished running command for workspace ${workspaceId}.`),
);
}
this.logger.log(chalk.green(`Command completed!`));
}
}
private async addIndexKeyToTasksAndNotesViews(
workspaceId: string,
dryRun: boolean,
): Promise<void> {
const viewRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace<ViewWorkspaceEntity>(
workspaceId,
'view',
false,
);
const allViews = await viewRepository.find();
const viewObjectMetadataIds = allViews.map((view) => view.objectMetadataId);
const objectMetadataEntities = await this.objectMetadataRepository.find({
where: {
id: In(viewObjectMetadataIds),
},
});
const tasksAndNotesObjectMetadataIds = objectMetadataEntities.filter(
(entity) =>
entity.standardId === STANDARD_OBJECT_IDS.task ||
entity.standardId === STANDARD_OBJECT_IDS.note,
);
const viewsToUpdate = allViews.filter(
(view) =>
tasksAndNotesObjectMetadataIds.some(
(entity) => entity.id === view.objectMetadataId,
) &&
['All Tasks', 'All Notes'].includes(view.name) &&
view.key === null,
);
if (dryRun) {
this.logger.log(
chalk.green(
`Found ${viewsToUpdate.length} views to update in workspace ${workspaceId}.`,
),
);
}
if (viewsToUpdate.length > 0 && !dryRun) {
await viewRepository.update(
viewsToUpdate.map((view) => view.id),
{
key: 'INDEX',
},
);
this.logger.log(chalk.green(`Updating ${viewsToUpdate.length} views.`));
}
if (viewsToUpdate.length === 0 && !dryRun) {
this.logger.log(chalk.green(`No views to update.`));
}
}
}

162
packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-backfill-workspace-favorites.command.ts
View File

@ -1,162 +0,0 @@
import { InjectRepository } from '@nestjs/typeorm';
import chalk from 'chalk';
import { Command } from 'nest-commander';
import { In, Repository } from 'typeorm';
import {
ActiveWorkspacesCommandOptions,
ActiveWorkspacesCommandRunner,
} from 'src/database/commands/active-workspaces.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';
import { FavoriteWorkspaceEntity } from 'src/modules/favorite/standard-objects/favorite.workspace-entity';
import { ViewWorkspaceEntity } from 'src/modules/view/standard-objects/view.workspace-entity';
@Command({
name: 'upgrade-0.31:backfill-workspace-favorites-migration',
description: 'Create a workspace favorite for all workspace views',
})
export class BackfillWorkspaceFavoritesCommand extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
@InjectRepository(ObjectMetadataEntity, 'metadata')
private readonly objectMetadataRepository: Repository<ObjectMetadataEntity>,
private readonly twentyORMGlobalManager: TwentyORMGlobalManager,
) {
super(workspaceRepository);
}
async executeActiveWorkspacesCommand(
_passedParam: string[],
_options: ActiveWorkspacesCommandOptions,
workspaceIds: string[],
): Promise<void> {
this.logger.log('Running command to fix migration');
for (const workspaceId of workspaceIds) {
this.logger.log(`Running command for workspace ${workspaceId}`);
try {
const allWorkspaceIndexViews = await this.getIndexViews(workspaceId);
const activeWorkspaceIndexViews =
await this.filterViewsWithoutObjectMetadata(
workspaceId,
allWorkspaceIndexViews,
);
await this.createViewWorkspaceFavorites(
workspaceId,
activeWorkspaceIndexViews.map((view) => view.id),
_options.dryRun ?? false,
);
this.logger.log(
chalk.green(`Backfilled workspace favorites to ${workspaceId}.`),
);
await this.twentyORMGlobalManager.destroyDataSourceForWorkspace(
workspaceId,
);
} catch (error) {
this.logger.log(
chalk.red(
`Running command on workspace ${workspaceId} failed with error: ${error}`,
),
);
continue;
} finally {
this.logger.log(
chalk.green(`Finished running command for workspace ${workspaceId}.`),
);
}
this.logger.log(chalk.green(`Command completed!`));
}
}
private async getIndexViews(
workspaceId: string,
): Promise<ViewWorkspaceEntity[]> {
const viewRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace<ViewWorkspaceEntity>(
workspaceId,
'view',
false,
);
return viewRepository.find({
where: {
key: 'INDEX',
},
});
}
private async filterViewsWithoutObjectMetadata(
workspaceId: string,
views: ViewWorkspaceEntity[],
): Promise<ViewWorkspaceEntity[]> {
const viewObjectMetadataIds = views.map((view) => view.objectMetadataId);
const objectMetadataEntities = await this.objectMetadataRepository.find({
where: {
workspaceId,
id: In(viewObjectMetadataIds),
},
});
const objectMetadataIds = new Set(
objectMetadataEntities.map((entity) => entity.id),
);
return views.filter((view) => objectMetadataIds.has(view.objectMetadataId));
}
private async createViewWorkspaceFavorites(
workspaceId: string,
viewIds: string[],
dryRun: boolean,
) {
const favoriteRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace<FavoriteWorkspaceEntity>(
workspaceId,
'favorite',
);
let nextFavoritePosition = await favoriteRepository.count();
let createdFavorites = 0;
for (const viewId of viewIds) {
const existingFavorites = await favoriteRepository.find({
where: {
viewId,
},
});
if (existingFavorites.length) {
continue;
}
if (!dryRun) {
await favoriteRepository.insert(
favoriteRepository.create({
viewId,
position: nextFavoritePosition,
}),
);
}
createdFavorites++;
nextFavoritePosition++;
}
this.logger.log(
chalk.green(
`Found ${createdFavorites} favorites to backfill in workspace ${workspaceId}.`,
),
);
}
}

119
packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-clean-views-associated-with-outdated-objects.command.ts
View File

@ -1,119 +0,0 @@
import { InjectRepository } from '@nestjs/typeorm';
import chalk from 'chalk';
import { Command } from 'nest-commander';
import { In, Repository } from 'typeorm';
import {
ActiveWorkspacesCommandOptions,
ActiveWorkspacesCommandRunner,
} from 'src/database/commands/active-workspaces.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';
import { STANDARD_OBJECT_IDS } from 'src/engine/workspace-manager/workspace-sync-metadata/constants/standard-object-ids';
import { ViewWorkspaceEntity } from 'src/modules/view/standard-objects/view.workspace-entity';
@Command({
name: 'upgrade-0.31:clean-views-associated-with-outdated-objects',
description:
'Clean views associated with deleted object metadata or activities',
})
export class CleanViewsAssociatedWithOutdatedObjectsCommand extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
@InjectRepository(ObjectMetadataEntity, 'metadata')
private readonly objectMetadataRepository: Repository<ObjectMetadataEntity>,
private readonly twentyORMGlobalManager: TwentyORMGlobalManager,
) {
super(workspaceRepository);
}
async executeActiveWorkspacesCommand(
_passedParam: string[],
_options: ActiveWorkspacesCommandOptions,
workspaceIds: string[],
): Promise<void> {
this.logger.log('Running command to fix migration');
for (const workspaceId of workspaceIds) {
this.logger.log(`Running command for workspace ${workspaceId}`);
try {
this.logger.log(chalk.green(`Cleaning views of ${workspaceId}.`));
await this.cleanViewsWithDeletedObjectMetadata(
workspaceId,
_options.dryRun ?? false,
);
await this.twentyORMGlobalManager.destroyDataSourceForWorkspace(
workspaceId,
);
} catch (error) {
this.logger.log(
chalk.red(
`Running command on workspace ${workspaceId} failed with error: ${error}`,
),
);
continue;
} finally {
this.logger.log(
chalk.green(`Finished running command for workspace ${workspaceId}.`),
);
}
this.logger.log(chalk.green(`Command completed!`));
}
}
private async cleanViewsWithDeletedObjectMetadata(
workspaceId: string,
dryRun: boolean,
): Promise<void> {
const viewRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace<ViewWorkspaceEntity>(
workspaceId,
'view',
false,
);
const allViews = await viewRepository.find();
const viewObjectMetadataIds = allViews.map((view) => view.objectMetadataId);
const objectMetadataEntities = await this.objectMetadataRepository.find({
where: {
id: In(viewObjectMetadataIds),
},
});
const validObjectMetadataIds = new Set(
objectMetadataEntities
.filter((entity) => entity.standardId !== STANDARD_OBJECT_IDS.activity)
.map((entity) => entity.id),
);
const viewIdsToDelete = allViews
.filter((view) => !validObjectMetadataIds.has(view.objectMetadataId))
.map((view) => view.id);
if (dryRun) {
this.logger.log(
chalk.green(
`Found ${viewIdsToDelete.length} views to clean in workspace ${workspaceId}.`,
),
);
}
if (viewIdsToDelete.length > 0 && !dryRun) {
await viewRepository.delete(viewIdsToDelete);
this.logger.log(chalk.green(`Cleaning ${viewIdsToDelete.length} views.`));
}
if (viewIdsToDelete.length === 0) {
this.logger.log(chalk.green(`No views to clean.`));
}
}
}

121
packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-delete-name-column-standard-object-tables.command.ts
View File

@ -1,121 +0,0 @@
import { InjectRepository } from '@nestjs/typeorm';
import chalk from 'chalk';
import { Command } from 'nest-commander';
import { Repository } from 'typeorm';
import {
ActiveWorkspacesCommandOptions,
ActiveWorkspacesCommandRunner,
} from 'src/database/commands/active-workspaces.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { FieldMetadataType } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';
@Command({
name: 'upgrade-0.31:delete-name-column-standard-object-tables',
description: 'Delete name column from standard object tables',
})
export class DeleteNameColumnStandardObjectTablesCommand extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
@InjectRepository(ObjectMetadataEntity, 'metadata')
private readonly objectMetadataRepository: Repository<ObjectMetadataEntity>,
private readonly twentyORMGlobalManager: TwentyORMGlobalManager,
) {
super(workspaceRepository);
}
async executeActiveWorkspacesCommand(
_passedParam: string[],
options: ActiveWorkspacesCommandOptions,
workspaceIds: string[],
): Promise<void> {
this.logger.log('Running command to fix migration');
for (const workspaceId of workspaceIds) {
this.logger.log(`Running command for workspace ${workspaceId}`);
try {
this.logger.log(
chalk.green(`Deleting name columns from workspace ${workspaceId}.`),
);
const standardObjects = await this.objectMetadataRepository.find({
where: {
isCustom: false,
workspaceId,
},
relations: ['fields'],
});
const dataSource =
await this.twentyORMGlobalManager.getDataSourceForWorkspace(
workspaceId,
);
dataSource.transaction(async (entityManager) => {
const queryRunner = entityManager.queryRunner;
for (const standardObject of standardObjects) {
if (options.dryRun) {
this.logger.log(
chalk.yellow(
`Dry run mode enabled. Skipping deletion of name column for workspace ${workspaceId} and table ${standardObject.nameSingular}.`,
),
);
continue;
}
const nameColumnExists = await queryRunner?.hasColumn(
standardObject.nameSingular,
'name',
);
const nameFieldMetadataExists = standardObject.fields.some(
(field) =>
field.name === 'name' && field.type === FieldMetadataType.TEXT,
);
if (nameFieldMetadataExists) {
this.logger.log(
chalk.yellow(
`Name field exists for workspace ${workspaceId} and table ${standardObject.nameSingular}. Skipping deletion.`,
),
);
continue;
}
if (!nameColumnExists) {
this.logger.log(
chalk.yellow(
`Name column does not exist for workspace ${workspaceId} and table ${standardObject.nameSingular}. Skipping deletion.`,
),
);
continue;
}
await queryRunner?.dropColumn(standardObject.nameSingular, 'name');
}
});
} catch (error) {
this.logger.log(
chalk.red(
`Running command on workspace ${workspaceId} failed with error: ${error}`,
),
);
continue;
} finally {
this.logger.log(
chalk.green(`Finished running command for workspace ${workspaceId}.`),
);
await this.twentyORMGlobalManager.destroyDataSourceForWorkspace(
workspaceId,
);
}
}
}
}

69
packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.command.ts
View File

@ -1,69 +0,0 @@
import { InjectRepository } from '@nestjs/typeorm';
import { Command } from 'nest-commander';
import { Repository } from 'typeorm';
import { ActiveWorkspacesCommandRunner } from 'src/database/commands/active-workspaces.command';
import { AddIndexKeyToTasksAndNotesViewsCommand } from 'src/database/commands/upgrade-version/0-31/0-31-add-index-key-to-tasks-and-notes-views.command';
import { BackfillWorkspaceFavoritesCommand } from 'src/database/commands/upgrade-version/0-31/0-31-backfill-workspace-favorites.command';
import { CleanViewsAssociatedWithOutdatedObjectsCommand } from 'src/database/commands/upgrade-version/0-31/0-31-clean-views-associated-with-outdated-objects.command';
import { DeleteNameColumnStandardObjectTablesCommand } from 'src/database/commands/upgrade-version/0-31/0-31-delete-name-column-standard-object-tables.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { SyncWorkspaceMetadataCommand } from 'src/engine/workspace-manager/workspace-sync-metadata/commands/sync-workspace-metadata.command';
interface UpdateTo0_31CommandOptions {
workspaceId?: string;
}
@Command({
name: 'upgrade-0.31',
description: 'Upgrade to 0.31',
})
export class UpgradeTo0_31Command extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
private readonly syncWorkspaceMetadataCommand: SyncWorkspaceMetadataCommand,
private readonly backfillWorkspaceFavoritesCommand: BackfillWorkspaceFavoritesCommand,
private readonly cleanViewsAssociatedWithOutdatedObjectsCommand: CleanViewsAssociatedWithOutdatedObjectsCommand,
private readonly addIndexKeyToTasksAndNotesViewsCommand: AddIndexKeyToTasksAndNotesViewsCommand,
private readonly deleteNameColumnStandardObjectTablesCommand: DeleteNameColumnStandardObjectTablesCommand,
) {
super(workspaceRepository);
}
async executeActiveWorkspacesCommand(
passedParam: string[],
options: UpdateTo0_31CommandOptions,
workspaceIds: string[],
): Promise<void> {
await this.syncWorkspaceMetadataCommand.executeActiveWorkspacesCommand(
passedParam,
{
...options,
force: true,
},
workspaceIds,
);
await this.cleanViewsAssociatedWithOutdatedObjectsCommand.executeActiveWorkspacesCommand(
passedParam,
options,
workspaceIds,
);
await this.addIndexKeyToTasksAndNotesViewsCommand.executeActiveWorkspacesCommand(
passedParam,
options,
workspaceIds,
);
await this.backfillWorkspaceFavoritesCommand.executeActiveWorkspacesCommand(
passedParam,
options,
workspaceIds,
);
await this.deleteNameColumnStandardObjectTablesCommand.executeActiveWorkspacesCommand(
passedParam,
options,
workspaceIds,
);
}
}

27
packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.module.ts
View File

@ -1,27 +0,0 @@
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { AddIndexKeyToTasksAndNotesViewsCommand } from 'src/database/commands/upgrade-version/0-31/0-31-add-index-key-to-tasks-and-notes-views.command';
import { BackfillWorkspaceFavoritesCommand } from 'src/database/commands/upgrade-version/0-31/0-31-backfill-workspace-favorites.command';
import { CleanViewsAssociatedWithOutdatedObjectsCommand } from 'src/database/commands/upgrade-version/0-31/0-31-clean-views-associated-with-outdated-objects.command';
import { DeleteNameColumnStandardObjectTablesCommand } from 'src/database/commands/upgrade-version/0-31/0-31-delete-name-column-standard-object-tables.command';
import { UpgradeTo0_31Command } from 'src/database/commands/upgrade-version/0-31/0-31-upgrade-version.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { WorkspaceSyncMetadataCommandsModule } from 'src/engine/workspace-manager/workspace-sync-metadata/commands/workspace-sync-metadata-commands.module';
@Module({
imports: [
TypeOrmModule.forFeature([Workspace], 'core'),
TypeOrmModule.forFeature([ObjectMetadataEntity], 'metadata'),
WorkspaceSyncMetadataCommandsModule,
],
providers: [
UpgradeTo0_31Command,
BackfillWorkspaceFavoritesCommand,
CleanViewsAssociatedWithOutdatedObjectsCommand,
AddIndexKeyToTasksAndNotesViewsCommand,
DeleteNameColumnStandardObjectTablesCommand,
],
})
export class UpgradeTo0_31CommandModule {}

303
packages/twenty-server/src/database/commands/upgrade-version/0-32/0-32-enforce-unique-constraints.command.ts Normal file
View File

@ -0,0 +1,303 @@
import { InjectRepository } from '@nestjs/typeorm';
import chalk from 'chalk';
import { Command, Option } from 'nest-commander';
import { IsNull, Repository } from 'typeorm';
import {
ActiveWorkspacesCommandOptions,
ActiveWorkspacesCommandRunner,
} from 'src/database/commands/active-workspaces.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { TwentyORMGlobalManager } from 'src/engine/twenty-orm/twenty-orm-global.manager';
interface EnforceUniqueConstraintsCommandOptions
extends ActiveWorkspacesCommandOptions {
person?: boolean;
company?: boolean;
viewField?: boolean;
viewSort?: boolean;
}
@Command({
name: 'upgrade-0.32:enforce-unique-constraints',
description:
'Enforce unique constraints on company domainName, person emailsPrimaryEmail, ViewField, and ViewSort',
})
export class EnforceUniqueConstraintsCommand extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
private readonly twentyORMGlobalManager: TwentyORMGlobalManager,
) {
super(workspaceRepository);
}
@Option({
flags: '--person',
description: 'Enforce unique constraints on person emailsPrimaryEmail',
})
parsePerson() {
return true;
}
@Option({
flags: '--company',
description: 'Enforce unique constraints on company domainName',
})
parseCompany() {
return true;
}
@Option({
flags: '--view-field',
description: 'Enforce unique constraints on ViewField',
})
parseViewField() {
return true;
}
@Option({
flags: '--view-sort',
description: 'Enforce unique constraints on ViewSort',
})
parseViewSort() {
return true;
}
async executeActiveWorkspacesCommand(
_passedParam: string[],
options: EnforceUniqueConstraintsCommandOptions,
workspaceIds: string[],
): Promise<void> {
this.logger.log('Running command to enforce unique constraints');
for (const workspaceId of workspaceIds) {
this.logger.log(`Running command for workspace ${workspaceId}`);
try {
await this.enforceUniqueConstraintsForWorkspace(
workspaceId,
options.dryRun ?? false,
options,
);
await this.twentyORMGlobalManager.destroyDataSourceForWorkspace(
workspaceId,
);
} catch (error) {
this.logger.log(
chalk.red(
`Running command on workspace ${workspaceId} failed with error: ${error}, ${error.stack}`,
),
);
continue;
} finally {
this.logger.log(
chalk.green(`Finished running command for workspace ${workspaceId}.`),
);
}
}
this.logger.log(chalk.green(`Command completed!`));
}
private async enforceUniqueConstraintsForWorkspace(
workspaceId: string,
dryRun: boolean,
options: EnforceUniqueConstraintsCommandOptions,
): Promise<void> {
if (options.company) {
await this.enforceUniqueCompanyDomainName(workspaceId, dryRun);
}
if (options.person) {
await this.enforceUniquePersonEmail(workspaceId, dryRun);
}
if (options.viewField) {
await this.enforceUniqueViewField(workspaceId, dryRun);
}
if (options.viewSort) {
await this.enforceUniqueViewSort(workspaceId, dryRun);
}
}
private async enforceUniqueCompanyDomainName(
workspaceId: string,
dryRun: boolean,
): Promise<void> {
const companyRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace(
workspaceId,
'company',
);
const duplicates = await companyRepository
.createQueryBuilder('company')
.select('company.domainNamePrimaryLinkUrl')
.addSelect('COUNT(*)', 'count')
.where('company.deletedAt IS NULL')
.where('company.domainNamePrimaryLinkUrl IS NOT NULL')
.where("company.domainNamePrimaryLinkUrl != ''")
.groupBy('company.domainNamePrimaryLinkUrl')
.having('COUNT(*) > 1')
.getRawMany();
for (const duplicate of duplicates) {
const { company_domainNamePrimaryLinkUrl } = duplicate;
const companies = await companyRepository.find({
where: {
domainName: {
primaryLinkUrl: company_domainNamePrimaryLinkUrl,
},
deletedAt: IsNull(),
},
order: { createdAt: 'DESC' },
});
for (let i = 1; i < companies.length; i++) {
const newdomainNamePrimaryLinkUrl = `${company_domainNamePrimaryLinkUrl}${i}`;
if (!dryRun) {
await companyRepository.update(companies[i].id, {
domainNamePrimaryLinkUrl: newdomainNamePrimaryLinkUrl,
});
}
this.logger.log(
chalk.yellow(
`Updated company ${companies[i].id} domainName from ${company_domainNamePrimaryLinkUrl} to ${newdomainNamePrimaryLinkUrl}`,
),
);
}
}
}
private async enforceUniquePersonEmail(
workspaceId: string,
dryRun: boolean,
): Promise<void> {
const personRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace(
workspaceId,
'person',
);
const duplicates = await personRepository
.createQueryBuilder('person')
.select('person.emailsPrimaryEmail')
.addSelect('COUNT(*)', 'count')
.where('person.deletedAt IS NULL')
.where('person.emailsPrimaryEmail IS NOT NULL')
.where("person.emailsPrimaryEmail != ''")
.groupBy('person.emailsPrimaryEmail')
.having('COUNT(*) > 1')
.getRawMany();
for (const duplicate of duplicates) {
const { person_emailsPrimaryEmail } = duplicate;
const persons = await personRepository.find({
where: {
emails: {
primaryEmail: person_emailsPrimaryEmail,
},
deletedAt: IsNull(),
},
order: { createdAt: 'DESC' },
});
for (let i = 1; i < persons.length; i++) {
const newEmail = person_emailsPrimaryEmail?.includes('@')
? `${person_emailsPrimaryEmail.split('@')[0]}+${i}@${person_emailsPrimaryEmail.split('@')[1]}`
: `${person_emailsPrimaryEmail}+${i}`;
if (!dryRun) {
await personRepository.update(persons[i].id, {
emailsPrimaryEmail: newEmail,
});
}
this.logger.log(
chalk.yellow(
`Updated person ${persons[i].id} emailsPrimaryEmail from ${person_emailsPrimaryEmail} to ${newEmail}`,
),
);
}
}
}
private async enforceUniqueViewField(
workspaceId: string,
dryRun: boolean,
): Promise<void> {
const viewFieldRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace(
workspaceId,
'viewField',
);
const duplicates = await viewFieldRepository
.createQueryBuilder('viewField')
.select(['viewField.fieldMetadataId', 'viewField.viewId'])
.addSelect('COUNT(*)', 'count')
.where('viewField.deletedAt IS NULL')
.groupBy('viewField.fieldMetadataId, viewField.viewId')
.having('COUNT(*) > 1')
.getRawMany();
for (const duplicate of duplicates) {
const { fieldMetadataId, viewId } = duplicate;
const viewFields = await viewFieldRepository.find({
where: { fieldMetadataId, viewId, deletedAt: IsNull() },
order: { createdAt: 'DESC' },
});
for (let i = 1; i < viewFields.length; i++) {
if (!dryRun) {
await viewFieldRepository.softDelete(viewFields[i].id);
}
this.logger.log(
chalk.yellow(
`Soft deleted duplicate ViewField ${viewFields[i].id} for fieldMetadataId ${fieldMetadataId} and viewId ${viewId}`,
),
);
}
}
}
private async enforceUniqueViewSort(
workspaceId: string,
dryRun: boolean,
): Promise<void> {
const viewSortRepository =
await this.twentyORMGlobalManager.getRepositoryForWorkspace(
workspaceId,
'viewSort',
);
const duplicates = await viewSortRepository
.createQueryBuilder('viewSort')
.select(['viewSort.fieldMetadataId', 'viewSort.viewId'])
.addSelect('COUNT(*)', 'count')
.where('viewSort.deletedAt IS NULL')
.groupBy('viewSort.fieldMetadataId, viewSort.viewId')
.having('COUNT(*) > 1')
.getRawMany();
for (const duplicate of duplicates) {
const { fieldMetadataId, viewId } = duplicate;
const viewSorts = await viewSortRepository.find({
where: { fieldMetadataId, viewId, deletedAt: IsNull() },
order: { createdAt: 'DESC' },
});
for (let i = 1; i < viewSorts.length; i++) {
if (!dryRun) {
await viewSortRepository.softDelete(viewSorts[i].id);
}
this.logger.log(
chalk.yellow(
`Soft deleted duplicate ViewSort ${viewSorts[i].id} for fieldMetadataId ${fieldMetadataId} and viewId ${viewId}`,
),
);
}
}
}
}

50
packages/twenty-server/src/database/commands/upgrade-version/0-32/0-32-upgrade-version.command.ts Normal file
View File

@ -0,0 +1,50 @@
import { InjectRepository } from '@nestjs/typeorm';
import { Command } from 'nest-commander';
import { Repository } from 'typeorm';
import { ActiveWorkspacesCommandRunner } from 'src/database/commands/active-workspaces.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { SyncWorkspaceMetadataCommand } from 'src/engine/workspace-manager/workspace-sync-metadata/commands/sync-workspace-metadata.command';
import { EnforceUniqueConstraintsCommand } from './0-32-enforce-unique-constraints.command';
interface UpdateTo0_32CommandOptions {
workspaceId?: string;
}
@Command({
name: 'upgrade-0.32',
description: 'Upgrade to 0.32',
})
export class UpgradeTo0_32Command extends ActiveWorkspacesCommandRunner {
constructor(
@InjectRepository(Workspace, 'core')
protected readonly workspaceRepository: Repository<Workspace>,
private readonly syncWorkspaceMetadataCommand: SyncWorkspaceMetadataCommand,
private readonly enforceUniqueConstraintsCommand: EnforceUniqueConstraintsCommand,
) {
super(workspaceRepository);
}
async executeActiveWorkspacesCommand(
passedParam: string[],
options: UpdateTo0_32CommandOptions,
workspaceIds: string[],
): Promise<void> {
await this.syncWorkspaceMetadataCommand.executeActiveWorkspacesCommand(
passedParam,
{
...options,
force: true,
},
workspaceIds,
);
await this.enforceUniqueConstraintsCommand.executeActiveWorkspacesCommand(
passedParam,
options,
workspaceIds,
);
}
}

18
packages/twenty-server/src/database/commands/upgrade-version/0-32/0-32-upgrade-version.module.ts Normal file
View File

@ -0,0 +1,18 @@
import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { EnforceUniqueConstraintsCommand } from 'src/database/commands/upgrade-version/0-32/0-32-enforce-unique-constraints.command';
import { UpgradeTo0_32Command } from 'src/database/commands/upgrade-version/0-32/0-32-upgrade-version.command';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { WorkspaceSyncMetadataCommandsModule } from 'src/engine/workspace-manager/workspace-sync-metadata/commands/workspace-sync-metadata-commands.module';
@Module({
imports: [
TypeOrmModule.forFeature([Workspace], 'core'),
TypeOrmModule.forFeature([ObjectMetadataEntity], 'metadata'),
WorkspaceSyncMetadataCommandsModule,
],
providers: [UpgradeTo0_32Command, EnforceUniqueConstraintsCommand],
})
export class UpgradeTo0_32CommandModule {}

9
packages/twenty-server/src/database/typeorm-seeds/core/feature-flags.ts
View File

@ -43,7 +43,7 @@ export const seedFeatureFlags = async (
{
key: FeatureFlagKey.IsWorkflowEnabled,
workspaceId: workspaceId,
value: false,
value: true,
},
{
key: FeatureFlagKey.IsMessageThreadSubscriberEnabled,
@ -53,7 +53,7 @@ export const seedFeatureFlags = async (
{
key: FeatureFlagKey.IsWorkspaceFavoriteEnabled,
workspaceId: workspaceId,
value: false,
value: true,
},
{
key: FeatureFlagKey.IsSearchEnabled,
@ -75,6 +75,11 @@ export const seedFeatureFlags = async (
workspaceId: workspaceId,
value: true,
},
{
key: FeatureFlagKey.IsUniqueIndexesEnabled,
workspaceId: workspaceId,
value: false,
},
])
.execute();
};

8
packages/twenty-server/src/database/typeorm-seeds/metadata/fieldsMetadata.ts
View File

@ -15,6 +15,7 @@ export const getDevSeedCompanyCustomFields = (
icon: 'IconAdCircle',
isActive: true,
isNullable: false,
isUnique: false,
defaultValue: "''",
objectMetadataId,
},
@ -27,6 +28,7 @@ export const getDevSeedCompanyCustomFields = (
icon: 'IconVideo',
isActive: true,
isNullable: true,
isUnique: false,
objectMetadataId,
},
{
@ -38,6 +40,7 @@ export const getDevSeedCompanyCustomFields = (
icon: 'IconHome',
isActive: true,
isNullable: true,
isUnique: false,
objectMetadataId,
options: [
{
@ -69,6 +72,7 @@ export const getDevSeedCompanyCustomFields = (
icon: 'IconBrandVisa',
isActive: true,
isNullable: true,
isUnique: false,
objectMetadataId,
defaultValue: false,
},
@ -89,6 +93,7 @@ export const getDevSeedPeopleCustomFields = (
icon: 'IconNote',
isActive: true,
isNullable: true,
isUnique: false,
objectMetadataId,
},
{
@ -100,6 +105,7 @@ export const getDevSeedPeopleCustomFields = (
icon: 'IconBrandWhatsapp',
isActive: true,
isNullable: false,
isUnique: false,
defaultValue: [
{
primaryPhoneNumber: '',
@ -118,6 +124,7 @@ export const getDevSeedPeopleCustomFields = (
icon: 'IconHome',
isActive: true,
isNullable: true,
isUnique: false,
objectMetadataId,
options: [
{
@ -149,6 +156,7 @@ export const getDevSeedPeopleCustomFields = (
icon: 'IconStars',
isActive: true,
isNullable: true,
isUnique: false,
objectMetadataId,
options: [
{

53
packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368824-migrationDebt.ts Normal file
View File

@ -0,0 +1,53 @@
import { MigrationInterface, QueryRunner } from 'typeorm';
export class MigrationDebt1726757368824 implements MigrationInterface {
name = 'MigrationDebt1726757368824';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TYPE "metadata"."relationMetadata_ondeleteaction_enum" RENAME TO "relationMetadata_ondeleteaction_enum_old"`,
);
await queryRunner.query(
`CREATE TYPE "metadata"."relationMetadata_ondeleteaction_enum" AS ENUM('CASCADE', 'RESTRICT', 'SET_NULL', 'NO_ACTION')`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."relationMetadata" ALTER COLUMN "onDeleteAction" DROP DEFAULT`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."relationMetadata" ALTER COLUMN "onDeleteAction" TYPE "metadata"."relationMetadata_ondeleteaction_enum" USING "onDeleteAction"::"text"::"metadata"."relationMetadata_ondeleteaction_enum"`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."relationMetadata" ALTER COLUMN "onDeleteAction" SET DEFAULT 'SET_NULL'`,
);
await queryRunner.query(
`DROP TYPE "metadata"."relationMetadata_ondeleteaction_enum_old"`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."workspaceMigration" ALTER COLUMN "name" SET NOT NULL`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."workspaceMigration" ALTER COLUMN "name" DROP NOT NULL`,
);
await queryRunner.query(
`CREATE TYPE "metadata"."relationMetadata_ondeleteaction_enum_old" AS ENUM('CASCADE', 'RESTRICT', 'SET_NULL')`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."relationMetadata" ALTER COLUMN "onDeleteAction" DROP DEFAULT`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."relationMetadata" ALTER COLUMN "onDeleteAction" TYPE "metadata"."relationMetadata_ondeleteaction_enum_old" USING "onDeleteAction"::"text"::"metadata"."relationMetadata_ondeleteaction_enum_old"`,
);
await queryRunner.query(
`ALTER TABLE "metadata"."relationMetadata" ALTER COLUMN "onDeleteAction" SET DEFAULT 'SET_NULL'`,
);
await queryRunner.query(
`DROP TYPE "metadata"."relationMetadata_ondeleteaction_enum"`,
);
await queryRunner.query(
`ALTER TYPE "metadata"."relationMetadata_ondeleteaction_enum_old" RENAME TO "relationMetadata_ondeleteaction_enum"`,
);
}
}

19
packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368825-addIsUniqueToIndexMetadata.ts Normal file
View File

@ -0,0 +1,19 @@
import { MigrationInterface, QueryRunner } from 'typeorm';
export class AddIsUniqueToIndexMetadata1726757368825
implements MigrationInterface
{
name = 'AddIsUniqueToIndexMetadata1726757368825';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."indexMetadata" ADD "isUnique" boolean NOT NULL DEFAULT false`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."indexMetadata" DROP COLUMN "isUnique"`,
);
}
}

19
packages/twenty-server/src/database/typeorm/metadata/migrations/1726766871572-addWhereToIndexMetadata.ts Normal file
View File

@ -0,0 +1,19 @@
import { MigrationInterface, QueryRunner } from 'typeorm';
export class AddWhereToIndexMetadata1726766871572
implements MigrationInterface
{
name = 'AddWhereToIndexMetadata1726766871572';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."indexMetadata" ADD "indexWhereClause" text`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."indexMetadata" DROP COLUMN "indexWhereClause"`,
);
}
}

17
packages/twenty-server/src/database/typeorm/metadata/migrations/1728563893694-addIsUniqueToFields.ts Normal file
View File

@ -0,0 +1,17 @@
import { MigrationInterface, QueryRunner } from 'typeorm';
export class AddIsUniqueToFields1728563893694 implements MigrationInterface {
name = 'AddIsUniqueToFields1728563893694';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."fieldMetadata" ADD "isUnique" boolean DEFAULT false`,
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "metadata"."indexMetadata" DROP COLUMN "indexWhereClause"`,
);
}
}