admin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Feat/twenty orm (#5153)

Browse Source 
## Introduction

This PR introduces "TwentyORM," a custom ORM module designed to
streamline database interactions within our workspace schema, reducing
the need for raw SQL queries. The API mirrors TypeORM's to provide a
familiar interface while integrating enhancements specific to our
project's needs.

To facilitate this integration, new decorators prefixed with `Workspace`
have been implemented. These decorators are used to define entity
metadata more explicitly and are critical in constructing our schema
dynamically.

## New Features

- **Custom ORM System**: Named "TwentyORM," which aligns closely with
TypeORM for ease of use but is tailored to our application's specific
requirements.
- **Decorator-Driven Configuration**: Entities are now configured with
`Workspace`-prefixed decorators that clearly define schema mappings and
relationships directly within the entity classes.
- **Injectable Repositories**: Repositories can be injected similarly to
TypeORM, allowing for flexible and straightforward data management.

## Example Implementations

### Decorated Entity Definitions

Entities are defined with new decorators that outline table and field
metadata, relationships, and constraints. Here are examples of these
implementations:

#### Company Metadata Object

```typescript
@WorkspaceObject({
  standardId: STANDARD_OBJECT_IDS.company,
  namePlural: 'companies',
  labelSingular: 'Company',
  labelPlural: 'Companies',
  description: 'A company',
  icon: 'IconBuildingSkyscraper',
})
export class CompanyObjectMetadata extends BaseObjectMetadata {
  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.name,
    type: FieldMetadataType.TEXT,
    label: 'Name',
    description: 'The company name',
    icon: 'IconBuildingSkyscraper',
  })
  name: string;

  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.xLink,
    type: FieldMetadataType.LINK,
    label: 'X',
    description: 'The company Twitter/X account',
    icon: 'IconBrandX',
  })
  @WorkspaceIsNullable()
  xLink: LinkMetadata;

  @WorkspaceField({
    standardId: COMPANY_STANDARD_FIELD_IDS.position,
    type: FieldMetadataType.POSITION,
    label: 'Position',
    description: 'Company record position',
    icon: 'IconHierarchy2',
  })
  @WorkspaceIsSystem()
  @WorkspaceIsNullable()
  position: number;

  @WorkspaceRelation({
    standardId: COMPANY_STANDARD_FIELD_IDS.accountOwner,
    label: 'Account Owner',
    description: 'Your team member responsible for managing the company account',
    type: RelationMetadataType.MANY_TO_ONE,
    inverseSideTarget: () => WorkspaceMemberObjectMetadata,
    inverseSideFieldKey: 'accountOwnerForCompanies',
    onDelete: RelationOnDeleteAction.SET_NULL,
  })
  @WorkspaceIsNullable()
  accountOwner: WorkspaceMemberObjectMetadata;
}
```

#### Workspace Member Metadata Object

```typescript
@WorkspaceObject({
  standardId: STANDARD_OBJECT_IDS.workspaceMember,
  namePlural: 'workspaceMembers',
  labelSingular: 'Workspace Member',
  labelPlural: 'Workspace Members',
  description: 'A workspace member',
  icon: 'IconUserCircle',
})
@WorkspaceIsSystem()
@WorkspaceIsNotAuditLogged()
export class WorkspaceMemberObjectMetadata extends BaseObjectMetadata {
  @WorkspaceField({
    standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.name,
    type: FieldMetadataType.FULL_NAME,
    label: 'Name',
    description: 'Workspace member name',
    icon: 'IconCircleUser',
  })
  name: FullNameMetadata;

  @WorkspaceRelation({
    standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.accountOwnerForCompanies,
    label: 'Account Owner For Companies',
    description: 'Account owner for companies',
    icon: 'IconBriefcase',
    type: RelationMetadataType.ONE_TO_MANY,
    inverseSideTarget: () => CompanyObjectMetadata,
    inverseSideFieldKey: 'accountOwner',
    onDelete: RelationOnDeleteAction.SET_NULL,
  })
  accountOwnerForCompanies: Relation

<CompanyObjectMetadata[]>;
}
```

### Injectable Repository Usage

Repositories can be directly injected into services, allowing for
streamlined query operations:

```typescript
export class CompanyService {
  constructor(
    @InjectWorkspaceRepository(CompanyObjectMetadata)
    private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    // Example queries demonstrating simple and relation-loaded operations
    const simpleCompanies = await this.companyObjectMetadataRepository.find({});
    const companiesWithOwners = await this.companyObjectMetadataRepository.find({
      relations: ['accountOwner'],
    });
    const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
      where: { xLinkLabel: 'MyLabel' },
    });

    return companiesFilteredByLinkLabel;
  }
}
```

## Conclusions

This PR sets the foundation for a decorator-driven ORM layer that
simplifies data interactions and supports complex entity relationships
while maintaining clean and manageable code architecture. This is not
finished yet, and should be extended.
All the standard objects needs to be migrated and all the module using
the old decorators too.

---------

Co-authored-by: Weiko <corentin@twenty.com>
This commit is contained in:
Jérémy M
2024-04-29 16:47:42 +02:00
committed by GitHub
parent 6e87554445
commit e2185448ed
42 changed files with 1597 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
packages/twenty-server/src/engine/twenty-orm/factories/entity-schema-column.factory.ts Normal file
View File

@ -0,0 +1,104 @@
import { Injectable } from '@nestjs/common';
import { ColumnType, EntitySchemaColumnOptions } from 'typeorm';
import { WorkspaceFieldMetadataArgs } from 'src/engine/twenty-orm/interfaces/workspace-field-metadata-args.interface';
import { fieldMetadataTypeToColumnType } from 'src/engine/metadata-modules/workspace-migration/utils/field-metadata-type-to-column-type.util';
import { isEnumFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-enum-field-metadata-type.util';
import { serializeDefaultValue } from 'src/engine/metadata-modules/field-metadata/utils/serialize-default-value';
import { computeCompositeColumnName } from 'src/engine/metadata-modules/field-metadata/utils/compute-column-name.util';
import { isCompositeFieldMetadataType } from 'src/engine/metadata-modules/field-metadata/utils/is-composite-field-metadata-type.util';
import { compositeTypeDefintions } from 'src/engine/metadata-modules/field-metadata/composite-types';
import { FieldMetadataType } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
type EntitySchemaColumnMap = {
[key: string]: EntitySchemaColumnOptions;
};
@Injectable()
export class EntitySchemaColumnFactory {
create(
fieldMetadataArgsCollection: WorkspaceFieldMetadataArgs[],
): EntitySchemaColumnMap {
let entitySchemaColumnMap: EntitySchemaColumnMap = {};
for (const fieldMetadataArgs of fieldMetadataArgsCollection) {
const key = fieldMetadataArgs.name;
if (isCompositeFieldMetadataType(fieldMetadataArgs.type)) {
const compositeColumns = this.createCompositeColumns(fieldMetadataArgs);
entitySchemaColumnMap = {
...entitySchemaColumnMap,
...compositeColumns,
};
continue;
}
const columnType = fieldMetadataTypeToColumnType(fieldMetadataArgs.type);
const defaultValue = serializeDefaultValue(
fieldMetadataArgs.defaultValue,
);
entitySchemaColumnMap[key] = {
name: key,
type: columnType as ColumnType,
primary: fieldMetadataArgs.isPrimary,
nullable: fieldMetadataArgs.isNullable,
createDate: key === 'createdAt',
updateDate: key === 'updatedAt',
array: fieldMetadataArgs.type === FieldMetadataType.MULTI_SELECT,
default: defaultValue,
};
if (isEnumFieldMetadataType(fieldMetadataArgs.type)) {
const values = fieldMetadataArgs.options?.map((option) => option.value);
if (values && values.length > 0) {
entitySchemaColumnMap[key].enum = values;
}
}
}
return entitySchemaColumnMap;
}
private createCompositeColumns(
fieldMetadataArgs: WorkspaceFieldMetadataArgs,
): EntitySchemaColumnMap {
const entitySchemaColumnMap: EntitySchemaColumnMap = {};
const compositeType = compositeTypeDefintions.get(fieldMetadataArgs.type);
if (!compositeType) {
throw new Error(
`Composite type ${fieldMetadataArgs.type} is not defined in compositeTypeDefintions`,
);
}
for (const compositeProperty of compositeType.properties) {
const columnName = computeCompositeColumnName(
fieldMetadataArgs.name,
compositeProperty,
);
const columnType = fieldMetadataTypeToColumnType(compositeProperty.type);
const defaultValue = serializeDefaultValue(
fieldMetadataArgs.defaultValue?.[compositeProperty.name],
);
entitySchemaColumnMap[columnName] = {
name: columnName,
type: columnType as ColumnType,
nullable: compositeProperty.isRequired,
default: defaultValue,
};
if (isEnumFieldMetadataType(compositeProperty.type)) {
throw new Error('Enum composite properties are not yet supported');
}
}
return entitySchemaColumnMap;
}
}

61
packages/twenty-server/src/engine/twenty-orm/factories/entity-schema-relation.factory.ts Normal file
View File

@ -0,0 +1,61 @@
import { Injectable } from '@nestjs/common';
import { EntitySchemaRelationOptions } from 'typeorm';
import { RelationType } from 'typeorm/metadata/types/RelationTypes';
import { WorkspaceRelationMetadataArgs } from 'src/engine/twenty-orm/interfaces/workspace-relation-metadata-args.interface';
import { convertClassNameToObjectMetadataName } from 'src/engine/workspace-manager/workspace-sync-metadata/utils/convert-class-to-object-metadata-name.util';
import { RelationMetadataType } from 'src/engine/metadata-modules/relation-metadata/relation-metadata.entity';
type EntitySchemaRelationMap = {
[key: string]: EntitySchemaRelationOptions;
};
@Injectable()
export class EntitySchemaRelationFactory {
create(
relationMetadataArgsCollection: WorkspaceRelationMetadataArgs[],
): EntitySchemaRelationMap {
const entitySchemaRelationMap: EntitySchemaRelationMap = {};
for (const relationMetadataArgs of relationMetadataArgsCollection) {
const oppositeTarget = relationMetadataArgs.inverseSideTarget();
const oppositeObjectName = convertClassNameToObjectMetadataName(
oppositeTarget.name,
);
const relationType = this.getRelationType(relationMetadataArgs);
entitySchemaRelationMap[relationMetadataArgs.name] = {
type: relationType,
target: oppositeObjectName,
inverseSide: relationMetadataArgs.inverseSideFieldKey,
joinColumn: relationMetadataArgs.joinColumn
? {
name: relationMetadataArgs.joinColumn,
}
: undefined,
};
}
return entitySchemaRelationMap;
}
private getRelationType(
relationMetadataArgs: WorkspaceRelationMetadataArgs,
): RelationType {
switch (relationMetadataArgs.type) {
case RelationMetadataType.ONE_TO_MANY:
return 'one-to-many';
case RelationMetadataType.MANY_TO_ONE:
return 'many-to-one';
case RelationMetadataType.ONE_TO_ONE:
return 'one-to-one';
case RelationMetadataType.MANY_TO_MANY:
return 'many-to-many';
default:
throw new Error('Invalid relation type');
}
}
}

43
packages/twenty-server/src/engine/twenty-orm/factories/entity-schema.factory.ts Normal file
View File

@ -0,0 +1,43 @@
import { Injectable, Type } from '@nestjs/common';
import { EntitySchema } from 'typeorm';
import { EntitySchemaColumnFactory } from 'src/engine/twenty-orm/factories/entity-schema-column.factory';
import { EntitySchemaRelationFactory } from 'src/engine/twenty-orm/factories/entity-schema-relation.factory';
import { metadataArgsStorage } from 'src/engine/twenty-orm/storage/metadata-args.storage';
@Injectable()
export class EntitySchemaFactory {
constructor(
private readonly entitySchemaColumnFactory: EntitySchemaColumnFactory,
private readonly entitySchemaRelationFactory: EntitySchemaRelationFactory,
) {}
create<T>(target: Type<T>): EntitySchema {
const objectMetadataArgs = metadataArgsStorage.filterObjects(target);
if (!objectMetadataArgs) {
throw new Error('Object metadata args are missing on this target');
}
const fieldMetadataArgsCollection =
metadataArgsStorage.filterFields(target);
const relationMetadataArgsCollection =
metadataArgsStorage.filterRelations(target);
const columns = this.entitySchemaColumnFactory.create(
fieldMetadataArgsCollection,
);
const relations = this.entitySchemaRelationFactory.create(
relationMetadataArgsCollection,
);
return new EntitySchema({
name: objectMetadataArgs.nameSingular,
tableName: objectMetadataArgs.nameSingular,
columns,
relations,
});
}
}

11
packages/twenty-server/src/engine/twenty-orm/factories/index.ts Normal file
View File

@ -0,0 +1,11 @@
import { EntitySchemaColumnFactory } from 'src/engine/twenty-orm/factories/entity-schema-column.factory';
import { EntitySchemaRelationFactory } from 'src/engine/twenty-orm/factories/entity-schema-relation.factory';
import { EntitySchemaFactory } from 'src/engine/twenty-orm/factories/entity-schema.factory';
import { WorkspaceDatasourceFactory } from 'src/engine/twenty-orm/factories/workspace-datasource.factory';
export const entitySchemaFactories = [
EntitySchemaColumnFactory,
EntitySchemaRelationFactory,
EntitySchemaFactory,
WorkspaceDatasourceFactory,
];

58
packages/twenty-server/src/engine/twenty-orm/factories/workspace-datasource.factory.ts Normal file
View File

@ -0,0 +1,58 @@
import { Inject, Injectable, Scope } from '@nestjs/common';
import { REQUEST } from '@nestjs/core';
import { DataSource, EntitySchema } from 'typeorm';
import { DataSourceService } from 'src/engine/metadata-modules/data-source/data-source.service';
import { EnvironmentService } from 'src/engine/integrations/environment/environment.service';
import { Workspace } from 'src/engine/core-modules/workspace/workspace.entity';
import { DataSourceStorage } from 'src/engine/twenty-orm/storage/data-source.storage';
@Injectable({ scope: Scope.REQUEST })
export class WorkspaceDatasourceFactory {
constructor(
@Inject(REQUEST) private readonly request: Request,
private readonly dataSourceService: DataSourceService,
private readonly environmentService: EnvironmentService,
) {}
public async createWorkspaceDatasource(entities: EntitySchema[]) {
const workspace: Workspace = this.request['req']['workspace'];
if (!workspace) {
return null;
}
const storedWorkspaceDataSource = DataSourceStorage.getDataSource(
workspace.id,
);
if (storedWorkspaceDataSource) {
return storedWorkspaceDataSource;
}
const dataSourceMetadata =
await this.dataSourceService.getLastDataSourceMetadataFromWorkspaceIdOrFail(
workspace.id,
);
const workspaceDataSource = new DataSource({
url:
dataSourceMetadata.url ??
this.environmentService.get('PG_DATABASE_URL'),
type: 'postgres',
// logging: this.environmentService.get('DEBUG_MODE')
// ? ['query', 'error']
// : ['error'],
logging: 'all',
schema: dataSourceMetadata.schema,
entities,
});
await workspaceDataSource.initialize();
DataSourceStorage.setDataSource(workspace.id, workspaceDataSource);
return workspaceDataSource;
}
}