# Introduction
Following https://github.com/twentyhq/twenty/pull/13264, this PR
introduces several `fieldMetadataEntity` typing enhancement suggestions.
Mainly any nullable field metadata entity properties are now either
nullable or defined.
Or never if field is dynamically required or not depending on the field
metadata type
This enhance DevX
## Standards
- field enum ( `MULTI_SELECT`, `SELECT`, `RATING` ) will never have
`options` set to `NULL` in db
- field `RELATION` or `MORH_RELATION` won't ever have its relation
fields set to `NULL` in db
- field of any type `settings`, even if possibly defined, can still be
`NULL` in db
- field of any type `defaultValue`, even if possibly defined, can still
be `NULL` in db
It coud be interesting to guard these standards by adding dedicated pg
constraints on each field
## TypesScript type tests
added coverage for each `settings`, `defaultValue`, and `options`
depending on the current `fieldMetadata`
Honestly I don' know if this typescript assertions test file is not
overkill, but regarding metadata staticness it might be very interesting
to have this guard
## Possible improvements
- We could type as `unknown` instead of "all" on `FieldMetadataType`
inferrance
- We still need to deprecate remaining duplicated entities such as
`Index/Field/MetadataInterface` etc not a huge refactor neither urgent
# Introduction
Following https://github.com/twentyhq/twenty/pull/13264
> After this PR merge will create a new one removing the type and
replacing it to FieldMetadataEntity.
This is it !
# Introduction
From the moment replaced the FieldMetadataInterface definition to:
```ts
import { FieldMetadataType } from 'twenty-shared/types';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
export type FieldMetadataInterface<
T extends FieldMetadataType = FieldMetadataType,
> = FieldMetadataEntity<T>;
```
After this PR merge will create a new one removing the type and
replacing it to `FieldMetadataEntity`.
Did not renamed it here to avoid conflicts on naming + type issues fixs
within the same PR
## Field metadata entity RELATION or MORPH
Relations fields cannot be null for those field metadata entity instance
anymore, but are never for the others see
`packages/twenty-server/src/engine/metadata-modules/field-metadata/types/field-metadata-entity-test.type.ts`
( introduced TypeScript tests )
## Concerns
- TS_VECTOR is the most at risk with the `generatedType` and
`asExpression` removal from interface
## What's next
- `FielMetadataInterface` removal and rename ( see introduction )
- Depcrecating `ObjectMetadataInterface`
- Refactor `FieldMetadataEntity` optional fiels to be nullable only
- TO DIG `never` occurences on settings, defaultValue etc
- Some interfaces will be replaced by the `FlatFieldMetadata` when
deprecating the current sync and comparators tools
This PR will create the migration to be run for the morph relations
- We created a dedicated util to generate the column name and refactored
a little the code in order to have less dependencies and a clearer devX
(updated the snapshot that changed because of this)
- Moved the `createMigrationActions` to its own util as well
- Created the `MorphRelationColumnActionFactory` based on the relation
one
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
resolve#12345
The issue was caused by the delete running after the update, which led
to both the old and new options being deleted when they shared the same
value.
---------
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
_(AI generated)_
### Summary
This PR fixes a validation bug in the GraphQL API that prevented
relation fields from being programmatically deactivated. The validation
was incorrectly triggering a "name cannot be changed" error even when
the update payload did not include a name, making it impossible to
disable the field.
Issue #13200
### Problem
- The [updateOneField] mutation failed when trying to set `isActive:
false` on a `RELATION` field.
- The root cause was a validation check in
[FieldMetadataValidationService] that compared the incoming `name` with
the existing one. If the input `name` was `undefined`, the check
`undefined !== existingName` would incorrectly fail.
- This created a catch-22 where a field could not be deleted (because it
had to be deactivated first) and could not be deactivated (due to this
validation error).
### Solution
- The validation logic in [field-metadata-validation.service.ts] has
been updated to only check for a name change if a new name is
**explicitly provided** in the input
(`isDefined(fieldMetadataInput.name)`).
- This change correctly enforces the rule that relation field names
cannot be changed, while allowing other properties like `isActive` to be
updated without issue.
### How to Test
1. Create a custom field of type `RELATION`.
2. Using the GraphQL API, call the [updateOneField] mutation with the
field's ID and the payload `{ "isActive": false }`.
3. Verify that the mutation succeeds and the field is now inactive.
4. Call the [deleteOneField] mutation to delete the field.
5. Verify that the deletion is successful.
### Additional Changes
_to be deleted if not necessary_
- Added a new integration test
[successful-field-metadata-relation-update.integration-spec.ts] to cover
this specific use case and prevent future regressions. The existing test
for failing updates remains untouched and continues to pass.
Fixes https://github.com/twentyhq/twenty/issues/12867
Issue:
when you have a variable `toto` which is: `Record<string, MyType>` and
you do toto['xxx'], this will be typed as `MyType` instead of `MyType |
undefined`
Solutions:
- activate `noUncheckedIndexedAccess` check in tsconfig, this is the
preferred solution but will take time to get there (this raises 600+
errors)
- use a Map: cf https://github.com/twentyhq/twenty/pull/13125/files
- set the type to Partial<Record<string, MyType>>. Drawback is that when
you do Object.values(toto), you'll get `Array<MyType | undefined>`.
Hence why we have to filter these behind
<img width="1512" alt="image"
src="https://github.com/user-attachments/assets/d0a0bfed-c441-4e53-84c2-2da98ccbcf50"
/>
## Expected behavior
Described behavior regarding: (update | create) x (custom | standard) x
(icon, label, name, isSynced)
**Custom:**
- Field RELATION create: name, label, isSynced, icon should be editable
- Field RELATION update: name should not, icon label, isSynced should
- For other fields, icon, label, name, isSynced should be editable at
field creation | update
To simplify: Field RELATION name should not be editable at update
**Standards**
- Field: create does not makes sense
- Field: name should not, icon label, isSynced should (this will end up
in overrides)
To simplify, no Field RELATION edge case, name should not be editable at
update
**Note:** the FE logic is quite different as the UI is hiding some
details behind the syncWithLabel. See my comments and TODO there
## What I've tested:
(update | create) x (custom | standard) x (icon, label, name, isSynced,
description)
Currently, when a server query or mutation from the front-end fails, the
error message defined server-side is displayed in a snackbar in the
front-end.
These error messages usually contain technical details that don't belong
to the user interface, such as "ObjectMetadataCollection not found" or
"invalid ENUM value for ...".
**BE**
In addition to the original error message that is still needed (for the
request response, debugging, sentry monitoring etc.), we add a
`displayedErrorMessage` that will be used in the snackbars. It's only
relevant to add it for the messages that will reach the FE (ie. not in
jobs or in rest api for instance) and if it can help the user sort out /
fix things (ie. we do add displayedErrorMessage for "Cannot create
multiple draft versions for the same workflow" or "Cannot delete
[field], please update the label identifier field first", but not
"Object metadata does not exist"), even if in practice in the FE users
should not be able to perform an action that will not work (ie should
not be able to save creation of multiple draft versions of the same
workflows).
**FE**
To ease the usage we replaced enqueueSnackBar with enqueueErrorSnackBar
and enqueueSuccessSnackBar with an api that only requires to pass on the
error.
If no displayedErrorMessage is specified then the default error message
is `An error occured.`
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
# Introduction
Diff description: ~500 tests and +500 additions
close https://github.com/twentyhq/core-team-issues/issues/731
## What has been done here
In a nutshell on a field metadata type ( `SELECT MULTI_SELECT` ) update,
we will be browsing all `ViewFilters` in a post hook searching for some
referencing related updated `fieldMetadata` select. In order to update
or delete the `viewFilter` depending on the associated mutations.
## How to test:
- Add FieldMetadata `SELECT | MULTI_SELECT` to an existing or a new
`objectMetadata`
- Create a filtered view on created `fieldMetadata` with any options you
would like
- Remove some options ( in the best of the world some that are selected
by the filter ) from the `fieldMetadata` settings page
- Go back to the filtered view, removed or updated options should have
been hydrated in the `displayValue` and the filtered data should make
sense
## All filtered options are deleted edge case
If an update implies that a viewFilter does not have any existing
related options anymore, then we remove the viewFilter
## Testing
```sh
PASS test/integration/metadata/suites/field-metadata/update-one-field-metadata-related-record.integration-spec.ts (27 s)
update-one-field-metadata-related-record
SELECT
✓ should delete related view filter if all select field options got deleted (2799 ms)
✓ should update related multi selected options view filter (1244 ms)
✓ should update related solo selected option view filter (1235 ms)
✓ should handle partial deletion of selected options in view filter (1210 ms)
✓ should handle reordering of options while maintaining view filter values (1487 ms)
✓ should handle no changes update of options while maintaining existing view filter values (1174 ms)
✓ should handle adding new options while maintaining existing view filter (1174 ms)
✓ should update display value with options label if less than 3 options are selected (1249 ms)
✓ should throw error if view filter value is not a stringified JSON array (1300 ms)
MULTI_SELECT
✓ should delete related view filter if all select field options got deleted (1127 ms)
✓ should update related multi selected options view filter (1215 ms)
✓ should update related solo selected option view filter (1404 ms)
✓ should handle partial deletion of selected options in view filter (1936 ms)
✓ should handle reordering of options while maintaining view filter values (1261 ms)
✓ should handle no changes update of options while maintaining existing view filter values (1831 ms)
✓ should handle adding new options while maintaining existing view filter (1610 ms)
✓ should update display value with options label if less than 3 options are selected (1889 ms)
✓ should throw error if view filter value is not a stringified JSON array (1365 ms)
Test Suites: 1 passed, 1 total
Tests: 18 passed, 18 total
Snapshots: 18 passed, 18 total
Time: 27.039 s
```
## Out of scope
- We should handle ViewFilter validation when extracting its definition
from the metadata
https://github.com/twentyhq/core-team-issues/issues/1009
## Concerns
- Are we able through the api to update an RATING fieldMetadata ? ( if
yes than that's an issue and we should handle RATING the same way than
for SELECT and MULTI_SELECT )
- It's not possible to group a view from a MULTI_SELECT field
The above points create a double nor a triple "lecture" to the post hook
effect:
- ViewGroup -> only SELECT
- VIewFilter -> only SELECT || MULTI_SELECT
- Rating nothing
I think we should determine the scope of all of that
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
Big diff a lot of tests and snapshots ( real diff < 500+ )
close https://github.com/twentyhq/twenty/issues/12117
close https://github.com/twentyhq/twenty/issues/12133
## What has been done here
Implemented a strong integration coverage on both fieldmetadata`SELECT`
`UPDATE` and `CREATE`.
Implemented server side validation for the options `value` `label` `id`
and collision issue with also `position`
We could improve:
- Position validation
- DefaultValue validation
## Update
```ts
PASS test/integration/metadata/suites/field-metadata/update-one-field-metadata-select.integration-spec.ts (41.054 s)
Field metadata select update tests group
✓ Update should succeed with provided option id (2565 ms)
✓ Update should succeed with valid default value (1469 ms)
✓ Update should succeed with various options id (1257 ms)
✓ Update should succeed without option id (1286 ms)
✓ Update should trim option values (1366 ms)
✓ Update should succeed with default value and no options (1122 ms)
✓ Update should fail with unknown default value and no options (1075 ms)
✓ Update should fail with only white spaces id (1195 ms)
✓ Update should fail with empty string id (1058 ms)
✓ Update should fail with null id (1066 ms)
✓ Update should fail with not a string id (1098 ms)
✓ Update should fail with too long id (1373 ms)
✓ Update should fail with only white spaces label (1034 ms)
✓ Update should fail with empty string label (1057 ms)
✓ Update should fail with null label (1100 ms)
✓ Update should fail with not a string label (1144 ms)
✓ Update should fail with too long label (1273 ms)
✓ Update should fail with only white spaces value (1385 ms)
✓ Update should fail with empty string value (1035 ms)
✓ Update should fail with null value (1068 ms)
✓ Update should fail with not a string value (1021 ms)
✓ Update should fail with too long value (1134 ms)
✓ Update should fail with invalid option id (1137 ms)
✓ Update should fail with empty options (1238 ms)
✓ Update should fail with invalid option value format (1104 ms)
✓ Update should fail with comma in option label (1004 ms)
✓ Update should fail with duplicated option values (1015 ms)
✓ Update should fail with duplicated option ids (1079 ms)
✓ Update should fail with duplicated option positions (1266 ms)
✓ Update should fail with duplicated trimmed option values (1220 ms)
✓ Update should fail with undefined option label (1029 ms)
✓ Update should fail with an invalid default value (1142 ms)
✓ Update should fail with an unknown default value (1081 ms)
✓ Update should fail with undefined option value (1086 ms)
Test Suites: 1 passed, 1 total
Tests: 34 passed, 34 total
Snapshots: 28 passed, 28 total
Time: 41.079 s
```
## Create
```ts
PASS test/integration/metadata/suites/field-metadata/create-one-field-metadata-select.integration-spec.ts (38.292 s)
Field metadata select creation tests group
✓ Create should succeed with provided option id (2096 ms)
✓ Create should succeed with valid default value (1316 ms)
✓ Create should succeed with various options id (1113 ms)
✓ Create should succeed without option id (1378 ms)
✓ Create should trim option values (1296 ms)
✓ Create should fail with only white spaces id (1000 ms)
✓ Create should fail with empty string id (1325 ms)
✓ Create should fail with null id (1060 ms)
✓ Create should fail with not a string id (1142 ms)
✓ Create should fail with too long id (1321 ms)
✓ Create should fail with only white spaces label (999 ms)
✓ Create should fail with empty string label (1163 ms)
✓ Create should fail with null label (1198 ms)
✓ Create should fail with not a string label (1678 ms)
✓ Create should fail with too long label (1527 ms)
✓ Create should fail with only white spaces value (1200 ms)
✓ Create should fail with empty string value (1102 ms)
✓ Create should fail with null value (1037 ms)
✓ Create should fail with not a string value (1462 ms)
✓ Create should fail with too long value (896 ms)
✓ Create should fail with invalid option id (997 ms)
✓ Create should fail with empty options (1058 ms)
✓ Create should fail with invalid option value format (1190 ms)
✓ Create should fail with comma in option label (1142 ms)
✓ Create should fail with duplicated option values (872 ms)
✓ Create should fail with duplicated option ids (860 ms)
✓ Create should fail with duplicated option positions (1002 ms)
✓ Create should fail with duplicated trimmed option values (1336 ms)
✓ Create should fail with undefined option label (754 ms)
✓ Create should fail with an invalid default value (696 ms)
✓ Create should fail with an unknown default value (678 ms)
✓ Create should fail with undefined option value (699 ms)
✓ Create should fail with null options (720 ms)
✓ Create should fail with undefined options (686 ms)
Test Suites: 1 passed, 1 total
Tests: 34 passed, 34 total
Snapshots: 29 passed, 29 total
Time: 38.314 s
```
## Conclusion
As always any suggestions are welcomed ! Please let me know
## Discussion about validation governance
### Front
Front side will be dealing with zod validations schema that he will
handle and maintain by himself
### Back validation instances
- Validation hold through DTO declarations ( run by yoga through the
resolvers )
- Server programmatic validation and exceptions handling ( run through
the services )
For this refactor/fix we decided to stick to the current implementation
only touching the `Server programmatic validation and exceptions
handling` we will handle validation centralization when we will onboard
the `nestjs-query` deprecation/integration refactor.
### Vision
In the best of the world we could think of an intermediary model that
will handle and take responsibility of the validation decorators that
would be run programmatically through the service, Yoga would still
consume it ? then we would need to have enough grain in the service to
know the input has already validated
## Notes
Introduced zod back side in order to handle very atomic and primitive
validation
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747
We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.
In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
`No Value` view groups wasn't properly created when we select a group by
field metadata, this PR fix the issue.
Also a script is added to backfill the current view groups.
---------
Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
This PR is fixing the following issues with record groups:
- [x] [Backend] - Only update view groups when a field is edited if this
one already has view groups
- [x] [Backend] - Editing a Select field metadata option brake view
groups
- [x] [Frontend] - Changing the group by field from one to another brake
record group and doesn't remove the previous ones
- [x] [Frontend & Backend] - Mark `kanbanFieldMetadataId` as deprecated
in favour of `viewGroups.fieldMetadataId`
Also the following has been checked:
- [x] Properly displayed a table with only one view groups
- [x] Properly displayed a table without view groups
