Context :
- IndexFieldMetadata was no longer available on 'objects' gql query
([since this PR](https://github.com/twentyhq/twenty/pull/12785)). Then,
unicity checks on import do not work anymore.
Fix :
- Add a dataloader logic in indexFieldMetadata
- Add extra check in unicity hook on import
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
## Introduction
For a custom object if the selected identifier field metadata is an
number type than it wouldn't get be converted to a string
#closes https://github.com/twentyhq/twenty/issues/12717
## Concerns
Kinda the same than for https://github.com/twentyhq/twenty/pull/12728
Here ObjectRecord unknown fields are typed as any, we might wanna do a
poc in order to migrate to `unknown` usage
```ts
import { BaseObjectRecord } from '@/object-record/types/BaseObjectRecord';
export type ObjectRecord = Record<string, any> & BaseObjectRecord;
```
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Introduction
closes https://github.com/twentyhq/core-team-issues/issues/591
Same than for `twenty-shared` made in
https://github.com/twentyhq/twenty/pull/11083.
## TODO
- [x] Manual migrate twenty-website twenty-ui imports
## What's next:
- Generate barrel and migration script factorization within own package
+ tests
- Refactoring using preconstruct ? TimeBox
- Lint circular dependencies
- Lint import from barrel and forbid them
### Preconstruct
We need custom rollup plugins addition, but preconstruct does not expose
its rollup configuration. It might be possible to handle this using the
babel overrides. But was a big tunnel.
We could give it a try afterwards ! ( allowing cjs interop and stuff
like that )
Stuck to vite lib app
Closed related PRs:
- https://github.com/twentyhq/twenty/pull/11294
- https://github.com/twentyhq/twenty/pull/11203
In this PR, I'm:
- fixing the root cause (we should not try to render a RecordChip if the
record is not defined in RelationFromMany Display)
- fixing related typing issues
- we won't be able to catch the issue from TS perspective as
ObjectRecord is a Record of string, any
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
This PR mainly fixes advanced filters on kanban view.
It also fixes various bugs and cleans some old states.
## Advanced filters on kanban views
Kanban views use a different hook to retrieve data from the backend :
useLoadRecordIndexBoardColumn, this hook wasn't using the new state
currentRecordFilterGroupsComponentState.
## Removal of confusing duplicate states
A few different states were used for filters and states, where we only
need one for filters and one for sorts for all indexes. So we remove
here the different states that can lead to confusion about what state
should be used in what case.
States removed :
- recordIndexFilterState
- recordIndexSortState
- recordIndexViewFilterGroupsState
- tableFiltersComponentState
- tableSortsComponentState
We also remove the logic that was used to manage those states.
## Abstracted non composite field type check into a util
We abstract the check made in mapFieldMetadataToGraphQLQuery into a util
isNonCompositeField, because those kinds of checks should be stored into
a separate unique file that acts as a source of truth.
## Bug with advanced filter rule position not saved
The position of an advanced filter rule wasn't correctly saved in the
backend, here we remove the WorkspaceIsSystem decorator on the
positionInViewFilterGroup fields.
The function that saved view filters was also ignoring the field
positionInViewFilterGroup, we add it back.
## Bug with view picker option dropdown closing weirdly
The view picker option dropdown was closing as soon as we hovered
outside of the option dropdown, which was annoying for the user, here we
apply the same behavior as every dropdown in the app : closing on click
outside.
Closes https://github.com/twentyhq/core-team-issues/issues/545
This PR:
- Introduces `commandMenuNavigationMorphItemsState` which stores the
information about the `recordId` and the `objectMetadataItemId` for each
page
- Creates `CommandMenuContextChipEffect`, which queries the records from
the previous pages in case a record has been updated during the
navigation, to keep up to date information and stores it inside
`commandMenuNavigationRecordsState`
- `useCommandMenuContextChips` returns the context chips information
- Style updates (icons background and color)
- Updates `useCommandMenu` to set and reset these new states
https://github.com/user-attachments/assets/8886848a-721d-4709-9330-8e84ebc0d51e
Refactor to only have MultipleRecordPicker and SingleRecordPicker
What's done:
- SingleRecordPicker, MultipleRecordPicker
- RelationToOneInput
- RelationFromManyInput
- usage in TableCell, InlineCell, RelationDetailSection, Workflow
What's left:
- Make a pass on the app, to make sure the hotkeyScopes, clickOutside
are properly set
- Fix flashing on ActivityTarget
- add more tests on the code
# Introduction
In this PR https://github.com/twentyhq/twenty/pull/10493 introduced a
regression on optimistic cache for record creation, by expecting a
`position` `fieldMetadataItem` on every `ObjectMetadataItem` which is a
wrong assertion
Some `Tasks` and `ApiKeys` do not have one
## Fix
Dynamically compute optimistic record input position depending on
current `ObjectMetadataItem`
## Refactor
Refactored a failing test following [jest
each](https://jestjs.io/docs/api#describeeachtablename-fn-timeout)
pattern to avoid error prone duplicated env tests. Created a "standard'
and applied it to an other test also using `it.each`.
# Context
To enable search records sorting by ts_rank_cd / ts_rank, we have
decided to add a new search resolver serving `GlobalSearchRecordDTO`.
-----
- [x] Test to add - work in progress
closes https://github.com/twentyhq/core-team-issues/issues/357
This PR implements a parallel code path to set record sorts without
impacting the actual sort, like we did on record filter.
We add a ViewBarRecordSortEffect which mirrors
ViewBarRecordFilterEffect.
It also adds availableFieldMetadataItemsForSortFamilySelector to replace
sortDefinitions progressively.
# Introduction
While importing records encountering missing expected fields when
writting a fragment from apollo cache
## Updates
### 1/ `createdBy` Default value
When inserting in cache in create single or many we will now make
optimistic behavior on the createdBy value
### 2/ `createRecordInCache` dynamically create `recordGrqlFields`
When creating an entry in cache, we will now dynamically generate fields
to be written in the fragment instead of expecting all of them. As by
nature record could be partial
### 3/ Strictly typed `RecordGqlFields`
# Conclusion
closes#9927
This PR progressively introduces fieldMetadataItemUsedInDropdown instead
of filterDefinitionUsedInDropdown where most easy to replace.
This allows to use `fieldMetadataItemUsedInDropdown.id` instead of
`filterDefinition.fieldMetadataId`, which is one easy dependency to
remove on filter definition.
We still derive filterDefinition instead of fully replacing it, because
it will be easier to remove RecordFilterDefinition usage in a bottom-up
approach instead.
In multiple components of the filter dropdown, we try to replace
filterDefinition by fieldMetadataItem derivation : Icon, label, id,
type, etc.
We also introduce the usage of subFieldNameUsedInDropdown instead of
storing it dynamically on filterDefinition, for handling filtering on
composite sub fields.
The method `formatFieldMetadataItemAsFilterDefinition()` that is used to
derive filterDefinition from fieldMetadataItem is what was being used
originally to create the availableFilterDefinition state. (That is
already removed)
Fixed associated unit tests accordingly.
The global record filter refactor will derive everything at runtime from
objectMetadataItemsState, thus removing the need for a filter definition
concept.
Here we don't yet remove available filter definition usage but we
replace the available filter definitions states, we now derive the same
value from objectMetadataItemsState.
This will allow us to progressively remove the usage of the concept of
filter definition, at the end it will then be easy to just remove from
the codebase because nothing will use it anymore.
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function
## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place
Related PR https://github.com/twentyhq/twenty/pull/9941
This PR doesn't remove or change the current behavior of the filter
definition used in filter dropdown, but adds a parallel code path where
we set the field metadata item used in filter dropdown, which is enough
to replace the filter definition.
The goal at the end is to compute dynamically the equivalent of filter
definition where needed, by deriving from objectMetadataItems global
state + fieldMetadataItemId used in dropdown, that way we don't create
any other source of truth for the concept of filter definition and
everything is easier to work with, especially with advanced filters.
The general spirit is that it's always better to derive everywhere from
a unique state as much as possible, and only create the equivalent of
selectors where needed that will only take the relevant chunk of state
for the small zone of the code operating some reading/writing.
- Added utils and hooks to get a FieldMetadataItem more easily
- Removed some properties from RecordFilterDefinition (the easiest to
remove) and replaced them with a dynamic logic, deriving what's needed
where it is needed
- Added a new fieldMetadataItemIdUsedInDropdownComponentState that is
set in parallel of filterDefinitionUsedInDropdown (to prepare the
removal of filter definition used in dropdown)
- Fixed some stories
---------
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
Implementing the Outlook icon for CreatedBy, only for emails.
Not in this PR original scope : The similar feature for calendar created
records. Since it was straightforward, I added it to the scope of this
PR.
Fix https://github.com/twentyhq/core-team-issues/issues/252
# Introduction
By initially fixing this Fixes#9381, discovered other behavior that
have been fix.
Overall we encountered a bug that corrupts a workspace and make the
browser + api crash
This issue https://github.com/twentyhq/core-team-issues/issues/25
suggests a refactor that has final save button instead of auto-save
## `labelIdentifierFieldMetadataId` form default value
The default value resulted in being undefined, resulting in react hook
form `labelIdentifierFieldMetadataId` is required field error.
### Fix
Setting default value fallback to `null` as field is `nullable`
## `SettingsDataModelObjectSettingsFormCard` never triggers form
Unless I'm mistaken in production touching any fields within
`SettingsDataModelObjectSettingsFormCard` would never trigger form
submission until you also modify `SettingsDataModelObjectAboutForm`
fields
### Fix
Provide and apply `onblur` that triggers the form on both
`SettingsDataModelObjectSettingsFormCard` inputs
## Wrong default `labelIdentifierFieldMetadataItem` on first page render
When landing on the page for the first time, if a custom
`labelIdentifierFieldMetadataItem` has been set it won't be computed
within the `PreviewCard`.
Occurs when `labelIdentifierFieldMetadataId` form default value is
undefined, due to `any` injection.
### Fix
In the `getLabelIdentifierFieldMetadataItem` check the
`labelIdentifierFieldMetadataIdFormValue` definition, if undefined
fallback to current `objectMetadata` identifier
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR is only moving and renaming types, hooks and utils to
record-filter module folder.
- Moved and renamed types from object filter modules to record filter…-
Moved and renamed types from object filter modules to record filter
module
- Moved useApplyRecordFilter to record filter module
- Renamed util getOperandsForFilterDefinition to
getRecordFilterOperandsForRecordFilterDefinition
# Introduction
For motivations and context please have a look to
https://github.com/twentyhq/twenty/pull/9394 whom this PR results from.
In this pull-request we remove any `metadataField` and `objectMetadata`
sluggification. We directly consume `objectMetadata.namePlural` and
`metadataField.name`, ***it seems like that historically the consumed
`metadataField.name`*** are we sure that we wanna change this behavior ?
## Notes
Unless I'm mistaken by reverting the `kebabcase` url formatting we might
be creating deadlinks that user could have save beforehand => Discussed
with Charles said it's controlled risk.
---------
Co-authored-by: Paul Rastoin <paulrastoin@Pauls-MacBook-Pro.local>
Fixes#8601
We had 3 implementations of getImageAbsoluteURI: in twenty-front, in
twenty-ui and in twenty-emails. I was able to remove the one in
twenty-front but I could not remove it from twenty-emails as this is a
standalone for now. The vision is to introduce shared utils in a
twenty-shared package
## Context
The recent addition of object renaming introduced issues with enum
names. Enum names should follow the pattern
`${schemaName}.${tableName}_${columnName}_enum`. To address this, and to
allow users to customize the API name (which is included in the enum
name, columnName), this PR implements behavior similar to object
renaming by introducing a `isLabelSyncedWithName` boolean.
<img width="624" alt="Screenshot 2024-12-02 at 11 58 49"
src="https://github.com/user-attachments/assets/690fb71c-83f0-4922-80c0-946c92dacc30">
<img width="596" alt="Screenshot 2024-12-02 at 11 58 39"
src="https://github.com/user-attachments/assets/af9a0037-7cf5-40c3-9ed5-d51b340c8087">
fix#8204
I changed "API keys" to "API values".
Stopped inputting special characters in Select field option keys.
@lucasbordeau please check the changes and tell me if I need to do any
other changes. :)
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
