Moves system-level operations (auth, billing, admin) to use the
/metadata endpoint instead of /graphql.
This cleans up the endpoint separation so /graphql is purely for core
objects (Company, People, etc.) and /metadata handles all system
operations.
Part of prep work for webhook/API key core migration.
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
## Introduction
Added coverage on the `useDeleteOneRecord` hooks, especially its
optimistic behavior feature.
Introduced a new testing tool `InMemoryTestingCacheInstance` that has
builtin very basic expectors in order to avoid future duplication when
covering others record hooks `update, create, destroy` etc etc
## Notes
Added few comments in this PR regarding some builtin functions I've
created around companies and people mocked object model and that I think
could be cool to spread and centralize within a dedicated "class
template"
Also put in light that unless I'm mistaken some tests are running on
`RecordNode` and not `RecordObject`
Took few directions on my own that as I always I would suggestion nor
remarks on them !
Let me know
## Misc
- Should we refactor `useDeleteOneRecord` tests to follow `eachTesting`
pattern ? => I feel like this is inappropriate as this hooks is already
high level, the only plus value would be less tests code despite
readability IMO
## Introduction
This refactor results from this
https://github.com/twentyhq/twenty/pull/10493 review
Introduced a new abstraction to the extinsting
`generateDepthOneRecordGqlFields` that was accepting an optional record
in arg in order to map generated `recordGqlFields` to the keys in the
record
1/ Created a dedicated util method
`generateDepthOneRecordGqlFieldsFromRecord` to do so
2/ Updated each previous `generateDepthOneRecordGqlFields` passing a
record to call new `generateDepthOneRecordGqlFieldsFromRecord`
# Introduction
The record does not appear in the table because the optimistic record
input cached does not fulfill the mutation response fields, which result
in it being considered as invalid response
close#10199
close https://github.com/twentyhq/twenty/issues/10153
# Introduction
While importing records encountering missing expected fields when
writting a fragment from apollo cache
## Updates
### 1/ `createdBy` Default value
When inserting in cache in create single or many we will now make
optimistic behavior on the createdBy value
### 2/ `createRecordInCache` dynamically create `recordGrqlFields`
When creating an entry in cache, we will now dynamically generate fields
to be written in the fragment instead of expecting all of them. As by
nature record could be partial
### 3/ Strictly typed `RecordGqlFields`
# Conclusion
closes#9927
# Introduction
At the moment when updating any record cache occurence, we will build a
fragment that will expect all of the object metadata item fields to be
provided.
Which result in the following traces: ( in the video companies aren't
fetch with companyId and other missing fields )
https://github.com/user-attachments/assets/56eab7c1-8f01-45ff-8f5d-78737b788b92
By definition as we're using graphql we might not request every record's
fields each time we wanna consume them.
In this way we will now dynamically compute or expect depending on the
CRUD operation specific fields to be written in the cache, and not all
of them
Tested all optimistic and failure management use cases
## Covering cache
Added coverage only for the `deleteOne` and `deleteMany` hooks, it cover
only the record record cache and not its relations hydratation ( for the
moment )
## Why not closing #9927
Unless I'm mistaken everything done here have fixed the same logs/traces
issue for updates and deletion but not creation.
Which means we still need to investigate the mass upload from import and
prefillRecord behavior
In a nutshell: went over each `updateRecordFromCache` calls, still need
to do all `createRecordInCache` calls
related to #9927
## Conlusion
Sorry for the big PR should have ejected into a specific one for the
`MinimalRecord` refactor
Will also continue covering others hooks later in my week as for the
`deleteOne`
As always any suggestions are welcomed !
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function
## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place
Related PR https://github.com/twentyhq/twenty/pull/9941
# Introduction
It seems like optimistic caching isn't working as expected for any
record relation mutation, CREATE UPDATE DELETE.
It should not have an impact on the destroy
We included a new `computeOptimisticRecordInput` that will calculate if
a relation is added or detach.
Updated the `triggerCreateRecordsOptimisticEffect` signature we should
have a look to each of its call to determine if it should be checking
cache or not
Related to #9580
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
At the moment the relationships are inferred from the record data
structure instead of its metadatas
We should refactor the code that computes or not the necessity to detach
a relation on a mutation
We've refactored the `isObjectRecordConnection` method to be consuming a
`relationDefintion` instead of "typeChecking" at the runtime the data
structure using zod validation schema
Related to #9580
In this PR, I'm fixing part of the impact of soft deletion on optimistic
rendering.
## Backend Vision
1) Backend endpoints will not return soft deleted records (having
deletedAt set) by default. To get the softDeleted records, we will pass
a { withSoftDelete: true } additional param in the query.
2) Record relations will NEVER contain softDeleted relations
## Backend current state
Right now, we have the following behavior:
- if the query filters do not mention deletedAt, we don't return
softDeletedRecords
- if the query filters mention deletedAt, we take it into consideration.
Meaning that if we want to have the softDeleted records in any way we
need to do { or: [ deletedAt: NULL, deletedAt: NOT_NULL] }
## Optimistic rendering strategy
1) useDestroyOne/Many is triggering destroyOptimisticEffects (previously
deleteOptimisticEffects)
2) UseDeleteOne/Many and useRestoreOne/Many are actually triggering
updateOptimisticEffects (as they only update deletedAt field) AND we
need updateOptimisticEffects to take into account deletedAt (future
withSoftDelete: true) filter.
In this PR:
- remove deprecated EMAIL, PHONE, LINK field types (except for Zapier
package as there is another work ongoing)
- remove composite currency filter on currencyCode, actor filter on name
and workspaceMember as the UX is not great yet
Fix all the broken CIs :p
This includes an ongoing effort to simplify test maintenance by having 1
unique source of truth about metadata and data mocks (that will later be
generated from a unique source of seeds: dev = demo = test)
Regressions:
- Unit line coverage: 60 > 55
- Storybook Pages branch coverage: 40 > 35
We will need to write tests to increase those coverage
- RelationFieldDisplay perf: 0.2ms to 0.22ms > We might have a
regression here
- Removed perf story about RawJSON > We will need to re-add it
Closes#5924.
Adding the "many" side of relations in the table view, and fixing some
issues (glitch in Multi record select, cache update after update).
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
After discussing with @charlesBochet, several fixes are needed on
fields:
- [x] Disable Boolean field `defaultValue` edition for now (On
`defaultValue` update, newly created records are not taking the updated
`defaultValue` into account. Setting the `defaultValue` on creation is
fine.)
- [x] Disable Phone field creation for now
- [x] For the Person object, display the "Phone" field as a field of
type Phone (right now its type is Text; later we'll migrate it to a
proper Phone field).
- [x] Fix RawJson field display (displaying `[object Object]` in Record
Table cells).
- [x] In Settings/Data Model, on Relation field creation/edition,
"Object destination" select is not working properly if an object was not
manually selected (displays Companies by default but creates a relation
to another random object than Companies).
## Query depth deprecation
I'm deprecating depth parameter in our graphql query / cache tooling.
They were obsolete since we introduce the possibility to provide
RecordGqlFields
## Refactor combinedFindManyRecordHook
The hook can now take an array of operationSignatures
## Fix tasks issues
Fix optimistic rendering issue. Note that we still haven't handle
optimisticEffect on creation properly
When writing to the normalized cache (record), it's crucial to use _refs
for relationships to avoid many problems. Essentially, we only deal with
level 0 and generate all fields to be comfortable with their defaults.
When writing in queries (which should be very rare, the only cases are
prefetch and the case of activities due to the nested query; I've
reduced this to a single file for activities
usePrepareFindManyActivitiesQuery 🙂), it's important to use queryFields
to avoid bugs. I've implemented them on the side of query generation and
record generation.
When doing an updateOne / createOne, etc., it's necessary to distinguish
between optimistic writing (which we actually want to do with _refs) and
the server response without refs. This allows for a clean write in the
optimistic cache without worrying about nesting (as the first point).
To simplify the whole activities part, write to the normalized cache
first. Then, base queries on it in an idempotent manner. This way,
there's no need to worry about the current page or action. The
normalized cache is up-to-date, so I update the queries. Same idea as
for optimisticEffects, actually.
Finally, I've triggered optimisticEffects rather than the manual update
of many queries.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
- Created addRecordInCache to inject a record in Apollo cache and inject single read query on this record
- Created createOneRecordInCache and createManyRecordsInCache that uses this addRecordInCache
- Created useOpenCreateActivityDrawerV2 hook to create an activity in cache and inject it into all other relevant requests in the app before opening activity drawer
- Refactored DEFAULT_SEARCH_REQUEST_LIMIT constant and hardcoded arbitrary request limits
- Added Apollo dev logs to see errors in the console when manipulating cache
