Fixes#11927
I have added 'format' in the zod schema of currency, and for using it, I
am separately passing 'format' to 'currencyDisplay.'
The feature is working correctly.
---------
Co-authored-by: prastoin <paul@twenty.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
- id field should only be available for search records action
- create record action does not work for relations. Requires to send
`accountOwner: { id: string }` instead of `accountOwner: string`
- hidding `runs` for version views as we did for workflows
Let's introduce an object-limited role for Tim, to test and/or spot
incompatibilities with restricted permissions in the future.
Our main user tim@apple.dev is now assigned a role that has all settings
permissions, and all object permissions except for update on Pets (to
test read-only view) and read on Rockets.
Since we still need an admin user for each workspace we are introducing
a new member, Jane, who has the admin role
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
In order to put back relations on track, we need to allow the
preexisting indexes to be created in indexmetadata, but also to avoid
failure when the migration runner will try to re-apply a pre-existing
index
- Fix an issue where custom object were seeded with 2 views, and with
the wrong icon
- ACME becomes YCombinator
- Allow 2 workspaces to have different metadata seeded
- Add many seeds for messages
- Add many seeds for calendar events
- Randomize createdBy for person and companies
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
## Why
After the changes on relations, index on relations were skipped by the
syncmetadata service, so no more migrations were generated for relation
fields.
We wanted to fix this.
## Test
This PR adds unit tests for the `createIndexMigration` utility in the
workspace migration builder. The tests cover:
- Creating index migrations for simple fields (e.g., text fields)
- Creating index migrations for relation fields (ensuring correct column
naming, e.g., `authorId` for the `author` objectmetadataname)
## Excluded
The delete index on relation does not need the column names so i don't
think i needed to work on this method. I might be wrong.
## Checklist
- [x] Added/updated unit tests for index migration creation
- [x] Verified correct handling of simple and relation fields
- [x] Ensured all tests pass
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
- Add seeds for notes/tasks
- Adds account manager to companies
- A companies and phone numbers to people
- Add many more opportunities
TODO: add timeline activities
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
## Goal
We have identified that sync-metadata (which is called during new
workspace initialization) is slow mainly because of workspaceMigration
application (migration-runner module). This is due to the fact that we
use typeORM API to perform schema changes, which often query the
existing schema. As querying the existing schema is costly (especially
with ~1M existing columns) and as we already have what we need described
as metadata, we will use raw SQL directly. This should divide the
workspace initialization time by x2.
## How
This PR can be read in two commits:
1. Extract functions tied to column migrations in a separate service
(`workspace-migration-column.service`) + deprecate COMMENT column
migration type which is not useful since we are not using pg-graphql
anymore
2. Re-work `workspace-migration-column.service` to make it clearer + use
raw SQL
## Result
Before:
<img width="1367" alt="image"
src="https://github.com/user-attachments/assets/e730df7b-db7f-4433-9ce5-52841b010990"
/>
After:
<img width="1367" alt="image"
src="https://github.com/user-attachments/assets/72d2c2b1-2475-4541-a3d5-50b70824a2e4"
/>
## Manual Testing
- Sync-metadata OK
- Workspace init OK
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
# Indexes
### TLDR:
Putting indexes back, except relation ones
### Details:
- Added index synchronization logic back (it was removed previously in
45d4845b26)
in the sync-metadata service.
- for unique inedexes, a command will create unicity again by handling
duplicates that were cretated since the
45d4845b26
was triggered
Closes https://github.com/twentyhq/core-team-issues/issues/748
In the frame of the work on permissions we
- remove all raw queries possible to use repositories instead
- forbid usage workspaceDataSource.executeRawQueries()
- restrict usage of workspaceDataSource.query() to force developers to
pass on shouldBypassPermissionChecks to use it.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# Introduction
Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder
Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles
## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc
## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment
## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
ignoreRestArgs?: boolean;
```
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
This is a first PR to remove old relation logic
Next steps:
- remove relationMetadata from cache
- remove relationMetadata table content and structure
- refactor relationDefinition to leverage field.settings instead
# Introduction
Encountering a blocking issue due to legacy upgrade history in staging
due to relation refactor
For the moment ( release 0.53 ) swallowing
@charlesBochet
## How to test
```ts
checkout v0.52.11
yarn
database:reset
checkout 0.53
yarn
build server
migrate
upgrade
```
## Context
While deploying the IS_NEW_RELATION_ENABLED (we don't compute relation
based on relationMetadata anymore) to existing workspace, I've tested to
run a sync-metadata post feature flag activation. This has raised two
issues:
- the workspaceMigration generator (which is over-complex and should be
refactored later) for fieldMetadata of type RELATION was not handling
settings update properly ;
- we need to delete existing fieldMetadata corresponding to the UUID
foreignKey as they are not needed anymore. This is handled as a 0.53
upgrade command as 0.53 will also come with the full removal of the old
relation system
---------
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747
We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.
In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
In this PR:
- this should fix the sync metadata for new relation system
This goes with the recent PR:
https://github.com/twentyhq/twenty/pull/11725
What we want:
- ONE_TO_MANY relations should have no joinColumn and no onDelete
- MANY_TO_ONE should have both
This PR implements what's missing to have sub-field filtering.
There is a backend modification to save subFieldName, we just add this
field on view filter workspace entity.
This PR adds subFieldName where missing in frontend, notably in
applyFilter calls, that will be refactored soon.
Also fixes a bug in ViewBar where Add Filter button was at the right
side of the ViewBar, while it should be right after the chips section.
Another bug fixed where we wouldn't delete an empty record filter on
dropdown click outside from the view bar, which was already the case
where using the filter chip dropdown.
<img width="512" alt="image"
src="https://github.com/user-attachments/assets/e9a2f8d2-a66f-4800-853a-4df5c6b627a9"
/>
<img width="495" alt="image"
src="https://github.com/user-attachments/assets/7542697b-0689-4095-9c3c-b5e47875355f"
/>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
## Context
When a trial ends, we receive a webhook from stripe to switch a
workspace from its previous status to SUSPENDED.
We also have some workspaces in ONGOING_CREATION that started the flow,
started the trial but did not finish completely the flow which means the
workspace has no data/metadata/schema.
Because of this, we can have workspaces that switch from
ONGOING_CREATION to SUSPENDED directly and ONGOING_CREATION workspaces
that didn't start the trial can stay in this state forever since they
are not cleaned up by the current cleaner.
To solve those 2 issues, I'm adding a new cron that will automatically
clean ONGOING_CREATION workspaces that are older than 7 days and I'm
updating the stripe webhook to only update the activationStatus to
SUSPENDED if it's already ACTIVE (then it will be deleted by the other
cron in the other case)
Closes https://github.com/twentyhq/core-team-issues/issues/526
(for reminder:
1. Make defaultRoleId non-nullable for an active workspace
2. Remove permissions V1 feature flag
3. Set member role as default role for new workspaces
About 1.:
An active workspace's defaultRoleId should never be null.
We can't rely on a simple postgres NOT NULL constraint as defaultRoleId
will always be initially null when the workspace is first created since
the roles do not exist at that time.
Let's add a more complex rule to ensure that
About 3.:
In the first phase of our deploy of permissions, we chose to assign
admin role to all existing users, not to break any existing behavior
with the introduction of the feature (= existing users have less rights
than before).
As we deploy permissions to all existing and future workspaces, let's
set the member role as default role for future workspaces.
)
