Commit Graph

324 Commits

Author SHA1 Message Date
4257f30f12 Permission checks on twentyORM global manager (#11477)
In this PR we are handling permissions when using
twentyORMGlobalManager,
and handling permissions for rest api and api key
2025-04-23 17:57:48 +02:00
fa5f758228 Fix storybook / chromatic tests flakyness and integration tests (#11687)
## Storybook flakyness

### Actor Display image flakyness

<img width="1512" alt="image"
src="https://github.com/user-attachments/assets/875c0738-5e31-4aba-9231-4ba5f78d1355"
/>

**Fix:** stop using a random usage

### Task Groups broken

<img width="1512" alt="image"
src="https://github.com/user-attachments/assets/c67e47a1-a027-43f1-9601-68d61a8052b4"
/>

**Fix:** add missing TabListComponentInstance

## Flaky dates

Add https://github.com/k35o/storybook-addon-mock-date

## Integration tests

Fix broken tests due to relation refactoring
2025-04-23 01:57:36 +02:00
cc29c25176 feat: new relation sync-metadata, twenty-orm, create/update (#10217)
Fix
https://github.com/twentyhq/core-team-issues/issues/330#issue-2827026606
and
https://github.com/twentyhq/core-team-issues/issues/327#issue-2827001814

What this PR does when `isNewRelationEnabled` is set to `true`:
- [x] Drop the creation of the  foreign key as a `FieldMetadata`
- [x] Stop creating `RelationMetadata`
- [x] Properly fill `FieldMetadata` of type `RELATION` during the sync
command
- [x] Use new relation settings in TwentyORM
- [x] Properly create `FieldMetadata` relations when we create a new
object
- [x] Handle `database:reset` with new relations

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2025-04-22 19:01:39 +02:00
f8b9e4d780 ignoreLock when recomputing rolePermissions when creating datasource (#11657)
[sentry](https://twenty-v7.sentry.io/issues/6545999328/?environment=prod&project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D&referrer=issue-stream&sort=date&stream_index=2)

Our workers have failing jobs because they are concurrently creating
datasources on different pods
While the datasources are independent, they read from the same redis
cache, with the same `RolesPermissionsOngoingCachingLock` value in
redis.
<img width="852" alt="Capture d’écran 2025-04-18 à 18 00 20"
src="https://github.com/user-attachments/assets/42ce8479-acc1-462b-af4c-b547cc2bb0b8"
/>
As a consequence they can fail to create a datasource: the first
datasource computes the permissions and sets ongoingCachingLock to true,
then the second arrives, there are still no available permissions in the
cache, but because of the lock it early returns without permissions and
fails. This behavior goes unnoticed on the product and helps
performances, but is annoying for workers as jobs are failing.

Let's remove the cache lock when creating the datasource !
2025-04-18 18:16:34 +02:00
fd750c7c0e Fix metadata cache flush (#11646)
Attempt to fix
https://twenty-v7.sentry.io/issues/6545999328/?environment=prod&project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D&referrer=issue-stream&sort=date&stream_index=2
2025-04-18 12:02:36 +00:00
53042cc9dc Date field format display (#11384)
## Introduction

This PR enables functionality discussed in [Layout Date
Formatting](https://github.com/twentyhq/core-team-issues/issues/97).

### TLDR;
It enables greater control of date formatting at the object's field
level by upgrading all DATE and DATE_TIME fields' settings from:

```ts
{
    displayAsRelativeDate: boolean
}
```

to:

```ts

type FieldDateDisplayFormat = 'full_date' | 'relative_date' | 'date' | 'time' | 'year' | 'custom'

{
    displayFormat: FieldDateDisplayFormat
}
```

PR also includes an upgrade command that will update any existing DATE
and DATE_TIME fields to the new settings value

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-04-18 07:00:02 +00:00
3fe12cd8b5 Fix select default value not in options (#11622)
Also fixing a bunch of places where validation exceptions were not
properly handled
2025-04-17 18:34:31 +02:00
9e9ba73a43 better handling the function argument without touching it (#11614)
leave intact the `input` argument to avoid side effects on the parent
caller

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2025-04-17 08:54:56 +00:00
b5e6600c73 Capitalize labels objectmetadata (#11609)
Capitalize labels singular and plural from objectmetadata

Fixes [#664](https://github.com/twentyhq/core-team-issues/issues/664)
2025-04-16 21:02:12 +00:00
587281a541 feat(analytics): add clickhouse (#11174) 2025-04-16 16:33:10 +00:00
4d78f5f97f [permissions] Improve performances using a cache for userWorkspaces roles (#11587)
In this PR we are 

- introducing a cached map `{ userworkspaceId: roleId } `to reduce calls
to get a userWorkspace's role (we were having N+1 around that with
combinedFindMany queries and generally having a lot of avoidable
queries)
- using the roles permissions cache (`{ roleId: { objectNameSingular:
{ canRead: bool, canUpdate: bool, ...} } `) in Permissions V1's
userHasObjectPermission, in order to 1) improve performances to avoid
calls to get roles 2) start using our permissions cache
2025-04-16 17:07:43 +02:00
43af5ceb5e Add object level permission permissions to role page (ReadOnly) (#11568)
## Context
This PR adds the display of object-level permissions. A following PR
will add the ability to update those permissions.
The PR contains the SettingsRoleObjectLevel page but it's not fully
implemented yet (save won't trigger the corresponding mutation)

<img width="616" alt="Screenshot 2025-04-14 at 18 02 40"
src="https://github.com/user-attachments/assets/f8c58193-31f3-468a-a96d-f06a9f2e1423"
/>
2025-04-15 18:46:36 +02:00
d4deca45e8 Read feature flags from cache (#11556)
We are now storing a workspace's feature flag map in our redis cache. 
The cache is invalidated upon feature flag update through the lab
resolver.
2025-04-14 17:31:13 +02:00
162c6bcaa3 [permissions] Implement object-records permissions in query builders (#11458)
In this PR we are

- (if permissionsV2 is enabled) executing permission checks at query
builder level. To do so we want to override the query builders methods
that are performing db calls (.execute(), .getMany(), ... etc.) For now
I have just overriden some of the query builders methods for the poc. To
do so I created custom query builder classes that extend typeorm's query
builder (selectQueryBuilder and updateQueryBuilder, for now and later I
will tackle softDeleteQueryBuilder, etc.).
- adding a notion of roles permissions version and roles permissions
object to datasources. We will now use one datasource per roleId and
rolePermissionVersion. Both rolesPermissionsVersion and rolesPermissions
objects are stored in redis and recomputed at role update or if queried
and found empty. Unlike for metadata version we don't need to store a
version in the db that stands for the source of truth. We also don't
need to destroy and recreate the datasource if the rolesPermissions
version changes, but only to update the value for rolesPermissions and
rolesPermissionsVersions on the existing datasource.

What this PR misses
- computing of roles permissions should take into account
objectPermissions table (for now it only looks at what's on the roles
table)
- pursue extension of query builder classes and overriding of their db
calling-methods
- what should the behaviour be for calls from twentyOrmGlobalManager
that don't have a roleId?
2025-04-11 17:34:02 +02:00
1794b3a611 Remove build lambda at creation (#11531)
Remove useless build call (done at first execution)
2025-04-11 14:40:00 +02:00
d69932c6c4 Fix boolean default value update not updating psql default value when false (#11513)
Fixes https://github.com/twentyhq/twenty/issues/11471

## Context
When sending false as a new defaultValue, this was not going through the
migration creation code due to this condition
```typescript
if (updatableFieldInput.defaultValue)
```
2025-04-10 16:23:16 +02:00
bd3ec6d5e3 rename core-module environment to twenty-config (#11445)
closes https://github.com/twentyhq/core-team-issues/issues/759
2025-04-09 14:11:26 +02:00
ff59658d39 Fix permission check during onboarding (#11429)
## Context
CurrentUser is fetched during onboarding however roles and permissions
are not created yet during that stage so an error was thrown. We only
want to fetch permissions after the onboarding of the workspace.
2025-04-07 18:53:53 +02:00
f403c551d7 Add settings permissions check on FE (#11425)
## Context
Now that we can update role settings permissions, we need to reflect
that on the FE as well (hiding/showing nav items + redirection logic).
Feature flag check here is not really needed because since not having
any setting permission will result in the same behavior as Permission
V1.
This PR updates the resolvers to return settings permissions of the
current user
2025-04-07 17:15:33 +02:00
e1f6c61651 add settings permissions update (#11377)
Fixes https://github.com/twentyhq/core-team-issues/issues/710
2025-04-04 17:40:14 +02:00
bb40bc9929 Fix update field default value (#11386)
Fixes https://github.com/twentyhq/twenty/issues/11379
2025-04-04 08:58:53 +00:00
06ff16e086 add role update (#11217)
## Context
This PR introduces the new Create and Edit role components, behind the
PERMISSIONS_ENABLED_V2 feature flag.
2025-03-31 17:57:14 +02:00
a683827e4b Fix inline cell height issue and allow field settings update (#11248)
In this PR:
- allow to update settings on fields metadata (regression introduced by
a recent refactoring of fields-metadata update)
- revert changes introduced by
https://github.com/twentyhq/twenty/pull/11221
2025-03-27 19:29:34 +01:00
22e56339a1 Fix field options not updatable anymore (#11213)
As per title
2025-03-26 23:40:44 +01:00
5bd10d40cb Fix deactivate objects impacts (#11185)
In this PR:
- Remove deactivated objects from ActivityTargetInlineCell record picker
- Prevent users to deactivate createdAt, updatedAt, deletedAt fields on
any objects

Still left:
- write unit tests on the assert utils
- write integration tests on field metadata service
- prevent users to deactivate createdAt, updatedAt, deletedAt on FE
2025-03-26 20:45:46 +01:00
7af90eb4c4 [permissions V2] Custom role deletion (#11187)
Closes https://github.com/twentyhq/core-team-issues/issues/616
2025-03-26 15:08:48 +01:00
6ec06be18d Overwrite standard translations (#11134)
Manage overwriting translations for standard fields and standard objects
properties
2025-03-25 22:17:29 +01:00
4680bc740a [permissions V2] Upsert object and setting permissions (#11119)
Closes https://github.com/twentyhq/core-team-issues/issues/639
2025-03-25 11:07:51 +01:00
52cf6f4795 Allow to edit labels of standard objects (#10922)
Fixes #10793

This PR is a work in progress.

**Still left to fix:**

- [x] When disabling synchronization of labels / api names, the edited
labels should be set to the English version. Currently the client just
send the localized versions together with the `isLabelSyncedWithName`
change. Could be an easy fix.
- [ ] Sometimes flipping the switch don't trigger the update function,
may be a regression as it seems to affect the custom objects too.
- [ ] There is a frontend problem where the labels inputs don't reflect
the changes made. When enabling back synchronisation after editing
labels, they are correctly back to their base values (backend,
navigation breadcrumb, etc) but the label inputs still have the old
values (switching pages will put them back to normal). I suspect this
could be linked to the above problem.
- [ ] API names are still displayed for standard objects per (kept them
for debugging, trivial fix)
- [ ] `SettingsDataModelObjectAboutForm` have a `disableEdition`
parameter which is now used only for a few fields, not sure if it's
worth keeping because it's a bit misleading since it doesn't "disable"
much?
- [ ] I don't know what these do, but I have seen "Remote" object types.
Not sure if they work with my patch or not (I don't know how to test
them)
- [ ] Make it work with metadata synchronisation


**What should work:**

- Disabling synchronization of standard objects should work, label
inputs should no longer be disabled
- Modifying labels should work
- Enabling back synchronization should reset back the labels to the base
value and disable the label inputs again (minus the mentioned display
bug)
- The synchronisation switch should still work as expected for custom
objects
- Creating custom objects should still work (it uses the same form)

---------

Signed-off-by: AFCMS <afcm.contact@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-03-24 20:19:52 +01:00
1c5f3ef5fa clean searchResolvers in server (#11114)
Introduces break in change

- remove search... resolvers
- rename globalSearch to search
- rename searchRecord.objectSingularName > objectNameSingular
closes https://github.com/twentyhq/core-team-issues/issues/643
2025-03-24 13:42:51 +01:00
9ad8287dbc [REFACTOR] twenty-shared multi barrel and CJS/ESM build with preconstruct (#11083)
# Introduction

In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)

For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682

close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590

## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )

## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617

## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960

## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level

## Conclusion
As always any suggestions are welcomed !
2025-03-22 19:16:06 +01:00
3960b0f99d App health check: Optimize pending migration query (#11049)
## Optimization: Efficient Health Check Query

This PR optimizes the workspace health check system by replacing the N+1
query pattern with efficient database queries.

### Key Improvements

- **Eliminated N+1 Query Problem**: Instead of fetching all workspaces
and then querying each one individually for pending migrations (which
caused slowness in production), we now use a single optimized query to
directly identify workspaces with pending migrations

- **Better Performance**: Reduced the number of database queries from
potentially hundreds/thousands (previous implementation) to just 2 fixed
queries regardless of workspace count

- **Full Coverage Instead of Sampling**: Rather than implementing a cap
on workspace checks at 100 (which was a workaround for performance
issues), this solution addresses the root cause by optimizing the query
pattern. We can now efficiently check all workspaces with pending
migrations without performance penalties.

@FelixMalfait This addresses the "always eager-load when you can"
feedback by handling the problem at the database level rather than just
applying a limit. The optimized query should solve both the performance
issues and provide more accurate health status information.
2025-03-21 16:30:38 +01:00
8b7188e85b [fix] [permissions] Fix role assignment at sign-up (#11045)
In [a previous PR](https://github.com/twentyhq/twenty/pull/11023) I
fixed the issue where users where re-assigned the default role when
signin up using SSO.
The "fix" actually introduced another error, calling
`assignRoleToUserWorkspace` only when a user is signing up to twenty,
forgetting about the case where an existing twenty user signs up to a
different workspace. They should still be assigned the role in that
case.

To fix this and improve clarity, I moved assignRoleToUserWorkspace to
addUserToWorkspace and renamed addUserToWorkspace to
addUserToWorkspaceIfUserNotInWorkspace since this is at each sign in but
does nothing if user is already in the workspace.

I think ideally we should refactor this part to improve readability and
understandability, maybe with different flows for each case: signIn and
signUp, to twenty or to a workspace
2025-03-20 10:37:26 +00:00
d99f027e8d 400 workflows webhooks trigger (#11041)
https://github.com/user-attachments/assets/dc0ece22-4d87-417f-b9e1-a11c3fd52ce8
2025-03-20 10:12:52 +00:00
9e83d902d8 [permissions V2] Create and update a custom role (without granularity) (#11003)
First steps for https://github.com/twentyhq/core-team-issues/issues/595
and https://github.com/twentyhq/core-team-issues/issues/621

Not handling granular permissions through objectPermissions and
settingsPermissions next; will come next !
2025-03-18 18:42:30 +01:00
aa6fd90424 Create objectPermissions and settingsPermissions tables (#10962)
Closes https://github.com/twentyhq/core-team-issues/issues/594
2025-03-18 10:45:31 +01:00
fc30ba57f8 Fix server integration tests 2 (#10818)
## Context
- Removing search* integration tests instead of fixing them because they
will be replaced by global search very soon
- Fixed billing + add missing seeds to make them work
- Fixed integration tests not using consistently the correct "test" db
- Fixed ci not running the with-db-reset configuration due to nx
configuration being used twice for different level of the command
- Enriched .env.test
- Fixed parts where exceptions were not thrown properly and not caught
by exception handler to convert to 400 when needed
- Refactored feature flag service that had 2 different implementations
in lab and admin panel + added tests
- Fixed race condition when migrations are created at the same timestamp
and doing the same type of operation, in this case object deletion could
break because table could be deleted earlier than its relations
- Fixed many integration tests that were not up to date since the CI has
been broken for a while

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2025-03-13 17:48:29 +01:00
5b34f86b6d Add Icon to roles (#10842)
## Context
Adding icon column to role table to distinguish roles.

<img width="621" alt="Screenshot 2025-03-13 at 11 02 37"
src="https://github.com/user-attachments/assets/dc7b121c-2901-4599-9638-d5dcdf443999"
/>
2025-03-13 11:34:30 +01:00
4036933c84 Add console logs to code step (#10786)
Works for local and lambda drivers

## After

![image](https://github.com/user-attachments/assets/ec082cf6-4159-4a15-99b6-243c73a13773)

![image](https://github.com/user-attachments/assets/f42e3f43-5ea4-4167-b4b5-9a17826fd224)
2025-03-12 10:40:59 +01:00
4d0450069c Fix fieldMetadata sync validation exceptions caught in exception handler (#10789)
## Context
Field metadata service was reusing validators from
validate-**OBJECT**-metadata-input which were throwing ObjectMetadata
exceptions not handled in fieldMetadataGraphqlApiExceptionHandler and
were going to Sentry.
To solve the issue since this validator is associated with both fields
and objects I'm moving the util to the root utils folder of metadata
module and throwing a common metadata user input exception
2025-03-11 18:41:29 +01:00
41f3a63962 [BUGFIX] ObjectMetadata item server validation (#10699)
# Introduction
This PR contains several SNAPSHOT files explaining big +

While refactoring the Object Model settings page in
https://github.com/twentyhq/twenty/pull/10653, encountered a critical
issue when submitting either one or both names with `""` empty string
hard corrupting a workspace.

This motivate this PR reviewing server side validation

I feel like we could share zod schema between front and back

## Refactored server validation
What to expect from Names:
- Plural and singular have to be different ( case insensitive and
trimmed check )
- Contains only a-z A-Z and 0-9
- Follows camelCase
- Is not empty => Is not too short ( 1 )
- Is not too long ( 63 )
- Is case insensitive( fooBar and fOoBar now rejected )

What to expect from Labels:
- Plural and singular have to be different ( case insensitive and
trimmed check )
- Is not empty => Is not too short ( 1 )
- Is not too long ( 63 )
- Is case insensitive ( fooBar and fOoBar now rejected )

close https://github.com/twentyhq/twenty/issues/10694

## Creation integrations tests
Created new integrations tests, following
[EachTesting](https://jestjs.io/docs/api#testeachtablename-fn-timeout)
pattern and uses snapshot to assert errors message. These tests cover
several failing use cases and started to implement ones for the happy
path but object metadata item deletion is currently broken unless I'm
mistaken @Weiko is on it

## Notes
- [ ] As we've added new validation rules towards names and labels we
should scan db in order to standardize existing values using either a
migration command or manual check
- [ ] Will review in an other PR the update path, adding integrations
tests and so on
2025-03-11 12:14:37 +01:00
f4fcf39eb5 [permissions] Prepare for roll-out (#10676)
Closes https://github.com/twentyhq/core-team-issues/issues/469 and
https://github.com/twentyhq/core-team-issues/issues/500

In this PR
1. stop conditioning permission initialization for a workspace to env
variable value. Instead we want to create and assign permissions and
roles in all new workspaces. For now that will be totally silent.
2. temporarily, the default role is set to the admin role for new
workspaces. it will also be the case for existing workspaces through the
backfill command. Member role is still being created though. (when we
will do the final roll-out we will update this so that future workspaces
have the member role as default role. our goal here is not to break any
current behaviour for users, that today have all have the equivalent of
admin rights).
2025-03-05 16:47:41 +01:00
baa3043954 Refactor upgrade commands (#10592)
Simplifying a lot the upgrade system.

New way to upgrade:
`yarn command:prod upgrade`

New way to write upgrade commands (all wrapping is done for you)
```
  override async runOnWorkspace({
    index,
    total,
    workspaceId,
    options,
  }: RunOnWorkspaceArgs): Promise<void> {}
```

Also cleaning CommandModule imports to make it lighter
2025-02-28 19:51:32 +01:00
122a6a7801 [permissions] Backfill command to prepare workspaces (#10581)
Closes https://github.com/twentyhq/core-team-issues/issues/317

---------

Co-authored-by: Weiko <corentin@twenty.com>
2025-02-28 15:46:51 +01:00
a3a05c63f6 [permisions] Bypass permission checks with api key (#10516)
Closes https://github.com/twentyhq/core-team-issues/issues/325
2025-02-28 07:50:49 +01:00
39543872e6 add new @WorkspaceIsSearchable decorator + updates services + add migration command (#10507)
closes https://github.com/twentyhq/core-team-issues/issues/345
2025-02-27 13:57:07 +01:00
17dbb634ca [permissions] forbid deletion of last admin user (#10504)
A user should not be able to delete their account if they are the last
admin of a workspace.

It means that if a user wants to sign out of twenty, they should delete
their workspace, not their account
2025-02-27 12:44:51 +01:00
83930551d8 Move workflow versions and steps building to workflow-builder folder (#10523)
We are starting to put too many services in common folder. Version and
step building should be separated into different services and go to the
builder folder. Today builder folder only manage schema.

We should:
- keep services responsible for only one action
- keep modules based on the actual action these provide rather than
having common module

This PR:
- creates a service for workflow version builder
- moves version and step builders to workflow builder folder rather than
commun
- creates separated folders for schema, version and steps

No logic has been added. Only modules created and functions moved.
2025-02-27 10:39:48 +01:00
d40a5ed74f Fix REST API not using metadata cache (#10521) 2025-02-26 17:59:35 +01:00
589a0c7b2d Fix graphql conflict between role custom object and role dto (#10479)
## Context
Following the strategy where we want to block custom object creation
when the type is reserved by core objects. The issue happened again with
the recently introduced role table.
2025-02-25 15:46:06 +01:00