In this PR
- Determine object record permissions on workflows objects (workflow,
workflowVersion, workflowRun) base on settings permissions @Weiko
- Add Workflow permission guards on workflow resolvers @thomtrp . **Any
method within a resolver that has the SettingsPermission Guard is only
callable by a apiKey or a user that has the permission** (so not by
external parties).
- Add checks bypass in workflow services since 1) for actions gated by
settings permissions, the gate should be done at resolver level, so it
will have been done before the call to the service 2) some service
methods may be called by workflowTriggerController which is callable by
external parties without permissions (ex:
workflowCommonWorkspaceService.getWorkflowVersionOrFail). This is
something we may want to change in the future (still to discuss), by
removing the guard at resolver-level and relying on
shouldBypassPermissionChecks at getRepository and made in a way that we
only bypass for external parties.
- Add checks bypass for actions performed by workflows since they should
not be restricted in our current vision
- Add tests
# Fix cursor-based pagination with lexicographic ordering for composite
fields
## Bug
The existing cursor-based pagination implementation had a bug when
handling composite fields.
When paginating through results sorted by composite fields (like
`fullName` with sub-properties `firstName` and`lastName`), the WHERE
conditions generated for cursor positioning were incorrect, leading to
records being skipped.
The previous implementation was generating wrong WHERE conditions:
For example, when paginating with a cursor like `{ firstName: 'John',
lastName: 'Doe' }`, it would generate:
```sql
WHERE firstName > 'John' AND lastName > 'Doe'
```
This is incorrect because it would miss records like `{ firstName:
'John', lastName: 'Smith' }` which should be included in forward
pagination.
## Fix
Create a new util to use proper lexicographic order when sorting a
composite field.
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR
1. adding tests on relations and nested relations to make sure that if
any permission is missing, the query fails
2. adding tests on objectRecord permissions to make sure that
permissions granted or restricted by objectPermissions take precedence
on the role's allObjectRecords permissions
# Integration test failing
- fix the local run by renaming folder in the jest configuration.
Otherwise "clickhouse" tests were failing
- falsy test introduced 2 days ago in
https://github.com/twentyhq/twenty/pull/12271/files
Closes https://github.com/twentyhq/core-team-issues/issues/605
Actually settingsPermissions checks were already implemented, but we had
no tests on them.
In the ticket we had mentioned
_TO DO: in pemissions.service we should stop calling
userRoleService.getRolesByUserWorkspaces and call
getRoleIdForUserWorkspace instead which relies on the cache._
But actually roleId is not enough for settings permissions because we
don't store them in the cache (unlien object records permissions - which
I think we had forgotten about when adding that TODO.), so we will still
need to make a db call to load the role's settingsPermissions. I think
it's better to make just one db call to get the role and
settingsPermissions from userWorkspaceId (as currently) than to make one
redis call to get roleId for userWorksapce then one db call to get role
and its settingsPermissions).
# Introduction
Diff description: ~500 tests and +500 additions
close https://github.com/twentyhq/core-team-issues/issues/731
## What has been done here
In a nutshell on a field metadata type ( `SELECT MULTI_SELECT` ) update,
we will be browsing all `ViewFilters` in a post hook searching for some
referencing related updated `fieldMetadata` select. In order to update
or delete the `viewFilter` depending on the associated mutations.
## How to test:
- Add FieldMetadata `SELECT | MULTI_SELECT` to an existing or a new
`objectMetadata`
- Create a filtered view on created `fieldMetadata` with any options you
would like
- Remove some options ( in the best of the world some that are selected
by the filter ) from the `fieldMetadata` settings page
- Go back to the filtered view, removed or updated options should have
been hydrated in the `displayValue` and the filtered data should make
sense
## All filtered options are deleted edge case
If an update implies that a viewFilter does not have any existing
related options anymore, then we remove the viewFilter
## Testing
```sh
PASS test/integration/metadata/suites/field-metadata/update-one-field-metadata-related-record.integration-spec.ts (27 s)
update-one-field-metadata-related-record
SELECT
✓ should delete related view filter if all select field options got deleted (2799 ms)
✓ should update related multi selected options view filter (1244 ms)
✓ should update related solo selected option view filter (1235 ms)
✓ should handle partial deletion of selected options in view filter (1210 ms)
✓ should handle reordering of options while maintaining view filter values (1487 ms)
✓ should handle no changes update of options while maintaining existing view filter values (1174 ms)
✓ should handle adding new options while maintaining existing view filter (1174 ms)
✓ should update display value with options label if less than 3 options are selected (1249 ms)
✓ should throw error if view filter value is not a stringified JSON array (1300 ms)
MULTI_SELECT
✓ should delete related view filter if all select field options got deleted (1127 ms)
✓ should update related multi selected options view filter (1215 ms)
✓ should update related solo selected option view filter (1404 ms)
✓ should handle partial deletion of selected options in view filter (1936 ms)
✓ should handle reordering of options while maintaining view filter values (1261 ms)
✓ should handle no changes update of options while maintaining existing view filter values (1831 ms)
✓ should handle adding new options while maintaining existing view filter (1610 ms)
✓ should update display value with options label if less than 3 options are selected (1889 ms)
✓ should throw error if view filter value is not a stringified JSON array (1365 ms)
Test Suites: 1 passed, 1 total
Tests: 18 passed, 18 total
Snapshots: 18 passed, 18 total
Time: 27.039 s
```
## Out of scope
- We should handle ViewFilter validation when extracting its definition
from the metadata
https://github.com/twentyhq/core-team-issues/issues/1009
## Concerns
- Are we able through the api to update an RATING fieldMetadata ? ( if
yes than that's an issue and we should handle RATING the same way than
for SELECT and MULTI_SELECT )
- It's not possible to group a view from a MULTI_SELECT field
The above points create a double nor a triple "lecture" to the post hook
effect:
- ViewGroup -> only SELECT
- VIewFilter -> only SELECT || MULTI_SELECT
- Rating nothing
I think we should determine the scope of all of that
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
### Primary Changes: Dynamic Driver Configuration
Refactors FileStorageService and EmailSenderService to support dynamic
driver configuration changes at runtime without requiring application
restarts.
**Key Architectural Change**: Instead of conditionally registering
drivers at build time based on configuration, we now **register all
possible drivers eagerly** and select the appropriate one at runtime.
### What Changed:
- **Before**: Modules conditionally registered only the configured
driver (e.g., only S3Driver if STORAGE_TYPE=S3)
- **After**: All drivers (LocalDriver, S3Driver, SmtpDriver,
LoggerDriver) are registered at startup
- **Runtime Selection**: Services dynamically choose and instantiate the
correct driver based on current configuration
### Secondary Fix: Integration Test Log Cleanup
Addresses ConfigStorageService error logs appearing in integration test
output by using injected LoggerService for consistent log handling.
## Context
- Introduced objectPermissions in currentUserWorkspace which uses role
permissions from cache so we can fetch granular permissions from the API
- Refactored cached role permissions to map permissions with object
metadata id instead of object metadata name singular to be more flexible
New Cache
<img width="574" alt="Screenshot 2025-05-27 at 11 59 06"
src="https://github.com/user-attachments/assets/1a090134-1b8a-4681-a630-29f1472178bd"
/>
GQL
<img width="977" alt="Screenshot 2025-05-27 at 11 58 53"
src="https://github.com/user-attachments/assets/3b9a82b0-6019-4a25-a6e2-a9e0fb4bb8a0"
/>
Next steps: Use the updated API in the FE to fetch granular permissions
and update useHasObjectReadOnlyPermission hook
This PR has several objectives:
- Ignore invalid and empty links in the frontend
- Ignore empty links when creating or updating a link field in the
backend
- Throw an error when trying to create or update a link field with an
invalid link
The logic is mostly the same in the frontend and the backend: we take
the initial primaryLink and the secondaryLinks, we discard all the empty
links (with `url === '' || url === null`), and the primaryLink becomes
the first remaining link.
## Frontend
There are three parts in the frontend where we have to remove the empty
links:
- LinksDisplay
- LinksFieldInput
- isFieldValueEmpty; used in RecordInlineCell
## Backend
I put the logic in
`packages/twenty-server/src/engine/core-modules/record-transformer/services/record-input-transformer.service.ts`
as it's used by the REST API, the GraphQL API, and by Create Record and
Update Record actions in the workflows.
# Introduction
Big diff a lot of tests and snapshots ( real diff < 500+ )
close https://github.com/twentyhq/twenty/issues/12117
close https://github.com/twentyhq/twenty/issues/12133
## What has been done here
Implemented a strong integration coverage on both fieldmetadata`SELECT`
`UPDATE` and `CREATE`.
Implemented server side validation for the options `value` `label` `id`
and collision issue with also `position`
We could improve:
- Position validation
- DefaultValue validation
## Update
```ts
PASS test/integration/metadata/suites/field-metadata/update-one-field-metadata-select.integration-spec.ts (41.054 s)
Field metadata select update tests group
✓ Update should succeed with provided option id (2565 ms)
✓ Update should succeed with valid default value (1469 ms)
✓ Update should succeed with various options id (1257 ms)
✓ Update should succeed without option id (1286 ms)
✓ Update should trim option values (1366 ms)
✓ Update should succeed with default value and no options (1122 ms)
✓ Update should fail with unknown default value and no options (1075 ms)
✓ Update should fail with only white spaces id (1195 ms)
✓ Update should fail with empty string id (1058 ms)
✓ Update should fail with null id (1066 ms)
✓ Update should fail with not a string id (1098 ms)
✓ Update should fail with too long id (1373 ms)
✓ Update should fail with only white spaces label (1034 ms)
✓ Update should fail with empty string label (1057 ms)
✓ Update should fail with null label (1100 ms)
✓ Update should fail with not a string label (1144 ms)
✓ Update should fail with too long label (1273 ms)
✓ Update should fail with only white spaces value (1385 ms)
✓ Update should fail with empty string value (1035 ms)
✓ Update should fail with null value (1068 ms)
✓ Update should fail with not a string value (1021 ms)
✓ Update should fail with too long value (1134 ms)
✓ Update should fail with invalid option id (1137 ms)
✓ Update should fail with empty options (1238 ms)
✓ Update should fail with invalid option value format (1104 ms)
✓ Update should fail with comma in option label (1004 ms)
✓ Update should fail with duplicated option values (1015 ms)
✓ Update should fail with duplicated option ids (1079 ms)
✓ Update should fail with duplicated option positions (1266 ms)
✓ Update should fail with duplicated trimmed option values (1220 ms)
✓ Update should fail with undefined option label (1029 ms)
✓ Update should fail with an invalid default value (1142 ms)
✓ Update should fail with an unknown default value (1081 ms)
✓ Update should fail with undefined option value (1086 ms)
Test Suites: 1 passed, 1 total
Tests: 34 passed, 34 total
Snapshots: 28 passed, 28 total
Time: 41.079 s
```
## Create
```ts
PASS test/integration/metadata/suites/field-metadata/create-one-field-metadata-select.integration-spec.ts (38.292 s)
Field metadata select creation tests group
✓ Create should succeed with provided option id (2096 ms)
✓ Create should succeed with valid default value (1316 ms)
✓ Create should succeed with various options id (1113 ms)
✓ Create should succeed without option id (1378 ms)
✓ Create should trim option values (1296 ms)
✓ Create should fail with only white spaces id (1000 ms)
✓ Create should fail with empty string id (1325 ms)
✓ Create should fail with null id (1060 ms)
✓ Create should fail with not a string id (1142 ms)
✓ Create should fail with too long id (1321 ms)
✓ Create should fail with only white spaces label (999 ms)
✓ Create should fail with empty string label (1163 ms)
✓ Create should fail with null label (1198 ms)
✓ Create should fail with not a string label (1678 ms)
✓ Create should fail with too long label (1527 ms)
✓ Create should fail with only white spaces value (1200 ms)
✓ Create should fail with empty string value (1102 ms)
✓ Create should fail with null value (1037 ms)
✓ Create should fail with not a string value (1462 ms)
✓ Create should fail with too long value (896 ms)
✓ Create should fail with invalid option id (997 ms)
✓ Create should fail with empty options (1058 ms)
✓ Create should fail with invalid option value format (1190 ms)
✓ Create should fail with comma in option label (1142 ms)
✓ Create should fail with duplicated option values (872 ms)
✓ Create should fail with duplicated option ids (860 ms)
✓ Create should fail with duplicated option positions (1002 ms)
✓ Create should fail with duplicated trimmed option values (1336 ms)
✓ Create should fail with undefined option label (754 ms)
✓ Create should fail with an invalid default value (696 ms)
✓ Create should fail with an unknown default value (678 ms)
✓ Create should fail with undefined option value (699 ms)
✓ Create should fail with null options (720 ms)
✓ Create should fail with undefined options (686 ms)
Test Suites: 1 passed, 1 total
Tests: 34 passed, 34 total
Snapshots: 29 passed, 29 total
Time: 38.314 s
```
## Conclusion
As always any suggestions are welcomed ! Please let me know
## Discussion about validation governance
### Front
Front side will be dealing with zod validations schema that he will
handle and maintain by himself
### Back validation instances
- Validation hold through DTO declarations ( run by yoga through the
resolvers )
- Server programmatic validation and exceptions handling ( run through
the services )
For this refactor/fix we decided to stick to the current implementation
only touching the `Server programmatic validation and exceptions
handling` we will handle validation centralization when we will onboard
the `nestjs-query` deprecation/integration refactor.
### Vision
In the best of the world we could think of an intermediary model that
will handle and take responsibility of the validation decorators that
would be run programmatically through the service, Yoga would still
consume it ? then we would need to have enough grain in the service to
know the input has already validated
## Notes
Introduced zod back side in order to handle very atomic and primitive
validation
- fix missing createBy injection in api createOne and createMany
endpoints
- add a command to fix null default value for createdBySource in
production entities
- tested on `1747159401197/` dump extract of production db without issue
Follow-up on https://github.com/twentyhq/twenty/pull/12007
In this PR
- adding a filter on HttpExceptionHandlerService to filter out 4xx
errors from driver handling (as we do for graphQL errors: see
useGraphQLErrorHandler hook - only filteredIssues are sent to`
exceptionHandlerService.captureExceptions()`.)
- grouping together more missing metadata issues
- attempting to use error codes as issues names in sentry to improve UI;
for now it says "Error" all the time
- In this PR the default value of IS_CONFIG_VARIABLES_IN_DB_ENABLED has
been changed to true,
- This is my first time writing integration tests, so I’d appreciate a
thorough review. :)
I’ve tried to follow the existing test patterns closely, but there might
be some small mistakes I may have missed.
Also let me know if I have missed any important test cases that should
be tested
UPDATE -
### Config Value Converter Refactoring
- Created a centralized type transformers registry with bidirectional
validation
- Refactored ConfigValueConverterService to support validation in both
directions:
- Maintained existing DB-to-app conversion behavior
- Added validation for app-to-DB conversion
- Added integration tests to verify validation works in both directions
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747
We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.
In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
isEmailVerified was set to false which was annoying in the staging
environment
Also updated password for tim@apple.dev from AppleCar2025 to just
tim@apple.dev since the joke is outdated
We want to have fewer base path for routing.
We will have:
- /files
- /webhooks
- /graphql
- /metadata
- /rest
- /auth
- /healthz
I'm moving /open-api under /rest, and centralizing the webhooks
(removing /stripe and /cloudflare)
