## Context
- Introduced objectPermissions in currentUserWorkspace which uses role
permissions from cache so we can fetch granular permissions from the API
- Refactored cached role permissions to map permissions with object
metadata id instead of object metadata name singular to be more flexible
New Cache
<img width="574" alt="Screenshot 2025-05-27 at 11 59 06"
src="https://github.com/user-attachments/assets/1a090134-1b8a-4681-a630-29f1472178bd"
/>
GQL
<img width="977" alt="Screenshot 2025-05-27 at 11 58 53"
src="https://github.com/user-attachments/assets/3b9a82b0-6019-4a25-a6e2-a9e0fb4bb8a0"
/>
Next steps: Use the updated API in the FE to fetch granular permissions
and update useHasObjectReadOnlyPermission hook
I encountered a bug where I was missing permissions while calling
searchResolver because the repository from
`twentyORMManager.getRepository` was missing permissions itself.
The repository was returned from the cached repositories map using a
repository key feature the roleId, the rolesVersion and
featureFlagMapVersion.
I was not able to reproduce but this error should not go unnoticed: we
always expect to find objectPermissions for every roleId in the
datasource now.
I was not able to understand what happened for now but I think throwing
the error will help keeping an eye on it
Introducing a class of RelationException extending CustomException to
help grouping those exception in sentries by ExceptionCode.
I did not introduce a filter as these are thrown in utils that can be
used in multiple places now or in the future, and filters are to be
added at resolver-level.
- fix missing createBy injection in api createOne and createMany
endpoints
- add a command to fix null default value for createdBySource in
production entities
- tested on `1747159401197/` dump extract of production db without issue
Fix the following error:
Cannot use a pool after calling end on a pool
<img width="917" alt="Screenshot 2025-05-17 at 14 56 18"
src="https://github.com/user-attachments/assets/63081831-9a7e-4633-8274-de9f8a48dbae"
/>
The problem was that the datasource manager was destroying the
connections when a datasource cache expired.
This PR implements a global PostgreSQL connection pool sharing
mechanism.
- Patches pg.Pool to reuse connection pools across the application when
connection parameters match, reducing resource overhead.
- New environment variables allow enabling/disabling sharing and
configuring pool size, idle timeout, and client exit behavior.
WorkspaceDatasourceFactory will now use shared pools if enabled, this
will avoid recreating 10 connections for each pods for each workspace.
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Follow-up on https://github.com/twentyhq/twenty/pull/12007
In this PR
- adding a filter on HttpExceptionHandlerService to filter out 4xx
errors from driver handling (as we do for graphQL errors: see
useGraphQLErrorHandler hook - only filteredIssues are sent to`
exceptionHandlerService.captureExceptions()`.)
- grouping together more missing metadata issues
- attempting to use error codes as issues names in sentry to improve UI;
for now it says "Error" all the time
As discussed with @Weiko
Even though we cache the datasource, the connection expire after
10minutes in TypeORM, that might be the reason why our app is spamming
the proxy asking for connections. Also lowering the pool size.
# Introduction
Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder
Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles
## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc
## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment
## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
ignoreRestArgs?: boolean;
```
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
Fixes https://github.com/twentyhq/twenty/issues/11864 and
https://github.com/twentyhq/core-team-issues/issues/908
We should not send `createManyXXX` mutations with FE-forged ids in the
payload if we want to do an upsert, because that 1) prevents records
from being merged 2) triggers optimistic rendering while we can't know
before-hand which records will actually be created and which records
will only be updated
Also noticed createdBy was being overriden even for records we are
updating and not creating, which did not seem right, so fixed that too
This PR attemps at improving sentry grouping and filtering by
- Using the exceptionCode as the fingerprint when the error is a
customException. For this to work in this PR we are now throwing
customExceptions instead of internalServerError deprived of their code.
They will still be converted to Internal server errors when sent back as
response
- Filtering 4xx issues where it was missing (for emailVerification
because errors were not handled, for invalid captcha and billing errors
because they are httpErrors and not graphqlErrors)
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
This is a first PR to remove old relation logic
Next steps:
- remove relationMetadata from cache
- remove relationMetadata table content and structure
- refactor relationDefinition to leverage field.settings instead
## Context
While deploying the IS_NEW_RELATION_ENABLED (we don't compute relation
based on relationMetadata anymore) to existing workspace, I've tested to
run a sync-metadata post feature flag activation. This has raised two
issues:
- the workspaceMigration generator (which is over-complex and should be
refactored later) for fieldMetadata of type RELATION was not handling
settings update properly ;
- we need to delete existing fieldMetadata corresponding to the UUID
foreignKey as they are not needed anymore. This is handled as a 0.53
upgrade command as 0.53 will also come with the full removal of the old
relation system
---------
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
In this PR we are
1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747
We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.
In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
# Introduction
Followup of https://github.com/twentyhq/twenty/pull/11784
Again some propaganda for the
[noUncheckedIndexedAccess](https://www.typescriptlang.org/tsconfig/#noUncheckedIndexedAccess)
that involved this @charlesBochet 👀
That's very risky to have this setup to false especially in the backend
regarding the twenty-server metadata nature
Also suggested that we do some integration testing ( e2e nestsJs tests )
on a related endpoint that could related we always retrieve the same
form result output
We could also do some unit testing of the method but like the idea to
ship it through the api itself
## Introduction
This PR enables functionality discussed in [Layout Date
Formatting](https://github.com/twentyhq/core-team-issues/issues/97).
### TLDR;
It enables greater control of date formatting at the object's field
level by upgrading all DATE and DATE_TIME fields' settings from:
```ts
{
displayAsRelativeDate: boolean
}
```
to:
```ts
type FieldDateDisplayFormat = 'full_date' | 'relative_date' | 'date' | 'time' | 'year' | 'custom'
{
displayFormat: FieldDateDisplayFormat
}
```
PR also includes an upgrade command that will update any existing DATE
and DATE_TIME fields to the new settings value
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
In this PR we are
- introducing a cached map `{ userworkspaceId: roleId } `to reduce calls
to get a userWorkspace's role (we were having N+1 around that with
combinedFindMany queries and generally having a lot of avoidable
queries)
- using the roles permissions cache (`{ roleId: { objectNameSingular:
{ canRead: bool, canUpdate: bool, ...} } `) in Permissions V1's
userHasObjectPermission, in order to 1) improve performances to avoid
calls to get roles 2) start using our permissions cache
In this PR we are
- (if permissionsV2 is enabled) executing permission checks at query
builder level. To do so we want to override the query builders methods
that are performing db calls (.execute(), .getMany(), ... etc.) For now
I have just overriden some of the query builders methods for the poc. To
do so I created custom query builder classes that extend typeorm's query
builder (selectQueryBuilder and updateQueryBuilder, for now and later I
will tackle softDeleteQueryBuilder, etc.).
- adding a notion of roles permissions version and roles permissions
object to datasources. We will now use one datasource per roleId and
rolePermissionVersion. Both rolesPermissionsVersion and rolesPermissions
objects are stored in redis and recomputed at role update or if queried
and found empty. Unlike for metadata version we don't need to store a
version in the db that stands for the source of truth. We also don't
need to destroy and recreate the datasource if the rolesPermissions
version changes, but only to update the value for rolesPermissions and
rolesPermissionsVersions on the existing datasource.
What this PR misses
- computing of roles permissions should take into account
objectPermissions table (for now it only looks at what's on the roles
table)
- pursue extension of query builder classes and overriding of their db
calling-methods
- what should the behaviour be for calls from twentyOrmGlobalManager
that don't have a roleId?
Comparing the last date a datasource was used instead of a fixed TTL :
should fix workers issues "error: Error: Connection terminated"
FYI was done in pair prog with @Weiko
## Context
This fix ensures that even if a datasource creation promise throws and
is cached, subsequent requests won't return that cached exception.
Also adding a TTL on MetadataObjectMetadataOngoingCachingLock, this is
not something that should stay in the cache forever and could
potentially unlock some race conditions (the origin of the issue is
probably due to performances where the lock is not removed as it should
be after metadata computation and caching)
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
Simplifying a lot the upgrade system.
New way to upgrade:
`yarn command:prod upgrade`
New way to write upgrade commands (all wrapping is done for you)
```
override async runOnWorkspace({
index,
total,
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {}
```
Also cleaning CommandModule imports to make it lighter
Proposal:
- Add a method in ActiveWorkspaceCommand to loop over workspace safely
(add counter, add try / catch, provide datasource with fresh cache,
destroy datasource => as we do always do it)
Also in this PR:
- make sure we clear all dataSources (and not only the one on metadata
version in RAM)
## Context
In some CustomException exceptions, we were instantiating a code without
initializing it which was overriding the parent code and it was then
lost when retrieving it in filters.
Removing them to make sure we don't reproduce this pattern
## Context
All objects have '...duplicates' resolver but only companies and people
have duplicate criteria (hard coded constant).
Gql schema and resolver should be created only if duplicate criteria
exist.
## Solution
- Add a new @WorkspaceDuplicateCriteria decorator at object level,
defining duplicate criteria for given object.
- Add a new duplicate criteria field in ObjectMetadata table
- Update schema and resolver building logic
- Update front requests for duplicate check (only for object with
criteria defined)
closes https://github.com/twentyhq/twenty/issues/9828
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function
## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place
Related PR https://github.com/twentyhq/twenty/pull/9941
