# Introduction
In this PR we've initialized the `workspace-migration-v2` folder.
Focusing on the builder in the first place.
From now it contains:
- Basic temporary types ( `fieldMetadataEntity` and
`ObjectMetadataEntity` )
- Object actions builder ( create, delete, update )
- Fields actions builder ( create, delete ) ( update coming in a
following PR )
We will still have to handle specific conditions such as:
- Index creation
- Uniqueness addition removal
- Relation
We also need to determine when we want to compute and transpile the
object no field `uniqueIdentifier`
We're aiming to merge this first in order to avoid accumulating code in
this PR
---------
Co-authored-by: prastoin <paul@twenty.com>
This PR is raised to close the issue #13044
But there are some doubts that needs to be approved.
If the fields are not custom then we were saving the changes in
**standardOverrides** obj in which only three fields are
overridableFields **label, icon & description** can be updated for a
field.
You can see this in _before-update-one-field.hook.ts_ on line 85
```ts
const overridableFields = ['label', 'icon', 'description'];
```
If the field to be updated are from these three we are putting this in a
**standardOverrides** obj and passing it
However in our _field-metadata.service.ts_ file. We have **updateOne**
function inside it we have wrote a condition if **isCustom** is false
then the purpose was to build the updatableFields from the
**standardOverrides** obj that we got in **fieldMetadataInput** but
there was an error in it. As you can see below
```ts
const updatableFieldInput =
existingFieldMetadata.isCustom === false
? this.buildUpdatableStandardFieldInput(
fieldMetadataInput,
existingFieldMetadata,
)
: fieldMetadataInput;
```
However, the issue was that we were placing the entire
**standardOverrides** object inside **updatableFieldInput** again —
instead of merging its individual fields (label, icon, description)
directly into the update payload.
This PR fixes that by correctly applying the overrides into the
top-level object.
Please refer to the file changes for the full context.
But the thing i don't know.
[ ] - Is this the correct expected flow??
[ ]- Will this change break anything elsewhere?
I still have doubts on these two. Let me know if I missed something.
---------
Co-authored-by: Jagss24 <btwitsjagannat12@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR, I'm fixing a bug introduced in recent performance work on
the cache.
Bug context: https://github.com/twentyhq/twenty/issues/12865
Related PR opened by a contributor:
https://github.com/twentyhq/twenty/pull/13003
## Root cause
We cache all objectMetadataItems at graphql level : see
`useCachedMetadata` hook:
- instead of going through the regular resolvers, we direlcty load data
from the cache. However this data must be localized regarding labels and
descriptions
In a precedent refactoring, we introduced the notion of locale in the
cache key. However, the user locale was not properly taken into account
as we did not have the information in this hook.
## Fix
1. **Introduce locale in userWorkspace entity**. The locale is stored on
workspaceMember in each postgres workspaceSchema (workspace_xxx) which
is the alter ego of userWorkspace in postgres core schema. Note that we
can't store it in user as a user can be part of multiple workspaces (the
locale already there must be seen as a default for this user), and we
cannot rely on workspaceMember as we would need to query the
workspaceSchema in the authentication layer which we want to avoid for
performance reasons.
2. During request hydration from token (containing the userWorkspaceId),
we fetch the userWorkspace and store it in the Request (this impact both
AuthContext and Request interface)
3. Leverage userWorkspace.locale in the useCachedMetadata hook
## Additional notes
There is no need to change the way we store and retrieve the
object-metadata-maps object itself which is different from the graphql
layer cache. object-metadadata-maps are not localized
Context :
- Phones import is a bit complex if not all subfields are provided.
- Phones subfield validation are absent or different from BE validation.
Solution :
- Normalize callingCode and countryCode validation (BE/FE)
- Ease phone import if only phoneNumber is provided
Currently, when a server query or mutation from the front-end fails, the
error message defined server-side is displayed in a snackbar in the
front-end.
These error messages usually contain technical details that don't belong
to the user interface, such as "ObjectMetadataCollection not found" or
"invalid ENUM value for ...".
**BE**
In addition to the original error message that is still needed (for the
request response, debugging, sentry monitoring etc.), we add a
`displayedErrorMessage` that will be used in the snackbars. It's only
relevant to add it for the messages that will reach the FE (ie. not in
jobs or in rest api for instance) and if it can help the user sort out /
fix things (ie. we do add displayedErrorMessage for "Cannot create
multiple draft versions for the same workflow" or "Cannot delete
[field], please update the label identifier field first", but not
"Object metadata does not exist"), even if in practice in the FE users
should not be able to perform an action that will not work (ie should
not be able to save creation of multiple draft versions of the same
workflows).
**FE**
To ease the usage we replaced enqueueSnackBar with enqueueErrorSnackBar
and enqueueSuccessSnackBar with an api that only requires to pass on the
error.
If no displayedErrorMessage is specified then the default error message
is `An error occured.`
When we use a record field in a form, record relations are displayed as
available variables in following step.
But those are actually empty at execution.
When choosing the record in the form and submitting, we enrich the
record id with the full record before starting the workflow again. But
we were not adding the relations to that enrichment.
This PR does not produce any functional change
First step of the workflow branch feature
- add gather `workflowRun.output` and `workflowRun.context` into one
column `workflowRun.runContext`
- add a command to fill `runContext` from `output` and `context` in
existing records
- maintain `runContext` up to date during workflow runs
Modifying the data-model can sometimes fail in the middle of your
operation, due to the way we handle both metadata update and schema
migration separately, a field can be created while the associated column
creation failed (same for object/table and such). This is also an issue
because WorkspaceMigrations are then stored as FAILED can never really
recovered by themselves so the schema is broken and we can't update the
models anymore.
This PR adds a executeMigrationFromPendingMigrationsWithinTransaction
method where we can (and must) pass a queryRunner executing a
transaction, which should come from the metadata services so that if
anything during metadata update OR schema update fails, it rolls back
everything (this also mean a workspaceMigration should never stay in a
failed state now).
This also fixes some issues with migration not running in the correct
order due to having the same timestamp and having to do some weird logic
to fix that.
This is a first step and fix before working on a much more reliable
solution in the upcoming weeks where we will refactor the way we
interact with the data model.
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
# Context
We had an error saying "Unknown error importing calendar events for
[...]: Access token is undefined or empty. Please provide a valid token.
For more help -
https://github.com/microsoftgraph/msgraph-sdk-javascript/blob/dev/docs/CustomAuthenticationProvider.md
"
Reason is that the access token method for microsoft is a bit different
than the one from google. And in microsoft case, we want to check the
access token in the authProvider in case it fails. Currently it was not
catched, so it broke services above that counted on the accesstoken to
be valid.
That ended in UNKNOWN failure for our calendar event fetch service.
# Solution
This PR should solve the issue since :
1. forcing the method to break if accesstoken renewal fails
2. logs will help to know what kind of errors will be sent in case we
need to tackle this issue again
3. we now throw TEMPORARY error instead of unknown, allowing 3
getClientConfig failure before it is definitive
Why so many changes while it should have been simple :
The root cause is the `authProvider` from
`'@microsoft/microsoft-graph-client'` npm package. It does not throw a
custom error, and we cannot catch it on calling `Client.init`. Errors
only occurs when the client from
```
const client = this.microsoftOAuth2ClientManagerService.getOAuth2Client(refreshtoken)
```
is used, as in `client.api('/messages').get().catch(err => [...])`
So we need to go in every call using the client and catch errors, and
rethrow whenver we need as a newly created message type
`MessageImportDriverExceptionCode.CLIENT_NOT_AVAILABLE`
We discussed 1. and 2. with @bosiraphael already
I added 3. to make our system more robust without waiting for more
failures
# Related
Fixes : https://github.com/twentyhq/twenty/issues/12880
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Previous logic was using the previous step output and filtering items
that were passing filters.
What we actually want is:
- send filters, right operand being always a step output key, left
operand being either a key, either a value
- resolve those filter variables
- apply the filters to decide whether the condition is passed or not
Fixes: #12722
The problem is that there is no TS_VECTOR field in workflow objects.
Thus, I have added this field to three objects: workflow,
workflowVersions, and workflowRuns.
---------
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
RestApiExceptionFilter is used as an exception filter for the core
controller which is used for crud operations on our objects (equivalent
of our dynamic queries findManyPeople etc. on the graphql API).
Exceptions were leading a 400 / BadRequestException response status
which can be confusing to users.
By default we should actually throw a 500 if the error was not handled
priorily, but we have not implemented input validation for the REST api
so we fear to be flooded with errors that should not be 500 but 400 due
to user inputs. A solution should be brought [with this
ticket](https://github.com/twentyhq/core-team-issues/issues/1027) but it
has not been prioritized yet.
- new status `ENQUEUED` added. With a command to backfill
- counter in cache per workspace, managed by a new service
[workflow-run-queue.workspace-service.ts](https://github.com/twentyhq/twenty/compare/tt-improve-workflow-run-queueing?expand=1#diff-1e2de2a48cd482a3bd7e8dedf1150a19d0b200afbd9282181a24ecddddb56927)
- cron added that will run every minute to look for not started
workflows
Here is the new flow:
- When executing a workflow, we check if the queue is not full. If not,
run is created as `ENQUEUED` and the run workflow job is triggered as
usual. If full, create the run as NOT_STARTED and do not trigger the job
- Cron will look for NOT_STARTED workflows and queue some if there is
some place again
- Only MANUAL and Form submit skip the queue limit
This PR introduces a significant enhancement to the role-based
permission system by extending it to support AI agents, enabling them to
perform database operations based on assigned permissions.
## Key Changes
### 1. Database Schema Migration
- **Table Rename**: `userWorkspaceRole` → `roleTargets` to better
reflect its expanded purpose
- **New Column**: Added `agentId` (UUID, nullable) to support AI agent
role assignments
- **Constraint Updates**:
- Made `userWorkspaceId` nullable to accommodate agent-only role
assignments
- Added check constraint `CHK_role_targets_either_agent_or_user`
ensuring either `agentId` OR `userWorkspaceId` is set (not both)
### 2. Entity & Service Layer Updates
- **RoleTargetsEntity**: Updated with new `agentId` field and constraint
validation
- **AgentRoleService**: New service for managing agent role assignments
with validation
- **AgentService**: Enhanced to include role information when retrieving
agents
- **RoleResolver**: Added GraphQL mutations for `assignRoleToAgent` and
`removeRoleFromAgent`
### 3. AI Agent CRUD Operations
- **Permission-Based Tool Generation**: AI agents now receive database
tools based on their assigned role permissions
- **Dynamic Tool Creation**: The `AgentToolService` generates CRUD tools
(`create_*`, `find_*`, `update_*`, `soft_delete_*`, `destroy_*`) for
each object based on role permissions
- **Granular Permissions**: Supports both global role permissions
(`canReadAllObjectRecords`) and object-specific permissions
(`canReadObjectRecords`)
### 4. Frontend Integration
- **Role Assignment UI**: Added hooks and components for
assigning/removing roles from agents
## Demo
https://github.com/user-attachments/assets/41732267-742e-416c-b423-b687c2614c82
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: Baptiste Devessier <baptiste@devessier.fr>
Co-authored-by: nitin <142569587+ehconitin@users.noreply.github.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
Co-authored-by: Vicky Wang <157669812+vickywxng@users.noreply.github.com>
Co-authored-by: Vicky Wang <vw92@cornell.edu>
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
### Added IMAP integration
This PR adds support for connecting email accounts via IMAP protocol,
allowing users to sync their emails without OAuth.
#### DB Changes:
- Added customConnectionParams and connectionType fields to
ConnectedAccountWorkspaceEntity
#### UI:
- Added settings pages for creating and editing IMAP connections with
proper validation and connection testing.
- Implemented reconnection flows for handling permission issues.
#### Backend:
- Built ImapConnectionModule with corresponding resolver and service for
managing IMAP connections.
- Created MessagingIMAPDriverModule to handle IMAP client operations,
message fetching/parsing, and error handling.
#### Dependencies:
Integrated `imapflow` and `mailparser` libraries with their type
definitions to handle the IMAP protocol communication.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Context :
- IndexFieldMetadata was no longer available on 'objects' gql query
([since this PR](https://github.com/twentyhq/twenty/pull/12785)). Then,
unicity checks on import do not work anymore.
Fix :
- Add a dataloader logic in indexFieldMetadata
- Add extra check in unicity hook on import
This PR aims at improving readability in sentry and user experience with
runtime errors.
**GraphQL errors (and ApolloError)**
1. In sentry we have a lot of "Object captured as exception with keys:
extensions, message" errors (2k over the last 90d), on which we have
zero information. This is because in apollo-factory we were passing on
GraphQL errors to sentry directly why sentry expects the structure of a
JS Error. We are now changing that, rebuilding an Error object and
attempting to help grouping by creating a fingerPrint based on error
code and truncated operationName (same as we do in the back for 500
graphql errors).
2. In sentry we have a lot of ApolloError, who actually correspond to
errors that should not be logged in sentry (Forbidden errors such as
"Email is not verified"), or errors that are already tracked by back-end
(Postgres errors such as "column xxx does not exist"). This is because
ApolloErrors become unhandled rejections errors if they are not caught
and automatically sent to sentry through the basic config. To change
that we are now filtering out ApolloErrors created from GraphQL Errors
before sending error to sentry:
<img width="524" alt="image"
src="https://github.com/user-attachments/assets/02974829-26d9-4a9e-8c4c-cfe70155e4ab"
/>
**Runtime errors**
4. Runtime errors were all caught by sentry with the name "Error",
making them not easy to differentiate on sentry (they were not grouped
together but all appeared in the list as "Error"). We are replacing the
"Error" name with the error message, or the error code if present. We
are introducing a CustomError class that allows errors whose message
contain dynamic text (an id for instance) to be identified on sentry
with a common code. _(TODO: if this approach is validated then I have
yet to replace Error with dynamic error messages with CustomError)_
5. Runtime error messages contain technical details that do not mean
anything to users (for instance, "Invalid folder ID: ${droppableId}",
"ObjectMetadataItem not found", etc.). Let's replace them with "Please
refresh the page." to users and keep the message error for sentry and
our dev experience (they will still show in the console as uncaught
errors).
Fixes https://github.com/twentyhq/twenty/issues/12726
## Context
Regression introduced in https://github.com/twentyhq/twenty/pull/12639
We now run raw queries for some migrations (column creations for
example) and we created a `typeormBuildCreateColumnSql` util for that.
The issue is that previously we were using typeorm methods which was
using isArray from the input to create $type[] (text[], number[])
properly which was not done in the new `typeormBuildCreateColumnSql`
util (so the type was text, number, etc...)
Edit: actually this was correctly implemented for Enum types (multi
select fields) but not Array type, I've updated the code accordingly
# Introduction
Following https://github.com/twentyhq/twenty/pull/12852
Discovered that:
- `relationCreationPayload` does not seem to be validated through the
input decorators
```ts
// TODO @prastoin implement validation for this with validate nested and dedicated class instance
@IsOptional()
@Field(() => GraphQLJSON, { nullable: true })
relationCreationPayload?: {
targetObjectMetadataId: string;
targetFieldLabel: string;
targetFieldIcon: string;
type: RelationType;
};
```
- Sending an unknown `targetObjectMetadataId` generates an
`internal_server_error` `500` @guillim on the go
## Coverage
```ts
PASS test/integration/metadata/suites/object-metadata/failing-field-metadata-relation-creation.integration-spec.ts
Field metadata relation creation should fail
✓ relation when targetFieldLabel is empty (109 ms)
✓ relation when targetFieldLabel exceeds maximum length (100 ms)
✓ relation when targetObjectMetadataId is unknown (97 ms)
✓ relation when targetFieldLabel contains only whitespace (103 ms)
✓ relation when targetFieldLabel conflicts with an existing field on target object metadata id (108 ms)
Test Suites: 1 passed, 1 total
Tests: 5 passed, 5 total
Snapshots: 5 passed, 5 total
Time: 2.629 s, estimated 3 s
```
Better catching label input
- there were absolutely no check on label when creating the target field
while doing a relation : we crearted these checks here.
- We keep the label quite open to special char as discussed with Felix.
so mostly checking length of label.
- We check that label does not already exists on the targetted object
- making sure the Target fieldinput label is checked before we create
it. The previous checks are not enough since the label goes through
anoteher merthod before going in the database
- validate-metadata-name-is-camel-case.utils.ts : making sure we can use
this error message for metadata name and for target label
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: prastoin <paul@twenty.com>
# Introduction
This PR might have a lot of impact on tested validation
Avoid catching programmatically thrown error
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
