# Context
We had an error saying "Unknown error importing calendar events for
[...]: Access token is undefined or empty. Please provide a valid token.
For more help -
https://github.com/microsoftgraph/msgraph-sdk-javascript/blob/dev/docs/CustomAuthenticationProvider.md
"
Reason is that the access token method for microsoft is a bit different
than the one from google. And in microsoft case, we want to check the
access token in the authProvider in case it fails. Currently it was not
catched, so it broke services above that counted on the accesstoken to
be valid.
That ended in UNKNOWN failure for our calendar event fetch service.
# Solution
This PR should solve the issue since :
1. forcing the method to break if accesstoken renewal fails
2. logs will help to know what kind of errors will be sent in case we
need to tackle this issue again
3. we now throw TEMPORARY error instead of unknown, allowing 3
getClientConfig failure before it is definitive
Why so many changes while it should have been simple :
The root cause is the `authProvider` from
`'@microsoft/microsoft-graph-client'` npm package. It does not throw a
custom error, and we cannot catch it on calling `Client.init`. Errors
only occurs when the client from
```
const client = this.microsoftOAuth2ClientManagerService.getOAuth2Client(refreshtoken)
```
is used, as in `client.api('/messages').get().catch(err => [...])`
So we need to go in every call using the client and catch errors, and
rethrow whenver we need as a newly created message type
`MessageImportDriverExceptionCode.CLIENT_NOT_AVAILABLE`
We discussed 1. and 2. with @bosiraphael already
I added 3. to make our system more robust without waiting for more
failures
# Related
Fixes : https://github.com/twentyhq/twenty/issues/12880
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Previous logic was using the previous step output and filtering items
that were passing filters.
What we actually want is:
- send filters, right operand being always a step output key, left
operand being either a key, either a value
- resolve those filter variables
- apply the filters to decide whether the condition is passed or not
Fixes: #12722
The problem is that there is no TS_VECTOR field in workflow objects.
Thus, I have added this field to three objects: workflow,
workflowVersions, and workflowRuns.
---------
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
- new status `ENQUEUED` added. With a command to backfill
- counter in cache per workspace, managed by a new service
[workflow-run-queue.workspace-service.ts](https://github.com/twentyhq/twenty/compare/tt-improve-workflow-run-queueing?expand=1#diff-1e2de2a48cd482a3bd7e8dedf1150a19d0b200afbd9282181a24ecddddb56927)
- cron added that will run every minute to look for not started
workflows
Here is the new flow:
- When executing a workflow, we check if the queue is not full. If not,
run is created as `ENQUEUED` and the run workflow job is triggered as
usual. If full, create the run as NOT_STARTED and do not trigger the job
- Cron will look for NOT_STARTED workflows and queue some if there is
some place again
- Only MANUAL and Form submit skip the queue limit
This PR introduces a significant enhancement to the role-based
permission system by extending it to support AI agents, enabling them to
perform database operations based on assigned permissions.
## Key Changes
### 1. Database Schema Migration
- **Table Rename**: `userWorkspaceRole` → `roleTargets` to better
reflect its expanded purpose
- **New Column**: Added `agentId` (UUID, nullable) to support AI agent
role assignments
- **Constraint Updates**:
- Made `userWorkspaceId` nullable to accommodate agent-only role
assignments
- Added check constraint `CHK_role_targets_either_agent_or_user`
ensuring either `agentId` OR `userWorkspaceId` is set (not both)
### 2. Entity & Service Layer Updates
- **RoleTargetsEntity**: Updated with new `agentId` field and constraint
validation
- **AgentRoleService**: New service for managing agent role assignments
with validation
- **AgentService**: Enhanced to include role information when retrieving
agents
- **RoleResolver**: Added GraphQL mutations for `assignRoleToAgent` and
`removeRoleFromAgent`
### 3. AI Agent CRUD Operations
- **Permission-Based Tool Generation**: AI agents now receive database
tools based on their assigned role permissions
- **Dynamic Tool Creation**: The `AgentToolService` generates CRUD tools
(`create_*`, `find_*`, `update_*`, `soft_delete_*`, `destroy_*`) for
each object based on role permissions
- **Granular Permissions**: Supports both global role permissions
(`canReadAllObjectRecords`) and object-specific permissions
(`canReadObjectRecords`)
### 4. Frontend Integration
- **Role Assignment UI**: Added hooks and components for
assigning/removing roles from agents
## Demo
https://github.com/user-attachments/assets/41732267-742e-416c-b423-b687c2614c82
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
Co-authored-by: martmull <martmull@hotmail.fr>
Co-authored-by: Thomas Trompette <thomas.trompette@sfr.fr>
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: Baptiste Devessier <baptiste@devessier.fr>
Co-authored-by: nitin <142569587+ehconitin@users.noreply.github.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
Co-authored-by: Vicky Wang <157669812+vickywxng@users.noreply.github.com>
Co-authored-by: Vicky Wang <vw92@cornell.edu>
Co-authored-by: Raphaël Bosi <71827178+bosiraphael@users.noreply.github.com>
### Added IMAP integration
This PR adds support for connecting email accounts via IMAP protocol,
allowing users to sync their emails without OAuth.
#### DB Changes:
- Added customConnectionParams and connectionType fields to
ConnectedAccountWorkspaceEntity
#### UI:
- Added settings pages for creating and editing IMAP connections with
proper validation and connection testing.
- Implemented reconnection flows for handling permission issues.
#### Backend:
- Built ImapConnectionModule with corresponding resolver and service for
managing IMAP connections.
- Created MessagingIMAPDriverModule to handle IMAP client operations,
message fetching/parsing, and error handling.
#### Dependencies:
Integrated `imapflow` and `mailparser` libraries with their type
definitions to handle the IMAP protocol communication.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Context :
- IndexFieldMetadata was no longer available on 'objects' gql query
([since this PR](https://github.com/twentyhq/twenty/pull/12785)). Then,
unicity checks on import do not work anymore.
Fix :
- Add a dataloader logic in indexFieldMetadata
- Add extra check in unicity hook on import
This PR improves error handling in `handleDriverException` by adding a
duck-typing check for `MessageImportDriverException` objects. It ensures
that errors are correctly identified and processed even if they are
received as plain objects rather than class instances. This change
prevents missed exception handling and increases the robustness of the
message import process.
This PR fixes this issue from the connected account refresh token
service that is
This PR fixes error handling in `handleDriverException` by ensuring that
errors resembling `MessageImportDriverException` are correctly detected,
even if they are plain objects and not true class instances. This
prevents missed exception handling due to failed `instanceof` checks.
Was introduced by [this
PR](https://github.com/twentyhq/twenty/pull/12233) that did not know all
provider cases that can occur.
Fixes https://github.com/twentyhq/twenty/issues/12589
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
Why : we had an issue impoting events du to CalendarEvents not yet
existing while inserting the CalendarChannelAssociation due to inverted
method in the service
This PR refactors the calendar event import logic by
- renaming
- splitting utility functions for better clarity and maintainability.
- adding TSDoc comments to explain the purpose and uniqueness of the
`eventExternalId` field in calendar event associations
Fixes#12690
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Fix inconsistent domain URL formats : removing the last / that was
caused by URL method
Standardize URL formatting to ensure consistent links storage and
retrieval of domain URLs across the application. Will improve the
dedpulicates in the links
Note: there is another temporary issue from google that was solved on
the 13th of june https://groups.google.com/g/adwords-api/c/tRSQMRZrJYM
but we consider this out of this scope
Fixes#12621
## Done
Update manually (without status update job) workflow and
workflowVersions statuses when workflow is deleted
## Not Done
Status optimistic rendering on workflow index deleted page. This page is
already buggy, this will be fix by
https://discord.com/channels/1130383047699738754/1384177035244732487
After release 55, we found out that CRON job monitor was red for
CronTriggerCronJob
While only 1 workspace was not in the appropriate state, meaning the
whole command was probably failing for only 1 workspace failing.
We suggest here to catch errors per worksspace and simply push to sentry
the error of the errored workspace relative to workflow trigger.
Closes#12303
### What’s Changed
- Replace auto‐save with explicit Save / Cancel
Webhook forms now use manual “Save” and “Cancel” buttons instead of the
old debounced auto‐save/update.
- Separate “New” and “Detail” routes
Two dedicated paths `/settings/webhooks/new` for creation and
/`settings/webhooks/:webhookId` for editing, making the UX clearer.
- URL hint & normalization
If a user omits the http(s):// scheme, we display a “Will be saved as
https://…” hint and automatically default to HTTPS.
- Centralized validation with Zod
Introduced a `webhookFormSchema` for client‐side URL, operations, and
secret validation.
- Storybook coverage
Added stories for both “New Webhook” and “Webhook Detail”
- Unit tests
Added tests for the new `useWebhookForm` hook
In this PR
1. fix workflow step creation by adding forgotten
`shouldBypassPermissionChecks` in WorkflowVersionStepWorkspaceService
2. clarify the rule for twentyORMGlobalManager: do not add unnecessary
`shouldBypassPermissionChecks` for system objects (there are no
object-records permission checks on system objects, they are dealt with
at resolver level)
In this PR
- Determine object record permissions on workflows objects (workflow,
workflowVersion, workflowRun) base on settings permissions @Weiko
- Add Workflow permission guards on workflow resolvers @thomtrp . **Any
method within a resolver that has the SettingsPermission Guard is only
callable by a apiKey or a user that has the permission** (so not by
external parties).
- Add checks bypass in workflow services since 1) for actions gated by
settings permissions, the gate should be done at resolver level, so it
will have been done before the call to the service 2) some service
methods may be called by workflowTriggerController which is callable by
external parties without permissions (ex:
workflowCommonWorkspaceService.getWorkflowVersionOrFail). This is
something we may want to change in the future (still to discuss), by
removing the guard at resolver-level and relying on
shouldBypassPermissionChecks at getRepository and made in a way that we
only bypass for external parties.
- Add checks bypass for actions performed by workflows since they should
not be restricted in our current vision
- Add tests
In this PR
1. Add missing override of insert() method on
WorkspaceSelectQueryBuilder to return our custom
WorkspaceInsertQueryBuilder with permission checks.
2. Replace override implementation of methods on WorkspaceEntityManager
that call createQueryBuilder at a nested internal layer of typeORM (i.e.
not directly in the initial implementation of EntityManager - unlike
findBy for instance -, but in calls done under the hood at a level which
would force us to override entire other classes to pass on our
permissionOptions. It is the case for methods which call typeORM's
EntityPersistExecutor for instance.), to validate permissions and then
allow the subsequent calls to be made without permission checks
3. adapt tests
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
First PR to add filters to send records. Lot of work left, but I want to
split. I mainly want to validate the architecture there.
https://github.com/user-attachments/assets/63375a75-ba88-49df-8c12-5e3e58de5342
TODO in next PRs:
- fix design
- make filters reliable. Some composite fields are not implemented and
some fields like datetime do not work well
- improve typing
# Improved participant matching with additional emails support
Closes#8991
This PR extends the participant matching system to support additional
emails in addition to primary emails for both calendar events and
messages. Previously, the system only matched participants based on
primary emails, missing matches with secondary email addresses.
- Contact creation now consider both primary and additional emails when
checking for existing contacts
- Calendar and message participant listeners now handle both primary and
additional email changes
- Added tests
## To test this PR:
Check that:
- Primary emails take precedence over additional emails in matching
- Case-insensitive email comparisons work correctly
- A contact is not created if a person already exists with the email as
its additional email
- Event listeners handle both creation and update scenarios
- Matching and unmatching logic works for complex email change scenarios
- When unmatching after a change in a primary or secondary email, events
and messages should be rematched if another person has this email as its
primary or secondary email.
---------
Co-authored-by: guillim <guigloo@msn.com>
I believe that some emails with invalid characters are breaking the sync
process.
this PR attempts to create a "safeParseAddress" function. Hopefully this
will change current behavior of a single email breaking the entire sync
process to the sync process "skipping" an invalid email address and
continuing on.
I opened this because of issues explained in #12336
---------
Co-authored-by: guillim <guigloo@msn.com>
