Catching "no licence - removed" microsoft message channels.
Current behabiour
> ` MessageImportException [Error]: The mailbox is either inactive,
soft-deleted, or is hosted on-premise.`
Goal:
better track errors VS user mistakes
Context:
A similar logic was already implemented for the calendar channels. I
just replicated it to message channels
# Introduction
Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder
Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles
## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc
## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment
## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
ignoreRestArgs?: boolean;
```
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
Context
workspaceMemberId is not always present in AuthContext. Solution
implemented here is to fetch workspaceMemberId via userId.
QRQC coming ...
Tested
- FindManyCalendarEvents / FindOneCalendarEvent
- FindManyMessages / FindOneMessage
closes https://github.com/twentyhq/twenty/issues/12027
This PR attemps at improving sentry grouping and filtering by
- Using the exceptionCode as the fingerprint when the error is a
customException. For this to work in this PR we are now throwing
customExceptions instead of internalServerError deprived of their code.
They will still be converted to Internal server errors when sent back as
response
- Filtering 4xx issues where it was missing (for emailVerification
because errors were not handled, for invalid captcha and billing errors
because they are httpErrors and not graphqlErrors)
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Several users have complained about not being able to read their emails
anymore.
This is because the find-messages post query hook is expecting
ObjectRecord[] as an input but is actually getting a graphql Connection
Typing was wrong. This PR fixes the typing and make sure the post query
hook always get an ObjectRecord[]
# TLDR
fix bug due to some event properties coming from the google calendar API
containing weird characters like this
"\u0000�4\u000b\u00042��K\u0001�z,\u001cm�",
it made the Postgres select and insert operator fail
## Details
We can have event properties (like cal UID below) encoded in a strange
way. From my research, the character \u0000 comes from `C` language to
signal end of line. It is wrongly interpreted by Postgres so must be
escaped. I decided to remove all possibility of failure with this regex
`[^\x20-\x7E]` basically "any character that is not a printable ASCII
character"
```
[
"5foijj28qb8smqiafjablo17vd@google.com",
"\u0011�\"�f�\\\u0019G_=��\u0005]x",
"?}|��\f}l��+�弴�",
"%���?t\u0007��n\u001e\u000eY�T<",
".\u0011�\u0016�!�\u000eIǹ� ��\u001f",
"!h\u0004��DŽ�6���h�]E",
"(�CX]�Q�7�^��n\u0006�",
"_040000008200E00074C5B7101A82E0080000000070105B958DEFD801000000000000000010000000EA30DB22E888B943A8EE0AD483F8DB35",
"\t�N�#�D��Ic�h",
"+�)�H���jJ|Ժ�'�",
"_040000008200E00074C5B7101A82E0080000000070A54736C6EED8010000000000000000100000006502334AFE61904595C2831FA4391034",
"_040000008200E00074C5B7101A82E0080000000072C80C3590EFD8010000000000000000100000001BA9FD5B330C1A4D85462AC9D70B9D9D",
"sg�fvUa:St>-<�d\u0006",
"\u0017ڦ��_\u001e\u001fGm-1����",
"_040000008200E00074C5B7101A82E00800000000F0F4F01F4EF0D8010000000000000000100000004C11CE0950C85549B79C456C13987AB8",
"$�����\u0007V\u0007��\u001e�OLN",
"_040000008200E00074C5B7101A82E00800000000341DA81151EDD8010000000000000000100000007453CEFB19AA2D4899B17F0BDB000493",
"_01C756CA-98DC-4799-9F06-883A540A065C",
"_040000008200E00074C5B7101A82E00800000000919548BBF1EDD80100000000000000001000000050AE1E41F3CD314CA9215F193EBE1D39",
"_040000008200E00074C5B7101A82E0080000000039CF64D718EDD801000000000000000010000000B3824EF0711436488CB5459BE83733B1",
"_040000008200E00074C5B7101A82E00800000000C50BDADCB5E4D8010000000000000000100000005B95FE762B2EF84B9C5AC53907B7E5E8",
"�Spx�\u0003ve��ss�X��",
"\b���>\u0013̈�ыh��0�",
"_040000008200E00074C5B7101A82E008000000005BCD492230E8D801000000000000000010000000B243A9CE99E94C4DAD201129A8F2A2F7",
"_040000008200E00074C5B7101A82E008000000009B540B82D1DCD801000000000000000010000000B4AFC9825D94994AA6C528C953BD3D96",
"_040000008200E00074C5B7101A82E008000000000D39EABDB7E4D801000000000000000010000000CD07BA2D05E61B47A597EE538ECE8CA5",
"\u0016���\u0003inv�=O����",
"4?b�-���\u001c\u0013ת�E�p",
">\u0000\u0015-;_�^�W&�p\u001f�",
"_040000008200E00074C5B7101A82E008000000006258818C85D9D80100000000000000001000000003346C625768FE43AF3F8D09917CA3C9",
"+K��ٔ�\u0006�\u0018G\u000b\u0000�s\u000e",
"/\f�\rj�IOD�g脅��",
"_68d820f9-e2c1-4d9a-bf61-83e4957c8261",
"xĠ�W4>���t�\u001d���",
"_040000008200E00074C5B7101A82E008000000003F1F314328E3D8010000000000000000100000002C8DC60F5369F44DA2B066441F882B35",
"_040000008200E00074C5B7101A82E008000000006DFCC204FDDED80100000000000000001000000046ECD444737D2E4992C349B2EA05637F",
"_1FF7C1BB-4E1A-485F-ADD0-5C4768601179",
"_040000008200E00074C5B7101A82E00800000000A0AA1223EFD3D801000000000000000010000000C83B72C2B424944199F3353D1442B27E",
"_040000008200E00074C5B7101A82E0080000000063D101EB06E4D801000000000000000010000000E489F61C89B8914BA6F8C8A2606405FE",
"\u0011��f�\"�b���rB�[�",
";t��\u001d\u001euDY+T\u001d��v",
]
```
Fixes https://github.com/twentyhq/core-team-issues/issues/946
Fixes
https://twenty-v7.sentry.io/issues/6568530279/?environment=prod&project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D%20b09d7a84-e6bc-45cf-b3ca-1e6047dddeed&referrer=issue-stream&stream_index=0
### Edit :
Changed the regex to match the chars to remove from
`[^\x20-\x7E]` to `replace('\u0000', '');`
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747
We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.
In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
Preview :
<img width="501" alt="Screenshot 2025-05-02 at 16 24 34"
src="https://github.com/user-attachments/assets/0c649df1-0e26-4ddc-8e13-ebd78af7ec09"
/>
Done :
- Fix getCalendarEventsFromPersonIds and getCalendarEventsFromCompanyId
(include accountOwner check)
- Fix permission check on pre-hook - Pre-hook seems useless, calendar
events are always on METADATA or SHARE_EVERYTHING visibility, else post
hook always has the responsibility of returning the data user can
access. >> To delete or to keep in case other visibility options are
added ?
- Add post hook to secure finOne / findMany calendarEvents resolver
- Update design
To do :
- same on messages (PR to arrive)
closes : https://github.com/twentyhq/twenty/issues/9826
Better error logging for messaging import exception handler.
Goal is to have better info on why Unknown errors are thrown and avoid
such messages `Unknown error occurred while importing messages for
message channel XXXXXXXX in workspace YYYYYYYYYY: Unknown error occurred
while importing messages for message channel XXXXXXXX...`
Done :
- move metrics and health cache services from health module to metrics
module
- refactor metrics counter from specific method to set up from enum keys
- add OpenTelemetry (Otel) instrumentation for metrics
- set up Otel SDK to send metrics to Otel collector
To do later :
- implement Otel instrumentation for traces + plug Sentry on top
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
### Context
For calendar and message sync job health monitoring, we used to
increment a counter in redis cache which could lead to concurrency
issue.
### Solution
- Update to a set structure in place of counter + use sAdd redis method
which is atomic
- Each minute another counter was incremented on a new cache key ->
Update to a 15s window
- Remove ONGOING status not needed. We only need status at job end (or
fail).
### Potential improvements
- Check for cache key existence before fetching data to avoid useless
call to redis ?
closes https://github.com/twentyhq/twenty/issues/10070
This PR is supposed to solve an issue with the syncrhonisation of
messages, specifically with microsoft driver. Microsoft calls don't need
access_Token so refreshing toekns was not implemented.
However, microsoft rely on its client which calls its refresfh_token,
and I might have missed some underlying dependency from microsoft
impelemtation so I setup the access token process to refresh it
Needs a talk before to be merged
Fix : https://github.com/twentyhq/twenty/issues/10367
EDIT:
it was a problem with microsoft making refreshtoken expire (contrarily
to google) which needs to be handled.
## Context
In some CustomException exceptions, we were instantiating a code without
initializing it which was overriding the parent code and it was then
lost when retrieving it in filters.
Removing them to make sure we don't reproduce this pattern
Fixes#9827
Also uncovered a conflict with `@objectType('Relation')` and
`@objectType('relation)`
I don't want to address it in this PR so I will create a followup issue
when we close this but I think there's a confusion between
Relation/RelationMetadata, it's unclear what is what
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
In this PR:
- migrate WorkspaceActivationStatus to twenty-shared (and update case to
make FE and BE consistent)
- introduce isWorkspaceActiveOrSuspended in twenty-shared
- refactor the code to use it (when we fetch data on the FE, we want to
keep SUSPENDED workspace working + when we sync workspaces we want it
too)
- [ ] Catch this specific `500` error
- [ ] Make sure catched `500` errors are sent to sentry for the Cloud
version
- [ ] Hide the option to sync email with google in this situation where
the according env var is missing
- [x] Add Worskpace information to all catched errors for better
debugging
fix#8607
Implemented:
* Account Connect
* Calendar sync via delta ids then requesting single events
I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
## Context
ObjectMetadataService became quite large and handles too many
responsibilities. I'm trying to refactor a bit this part in preparation
of a larger work that will combine object-metadata services and
sync-metadata logic
- Created a STANDARD_OBJECT_ICONS that can be reused in relation
creation to refer to a standard object icon.
- Created a STANDARD_OBJECT_FIELD_IDS that can be used with an object
name to get its standard field ids.
- Moved migration, record and relation creations to dedicated services,
refactored to improve performances and readability
- Refactored some validation logic
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR:
- remove deprecated EMAIL, PHONE, LINK field types (except for Zapier
package as there is another work ongoing)
- remove composite currency filter on currencyCode, actor filter on name
and workspaceMember as the UX is not great yet
Fix syncedAt no longer been set on message sync.
Fix calendar data model:
- Add `syncedAt` to `CalendarChannelWorkspaceEntity`
- Move `recurringEventExternalId` from `CalendarEventWorkspaceEntity` to
`CalendarChannelEventAssociationWorkspaceEntity` since the id is
relative to one channel
Fix save queries on calendar sync after regression.
## Context
Name shouldn't be added to all tables, especially standard objects
because they already have their own labelIdentifierFieldMetadata
specified in the workspace-entity schema. This PR removes this column
from the "base" list of columns to add when creating a new object/table
and moves it to the object-metadata service that is, as of today, only
used for custom objects. Also had to modify the migration-runner to
handle column creation in a table creation migration (this was available
in the migration definition already but was not doing anything)
This also fixes an issue in standard objects that already have a "name"
field defined with a different field type, this is even more important
when the said field is a composite field. For example people already has
a FULL_NAME name field which clashes with the default TEXT name field
meaning it was only creating 1 field metadata for 'name' but 3 columns
were created: `name, nameFirstName, nameLastName`. This inconsistency
with metadata (which is our source of truth everywhere) brought some
issues (lately, converting back typeorm response to gql (including
composition) was broken).
- Add message deletion and thread cleaning during full message list
fetch
- Add thread cleaning during partial message list fetch
- Delete provider from cache key
