admin/twenty
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
2,734 Commits 1 Branch 0 Tags
756de8a31b5f74ff790fcea7e464a4a9cea2c3ec
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Weiko f4fda221b7 Fix cron module structure (#4933) 
This PR introduces a new folder structure for business modules.
Cron commands and jobs are now stored within the same module/folder at
the root of the business module
e.g: /modules/messaging/crons/commands instead of
/modules/messaging/commands/crons
Patterns are now inside their own cron-command files since they don't
need to be exported
Ideally cronJobs and cronCommands should have their logic within the
same class but it's a bit harder than expected due to how commanderjs
and our worker need both some class heritage check, hence the first
approach is to move them in the same folder

Also Messaging fullsync/partialsync V2 has been dropped since this is
the only used version => Breaking change for ongoing jobs and crons.
Jobs can be dropped but we will need to re-run our crons (only
cron:messaging:gmail-fetch-messages-from-cache)
 2024-04-12 14:43:03 +02:00
Weiko dc542a395e fix default value for message channel enums (#4934) 2024-04-11 18:42:31 +02:00
Weiko fc56775c2a [calendar/messaging] fix connected account auth failed should skip sync (#4920) 
- AuthFailedAt is set when a refreshToken is not valid and an
accessToken can't be generated, meaning it will need a manual action
from the user to provide a new refresh token.
- Calendar/messaging jobs should not be executed if authFailedAt is not
null.
 2024-04-11 17:57:48 +02:00
Weiko 4f2c29dce0 uuid codegen update (#4897) 
Following https://github.com/twentyhq/twenty/pull/4895/files
 2024-04-09 18:06:39 +02:00
Weiko 19df43156e [calendar] change api scope (#4888) 
## Context
Calendar scope was too broad, this PR updates it to events only.
Also changing "Cannot connect Google account to demo workspace" error to
a 404 to avoid having a 500 for something expected
 2024-04-09 11:18:48 +02:00
Weiko ab60b8be65 [messaging]Add temporary capture to gmail refresh token exceptions (#4875) 
## Context
This exception is currently caught since this is expected but it seems
to be rejected more than it should so we want to have more visibility on
it

## Test
<img width="562" alt="Screenshot 2024-04-08 at 11 32 28"
src="https://github.com/twentyhq/twenty/assets/1834158/43bb6de9-191a-42d4-911b-6e83c7d8aa18">
 2024-04-08 13:23:31 +02:00
Weiko f8da8f9805 [messaging] remove v2 feature flag (#4845) 
## Context
We are now removing Messaging V2 feature flag to use it everywhere.

## Implementation
- renaming FetchWorkspaceMessagesCommandsModule to
MessagingCommandModule to make it more generic since it it hosts all
commands related to the messaging module
- creating a crons folder inside commands and jobs crons should be named
with xxx.cron.command.ts instead of xxx.command.ts. Same for jobs, jobs
should be named with xxx.cron.job.ts. In a future PR we should make sure
those CronJobs implement a CronJob interface since it's a bit different
(a CronJob does not contain a payload compared to a Job)
- Cron commands have been renamed to "cron:$module:command" so
`fetch-all-workspaces-messages-from-cache:cron:start` has been renamed
to `cron:messaging:gmail-fetch-messages-from-cache`. Also having to
create a command to stop the cron is a bit painful to maintain so I
removed them for now, this can be easily done manually with pg-boss or
bull-mq
- Removing full-sync and partial-sync commands as they were there for
testing only, we might put them back at some point but we will have to
adapt the code anyway.
- Feature flag has been removed from the MessageChannel standard object
to make sure those new columns are created during the next sync-metadata
 2024-04-05 16:59:48 +02:00
Weiko e0918c89c1 Fix contact creation when calendar is not enabled (#4843) 
## Context
Calendar tables are behind a featureFlag, they do not exist if the
feature flag is off which means we should not use them for the same
reason. I'm adding a check on the featureFlag before calling the
repository.

```
Error executing raw query for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419: relation "workspace_1wgvd1injqtife6y4rvfbu3h5.calendarEventParticipant" does not exist
```
## Test
locally with and without featureflag
 2024-04-05 15:55:12 +02:00
Weiko f8edb6652e Gracefully close Redis connection when cacheStorageModule is destroyed (#4812) 
## Context
When running a command, the process should end normally however it stays
hanging due to the open connection with redis client (when
CACHE_STORAGE_TYPE=redis)
This PR adds the necessary logic to gracefully close the connection once
the module is destroyed. Thanks to that, the command process now
properly ends once executed.
 2024-04-04 16:15:22 +02:00
Weiko 35ddb9acb5 [messaging] rename syncExternalId to syncCursor (#4752) 
## Context

SyncExternalId should be renamed because this won't always represent an
id. For example, microsoft API does not use ids but dates for their
sync. Also we think external is a bit redundant so we are removing it.

Note: this field is not used by the front-end (and will probably never
be)
 2024-04-02 18:18:43 +02:00
Weiko a3a15957f4 Revert company address field type (#4737) 
## Context
A new ADDRESS field type has been introduced and the company object has
been updated to use this new type however this introduced a few
regressions.
The good strategy would be to introduce a new field and rename the old
one.

This PR revert that change to fix the issue.
 2024-04-02 11:11:14 +02:00
Weiko 1829f4d009 Switch timestamp to timestamptz (#4696) 
* Switch timestamps to timestamptz

* update standard/custom objects logic to use timestamptz

* fix test
 2024-03-28 22:39:41 +01:00
Weiko 22d4af2e0c Fix events being created for system objects (#4694) 
* Fix events being created for system objects

* move the condition above to avoid unnecessary db calls
 2024-03-28 16:15:14 +01:00
Weiko 5c40e3608b [messaing] improve messaging import (#4650) 
* [messaging] improve full-sync fetching strategy

* fix

* rebase

* fix

* fix

* fix rebase

* fix

* fix

* fix

* fix

* fix

* remove deletion

* fix setPop with memory storage

* fix pgBoss and remove unnecessary job

* fix throw

* fix

* add timeout to ongoing sync
 2024-03-27 12:44:03 +01:00
Weiko a560746fcd Add worker run step to server cli (#4619) 
* Add worker run step to server cli

* add build step

* fix
 2024-03-25 11:00:11 +01:00
Weiko 8fb1ab8933 [backend] rename repository services and replace repository modules by dynamicModule (#4536) 
* rename database services to repository

* refactor more repositories

* more refactoring

* followup

* remove unused imports

* fix

* fix

* Fix calendar listener being called when flag is off

* remove folders
 2024-03-18 16:26:23 +01:00
WeikoandCharles Bochet 2c09096edd Refactor backend folder structure (#4505) 
* Refactor backend folder structure

Co-authored-by: Charles Bochet <charles@twenty.com>

* fix tests

* fix

* move yoga hooks

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
 2024-03-15 18:37:09 +01:00
Weiko 7555e7aad5 [messaging] Fix messaging formatAddress tests (#4482) 
* [messaging] Fix messaging formatAddress tests

* rebase

* remove unused test
 2024-03-15 14:58:02 +01:00
Weiko 991bb09622 [messaging] fix participant handles with trailing spaces (#4457) 2024-03-14 13:46:38 +01:00
Weiko 4476f5215b [messaging] Fix thread cleaner service subqueries (#4416) 
* [messaging] Fix thread cleaner service subqueries

* add pagination

* various fixes

* Fix thread merging

* fix

* fix
 2024-03-12 17:49:45 +01:00
Weiko 0d8e700239 [backend] use cache storage service for workspace schema cache (#4342) 
* [backend] add cache storage module

* update docs

* update default TTL to a week

* Use cacheStorageService for WorkspaceSchemaCache

* remove memory storage module

* revert pattern

* remove logs
 2024-03-12 13:51:39 +01:00
Weiko 250bb6134e [messaging] remove partial sync retry and fix missing datasource error (#4371) 
* [messaging] remove partial sync retry and fix missing datasource error

* revert

* fix

* add 429

* fix

* fix

* fix

* remove duplicate log

* fix cron pattern
 2024-03-08 14:06:21 +01:00
Weiko 41bed57be9 [backend] add cache storage module (#4320) 
* [backend] add cache storage module

* update docs

* update default TTL to a week
 2024-03-07 14:07:01 +01:00
Weiko e7733a1b7a Fix fetchAllMessages cronJob when deleted datasource (#4355) 2024-03-07 12:31:57 +01:00
Weiko 3c63584ef8 [messaging] add more details in exceptions (#4256) 
[messaging] add more logs in exceptions
 2024-03-04 14:05:01 +01:00
Weiko 8a669cc540 [messaging] add better logs to messaging sync jobs (#4245) 2024-02-29 17:30:42 +01:00
Weiko 47656479ba [messaging] fix empty history (#4218) 2024-02-28 14:32:05 +01:00
Weiko 214807588a [messaging] clean orphan threads and messages after connected account deletion (#4195) 
* [messaging] add connected account associated data delete

* add threadCleanerService

* fix

* fix import

* add thread cleaner import

* remove log
 2024-02-26 21:29:44 +01:00
Weiko d5e8844521 Fix referential_constraints health check (#4139) 2024-02-22 16:33:19 +01:00
Weiko 70511dc860 Add writeLog for relation update sync metadata (#4136) 
* Add writeLog for relation update sync metadata

* fix health check
 2024-02-22 16:02:13 +01:00
Weiko 8425ce4987 Add onDeleteAction to RelationMetadata (#4100) 
* Add onDeleteAction to relationMetadata

* rename to SET NULL

* fix migration

* fix migration

* fix after review
 2024-02-22 10:27:15 +01:00
Weiko 8c46e66cf5 Fix delete with no result should not throw (#4091) 
* Fix delete with no result should not throw

* add logs

* Delete packages/twenty-server/src/database/typeorm/metadata/migrations/1708442904165-addOnDeleteActionToRelationMetadata.ts
 2024-02-20 17:27:31 +01:00
Weiko 8e3723b88e Fix update on delete record (#3996) 2024-02-16 13:08:24 +01:00
Weiko 4b3eeac333 [messaging] add defaultValue to isContactAutoCreationEnabled (#3992) 2024-02-15 16:42:03 +01:00
Weiko 62058dd0e9 [worker] add more logs to queue-worker (#3982) 2024-02-14 21:28:17 +01:00
Weiko 49cc01d7d8 [messaging] fix add messageParticipant not in a transaction (#3981) 2024-02-14 19:28:25 +01:00
Weiko 2055f64acd fix CreateCompaniesAndContactsAfterSyncJobData import (#3979) 2024-02-14 18:58:49 +01:00
Weiko 0b2ffb0ee6 add rimraf to server dependencies (#3977) 2024-02-14 16:56:37 +01:00
Weiko 1afe8aecd0 Fix missing feature flag on messaging listeners (#3952) 
* Fix missing feature flag on messaging listeners

* Update workspace-query-runner.service.ts
 2024-02-13 20:23:09 +01:00
Weiko 87fafae9be [messaging] Removing TO requirement for email import (#3949) 2024-02-13 19:19:57 +01:00
Weiko 458e8c839f Add workspacePreQueryHook module (#3879) 
* rebase

* reorganise messaging folders

* fix

* fix after review

* fix yarn lock
 2024-02-13 18:23:29 +01:00
Weiko 36b69a8625 [messaging] Fix messaging import with no person skipped email (#3941) 2024-02-13 16:55:06 +01:00
Weiko 7f122a4671 [messaging] Remove ids from enqueued jobs (#3936) 
* [messaging] Fix import message participant uppercase

* fix job not enqueuing
 2024-02-13 16:13:34 +01:00
Weiko b6a86ebf96 [messaging] Fix import message participant uppercase (#3934) 2024-02-13 16:03:44 +01:00
Weiko c13e55a753 Fix bull-mq retry option and exceptions not being captured for jobs (#3905) 
* Fix bull-mq retry option

* fix exception inside worker

* add logs

* fix after review
 2024-02-12 15:12:52 +01:00
Weiko 99e2dd6899 [messaging] Add messageParticipant matching once people emails are updated (#3887) 
* poc nest event emitter

* add match message participant listener

* add workspacemember listener

* fix after review

* fix deep-equal
 2024-02-08 17:42:33 +01:00
Weiko 0096e60489 [messaging] add cronjob for workspaces messages partial sync (#3800) 
* [messaging] add cronjob for workspaces messages partial sync

* run cron every 10 minutes

* use logger
 2024-02-05 17:15:11 +01:00
Weiko 729e2dc651 [Messaging] Delete empty threads after message deletion import (#3716) 
* [Messaging] Delete empty threads after message deletion import

* fix
 2024-02-02 18:13:41 +01:00
Weiko ae5f82df59 [messaging] add fallback if lastHistoryId has been invalidated (#3782) 2024-02-02 15:28:38 +01:00
Weiko 142affbeea [messaging] reorder messages desc to asc (#3755) 2024-02-01 15:28:00 +01:00
Weiko e787b4e3b8 [messaging] add more logs to gmail full-sync (#3728) 2024-01-31 17:03:11 +01:00
Weiko 82d99afe2f Fix workspace-sync when alter with relation (#3721) 2024-01-31 15:50:54 +01:00
Weiko 03e5c792f0 [messaging] fix message-channel-message-association field name as dependencies (#3712) 2024-01-30 19:35:41 +01:00
Weiko 64b2ef3dc2 Delete message when no more association (#3701) 
* Delete message when no more association

* remove unused injections

* rename methods

* fix after review
 2024-01-30 17:58:36 +01:00
WeikoandCharles Bochet a9349f9fea Add deleteOneObject mutation (#3682) 
* Add deleteOneObject mutation

* codegen

* move relationToDelete to dedicated file

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
 2024-01-30 09:47:58 +01:00
Weiko d66d8c9907 [Messaging] Gmail Full sync pagination (#3664) 2024-01-29 11:57:54 +01:00
Weiko 9053769616 [Messaging] Fix gmail connected account creation redirect url (#3653) 2024-01-26 18:28:06 +01:00
Weiko 6d997edabb [Messaging] Fix duplicate messageChannelMessage (#3616) 
* [Messaging] Fix duplicate channelMessageChannel

* add messageChannelMessage check before querying gmail

* rename messageChannelMessage to messageChannelMessageAssociation
 2024-01-25 14:15:57 +01:00
Weiko c811206c47 Fix message table plural name (#3604) 2024-01-24 14:33:08 +01:00
Weiko c7ad6a0de7 [messaging] Rename body to text (#3595) 
* Store HTML message

* remove console log

* [messaging] rename body to text

* use CoreObjectNameSingular
 2024-01-23 19:56:42 +01:00
Weiko d176ba95d3 Store HTML message (#3594) 
* Store HTML message

* remove console log

* fix html label/description
 2024-01-23 19:40:05 +01:00
Weiko dc7fccb0a8 Merge messages and threads #1 (#3583) 
* Merge messages and threads

* rename messageChannelSync to messageChannelMessage

* add merge logic

* remove deprecated methods

* restore enqueue GmailFullSyncJob after connectedAccount creation
 2024-01-23 17:28:14 +01:00
Weiko 64110c591a Add performance logs to object metadata query (#3463) 2024-01-17 17:04:16 +01:00
Weiko f3c9854be3 Add select type to field metadata decorator (#3471) 
* Add select type to field metadata decorator

* add option id generation for new field
 2024-01-17 15:03:11 +01:00
Weiko ed6458e833 Fix: check if relation creates existing field name (#3433) 
* Fix: check if relation creates existing field name

* fix rebase

* add object name to performance log
 2024-01-15 14:13:57 +01:00
Weiko 0a77a376fd Import messages after connected-account creation (#3389) 2024-01-11 20:05:08 +01:00
Weiko 22047fa2bf Fix metadata exception handler #2 (#3357) 2024-01-10 17:02:54 +01:00
Weiko 6c00aa92a4 Fix capture exception for metadata and core (#3335) 2024-01-09 17:46:16 +01:00
Weiko ea2cb8938f Add fetch connected account job (#3313) 
* Add fetch connected account job

* add featureFlag check
 2024-01-08 18:24:39 +01:00
Weiko 4fddafceed Fix Event core object (#3232) 2024-01-04 13:40:34 +01:00
Weiko 7f66eb9459 Filter out system object for relation object destination (#3108) 
filter out system object for relation object destination
 2023-12-21 16:32:49 +01:00
Weiko e67f6873d3 Fix missing isNullable for object creation activity target (#3109) 2023-12-21 16:32:04 +01:00
Weiko 0d00e3d62d send pg graphql exception to sentry + fix missing nullable for relations (#3101) 
* Send pg_graphql errors to sentry

* Send pg_graphql errors to sentry

* fix

* fix

* fix

* fix relation nullable
 2023-12-21 16:07:25 +01:00
Weiko 4637a92f09 Fix queue setup (#3075) 
fix bullmq setup
 2023-12-19 17:12:22 +01:00
Weiko e799c84233 Add sync driver for queue messages (#3070) 
* Add sync driver for queue messages

* rename moduleRef

* use switch instead
 2023-12-19 13:30:40 +01:00
Weiko 36164ab59b Add pg-boss worker poc (#2991) 
* Add pg-boss worker poc

* add Example job

* add retry limit

* rename MessageQueue
 2023-12-14 18:57:25 +01:00
Weiko f126bd95d6 Add featureFlag gateDecorator for sync-metadata (#2956) 
* Add featureFlag gateDecorator for sync-metadata

* remove gate exampels

* gate messaging objects

* gate messaging recipient object

* add missing gate
 2023-12-12 17:34:59 +01:00
Weiko d2a42c14d2 Add field create and delete migration to metadata sync (#2942) 
add field create and delete migration to metadata sync
 2023-12-11 17:42:09 +01:00
Weiko a48c9293f6 Fix missing isNullable (#2892) 
* Fix missing isNullable

* fix
 2023-12-08 16:33:34 +01:00
Weiko b68f5cda97 Fix relation between standard objects (#2878) 2023-12-08 15:50:12 +01:00
Weiko 53b6005d73 Improve Metadata sync setup (#2874) 
* Improve Metadata sync setup

* add missing IsNullable()

* add composite fields types
 2023-12-08 12:27:47 +01:00
Weiko 5efc2f00b9 Sync metadata generate migrations (#2864) 
* Sync Metadata generates migrations

* add execute migrations

* fix relations + add isActive on creation

* fix composite fields migration

* remove dependency

* use new metadata setup for seed-dev

* fix rebase

* remove unused code

* fix viewField dev seeds

* fix isSystem
 2023-12-07 19:22:34 +01:00
Weiko 6d4ad6ec18 Sync standard object metadata (#2807) 
* Sync standard object metadata

* remove debug logging

* remove unused func

* fix comments

* fix empty objectsToDelete list
 2023-12-05 14:10:50 +01:00
Weiko 65aa91c774 Allow field/object update if name/label are similar values (#2709) 2023-11-24 21:25:48 +01:00
Weiko 323c69948c Fix check identical nameSingular/namePlural (#2689) 2023-11-24 11:30:22 +01:00
Weiko cc526517b3 Add viewField creation to fieldMetadata creation service (#2685) 
* Add viewField creation to fieldMetadata creation service

* fix reduce with no initial value
 2023-11-24 10:02:37 +01:00
Weiko c395955f12 Fix activity creation (#2686) 2023-11-24 10:01:29 +01:00
Weiko 851ce73609 Fix company update with accountOwner (#2687) 2023-11-24 10:00:45 +01:00
Weiko 5038c36df4 Update seed default values (#2681) 
* Update seed default values

* Update seed default values

* Update seed default values

* remove allowImpersonation in workspaceMembers

* remove USD from currencyCode defaultValue

* fix tests
 2023-11-23 23:30:53 +01:00
Weiko c795db33b2 Add activityTarget relation after custom object creation (#2670) 
* Add activityTarget relation after custom object creation

* add isCustom check for relations
 2023-11-23 16:26:33 +01:00
Weiko 0fd823af21 Allow isActive to be updated for standard objects (#2651) 
* Allow isActive to be updated for standard objects

* Allow isActive to be updated for standard objects/fields
 2023-11-22 17:17:13 +01:00
Weiko ccaa96bc58 Fix workspace/user deletion (#2648) 
* Fix workspace/user deletion

* remove logs

* add defaultWorkspace check
 2023-11-22 17:12:46 +01:00
Weiko a6abe09163 Move Impersonate from User to Workspace (#2630) 
* Fix impersonate

* align core typeorm config with metadata config + add allowImpersonation to workspace

* move allowImpersonation to workspace

* remove allowImpersonation from workspaceMember workspace table
 2023-11-22 14:12:39 +01:00
Weiko c74bde28b8 Add identifier fields to ObjectMetadata (#2616) 
* Add indentifier fields to ObjectMetadata

* Add indentifier fields to ObjectMetadata

* Add indentifier fields to ObjectMetadata

* temporarily block name/label edition
 2023-11-21 18:41:48 +01:00
Weiko 6f4a952381 Add [from/to]Description to CreateRelationInput (#2613) 2023-11-21 11:38:27 +01:00
Weiko a33d4c8b8d Rename DATE to DATETIME (#2576) 
* Rename DATE to DATETIME

* rename DATE to DATE_TIME

* fix server tests

* rename date to datetime

* rename date to datetime
 2023-11-21 00:16:42 +01:00
Weiko 1deb742ac9 Add deleteOneRelation resolver (#2569) 2023-11-17 19:13:42 +01:00
Weiko ed71ef67af Add Name defaultColumn for custom objects (#2568) 2023-11-17 19:13:07 +01:00
Weiko f62108d539 Add missing default values (#2567) 
* add missing default values

* add missing default values
 2023-11-17 19:11:25 +01:00
Weiko 4870b0ac30 Add fullName field metadata type (#2563) 
* Add fullName field metadata type

* update seeds
 2023-11-17 16:52:51 +01:00