Commit Graph

38 Commits

Author SHA1 Message Date
458e8c839f Add workspacePreQueryHook module (#3879)
* rebase

* reorganise messaging folders

* fix

* fix after review

* fix yarn lock
2024-02-13 18:23:29 +01:00
36b69a8625 [messaging] Fix messaging import with no person skipped email (#3941) 2024-02-13 16:55:06 +01:00
7f122a4671 [messaging] Remove ids from enqueued jobs (#3936)
* [messaging] Fix import message participant uppercase

* fix job not enqueuing
2024-02-13 16:13:34 +01:00
b6a86ebf96 [messaging] Fix import message participant uppercase (#3934) 2024-02-13 16:03:44 +01:00
ec48e66eeb 3921 delete messagethreads after deleting connectedaccount (#3925)
* created listener

* working

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-02-13 14:36:55 +01:00
1d1976ef22 3807 auto creation of contact when importing emails (#3888)
* Add CreateContactService to messaging services

* Add logic to create a contact if it doesn't exist

* Add name

* Improvements

* contact creation working

* fix bug

* Add IsPersonEmailService to check if an email is personal or not

* filter is working

* improve filter

* create companies and people

* Refactor createContactFromHandleAndDisplayName to createContactAndCompanyFromHandleAndDisplayName

* improve regex

* reorganizing services

* updates

* reorganize folders

* wip

* use transaction

* wip

* wip

* wip

* batch queries

* almost working

* working
2024-02-13 14:24:28 +01:00
c13e55a753 Fix bull-mq retry option and exceptions not being captured for jobs (#3905)
* Fix bull-mq retry option

* fix exception inside worker

* add logs

* fix after review
2024-02-12 15:12:52 +01:00
11d1c4c161 3808 auto creation of company when importing emails (#3881)
* create service

* wip

* use raw queries

* creating companies is working

* Fix participant handle domain name extraction

* Add HTTP service to fetch company info from domain name

* Handle 404

* Fix missing parameter in INSERT query

* wip

* renaming

* Add typing
2024-02-09 15:28:35 +01:00
99e2dd6899 [messaging] Add messageParticipant matching once people emails are updated (#3887)
* poc nest event emitter

* add match message participant listener

* add workspacemember listener

* fix after review

* fix deep-equal
2024-02-08 17:42:33 +01:00
7001ca83d1 3491 launch cleaning cron (#3872)
* Add command to delete incomplete workspaces

* Inject command dependencies

* Fix command

* Do not delete core.workspace

* Reorganize files

* Delete src/workspace/cron

* Fix

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-02-07 18:52:48 +01:00
0096e60489 [messaging] add cronjob for workspaces messages partial sync (#3800)
* [messaging] add cronjob for workspaces messages partial sync

* run cron every 10 minutes

* use logger
2024-02-05 17:15:11 +01:00
729e2dc651 [Messaging] Delete empty threads after message deletion import (#3716)
* [Messaging] Delete empty threads after message deletion import

* fix
2024-02-02 18:13:41 +01:00
ae5f82df59 [messaging] add fallback if lastHistoryId has been invalidated (#3782) 2024-02-02 15:28:38 +01:00
142affbeea [messaging] reorder messages desc to asc (#3755) 2024-02-01 15:28:00 +01:00
e787b4e3b8 [messaging] add more logs to gmail full-sync (#3728) 2024-01-31 17:03:11 +01:00
64b2ef3dc2 Delete message when no more association (#3701)
* Delete message when no more association

* remove unused injections

* rename methods

* fix after review
2024-01-30 17:58:36 +01:00
d66d8c9907 [Messaging] Gmail Full sync pagination (#3664) 2024-01-29 11:57:54 +01:00
b0c14ba5b9 3571 remove reply quotations from emails (#3630)
* lib is working

* update

* plural
2024-01-25 18:26:29 +01:00
6d997edabb [Messaging] Fix duplicate messageChannelMessage (#3616)
* [Messaging] Fix duplicate channelMessageChannel

* add messageChannelMessage check before querying gmail

* rename messageChannelMessage to messageChannelMessageAssociation
2024-01-25 14:15:57 +01:00
c7ad6a0de7 [messaging] Rename body to text (#3595)
* Store HTML message

* remove console log

* [messaging] rename body to text

* use CoreObjectNameSingular
2024-01-23 19:56:42 +01:00
d176ba95d3 Store HTML message (#3594)
* Store HTML message

* remove console log

* fix html label/description
2024-01-23 19:40:05 +01:00
dc7fccb0a8 Merge messages and threads #1 (#3583)
* Merge messages and threads

* rename messageChannelSync to messageChannelMessage

* add merge logic

* remove deprecated methods

* restore enqueue GmailFullSyncJob after connectedAccount creation
2024-01-23 17:28:14 +01:00
4695e99458 3369 rename messagerecipients table into messageparticipants (#3457)
* renaming

* renaming
2024-01-15 16:55:19 +01:00
8682f3c0c0 3441 modify message table change date to receivedat (#3452)
* changed date to receivedAt

* update saving messages

* update custom resolver
2024-01-15 14:48:25 +01:00
a8efc17fff Fix post merge conflict 2024-01-13 12:06:37 +01:00
49a9a2c2be 2252 build a script to cleanup inactive workspaces (#3307)
* Add cron to message queue interfaces

* Add command to launch cron job

* Add command to stop cron job

* Update clean inactive workspaces job

* Add react-email

* WIP

* Fix import error

* Rename services

* Update logging

* Update email template

* Update email template

* Add Base Email template

* Move to proper place

* Remove test files

* Update logo

* Add email theme

* Revert "Remove test files"

This reverts commit fe062dd05166b95125cf99f2165cc20efb6c275a.

* Add email theme 2

* Revert "Revert "Remove test files""

This reverts commit 6c6471273ad765788f2eaf5a5614209edfb965ce.

* Revert "Revert "Revert "Remove test files"""

This reverts commit f851333c24e9cfe3f425c9cbbd1e079efce5c3dd.

* Revert "Revert "Revert "Revert "Remove test files""""

This reverts commit 7838e19e88e269026e24803f26cd52b467b4ef36.

* Fix theme

* Reorganize files

* Update clean inactive workspaces job

* Use env variable to define inactive days

* Remove FROM variable

* Use feature flag

* Fix cron command

* Remove useless variable

* Reorganize files

* Refactor some code

* Update email template

* Update email object

* Remove verbose log

* Code review returns

* Code review returns

* Simplify handle

* Code review returns

* Review

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-01-13 12:03:41 +01:00
5a61e34f4c 3239 create a command to do a partial sync with the gmail api using the historyid (#3405)
* create utils service

* getLastSyncHistoryId

* getHistory

* add historyTypes messageAdded and messageDeleted

* getMessageIdsAndThreadIdsNotInDatabase

* wip

* fix messageThreadId null

* no need to fetch threads anymore

* get messagesAdded in partial sync

* adding errors

* save lastSyncHistoryId

* improve

* renaming

* create partial sync job

* improve partial sync

* adding messages with partial sync is working

* now adding messages with partial sync is working

* deleting messages and empty threads is working

* wip

* wip

* fix bug to delete threads

* update partial sync to cover edge cases

* renaming

* modify ambiguous naming

* renaming
2024-01-12 17:46:55 +01:00
bdd0a7ed95 3242 all message recipients should be stored (#3320)
* saveMessageRecipients

* update

* workspaceMemberId is working

* merge

* get direction of the message

* fix

* improve code

* modify GmailMessage type
2024-01-09 14:14:32 +01:00
ea2cb8938f Add fetch connected account job (#3313)
* Add fetch connected account job

* add featureFlag check
2024-01-08 18:24:39 +01:00
54c1d245ab 3218 make the function fetchworkspacememberthreads idempotent (#3230)
* wip

* fetch only the messages which are not in the db

* fetch only the messages and threads which are not in the db

* fix bugs

* merge

* remove eslint-plugins-twenty

* get saved message thread ids and message ids at the same time
2024-01-04 13:36:37 +01:00
8483cf0b4b POC: chore: use Nx workspace lint rules (#3163)
* chore: use Nx workspace lint rules

Closes #3162

* Fix lint

* Fix lint on BE

* Fix tests

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-01-03 23:07:25 +01:00
4ebaacc306 3216 request a new access token for the gmail api when it expires (#3224)
* refresh access token

* refresh and save access token

* update module

* refreshing access token before fetching the emails

* remove log
2024-01-03 18:00:31 +01:00
67fca68480 3202 fetch emails by threads (#3214)
* change fetchAllByBatches and fetchBatch to allow messages and threads to be fetched by batches

* wip

* format threads batches

* command is working

* command is working

* fix typing

* updates
2024-01-03 15:01:22 +01:00
526a3d7d9a 3019 timebox add typing and checks in workspacemessagingservices (#3112)
* throw error

* fetchWorkspaceMessages fetches messages and threads

* renaming

* improve typing

* improve typing and error handling

* improve typing and error handling

* improve typing and error handling

* improve fetch-batch

* fix bug

* replace return types

* imporving typing and error handling

* improve typing and error handling

* improve typing and error handling

* improve typing and error handling

* improve typing and error handling

* remove console log
2023-12-26 18:07:40 +01:00
5afcab4e78 3011 fill the messagerecipient table when fetching messages (#3073)
* wip

* trying to parse display names and emails

* add nodemailer mailparser

* mail parsing is working

* add personId and workspaceMemberId

* add date to messages

* Fix PR

* Run tsc on bigger machine

* Fix lint

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2023-12-19 17:08:54 +01:00
f95c56b1cb 2880 timebox create a poc to fetch emails from the gmail api (#2993)
* create empty service

* getting threads is working

* insert message channel

* save threads in the db

* clean

* fetch messages

* create a service to fetch a batch of messages

* batch messages

* use httpService instead

* parse batch

* base 64 decoding working

* solve parsing bug

* saving messages is working

* bug to fix in fetchAllByBatches

* fetching all messages is working but not saving yet

* fecth 500 messages and threads is working

* remove unused package and console log

* set direction to incoming

* fix bug after merging main
2023-12-15 16:35:56 +01:00
36164ab59b Add pg-boss worker poc (#2991)
* Add pg-boss worker poc

* add Example job

* add retry limit

* rename MessageQueue
2023-12-14 18:57:25 +01:00
6594055317 Create empty command (#2963)
* create empty command

* update description

* rebase

---------

Co-authored-by: corentin <corentin@twenty.com>
2023-12-12 18:22:19 +01:00