4476f5215b
[messaging] Fix thread cleaner service subqueries ( #4416 )
...
* [messaging] Fix thread cleaner service subqueries
* add pagination
* various fixes
* Fix thread merging
* fix
* fix
2024-03-12 17:49:45 +01:00
250bb6134e
[messaging] remove partial sync retry and fix missing datasource error ( #4371 )
...
* [messaging] remove partial sync retry and fix missing datasource error
* revert
* fix
* add 429
* fix
* fix
* fix
* remove duplicate log
* fix cron pattern
2024-03-08 14:06:21 +01:00
735e75b3b1
Fix domain name parsing on company creation ( #4297 )
...
* add domain parsing library
* change package for psl
* trying to fix error
* fix
* update
* remove unused function
2024-03-04 17:50:41 +01:00
4242b546b6
Handle error 429 during partial sync ( #4258 )
...
handle error 429
2024-03-01 14:41:42 +01:00
72ae6e44b3
Remove Gate decorator IS_BLOCKLIST_ENABLED from BlocklistObjectMetadata ( #4250 )
...
* Remove Gate decorator from BlocklistObjectMetadata
* check if blocklist is enabled
* wip
* done
2024-02-29 19:25:11 +01:00
773f698faf
Add error handling in GmailPartialSyncService ( #4248 )
...
* Add error handling in GmailPartialSyncService
* improve typing
2024-02-29 18:11:53 +01:00
8a669cc540
[messaging] add better logs to messaging sync jobs ( #4245 )
2024-02-29 17:30:42 +01:00
bc11cf80fa
4209 speed up gmail full sync by using search params to query only the relevant emails ( #4213 )
...
* create blocklist service
* blocklist is working on email import in full sync
* add log
* add blocklist to partial sync
* define rule for blocklist imports
* gmail filter is working
* correct typo
* fix bugs
* getCompanyNameFromDomainName
* renaming
* remove unused service
* add transaction
2024-02-29 12:26:58 +01:00
8c08f1b603
Remove html from messages ( #4229 )
...
* remove html
* remove html
* remove html from db
2024-02-29 12:01:01 +01:00
fcfc6796f7
Add pagination to partial sync and add logs ( #4223 )
...
* update gmail partial sync to add pagination
* adding logs
* update
* improve readability
2024-02-28 14:55:54 +01:00
47656479ba
[messaging] fix empty history ( #4218 )
2024-02-28 14:32:05 +01:00
a19de71fad
4017 improve queries on messages write ( #4207 )
...
* modify code to reduce nested loops and improve performances
* is working
* fix lastSyncHistoryId
* create new service to share it betweent partial sync and full sync
* update partial sync
* update batch limit
* renaming
* adding logs
* update logs
* update logs
* update logs
* delete messages if error while saving the participants
* refactoring
* improving logs
* update logs
* delete historyId if outdated
2024-02-27 16:06:19 +01:00
214807588a
[messaging] clean orphan threads and messages after connected account deletion ( #4195 )
...
* [messaging] add connected account associated data delete
* add threadCleanerService
* fix
* fix import
* add thread cleaner import
* remove log
2024-02-26 21:29:44 +01:00
4e798ba2a3
3933 filter non work email from auto contact creation ( #4131 )
...
* use isWorkEmail
* working
* improvement
* Refactor lodash import in create-companies-and-contacts.service.ts
* refactor lodash import
2024-02-22 15:25:14 +01:00
ee7c1fbf5c
4008 dont create a contact company if it matches the persons domain ( #4088 )
...
* Add SettingsAccountsEmailsBlocklistInput story
* prevent contact creation from the same company
* add todo
* improvements
* Delete packages/twenty-front/src/modules/settings/accounts/components/__stories__/SettingsAccountsEmailsBlocklistInput.stories.tsx
* refactor
* modify after review
* improve code
* create utils
* fix
* Fix getAllByWorkspaceId to throw NotFoundException when no workspace member found
* fix after merge
* use map
* modify after review
2024-02-21 13:22:01 +01:00
09783912f3
4008 dont create a contact company if it matches the persons domain ( #4057 )
...
* Add SettingsAccountsEmailsBlocklistInput story
* prevent contact creation from the same company
* add todo
* improvements
* Delete packages/twenty-front/src/modules/settings/accounts/components/__stories__/SettingsAccountsEmailsBlocklistInput.stories.tsx
* refactor
* modify after review
* improve code
2024-02-19 18:46:49 +01:00
e34e341ddc
3938 change email auto import to contact ive sent email to ( #3995 )
...
* done
* working
* wip
* merge main
* almost done
* improvement
2024-02-19 17:29:38 +01:00
94ad0e33ec
3889 activate settingsaccountsemailsinboxsettings ( #3962 )
...
* update email visibility in settings
* improve styling
* Add contact auto creation toggle to inbox settings
* re
move soonpill
* update Icon
* create job
* Add logic to create contacts and companies for message participants without personId and workspaceMemberId
* add listener
* wip
* wip
* refactoring
* improve structure
* Add isContactAutoCreationEnabled method to MessageChannelService
* wip
* wip
* clean
* add job
* fix bug
* contact creation is working
* wip
* working
* improve code
* improve typing
* resolve conflicts
* fix
* create company repository
* move util
* wip
* fix
2024-02-14 17:30:17 +01:00
87fafae9be
[messaging] Removing TO requirement for email import ( #3949 )
2024-02-13 19:19:57 +01:00
458e8c839f
Add workspacePreQueryHook module ( #3879 )
...
* rebase
* reorganise messaging folders
* fix
* fix after review
* fix yarn lock
2024-02-13 18:23:29 +01:00
36b69a8625
[messaging] Fix messaging import with no person skipped email ( #3941 )
2024-02-13 16:55:06 +01:00
7f122a4671
[messaging] Remove ids from enqueued jobs ( #3936 )
...
* [messaging] Fix import message participant uppercase
* fix job not enqueuing
2024-02-13 16:13:34 +01:00
b6a86ebf96
[messaging] Fix import message participant uppercase ( #3934 )
2024-02-13 16:03:44 +01:00
1d1976ef22
3807 auto creation of contact when importing emails ( #3888 )
...
* Add CreateContactService to messaging services
* Add logic to create a contact if it doesn't exist
* Add name
* Improvements
* contact creation working
* fix bug
* Add IsPersonEmailService to check if an email is personal or not
* filter is working
* improve filter
* create companies and people
* Refactor createContactFromHandleAndDisplayName to createContactAndCompanyFromHandleAndDisplayName
* improve regex
* reorganizing services
* updates
* reorganize folders
* wip
* use transaction
* wip
* wip
* wip
* batch queries
* almost working
* working
2024-02-13 14:24:28 +01:00
c13e55a753
Fix bull-mq retry option and exceptions not being captured for jobs ( #3905 )
...
* Fix bull-mq retry option
* fix exception inside worker
* add logs
* fix after review
2024-02-12 15:12:52 +01:00
11d1c4c161
3808 auto creation of company when importing emails ( #3881 )
...
* create service
* wip
* use raw queries
* creating companies is working
* Fix participant handle domain name extraction
* Add HTTP service to fetch company info from domain name
* Handle 404
* Fix missing parameter in INSERT query
* wip
* renaming
* Add typing
2024-02-09 15:28:35 +01:00
99e2dd6899
[messaging] Add messageParticipant matching once people emails are updated ( #3887 )
...
* poc nest event emitter
* add match message participant listener
* add workspacemember listener
* fix after review
* fix deep-equal
2024-02-08 17:42:33 +01:00
0096e60489
[messaging] add cronjob for workspaces messages partial sync ( #3800 )
...
* [messaging] add cronjob for workspaces messages partial sync
* run cron every 10 minutes
* use logger
2024-02-05 17:15:11 +01:00
729e2dc651
[Messaging] Delete empty threads after message deletion import ( #3716 )
...
* [Messaging] Delete empty threads after message deletion import
* fix
2024-02-02 18:13:41 +01:00
ae5f82df59
[messaging] add fallback if lastHistoryId has been invalidated ( #3782 )
2024-02-02 15:28:38 +01:00
142affbeea
[messaging] reorder messages desc to asc ( #3755 )
2024-02-01 15:28:00 +01:00
e787b4e3b8
[messaging] add more logs to gmail full-sync ( #3728 )
2024-01-31 17:03:11 +01:00
64b2ef3dc2
Delete message when no more association ( #3701 )
...
* Delete message when no more association
* remove unused injections
* rename methods
* fix after review
2024-01-30 17:58:36 +01:00
d66d8c9907
[Messaging] Gmail Full sync pagination ( #3664 )
2024-01-29 11:57:54 +01:00
b0c14ba5b9
3571 remove reply quotations from emails ( #3630 )
...
* lib is working
* update
* plural
2024-01-25 18:26:29 +01:00
6d997edabb
[Messaging] Fix duplicate messageChannelMessage ( #3616 )
...
* [Messaging] Fix duplicate channelMessageChannel
* add messageChannelMessage check before querying gmail
* rename messageChannelMessage to messageChannelMessageAssociation
2024-01-25 14:15:57 +01:00
c7ad6a0de7
[messaging] Rename body to text ( #3595 )
...
* Store HTML message
* remove console log
* [messaging] rename body to text
* use CoreObjectNameSingular
2024-01-23 19:56:42 +01:00
d176ba95d3
Store HTML message ( #3594 )
...
* Store HTML message
* remove console log
* fix html label/description
2024-01-23 19:40:05 +01:00
dc7fccb0a8
Merge messages and threads #1 ( #3583 )
...
* Merge messages and threads
* rename messageChannelSync to messageChannelMessage
* add merge logic
* remove deprecated methods
* restore enqueue GmailFullSyncJob after connectedAccount creation
2024-01-23 17:28:14 +01:00
4695e99458
3369 rename messagerecipients table into messageparticipants ( #3457 )
...
* renaming
* renaming
2024-01-15 16:55:19 +01:00
8682f3c0c0
3441 modify message table change date to receivedat ( #3452 )
...
* changed date to receivedAt
* update saving messages
* update custom resolver
2024-01-15 14:48:25 +01:00
5a61e34f4c
3239 create a command to do a partial sync with the gmail api using the historyid ( #3405 )
...
* create utils service
* getLastSyncHistoryId
* getHistory
* add historyTypes messageAdded and messageDeleted
* getMessageIdsAndThreadIdsNotInDatabase
* wip
* fix messageThreadId null
* no need to fetch threads anymore
* get messagesAdded in partial sync
* adding errors
* save lastSyncHistoryId
* improve
* renaming
* create partial sync job
* improve partial sync
* adding messages with partial sync is working
* now adding messages with partial sync is working
* deleting messages and empty threads is working
* wip
* wip
* fix bug to delete threads
* update partial sync to cover edge cases
* renaming
* modify ambiguous naming
* renaming
2024-01-12 17:46:55 +01:00
bdd0a7ed95
3242 all message recipients should be stored ( #3320 )
...
* saveMessageRecipients
* update
* workspaceMemberId is working
* merge
* get direction of the message
* fix
* improve code
* modify GmailMessage type
2024-01-09 14:14:32 +01:00
ea2cb8938f
Add fetch connected account job ( #3313 )
...
* Add fetch connected account job
* add featureFlag check
2024-01-08 18:24:39 +01:00
54c1d245ab
3218 make the function fetchworkspacememberthreads idempotent ( #3230 )
...
* wip
* fetch only the messages which are not in the db
* fetch only the messages and threads which are not in the db
* fix bugs
* merge
* remove eslint-plugins-twenty
* get saved message thread ids and message ids at the same time
2024-01-04 13:36:37 +01:00
8483cf0b4b
POC: chore: use Nx workspace lint rules ( #3163 )
...
* chore: use Nx workspace lint rules
Closes #3162
* Fix lint
* Fix lint on BE
* Fix tests
---------
Co-authored-by: Charles Bochet <charles@twenty.com >
2024-01-03 23:07:25 +01:00
4ebaacc306
3216 request a new access token for the gmail api when it expires ( #3224 )
...
* refresh access token
* refresh and save access token
* update module
* refreshing access token before fetching the emails
* remove log
2024-01-03 18:00:31 +01:00
67fca68480
3202 fetch emails by threads ( #3214 )
...
* change fetchAllByBatches and fetchBatch to allow messages and threads to be fetched by batches
* wip
* format threads batches
* command is working
* command is working
* fix typing
* updates
2024-01-03 15:01:22 +01:00
526a3d7d9a
3019 timebox add typing and checks in workspacemessagingservices ( #3112 )
...
* throw error
* fetchWorkspaceMessages fetches messages and threads
* renaming
* improve typing
* improve typing and error handling
* improve typing and error handling
* improve typing and error handling
* improve fetch-batch
* fix bug
* replace return types
* imporving typing and error handling
* improve typing and error handling
* improve typing and error handling
* improve typing and error handling
* improve typing and error handling
* remove console log
2023-12-26 18:07:40 +01:00
5afcab4e78
3011 fill the messagerecipient table when fetching messages ( #3073 )
...
* wip
* trying to parse display names and emails
* add nodemailer mailparser
* mail parsing is working
* add personId and workspaceMemberId
* add date to messages
* Fix PR
* Run tsc on bigger machine
* Fix lint
---------
Co-authored-by: Charles Bochet <charles@twenty.com >
2023-12-19 17:08:54 +01:00
