[messaging] Add message deletion during partial sync (#4972)

## Context

- Rename remaining V2 services.
- Delete messages in DB when gmail history tells us they've been
deleted. I removed the logic where we store those in a cache since it's
a bit overkill because we don't need to query gmail and can use those
ids directly. The strategy is to delete the message channel message
association of the current channel, not the message or the thread since
they can still be linked to other channels. However, we will need to
call the threadCleaner service on the workspace to remove orphan
threads/non-associated messages.

Note: deletion for full-sync is a bit tricky because we need the full
list of message ids to compare with the DB and make sure we don't
over-delete. Currently, to keep memory, we don't have a variable that
holds all ids as we flush it after each page. Easier solution would be
to wipe everything before each full sync but it's probably not great for
the user experience if they are currently manipulating messages since
full-sync can happen without a user intervention (if a partial sync
fails due to historyId being invalidated by google for some reason)
This commit is contained in:
Weiko
2024-04-16 17:18:06 +02:00
committed by GitHub
parent 19a3be7b1b
commit 2efc794b43
8 changed files with 42 additions and 19 deletions

View File

@ -58,6 +58,11 @@ export class TimelineMessagingService {
${dataSourceSchema}."messageParticipant" "messageParticipant" ON "messageParticipant"."messageId" = message.id
WHERE
"messageParticipant"."personId" = ANY($1)
AND EXISTS (
SELECT 1
FROM ${dataSourceSchema}."messageChannelMessageAssociation" mcma
WHERE mcma."messageId" = message.id
)
GROUP BY
message."messageThreadId",
message.id
@ -128,6 +133,11 @@ export class TimelineMessagingService {
${dataSourceSchema}."message" message
WHERE
message."messageThreadId" = ANY($1)
AND EXISTS (
SELECT 1
FROM ${dataSourceSchema}."messageChannelMessageAssociation" mcma
WHERE mcma."messageId" = message.id
)
GROUP BY
message."messageThreadId"
`,
@ -248,6 +258,11 @@ export class TimelineMessagingService {
${dataSourceSchema}."messageParticipant" "messageParticipant" ON "messageParticipant"."messageId" = message.id
WHERE
"messageParticipant"."personId" = ANY($1)
AND EXISTS (
SELECT 1
FROM ${dataSourceSchema}."messageChannelMessageAssociation" mcma
WHERE mcma."messageId" = message.id
)
`,
[personIds],
workspaceId,

View File

@ -48,8 +48,8 @@ import { GmailFullSyncJob } from 'src/modules/messaging/jobs/gmail-full-sync.job
import { GmailPartialSyncJob } from 'src/modules/messaging/jobs/gmail-partial-sync.job';
import { MessagingCreateCompanyAndContactAfterSyncJob } from 'src/modules/messaging/jobs/messaging-create-company-and-contact-after-sync.job';
import { GmailFetchMessageContentFromCacheModule } from 'src/modules/messaging/services/gmail-fetch-message-content-from-cache/gmail-fetch-message-content-from-cache.module';
import { GmailFullSynV2Module } from 'src/modules/messaging/services/gmail-full-sync-v2/gmail-full-sync.v2.module';
import { GmailPartialSyncV2Module } from 'src/modules/messaging/services/gmail-partial-sync-v2/gmail-partial-sync-v2.module';
import { GmailFullSyncModule } from 'src/modules/messaging/services/gmail-full-sync/gmail-full-sync.module';
import { GmailPartialSyncModule } from 'src/modules/messaging/services/gmail-partial-sync/gmail-partial-sync.module';
import { MessageParticipantModule } from 'src/modules/messaging/services/message-participant/message-participant.module';
import { ThreadCleanerModule } from 'src/modules/messaging/services/thread-cleaner/thread-cleaner.module';
import { MessageChannelObjectMetadata } from 'src/modules/messaging/standard-objects/message-channel.object-metadata';
@ -81,9 +81,9 @@ import { MessageChannelObjectMetadata } from 'src/modules/messaging/standard-obj
MessageChannelObjectMetadata,
EventObjectMetadata,
]),
GmailFullSynV2Module,
GmailFullSyncModule,
GmailFetchMessageContentFromCacheModule,
GmailPartialSyncV2Module,
GmailPartialSyncModule,
CalendarEventParticipantModule,
],
providers: [