## Context
- Rename remaining V2 services.
- Delete messages in DB when gmail history tells us they've been
deleted. I removed the logic where we store those in a cache since it's
a bit overkill because we don't need to query gmail and can use those
ids directly. The strategy is to delete the message channel message
association of the current channel, not the message or the thread since
they can still be linked to other channels. However, we will need to
call the threadCleaner service on the workspace to remove orphan
threads/non-associated messages.
Note: deletion for full-sync is a bit tricky because we need the full
list of message ids to compare with the DB and make sure we don't
over-delete. Currently, to keep memory, we don't have a variable that
holds all ids as we flush it after each page. Easier solution would be
to wipe everything before each full sync but it's probably not great for
the user experience if they are currently manipulating messages since
full-sync can happen without a user intervention (if a partial sync
fails due to historyId being invalidated by google for some reason)
## Context
The full-sync job was enqueued within a transaction, which means it
could be executed before the transaction was commit and
connected-account was not created yet.
This PR re-arrange the code a bit to avoid this
cc @bosiraphael thx for flagging this!
## Context
Recent PR introduced a verifyTransientToken inside the
GoogleAPIsProviderEnabledGuard guard. This is used to extract the
workspaceId from the token. This is working fine for the first call sent
to google however the callback is calling the same guard which is
causing an issue because the transientToken is missing from the
callback.
Imho, the same guard shouldn't be used by the callback but for the time
being I'm adding a check to prevent using feature flag when
transientToken is absent. In fact, it is present in the request but not
in the same key. Because the scope is only relevant for the first call,
I'm simply adding a check there.
## Context
Currently the calendar scope is bound to an env variable. We want to
rollout this feature to some users so this PR adds a check on the
existing IS_CALENDAR_ENABLED flag
## Context
With the addition of cronjobs, the app is building a lot of jobs and
stores them indefinitely. There is no real point to keep all of them in
the queue once they have been processed (completed or failed) so we are
adding a new default option to the bull-mq driver
## Implementation
See bull-mq JobsOption doc
```typescript
/**
* If true, removes the job when it successfully completes
* When given a number, it specifies the maximum amount of
* jobs to keep, or you can provide an object specifying max
* age and/or count to keep. It overrides whatever setting is used in the worker.
* Default behavior is to keep the job in the completed set.
*/
removeOnComplete?: boolean | number | KeepJobs;
/**
* If true, removes the job when it fails after all attempts.
* When given a number, it specifies the maximum amount of
* jobs to keep, or you can provide an object specifying max
* age and/or count to keep. It overrides whatever setting is used in the worker.
* Default behavior is to keep the job in the failed set.
*/
removeOnFail?: boolean | number | KeepJobs;
```
removeOnFail should be a bit higher since they are the ones we are most
likely looking at when needed.
This PR introduces a new folder structure for business modules.
Cron commands and jobs are now stored within the same module/folder at
the root of the business module
e.g: /modules/messaging/crons/commands instead of
/modules/messaging/commands/crons
Patterns are now inside their own cron-command files since they don't
need to be exported
Ideally cronJobs and cronCommands should have their logic within the
same class but it's a bit harder than expected due to how commanderjs
and our worker need both some class heritage check, hence the first
approach is to move them in the same folder
Also Messaging fullsync/partialsync V2 has been dropped since this is
the only used version => Breaking change for ongoing jobs and crons.
Jobs can be dropped but we will need to re-run our crons (only
cron:messaging:gmail-fetch-messages-from-cache)
- AuthFailedAt is set when a refreshToken is not valid and an
accessToken can't be generated, meaning it will need a manual action
from the user to provide a new refresh token.
- Calendar/messaging jobs should not be executed if authFailedAt is not
null.
## Context
Calendar scope was too broad, this PR updates it to events only.
Also changing "Cannot connect Google account to demo workspace" error to
a 404 to avoid having a 500 for something expected
## Context
We are now removing Messaging V2 feature flag to use it everywhere.
## Implementation
- renaming FetchWorkspaceMessagesCommandsModule to
MessagingCommandModule to make it more generic since it it hosts all
commands related to the messaging module
- creating a crons folder inside commands and jobs crons should be named
with xxx.cron.command.ts instead of xxx.command.ts. Same for jobs, jobs
should be named with xxx.cron.job.ts. In a future PR we should make sure
those CronJobs implement a CronJob interface since it's a bit different
(a CronJob does not contain a payload compared to a Job)
- Cron commands have been renamed to "cron:$module:command" so
`fetch-all-workspaces-messages-from-cache:cron:start` has been renamed
to `cron:messaging:gmail-fetch-messages-from-cache`. Also having to
create a command to stop the cron is a bit painful to maintain so I
removed them for now, this can be easily done manually with pg-boss or
bull-mq
- Removing full-sync and partial-sync commands as they were there for
testing only, we might put them back at some point but we will have to
adapt the code anyway.
- Feature flag has been removed from the MessageChannel standard object
to make sure those new columns are created during the next sync-metadata
## Context
Calendar tables are behind a featureFlag, they do not exist if the
feature flag is off which means we should not use them for the same
reason. I'm adding a check on the featureFlag before calling the
repository.
```
Error executing raw query for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419: relation "workspace_1wgvd1injqtife6y4rvfbu3h5.calendarEventParticipant" does not exist
```
## Test
locally with and without featureflag
## Context
When running a command, the process should end normally however it stays
hanging due to the open connection with redis client (when
CACHE_STORAGE_TYPE=redis)
This PR adds the necessary logic to gracefully close the connection once
the module is destroyed. Thanks to that, the command process now
properly ends once executed.
## Context
SyncExternalId should be renamed because this won't always represent an
id. For example, microsoft API does not use ids but dates for their
sync. Also we think external is a bit redundant so we are removing it.
Note: this field is not used by the front-end (and will probably never
be)
## Context
A new ADDRESS field type has been introduced and the company object has
been updated to use this new type however this introduced a few
regressions.
The good strategy would be to introduce a new field and rename the old
one.
This PR revert that change to fix the issue.
* rename database services to repository
* refactor more repositories
* more refactoring
* followup
* remove unused imports
* fix
* fix
* Fix calendar listener being called when flag is off
* remove folders
* Fix delete with no result should not throw
* add logs
* Delete packages/twenty-server/src/database/typeorm/metadata/migrations/1708442904165-addOnDeleteActionToRelationMetadata.ts
Weiko