In this PR:
- remove deprecated EMAIL, PHONE, LINK field types (except for Zapier
package as there is another work ongoing)
- remove composite currency filter on currencyCode, actor filter on name
and workspaceMember as the UX is not great yet
Fix syncedAt no longer been set on message sync.
Fix calendar data model:
- Add `syncedAt` to `CalendarChannelWorkspaceEntity`
- Move `recurringEventExternalId` from `CalendarEventWorkspaceEntity` to
`CalendarChannelEventAssociationWorkspaceEntity` since the id is
relative to one channel
Fix save queries on calendar sync after regression.
## Context
Name shouldn't be added to all tables, especially standard objects
because they already have their own labelIdentifierFieldMetadata
specified in the workspace-entity schema. This PR removes this column
from the "base" list of columns to add when creating a new object/table
and moves it to the object-metadata service that is, as of today, only
used for custom objects. Also had to modify the migration-runner to
handle column creation in a table creation migration (this was available
in the migration definition already but was not doing anything)
This also fixes an issue in standard objects that already have a "name"
field defined with a different field type, this is even more important
when the said field is a composite field. For example people already has
a FULL_NAME name field which clashes with the default TEXT name field
meaning it was only creating 1 field metadata for 'name' but 3 columns
were created: `name, nameFirstName, nameLastName`. This inconsistency
with metadata (which is our source of truth everywhere) brought some
issues (lately, converting back typeorm response to gql (including
composition) was broken).
- Add message deletion and thread cleaning during full message list
fetch
- Add thread cleaning during partial message list fetch
- Delete provider from cache key
In this PR:
- removing ugprade-0.24 commands as we are releasing 0.30
- introducing cache:flush command
- refactoring upgrade command and sync-metadata command to use the
ActiveWorkspacesCommand so they consistently run on all workspaces or
selected workspaces
Fixes:
- clear localStorage on sign out
- fix missing workspaceMember in verify resolver
- do not throw on datasource already destroyed exception which can
happen with race condition when several resolvers are resolving in
parallel
This is the second PR on TWNTY-6261 which handlesdata migration of Email
field to Emails field.\
\
How to Test?\
Firstly make sure that you have completed the testing steps on first PR
then follow the below steps:
- Checkout to TWNTY-6261-emails-migrations branch
- Rebuild typescript using "npx nx build twenty-server"
- Run command "yarn command:prod upgrade-0.25" to do migration\
\
Loom Video:\
<https://www.loom.com/share/f82b8d29f8f64f92abe3c59c01147b45?sid=9f8ccc05-aa38-4c49-b139-fd0823066273>
**Testing Messaging Sync functionality:**
Please watch the below video to see that the synchronization of contacts
is working fine after migrating Email field to Emails field:\
<https://www.loom.com/share/400949464b244272b78c25e338cc6ab2?sid=103f6625-5933-4b99-9825-0fed33782f36>
**Question to the client**
should we rename email to emails here? in the DomainName PR, the name
did not change.
```typescript
@WorkspaceField({
standardId: PERSON_STANDARD_FIELD_IDS.email,
type: FieldMetadataType.EMAILS,
label: 'Email',
description: 'Contact’s Email',
icon: 'IconMail',
})
email: EmailsMetadata;
```
**Test Messaging Sync**
This pr will update messaging sync files so the changes shouldn't break
existing functionality of importing people and companies in the app.\
To test messaging sync you should follow the below steps:\
1. you need to connect a google account to see the importing
functionality. For this purpose you
have to create a project inside Google Cloud. But to make things easier
you can use the below credentials of an already created project. Put
them in .env of twenty-server package:
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=951231465939-h61tg6nkpkv1821qi899fjbj9looquto.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
Alternative env
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=622006708006-dc4n3vrtf3cs2h6k7hgbborudme7ku9l.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
1. Launch your worker with `npx nx run twenty-server:worker`
2. npx nx run twenty-server:command cron:messaging:messages-import
3. npx nx run twenty-server:command cron:messaging:message-list-fetch
4. npx nx run twenty-server:command
cron📆calendar-event-list-fetch
5. Run the app and navigate to Settings/Accounts then connect your
Google account
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
Closes#6657
- Fix listeners
- Refactor jobs to take array of events
- Fix calendar events and messages deletion
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
The code removed in the PR was flagged as unused by the JetBrains
inspector.
I did a QA on the dev environment but other checks are highly
recommended.
There is one commit by scope to make the review easier.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Context
The goal is to replace pg_graphql with our own ORM wrapper (TwentyORM).
This PR tries to add some parsing logic to convert graphql requests to
send to the ORM to replace pg_graphql implementation.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Context
As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!
I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
Closes#6382
Create SetUserVarsAccountsToReconnectCommand.
This command loops on all workspaces and:
- deletes all user vars with deprecated key `ACCOUNTS_TO_RECONNECT`
- creates a key value pair of type `USER_VAR` with a key of
`ACCOUNTS_TO_RECONNECT_INSUFFICIENT_PERMISSIONS` for all connect
accounts with a message channel or calendar channel with status
`FAILED_INSUFFICIENT_PERMISSIONS`
This pull request introduces a new `FieldMetadataType` called `ACTOR`.
The primary objective of this new type is to add an extra column to the
following objects: `person`, `company`, `opportunity`, `note`, `task`,
and all custom objects.
This composite type contains three properties:
- `source`
```typescript
export enum FieldActorSource {
EMAIL = 'EMAIL',
CALENDAR = 'CALENDAR',
API = 'API',
IMPORT = 'IMPORT',
MANUAL = 'MANUAL',
}
```
- `workspaceMemberId`
- This property can be `undefined` in some cases and refers to the
member who created the record.
- `name`
- Serves as a fallback if the `workspaceMember` is deleted and is used
for other source types like `API`.
### Functionality
The pre-hook system has been updated to allow real-time argument
updates. When a record is created, a pre-hook can now compute and update
the arguments accordingly. This enhancement enables the `createdBy`
field to be populated with the correct values based on the
`authContext`.
The `authContext` now includes:
- An optional User entity
- An optional ApiKey entity
- The workspace entity
This provides access to the necessary data for the `createdBy` field.
In the GraphQL API, only the `source` can be specified in the
`createdBy` input. This allows the front-end to specify the source when
creating records from a CSV file.
### Front-End Handling
On the front-end, `orderBy` and `filter` are only applied to the name
property of the `ACTOR` composite type. Currently, we are unable to
apply these operations to the workspace member relation. This means that
if a workspace member changes their first name or last name, there may
be a mismatch because the name will differ from the new one. The name
displayed on the screen is based on the workspace member entity when
available.
### Missing Components
Currently, this PR does not include a `createdBy` value for the `MAIL`
and `CALENDAR` sources. These records are created in a job, and at
present, we only have access to the workspaceId within the job. To
address this, we should use a function similar to
`loadServiceWithContext`, which was recently removed from `TwentyORM`.
This function would allow us to pass the `authContext` to the jobs
without disrupting existing jobs.
Another PR will be created to handle these cases.
### Related Issues
Fixes issue #5155.
### Additional Notes
This PR doesn't include the migrations of the current records and views.
Everything works properly when the database is reset but this part is
still missing for now. We'll add that in another PR.
- There is a minor issue: front-end tests are broken since this commit:
[80c0fc7ff1).
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
An ACTIVE workspace is a workspace that has a complete workspaceSchema
and is authorized to be browsed by users.
In this PR, I'm:
- introducing a new activationStatus: PENDING_CREATION (existing ACTIVE
/ INACTIVE)
- removing workspaceService.isWorkspaceActivated (based on
workspaceSchema existence which is not robust and checking
activationStatus.ACTIVE instead)
- removing dynamic activationStatus field on worksapce resolver (we can
use the postgres column directly now that data has been migrated)
- on user sign up creating the workspace in PENDING_CREATION, and on
workspace activation setting it to ACTIVE
- only re-activating a workspace if the current activationStatus is
INACTIVE through billing webhooks (a PENDING_CREATION should stay
PENDING and ACTIVE should stay ACTIVE)
Closes#6255
- Move files from `messaging/common` into the correct module
- Remove common module between calendar and messaging
`calendar-messaging-participant-manager`
- Update and fix massaging and calendar participant matching
- Create `MatchParticipantModule`
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR refactors the ORM-Manager to simplify and unify the datasource
creation. I'm deprecating all usages if InjectWorkspaceDatasource and
InjectWorkspaceRepository as we can't be sure they are up-to-date
This PR fixes a few bugs on TwentyORM:
- fix many to one relations that were not properly queries
- fix many to one relations that were not properly parsed
- compute datasource (or use from cache) at run-time and do not use
injected one that could be outdated
We still need to refactor it to simplify, I feel the API are too complex
and we have too many cache layers. Also the relation computation part is
very complex and bug prone
## Context
LabelIdentifier and ImageIdentifier are metadata info attached to
objectMetadata that are used to display a record in a more readable way.
Those columns point to existing fields that are part of the object.
For example, for a relation picker of a person, we will show a record
using the "name" labelIdentifier and the "avatarUrl" imageIdentifier.
<img width="215" alt="Screenshot 2024-07-11 at 18 45 51"
src="https://github.com/twentyhq/twenty/assets/1834158/488f8294-0d7c-4209-b763-2499716ef29d">
Currently, the FE has a specific logic for company and people objects
and we have a way to update this value via the API for custom objects,
but the code is not flexible enough to change other standard objects.
This PR updates the WorkspaceEntity API so we can now provide the
labelIdentifier and imageIdentifier in the WorkspaceEntity decorator.
Example:
```typescript
@WorkspaceEntity({
standardId: STANDARD_OBJECT_IDS.activity,
namePlural: 'activities',
labelSingular: 'Activity',
labelPlural: 'Activities',
description: 'An activity',
icon: 'IconCheckbox',
labelIdentifierStandardId: ACTIVITY_STANDARD_FIELD_IDS.title,
})
@WorkspaceIsSystem()
export class ActivityWorkspaceEntity extends BaseWorkspaceEntity {
@WorkspaceField({
standardId: ACTIVITY_STANDARD_FIELD_IDS.title,
type: FieldMetadataType.TEXT,
label: 'Title',
description: 'Activity title',
icon: 'IconNotes',
})
title: string;
...
```
- Refactor connected account module
- Move blocklist into it's own module
- Move contact-creation-manager into it's own module
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
- Refactor calendar modules and some messaging modules to better
organize them by business rules and decouple them
- Work toward a common architecture for the different calendar providers
by introducing interfaces for the drivers
- Modify cron job to use the new sync statuses and stages
We have recently decided that boolean fields should only accept truthy
or falsy value, with users deciding of a default value at creation.
This command helps cleaning the existing data, by
1. updating all boolean fields default values from null to false
2. updating all boolean fields values for records from null to false
---------
Co-authored-by: Weiko <corentin@twenty.com>
An error was introduced in the calendar cron job because we tried to
inject the workspace context inside the calendarChannelRepository where
we didn't have access to that context.
Closes#5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
- move front `onboardingStatus` computing to server side
- add logic to `useSetNextOnboardingStatus`
- update some missing redirections in
`usePageChangeEffectNavigateLocation`
- separate subscriptionStatus from onboardingStatus
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.
This decorator can be used that way:
```typescript
@WorkspaceRelation({
standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
type: RelationMetadataType.MANY_TO_ONE,
label: 'Company',
description: 'ActivityTarget company',
icon: 'IconBuildingSkyscraper',
inverseSideTarget: () => CompanyWorkspaceEntity,
inverseSideFieldKey: 'activityTargets',
})
@WorkspaceIsNullable()
company: Relation<CompanyWorkspaceEntity> | null;
// The argument is the name of the relation above
@WorkspaceJoinColumn('company')
companyId: string | null;
```
#### Overview
This PR introduces a new API for dynamically registering and executing
pre and post query hooks in the Workspace Query Hook system using the
`@WorkspaceQueryHook` decorator. This approach eliminates the need for
manual provider registration, and fix the issue of `undefined` or `null`
repository using `@InjectWorkspaceRepository`.
#### New API
**Define a Hook**
Use the `@WorkspaceQueryHook` decorator to define pre or post hooks:
```typescript
@WorkspaceQueryHook({
key: `calendarEvent.findMany`,
scope: Scope.REQUEST,
})
export class CalendarEventFindManyPreQueryHook implements WorkspaceQueryHookInstance {
async execute(userId: string, workspaceId: string, payload: FindManyResolverArgs): Promise<void> {
if (!payload?.filter?.id?.eq) {
throw new BadRequestException('id filter is required');
}
// Implement hook logic here
}
}
```
This API simplifies the registration and execution of query hooks,
providing a more flexible and maintainable approach.
---------
Co-authored-by: Weiko <corentin@twenty.com>
This PR is replacing and removing all the raw queries and repositories
with the new `TwentyORM` and injection system using
`@InjectWorkspaceRepository`.
Some logic that was contained inside repositories has been moved to the
services.
In this PR we're only replacing repositories for calendar feature.
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
