Commit Graph

131 Commits

Author SHA1 Message Date
e7bbc7ddfc Fix participant listeners (#6767)
Fixes a bug where all the messagesParticipants and the
calendarEventParticipants were linked to a contact after its
restoration.
2024-08-28 16:26:08 +02:00
8dd772aee2 Fix post merge conflicts on messaging services 2024-08-27 19:30:25 +02:00
e6a55f270b 5617 Create CalendarOngoingStaleCron Job (#6748)
Closes #5617

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 19:13:15 +02:00
e771793626 6655 remove field direction in message and add it in mcma (#6743)
Closes #6655 
- Remove direction from message
- Add direction do mcma
- Create migration command
- Create upgrade 0.24
2024-08-27 19:11:04 +02:00
81fa3f0c41 6256 refactor messaging module to remove all provider specific code and put it inside the drivers folders (#6721)
Closes #6256 
Closes #6257 
+ Create custom exceptions

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 18:14:45 +02:00
f6fd92adcb [POC] add graphql query runner (#6747)
## Context
The goal is to replace pg_graphql with our own ORM wrapper (TwentyORM).
This PR tries to add some parsing logic to convert graphql requests to
send to the ORM to replace pg_graphql implementation.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-27 17:06:39 +02:00
1eeeae8564 6686 Add try catch on every cron job, and send exception to exceptionHandler (#6705)
Closes #6686

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-22 18:23:05 +02:00
579c2ebcea 6687 change messaging import cron job to run every minute (#6704)
Closes #6687
2024-08-22 17:51:08 +02:00
091c0f83be 6619 modify event emitter to emit an array of events (#6625)
Closes #6619

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-20 19:44:29 +02:00
17a1760afd Improve performance twenty orm (#6691)
## Context

As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!

I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
2024-08-20 19:42:02 +02:00
12a657ce29 Patch messaging import crons not running (#6688)
In 0.23.1, we have introduced a regression by migrating to TwentyORM ;
messageChannels were not considered as syncable anymore
2024-08-19 19:19:01 +02:00
d1c278d6b2 6430 Part 1: remove all raw queries from the messaging and calendar modules (#6572)
Part 1 of #6430
- Remove all repositories which contained raw queries in `messaging`
module
- Replace them using `twentyORMManager`
2024-08-13 19:40:50 +02:00
d5350e11a3 Remove some dead code (#6611)
We could remove a lot more than this, this is just a start.

There are various tools to help with this, knip is a good one
2024-08-11 20:43:18 +02:00
11a41b3d97 feat: created by email calendar (#6536)
This PR is a followup of #6324 to add support of EMAIL and CALENDAR
source for the created by composite field.
2024-08-07 15:03:06 +02:00
5a72b949d7 6382 create a command to add a uservar in the key value pair table for every account which needs to reconnect (#6553)
Closes #6382

Create SetUserVarsAccountsToReconnectCommand.
This command loops on all workspaces and:
- deletes all user vars with deprecated key `ACCOUNTS_TO_RECONNECT`
- creates a key value pair of type `USER_VAR` with a key of
`ACCOUNTS_TO_RECONNECT_INSUFFICIENT_PERMISSIONS` for all connect
accounts with a message channel or calendar channel with status
`FAILED_INSUFFICIENT_PERMISSIONS`
2024-08-07 11:43:18 +02:00
018b8220dc Remove message thread id from mcma and update scripts (#6500)
- Remove `messageThreadId` from `messageChannelMessageAssociation`
- Update thread merging
- Update all queries which were dependent on this field
- Update some raw queries by using `twentyORM` instead

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-08-06 18:19:59 +02:00
2073d8e6e1 6446 improve information banner component to make it scale better (#6545)
Closes #6446
2024-08-05 16:00:52 +02:00
03204021cb Refactor onboarding user vars to be absent when user is fully onboarded (#6531)
In this PR:
- take feedbacks from: https://github.com/twentyhq/twenty/pull/6530 /
https://github.com/twentyhq/twenty/pull/6529 /
https://github.com/twentyhq/twenty/pull/6526 /
https://github.com/twentyhq/twenty/pull/6512
- refactor onboarding uservars to be absent when the user is fully
onboarded: isStepComplete ==> isStepIncomplete
- introduce a new workspace.activationStatus: CREATION_ONGOING

I'm retesting the whole flow:
- with/without BILLING
- sign in with/without SSO
- sign up with/without SSO
- another workspaceMembers join the team
- subscriptionCanceled
- access to billingPortal
2024-08-04 20:37:36 +02:00
7cd5427589 Fix onboarding status performance issues (#6512)
Updated the onboardingStatus computation to improve performances

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-04 00:33:33 +02:00
6432ad39b9 feat: add new ACTOR field type and createdBy standard fields (#6324)
This pull request introduces a new `FieldMetadataType` called `ACTOR`.
The primary objective of this new type is to add an extra column to the
following objects: `person`, `company`, `opportunity`, `note`, `task`,
and all custom objects.

This composite type contains three properties:

- `source`
    ```typescript
    export enum FieldActorSource {
      EMAIL = 'EMAIL',
      CALENDAR = 'CALENDAR',
      API = 'API',
      IMPORT = 'IMPORT',
      MANUAL = 'MANUAL',
    }
    ```
- `workspaceMemberId`
- This property can be `undefined` in some cases and refers to the
member who created the record.
- `name`
- Serves as a fallback if the `workspaceMember` is deleted and is used
for other source types like `API`.

### Functionality

The pre-hook system has been updated to allow real-time argument
updates. When a record is created, a pre-hook can now compute and update
the arguments accordingly. This enhancement enables the `createdBy`
field to be populated with the correct values based on the
`authContext`.

The `authContext` now includes:
- An optional User entity
- An optional ApiKey entity
- The workspace entity

This provides access to the necessary data for the `createdBy` field.

In the GraphQL API, only the `source` can be specified in the
`createdBy` input. This allows the front-end to specify the source when
creating records from a CSV file.

### Front-End Handling

On the front-end, `orderBy` and `filter` are only applied to the name
property of the `ACTOR` composite type. Currently, we are unable to
apply these operations to the workspace member relation. This means that
if a workspace member changes their first name or last name, there may
be a mismatch because the name will differ from the new one. The name
displayed on the screen is based on the workspace member entity when
available.

### Missing Components

Currently, this PR does not include a `createdBy` value for the `MAIL`
and `CALENDAR` sources. These records are created in a job, and at
present, we only have access to the workspaceId within the job. To
address this, we should use a function similar to
`loadServiceWithContext`, which was recently removed from `TwentyORM`.
This function would allow us to pass the `authContext` to the jobs
without disrupting existing jobs.
Another PR will be created to handle these cases.

### Related Issues

Fixes issue #5155.

### Additional Notes

This PR doesn't include the migrations of the current records and views.
Everything works properly when the database is reset but this part is
still missing for now. We'll add that in another PR.

- There is a minor issue: front-end tests are broken since this commit:
[80c0fc7ff1).

---------

Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-03 15:43:31 +02:00
c3417ddba1 Share an email thread to workspace members chip and dropdown (#4199) (#5640)
# Feature: Email thread members visibility

For this feature we implemented a chip and a dropdown menu that allows
users to check which workspace members can see an email thread, as
depicted on issue (#4199).

## Implementations

- create a new database table (messageThreadMember)
- relations between `messageThreadMembers` and the relevant existing
tables (`MessageThread` and `WorkspaceMembers`)
- added a new column to the `MessageThread table`: `everyone` - to
indicate that all workspace members can see the email thread
- create a new repository for the new table, including new queries
- edit the queries so that the new fields could be fetched from the
frontend
- created a component `MultiChip`, that shows a group of user avatars,
instead of just one
- created a component, `ShareDropdownMenu`, that shows up once the
`EmailThreadMembersChip` is clicked. On this menu you can see which
workspace members can view the email thread.

## Screenshots

Here are some screenshots of the frontend components that were created:

Chip with everyone in the workspace being part of the message thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/80d75cdc-656f-490d-9eb1-a07346aad75c)

Chip with just one member of the workspace (the owner) being part of the
message thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/c26677c6-ab93-4149-8201-b110d7346a28)

Chip with some members of the workspace being part of the message
thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/9eccf5f8-134c-4c62-9145-5d5aa2346071)

How the chip looks in a message thread:

![image](https://github.com/twentyhq/twenty/assets/26422084/a9de981d-7288-4aed-8616-c1cb7de524e2)

Dropdown that opens when you click on the chip:

![image](https://github.com/twentyhq/twenty/assets/26422084/a1bb9cd4-01bb-45c5-bf8b-b31c2f3d85e0)

## Testing and Mock data

We also added mock data (TypeORM seeds), focusing on adding mock data
related to message thread members.

## Conclusion

As some of the changes that we needed to do, regarding the change of
visibility of the message thread, were not covered by the existing
documentation, we were told to open a PR and ask for feedback on this
part of the implementation. Right now, our implementation is focused on
displaying who is part of an email thread.

Feel free to let us know which steps we should follow next :)

---------

Co-authored-by: Simão Sanguinho <simao.sanguinho@tecnico.ulisboa.pt>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2024-07-31 18:50:27 +02:00
b85ae7e1ac Fix googleApisSetRequestExtraParams (#6455) 2024-07-30 16:08:59 +02:00
6728e40256 5899 display a banner to alert users which need to reconnect their account (#6301)
Closes #5899

<img width="1280" alt="Index - banner"
src="https://github.com/twentyhq/twenty/assets/71827178/313cf20d-eb34-496a-8c7c-7589fbd55954">

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-27 18:34:52 +02:00
d0db3b765f 6255 move services from messaging common module into the correct module and refactor them (#6409)
Closes #6255 

- Move files from `messaging/common` into the correct module
- Remove common module between calendar and messaging
`calendar-messaging-participant-manager`
- Update and fix massaging and calendar participant matching
- Create `MatchParticipantModule`

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-27 12:29:02 +02:00
1529148c04 Updated MessageChannelSyncStatus enum: changed COMPLETED to ACTIVE (#5965)
Fixes #5961

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-25 21:17:55 +02:00
d8cadad0fa Deprecate inject workspace repo (#6353) 2024-07-20 00:43:29 +02:00
67e2d5c73a Add label identifier to object decorator (#6227)
## Context
LabelIdentifier and ImageIdentifier are metadata info attached to
objectMetadata that are used to display a record in a more readable way.
Those columns point to existing fields that are part of the object.
For example, for a relation picker of a person, we will show a record
using the "name" labelIdentifier and the "avatarUrl" imageIdentifier.
<img width="215" alt="Screenshot 2024-07-11 at 18 45 51"
src="https://github.com/twentyhq/twenty/assets/1834158/488f8294-0d7c-4209-b763-2499716ef29d">

Currently, the FE has a specific logic for company and people objects
and we have a way to update this value via the API for custom objects,
but the code is not flexible enough to change other standard objects.

This PR updates the WorkspaceEntity API so we can now provide the
labelIdentifier and imageIdentifier in the WorkspaceEntity decorator.

Example:
```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.activity,
  namePlural: 'activities',
  labelSingular: 'Activity',
  labelPlural: 'Activities',
  description: 'An activity',
  icon: 'IconCheckbox',
  labelIdentifierStandardId: ACTIVITY_STANDARD_FIELD_IDS.title,
})
@WorkspaceIsSystem()
export class ActivityWorkspaceEntity extends BaseWorkspaceEntity {
  @WorkspaceField({
    standardId: ACTIVITY_STANDARD_FIELD_IDS.title,
    type: FieldMetadataType.TEXT,
    label: 'Title',
    description: 'Activity title',
    icon: 'IconNotes',
  })
  title: string;
...
```
2024-07-19 14:24:04 +02:00
11da718482 Refactor connected account module (#6225)
- Refactor connected account module
- Move blocklist into it's own module
- Move contact-creation-manager into it's own module

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-12 20:15:33 +02:00
52aa9abd73 Remove old message channel sync statuses and create migration command (#6177)
Remove old message channel sync statuses and create migration command

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-12 18:36:42 +02:00
28387003d2 Fix contact creation and rename email aliases to handle aliases (#6176)
Fix contact creation (linked to #6162) and rename email aliases to
handle aliases
2024-07-09 17:49:03 +02:00
37fb88c533 Modify messaging message channel sync status monitoring cron pattern (#6173)
Change from every hour to every 10 minutes, starting at 2 minutes past
the hour
2024-07-09 14:42:57 +02:00
3e453054b7 Fix contactCreation ignoring connectedAccount mainHandle 2024-07-08 18:13:10 +02:00
5638af4385 Fix wrong email direction (#6163)
Closes #6162
2024-07-08 17:55:49 +02:00
f458322303 Refactor calendar to use new sync statuses and stages (#6141)
- Refactor calendar modules and some messaging modules to better
organize them by business rules and decouple them
- Work toward a common architecture for the different calendar providers
by introducing interfaces for the drivers
- Modify cron job to use the new sync statuses and stages
2024-07-08 17:01:06 +02:00
9ba211055a Add message import granulary on non-pro emails, group emails and received contact creation (#6156)
1) Remove featureFlag
2) Base contactCreation on messageChannel.autoContactCreationPolicy
4) add excludeProfessionalEmails + excludeGroupEmails logic
2024-07-08 14:33:48 +02:00
14cd6f8b6b Decrease messaging import batch size 2024-07-03 22:24:21 +02:00
f8dd2cc733 Reorganise calendar module (#6089)
Refactor Calendar into functional sub modules
<img width="437" alt="image"
src="https://github.com/twentyhq/twenty/assets/12035771/d9de3285-a226-4fe8-b3ef-2d8a21def2a5">

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-07-02 13:55:11 +02:00
b371c77284 Change Messaging import frequency 2024-07-01 14:22:31 +02:00
8c33d91734 5748 Create contacts for emails sent and received by email aliases (#5855)
Closes #5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-01 14:21:34 +02:00
632e34fc7b fix: message cleaner find operator (#6080)
This PR fix an issue with the `IsNull()` find operator applied on
one-to-many relation, this one is not supported by TypeORM.
We can instead filter by an empty array to retrieve object with empty
relations.
2024-07-01 11:23:22 +02:00
13f213a05e feat: message cleaner drop repository (#6052)
This PR use the new `TwentyORM` for the message-cleaner module by using
the new injection system with `@InjectWorkspaceRepository`.
2024-06-30 21:57:18 +02:00
b8f33f6f59 5095 move onboardingstatus computation from frontend to backend (#5954)
- move front `onboardingStatus` computing to server side
- add logic to `useSetNextOnboardingStatus`
- update some missing redirections in
`usePageChangeEffectNavigateLocation`
- separate subscriptionStatus from onboardingStatus
2024-06-28 17:32:02 +02:00
1a66db5bff Refactor messaging refresh access token (#6034)
- Put error handling outside of `refreshAndSaveAccessToken`
- return after failing to refresh access token in
`processMessageBatchImport`
- remove unnecessary token refresh in `processMessageListFetch`
2024-06-27 17:07:45 +02:00
d1bb0fb822 Change messaging batch size and cron pattern (#6063)
Change messaging batch size and cron pattern to accelerate messages
import
2024-06-27 16:23:47 +02:00
ac7d905c84 Create fields for calendar and messaging settings v2 (#6049)
Create fields for calendar and messaging settings v2
2024-06-27 16:23:25 +02:00
845fcb65ce Replace ObjectRecord<MessageChannelWorkspaceEntity> with MessageChannelWorkspaceEntity (#6059)
Replace ObjectRecord<MessageChannelWorkspaceEntity> with
MessageChannelWorkspaceEntity
2024-06-27 14:35:25 +02:00
95c5602a4e feat: manually implement joinColumn (#6022)
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.

This decorator can be used that way:

```typescript
  @WorkspaceRelation({
    standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
    type: RelationMetadataType.MANY_TO_ONE,
    label: 'Company',
    description: 'ActivityTarget company',
    icon: 'IconBuildingSkyscraper',
    inverseSideTarget: () => CompanyWorkspaceEntity,
    inverseSideFieldKey: 'activityTargets',
  })
  @WorkspaceIsNullable()
  company: Relation<CompanyWorkspaceEntity> | null;

  // The argument is the name of the relation above
  @WorkspaceJoinColumn('company')
  companyId: string | null;
```
2024-06-27 11:41:22 +02:00
97822533db Fix cache flush in messaging-channel-sync-status.service (#6024)
Fix cache flush in messaging-channel-sync-status.service
2024-06-27 09:58:37 +02:00
03b00c49d5 Improve gmail error handling by catching and throttling for 400 failedPrecondition (#6044)
Closes #5897
2024-06-27 09:57:19 +02:00
1736aee7ff Remove message-import cache when connectedAccount is removed (#6021) 2024-06-26 11:23:08 +02:00