Commit Graph

691 Commits

Author SHA1 Message Date
b33b468679 Add command to update boolean fields null values (#6113)
We have recently decided that boolean fields should only accept truthy
or falsy value, with users deciding of a default value at creation.

This command helps cleaning the existing data, by
1. updating all boolean fields default values from null to false
2. updating all boolean fields values for records from null to false

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-07-05 12:27:12 +02:00
6cd154aac6 Forbid names above 63 characters to comply with pg identifier limit (#6095)
Fixes #6032.

Pg has a char limit on identifiers (= table, columns, enum names) of 63
bytes.
Let's limit the metadata names that will be converted to identifiers
(objects names, fields names, relation names, enum values) to 63 chars.
For the sake of simplicity in the FE we will limit the input length of
labels.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-04 15:32:42 +02:00
5df0ea6466 fix: message queue injection issue (#6126) 2024-07-04 12:22:58 +02:00
5b4d2d989a Hotfix tests after AI PR (#6124)
I merged https://github.com/twentyhq/twenty/pull/5788 too quickly and
didn't noticed it was broken with
https://github.com/twentyhq/twenty/pull/6069
2024-07-04 10:21:33 +02:00
4c642a0bb8 Text-to-SQL proof of concept (#5788)
Added:
- An "Ask AI" command to the command menu.
- A simple GraphQL resolver that converts the user's question into a
relevant SQL query using an LLM, runs the query, and returns the result.

<img width="428" alt="Screenshot 2024-06-09 at 20 53 09"
src="https://github.com/twentyhq/twenty/assets/171685816/57127f37-d4a6-498d-b253-733ffa0d209f">

No security concerns have been addressed, this is only a
proof-of-concept and not intended to be enabled in production.

All changes are behind a feature flag called `IS_ASK_AI_ENABLED`.

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2024-07-04 08:57:26 +02:00
25fce27fe3 Improve performance/robustness of worker 2024-07-03 22:54:39 +02:00
14cd6f8b6b Decrease messaging import batch size 2024-07-03 22:24:21 +02:00
921b311c65 fix: small PR fixes workspace-sync-fields (#6107)
Small fixes of PR #6069
2024-07-03 19:19:37 +02:00
c2da7c4c6e Fix workspace sync issue (#6121)
Fixing an issue with typeORM not fetching existing metadata collection
while syncing fields and relations
2024-07-03 19:18:57 +02:00
4c57e838ca Fix pg-boss worker not working with dynamic injection (#6119) 2024-07-03 19:17:29 +02:00
4183e5460d Use return await to catch exceptions (#6109)
So the exceptions are handled properly and filtered in sentry
2024-07-03 11:14:28 +02:00
5b26452649 feat: refactor workspace sync fields (#6069)
This PR was first here to fix the issue related to ticket #5004, after
some testing it seems that changing the name of a relation is actually
properly working, if we rename `ONE-TO-MANY` side, the only things that
is going to be updated is the FieldMetadata as the `joinColumn` is
stored on the opposite object.
For `MANY-TO-ONE` relations, the `joinColumn` migration is properly
generated. We need to take care that if we rename a side of a relation,
sometimes the opposite side doesn't have `inverseSideFieldKey`
implemented and used by default the name of the opposite object, so this
is going to throw an error as the field can't be found in the object.

---------

Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
2024-07-02 17:21:13 +02:00
a163ccced6 Fix calendar import cron job (#6096)
An error was introduced in the calendar cron job because we tried to
inject the workspace context inside the calendarChannelRepository where
we didn't have access to that context.
2024-07-02 16:51:10 +02:00
f8dd2cc733 Reorganise calendar module (#6089)
Refactor Calendar into functional sub modules
<img width="437" alt="image"
src="https://github.com/twentyhq/twenty/assets/12035771/d9de3285-a226-4fe8-b3ef-2d8a21def2a5">

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-07-02 13:55:11 +02:00
88915291d9 Fix Active Workspaces check (#6084)
We have recently deprecated our subscriptionStatus on workspace to
replace it by a check on existing subscription (+ freeAccess
featureFlag) but the logic was not properly implemented
2024-07-01 16:45:56 +02:00
bb627a91e2 Use invalid field input error for invalid object metadata input (#6083)
As title
2024-07-01 16:33:48 +02:00
fbbcdb20e5 Fix demo seed script by using usage of deprecated sub status 2024-07-01 15:13:31 +02:00
b371c77284 Change Messaging import frequency 2024-07-01 14:22:31 +02:00
8c33d91734 5748 Create contacts for emails sent and received by email aliases (#5855)
Closes #5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-07-01 14:21:34 +02:00
a15884ea0a Add exceptions for metadata modules (#6070)
Class exception for each metadata module + handler to map on graphql
error

TODO left :
- find a way to call handler on auto-resolvers nestjs query (probably
interceptors)
- discuss what should be done for pre-hooks errors
- discuss what should be done for Unauthorized exception
2024-07-01 13:49:17 +02:00
632e34fc7b fix: message cleaner find operator (#6080)
This PR fix an issue with the `IsNull()` find operator applied on
one-to-many relation, this one is not supported by TypeORM.
We can instead filter by an empty array to retrieve object with empty
relations.
2024-07-01 11:23:22 +02:00
13f213a05e feat: message cleaner drop repository (#6052)
This PR use the new `TwentyORM` for the message-cleaner module by using
the new injection system with `@InjectWorkspaceRepository`.
2024-06-30 21:57:18 +02:00
afb3f4b7b7 Allow s3 credentials via env (#6066)
closes #5072
2024-06-30 21:47:51 +02:00
cce9bf5730 Improve use set next onboarding state (#6076)
querying workspaceMembers may be slow leads to wrong
setNextOnboardingStatus value. So we added a resolved field in workspace
to get workspaceMemberCount directly
2024-06-30 21:00:20 +02:00
b8f33f6f59 5095 move onboardingstatus computation from frontend to backend (#5954)
- move front `onboardingStatus` computing to server side
- add logic to `useSetNextOnboardingStatus`
- update some missing redirections in
`usePageChangeEffectNavigateLocation`
- separate subscriptionStatus from onboardingStatus
2024-06-28 17:32:02 +02:00
1a66db5bff Refactor messaging refresh access token (#6034)
- Put error handling outside of `refreshAndSaveAccessToken`
- return after failing to refresh access token in
`processMessageBatchImport`
- remove unnecessary token refresh in `processMessageListFetch`
2024-06-27 17:07:45 +02:00
4f9527c860 5901 refactor email and calendar auto contact creation to create them by batch (#6038)
Closes #5901
2024-06-27 16:37:34 +02:00
d1bb0fb822 Change messaging batch size and cron pattern (#6063)
Change messaging batch size and cron pattern to accelerate messages
import
2024-06-27 16:23:47 +02:00
ac7d905c84 Create fields for calendar and messaging settings v2 (#6049)
Create fields for calendar and messaging settings v2
2024-06-27 16:23:25 +02:00
845fcb65ce Replace ObjectRecord<MessageChannelWorkspaceEntity> with MessageChannelWorkspaceEntity (#6059)
Replace ObjectRecord<MessageChannelWorkspaceEntity> with
MessageChannelWorkspaceEntity
2024-06-27 14:35:25 +02:00
95c5602a4e feat: manually implement joinColumn (#6022)
This PR introduce a new decorator named `@WorkspaceJoinColumn`, the goal
of this one is to manually declare the join columns inside the workspace
entities, so we don't have to rely on `ObjectRecord` type.

This decorator can be used that way:

```typescript
  @WorkspaceRelation({
    standardId: ACTIVITY_TARGET_STANDARD_FIELD_IDS.company,
    type: RelationMetadataType.MANY_TO_ONE,
    label: 'Company',
    description: 'ActivityTarget company',
    icon: 'IconBuildingSkyscraper',
    inverseSideTarget: () => CompanyWorkspaceEntity,
    inverseSideFieldKey: 'activityTargets',
  })
  @WorkspaceIsNullable()
  company: Relation<CompanyWorkspaceEntity> | null;

  // The argument is the name of the relation above
  @WorkspaceJoinColumn('company')
  companyId: string | null;
```
2024-06-27 11:41:22 +02:00
97822533db Fix cache flush in messaging-channel-sync-status.service (#6024)
Fix cache flush in messaging-channel-sync-status.service
2024-06-27 09:58:37 +02:00
03b00c49d5 Improve gmail error handling by catching and throttling for 400 failedPrecondition (#6044)
Closes #5897
2024-06-27 09:57:19 +02:00
1eb9c582f3 Rename mutation maximum affected records (#6042)
As per my last comment on https://github.com/twentyhq/twenty/pull/6039,
we decided to rename this var
2024-06-26 18:00:25 +02:00
6599bc136e Add mutationMaximumRecordAffected to clientConfig (#6039)
We are exposing the server mutationMaximumRecordAffected value via
clientConfig so it can be used by the FE.

This is a first step for https://github.com/twentyhq/twenty/issues/6025

<img width="610" alt="Screenshot 2024-06-26 at 14 58 26"
src="https://github.com/twentyhq/twenty/assets/1834158/9d192976-fd22-45d2-bdaa-a8ff6bb90ca2">
2024-06-26 15:21:13 +02:00
cf67ed09d0 Upsert endpoint and CSV import upsert (#5970)
This PR introduces an `upsert` parameter (along the existing `data`
param) for `createOne` and `createMany` mutations.

When upsert is set to `true`, the function will look for records with
the same id if an id was passed. If not id was passed, it will leverage
the existing duplicate check mechanism to find a duplicate. If a record
is found, then the function will perform an update instead of a create.

Unfortunately I had to remove some nice tests that existing on the args
factory. Those tests where mostly testing the duplication rule
generation logic but through a GraphQL angle. Since I moved the
duplication rule logic to a dedicated service, if I kept the tests but
mocked the service we wouldn't really be testing anything useful. The
right path would be to create new tests for this service that compare
the JSON output and not the GraphQL output but I chose not to work on
this as it's equivalent to rewriting the tests from scratch and I have
other competing priorities.
2024-06-26 11:39:16 +02:00
1736aee7ff Remove message-import cache when connectedAccount is removed (#6021) 2024-06-26 11:23:08 +02:00
78865ee73e Fix billing signup when workspace does not exist (#6018) 2024-06-25 15:51:01 +02:00
7c2e745b45 feat: Dynamic hook registration for WorkspaceQueryHooks (#6008)
#### Overview

This PR introduces a new API for dynamically registering and executing
pre and post query hooks in the Workspace Query Hook system using the
`@WorkspaceQueryHook` decorator. This approach eliminates the need for
manual provider registration, and fix the issue of `undefined` or `null`
repository using `@InjectWorkspaceRepository`.

#### New API

**Define a Hook**

Use the `@WorkspaceQueryHook` decorator to define pre or post hooks:

```typescript
@WorkspaceQueryHook({
  key: `calendarEvent.findMany`,
  scope: Scope.REQUEST,
})
export class CalendarEventFindManyPreQueryHook implements WorkspaceQueryHookInstance {
  async execute(userId: string, workspaceId: string, payload: FindManyResolverArgs): Promise<void> {
    if (!payload?.filter?.id?.eq) {
      throw new BadRequestException('id filter is required');
    }

    // Implement hook logic here
  }
}
```

This API simplifies the registration and execution of query hooks,
providing a more flexible and maintainable approach.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-06-25 12:41:46 +02:00
4dfca45fd3 5615 create messageongoingstalecron (#6005)
Closes #5615
2024-06-25 11:57:02 +02:00
f8c057deea Fix sign up broken because of missing workspace schema (#6013)
Allow workspace datasource factory to return null if the workspace
schema has not been created yet
2024-06-25 10:59:07 +02:00
7fb5c9b60f Remove useless api position parameter (#6010)
- remove buggy addition of position parameter
- check created records are in first position by default
2024-06-25 10:30:19 +02:00
a001bf1514 5951 create a command to trigger the import of a single message (#5962)
Closes #5951

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-24 18:01:22 +02:00
f3701281e9 Create new sync statuses and stages for calendar (#5997)
Create fields:
- syncStatus
- syncStage
- syncStageStartedAt
2024-06-24 16:39:56 +02:00
ad61efe6ed Remove multi select usage (#6004)
As per title!

Also, I'm removing an incorrect logic in the enum migration runner that
takes care of the case where we have no defaultValue but non nullable
which is not a valid business case.
2024-06-24 16:39:17 +02:00
77f9f6473b Create feature flag for calendar V2 (#5998)
Create feature flag for calendar V2
2024-06-24 13:54:52 +02:00
158e7a31f4 Improve tests (#5994)
Our tests on FE are red, which is a threat to code quality. I'm adding a
few unit tests to improve the coverage and lowering a bit the lines
coverage threshold
2024-06-23 20:12:18 +02:00
e13dc7a1fc [FlexibleSchema] Add IndexMetadata decorator (#5981)
## Context
Our Flexible Schema engine dynamically generates entities/tables/APIs
for us but was not flexible enough to build indexes in the DB. With more
and more features involving heavy queries such as Messaging, we are now
adding a new WorkspaceIndex() decorator for our standard objects (will
come later for custom objects). This decorator will give enough
information to the workspace sync metadata manager to generate the
proper migrations that will create or drop indexes on demand.
To be aligned with the rest of the engine, we are adding 2 new tables:
IndexMetadata and IndexFieldMetadata, that will store the info of our
indexes.

## Implementation

```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.person,
  namePlural: 'people',
  labelSingular: 'Person',
  labelPlural: 'People',
  description: 'A person',
  icon: 'IconUser',
})
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
  @WorkspaceField({
    standardId: PERSON_STANDARD_FIELD_IDS.email,
    type: FieldMetadataType.EMAIL,
    label: 'Email',
    description: 'Contact’s Email',
    icon: 'IconMail',
  })
  @WorkspaceIndex()
  email: string;
```
By simply adding the WorkspaceIndex decorator, sync-metadata command
will create a new index for that column.
We can also add composite indexes, note that the order is important for
PSQL.
```typescript
@WorkspaceEntity({
  standardId: STANDARD_OBJECT_IDS.person,
  namePlural: 'people',
  labelSingular: 'Person',
  labelPlural: 'People',
  description: 'A person',
  icon: 'IconUser',
})
@WorkspaceIndex(['phone', 'email'])
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```

Currently composite fields and relation fields are not handled by
@WorkspaceIndex() and you will need to use this notation instead
```typescript
@WorkspaceIndex(['companyId', 'nameFirstName'])
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```
<img width="700" alt="Screenshot 2024-06-21 at 15 15 45"
src="https://github.com/twentyhq/twenty/assets/1834158/ac6da1d9-d315-40a4-9ba6-6ab9ae4709d4">

Next step: We might need to implement more complex index expressions,
this is why we have an expression column in IndexMetadata.
What I had in mind for the decorator, still open to discussion
```typescript
@WorkspaceIndex(['nameFirstName', 'nameLastName'], { expression: "$1 || ' ' || $2"})
export class PersonWorkspaceEntity extends BaseWorkspaceEntity {
```

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-22 12:39:57 +02:00
0b4bfce324 feat: drop calendar repository (#5824)
This PR is replacing and removing all the raw queries and repositories
with the new `TwentyORM` and injection system using
`@InjectWorkspaceRepository`.
Some logic that was contained inside repositories has been moved to the
services.
In this PR we're only replacing repositories for calendar feature.

---------

Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-06-22 09:26:58 +02:00
9228667a57 Add the support of Empty and Non-Empty filter (#5773) 2024-06-20 18:18:12 +02:00