Commit Graph

499 Commits

Author SHA1 Message Date
8b5f79ddbf fix: multiple twenty orm issues & show an example of use (#5439)
This PR is fixing some issues and adding enhancement in TwentyORM:

- [x] Composite fields in nested relations are not formatted properly
- [x] Passing operators like `Any` in `where` condition is breaking the
query
- [x] Ability to auto load workspace-entities based on a regex path

I've also introduced an example of use for `CalendarEventService`:


https://github.com/twentyhq/twenty/pull/5439/files#diff-3a7dffc0dea57345d10e70c648e911f98fe237248bcea124dafa9c8deb1db748R15
2024-05-20 11:01:47 +02:00
66637a3770 Add more details to mutation limit exception message and fix update many query (#5460)
## Context
Since we rely on PgGraphql to query the DB, we have to map its errors to
more comprehensible errors before sending them back to the FE. This has
already been done for unicity constraint and mutation maximum records
but for the last one the message wasn't clear enough. This PR introduces
a new pgGraphqlConfig param to the util to pass down the 'atMost' config
that we are actually overwriting with an
'MUTATION_MAXIMUM_RECORD_AFFECTED' env variable. See how atMost works in
this doc (https://supabase.github.io/pg_graphql/api/#delete)

Also adding the same message for the update since this mutation is also
affected. Create is not though.

Lastly, this PR introduces a fix on the updateMany. Since the current FE
is not using updateMany, this was missed for a few weeks but a
regression has been introduced when we started checking if the id is a
valid UUID however for updateMany this was checking the data object
instead of the filter object. Actually, the data object should never
contain id because it wouldn't make sense to allow the update of the id
and even more for multiple records since the id should be unique.

## Test
locally with MUTATION_MAXIMUM_RECORD_AFFECTED=5

<img width="1408" alt="Screenshot 2024-05-18 at 02 11 59"
src="https://github.com/twentyhq/twenty/assets/1834158/06bf25ce-4a44-4851-8456-aed7689bb33e">
<img width="1250" alt="Screenshot 2024-05-18 at 02 12 10"
src="https://github.com/twentyhq/twenty/assets/1834158/06fc4329-147b-4bb4-9223-c3bce340a8d2">
<img width="1222" alt="Screenshot 2024-05-18 at 02 12 36"
src="https://github.com/twentyhq/twenty/assets/1834158/0674546e-73e2-4e5c-918f-9825f2ee5967">
<img width="1228" alt="Screenshot 2024-05-18 at 02 13 01"
src="https://github.com/twentyhq/twenty/assets/1834158/f50df435-1fd4-45df-a953-8fefa8f36e75">
<img width="1174" alt="Screenshot 2024-05-18 at 02 13 09"
src="https://github.com/twentyhq/twenty/assets/1834158/707b9300-2779-43df-8177-9658b8965b49">


<img width="1393" alt="Screenshot 2024-05-18 at 02 19 11"
src="https://github.com/twentyhq/twenty/assets/1834158/2cd167b6-1261-4914-a4db-36f792d810c0">
2024-05-18 08:00:00 +02:00
36e54119a3 Enable remotes with existing name (#5433)
- Check if a table with the same name already exists
- If yes, add a number suffix, and check again

Co-authored-by: Thomas Trompette <thomast@twenty.com>
2024-05-17 10:38:17 +02:00
1694e0cccd Fix missing name validation on object names at update (#5434)
## Context
as per title

## How was it tested? 
local (/metadata + in product)
2024-05-16 18:15:56 +02:00
d741f4a5bd Minor refacto and fixes on Remotes updates (#5438)
In this PR

- Code refactoring
- v0 of adding "updates available" info in Connection sync status
<img width="835" alt="Capture d’écran 2024-05-16 à 17 02 07"
src="https://github.com/twentyhq/twenty/assets/51697796/9674d3ca-bed2-4520-a5a6-ba37bc242d06">

- fix distant table columns with not-camel case names are always
considered as new
2024-05-16 17:31:34 +02:00
afad993bb3 Fix main (#5435)
- fix lint issue
- fix Apply Cors exception handler (do not work when logged out)
2024-05-16 15:29:27 +02:00
fdf10f17e2 4655 batch endpoints on the rest api (#5411)
- add POST rest/batch/<OBJECT> endpoint
- rearrange rest api code with Twenty quality standard
- unify REST API error format
- Added PATCH verb to update objects
- In openapi schema, we replaced PUT with PATCH verb to comply with REST
standard
- fix openApi schema to match the REST api

### Batch Create

![image](https://github.com/twentyhq/twenty/assets/29927851/fe8cd91d-7b35-477f-9077-3477b57b054c)

### Replace PUT by PATCH in open Api

![image](https://github.com/twentyhq/twenty/assets/29927851/9a95060d-0b21-4a04-a3fa-c53390897b5b)

### Error format unification

![image](https://github.com/twentyhq/twenty/assets/29927851/f47dfcef-a4f8-4f93-8504-22f82a8d8057)

![image](https://github.com/twentyhq/twenty/assets/29927851/d76a87e2-2bf6-4ed9-a142-71ad7c123beb)

![image](https://github.com/twentyhq/twenty/assets/29927851/6db59ad3-0ba7-4390-a02d-be15884e2516)
2024-05-16 14:15:49 +02:00
ea5a7ba70e feat: add renew token query for apollo client (chrome-extension) (#5200)
fixes - #5203
2024-05-16 10:21:16 +02:00
63387424c3 Fix transliteration for metadata + transliterate select options (#5430)
## Context 
Fixes #5403

Transliteration is now integrated to form validation through the schema.
While it does not impede inputting an invalid value, it impedes
submitting a form that will fail as the transliteration is not possible.
Until then we were only performing the transliteration at save time in
the front-end, but it's best to provide the information as soon as
possible. Later we will add helpers to guide the user (eg "This name is
not valid": https://github.com/twentyhq/twenty/issues/5428).

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-15 21:43:58 +02:00
e1eead56c6 Alter comment on foreign key deletion (#5406)
We do not update the comment on the local table when a foreign table key
is deleted.
This was not breaking, which is why we did not see it. But comments
should be kept up to date.

---------

Co-authored-by: Thomas Trompette <thomast@twenty.com>
2024-05-15 17:05:30 +02:00
f0383e3147 feat: twenty orm sync (#5266)
This PR is updating all object metadata entities with the new
decorators, and deleting the old ones.
This way we can use the new TwentyORM with all the standard objects.

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-05-15 16:58:47 +02:00
38eb293b3c Compare distant tables schema with remote tables schema (#5413)
Closes #4532 and part of #5062
2024-05-15 15:47:54 +02:00
a53ce1c488 Track backend events (#5405)
Add tracking to backend events, we might disable frontend tracking which
doesn't bring much value to improve the product
2024-05-14 16:42:28 +02:00
ffdd3a7d4e Return graphql errors when exists (#5389)
- throw badRequest with graphql error messages when graphql request
fails
- clean some code

Before
<img width="1470" alt="image"
src="https://github.com/twentyhq/twenty/assets/29927851/0b700d9a-2bbe-41f7-84a9-981dc7dd5344">

After

![image](https://github.com/twentyhq/twenty/assets/29927851/6bbaaf7c-1244-473d-9ae5-4fefc6a1b994)
2024-05-14 13:21:55 +02:00
b9154f315e Enable deletion of relation fields (#5338)
In this PR
1. Enable deletion of relation fields in the product and via the api
(migration part was missing in the api)
3. Change wording, only use "deactivate" and "delete" everywhere (and
not a mix of the two + "disable", "erase")
2024-05-13 17:43:51 +02:00
0018ec78b0 4840 multi workspace update user userworkspace inconsistent on delete set null constraint (#5373) 2024-05-13 14:50:27 +02:00
8576127b47 Add migration to restrict users without workspaces (#5369)
- update set null ON DELETE constraint to RESTRICT
- update missing updates
2024-05-13 14:18:45 +02:00
1ac8abb118 5188 bug some canceled subscriptions are billed (#5254)
When user is deleting its account on a specific workspace, we remove it
as if it was a workspaceMember, and if no workspaceMember remains, we
delete the workspace and the associated stripe subscription
2024-05-13 10:23:32 +02:00
ce4e78aa85 Fix Rest API id UUID error (#5321)
A user has reported an issue with REST API.
We have recently migrated the graphql IDs from UUID to ID type. As Rest
API is leveraging the graphql API under the hood, the Rest API query
builder should be updated accordingly
2024-05-07 21:04:45 +02:00
b691894254 Fix query runner throwing 500 when pg_graphql detects unique constraint (#5323)
## Context
Since pg_graphql does not return specific error/exception, we have to
map the error message and throw validation errors when needed
This PR adds a check on unicity constraint error returned by pg_graphql
when we are trying to insert duplicate records and returns a 400 instead
of being handled by the exceptionHandler as a 500.
2024-05-07 21:03:15 +02:00
e802cef8f1 Fix 400 yoga errors being sent to exception handlers (#5322)
## Context
Yoga can catch its own errors and we don't want to convert them again.
Moreover those errors don't have an "originalError" property and should
be schema related only (400 validation) so we only want to send them
back to the API caller without going through the exception handler.

Also fixed an issue in the createMany which was throwing a 500 when id
was missing from the creation payload. It seems the FE is always sending
an ID but it should actually be optional since the DB can generate one.
This is a regression from the new UUID validation introduced a few weeks
ago.
2024-05-07 20:54:10 +02:00
7c3e82870c [fix] Increment cache version after object/field/relation update (#5316)
Fixes #5276.

Updates were not triggering a cache version incrementation because they
do not trigger migrations while that is where the caching version logic
was.
We have decided to move the cache incrementation logic to the services.
2024-05-07 16:30:25 +02:00
8074aae449 Split job modules (#5318)
## Context
JobsModule is hard to maintain because we provide all the jobs there,
including their dependencies. This PR aims to split jobs in dedicated
modules.
2024-05-07 14:08:20 +02:00
ffd804d04b Fix convertRecordPositionsToIntegers command for camelCase tables (#5315)
## Context
Per title, postgresql will use lowercase if not surrounded by quotes
2024-05-07 11:07:20 +02:00
a2017eaeb7 Improve messaging/calendar create contact performance (#5314)
In this PR, I'm refactoring the way we associate messageParticipant post
person/company creation. Instead of looking a all person without
participant, we are passing the one that were just created.

Also, I'm making sure the message and messageParticipant creation
transaction is commited before creating person/company creation (and
then messageParticipant association)
2024-05-06 23:43:18 +02:00
2c9f50ecb1 Fix enum defaultValue issues (#5307)
This PR fixes several issues:
- enum naming should be: {tableName}_{fieldName}_enum and respecting the
case
- defaultValue format handled in the FE should respect the one in the BE

In my opinion we should refactor the defaultValue:
- we should respect backend format: "'myDefault'" for constant default
and "0" for float, "now" for expressions, "true" for booleans. we can
rename it to defaultValueExpression if it is more clear but we should
not maintain a parallel system
- we should deprecate option: isDefaultValue which is confusing
- we should re-work backend to have a more unified approach between
fields and avoid having if everywhere about select, multiselect, and
currency cases. one unified "computeDefaultValue" function should do the
job

What is still broken:
- currency default Value on creation. I think we should do the refactor
first
- select default value edition.
These cases do not break the schema but are ignored currently
2024-05-06 17:00:38 +02:00
ff77a4ee21 Feat/migrate password reset token to app token table (#5051)
# This PR

- Fix #5021 
- Migrates `passwordResetToken` and `passwordResetTokenExpiresAt` fields
from `core.users` to `core.appToken`
- Marks those fields as `deprecated` so we can remove them later if we
are happy with the transition -- I took this decision on my own,
@FellipeMTX let me know what you think about it, we can also remove them
straight away if you think it's better
- Fixed the `database:migration` script from the `twenty-server` to:
```json
    "database:migrate": {
      "executor": "nx:run-commands",
      "dependsOn": ["build"], // added this line
      "options": {
        "cwd": "packages/twenty-server",
        "commands": [
          "nx typeorm -- migration:run -d src/database/typeorm/metadata/metadata.datasource",
          "nx typeorm -- migration:run -d src/database/typeorm/core/core.datasource"
        ],
        "parallel": false
      }
    },
```
The migration script wasn't running because the builds were not executed
- [x] Added unit tests for the token.service file's changes

Looking forward to hearing feedback from you

cc: @charlesBochet

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-05-06 15:30:03 +02:00
b207d10312 feat: extend twenty orm (#5238)
This PR is a follow up of PR #5153.
This one introduce some changes on how we're querying composite fields.
We can do:

```typescript
export class CompanyService {
  constructor(
    @InjectWorkspaceRepository(CompanyObjectMetadata)
    private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    // Old way
    // const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
    //   where: { xLinkLabel: 'MyLabel' },
    // });
    // Result will return xLinkLabel property

    // New way
    const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
      where: { xLink: { label:  'MyLabel' } },
    });
    // Result will return { xLink: { label: 'MyLabel' } } property instead of  { xLinkLabel: 'MyLabel' }

    return companiesFilteredByLinkLabel;
  }
}
```

Also we can now inject `TwentyORMManage` class to manually create a
repository based on a given `workspaceId` using
`getRepositoryForWorkspace` function that way:

```typescript
export class CompanyService {
  constructor(
    // TwentyORMModule should be initialized
    private readonly twentyORMManager,
  ) {}

  async companies(): Promise<CompanyObjectMetadata[]> {
    const repository = await this.twentyORMManager.getRepositoryForWorkspace(
      '8bb6e872-a71f-4341-82b5-6b56fa81cd77',
      CompanyObjectMetadata,
    );

    const companies = await repository.find();

    return companies;
  }
}
```
2024-05-06 14:12:11 +02:00
154ae99ed3 [flexible-schema] Add reserved keyword check on object creation (#5303)
## Context
Because creating an object in metadata also generates a graphql type and
because graphql does not allow 2 types with the same name, we have to
manage a list of reserved keywords that can't be used as object names.

Currently we were maintaining a list of the core objects but we also
have to introduce composite fields that are also generated as gql types.
2024-05-06 13:44:40 +02:00
fc87a51acf fix: fix storybook:build memory allocation error in CI (#5284) 2024-05-03 19:19:21 +02:00
a750901582 Remove Feature Flag on Calendar (#5288)
Remove Calendar feature Flag!
2024-05-03 19:10:33 +02:00
381bf0fc8d Create convert record positions to integers command (#5287)
## Context
Positions are used within a view to display and sort the different
records of standard/custom object.
When we add a new record and want to put it before the existing first
record, we have to use float values to insert them in the DB and respect
the desired order. We are adding a new command that can be executed to
flatten those positions.

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-05-03 19:05:56 +02:00
a5a9e0e238 Remove isMultiSelect feature flag (#5280)
As title

Co-authored-by: Thomas Trompette <thomast@twenty.com>
2024-05-03 16:30:58 +02:00
1d9cd234ea Fix white screen on token expire (#5271)
While using middleware (executed pre-graphql) for graphql endpoint, we
need to swallow exception and return errors with a 200. Otherwise it's
not a valid graphql response
2024-05-03 15:35:49 +02:00
50421863d4 Fix filter transform with logic operators (#5269)
Various fixes

- Remote objects are read-only for now, we already hide and block most
of the write actions but the button that allows you to add a new record
in an empty collection was still visible.
- CreatedAt is not mandatory on remote objects (at least for now) so it
was breaking the show page, it now checks if createdAt exists and is not
null before trying to display the human readable format `Added x days
ago`
- The filters are overwritten in query-runner-args.factory.ts to handle
NUMBER field type, this was only working with filters like
```
      {
        "id": {
          "in": [
            1
          ]
        }
```
but not with more depth such as 
```
    "and": [
      {},
      {
        "id": {
          "in": [
            1
          ]
        }
      }
    ]
 ```
- Fixes CREATE FOREIGN TABLE raw query which was missing ",".
2024-05-03 14:52:20 +02:00
30ffe0160e Fix token validation on graphql IntrospectionQuery (#5255)
## Context
We recently introduced a change that now throws a 401 if the token is
invalid or expired.
The first implementation is using an allow list and 'IntrospectionQuery'
was missing so the playground was broken.

The check has been updated and we now only check the excludedOperations
list if a token is not present. This is because some operations can be
both used as loggedIn and loggedOut so we want to validate the token for
those sometimes (and set the workspace, user, cache version, etc). Still
not a very clean solution imho.
2024-05-03 10:30:47 +02:00
fe758e193f fix workspace-member deletion with existing attachments/documents (#5232)
## Context
We have a non-nullable constraint on authorId in attachments and
documents, until we have soft-deletion we need to handle deletion of
workspace-members and their attachments/documents.
This PR introduces pre-hooks to deleteOne/deleteMany
This is called when a user deletes a workspace-member from the members
page

Next: needs to be done on user level as well. This is called when users
try to delete their own accounts. I've seen other issues such as
re-creating a user with a previously used email failing.
2024-05-02 17:36:57 +02:00
f9c19c839b Build stripe integration on backend side (#5246)
Adding stripe integration by making the server logic independent of the
input fields:
- query factories (remote server, foreign data wrapper, foreign table)
to loop on fields and values without hardcoding the names of the fields
- adding stripe input and type
- add the logic to handle static schema. Simply creating a big object to
store into the server

Additional work:
- rename username field to user. This is the input intended for postgres
user mapping and we now need a matching by name

---------

Co-authored-by: Thomas Trompette <thomast@twenty.com>
2024-05-02 17:13:15 +02:00
3015f4ce31 Fix sync metadata script (#5253)
While troubleshooting self-hosting migration, we run into issues with
sync-metadata script introduced by recent changes
2024-05-02 15:50:40 +02:00
05a90d6153 Constant api version (#5248)
closes #5206
2024-05-02 14:21:19 +02:00
9a116b08a4 User workspace middleware throws 401 if token is invalid (#5245)
## Context
Currently, this middleware validates the token and stores the user,
workspace and cacheversion in the request object.
It only does so when a token is provided and ignores the middleware
logic if not. If the token is invalid or expired, the exception is
swallowed.

This PR removes the try/catch and adds an allowlist to skip the token
validation for operations executed while not signed-in.
I don't know a better way to do that with Nestjs. We can't easily add
the middleware per resolver without refactoring the flexible schema
engine so I'm doing it the other way around.

Fixes https://github.com/twentyhq/twenty/issues/5224
2024-05-02 12:54:01 +02:00
0d023e5e77 feat: update links field (#5212)
Closes #5113

---------

Co-authored-by: Jérémy Magrin <jeremy.magrin@gmail.com>
2024-05-01 14:56:55 +02:00
8853226d17 feat: add Links field type (#5176)
Closes #5113

---------

Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2024-05-01 11:56:14 +02:00
e0ece3c917 Rename types for UserMappingOptions (#5230)
Following #5210

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-05-01 11:46:47 +02:00
1b2ed80c1c [feat][Remote objects] Edit a connection (for pg) (#5210)
## Context
#4774 

## How was it tested
Locally

## In further PRs
- Update connection status upon page change
- Adapt Info banner to dark mode
- placeholders for form
2024-04-30 17:46:30 +02:00
3bf9045990 Fix record position on contact creation (#5227)
Fix record position on contact creation

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-04-30 17:09:29 +02:00
bc0d30f28b [flexible-backend] handle object already exists error (#5225)
## Context
Currently we have an unicity constraint in the DB but we don't return a
clear error to the frontend before reaching the DB (which then throws a
500). This PR adds a validation check similar to what we have with field
creation
2024-04-30 16:36:02 +02:00
a77cb023c0 Flush cache when reset db (#5214)
Now that we have persistent cache for schemas, we want to be able to
reset its state when users run the database:reset db otherwise schemas
won't be synced with the new DB state.

Note: In an upcoming PR, we want to be able to invalidate the cache on a
workspace level when we change the metadata schema through twenty
version upgrade
2024-04-30 15:03:24 +02:00
f512049381 [messaging/calendar] cron jobs can run regardless of sub status if billing is disabled (#5218)
## Context
Messaging and calendar cron jobs are only working for workspace that
have sub status different than incomplete, this is because currently
this is the simplest way to know if a user is onboarded. This should not
be the source of truth and this will be updated in a later version. In
the meantime, to make self-hosting easier, we are adding an extra check
on IS_BILLING_ENABLED env var since sub status is not relevant for
people not using billing.
2024-04-30 15:01:22 +02:00
7c605fc2f9 4002 prevent user from creating twice the same blocklist item (#5213)
Closes #4002
2024-04-30 14:36:33 +02:00