Commit Graph

1825 Commits

Author SHA1 Message Date
f129bc0ac4 Upgrade infer commands from APP_VERSION (#11881)
# Introduction
This PR refactors the way we previously manually handled the upgrade
command `versionTo` and `versionFrom` values to be replaced by a
programmatic inferring using the `APP_VERSION` env variable. It raises
new invariant edge cases that are covered by new tests and so on

Please keep in mind that an upgrade will run agnostically of any `patch`
semver value as it should be done only when releasing a `major/minor`
version update
[Related discord
thread](https://discord.com/channels/1130383047699738754/1368953221921505280)

## Testing in local
In order to test in local we have to define an `APP_VERSION` value in
`packages/twenty-server/.env` following semver ( or not 🙃 )

## Logs example
```ts
Computing new Datasource for cacheKey: 20202020-1c25-4d02-bf25-6aeccf7ea419-8 out of 0
query: SELECT * FROM current_schema()
query: SELECT version();
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Initialized upgrade context with:
   - currentVersion (migrating to): 0.53.0
   - fromWorkspaceVersion: 0.52.0
   - 2 commands
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Upgrading workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 from=0.52.0 to=0.53.0 1/2
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Upgrade for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 ignored as is already at a higher version.
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Running command on workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db 2/2
Computing new Datasource for cacheKey: 3b8e6458-5fc1-4e63-8563-008ccddaa6db-8 out of 0
query: SELECT * FROM current_schema()
query: SELECT version();
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Upgrading workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db from=0.52.0 to=0.53.0 2/2
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Upgrade for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db ignored as is already at a higher version.
[Nest] 37872  - 05/06/2025, 4:07:21 PM     LOG [UpgradeCommand] Command completed!
```

## Misc
Related to https://github.com/twentyhq/twenty/issues/11780
2025-05-07 13:48:19 +00:00
e96afe444f Create filter action (#11904)
Figma
https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=59956-288587&t=Dkp83eigIgb3DO6W-11

Issue
https://github.com/orgs/twentyhq/projects/1/views/3?filterQuery=sprint%3A%40current+assignee%3A%40me&pane=issue&itemId=108202682&issue=twentyhq%7Ccore-team-issues%7C897

- filters will be stored as existing GQL filters. It will avoid
re-building frontend
- `applyFilter` function will take the filter and a JS array in input
and returns the filtered array
- filter action calls the util then returns an empty result and error if
no data in the output array. It will end the workflow gracefully.

Example of action:

```
{
      "id": "9d4aeee9-5b78-4053-9615-c367e901ed71",
      "name": "Filter",
      "type": "FILTER",
      "valid": false,
      "settings": {
        "input": {
          "filter": {
            "employees": {
              "gt": 300,
            },
          }
        }
      }
}
```
2025-05-07 14:52:03 +02:00
463dee3fe6 Remove usages of connectToDataSource and use workspaceDataSource (#11873)
In this PR we are

1. cleaning typeORM service by removing connectToDataSource method
2. using workspaceDataSource instead of mainDataSource when possible,
and replacing raw SQL with workspaceRepository methods to use
2025-05-07 10:42:51 +02:00
7b78b64bca Update clickhouse tables (#11905)
Following a discussion with @Bonapara - changing the base tables
2025-05-07 09:39:18 +02:00
6810ca0204 Move workflow backfill command to 0.53 (#11892)
As title. It has not been added to 0.52 list of commands
2025-05-06 14:09:18 +02:00
e92117d556 Critical email sync bug (#11885)
Removing the upsert to avoid the ON CONFLICT
2025-05-06 11:17:27 +02:00
a60711c808 Ej/fix message visibility (#11874)
<img width="257" alt="Screenshot 2025-05-05 at 15 30 09"
src="https://github.com/user-attachments/assets/5a8e18e0-efc5-4521-9c3a-bf73277ecdf9"
/>
<img width="257" alt="Screenshot 2025-05-05 at 15 29 05"
src="https://github.com/user-attachments/assets/c1a784af-a744-497a-b6ce-ec3a9e8b851a"
/>
<img width="257" alt="Screenshot 2025-05-05 at 15 33 06"
src="https://github.com/user-attachments/assets/c5fabd1d-a125-49d7-aade-0a208a0eff95"
/>

related to PR https://github.com/twentyhq/twenty/pull/11840 and issue
https://github.com/twentyhq/twenty/issues/9826
2025-05-05 17:23:27 +02:00
a9e73c6340 [permissions] Add permissions check layer in entityManager (#11818)
First and main step of
https://github.com/twentyhq/core-team-issues/issues/747

We are implementing a permission check layer in our custom
WorkspaceEntityManager by overriding all the db-executing methods (this
PR only overrides some as a POC, the rest will be done in the next PR).
Our custom repositories call entity managers under the hood to interact
with the db so this solves the repositories case too.
This is still behind the feature flag IsPermissionsV2Enabled.

In the next PR
- finish overriding all the methods required in WorkspaceEntityManager
- add tests
2025-05-05 14:06:54 +00:00
5f8040af5d Modify Decimal Fields to be treated as number in OpenAPI schema. (#11871)
Closes https://github.com/twentyhq/twenty/issues/10807

## Description
This PR will Modify Decimal Fields to be treated as `number` in OpenAPI
schema.

## Testing
<img width="989" alt="スクリーンショット 2025-05-05 20 49 05"
src="https://github.com/user-attachments/assets/2f120317-5860-4c93-91a2-f521a69a1cd5"
/>
<img width="872" alt="スクリーンショット 2025-05-05 20 49 52"
src="https://github.com/user-attachments/assets/0d319785-e30b-4132-be9e-12ed6f3cc46a"
/>

---------

Co-authored-by: Takuya Kurimoto <takuya004869@gmail.com>
Co-authored-by: prastoin <paul@twenty.com>
2025-05-05 15:48:23 +02:00
6128d660c2 Fix form record picker field (#11817)
- enrich response so the record is available in the step output. Today
this is available in the schema but only the id is set
- make the full record picker clickable instead of the arrow only

<img width="467" alt="Capture d’écran 2025-04-30 à 16 08 04"
src="https://github.com/user-attachments/assets/db74b9a6-7f1d-4e54-bf06-9be3d67ee398"
/>
2025-05-05 12:58:11 +00:00
521e75981a Fix calendar events & messages fetching + fix timeline design (#11840)
Preview : 

<img width="501" alt="Screenshot 2025-05-02 at 16 24 34"
src="https://github.com/user-attachments/assets/0c649df1-0e26-4ddc-8e13-ebd78af7ec09"
/>


Done : 
- Fix getCalendarEventsFromPersonIds and getCalendarEventsFromCompanyId
(include accountOwner check)
- Fix permission check on pre-hook - Pre-hook seems useless, calendar
events are always on METADATA or SHARE_EVERYTHING visibility, else post
hook always has the responsibility of returning the data user can
access. >> To delete or to keep in case other visibility options are
added ?
- Add post hook to secure finOne / findMany calendarEvents resolver
- Update design

To do :
- same on messages (PR to arrive)

closes : https://github.com/twentyhq/twenty/issues/9826
2025-05-05 13:12:16 +02:00
a570be0ad1 i18n - translations (#11861)
Created by Github action

Co-authored-by: github-actions <github-actions@twenty.com>
2025-05-04 14:42:30 +02:00
49b7f5255f Update what is being audit logged (#11833)
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-05-04 14:35:41 +02:00
48011ac012 fix: do not report EmailNotVerifiedError to Sentry (#11753) (#11835)
This PR prevents EmailNotVerifiedError from being reported to Sentry, as
discussed in issue #11753.

edit:
close #11753
2025-05-02 15:35:05 +02:00
3e1b4ace37 Standardize isDefined usage for metadata version assertions (#11829)
# Introduction
`!value` is risky as `!falsy` would return `true`
2025-05-02 11:07:12 +02:00
8c3a2d8640 Verify seed users and change AppleCar password (#11825)
isEmailVerified was set to false which was annoying in the staging
environment

Also updated password for tim@apple.dev from AppleCar2025 to just
tim@apple.dev since the joke is outdated
2025-04-30 16:59:13 +00:00
17334f8800 Fix undefined read attempt (#11816)
# Introduction
Followup of https://github.com/twentyhq/twenty/pull/11784
Again some propaganda for the
[noUncheckedIndexedAccess](https://www.typescriptlang.org/tsconfig/#noUncheckedIndexedAccess)
that involved this @charlesBochet 👀
That's very risky to have this setup to false especially in the backend
regarding the twenty-server metadata nature

Also suggested that we do some integration testing ( e2e nestsJs tests )
on a related endpoint that could related we always retrieve the same
form result output

We could also do some unit testing of the method but like the idea to
ship it through the api itself
2025-04-30 17:58:27 +02:00
4ce7834f94 i18n - translations (#11822)
Created by Github action

---------

Co-authored-by: github-actions <github-actions@twenty.com>
2025-04-30 17:20:54 +02:00
849a35955a 866 refactor cron trigger only one cron each minutes triggers all cron triggers (#11809)
<img width="1123" alt="image"
src="https://github.com/user-attachments/assets/75447922-81dd-4cfc-805d-f511f73cc778"
/>
2025-04-30 17:08:47 +02:00
357649db95 fix-discord-timeline (#11784)
In some cases, the person is undefined so we fallback on the
workspacemember, and sometimes he is missing the firstname/lastname.

Not sure how this can happen, but one user experiencd such a thing so
better to catch this


![image](https://github.com/user-attachments/assets/c91445fe-365e-4fc9-bf14-69f71d344aa7)

fix https://discord.com/channels/1130383047699738754/1366145144838951022

close https://github.com/twentyhq/core-team-issues/issues/918
2025-04-30 15:11:00 +02:00
e957b1acd6 Twenty config admin panel integration (#11755)
closes https://github.com/twentyhq/core-team-issues/issues/761
closes https://github.com/twentyhq/core-team-issues/issues/762

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2025-04-30 09:12:59 +02:00
d8b2e1fb34 Start using next step ids (#11683)
- update workflow executor
- update next step ids on step creation/deletion
- use these in workflow run
- use these in variables
2025-04-29 16:29:25 +02:00
23d71915f6 Improve metadata version caching (#11775)
Investigating https://github.com/twentyhq/core-team-issues/issues/756, I
found that the error actually stemmed from "Object metadata collection
not found" error.

While this is planned to be fixed by metadata performance improvements
(as stated in [sentry-boss
doc](https://docs.google.com/document/d/1ytbC5W6ZFUSJ3PoJ4IfKi2IehKZYw65mqCnc24aP4RM/edit?tab=t.0)
in "known issues"), I tried some easy improvements to reduce the number
of errors.
2025-04-29 09:53:19 +02:00
b182a676d7 Fix next step ids backfill command (#11769)
Twenty prod DB has been exported for testing.

Main updates:
- do not process workflow runs with less than 2 steps. Nothing to do.
- update runs by batches of 500. Will avoid java heap space issue
- add more logs
2025-04-28 16:24:15 +02:00
2586d0ec4f fix workflow chips not clickable (#11767)
bug introduced in this PR https://github.com/twentyhq/twenty/pull/1174
2025-04-28 15:30:26 +02:00
a15b87649a Twenty config core implementation (#11595)
closes https://github.com/twentyhq/core-team-issues/issues/760

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-04-26 09:21:59 +02:00
95808b06b9 Messaging-logging (#11723)
parseBatchResponse:
we need more logs to understand why we have some empty repsonse in the
body.

For anothe PR:  
~~UNKNOWN_NETWORK_ERROR : moving from handleUnknownException to
handleTemporaryException. This is important since in the logs I saw "The
service is currently unavailable" which makes me think we should handle
this type of error with as temporary~~
2025-04-25 18:01:53 +02:00
0c8eb149e6 Refactor new relation sync (#11711)
In this PR:
- this should fix the sync metadata for new relation system

This goes with the recent PR:
https://github.com/twentyhq/twenty/pull/11725

What we want:
- ONE_TO_MANY relations should have no joinColumn and no onDelete
- MANY_TO_ONE should have both
2025-04-25 01:02:49 +02:00
4d7dbb1991 Fix missing on delete cascade for relation (#11725)
Follow up from https://github.com/twentyhq/twenty/pull/10217
Adding onDelete action on Many_to_one sides
2025-04-24 18:32:14 +02:00
e750ef28a1 [permission] Override query builders db-executing methods (#11714)
closes https://github.com/twentyhq/core-team-issues/issues/843
2025-04-24 18:20:21 +02:00
e55ecb4dcd object level override form (#11672) 2025-04-24 18:15:43 +02:00
ccc6d968aa update price on subscription - command (#11698) 2025-04-24 09:32:03 +02:00
0b729cb000 Fix json type in clickhouse migrations (#11710)
Settings in clickhouse are only set for the duration of the session so
we need to keep the same client
2025-04-23 22:08:52 +02:00
8076ff5b82 Fix broken metadata sync due to index removal (#11706)
Changing strategy regarding disabling index creation with new relation
system
2025-04-23 18:06:19 +02:00
4257f30f12 Permission checks on twentyORM global manager (#11477)
In this PR we are handling permissions when using
twentyORMGlobalManager,
and handling permissions for rest api and api key
2025-04-23 17:57:48 +02:00
28a1354928 Allow webhook only on active versions (#11705)
- webhook call should only be allow on active versions
- fixing exceptions that are wrongly sent to sentry
2025-04-23 17:28:41 +02:00
a4882089b1 Add is relation enabled to default feature flags (#11703) 2025-04-23 16:26:53 +02:00
1333c64d98 Add diff in workflow update events + fix existing (#11700)
Closes https://github.com/twentyhq/twenty/issues/11676.

Diff should be available for objects displaying timeline activities. It
should also fit the existing type.
2025-04-23 15:43:41 +02:00
e1054b0474 Fix clickhouse migration (#11699)
It looks like Clickhouse Cloud uses a different config var than the docs
2025-04-23 15:09:23 +02:00
b6e8103df7 Mime encode names in email headers (#11691)
# Before
<img width="618" alt="image"
src="https://github.com/user-attachments/assets/e6d7c0ac-10f8-4d2e-abe4-05bf86a2ea9e"
/>

# After
<img width="504" alt="image"
src="https://github.com/user-attachments/assets/db147cb9-be19-4d08-b357-cb554dc2a4ae"
/>

Fixes also Emojis encoding in Subject ->
<img width="540" alt="image"
src="https://github.com/user-attachments/assets/8c27743f-3a50-46eb-b0f3-492219bbd646"
/>

 Works with microsoft accounts
<img width="360" alt="image"
src="https://github.com/user-attachments/assets/d924440f-c107-4c1c-ad4d-b4b27016f102"
/>


Fixes https://github.com/twentyhq/twenty/issues/9194
2025-04-23 11:37:40 +02:00
fa5f758228 Fix storybook / chromatic tests flakyness and integration tests (#11687)
## Storybook flakyness

### Actor Display image flakyness

<img width="1512" alt="image"
src="https://github.com/user-attachments/assets/875c0738-5e31-4aba-9231-4ba5f78d1355"
/>

**Fix:** stop using a random usage

### Task Groups broken

<img width="1512" alt="image"
src="https://github.com/user-attachments/assets/c67e47a1-a027-43f1-9601-68d61a8052b4"
/>

**Fix:** add missing TabListComponentInstance

## Flaky dates

Add https://github.com/k35o/storybook-addon-mock-date

## Integration tests

Fix broken tests due to relation refactoring
2025-04-23 01:57:36 +02:00
8694840b92 Rename webhook and open api urls (#11684)
We want to have fewer base path for routing.

We will have:
- /files
- /webhooks
- /graphql
- /metadata
- /rest
- /auth
- /healthz

I'm moving /open-api under /rest, and centralizing the webhooks
(removing /stripe and /cloudflare)
2025-04-22 22:24:26 +02:00
cc29c25176 feat: new relation sync-metadata, twenty-orm, create/update (#10217)
Fix
https://github.com/twentyhq/core-team-issues/issues/330#issue-2827026606
and
https://github.com/twentyhq/core-team-issues/issues/327#issue-2827001814

What this PR does when `isNewRelationEnabled` is set to `true`:
- [x] Drop the creation of the  foreign key as a `FieldMetadata`
- [x] Stop creating `RelationMetadata`
- [x] Properly fill `FieldMetadata` of type `RELATION` during the sync
command
- [x] Use new relation settings in TwentyORM
- [x] Properly create `FieldMetadata` relations when we create a new
object
- [x] Handle `database:reset` with new relations

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2025-04-22 19:01:39 +02:00
0b7024c94a Perform batch update on workflow runs (#11680)
Command times out on workflow versions with too many runs. Performing
batch update instead
2025-04-22 16:07:29 +02:00
5250d5c8d6 fix subscription item update (#11648)
closes
https://twenty-v7.sentry.io/issues/6550388239/?environment=prod&project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D&referrer=issue-stream&stream_index=6
2025-04-22 14:01:36 +02:00
33e0794da9 Update workflow statuses in real time (#11653)
Statuses are maintained by an async job. Those are calculations that we
would like to avoid using in both frontend and backend. Using push
updates are an easier way.



https://github.com/user-attachments/assets/31e44a82-08a8-4100-a38e-c965d5c73ee8
2025-04-22 11:03:12 +02:00
83434deb22 refactor(auth): optimize work email handling in sign-up flow (#11655)
Extracted isWorkEmail check into a variable for reusability and adjusted
subdomain generation to conditionally include email. This enhances code
readability and maintains logic consistency.
2025-04-19 15:28:41 +02:00
f8b9e4d780 ignoreLock when recomputing rolePermissions when creating datasource (#11657)
[sentry](https://twenty-v7.sentry.io/issues/6545999328/?environment=prod&project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D&referrer=issue-stream&sort=date&stream_index=2)

Our workers have failing jobs because they are concurrently creating
datasources on different pods
While the datasources are independent, they read from the same redis
cache, with the same `RolesPermissionsOngoingCachingLock` value in
redis.
<img width="852" alt="Capture d’écran 2025-04-18 à 18 00 20"
src="https://github.com/user-attachments/assets/42ce8479-acc1-462b-af4c-b547cc2bb0b8"
/>
As a consequence they can fail to create a datasource: the first
datasource computes the permissions and sets ongoingCachingLock to true,
then the second arrives, there are still no available permissions in the
cache, but because of the lock it early returns without permissions and
fails. This behavior goes unnoticed on the product and helps
performances, but is annoying for workers as jobs are failing.

Let's remove the cache lock when creating the datasource !
2025-04-18 18:16:34 +02:00
cf5649a1df Deprecate Sentry release (#11651)
Let's deprecate Sentry Release and use APP_VERSION instead. 

It'll make it more clear in the interface to use named version for bug
analysis, than commit sha
2025-04-18 15:48:48 +02:00
fba4192d4b Fix rest api integration tests (#11645)
as title
2025-04-18 15:23:36 +02:00