Commit Graph

51 Commits

Author SHA1 Message Date
322c8a1852 Upgrade to Node22 (#12488)
BlocknoteJS requires an ESM module where our server is CJS, this forced
us to pin the server-util version, which led us to force the resolution
of several packages, leading to bugs downstream.

From Node 22.12 Node supports requiring ESM modules (available from Node
22.0 with a flag). So I upgrade the module.
I picked Node 22 and not Node 23 or Node 24 because 22 is the LTS and we
don't plan to change node versions frequently.

If you remain on Node 18, things should still mostly work, except if you
edit a Rich Text field.

I also starting changing the default runtime for Serverless Functions
which isn't directly related. This means new serverless functions will
be created on Node 22, but we will still need another PR to migrate
existing serverless functions before September (end of support by AWS).

(In this PR I also remove the upgrade commands from 0.43 since they rely
on Blocknote and I didn't want to have to deal with this)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-06-06 18:35:30 +02:00
cb010d90fe 998 workflow restore (#12417)
Add a post hook to restore workflow sub-entities
2025-06-03 15:28:43 +02:00
4485e8e3db Update enums to be all caps (#12372)
- Make custom domain public (remove from lab)
- Use ALL_CAPS definition for enums
2025-05-29 14:08:36 +02:00
a5b212369b Set findOneServerlessFunction result nullable (#12179)
solves https://github.com/twentyhq/twenty/issues/11819
remove fetch query when deleting code step
2025-05-21 13:59:20 +00:00
442f8dbe3c [QRQC_2] No implicitAny in twenty-server (#12075)
# Introduction
Following https://github.com/twentyhq/twenty/pull/12068
Related with https://github.com/twentyhq/core-team-issues/issues/975

We're enabling `noImplicitAny` handled few use case manually, added a
`ts-expect-error` to the others, we should plan to handle them in the
future
2025-05-15 18:23:22 +02:00
a8423e8503 [QRQC_2] No explicit any in twenty-server (#12068)
# Introduction

Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder

Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles

## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc

## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment

## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
    ignoreRestArgs?: boolean;
```

---------

Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
2025-05-15 16:26:38 +02:00
9c2b88870f Improve sentry grouping (#12007)
This PR attemps at improving sentry grouping and filtering by 
- Using the exceptionCode as the fingerprint when the error is a
customException. For this to work in this PR we are now throwing
customExceptions instead of internalServerError deprived of their code.
They will still be converted to Internal server errors when sent back as
response
- Filtering 4xx issues where it was missing (for emailVerification
because errors were not handled, for invalid captcha and billing errors
because they are httpErrors and not graphqlErrors)

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2025-05-14 09:00:06 +00:00
7b78b64bca Update clickhouse tables (#11905)
Following a discussion with @Bonapara - changing the base tables
2025-05-07 09:39:18 +02:00
49b7f5255f Update what is being audit logged (#11833)
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-05-04 14:35:41 +02:00
587281a541 feat(analytics): add clickhouse (#11174) 2025-04-16 16:33:10 +00:00
1794b3a611 Remove build lambda at creation (#11531)
Remove useless build call (done at first execution)
2025-04-11 14:40:00 +02:00
bd3ec6d5e3 rename core-module environment to twenty-config (#11445)
closes https://github.com/twentyhq/core-team-issues/issues/759
2025-04-09 14:11:26 +02:00
9ad8287dbc [REFACTOR] twenty-shared multi barrel and CJS/ESM build with preconstruct (#11083)
# Introduction

In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)

For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682

close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590

## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )

## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617

## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960

## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level

## Conclusion
As always any suggestions are welcomed !
2025-03-22 19:16:06 +01:00
4036933c84 Add console logs to code step (#10786)
Works for local and lambda drivers

## After

![image](https://github.com/user-attachments/assets/ec082cf6-4159-4a15-99b6-243c73a13773)

![image](https://github.com/user-attachments/assets/f42e3f43-5ea4-4167-b4b5-9a17826fd224)
2025-03-12 10:40:59 +01:00
83930551d8 Move workflow versions and steps building to workflow-builder folder (#10523)
We are starting to put too many services in common folder. Version and
step building should be separated into different services and go to the
builder folder. Today builder folder only manage schema.

We should:
- keep services responsible for only one action
- keep modules based on the actual action these provide rather than
having common module

This PR:
- creates a service for workflow version builder
- moves version and step builders to workflow builder folder rather than
commun
- creates separated folders for schema, version and steps

No logic has been added. Only modules created and functions moved.
2025-02-27 10:39:48 +01:00
927b8c717e Poc lambda deployment duration (#10340)
closes https://github.com/twentyhq/core-team-issues/issues/436

## Acheivements
Improve aws lambda deployment time from ~10/15 secs to less that 1 sec

## Done
- migrate with the new code executor architecture for local and lambda
drivers
- support old and new executor architecture to avoid breaking changes
- first run is long, next runs are quick even if code step is updated

## Demo using `lambda` driver
### Before


https://github.com/user-attachments/assets/7f7664b4-658f-4689-8949-ea2c31131252


### After



https://github.com/user-attachments/assets/d486c8e2-f8f8-4dbd-a801-c9901e440b29
2025-02-20 10:49:57 +01:00
934a97df51 Fix custom exceptions not propagating error code (#10210)
## Context
In some CustomException exceptions, we were instantiating a code without
initializing it which was overriding the parent code and it was then
lost when retrieving it in filters.
Removing them to make sure we don't reproduce this pattern
2025-02-14 10:27:47 +01:00
b66289c44c replace log by throw (#10167)
as title
2025-02-12 16:31:41 +00:00
7fd89678b7 [CHORE] Avoid isDefined duplicated reference, move it to twenty-shared (#9967)
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function

## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place

Related PR https://github.com/twentyhq/twenty/pull/9941
2025-02-01 12:10:10 +01:00
ae62789159 Serverless function follow up (#9924)
- remove asynchronous serverless function build
- build serverless function synchronously instead on activate workflow
or execute
- add a loader on workflow code step test tab test button
- add a new `ServerlessFunctionSyncStatus` `BUILDING`
- add a new route to build a serverless function draft version 
- delay artificially execution to avoid UI flashing



https://github.com/user-attachments/assets/8d958d9a-ef41-4261-999e-6ea374191e33
2025-01-31 17:12:42 +01:00
af8d22ee99 Fix ObjectType casing and conflict between Relation and RelationMetadata (#9849)
Fixes #9827 

Also uncovered a conflict with `@objectType('Relation')` and
`@objectType('relation)`

I don't want to address it in this PR so I will create a followup issue
when we close this but I think there's a confusion between
Relation/RelationMetadata, it's unclear what is what

---------

Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
2025-01-28 10:06:18 +01:00
ed7c48e12a Fix use as draft (#9718)
- remove delete serverless function when archiving workflow version
- update copy serverless function to reset serverless function to old
version
- remove createNewWorkflowVersion and use createDraftFromWorkflowVersion
- fix step update issue and optimistic rendering when generate draft
from active version
2025-01-21 15:44:52 +01:00
f8f9bb2b78 Serverless function timeout concerns (#9689)
closes https://github.com/twentyhq/core-team-issues/issues/242
- unify timeout behavior between local and lambda
- add timeout in serverless entity
- set timeout default to 300s (5min)
2025-01-17 13:49:02 +00:00
5783d68d62 Remove serverless functions on version archivation (#9535)
Fixes https://github.com/twentyhq/core-team-issues/issues/52
- contrary to title, we do not remove serverless functions on workflow
version archivation because serverless fucntion might be used in another
workflow version
- we fix the serverless funciton version displayed in the code step
- we allow test function version in step display right drawer
- we delete serverless function only when serverless function has no
published version
2025-01-13 13:09:57 +00:00
0c75b244ba Remove serverless functions on soft delete (#9438)
Delete workflow versions serverless functions when soft delete workflow
2025-01-07 15:51:34 +00:00
17bf2b6173 Move the workflow draft version overriding to the backend (#9328)
- In the `formatFieldMetadataValue` function, allow people to call
TypeORM's `save()` method with unserialized JSON data.
- Create an `overrideWorkflowDraftVersion` mutation that takes a
workflow id and the id of the workflow version to use as the new draft
  - If no draft exists yet, create one
  - If a draft already exists, deactivate its serverless functions
- Duplicate every step. For serverless function steps, it includes
duplicating the functions
  - Save the data of the step in DB
- Call the `overrideWorkflowDraftVersion` mutation in the old workflow
header and in the new Cmd+K actions
- I chose to not update the Apollo cache manually as the information of
the new draft are going to be automatically fetched once the user lands
on the workflow's show page. Note that we redirect the user to this page
after overriding the draft version.
2025-01-06 14:56:09 +01:00
e492efb79e Remove serverless functions from settings (#9105)
closes https://github.com/twentyhq/twenty/issues/8727

@Bonapara wants to keep the code for now so I ended up by setting the
`isFunctionSettingsEnabled` constants to false in the codebase
2024-12-17 17:57:17 +01:00
5dfcc413cf 9024 workflow test serverless function follow up (#9066)
-  Fix Tablist style
- Fix dropdown style (wrong grey background)
- Update dropdown variable when no outputSchema is available 



https://github.com/user-attachments/assets/56698fe8-8dd3-404a-b2b2-f1eca6f5fa28
2024-12-17 10:35:38 +01:00
b10d831371 8726 workflow add a test button in workflow code step (#9016)
- add test button to workflow code step
- add test tab to workflow code step


https://github.com/user-attachments/assets/e180a827-7321-49a2-8026-88490c557da2



![image](https://github.com/user-attachments/assets/cacbd756-de3f-4141-a84c-8e1853f6556b)

![image](https://github.com/user-attachments/assets/ee170d81-8a22-4178-bd6d-11a0e8c73365)
2024-12-13 10:16:29 +00:00
2b3b073570 8725 workflow avoid serverless function autosave errors (#8916)
See issue #8725 
- Build function asynchronously using a job
- prevent useless builds
- run promises simultaneously

Todo:
- fix outputSchema computing
2024-12-06 10:13:12 +00:00
d0ff1ffd5f 8723 workflow add editor in serverless function code step (#8805)
- create a serverless function when creating a new workflow code step
- add code editor in workflow code step
- move workflowVersion steps management from frontend to backend
  - add a custom resolver for workflow-version management
  - fix optimistic rendering on frontend
- fix css
- delete serverless function when deleting workflow code step

TODO
- Don't update serverlessFunction if no code change
- Factorize what can be between crud trigger and crud step
- Publish serverless version when activating workflow
- delete serverless functions when deleting workflow or workflowVersion
- fix optimistic rendering for code updates
- Unify CRUD types

<img width="1279" alt="image"
src="https://github.com/user-attachments/assets/3d97ee9f-4b96-4abc-9d36-5c0280058be4">
2024-12-03 08:41:13 +00:00
39373b4a28 8643 fix sentry error (#8644)
- fixes missing data in event payload when adding a new workspaceMember
- add strong typing to database event emitters
2024-11-21 16:09:36 +00:00
354ee86cb9 8311 serverless function functions can be executed with any input (#8380)
- remove ts-morph
- update inputSchema shape

![image](https://github.com/user-attachments/assets/e62f3fdb-5be8-4666-8172-44f73a1981b9)


https://github.com/user-attachments/assets/913cd305-9e7c-48da-b20f-c974a8ac7cea

## TODO
- have inputTypes to match the inputSchema type (string, number,
boolean, etc...), only string for now
- handle required/optional inputs
- handle case when inputSchema changes, fix data reset when switching
function
2024-11-08 16:15:27 +00:00
be8141ce5e Infer function input in workflow step (#8308)
- add `inputSchema` column in serverless function. This is an array of
parameters, with their name and type
- on serverless function id update, get the `inputSchema` + store empty
settings in step
- from step settings, build the form 

TODO in next PR:
- use field type to decide what kind of form should be printed
- have a strategy to handle object as input



https://github.com/user-attachments/assets/ed96f919-24b5-4baf-a051-31f76f45e575
2024-11-05 14:57:06 +01:00
29bd4e5f2d track serverless functions executions (#7963)
Solves: 
https://github.com/twentyhq/private-issues/issues/74

**TLDR**
When a serverless function is executed, the result is send to tinybird
event data source.

**In order to test:**

1. Set ANALYTICS_ENABLED to true
2. Put your TINYBIRD_INGEST_TOKEN from twenty_event_playground in your
.env file
3.  Don't forget to run the worker
4. Create your serverless function and run it
5. The event should be logged on the event datasource in
twenty_event_playground

**What is the structure of the payload of a serverless function event?**

Here are two examples of the payload:

`{"duration":37,"status":"SUCCESS","functionId":"a9fd87c0-af86-4e17-be3a-a6d3d961678a","functionName":"testingFunction"}`

`

{"duration":34,"status":"ERROR","errorType":"ReferenceError","functionId":"a9fd87c0-af86-4e17-be3a-a6d3d961678a","functionName":"testingFunction"}`

**Possible improvments**

- Change the status(str) to success(bool) 
- Enrich data in the payload
2024-10-22 17:51:51 +02:00
e767f16dbe 7415 serverless functions update environment variables in a dedicated tab in settings functions not a env file (#7939)
![image](https://github.com/user-attachments/assets/0ef9551d-d867-479e-9a76-faee6930bc0a)

![image](https://github.com/user-attachments/assets/a7aac417-4dd8-401f-8d5b-5b72f31710f6)

![image](https://github.com/user-attachments/assets/16c98e52-a2db-4ed3-b5d2-77745b4d2918)

![image](https://github.com/user-attachments/assets/847d23d6-8a58-4d8f-aff1-4f8a81862964)
2024-10-22 14:51:03 +02:00
f338d01b4f Build code introspection service (#7760)
Starting to use ts-morph to retrieve function parameters
2024-10-17 15:08:42 +02:00
62fe1d0e88 6653 serverless functions store and use environment variables in serverless function scripts (#7390)
![image](https://github.com/user-attachments/assets/a15bd4c1-3db4-4466-b748-06bdf3874354)

![image](https://github.com/user-attachments/assets/71242dfb-956b-43ed-9704-87cb0dfbc98d)
2024-10-03 13:56:17 +02:00
523df5398a Optimize metadata queries (#7013)
In this PR:

1. Refactor guards to avoid duplicated queries: WorkspaceAuthGuard and
UserAuthGuard only check for existence of workspace and user in the
request without querying the database
2024-09-13 19:42:22 +02:00
3190f4a87b 6658 workflows add a first twenty piece email sender (#6965) 2024-09-12 11:00:25 +02:00
7e03419c16 Serverless function improvements (#6769)
- add layer for lambda execution
- add layer for local execution
- add package resolve for the monaco editor
- add route to get installed package for serverless functions
- add layer versioning
2024-09-02 15:25:20 +02:00
cd06ae20e8 chore(*): remove unused code (#6781)
The code removed in the PR was flagged as unused by the JetBrains
inspector.

I did a QA on the dev environment but other checks are highly
recommended.

There is one commit by scope to make the review easier.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-29 18:16:50 +02:00
9f69383aa2 Add function execution throttler (#6742)
Add throttler service to limit the number of function execution
2024-08-27 18:56:47 +02:00
873a4c1bd1 Fix serverless save when name empty (#6720)
- fix serverless function error on save when name empty
- remove useless unique constraint on serverless function names
2024-08-23 17:34:31 +02:00
6f9aa1e870 6654 serverless functions add a deploy button disable deploy when autosave (#6715)
- improvements on serverless function behavior (autosave performances,
deploy on execution only)
- add versioning to serverless functions
- add a publish endpoint to create a new version of a serverless
function
  - add deploy and reset to lastVersion button in the settings section:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/2001f8d2-07a4-4f79-84dd-ec74b6f301d3">
2024-08-23 12:06:03 +02:00
4157a67bf8 Fix permissions for serverless functions (#6555)
Fixes #6525

(@martmull fyi it was not related to AWS but linked to the fact that we
recently enforced passing a token to access files)
2024-08-06 15:31:22 +02:00
6b4c79ff0d Add workflow runner (#6458)
- add workflow runner module
- add an endpoint to trigger a workflow via api
- improve error handling for serverless functions

## Testing
- create 2 serverless functions
- create a workflow
- create this workflow Version
```
{
  "type": "MANUAL",
  "input": {"b": "bb"},
  "nextAction": {
    "name": "step_1",
    "displayName": "Code",
    "type": "CODE",
    "valid": true,
    "settings": {
      "serverlessFunctionId": "Serverless function 1 Id",
      "errorHandlingOptions": {
        "retryOnFailure": {
          "value": false
        },
        "continueOnFailure": {
          "value": false
        }
      }
    },
    "nextAction": {
      "name": "step_1",
      "displayName": "Code",
      "type": "CODE",
      "valid": true,
      "settings": {
        "serverlessFunctionId": "Serverless function 1 Id",
        "errorHandlingOptions": {
          "retryOnFailure": {
            "value": false
          },
          "continueOnFailure": {
            "value": false
          }
        }
      },
      "nextAction": {
        "name": "step_1",
        "displayName": "Code",
        "type": "CODE",
        "valid": true,
        "settings": {
          "serverlessFunctionId": "Serverless function 2 Id",
          "errorHandlingOptions": {
            "retryOnFailure": {
              "value": false
            },
            "continueOnFailure": {
              "value": false
            }
          }
        }
      }
    }
  }
}

`
``
- call 
```
mutation Trigger {
  triggerWorkflow(workflowVersionId: "WORKFLOW_VERSION_ID") {
    result
  }
}
```
- try when errors are injected in serverless function
2024-07-31 12:48:33 +02:00
b85ae7e1ac Fix googleApisSetRequestExtraParams (#6455) 2024-07-30 16:08:59 +02:00
fb0fd99a38 Fix error handling in serverless service (#6442)
- narrowing error handling in aws sdk usage
- updating dto to authorize null descriptions
2024-07-29 18:51:57 +02:00
00fea17920 Serverless function UI (#6388)
https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=36235-120877

Did not do the file manager part. A Function is defined using one unique
file at the moment

Feature protected by featureFlag `IS_FUNCTION_SETTINGS_ENABLED`

## Demo


https://github.com/user-attachments/assets/0acb8291-47b4-4521-a6fa-a88b9198609b
2024-07-29 13:03:09 +02:00