Commit Graph

37 Commits

Author SHA1 Message Date
83930551d8 Move workflow versions and steps building to workflow-builder folder (#10523)
We are starting to put too many services in common folder. Version and
step building should be separated into different services and go to the
builder folder. Today builder folder only manage schema.

We should:
- keep services responsible for only one action
- keep modules based on the actual action these provide rather than
having common module

This PR:
- creates a service for workflow version builder
- moves version and step builders to workflow builder folder rather than
commun
- creates separated folders for schema, version and steps

No logic has been added. Only modules created and functions moved.
2025-02-27 10:39:48 +01:00
927b8c717e Poc lambda deployment duration (#10340)
closes https://github.com/twentyhq/core-team-issues/issues/436

## Acheivements
Improve aws lambda deployment time from ~10/15 secs to less that 1 sec

## Done
- migrate with the new code executor architecture for local and lambda
drivers
- support old and new executor architecture to avoid breaking changes
- first run is long, next runs are quick even if code step is updated

## Demo using `lambda` driver
### Before


https://github.com/user-attachments/assets/7f7664b4-658f-4689-8949-ea2c31131252


### After



https://github.com/user-attachments/assets/d486c8e2-f8f8-4dbd-a801-c9901e440b29
2025-02-20 10:49:57 +01:00
934a97df51 Fix custom exceptions not propagating error code (#10210)
## Context
In some CustomException exceptions, we were instantiating a code without
initializing it which was overriding the parent code and it was then
lost when retrieving it in filters.
Removing them to make sure we don't reproduce this pattern
2025-02-14 10:27:47 +01:00
b66289c44c replace log by throw (#10167)
as title
2025-02-12 16:31:41 +00:00
7fd89678b7 [CHORE] Avoid isDefined duplicated reference, move it to twenty-shared (#9967)
# Introduction
Avoid having multiple `isDefined` definition across our pacakges
Also avoid importing `isDefined` from `twenty-ui` which exposes a huge
barrel for a such little util function

## In a nutshell
Removed own `isDefined.ts` definition from `twenty-ui` `twenty-front`
and `twenty-server` to move it to `twenty-shared`.
Updated imports for each packages, and added explicit dependencies to
`twenty-shared` if not already in place

Related PR https://github.com/twentyhq/twenty/pull/9941
2025-02-01 12:10:10 +01:00
ae62789159 Serverless function follow up (#9924)
- remove asynchronous serverless function build
- build serverless function synchronously instead on activate workflow
or execute
- add a loader on workflow code step test tab test button
- add a new `ServerlessFunctionSyncStatus` `BUILDING`
- add a new route to build a serverless function draft version 
- delay artificially execution to avoid UI flashing



https://github.com/user-attachments/assets/8d958d9a-ef41-4261-999e-6ea374191e33
2025-01-31 17:12:42 +01:00
af8d22ee99 Fix ObjectType casing and conflict between Relation and RelationMetadata (#9849)
Fixes #9827 

Also uncovered a conflict with `@objectType('Relation')` and
`@objectType('relation)`

I don't want to address it in this PR so I will create a followup issue
when we close this but I think there's a confusion between
Relation/RelationMetadata, it's unclear what is what

---------

Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
2025-01-28 10:06:18 +01:00
ed7c48e12a Fix use as draft (#9718)
- remove delete serverless function when archiving workflow version
- update copy serverless function to reset serverless function to old
version
- remove createNewWorkflowVersion and use createDraftFromWorkflowVersion
- fix step update issue and optimistic rendering when generate draft
from active version
2025-01-21 15:44:52 +01:00
f8f9bb2b78 Serverless function timeout concerns (#9689)
closes https://github.com/twentyhq/core-team-issues/issues/242
- unify timeout behavior between local and lambda
- add timeout in serverless entity
- set timeout default to 300s (5min)
2025-01-17 13:49:02 +00:00
5783d68d62 Remove serverless functions on version archivation (#9535)
Fixes https://github.com/twentyhq/core-team-issues/issues/52
- contrary to title, we do not remove serverless functions on workflow
version archivation because serverless fucntion might be used in another
workflow version
- we fix the serverless funciton version displayed in the code step
- we allow test function version in step display right drawer
- we delete serverless function only when serverless function has no
published version
2025-01-13 13:09:57 +00:00
0c75b244ba Remove serverless functions on soft delete (#9438)
Delete workflow versions serverless functions when soft delete workflow
2025-01-07 15:51:34 +00:00
17bf2b6173 Move the workflow draft version overriding to the backend (#9328)
- In the `formatFieldMetadataValue` function, allow people to call
TypeORM's `save()` method with unserialized JSON data.
- Create an `overrideWorkflowDraftVersion` mutation that takes a
workflow id and the id of the workflow version to use as the new draft
  - If no draft exists yet, create one
  - If a draft already exists, deactivate its serverless functions
- Duplicate every step. For serverless function steps, it includes
duplicating the functions
  - Save the data of the step in DB
- Call the `overrideWorkflowDraftVersion` mutation in the old workflow
header and in the new Cmd+K actions
- I chose to not update the Apollo cache manually as the information of
the new draft are going to be automatically fetched once the user lands
on the workflow's show page. Note that we redirect the user to this page
after overriding the draft version.
2025-01-06 14:56:09 +01:00
e492efb79e Remove serverless functions from settings (#9105)
closes https://github.com/twentyhq/twenty/issues/8727

@Bonapara wants to keep the code for now so I ended up by setting the
`isFunctionSettingsEnabled` constants to false in the codebase
2024-12-17 17:57:17 +01:00
5dfcc413cf 9024 workflow test serverless function follow up (#9066)
-  Fix Tablist style
- Fix dropdown style (wrong grey background)
- Update dropdown variable when no outputSchema is available 



https://github.com/user-attachments/assets/56698fe8-8dd3-404a-b2b2-f1eca6f5fa28
2024-12-17 10:35:38 +01:00
b10d831371 8726 workflow add a test button in workflow code step (#9016)
- add test button to workflow code step
- add test tab to workflow code step


https://github.com/user-attachments/assets/e180a827-7321-49a2-8026-88490c557da2



![image](https://github.com/user-attachments/assets/cacbd756-de3f-4141-a84c-8e1853f6556b)

![image](https://github.com/user-attachments/assets/ee170d81-8a22-4178-bd6d-11a0e8c73365)
2024-12-13 10:16:29 +00:00
2b3b073570 8725 workflow avoid serverless function autosave errors (#8916)
See issue #8725 
- Build function asynchronously using a job
- prevent useless builds
- run promises simultaneously

Todo:
- fix outputSchema computing
2024-12-06 10:13:12 +00:00
d0ff1ffd5f 8723 workflow add editor in serverless function code step (#8805)
- create a serverless function when creating a new workflow code step
- add code editor in workflow code step
- move workflowVersion steps management from frontend to backend
  - add a custom resolver for workflow-version management
  - fix optimistic rendering on frontend
- fix css
- delete serverless function when deleting workflow code step

TODO
- Don't update serverlessFunction if no code change
- Factorize what can be between crud trigger and crud step
- Publish serverless version when activating workflow
- delete serverless functions when deleting workflow or workflowVersion
- fix optimistic rendering for code updates
- Unify CRUD types

<img width="1279" alt="image"
src="https://github.com/user-attachments/assets/3d97ee9f-4b96-4abc-9d36-5c0280058be4">
2024-12-03 08:41:13 +00:00
39373b4a28 8643 fix sentry error (#8644)
- fixes missing data in event payload when adding a new workspaceMember
- add strong typing to database event emitters
2024-11-21 16:09:36 +00:00
354ee86cb9 8311 serverless function functions can be executed with any input (#8380)
- remove ts-morph
- update inputSchema shape

![image](https://github.com/user-attachments/assets/e62f3fdb-5be8-4666-8172-44f73a1981b9)


https://github.com/user-attachments/assets/913cd305-9e7c-48da-b20f-c974a8ac7cea

## TODO
- have inputTypes to match the inputSchema type (string, number,
boolean, etc...), only string for now
- handle required/optional inputs
- handle case when inputSchema changes, fix data reset when switching
function
2024-11-08 16:15:27 +00:00
be8141ce5e Infer function input in workflow step (#8308)
- add `inputSchema` column in serverless function. This is an array of
parameters, with their name and type
- on serverless function id update, get the `inputSchema` + store empty
settings in step
- from step settings, build the form 

TODO in next PR:
- use field type to decide what kind of form should be printed
- have a strategy to handle object as input



https://github.com/user-attachments/assets/ed96f919-24b5-4baf-a051-31f76f45e575
2024-11-05 14:57:06 +01:00
29bd4e5f2d track serverless functions executions (#7963)
Solves: 
https://github.com/twentyhq/private-issues/issues/74

**TLDR**
When a serverless function is executed, the result is send to tinybird
event data source.

**In order to test:**

1. Set ANALYTICS_ENABLED to true
2. Put your TINYBIRD_INGEST_TOKEN from twenty_event_playground in your
.env file
3.  Don't forget to run the worker
4. Create your serverless function and run it
5. The event should be logged on the event datasource in
twenty_event_playground

**What is the structure of the payload of a serverless function event?**

Here are two examples of the payload:

`{"duration":37,"status":"SUCCESS","functionId":"a9fd87c0-af86-4e17-be3a-a6d3d961678a","functionName":"testingFunction"}`

`

{"duration":34,"status":"ERROR","errorType":"ReferenceError","functionId":"a9fd87c0-af86-4e17-be3a-a6d3d961678a","functionName":"testingFunction"}`

**Possible improvments**

- Change the status(str) to success(bool) 
- Enrich data in the payload
2024-10-22 17:51:51 +02:00
e767f16dbe 7415 serverless functions update environment variables in a dedicated tab in settings functions not a env file (#7939)
![image](https://github.com/user-attachments/assets/0ef9551d-d867-479e-9a76-faee6930bc0a)

![image](https://github.com/user-attachments/assets/a7aac417-4dd8-401f-8d5b-5b72f31710f6)

![image](https://github.com/user-attachments/assets/16c98e52-a2db-4ed3-b5d2-77745b4d2918)

![image](https://github.com/user-attachments/assets/847d23d6-8a58-4d8f-aff1-4f8a81862964)
2024-10-22 14:51:03 +02:00
f338d01b4f Build code introspection service (#7760)
Starting to use ts-morph to retrieve function parameters
2024-10-17 15:08:42 +02:00
62fe1d0e88 6653 serverless functions store and use environment variables in serverless function scripts (#7390)
![image](https://github.com/user-attachments/assets/a15bd4c1-3db4-4466-b748-06bdf3874354)

![image](https://github.com/user-attachments/assets/71242dfb-956b-43ed-9704-87cb0dfbc98d)
2024-10-03 13:56:17 +02:00
523df5398a Optimize metadata queries (#7013)
In this PR:

1. Refactor guards to avoid duplicated queries: WorkspaceAuthGuard and
UserAuthGuard only check for existence of workspace and user in the
request without querying the database
2024-09-13 19:42:22 +02:00
3190f4a87b 6658 workflows add a first twenty piece email sender (#6965) 2024-09-12 11:00:25 +02:00
7e03419c16 Serverless function improvements (#6769)
- add layer for lambda execution
- add layer for local execution
- add package resolve for the monaco editor
- add route to get installed package for serverless functions
- add layer versioning
2024-09-02 15:25:20 +02:00
cd06ae20e8 chore(*): remove unused code (#6781)
The code removed in the PR was flagged as unused by the JetBrains
inspector.

I did a QA on the dev environment but other checks are highly
recommended.

There is one commit by scope to make the review easier.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-08-29 18:16:50 +02:00
9f69383aa2 Add function execution throttler (#6742)
Add throttler service to limit the number of function execution
2024-08-27 18:56:47 +02:00
873a4c1bd1 Fix serverless save when name empty (#6720)
- fix serverless function error on save when name empty
- remove useless unique constraint on serverless function names
2024-08-23 17:34:31 +02:00
6f9aa1e870 6654 serverless functions add a deploy button disable deploy when autosave (#6715)
- improvements on serverless function behavior (autosave performances,
deploy on execution only)
- add versioning to serverless functions
- add a publish endpoint to create a new version of a serverless
function
  - add deploy and reset to lastVersion button in the settings section:
<img width="736" alt="image"
src="https://github.com/user-attachments/assets/2001f8d2-07a4-4f79-84dd-ec74b6f301d3">
2024-08-23 12:06:03 +02:00
4157a67bf8 Fix permissions for serverless functions (#6555)
Fixes #6525

(@martmull fyi it was not related to AWS but linked to the fact that we
recently enforced passing a token to access files)
2024-08-06 15:31:22 +02:00
6b4c79ff0d Add workflow runner (#6458)
- add workflow runner module
- add an endpoint to trigger a workflow via api
- improve error handling for serverless functions

## Testing
- create 2 serverless functions
- create a workflow
- create this workflow Version
```
{
  "type": "MANUAL",
  "input": {"b": "bb"},
  "nextAction": {
    "name": "step_1",
    "displayName": "Code",
    "type": "CODE",
    "valid": true,
    "settings": {
      "serverlessFunctionId": "Serverless function 1 Id",
      "errorHandlingOptions": {
        "retryOnFailure": {
          "value": false
        },
        "continueOnFailure": {
          "value": false
        }
      }
    },
    "nextAction": {
      "name": "step_1",
      "displayName": "Code",
      "type": "CODE",
      "valid": true,
      "settings": {
        "serverlessFunctionId": "Serverless function 1 Id",
        "errorHandlingOptions": {
          "retryOnFailure": {
            "value": false
          },
          "continueOnFailure": {
            "value": false
          }
        }
      },
      "nextAction": {
        "name": "step_1",
        "displayName": "Code",
        "type": "CODE",
        "valid": true,
        "settings": {
          "serverlessFunctionId": "Serverless function 2 Id",
          "errorHandlingOptions": {
            "retryOnFailure": {
              "value": false
            },
            "continueOnFailure": {
              "value": false
            }
          }
        }
      }
    }
  }
}

`
``
- call 
```
mutation Trigger {
  triggerWorkflow(workflowVersionId: "WORKFLOW_VERSION_ID") {
    result
  }
}
```
- try when errors are injected in serverless function
2024-07-31 12:48:33 +02:00
b85ae7e1ac Fix googleApisSetRequestExtraParams (#6455) 2024-07-30 16:08:59 +02:00
fb0fd99a38 Fix error handling in serverless service (#6442)
- narrowing error handling in aws sdk usage
- updating dto to authorize null descriptions
2024-07-29 18:51:57 +02:00
00fea17920 Serverless function UI (#6388)
https://www.figma.com/design/xt8O9mFeLl46C5InWwoMrN/Twenty?node-id=36235-120877

Did not do the file manager part. A Function is defined using one unique
file at the moment

Feature protected by featureFlag `IS_FUNCTION_SETTINGS_ENABLED`

## Demo


https://github.com/user-attachments/assets/0acb8291-47b4-4521-a6fa-a88b9198609b
2024-07-29 13:03:09 +02:00
47ddc7be83 6181 workflows create a custom code executor (#6235)
Closes #6181

## Testing
- download Altair graphql dev tool https://altairgraphql.dev/#download
- create a file locally `test.ts` containing:
```
export const handler = async (event: object, context: object) => {
  return { test: 'toto', data: event['data'] };
}
```
- play those requests in Altair:
mutation UpsertFunction($file: Upload!) {
  upsertFunction(name: "toto", file: $file)
}

mutation ExecFunction {
  executeFunction(name:"toto", payload: {data: "titi"})
}
- it will run the local driver, add those env variable to test with
lambda driver
```
CUSTOM_CODE_ENGINE_DRIVER_TYPE=lambda
LAMBDA_REGION=eu-west-2
LAMBDA_ROLE=<ASK_ME>
```
2024-07-17 17:53:01 +02:00