Commit Graph

1099 Commits

Author SHA1 Message Date
736635a94b Begin moving to postgres spilo + adding pgvector (#8309)
We will remove the `twenty-postgres` image that was used for local
development and only use `twenty-postgres-pilo` (which we use in prod),
bringing the development environment closer to prod and avoiding having
to maintain 2 images.


Instead of provisioning the super user after the db initialization, we
directly rely on the superuser provided by Spilo for simplicity. We also
introduce a change that tries to create the right database (`default` or
`test`) based on the context.
  

How to test:
```
docker build -t twentycrm/twenty-postgres-spilo:latest -f ./packages/twenty-docker/twenty-postgres-spilo/Dockerfile .
docker images --no-trunc | grep twenty-postgres-spilo
postgres-on-docker:
	docker run \
	--name twenty_pg \
	-e PGUSER_SUPERUSER=twenty \
	-e PGPASSWORD_SUPERUSER=twenty \
	-e ALLOW_NOSSL=true \
	-v twenty_db_data:/home/postgres/pgdata \
	-p 5432:5432 \
	REPLACE_WITH_IMAGE_ID
```
2024-11-15 09:38:30 +01:00
cfe3515aa6 Set cache metadata version to infinite TTL (#8507)
## Context
To avoid having a corrupt metadata version, we want to remove TTL for
that key.

## Test
<img width="592" alt="Screenshot 2024-11-15 at 00 02 05"
src="https://github.com/user-attachments/assets/9da0ae33-26a8-4e7b-82d0-dd691135a08f">
2024-11-15 00:05:42 +01:00
a799370483 Aggregated queries #1 (#8345)
First step of https://github.com/twentyhq/twenty/issues/6868

Adds min.., max.. queries for DATETIME fields
adds min.., max.., avg.., sum.. queries for NUMBER fields 

(count distinct operation and composite fields such as CURRENCY handling
will be dealt with in a future PR)

<img width="1422" alt="Capture d’écran 2024-11-06 à 15 48 46"
src="https://github.com/user-attachments/assets/4bcdece0-ad3e-4536-9720-fe4044a36719">

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Weiko <corentin@twenty.com>
2024-11-14 18:05:05 +01:00
51c54d4c5b validation on Select field (#8316)
fix #8204
I changed "API keys" to "API values".
Stopped inputting special characters in Select field option keys.

@lucasbordeau please check the changes and tell me if I need to do any
other changes. :)

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2024-11-14 17:46:18 +01:00
15b8b9b158 Feature : Adding percentage option to Input Number (#8481)
fixing #7375

---------

Co-authored-by: guillim <guillaume@twenty.com>
2024-11-14 14:23:25 +01:00
090f612c4b Fix open api specs for MULTI_SELECT (#8494)
## Before

![image](https://github.com/user-attachments/assets/b7384fb3-0f32-498b-99bb-27b8c15db00c)

## After

![image](https://github.com/user-attachments/assets/a8429849-559d-4494-8636-e5a6ddda7b47)
2024-11-14 12:07:31 +01:00
d72068eb99 Fix BullMQ deprecation warning (#8486)
Related to #8470
2024-11-13 19:38:29 +01:00
2fb920f68c Fix typo in branch name (#8480)
Fixes #8471 

Updated `0.33.0-canaray` -> `0.33.0-canary`

![CleanShot 2024-11-13 at 09 45
36](https://github.com/user-attachments/assets/6a365542-9b4d-4279-9ef4-966d22863eba)
2024-11-13 19:35:15 +01:00
cde96cf526 Fix/validate access token user not found (#8484)
# Description
Closes #7244 

See details about implementation:
https://github.com/twentyhq/twenty/issues/7244#issuecomment-2473845859
and
https://github.com/twentyhq/twenty/issues/7244#issuecomment-2473905514

# Changes
- return a `USER_NOT_FOUND` error instead of `INVALID_INPUT` error
- tweak unit tests to correctly test `AuthExceptionCode`, as it wasn't
properly tested; it was actually a _false positive_. This is because
[`toThrow`](https://jestjs.io/docs/expect#tothrowerror) from jest only
checks the `message`, and not any other method / attributes from the
`Error`. It's a know behaviour and not considered a bug, see
https://github.com/jestjs/jest/issues/13232#issuecomment-1252392845
2024-11-13 18:50:30 +01:00
faeea2b887 Create record action (#8460)
- Add create record action
- Migrate all step name => action in a common folder
- Separate types
2024-11-13 15:21:40 +01:00
269eaf4422 Update zapier trigger payload (#8464)
- fixes zapier tests
2024-11-12 17:58:36 +01:00
aadcb49dcb Fix missing timeline activity events (#8459)
Fixes bug introduced in https://github.com/twentyhq/twenty/pull/8193

Taking into account linked event name `linked-{eventName}` as before
issue

## Before
`linked-created` and `linked-updated` activity targets were not created
in `timelineActivity` table

## After
`linked-created` and `linked-updated` activity targets are created in
`timelineActivity` table
2024-11-12 12:04:58 +01:00
9d6a850ee8 Mark main version as 0.33 canary (#8453) 2024-11-11 11:59:33 +01:00
354ee86cb9 8311 serverless function functions can be executed with any input (#8380)
- remove ts-morph
- update inputSchema shape

![image](https://github.com/user-attachments/assets/e62f3fdb-5be8-4666-8172-44f73a1981b9)


https://github.com/user-attachments/assets/913cd305-9e7c-48da-b20f-c974a8ac7cea

## TODO
- have inputTypes to match the inputSchema type (string, number,
boolean, etc...), only string for now
- handle required/optional inputs
- handle case when inputSchema changes, fix data reset when switching
function
2024-11-08 16:15:27 +00:00
0bf2cb69da Fix multi-dropdown field can't have no options selected (#8391)
Fixes: #8224 

1. Summary
All multi-dropdowns across the application don't accept empty options
because emptiness is perceived as an invalid state.

2. Solution
The issue came down to the back-end and the specific string utility
function that converts a string separated by `,` to an array of options.
But the problem with it is that it returns `['']` for an empty string
representing an emptiness. And then `['']` fails on the parser because
simply `''` is not a valid option the user selected for the dropdown. So
I updated the string utility function to return an empty array for cases
like `'', '{}', '{ }'`, etc

3. Recording


https://github.com/user-attachments/assets/071fe5d2-2123-4deb-878c-67f62d9b3431
2024-11-08 16:10:14 +01:00
8b5e90aca9 Update searchVector expression based on rich text fields (#8390)
Search vector fields based on `RICH_TEXT` fields were generated using a
treatment on `RICH_TEXT` fields's `body`
column, to only extract and index the core text.
([PR](https://github.com/twentyhq/twenty/pull/7953))

Actually our RICH_TEXT fields are of datatype `text` in our database,
allowing users to insert non-json values, which breaks the search vector
generation (as it expects json values).

Our vision is unclear for now: for instance we may want to turn
RICH_TEXT into a composite field where the plain text would be stored in
a different column.

So for now, we will (1) treat rich_text data as text, and (2) update the
search vector expressions for the existing workspaces.
2024-11-08 12:41:11 +01:00
813c57fba6 Create workflow setup command (#8406)
Steps to enable workflows:
- enable feature flags
- run metadata sync
- run this command

If the feature flag is not true, the command will fail. Which will be
useful to prevent seeding a wrong workspace.

Including a little fix for send email aciton error
2024-11-08 12:40:11 +01:00
f06cdbdfc6 refactor webhookAnalytics call and enrich analytics module (#8253)
**TLDR**

Refactor WebhoonAnalytics Graph to a more abstract version
AnalyticsGraph (in analytics module). Thus enabling the components to be
used on different instances (ex: new endpoint, new kind of graph).

**In order to test:**

1. Set ANALYTICS_ENABLED to true
2. Set TINYBIRD_JWT_TOKEN to the ADMIN token from the workspace
twenty_analytics_playground
3. Set TINYBIRD_JWT_TOKEN to the datasource or your admin token from the
workspace twenty_analytics_playground
4. Create a Webhook in twenty and set wich events it needs to track
5. Run twenty-worker in order to make the webhooks work.
6. Do your tasks in order to populate the data
7. Enter to settings> webhook>your webhook and the statistics section
should be displayed.

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2024-11-08 10:00:51 +01:00
f9c076df31 o365 calendar sync (#8044)
Implemented:

* Account Connect
* Calendar sync via delta ids then requesting single events


I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)

---------

Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
2024-11-07 18:13:22 +01:00
f9a136ab6d Update CanAccessMessageThreadService (#8388)
Trying to fix #7830
2024-11-07 17:32:41 +01:00
fb22efd9e9 Fix CalendarOngoingStaleJob being in wrong queue (#8385)
Fix CalendarOngoingStaleJob being in the wrong queue
(`MessageQueue.messagingQueue` instead of `MessageQueue.calendarQueue`)
2024-11-07 17:29:00 +01:00
d9c0530dd3 Integration test : Sync mode + db reset option + cleaning (#8376)
Run the CI integrationin sync mode
and add the option to run it without db reset
cleaning all the useless integration test

---------

Co-authored-by: guillim <guillaume@twenty.com>
2024-11-07 17:22:47 +01:00
7bab65b569 Implement object fields and settings new layout (#7979)
### Description

- This PR has as the base branch the TWNTY-5491 branch, but we also had
to include updates from the main branch, and currently, there are
conflicts in the TWNTY-5491, that cause errors on typescript in this PR,
so, we can update once the conflicts are resolved on the base branch,
but the functionality can be reviewed anyway
- We Implemented a new layout of object details settings and new, the
data is auto-saved in `Settings `tab of object detail
- There is no indication to the user that data are saved automatically
in the design, currently we are disabling the form

### Demo\

<https://www.loom.com/share/4198c0aa54b5450780a570ceee574838?sid=b4ef0a42-2d41-435f-9f5f-1b16816939f7>

### Refs

#TWNTY-5491

---------

Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: gitstart-twenty <140154534+gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
2024-11-07 14:50:53 +01:00
f3a38679ae Add try catch to command (#8381)
fixes 0-32 command
2024-11-07 11:27:33 +01:00
117e502e80 Clean available fields in variable dropdown (#8371)
As title

We do not want relations yet in picker
2024-11-06 16:54:23 +01:00
278ab4c513 Fix loginToken and legacy apiToken not working (#8369) 2024-11-06 15:56:14 +01:00
4b5d096441 Fix of broken API Auth (#8338)
Fix done this morning with @FelixMalfait  from #8295

---------

Co-authored-by: guillim <guillaume@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
2024-11-06 14:45:33 +01:00
ac7d740135 fix: when field metadata SELECT type is edited update view groups (#8344)
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-11-06 11:41:44 +01:00
be8141ce5e Infer function input in workflow step (#8308)
- add `inputSchema` column in serverless function. This is an array of
parameters, with their name and type
- on serverless function id update, get the `inputSchema` + store empty
settings in step
- from step settings, build the form 

TODO in next PR:
- use field type to decide what kind of form should be printed
- have a strategy to handle object as input



https://github.com/user-attachments/assets/ed96f919-24b5-4baf-a051-31f76f45e575
2024-11-05 14:57:06 +01:00
88d6f5e59e Bump to 0.32 (#8324) 2024-11-05 00:19:05 +01:00
dbd4292b6a Add backfill view groups command 2024-11-04 23:59:34 +01:00
4fc690b42c Fix 0.32 (#8319)
- Removing unique constraint creation in 0.32 as we have a dependency on
sync-metadata and sync-metadata has a dependency on it if we keep it.
- sync-metadata does not support well the deletion of activity +
activityTarget, silently swallow the exception for now
2024-11-04 22:08:07 +01:00
695991881f 6071 return only updated fields of records in zapier update trigger (#8193)
- move webhook triggers into `entity-events-to-db.listener.ts`
- refactor event management
- add a `@OnDatabaseEvent` decorator to manage database events
- add updatedFields in updated events
- update openApi webhooks docs
- update zapier integration
2024-11-04 17:44:36 +01:00
0c6c22b852 Fix typo in default member's last name (#8293)
Fixes #8290

Correct the last name of the default member "Phil Shiler" to "Phil
Schiler" in the
`packages/twenty-server/src/database/typeorm-seeds/workspace/workspace-members.ts`
file.

* Change the last name from "Shiler" to "Schiler" for the user with the
email `phil.schiler@apple.dev`.

---

For more details, open the [Copilot Workspace
session](https://copilot-workspace.githubnext.com/twentyhq/twenty/issues/8290?shareId=13cc0610-8aaf-45f9-9e96-2a723ed60218).
2024-11-03 16:41:25 +01:00
6630fc4a6a Fixed typo in task's status (#8274)
Related to #8100

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2024-11-01 11:33:30 +01:00
99837f9752 Fix integration tests (#8229)
Removing leftovers of the deprecated "activity" object which is fixing
our integration tests
2024-10-31 10:36:29 +01:00
82f33efe9c Add missing CopyWebhookOperationIntoOperationsCommand to 0.32 upgrade global command (#8219)
## Context
This command was missing in the global upgrade command, this PR simply
adds it.
2024-10-31 09:56:44 +01:00
d19bc2c224 Fix SimplifySearchVectorExpressionCommand (#8218)
## Context
This command was introduced to simplify searchVector expressions that
were added in v0.31.x for existing workspaces.
New search vector columns have been added later during 0.32 development
and should not be migrated, this actually breaks the command because we
run sync-metadata before the upgrade command.

The fix removes the throw since this is expected and return early if the
search vector was not matched with an object that needs migration.

## Test
checkout v0.31.0
reset:db command
checkout this PR
run typeorm migrations
run upgrade 0-32 command
2024-10-30 17:47:52 +01:00
692c4ba6fb Add simple configuration to ts_query (#8215)
In the expression of our searchVector fields, we use the "simple"
configuration (over the default "english" one), to avoid picking a
language that's irrelevant to the user.
I initially forgot to add the same configuration to the query that is
being sent using ts_query.
Adding it will also allow the search to work for a single character,
while so far a single letter was most of the time considered a "stop
word" (a word with no semantic value, like "a").
2024-10-30 16:58:51 +01:00
50c912d57f Delete unused objects (#7823)
Fixes #7113

---------

Co-authored-by: Weiko <corentin@twenty.com>
2024-10-30 13:35:43 +01:00
57d9b8e8b4 feat: generate secret function and replaced few instances (#7810)
This PR fixes #4588

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-30 12:07:11 +01:00
1782865ff8 Added feature flag for advanced filter (#8194)
Added feature flag for advanced filter
2024-10-30 12:06:46 +01:00
31ecaf2a0d fix object metadata renaming (#8175)
## Context
Latest refactoring broke the findOneWithinWorkspace method which is
called during object update, I'm simply reverting this change.
object-metadata-relation.service was naively computing a namePlural
based on the nameSingular while we already had that info in the DB...
Should fix some issues with renaming as well because the original field
was not computed with the right name.
2024-10-29 14:45:27 +00:00
8bb07c4a4f 8130 creating a new company in twenty doesnt activate on zapier (#8166)
- fix webhook.operation format change from august 2024 not spread in
twenty-zapier
- added a comment so it does not happen again
- add a fix for the new webhook.operations column that would produce
another issue
2024-10-29 13:56:01 +01:00
2e10070fdb fix(sso): improve enterprise key var management (#8152)
Resolve https://github.com/twentyhq/twenty/issues/8070

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
2024-10-29 11:39:00 +01:00
88de2872d8 Don't require host for postgres (twentyhq#8157) (#8159)
Not sure if this will pass the tests; I don't have TS experience so I'll
see what CI does with it.

Fixes #8157
2024-10-29 08:46:54 +01:00
409def8431 Fix manual trigger output schema (#8150)
- add schema for manual trigger
- split into sub functions
- handle case with no variables
2024-10-28 17:42:09 +00:00
ad1e835150 Remove broken e2e tests 2024-10-28 14:48:14 +01:00
f202cdb4cb [Server Integration tests] Enrich integration GraphQL API tests #4 (#8015)
This PR was created by [GitStart](https://gitstart.com/) to address the
requirements from this ticket:
[TWNTY-7526](https://clients.gitstart.com/twenty/5449/tickets/TWNTY-7526).

 --- 

### Description

We are using standard field and object IDs because there is a problem
when creating them as we would with helpers.

We were creating custom objects and removing them in afterAll, but there
is a problem that causes a conflict in the object IDs if the tests run
at the same time. The solution would be to modify the test command as
mentioned here:

<https://github.com/typeorm/typeorm/issues/6125>

Currently, we are using the standard IDs but we can create the objects
within the test if you want, but we would need to make them run in
sequence

### Demo


![](https://assets-service.gitstart.com/16336/70c5550e-8570-45d7-a613-a3d6b79e7c00.png)

### Refs

#7526

---------

Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-28 14:24:24 +01:00
28815f54fe Refactor object metadata service (#8123)
## Context
ObjectMetadataService became quite large and handles too many
responsibilities. I'm trying to refactor a bit this part in preparation
of a larger work that will combine object-metadata services and
sync-metadata logic

- Created a STANDARD_OBJECT_ICONS that can be reused in relation
creation to refer to a standard object icon.
- Created a STANDARD_OBJECT_FIELD_IDS that can be used with an object
name to get its standard field ids.
- Moved migration, record and relation creations to dedicated services,
refactored to improve performances and readability
- Refactored some validation logic

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2024-10-28 14:13:15 +01:00