Commit Graph

357 Commits

Author SHA1 Message Date
293e3f58c4 Fix TEST env not using the correct DB for datasources (#10780)
## Context
Config was programmatically loaded in our datasources however the
default behavior of dotenv is to ignore vars if they are already
defined. This means we need to be careful about the order of env
injection and sometimes it's done at a higher level (for example
db:reset will depend on build). To make things easier I'm using the
override flag to properly override the PG_DATABASE_URL if different (and
to properly work with the 'test' DB instead of 'default' during
testing).
2025-03-11 16:20:01 +01:00
4be75fb7da eliminate IS_BILLING_PLANS_ENABLED feature flag (#10678)
Solves: https://github.com/twentyhq/core-team-issues/issues/527

**TLDR:**
Basically the title. Fetches the product and prices from the database
instead of the environment variables.

**What this means:**

- new subscriptions in twenty will be hybrid (per seat subscription plus
an usage base product)
- right now the price for the usage base product  is 0$ per unit
- The existing subscription will work normally, however we will need to
update their subscription items in order to contain the usage base
product (remember that the pricing intervals like monthly or yearly
should match in all the subscription items)
-  The previous point can be done using Stripe Postman


**In order to test:**

- Have the environment variable IS_BILLING_ENABLED set to true and add
the other required environment variables for Billing to work
- Do a database reset (to ensure that the new feature flag is deleted
and that the billing tables are created)
- Run the command: npx nx run twenty-server:command
billing:sync-plans-data (if you don't do that the products and prices
will not be present in the database)
- Run the server , the frontend, the worker, and the stripe listen
command (stripe listen --forward-to
http://localhost:3000/billing/webhooks)
- Buy a subscription for acme workspace
- Update the quantity of members in a workspace (add or delete)
- Change the subscription interval

---------

Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
2025-03-07 11:46:22 +01:00
f4fcf39eb5 [permissions] Prepare for roll-out (#10676)
Closes https://github.com/twentyhq/core-team-issues/issues/469 and
https://github.com/twentyhq/core-team-issues/issues/500

In this PR
1. stop conditioning permission initialization for a workspace to env
variable value. Instead we want to create and assign permissions and
roles in all new workspaces. For now that will be totally silent.
2. temporarily, the default role is set to the admin role for new
workspaces. it will also be the case for existing workspaces through the
backfill command. Member role is still being created though. (when we
will do the final roll-out we will update this so that future workspaces
have the member role as default role. our goal here is not to break any
current behaviour for users, that today have all have the equivalent of
admin rights).
2025-03-05 16:47:41 +01:00
ad628c1266 [ENH] Do not fail on missed cache within server commands (#10634)
Avoid critical failure if cache is missed when interacting with the
twenty-orm in the upgrade commands
2025-03-04 10:28:21 +01:00
b8d944bd6e fix try catch logic in upgrade command (#10616) 2025-03-03 16:37:39 +01:00
d19144208b Improve migration runner performances (#10572)
## Context
Workspace creation and more specifically sync-metadata performances are
bad at the moment. We are trying to identify bottlenecks and one of the
root causes is the migration runner that can take up to 10s when setting
up a new workspaces with all its tables.
First observation is we do a lot of things sequentially, mostly to make
the code easier to read and debug but it impacts performances. For
example, a table creation is done in two steps, we first ask typeorm to
create the table then ask typeorm to create columns (and sometimes
columns one by one), each instruction can take time because typeorm
seems to do some checks internally.

The proposition here is to try to merge migrations when possible, for
example when we create a table we want the migration to also contain the
columns it will contain so we can ask typeorm to add the columns at the
same time. We are also using batch operations when possible (addColumns
instead of addColumn, dropColumns instead of dropColumn)
Still, we could go further with foreign keys creations or/and try with
raw query directly.

## Test
New workspace creation:
See screenshot, 9865.40233296156ms is on main, the rest is after the
changes:

<img width="610" alt="Screenshot 2025-02-28 at 09 27 21"
src="https://github.com/user-attachments/assets/42e880ff-279e-4170-b705-009e4b72045c"
/>

ResetDB and Sync-metadata on an existing workspace commands still work
2025-02-28 19:51:45 +01:00
baa3043954 Refactor upgrade commands (#10592)
Simplifying a lot the upgrade system.

New way to upgrade:
`yarn command:prod upgrade`

New way to write upgrade commands (all wrapping is done for you)
```
  override async runOnWorkspace({
    index,
    total,
    workspaceId,
    options,
  }: RunOnWorkspaceArgs): Promise<void> {}
```

Also cleaning CommandModule imports to make it lighter
2025-02-28 19:51:32 +01:00
00b650a121 Fix export view missing column and viewGroups not persisted bug (#10584)
Fixes https://github.com/twentyhq/twenty/issues/10535
Also fixes missing columns in export view
2025-02-28 15:59:13 +01:00
122a6a7801 [permissions] Backfill command to prepare workspaces (#10581)
Closes https://github.com/twentyhq/core-team-issues/issues/317

---------

Co-authored-by: Weiko <corentin@twenty.com>
2025-02-28 15:46:51 +01:00
fba63d9cb7 migrate rich text v1 workspace + move relation migration to 0.44 (#10582)
Adapt from MigrateRichTextFieldCommand
2025-02-28 14:46:34 +01:00
2aa72de210 remove isRichTextV2Enabled feature flag (#10562) 2025-02-28 14:34:06 +01:00
0dc1cd9df1 [REG/FEAT] Sync metadata after upgrade (#10557)
# Introduction
Historically we've been programmatically running sync metadata just
after all upgrade command's migration.
Adding back this behavior as default to the new dynamic modules

Duplicated already existing synchronize metadata logic as a quick fix as
we're about to iterate over commands next sprint
2025-02-27 18:44:41 +01:00
39543872e6 add new @WorkspaceIsSearchable decorator + updates services + add migration command (#10507)
closes https://github.com/twentyhq/core-team-issues/issues/345
2025-02-27 13:57:07 +01:00
0220672fa9 Add default role to workspace (#10444)
## Context
Adding a defaultRole to each workspace, this role will be automatically
added when a member joins a workspace via invite link or public link
(seeds work differently though).
Took the occasion to refactor a bit the frontend components, splitting
them in smaller components for more readability.

## Test
<img width="948" alt="Screenshot 2025-02-24 at 14 54 02"
src="https://github.com/user-attachments/assets/13ef1452-d3c9-4385-940c-2ced0f0b05ef"
/>
2025-02-25 11:26:35 +01:00
a1eea40cf7 feat: populate relation join column (#10212)
Fix
https://github.com/twentyhq/core-team-issues/issues/241#issue-2793030259
2025-02-25 11:24:05 +01:00
dde70ee3b0 Add fields for admin panel access and workspace version (#10451)
Prepare for better version upgrade system + split admin panel into two
permissions + fix GraphQL generation detection

---------

Co-authored-by: ehconitin <nitinkoche03@gmail.com>
2025-02-24 21:38:41 +01:00
1b64f87d75 Add error management to the Billing Package (#10343)
Solves https://github.com/twentyhq/core-team-issues/issues/403

**TLDR:**
Enhance error management in Billing and when a customer is updated it
updates automatically the Stripecustomer id in the entitlements.


- Add Billing exceptions to filter. 
- Add onUpdate for billing customer and entitlement.
- Remember to run the migrations with is BILLING_ENABLED set to true.

**In order to test (a simple test case)**

- Ensure that the environment variables for Sentry and Billing are set,
ensuring that SENTRY_ENVIRONMENT=staging
- Run the server, the worker and the stripe cli
- Do a database reset with IS_BILLING_ENABLED set to true
- Go to stripe in test mode and update a random price description, this
causes an exception because you are trying to write a price of. a
product that doesn't exists in the database
- You should see an error in Sentry:


![image](https://github.com/user-attachments/assets/7b3c8c7f-6628-4a20-9889-a691e7838d79)
2025-02-24 10:10:06 +01:00
bf92860d19 feat(twenty-server): add trusted domain - backend crud (#10290)
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
2025-02-21 16:02:48 +00:00
d747366bf3 Provide a wrapper to execute command on workspace with easier devXP (#10391)
Proposal:
- Add a method in ActiveWorkspaceCommand to loop over workspace safely
(add counter, add try / catch, provide datasource with fresh cache,
destroy datasource => as we do always do it)

Also in this PR:
- make sure we clear all dataSources (and not only the one on metadata
version in RAM)
2025-02-21 16:40:33 +01:00
311fc402d5 Enhance ActiveWorkspaceCommand to order, limit and startFrom workspaceId (#10378)
I have tested it locally with all combinaisons of params
2025-02-21 10:37:04 +01:00
9f454c565b 410 open in side panel (#10363)
Closes https://github.com/twentyhq/core-team-issues/issues/410

- Added `openRecordIn` column in the `view` entity, which is set to
`SIDE_PANEL` by default
- Created a new option inside the view option dropdown to be able to set
`openRecordIn`
- Updated all record show page openings to reflect the setting behavior
- For `workflow`, `workflowVersion` and `workflowRun` (what I call
workflow objects), we want the default view `openRecordIn` to be set to
`RECORD_PAGE`. When seeding the views for the new workspaces, we set
`openRecordIn` to `RECORD_PAGE` for workflow objects. Since the workflow
objects views `openRecordIn` will be set to the default value
`SIDE_PANEL` for the existing workspaces when the sync metadata runs, I
created a script to run in the 0.43 update to update this value.
- Updated `closeCommandMenu` because of problems introduced by the
animate presence wrapper around the command menu. We now reset the
states at the end of the animation.

Note: We want to be able to open all workflow objects pages in the side
panel, but this requires some refactoring of the workflow module. For
now @Bonapara wanted to allow the possibility to change the
`openRecordIn` setting to `SIDE_PANEL` even for the workflows even if
it's buggy and not ready for the moment. Since this is an experimental
feature, it shouldn't cause too many problems.
2025-02-21 09:27:33 +00:00
41bbb4b47f Fix upgrade command 0.42 2025-02-21 00:20:44 +01:00
4037fb5860 Fix upgrade command 0.42 (#10373) 2025-02-20 20:22:27 +01:00
68cb94f365 [BUGFIX] Call and await destroyDataSourceForWorkspace to avoid killing poor pod (#10364)
# Motivations
Upgrade migration is not possible atm in production because running over
all workspaces cause cpu raise leading to pod termination
2025-02-20 16:45:21 +01:00
94c0d0f8d2 [BUG] Refactor actor composite type (#10232)
fixes #10200 

The FieldActor Zod schema was updated to correctly handle null context.

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-02-20 11:05:26 +01:00
422e4e33c0 [BUGFIX][PROD] RICH_TEXT_V2 command handle {} body col value (#10324)
# Introduction
Encountered in issue in production where we have a lot of records that
has RICH_TEXT_FIELD set to `{}`

```sh
[Nest] 20106  - 02/19/2025, 12:43:08 PM     LOG [MigrateRichTextFieldCommand] Generating markdown for 1 records
[Nest] 20106  - 02/19/2025, 12:43:09 PM     LOG [MigrateRichTextFieldCommand] Error in workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db: TypeError: o is not iterable
```

## Fix
While reading `fieldValue` definition also strictly check if it's `{}` +
checking after JSON parse if it's an iterable to pass to the
`serverBlockNoteEditor` in order to be 100 bullet proof for prod
migration command

## Refactor Dry run
Implemented dry run

## Refactor to Idempotency
Made the script idempotent in order to avoid issues with re-running
commands

## Error repro
- In local checkout on v0.41.5 run `yarn && npx nx reset && npx nx
start`
- Create record manually in db that has a RICH_TEXT body to `{}`
- Checkout to main, `yarn && npx nx reset && npx nx build twenty-server
&& yarn command:prod upgrade-0.42:migrate-rich-text-field -d`
2025-02-20 09:16:58 +00:00
222f475cc6 [permissions] Fix db reset leads to broken state if env var permission is not true (#10309) 2025-02-18 17:25:54 +00:00
af8a167fb2 Update searchVector on new richTextV2 note and task entities + migration command (#10303)
closes https://github.com/twentyhq/core-team-issues/issues/343
closes https://github.com/twentyhq/core-team-issues/issues/340
2025-02-18 18:11:29 +01:00
cb3bd1353a [permissions] Add object records permissions to role entity (#10255)
Closes https://github.com/twentyhq/core-team-issues/issues/388

- Add object records-related permissions to role entity
- Add it to queriable `currentUserWorkspace` (used in FE)
2025-02-17 18:32:39 +01:00
5b961cbb7f Tasks assigned to me view (#9567) (#9568)
Issue: https://github.com/twentyhq/core-team-issues/issues/154

View is now added.

TODO: Fix filtering a relation field by 'Me' in Tasks.

---------

Co-authored-by: ad-elias <elias@autodiligence.com>
2025-02-17 10:13:55 +00:00
db1b11742b [0.42][FIX] Seeder activate rich text v2 feature flag (#10229)
as per title
2025-02-14 17:57:32 +00:00
258025a18f Fix upgrade-0-42 command to migrate RICH_TEXT field metadata to RICH_TEXT_V2 (#10218)
While making sure the upgrade-0.42 command was working as expected to
upgrade from 0.41 to 0.42, we've detected that the standardId of the new
bodyV2 (type RICH_TEXT_V2) was not properly set ; standardId was not
correct.

This was forcing the sync-metadata command to try to re-create the field
again.
2025-02-14 13:50:06 +01:00
1090fb8fad Remove localization feature flag and improve CI workflow (#10208) 2025-02-14 09:41:35 +01:00
8a425456f2 feat(workspace): add support for custom domain status toggle (#10114)
Introduce isCustomDomainEnabled field in Workspace entity to manage
custom domain activation. Update related services, types, and logic to
validate and toggle the custom domain's status dynamically based on its
current state. This ensures accurate domain configurations are reflected
across the system.

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-02-13 15:01:33 +00:00
41dfeb34d1 Add sync metadata to migrate command (#10182)
Add sync-metdata to 0-42 upgrade command
2025-02-13 13:36:29 +01:00
6e57f02ae3 Feature flag IS_MICROSOFT_SYNC_ENABLED removal (#10170)
Since we have already removed the feature flag from twenty-infra, and
the most part of the application. we can now remove it from these
declaration

fixes https://github.com/twentyhq/core-team-issues/issues/331
2025-02-13 00:52:31 +01:00
193ef432a0 add object settings permission tab (#10159)
## Context
Introducing the "Permissions" tab in the role page

Next: Need to address some css improvements, some components might be
reusable and it still does not fully match the figma (icon missing for
permission types for example). We decided to merge like this for now so
we have something functional and I will update the code in an upcoming
PR

<img width="633" alt="Screenshot 2025-02-12 at 13 54 16"
src="https://github.com/user-attachments/assets/762db5d7-e0a6-4ee1-b299-24de6645bad1"
/>
2025-02-12 18:49:50 +01:00
0609b31c64 add WorkspaceDuplicateCriteria decorator + update duplicate resolver logic (#10128)
## Context

All objects have '...duplicates' resolver but only companies and people
have duplicate criteria (hard coded constant).
Gql schema and resolver should be created only if duplicate criteria
exist.

## Solution

- Add a new @WorkspaceDuplicateCriteria decorator at object level,
defining duplicate criteria for given object.
- Add a new duplicate criteria field in ObjectMetadata table
- Update schema and resolver building logic
- Update front requests for duplicate check (only for object with
criteria defined)



closes https://github.com/twentyhq/twenty/issues/9828
2025-02-12 17:32:59 +01:00
23d2e54439 RICH_TEXT_V2 upgrade command (#10094)
Adds two migration commands:
- copy note and task `body` data to `bodyV2`
- hide `body` view field and swap position with `bodyV2` view field

Related to issue https://github.com/twentyhq/twenty/issues/7613

---------

Co-authored-by: ad-elias <elias@autodiligence.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
2025-02-12 12:26:29 +01:00
ed4a5b0c15 fix: many fields in an object (#10061)
Co-authored-by: Charles Bochet <charles@twenty.com>
2025-02-11 17:15:30 +01:00
de91a5e39e chore(twenty-server): remove eslint warn + add maxWarning 0 (#10103) 2025-02-11 10:38:43 +01:00
3eaafbde55 fix log + add 3 indexes on fielMetadata and indexFieldMetadata (#10113) 2025-02-11 00:34:33 +01:00
f733307517 refacto(*): rename hostname to custom domain (#10100) 2025-02-10 15:12:36 +00:00
bb24c97f80 Translations - Crowdin, Set workspace member locale on signup, and optimizations (#10091)
More progress on translations:
- Migrate from translations.io to crowdin
- Optimize performance and robustness 
- Set workspaceMember/user locale upon signup
2025-02-09 22:10:41 +01:00
68183b7c85 feat(): enable custom domain usage (#9911)
# Content
- Introduce the `workspaceUrls` property. It contains two
sub-properties: `customUrl, subdomainUrl`. These endpoints are used to
access the workspace. Even if the `workspaceUrls` is invalid for
multiple reasons, the `subdomainUrl` remains valid.
- Introduce `ResolveField` workspaceEndpoints to avoid unnecessary URL
computation on the frontend part.
- Add a `forceSubdomainUrl` to avoid custom URL using a query parameter
2025-02-07 14:34:26 +01:00
8df59c085d Lingui working with NODE ENV=production again (#10067)
Lingui now offers an option to disable stripping even in prod mode so we
can bring it back
2025-02-07 10:05:07 +01:00
1b150e1da6 Environment variables in admin panel (read only) - front (#10011)
Frontend for https://github.com/twentyhq/core-team-issues/issues/293

POC - https://github.com/twentyhq/twenty/pull/9903

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-02-06 17:08:44 +01:00
736b845c98 335 workflow implement workflow cron triggers backend (#9988)
[Backend side] Add cron triggers to workflow
Closes https://github.com/twentyhq/core-team-issues/issues/335
2025-02-05 11:02:49 +00:00
40f43a4076 add createMany fields to fieldMetadataService to batch field creation (#9957)
## Context
Not exposed in the API yet, this new method allows us to reduce the time
to create multiple fields at once, mostly during seeding. This allows us
to batch transactions and avoid recomputing the cache everytime.

With this change, we recompute the cache 7 times instead of 35 during
seeding. We could do the same for objects.
2025-02-04 11:18:57 +01:00
5a13d5ae29 Sort active workspaces (#9973)
Adding an order when fetching active workspaces so that when we run
command we can better understand the workspaces from the logs. Also, if
the command fails, we know we could start from a specific createdAt
later on
2025-02-03 13:33:00 +01:00