Commit Graph

371 Commits

Author SHA1 Message Date
9e83d902d8 [permissions V2] Create and update a custom role (without granularity) (#11003)
First steps for https://github.com/twentyhq/core-team-issues/issues/595
and https://github.com/twentyhq/core-team-issues/issues/621

Not handling granular permissions through objectPermissions and
settingsPermissions next; will come next !
2025-03-18 18:42:30 +01:00
aa6fd90424 Create objectPermissions and settingsPermissions tables (#10962)
Closes https://github.com/twentyhq/core-team-issues/issues/594
2025-03-18 10:45:31 +01:00
8b5d5b35ad Workspace seeders with version (#10895)
# Introduction
close https://github.com/twentyhq/core-team-issues/issues/487
Updated the seeders to infer the workspace's version from the
`APP_VERSION` env var

To test in local run: `npx nx database:reset twenty-server` with either
a defined or not defined `APP_VERSION` in your `.env`
( note that invalid semver values will throw an error and stop the
process ) ( valid version ex: `APP_VERSION=1.0.0`)
2025-03-17 15:32:49 +01:00
23b4605987 [REFACTOR] Workspace version only x.y.z (#10910)
# Introduction
We want the APP_VERSION to be able to contains pre-release options, in a
nutshell to be semVer compatible.
But we want to have workspace, at least for the moment, that only store
`x.y.z` and not `vx.y.z` or `x.y.z-alpha` version in database

Explaining this refactor

Related https://github.com/twentyhq/twenty/pull/10907
2025-03-14 19:21:44 +01:00
c371dd5d27 Add custom domain to Lab and move countBoolean command to 0.44 from 0.50 (#10884)
- Add customs domain to Lab
- Moving back countBoolean command to 0.44 from 0.50 (I inadvertently
moved it yesterday night...)

<img width="592" alt="Screenshot 2025-03-14 at 12 07 44"
src="https://github.com/user-attachments/assets/1d8d7b36-f392-4df0-974d-426a55760af2"
/>
2025-03-14 15:13:44 +01:00
362bc333d1 [ENH] APP_VERSIONdocumentation (#10880)
# Introduction
In https://github.com/twentyhq/twenty/pull/10751 we decided not to put
`APP_VERSION` references in `.env.example` as it's programmatically
defined by our CD and should not be override by any manual interaction.

Still, as a dev testing the upgrade command in local, if you do not set
the `APP_VERSION` in local you will encounter the following error:
```ts
'Cannot run upgrade command when APP_VERSION is not defined'
```

@guillim currently doing the release legitimately raised that it was not
very intuitive

## Levers
- Improve error message such as adding reference to checking env
variables
- App local upgrade command dev dedicated documentation ?

## Conclusion
Any suggestions are more than welcomed !
2025-03-14 10:00:32 +00:00
1bffe57f6b feat: added countTrue and countFalse (#10741)
fix: #10603 


https://www.loom.com/share/cebc8a19bd8e4ae684a5a215d0fd1f94?sid=cadaa395-285c-45c9-b3ce-2ae6d1330a3c

---------

Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-03-14 07:34:21 +01:00
dfb00f2b61 [FIX] Omition on previous integ of upgrade command compare version based on integer result (#10864) 2025-03-13 18:12:05 +00:00
fc30ba57f8 Fix server integration tests 2 (#10818)
## Context
- Removing search* integration tests instead of fixing them because they
will be replaced by global search very soon
- Fixed billing + add missing seeds to make them work
- Fixed integration tests not using consistently the correct "test" db
- Fixed ci not running the with-db-reset configuration due to nx
configuration being used twice for different level of the command
- Enriched .env.test
- Fixed parts where exceptions were not thrown properly and not caught
by exception handler to convert to 400 when needed
- Refactored feature flag service that had 2 different implementations
in lab and admin panel + added tests
- Fixed race condition when migrations are created at the same timestamp
and doing the same type of operation, in this case object deletion could
break because table could be deleted earlier than its relations
- Fixed many integration tests that were not up to date since the CI has
been broken for a while

---------

Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
2025-03-13 17:48:29 +01:00
37afb38479 refactor(server): upgrade command to more human friendly (#10858)
# Introduction
Refactored the upgrade command to be more intuitive to anyone wanting to
add a command to the next relase upgrade instance

Also updated the upgrade command for the next 0.44 release
2025-03-13 16:48:58 +01:00
bd5d211590 [FEAT] New APP_VERSION env var inferred from tag & refactor upgrade-command to integrate versioning (#10751)
# Introduction
This PR contains a big test file and few snapshots
Related to https://github.com/twentyhq/core-team-issues/issues/487

## New env var `APP_VERSION`
Now will be injected directly in a built docker image the twenty's built
version. Inferred from the build git tag name.
Which mean on main or other `not a tag version` built APP_VERSION will
be `null`

## New upgrade-commander-runner
Refactored the upgrade command to be more strict regarding:
 - Version management
 - Sync metadata command always run
 - Added failing workspaces aggregator + logs on cleanup
 
From now on the `upgrade` command will compare the `WORKSPACE_VERSION`
to the `APP_VERSION` in order to bypass any workspace version != than
the upgrade version `fromVersion`
## Existing commands
Note that the version validation will be done only when passing by the
`upgrade` command.
Which means that running the following command
`upgrade:x.y-some-specific-command` won't result in workspace version
mutation

This is to enforce that all an upgrade commands + sync-metadata has been
run on a workspace



## Will do in other PR but related
### New workspace
New workspace will now be inserted with version equal to the APP_VERSION
they've been created by

### Old workspace
Will create a command that should be ran outside of any `upgrade-runner`
extending command, the command will have to be ran on every workspace
before making the next release upgrade
This command iterates over any active and suspended workspace that has
`version` to `NULL` in order to update it `APP_VERSION` -1 minor

### SENTRY_RELEASE
- Either deprecate SENTRY_RELEASE in favor of `APP_VERSION` => What
about main with null version ? or create a new env var that would be
`APP_COMMIT_SHA` instead of SENTRY third party ref

### Update CD to inject APP_VERSION from branch name

### Update docs and release logs
Adding documentation for `APP_VERSION`

## Related PRs:
https://github.com/twentyhq/twenty-infra/pull/181
2025-03-13 15:46:27 +01:00
5b34f86b6d Add Icon to roles (#10842)
## Context
Adding icon column to role table to distinguish roles.

<img width="621" alt="Screenshot 2025-03-13 at 11 02 37"
src="https://github.com/user-attachments/assets/dc7b121c-2901-4599-9638-d5dcdf443999"
/>
2025-03-13 11:34:30 +01:00
f4a362b53a Add base form action without logic (#10811)
<img width="1298" alt="Capture d’écran 2025-03-12 à 15 32 27"
src="https://github.com/user-attachments/assets/8a3140e5-e165-445e-a718-748aa76b525c"
/>
2025-03-12 17:05:31 +00:00
daa501549e 271 remove is command menu v2 enabled (#10809)
Closes https://github.com/twentyhq/core-team-issues/issues/271

This PR
- Removes the feature flag IS_COMMAND_MENU_V2_ENABLED
- Removes all old Right drawer components
- Removes the Action menu bar
- Removes unused Copilot page
2025-03-12 16:26:29 +01:00
293e3f58c4 Fix TEST env not using the correct DB for datasources (#10780)
## Context
Config was programmatically loaded in our datasources however the
default behavior of dotenv is to ignore vars if they are already
defined. This means we need to be careful about the order of env
injection and sometimes it's done at a higher level (for example
db:reset will depend on build). To make things easier I'm using the
override flag to properly override the PG_DATABASE_URL if different (and
to properly work with the 'test' DB instead of 'default' during
testing).
2025-03-11 16:20:01 +01:00
4be75fb7da eliminate IS_BILLING_PLANS_ENABLED feature flag (#10678)
Solves: https://github.com/twentyhq/core-team-issues/issues/527

**TLDR:**
Basically the title. Fetches the product and prices from the database
instead of the environment variables.

**What this means:**

- new subscriptions in twenty will be hybrid (per seat subscription plus
an usage base product)
- right now the price for the usage base product  is 0$ per unit
- The existing subscription will work normally, however we will need to
update their subscription items in order to contain the usage base
product (remember that the pricing intervals like monthly or yearly
should match in all the subscription items)
-  The previous point can be done using Stripe Postman


**In order to test:**

- Have the environment variable IS_BILLING_ENABLED set to true and add
the other required environment variables for Billing to work
- Do a database reset (to ensure that the new feature flag is deleted
and that the billing tables are created)
- Run the command: npx nx run twenty-server:command
billing:sync-plans-data (if you don't do that the products and prices
will not be present in the database)
- Run the server , the frontend, the worker, and the stripe listen
command (stripe listen --forward-to
http://localhost:3000/billing/webhooks)
- Buy a subscription for acme workspace
- Update the quantity of members in a workspace (add or delete)
- Change the subscription interval

---------

Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
2025-03-07 11:46:22 +01:00
f4fcf39eb5 [permissions] Prepare for roll-out (#10676)
Closes https://github.com/twentyhq/core-team-issues/issues/469 and
https://github.com/twentyhq/core-team-issues/issues/500

In this PR
1. stop conditioning permission initialization for a workspace to env
variable value. Instead we want to create and assign permissions and
roles in all new workspaces. For now that will be totally silent.
2. temporarily, the default role is set to the admin role for new
workspaces. it will also be the case for existing workspaces through the
backfill command. Member role is still being created though. (when we
will do the final roll-out we will update this so that future workspaces
have the member role as default role. our goal here is not to break any
current behaviour for users, that today have all have the equivalent of
admin rights).
2025-03-05 16:47:41 +01:00
ad628c1266 [ENH] Do not fail on missed cache within server commands (#10634)
Avoid critical failure if cache is missed when interacting with the
twenty-orm in the upgrade commands
2025-03-04 10:28:21 +01:00
b8d944bd6e fix try catch logic in upgrade command (#10616) 2025-03-03 16:37:39 +01:00
d19144208b Improve migration runner performances (#10572)
## Context
Workspace creation and more specifically sync-metadata performances are
bad at the moment. We are trying to identify bottlenecks and one of the
root causes is the migration runner that can take up to 10s when setting
up a new workspaces with all its tables.
First observation is we do a lot of things sequentially, mostly to make
the code easier to read and debug but it impacts performances. For
example, a table creation is done in two steps, we first ask typeorm to
create the table then ask typeorm to create columns (and sometimes
columns one by one), each instruction can take time because typeorm
seems to do some checks internally.

The proposition here is to try to merge migrations when possible, for
example when we create a table we want the migration to also contain the
columns it will contain so we can ask typeorm to add the columns at the
same time. We are also using batch operations when possible (addColumns
instead of addColumn, dropColumns instead of dropColumn)
Still, we could go further with foreign keys creations or/and try with
raw query directly.

## Test
New workspace creation:
See screenshot, 9865.40233296156ms is on main, the rest is after the
changes:

<img width="610" alt="Screenshot 2025-02-28 at 09 27 21"
src="https://github.com/user-attachments/assets/42e880ff-279e-4170-b705-009e4b72045c"
/>

ResetDB and Sync-metadata on an existing workspace commands still work
2025-02-28 19:51:45 +01:00
baa3043954 Refactor upgrade commands (#10592)
Simplifying a lot the upgrade system.

New way to upgrade:
`yarn command:prod upgrade`

New way to write upgrade commands (all wrapping is done for you)
```
  override async runOnWorkspace({
    index,
    total,
    workspaceId,
    options,
  }: RunOnWorkspaceArgs): Promise<void> {}
```

Also cleaning CommandModule imports to make it lighter
2025-02-28 19:51:32 +01:00
00b650a121 Fix export view missing column and viewGroups not persisted bug (#10584)
Fixes https://github.com/twentyhq/twenty/issues/10535
Also fixes missing columns in export view
2025-02-28 15:59:13 +01:00
122a6a7801 [permissions] Backfill command to prepare workspaces (#10581)
Closes https://github.com/twentyhq/core-team-issues/issues/317

---------

Co-authored-by: Weiko <corentin@twenty.com>
2025-02-28 15:46:51 +01:00
fba63d9cb7 migrate rich text v1 workspace + move relation migration to 0.44 (#10582)
Adapt from MigrateRichTextFieldCommand
2025-02-28 14:46:34 +01:00
2aa72de210 remove isRichTextV2Enabled feature flag (#10562) 2025-02-28 14:34:06 +01:00
0dc1cd9df1 [REG/FEAT] Sync metadata after upgrade (#10557)
# Introduction
Historically we've been programmatically running sync metadata just
after all upgrade command's migration.
Adding back this behavior as default to the new dynamic modules

Duplicated already existing synchronize metadata logic as a quick fix as
we're about to iterate over commands next sprint
2025-02-27 18:44:41 +01:00
39543872e6 add new @WorkspaceIsSearchable decorator + updates services + add migration command (#10507)
closes https://github.com/twentyhq/core-team-issues/issues/345
2025-02-27 13:57:07 +01:00
0220672fa9 Add default role to workspace (#10444)
## Context
Adding a defaultRole to each workspace, this role will be automatically
added when a member joins a workspace via invite link or public link
(seeds work differently though).
Took the occasion to refactor a bit the frontend components, splitting
them in smaller components for more readability.

## Test
<img width="948" alt="Screenshot 2025-02-24 at 14 54 02"
src="https://github.com/user-attachments/assets/13ef1452-d3c9-4385-940c-2ced0f0b05ef"
/>
2025-02-25 11:26:35 +01:00
a1eea40cf7 feat: populate relation join column (#10212)
Fix
https://github.com/twentyhq/core-team-issues/issues/241#issue-2793030259
2025-02-25 11:24:05 +01:00
dde70ee3b0 Add fields for admin panel access and workspace version (#10451)
Prepare for better version upgrade system + split admin panel into two
permissions + fix GraphQL generation detection

---------

Co-authored-by: ehconitin <nitinkoche03@gmail.com>
2025-02-24 21:38:41 +01:00
1b64f87d75 Add error management to the Billing Package (#10343)
Solves https://github.com/twentyhq/core-team-issues/issues/403

**TLDR:**
Enhance error management in Billing and when a customer is updated it
updates automatically the Stripecustomer id in the entitlements.


- Add Billing exceptions to filter. 
- Add onUpdate for billing customer and entitlement.
- Remember to run the migrations with is BILLING_ENABLED set to true.

**In order to test (a simple test case)**

- Ensure that the environment variables for Sentry and Billing are set,
ensuring that SENTRY_ENVIRONMENT=staging
- Run the server, the worker and the stripe cli
- Do a database reset with IS_BILLING_ENABLED set to true
- Go to stripe in test mode and update a random price description, this
causes an exception because you are trying to write a price of. a
product that doesn't exists in the database
- You should see an error in Sentry:


![image](https://github.com/user-attachments/assets/7b3c8c7f-6628-4a20-9889-a691e7838d79)
2025-02-24 10:10:06 +01:00
bf92860d19 feat(twenty-server): add trusted domain - backend crud (#10290)
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: Paul Rastoin <45004772+prastoin@users.noreply.github.com>
2025-02-21 16:02:48 +00:00
d747366bf3 Provide a wrapper to execute command on workspace with easier devXP (#10391)
Proposal:
- Add a method in ActiveWorkspaceCommand to loop over workspace safely
(add counter, add try / catch, provide datasource with fresh cache,
destroy datasource => as we do always do it)

Also in this PR:
- make sure we clear all dataSources (and not only the one on metadata
version in RAM)
2025-02-21 16:40:33 +01:00
311fc402d5 Enhance ActiveWorkspaceCommand to order, limit and startFrom workspaceId (#10378)
I have tested it locally with all combinaisons of params
2025-02-21 10:37:04 +01:00
9f454c565b 410 open in side panel (#10363)
Closes https://github.com/twentyhq/core-team-issues/issues/410

- Added `openRecordIn` column in the `view` entity, which is set to
`SIDE_PANEL` by default
- Created a new option inside the view option dropdown to be able to set
`openRecordIn`
- Updated all record show page openings to reflect the setting behavior
- For `workflow`, `workflowVersion` and `workflowRun` (what I call
workflow objects), we want the default view `openRecordIn` to be set to
`RECORD_PAGE`. When seeding the views for the new workspaces, we set
`openRecordIn` to `RECORD_PAGE` for workflow objects. Since the workflow
objects views `openRecordIn` will be set to the default value
`SIDE_PANEL` for the existing workspaces when the sync metadata runs, I
created a script to run in the 0.43 update to update this value.
- Updated `closeCommandMenu` because of problems introduced by the
animate presence wrapper around the command menu. We now reset the
states at the end of the animation.

Note: We want to be able to open all workflow objects pages in the side
panel, but this requires some refactoring of the workflow module. For
now @Bonapara wanted to allow the possibility to change the
`openRecordIn` setting to `SIDE_PANEL` even for the workflows even if
it's buggy and not ready for the moment. Since this is an experimental
feature, it shouldn't cause too many problems.
2025-02-21 09:27:33 +00:00
41bbb4b47f Fix upgrade command 0.42 2025-02-21 00:20:44 +01:00
4037fb5860 Fix upgrade command 0.42 (#10373) 2025-02-20 20:22:27 +01:00
68cb94f365 [BUGFIX] Call and await destroyDataSourceForWorkspace to avoid killing poor pod (#10364)
# Motivations
Upgrade migration is not possible atm in production because running over
all workspaces cause cpu raise leading to pod termination
2025-02-20 16:45:21 +01:00
94c0d0f8d2 [BUG] Refactor actor composite type (#10232)
fixes #10200 

The FieldActor Zod schema was updated to correctly handle null context.

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-02-20 11:05:26 +01:00
422e4e33c0 [BUGFIX][PROD] RICH_TEXT_V2 command handle {} body col value (#10324)
# Introduction
Encountered in issue in production where we have a lot of records that
has RICH_TEXT_FIELD set to `{}`

```sh
[Nest] 20106  - 02/19/2025, 12:43:08 PM     LOG [MigrateRichTextFieldCommand] Generating markdown for 1 records
[Nest] 20106  - 02/19/2025, 12:43:09 PM     LOG [MigrateRichTextFieldCommand] Error in workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db: TypeError: o is not iterable
```

## Fix
While reading `fieldValue` definition also strictly check if it's `{}` +
checking after JSON parse if it's an iterable to pass to the
`serverBlockNoteEditor` in order to be 100 bullet proof for prod
migration command

## Refactor Dry run
Implemented dry run

## Refactor to Idempotency
Made the script idempotent in order to avoid issues with re-running
commands

## Error repro
- In local checkout on v0.41.5 run `yarn && npx nx reset && npx nx
start`
- Create record manually in db that has a RICH_TEXT body to `{}`
- Checkout to main, `yarn && npx nx reset && npx nx build twenty-server
&& yarn command:prod upgrade-0.42:migrate-rich-text-field -d`
2025-02-20 09:16:58 +00:00
222f475cc6 [permissions] Fix db reset leads to broken state if env var permission is not true (#10309) 2025-02-18 17:25:54 +00:00
af8a167fb2 Update searchVector on new richTextV2 note and task entities + migration command (#10303)
closes https://github.com/twentyhq/core-team-issues/issues/343
closes https://github.com/twentyhq/core-team-issues/issues/340
2025-02-18 18:11:29 +01:00
cb3bd1353a [permissions] Add object records permissions to role entity (#10255)
Closes https://github.com/twentyhq/core-team-issues/issues/388

- Add object records-related permissions to role entity
- Add it to queriable `currentUserWorkspace` (used in FE)
2025-02-17 18:32:39 +01:00
5b961cbb7f Tasks assigned to me view (#9567) (#9568)
Issue: https://github.com/twentyhq/core-team-issues/issues/154

View is now added.

TODO: Fix filtering a relation field by 'Me' in Tasks.

---------

Co-authored-by: ad-elias <elias@autodiligence.com>
2025-02-17 10:13:55 +00:00
db1b11742b [0.42][FIX] Seeder activate rich text v2 feature flag (#10229)
as per title
2025-02-14 17:57:32 +00:00
258025a18f Fix upgrade-0-42 command to migrate RICH_TEXT field metadata to RICH_TEXT_V2 (#10218)
While making sure the upgrade-0.42 command was working as expected to
upgrade from 0.41 to 0.42, we've detected that the standardId of the new
bodyV2 (type RICH_TEXT_V2) was not properly set ; standardId was not
correct.

This was forcing the sync-metadata command to try to re-create the field
again.
2025-02-14 13:50:06 +01:00
1090fb8fad Remove localization feature flag and improve CI workflow (#10208) 2025-02-14 09:41:35 +01:00
8a425456f2 feat(workspace): add support for custom domain status toggle (#10114)
Introduce isCustomDomainEnabled field in Workspace entity to manage
custom domain activation. Update related services, types, and logic to
validate and toggle the custom domain's status dynamically based on its
current state. This ensures accurate domain configurations are reflected
across the system.

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2025-02-13 15:01:33 +00:00
41dfeb34d1 Add sync metadata to migrate command (#10182)
Add sync-metdata to 0-42 upgrade command
2025-02-13 13:36:29 +01:00
6e57f02ae3 Feature flag IS_MICROSOFT_SYNC_ENABLED removal (#10170)
Since we have already removed the feature flag from twenty-infra, and
the most part of the application. we can now remove it from these
declaration

fixes https://github.com/twentyhq/core-team-issues/issues/331
2025-02-13 00:52:31 +01:00