# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
BlocknoteJS requires an ESM module where our server is CJS, this forced
us to pin the server-util version, which led us to force the resolution
of several packages, leading to bugs downstream.
From Node 22.12 Node supports requiring ESM modules (available from Node
22.0 with a flag). So I upgrade the module.
I picked Node 22 and not Node 23 or Node 24 because 22 is the LTS and we
don't plan to change node versions frequently.
If you remain on Node 18, things should still mostly work, except if you
edit a Rich Text field.
I also starting changing the default runtime for Serverless Functions
which isn't directly related. This means new serverless functions will
be created on Node 22, but we will still need another PR to migrate
existing serverless functions before September (end of support by AWS).
(In this PR I also remove the upgrade commands from 0.43 since they rely
on Blocknote and I didn't want to have to deal with this)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
- fix missing createBy injection in api createOne and createMany
endpoints
- add a command to fix null default value for createdBySource in
production entities
- tested on `1747159401197/` dump extract of production db without issue
In this PR:
- Set the default position for the DONE option of the task's status
option to `2` instead of `1`, which was the same as `IN_PROGRESS`
option's position.
- Write a command to prevent position duplicates in the database for the
task's status field.
What I've checked before setting this PR as ready to be reviewed:
- De-duplicating the position solves the issue and it's possible to edit
the field (solves the related issue)
- The upgrade command de-duplicates the position for each workspace.
There are no more DONE options with `position=2`. I ran the upgrade
command on the `database-snapshot-manager` dataset.
- Suspended workspaces aren't fixed
---
To test the script:
```ts
const scannedPositions = new Set();
let biggestPosition = -1;
// Sort options by position for consistent processing
const sortedOptions = [
{ name: 'a', position: 2 },
{ name: 'b', position: 1 },
{ name: 'c', position: 1 },
{ name: 'd', position: 2 },
].sort((a, b) => a.position - b.position);
for (const option of sortedOptions) {
if (scannedPositions.has(option.position)) {
option.position = biggestPosition + 1;
}
biggestPosition = Math.max(biggestPosition, option.position);
scannedPositions.add(option.position);
}
console.log('Sorted options:', sortedOptions);
```
Closes https://github.com/twentyhq/twenty/issues/11790
## Context
While deploying the IS_NEW_RELATION_ENABLED (we don't compute relation
based on relationMetadata anymore) to existing workspace, I've tested to
run a sync-metadata post feature flag activation. This has raised two
issues:
- the workspaceMigration generator (which is over-complex and should be
refactored later) for fieldMetadata of type RELATION was not handling
settings update properly ;
- we need to delete existing fieldMetadata corresponding to the UUID
foreignKey as they are not needed anymore. This is handled as a 0.53
upgrade command as 0.53 will also come with the full removal of the old
relation system
---------
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
Co-authored-by: prastoin <paul@twenty.com>
## Architecture Detail
The goal is to merge the two TypeORM schemas.
Having two schemas prevent doing things like fieldMetadata.workspace in
TypeORM, and create useless debates since there is no clear line (is a
serverlessFunction core or metadata? What about events? etc.)
### Before
```
┌───────────────────┐ ┌───────────────────┐
│ core schema │ │ metadata schema │
├───────────────────┤ ├───────────────────┤
│- User │ │- ObjectMetadata │
│- Workspace │ │- FieldMetadata │
│- UserWorkspace │ │- RelationMetadata │
│- etc. │ │- etc. │
└───────────────────┘ └───────────────────┘
```
### After the Migration
```
┌───────────────────────────────────────────┐
│ engine schema │
├───────────────────────────────────────────┤
│- User - ObjectMetadata │
│- Workspace - FieldMetadata │
│- UserWorkspace - RelationMetadata │
│- etc. - etc. │
└───────────────────────────────────────────┘
```
## Strategy
1. During 0.53 we backfill the *_typeorm_migrations* table of the core
schema with all metadata migrations
2. That way in 0.54 we can move the metadata migrations from the
metadata folder to the core folder. We will also edit the migration
files to reference "core" instead of "metadata". For people doing a
fresh install this will run smoothly and create the tables in Core
directly. For people on an existing install, this migrations will not
run because they were added to the *_typeorm_migrations* in 0.53
3. In 0.55 we will rename "core" to something else (for example
"engine")
Note: if someone jumps version, for example skips to 0.54 directly
without having run 0.53 then this could cause issue. In 0.54 we should
consider gating the "migrate:prod" in the docker file so that it's
controlled and run by the upgrade command (and not run if the command
wasn't executed properly)
# Introduction
This PR refactors the way we previously manually handled the upgrade
command `versionTo` and `versionFrom` values to be replaced by a
programmatic inferring using the `APP_VERSION` env variable. It raises
new invariant edge cases that are covered by new tests and so on
Please keep in mind that an upgrade will run agnostically of any `patch`
semver value as it should be done only when releasing a `major/minor`
version update
[Related discord
thread](https://discord.com/channels/1130383047699738754/1368953221921505280)
## Testing in local
In order to test in local we have to define an `APP_VERSION` value in
`packages/twenty-server/.env` following semver ( or not 🙃 )
## Logs example
```ts
Computing new Datasource for cacheKey: 20202020-1c25-4d02-bf25-6aeccf7ea419-8 out of 0
query: SELECT * FROM current_schema()
query: SELECT version();
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Initialized upgrade context with:
- currentVersion (migrating to): 0.53.0
- fromWorkspaceVersion: 0.52.0
- 2 commands
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Upgrading workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 from=0.52.0 to=0.53.0 1/2
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Upgrade for workspace 20202020-1c25-4d02-bf25-6aeccf7ea419 ignored as is already at a higher version.
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Running command on workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db 2/2
Computing new Datasource for cacheKey: 3b8e6458-5fc1-4e63-8563-008ccddaa6db-8 out of 0
query: SELECT * FROM current_schema()
query: SELECT version();
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Upgrading workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db from=0.52.0 to=0.53.0 2/2
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Upgrade for workspace 3b8e6458-5fc1-4e63-8563-008ccddaa6db ignored as is already at a higher version.
[Nest] 37872 - 05/06/2025, 4:07:21 PM LOG [UpgradeCommand] Command completed!
```
## Misc
Related to https://github.com/twentyhq/twenty/issues/11780
## Introduction
This PR enables functionality discussed in [Layout Date
Formatting](https://github.com/twentyhq/core-team-issues/issues/97).
### TLDR;
It enables greater control of date formatting at the object's field
level by upgrading all DATE and DATE_TIME fields' settings from:
```ts
{
displayAsRelativeDate: boolean
}
```
to:
```ts
type FieldDateDisplayFormat = 'full_date' | 'relative_date' | 'date' | 'time' | 'year' | 'custom'
{
displayFormat: FieldDateDisplayFormat
}
```
PR also includes an upgrade command that will update any existing DATE
and DATE_TIME fields to the new settings value
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
# Introduction
As we deploy patch from the main branch we've ship the upgrade for the
`0.51` within the `0.50`.
We should only do that when about to release
We should find a way for this not to occur again
# Introduction
Refactored the upgrade command to be more intuitive to anyone wanting to
add a command to the next relase upgrade instance
Also updated the upgrade command for the next 0.44 release
# Introduction
This PR contains a big test file and few snapshots
Related to https://github.com/twentyhq/core-team-issues/issues/487
## New env var `APP_VERSION`
Now will be injected directly in a built docker image the twenty's built
version. Inferred from the build git tag name.
Which mean on main or other `not a tag version` built APP_VERSION will
be `null`
## New upgrade-commander-runner
Refactored the upgrade command to be more strict regarding:
- Version management
- Sync metadata command always run
- Added failing workspaces aggregator + logs on cleanup
From now on the `upgrade` command will compare the `WORKSPACE_VERSION`
to the `APP_VERSION` in order to bypass any workspace version != than
the upgrade version `fromVersion`
## Existing commands
Note that the version validation will be done only when passing by the
`upgrade` command.
Which means that running the following command
`upgrade:x.y-some-specific-command` won't result in workspace version
mutation
This is to enforce that all an upgrade commands + sync-metadata has been
run on a workspace
## Will do in other PR but related
### New workspace
New workspace will now be inserted with version equal to the APP_VERSION
they've been created by
### Old workspace
Will create a command that should be ran outside of any `upgrade-runner`
extending command, the command will have to be ran on every workspace
before making the next release upgrade
This command iterates over any active and suspended workspace that has
`version` to `NULL` in order to update it `APP_VERSION` -1 minor
### SENTRY_RELEASE
- Either deprecate SENTRY_RELEASE in favor of `APP_VERSION` => What
about main with null version ? or create a new env var that would be
`APP_COMMIT_SHA` instead of SENTRY third party ref
### Update CD to inject APP_VERSION from branch name
### Update docs and release logs
Adding documentation for `APP_VERSION`
## Related PRs:
https://github.com/twentyhq/twenty-infra/pull/181
Simplifying a lot the upgrade system.
New way to upgrade:
`yarn command:prod upgrade`
New way to write upgrade commands (all wrapping is done for you)
```
override async runOnWorkspace({
index,
total,
workspaceId,
options,
}: RunOnWorkspaceArgs): Promise<void> {}
```
Also cleaning CommandModule imports to make it lighter
