Prepare for better version upgrade system + split admin panel into two
permissions + fix GraphQL generation detection
---------
Co-authored-by: ehconitin <nitinkoche03@gmail.com>
Fairly straightforward change, it's my first contribution in this
project so tried to follow the existing patterns.
I couldn't find any component tests or e2e Playwright tests touching
this area to update - happy to write some from scratch if necessary!
Closes#10285
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
## Context
Integration test ci was not running properly. This PR should fix that
Todo: This was not running for a while, many tests are not succeeding so
I'll try to fix them in this PR now that the action is running properly.
More progress on translations:
- Migrate from translations.io to crowdin
- Optimize performance and robustness
- Set workspaceMember/user locale upon signup
Fortunately we're not using NODE_ENV=production for our deployment, but
we need to think of a better long-term fix. Not easy because Lingui
strips out messages in production environment
This PR updates the docker-compose installation documentation and env
sample to improve the setup experience.
- Updates the URLs for raw files to reference main branch, which is
likely where new users will be pulling from initially. This seems to be
the most straightforward option; assume that advanced users who want to
retrieve it from a particular tag will know to change the URL for their
scenario.
- Fixes an improperly stated curl command.
- Adds a note that the PGPASSWORD_SUPERUSER should be URL-safe. This is
required since the value is later concat into a PG_DATABASE_URL as a
URL, and expected to be in proper URL format. Touches on #8597.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
This pull request addresses the issue of ensuring that npx nx run
twenty-front:graphql:generate and npx nx run
twenty-front:graphql:generate --configuration=metadata commands are run
to generate the necessary GraphQL files. This prevents changes from
being missed and ending up in subsequent unrelated PRs.
Changes:
Added a step in the ci-server.yml workflow to check for pending GraphQL
generation.
If any GraphQL changes are detected, the CI will fail, and an error
message will be displayed instructing the developer to run the necessary
commands and commit the changes.
This approach is similar to the existing TypeORM migration check and
helps maintain consistency and correctness in the codebase.
Issue Resolved: #9726
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
- Clean Playwright's configuration:
- Remove artificial 500ms delay between each step
- Group all tests under a `chrome` project relying on a `setup` project
to get an authentication state which all tests can reuse
- Changes on the `Sign up with invite link via email` test:
- Generate a new email for each test trial, as previously it was failing
when run many times
- Make deleting the account part of the test; if we write other tests
for account sign-up, we'll prefer to delete the accounts with an HTTP
call to speed up things
- Added some assertions to ensure we reached steps when expected, as we
removed the 500ms delay between each step, and it made some assertions
fail
- Wrote new tests for workflows:
- Created `Create workflow`, a test asserting we can create a workflow
from the record table
- Created `Create simple workflow`, a test asserting we can create a
simple flow; I will add more assertions to this test and write other
tests once this first PR is approved
- I make HTTP calls to delete and destroy workflows after they run to
keep the database clean
- Added a data-testid to ensure we focus elements from the Cmd+K; our
selectors are not strong – see `getByRole('textbox')` – and I preferred
to scope them to a root element
- Added an `aria-label` to a button
---------
Co-authored-by: prastoin <paul@twenty.com>
# Introduction
Running `storybook` with `configuration=pages` outputs a coverage that
is not replicated afterwards by the `storybook:coverage` command.
```sh
npx nx storybook:serve-and-test:static twenty-front --configuration=pages --shard=1/1 --checkCoverage=true
# ...
[TEST] Test Suites: 40 passed, 40 total
[TEST] Tests: 52 passed, 52 total
[TEST] Snapshots: 0 total
[TEST] Time: 84.786 s
[TEST] Ran all test suites.
[TEST] Coverage file (13067196 bytes) written to .nyc_output/coverage.json
[TEST] > nx storybook:coverage twenty-front --coverageDir=coverage/storybook --checkCoverage=true
[TEST]
[TEST]
[TEST] > nx run twenty-front:"storybook:coverage" --coverageDir=coverage/storybook --checkCoverage=true
[TEST]
[TEST] > npx nyc report --reporter=lcov --reporter=text-summary -t coverage/storybook --report-dir coverage/storybook --check-coverage=true --cwd=packages/twenty-front
[TEST]
[TEST]
[TEST] =============================== Coverage summary ===============================
[TEST] Statements : 70.45% ( 775/1100 )
[TEST] Branches : 45.39% ( 197/434 )
[TEST] Functions : 63.52% ( 209/329 )
[TEST] Lines : 71.28% ( 767/1076 )
[TEST] ================================================================================
[TEST]
```
```sh
> npx nyc report --reporter=lcov --reporter=text-summary -t coverage/storybook --report-dir coverage/storybook --check-coverage=true --cwd=packages/twenty-front
=============================== Coverage summary ===============================
Statements : 37.4% ( 9326/24931 )
Branches : 22.99% ( 2314/10063 )
Functions : 28.27% ( 2189/7741 )
Lines : 37.81% ( 9261/24488 )
================================================================================
ERROR: Coverage for lines (37.81%) does not meet global threshold (39%)
ERROR: Coverage for branches (22.99%) does not meet global threshold (23%)
ERROR: Coverage for statements (37.4%) does not meet global threshold (39%)
Warning: command "npx nyc report --reporter=lcov --reporter=text-summary -t coverage/storybook --report-dir coverage/storybook --check-coverage=true --cwd=packages/twenty-front" exited with non-zero status code
```
## Fix
Persist configuration scope arg to the `check-coverage` command
## Question
Should we add a step in the `ci-front` what would merge all
`performance,modules,pages` coverage and calculate the `global` coverage
? => I think that this has no plus value as we still compute each of
them individualy
# Introduction
It seems like I've just oversight adding it back while debugging
# Centralization
Please note that we only have access to the following context within a
job matrix properties:
```
Available expression contexts: github, inputs, vars, needs
```
We could centralize the `storybook_scope` as a job `outputs` in order to
avoid this to re-occurs.
# Introduction
Applying the most present `yaml` file extension to all
`.github/workflows/` files
## Notes
Regarding the `restore-cache` and `save-cache` composite actions github
agnostically searches for any `DockerFile` `action.yml` and
`action.yaml` within the target folder when invoking composite actions
as follows:
```yml
- name: Restore storybook build cache
if: steps.changed-files.outputs.any_changed == 'true'
uses: ./.github/workflows/actions/restore-cache
with:
key: ${{ env.STORYBOOK_BUILD_CACHE_KEY }}
```
# Introduction
Unless I'm mistaken in the existing `yarn-install` composite action
`dependencies` are cached twice.
## Cached through `setup-node`
As we're providing `yarn` value to cache inputs.
Setup node will cache the global `.yarn`
```yml
- name: Setup Node.js and get yarn cache
uses: actions/setup-node@v3
with:
node-version: 18
cache: yarn
```
## Programmatic `node_modules` caching
We even manually still cache `node_modules` folder using:
```yml
- name: Cache node modules
id: node-modules-cache
uses: actions/cache@v3
with:
path: |
node_modules
packages/*/node_modules
key: root-node_modules-${{ hashFiles('yarn.lock') }}
restore-keys: root-node_modules-
```
By the way it seems like that the `yarn.lock` file is useless as the
restore-key pattern omits it
This could, unless I'm mistaken, leads to invalid cache restore
## Runtime
The current infra results in the following install deps logs:
```sh
Prepare all required actions
Getting action download info
Download action repository 'actions/setup-node@v3' (SHA:1a444[2](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:2)cacd436585916779262731d5b162bc6ec7)
Download action repository 'actions/cache@v[3](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:3)' (SHA:f4b3439a656ba812b8cb417d2d49f9c810103092)
Run ./.github/workflows/actions/yarn-install
Run actions/setup-node@v3
Found in cache @ /opt/hostedtoolcache/node/18.20.5/x6[4](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:4)
Environment details
/usr/local/bin/yarn --version
4.4.0
/usr/local/bin/yarn config get cacheFolder
/home/runner/.yarn/berry/cache
/usr/local/bin/yarn config get enableGlobalCache
true
Cache Size: ~421 MB (441369819 B)
/usr/bin/tar -xf /home/runner/work/_temp/883eae34-9b21-4684-96[5](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:5)8-cdb937f62965/cache.tzst -P -C /home/runner/work/twenty/twenty --use-compress-program unzstd
Cache restored successfully
Cache restored from key: node-cache-Linux-yarn-a8c18b6600f1bc685e60192f39a0a0c5c51b918829090129d3787302789547fc
Run actions/cache@v3
Cache Size: ~506 MB (530305961 B)
/usr/bin/tar -xf /home/runner/work/_temp/becd3767-ac80-4ca9-8d[21](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:23)-93fa61e3070c/cache.tzst -P -C /home/runner/work/twenty/twenty --use-compress-program unzstd
Cache restored successfully
Cache restored from key: root-node_modules-a8c18b6600f1bc685e60192f39a0a0c5c51b918829090129d378730[27](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:30)89547fc
```
Where we can see in details that a cache is hit twice.
```
Cache Size: ~421 MB (441369819 B)
Cache Size: ~506 MB (530305961 B)
```
## Suggestion
We should stick to only one deps caching mechanism.
Also caching the node_modules folder is not
[recommended](611465405c/examples.md (node---npm)).
That's why I would factorize our caching to the `setup-node` scope only.
## The cache-primary-key crafting
Within the cache-primary-key we will inject the following metrics:
- Node version, as we're caching `node_modules` directly in this way to
avoid cache desync in case we upgrade node.
- Hash of the lockfile, in this way if the lockfile comes to mutate we
will re-install the deps from scratch, in this way no need to
programmatically listen for `changed-files` on the `package.json` and
the `lockfile`
- Strict installation with `check-cache` and `enableHardenedMode` to
true and obviously the `--immutable` flag ( note adding the
`--immutable-cache` by principle even if on paper is overkill as a cache
restoration and install should never occurs )
## Motivations
This is not because we've hit the `actions/cache/restore` cache that NX
won't be re-building sub commands.
Which means the grain to save or not the cache should be extracted from
nx sub commands and not from the `actions/cache` outputs.
We should investigate on the way to retrieve this granularity level
From the moment we will be saving everything, which could result
sometimes in duplicating cache instances.
We should keep in mind that our cache occurrences have a 1 day retention
date, is low cost and pretty fast to perform
## Introduction
Unless I'm mistaken neither the `inputs.type` or `inputs.type` syntax
exists for `composite actions`.
By default it seems like any `Composite action` inputs are considered as
`string`.
```yml
name: Nx Affected CI
inputs:
parallel:
required: false
types: [number]
default: 3
tag:
required: false
types: [string]
tasks:
required: true
types: [string]
```
## Suggestion
To avoid misunderstanding of our composite actions inputs signature I
would recommend removing any `type` props but also default any inputs as
`strings`
## Misc
- Please find related GitHub community discussion
https://github.com/orgs/community/discussions/65588
## Introduction
In this previous PR https://github.com/twentyhq/twenty/pull/9448 we've
refactored the storybook build caching flow to be using the new
[actions/cache](https://github.com/actions/cache) `restore` and `save`
functions, which significantly improve caching operations duration.
In this way, in this PR, we've standardize both of the `restore` and
`save` by refactoring the `task-cache` composite action. By creating two
new composite actions `save-cache` and `restore-cache` that centralize
the paths to cache and the way to compute the primary key.
## Misc
- **If no cache** is hit, then a job duration will long for its task
duration and nothing else, previously the cache upload would sometimes
take up to 3 mins.
- **if cache** is hit, then mainly the only time consuming step is the
dependencies installation ( which is theory is also cached, in fact
twice. We will be having a look on this issue in an upcoming PR )
Solves [ https://github.com/twentyhq/private-issues/issues/214 ]
**TLDR**
Add unit and integration tests to Billing. First approach to run jest
integration tests directly from VSCode.
**In order to run the unit tests:**
Run unit test using the CLI or with the jest extension directly from
VSCode.
**In order to run the integration tests:**
Ensure that your database has the billingTables. If that's not the case,
migrate the database with IS_BILLING_ENABLED set to true:
` npx nx run twenty-server:test:integration
test/integration/billing/suites/billing-controller.integration-spec.ts`
**Doing:**
- Unit test on transformSubscriptionEventToSubscriptionItem
- More tests cases in billingController integration tests.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
