# Introduction
Running `storybook` with `configuration=pages` outputs a coverage that
is not replicated afterwards by the `storybook:coverage` command.
```sh
npx nx storybook:serve-and-test:static twenty-front --configuration=pages --shard=1/1 --checkCoverage=true
# ...
[TEST] Test Suites: 40 passed, 40 total
[TEST] Tests: 52 passed, 52 total
[TEST] Snapshots: 0 total
[TEST] Time: 84.786 s
[TEST] Ran all test suites.
[TEST] Coverage file (13067196 bytes) written to .nyc_output/coverage.json
[TEST] > nx storybook:coverage twenty-front --coverageDir=coverage/storybook --checkCoverage=true
[TEST]
[TEST]
[TEST] > nx run twenty-front:"storybook:coverage" --coverageDir=coverage/storybook --checkCoverage=true
[TEST]
[TEST] > npx nyc report --reporter=lcov --reporter=text-summary -t coverage/storybook --report-dir coverage/storybook --check-coverage=true --cwd=packages/twenty-front
[TEST]
[TEST]
[TEST] =============================== Coverage summary ===============================
[TEST] Statements : 70.45% ( 775/1100 )
[TEST] Branches : 45.39% ( 197/434 )
[TEST] Functions : 63.52% ( 209/329 )
[TEST] Lines : 71.28% ( 767/1076 )
[TEST] ================================================================================
[TEST]
```
```sh
> npx nyc report --reporter=lcov --reporter=text-summary -t coverage/storybook --report-dir coverage/storybook --check-coverage=true --cwd=packages/twenty-front
=============================== Coverage summary ===============================
Statements : 37.4% ( 9326/24931 )
Branches : 22.99% ( 2314/10063 )
Functions : 28.27% ( 2189/7741 )
Lines : 37.81% ( 9261/24488 )
================================================================================
ERROR: Coverage for lines (37.81%) does not meet global threshold (39%)
ERROR: Coverage for branches (22.99%) does not meet global threshold (23%)
ERROR: Coverage for statements (37.4%) does not meet global threshold (39%)
Warning: command "npx nyc report --reporter=lcov --reporter=text-summary -t coverage/storybook --report-dir coverage/storybook --check-coverage=true --cwd=packages/twenty-front" exited with non-zero status code
```
## Fix
Persist configuration scope arg to the `check-coverage` command
## Question
Should we add a step in the `ci-front` what would merge all
`performance,modules,pages` coverage and calculate the `global` coverage
? => I think that this has no plus value as we still compute each of
them individualy
# Introduction
It seems like I've just oversight adding it back while debugging
# Centralization
Please note that we only have access to the following context within a
job matrix properties:
```
Available expression contexts: github, inputs, vars, needs
```
We could centralize the `storybook_scope` as a job `outputs` in order to
avoid this to re-occurs.
# Introduction
Applying the most present `yaml` file extension to all
`.github/workflows/` files
## Notes
Regarding the `restore-cache` and `save-cache` composite actions github
agnostically searches for any `DockerFile` `action.yml` and
`action.yaml` within the target folder when invoking composite actions
as follows:
```yml
- name: Restore storybook build cache
if: steps.changed-files.outputs.any_changed == 'true'
uses: ./.github/workflows/actions/restore-cache
with:
key: ${{ env.STORYBOOK_BUILD_CACHE_KEY }}
```
# Introduction
Unless I'm mistaken in the existing `yarn-install` composite action
`dependencies` are cached twice.
## Cached through `setup-node`
As we're providing `yarn` value to cache inputs.
Setup node will cache the global `.yarn`
```yml
- name: Setup Node.js and get yarn cache
uses: actions/setup-node@v3
with:
node-version: 18
cache: yarn
```
## Programmatic `node_modules` caching
We even manually still cache `node_modules` folder using:
```yml
- name: Cache node modules
id: node-modules-cache
uses: actions/cache@v3
with:
path: |
node_modules
packages/*/node_modules
key: root-node_modules-${{ hashFiles('yarn.lock') }}
restore-keys: root-node_modules-
```
By the way it seems like that the `yarn.lock` file is useless as the
restore-key pattern omits it
This could, unless I'm mistaken, leads to invalid cache restore
## Runtime
The current infra results in the following install deps logs:
```sh
Prepare all required actions
Getting action download info
Download action repository 'actions/setup-node@v3' (SHA:1a444[2](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:2)cacd436585916779262731d5b162bc6ec7)
Download action repository 'actions/cache@v[3](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:3)' (SHA:f4b3439a656ba812b8cb417d2d49f9c810103092)
Run ./.github/workflows/actions/yarn-install
Run actions/setup-node@v3
Found in cache @ /opt/hostedtoolcache/node/18.20.5/x6[4](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:4)
Environment details
/usr/local/bin/yarn --version
4.4.0
/usr/local/bin/yarn config get cacheFolder
/home/runner/.yarn/berry/cache
/usr/local/bin/yarn config get enableGlobalCache
true
Cache Size: ~421 MB (441369819 B)
/usr/bin/tar -xf /home/runner/work/_temp/883eae34-9b21-4684-96[5](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:5)8-cdb937f62965/cache.tzst -P -C /home/runner/work/twenty/twenty --use-compress-program unzstd
Cache restored successfully
Cache restored from key: node-cache-Linux-yarn-a8c18b6600f1bc685e60192f39a0a0c5c51b918829090129d3787302789547fc
Run actions/cache@v3
Cache Size: ~506 MB (530305961 B)
/usr/bin/tar -xf /home/runner/work/_temp/becd3767-ac80-4ca9-8d[21](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:23)-93fa61e3070c/cache.tzst -P -C /home/runner/work/twenty/twenty --use-compress-program unzstd
Cache restored successfully
Cache restored from key: root-node_modules-a8c18b6600f1bc685e60192f39a0a0c5c51b918829090129d378730[27](https://github.com/twentyhq/twenty/actions/runs/12770942233/job/35596946506#step:6:30)89547fc
```
Where we can see in details that a cache is hit twice.
```
Cache Size: ~421 MB (441369819 B)
Cache Size: ~506 MB (530305961 B)
```
## Suggestion
We should stick to only one deps caching mechanism.
Also caching the node_modules folder is not
[recommended](611465405c/examples.md (node---npm)).
That's why I would factorize our caching to the `setup-node` scope only.
## The cache-primary-key crafting
Within the cache-primary-key we will inject the following metrics:
- Node version, as we're caching `node_modules` directly in this way to
avoid cache desync in case we upgrade node.
- Hash of the lockfile, in this way if the lockfile comes to mutate we
will re-install the deps from scratch, in this way no need to
programmatically listen for `changed-files` on the `package.json` and
the `lockfile`
- Strict installation with `check-cache` and `enableHardenedMode` to
true and obviously the `--immutable` flag ( note adding the
`--immutable-cache` by principle even if on paper is overkill as a cache
restoration and install should never occurs )
## Motivations
This is not because we've hit the `actions/cache/restore` cache that NX
won't be re-building sub commands.
Which means the grain to save or not the cache should be extracted from
nx sub commands and not from the `actions/cache` outputs.
We should investigate on the way to retrieve this granularity level
From the moment we will be saving everything, which could result
sometimes in duplicating cache instances.
We should keep in mind that our cache occurrences have a 1 day retention
date, is low cost and pretty fast to perform
## Introduction
Unless I'm mistaken neither the `inputs.type` or `inputs.type` syntax
exists for `composite actions`.
By default it seems like any `Composite action` inputs are considered as
`string`.
```yml
name: Nx Affected CI
inputs:
parallel:
required: false
types: [number]
default: 3
tag:
required: false
types: [string]
tasks:
required: true
types: [string]
```
## Suggestion
To avoid misunderstanding of our composite actions inputs signature I
would recommend removing any `type` props but also default any inputs as
`strings`
## Misc
- Please find related GitHub community discussion
https://github.com/orgs/community/discussions/65588
## Introduction
In this previous PR https://github.com/twentyhq/twenty/pull/9448 we've
refactored the storybook build caching flow to be using the new
[actions/cache](https://github.com/actions/cache) `restore` and `save`
functions, which significantly improve caching operations duration.
In this way, in this PR, we've standardize both of the `restore` and
`save` by refactoring the `task-cache` composite action. By creating two
new composite actions `save-cache` and `restore-cache` that centralize
the paths to cache and the way to compute the primary key.
## Misc
- **If no cache** is hit, then a job duration will long for its task
duration and nothing else, previously the cache upload would sometimes
take up to 3 mins.
- **if cache** is hit, then mainly the only time consuming step is the
dependencies installation ( which is theory is also cached, in fact
twice. We will be having a look on this issue in an upcoming PR )
Solves [ https://github.com/twentyhq/private-issues/issues/214 ]
**TLDR**
Add unit and integration tests to Billing. First approach to run jest
integration tests directly from VSCode.
**In order to run the unit tests:**
Run unit test using the CLI or with the jest extension directly from
VSCode.
**In order to run the integration tests:**
Ensure that your database has the billingTables. If that's not the case,
migrate the database with IS_BILLING_ENABLED set to true:
` npx nx run twenty-server:test:integration
test/integration/billing/suites/billing-controller.integration-spec.ts`
**Doing:**
- Unit test on transformSubscriptionEventToSubscriptionItem
- More tests cases in billingController integration tests.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
We will remove the `twenty-postgres` image that was used for local
development and only use `twenty-postgres-pilo` (which we use in prod),
bringing the development environment closer to prod and avoiding having
to maintain 2 images.
Instead of provisioning the super user after the db initialization, we
directly rely on the superuser provided by Spilo for simplicity. We also
introduce a change that tries to create the right database (`default` or
`test`) based on the context.
How to test:
```
docker build -t twentycrm/twenty-postgres-spilo:latest -f ./packages/twenty-docker/twenty-postgres-spilo/Dockerfile .
docker images --no-trunc | grep twenty-postgres-spilo
postgres-on-docker:
docker run \
--name twenty_pg \
-e PGUSER_SUPERUSER=twenty \
-e PGPASSWORD_SUPERUSER=twenty \
-e ALLOW_NOSSL=true \
-v twenty_db_data:/home/postgres/pgdata \
-p 5432:5432 \
REPLACE_WITH_IMAGE_ID
```
Run the CI integrationin sync mode
and add the option to run it without db reset
cleaning all the useless integration test
---------
Co-authored-by: guillim <guillaume@twenty.com>
https://github.com/twentyhq/private-issues/issues/75
**TLDR**
Add twenty-server package, wich contains the last tinybird datasources
and pipes used in analytics.
This new version of the API endpoints, pipes and datasources are
inspired by the implementation of dub.co
(https://github.com/dubinc/dub/blob/main/packages/tinybird/).
It contains the webhooks analytics, serverless functions duration and
serverless functions error count analytics. As well as
**In order to test**
- Follow the instructions in the README.md on twenty-tinybird using your
admin token from twenty_analytics_playground if you want to modify them.
- For a better experience add the extension Tinybird support for VsCode
- If you want more info about datasources and pipes please read the
tinybird docs.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
TLDR:
Secure connexion between tinybird and twenty using jwt when accessing
datasource from tinybird.
Solves:
https://github.com/twentyhq/private-issues/issues/73
In order to test:
1. Set ANALYTICS_ENABLED to true
2. Set TINYBIRD_JWT_TOKEN to the ADMIN token from the workspace
twenty_analytics_playground
3. Set TINYBIRD_JWT_TOKEN to the datasource or your admin token from the
workspace twenty_analytics_playground
4. Create a Webhook in twenty and set wich events it needs to track
5. Run twenty-worker in order to make the webhooks work.
6. Do your tasks in order to populate the data
7. Enter to settings> webhook>your webhook and the statistics section
should be displayed.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This Pull Request addresses the need to optimize our Continuous
Integration (CI) workflows for Playwright tests and release processes.
The changes implemented aim to reduce unnecessary resource usage by
conditionally executing jobs based on relevant file changes and
Implement https://github.com/tj-actions/changed-files step
## Changes logs
- Updated `ci-test-docker-compose.yaml , ci-chrome-extension.yaml ` to
check for changed files before running tests.
- Updated `ci-front.yaml , ci-utils.yaml , ci-website.yaml ,
ci-server.yaml` to check for changed files before running tests.
- Enhanced `playwright.yml` to skip unnecessary tests based on file
changes.
