This PR changes the way we do automatching in the import feature.
It uses [Fuse.js](https://www.fusejs.io/) to do a fuzzy text search on
fields and sub-fields.
The labels of sub-fields are now derived from the common config constant
we have for sub-fields.
This PR removes use-context-selector completely, so that any bug
associated with state synchronization between recoil and
use-context-selector disappears.
There might be a slight performance decrease on the table, but since we
have already improved the average performance per line by a lot, and
that the performance bottleneck right now is the fetch more logic and
the windowing solution we use, it is not relevant.
Also the DX has become so hindered by this parallel state logic recently
(think [cache
invalidation](https://martinfowler.com/bliki/TwoHardThings.html)), that
the main benefit we gain from this removal is the DX improvement.
Fixes https://github.com/twentyhq/twenty/issues/12123
Fixes https://github.com/twentyhq/twenty/issues/12109
related to https://github.com/twentyhq/core-team-issues/issues/601
## Done
- add a `onDbEvent` `Subscription` graphql endpoint to listen to
database_event using what we have done with webhooks:
- you can subscribe to any `action` (created, updated, ...) for any
`objectNameSingular` or a specific `recordId`. Parameters are nullable
and treated as wildcards when null.
- returns events with following shape
```typescript
@Field(() => String)
eventId: string;
@Field()
emittedAt: string;
@Field(() => DatabaseEventAction)
action: DatabaseEventAction;
@Field(() => String)
objectNameSingular: string;
@Field(() => GraphQLJSON)
record: ObjectRecord;
@Field(() => [String], { nullable: true })
updatedFields?: string[];
```
- front provide a componentEffect `<ListenRecordUpdatesEffect />` that
listen for an `objectNameSingular`, a `recordId` and a list of
`listenedFields`. It subscribes to record updates and updates its apollo
cached value for specified `listenedFields`
- subscription is protected with credentials
## Result
Here is an application with `workflowRun`
https://github.com/user-attachments/assets/c964d857-3b54-495f-bf14-587ba26c5a8c
---------
Co-authored-by: prastoin <paul@twenty.com>
Recoil-sync was causing issues with Firefox, replacing it with a simpler
mechanism to hydrate variables on page load
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
## What
- Deprecate overlayscrollbars as we decided to follow the native
behavior
- rework on performances (avoid calling recoil states too much at field
level which is quite expensive)
- Also implements:
https://github.com/twentyhq/core-team-issues/issues/569
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
# Introduction
closes https://github.com/twentyhq/core-team-issues/issues/591
Same than for `twenty-shared` made in
https://github.com/twentyhq/twenty/pull/11083.
## TODO
- [x] Manual migrate twenty-website twenty-ui imports
## What's next:
- Generate barrel and migration script factorization within own package
+ tests
- Refactoring using preconstruct ? TimeBox
- Lint circular dependencies
- Lint import from barrel and forbid them
### Preconstruct
We need custom rollup plugins addition, but preconstruct does not expose
its rollup configuration. It might be possible to handle this using the
babel overrides. But was a big tunnel.
We could give it a try afterwards ! ( allowing cjs interop and stuff
like that )
Stuck to vite lib app
Closed related PRs:
- https://github.com/twentyhq/twenty/pull/11294
- https://github.com/twentyhq/twenty/pull/11203
# Introduction
Lately encountering a lot of out of memory error when running
twenty-front in watch mode with both TypeScript and lint checkers
```ts
Error: Worker terminated due to reaching memory limit: JS heap out of memory
at new NodeError (node:internal/errors:405:5)
at [kOnExit] (node:internal/worker:287:26)
at Worker.<computed>.onexit (node:internal/worker:209:20)
```
The existing configuration looks like this:
```ts
// packages/twenty-front/vite.config.ts
'cd ../.. && eslint packages/twenty-front --report-unused-disable-directives --max-warnings 0 --config .eslintrc.cjs',
```
This is wrong as computing the root eslintrc completely omitting
twenty-front's one ***and its ignorePattern*** so will be checking in
`node_modules` etc checking for project-structure :).
For example this a
[snippet](https://gist.github.com/prastoin/d7f8ad4ef5eb2f7732209b756a38094c)
of the above commands errors. We can see rule that should be disabled by
`eslintrc.react.cjs` extension made from twenty-front `eslintrc` :
```ts
/Users/paulrastoin/ws/twenty-two/packages/twenty-front/src/modules/settings/data-model/fields/forms/components/__stories__/SettingsDataModelFieldSettingsFormCard.stories.tsx
23:27 warning Forbidden non-null assertion @typescript-eslint/no-non-null-assertion
```
## Fixes
- consume the `twenty-front` package eslint configuration within the
vite lint checker
- eslint overrides extends are getting merged based on glob inclusion of
their files declarations
- any linted files should be included in one of our `tsconfig`
- removed redundant and counter-productive negative `ignorePatterns`, as
eslint will naturally only lint files within configuration file
directory by default which will result making it go through local
`node_modules` project structure
## Now
Less cpu usage <3.5 gb and faster
```ts
// from packages/twenty-front
TIMING=1 npx eslint . --report-unused-disable-directives --max-warnings 0 --config .eslintrc.cjs --debug
#...
Rule | Time (ms) | Relative
:-----------------------------------------------|----------:|--------:
project-structure/folder-structure | 19578.927 | 20.2%
prettier/prettier | 13746.156 | 14.2%
no-redeclare | 9546.570 | 9.9%
@nx/workspace-explicit-boolean-predicates-in-if | 8167.805 | 8.4%
@typescript-eslint/no-unused-vars | 6872.803 | 7.1%
import/no-relative-packages | 6577.273 | 6.8%
@nx/enforce-module-boundaries | 6520.945 | 6.7%
import/no-duplicates | 4987.476 | 5.2%
react/no-direct-mutation-state | 2323.082 | 2.4%
react/require-render-return | 1155.261 | 1.2%
```
## Conclusion
Please note that `nx linter` might not be as strict as vite config
eslint runner
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Done :
- move metrics and health cache services from health module to metrics
module
- refactor metrics counter from specific method to set up from enum keys
- add OpenTelemetry (Otel) instrumentation for metrics
- set up Otel SDK to send metrics to Otel collector
To do later :
- implement Otel instrumentation for traces + plug Sentry on top
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
In this PR, I'm specifying to vite build that
'@scalar/api-reference-react' is an external dependency and should not
be considered as a module we maintain (it won't get its own chunk at
build time and we won't generate sourcemaps on our end).
I'm not sure why vite is considering it internal in the first place (I
can see that it's generating .vue.js files, might be the first time we
are relying on a vue library)
This PR introduces Keystatic to let us edit twenty.com's content with a
CMS. For now, we'll focus on creating release notes through Keystatic as
it uses quite simple Markdown. Other types of content will need some
refactoring to work with Keystatic.
https://github.com/user-attachments/assets/e9f85bbf-daff-4b41-bc97-d1baf63758b2
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
## Introduction
This is PR is a suggestion ! And should be discussed
With yarn `^4`, during installation won't raise an error if current dev
env does not satisfies the `engines` policy.
We have usually 10+ contributors support request regarding higher node
version issue per week
I would have preferred a very declarative integration using npm
[engines](https://docs.npmjs.com/cli/v11/configuring-npm/package-json#engines)
but this does not seems to be natively supported by `yarn`
We should keep in mind that this might block any machines from our CICD
if they have diff node version installed ( such as running the project
on a different node version could result in bugs too )
## Implem
Created a yarn [constraints](https://yarnpkg.com/features/constraints)
run after each installation that checking if current node version
satisfies defined engines range ( might also be done for others engines
entries )
I assume we will always have the same engines policy for every packages,
at least that's not a consideration from now
## Further
We could refactor our package.json engines into only one using
`Yarn.set` etc
## Resource
- https://yarnpkg.com/configuration/yarnrc
- https://yarnpkg.com/features/constraints
## Note
- Not running constraints in `preInstall` hook as won't be effective on
fresh install
-
[engine-strict](https://docs.npmjs.com/cli/v8/using-npm/config#engine-strict)
is an npm-config
-
[devEngines](https://docs.npmjs.com/cli/v11/configuring-npm/package-json#devengines)
are npm feature too ( for instance pnpm current PR
https://github.com/pnpm/pnpm/issues/8153 )
## Conclusion
As always any suggestions are more than welcomed !
# Health Monitoring for Self-Hosted Instances
This PR implements basic health monitoring for self-hosted instances in
the admin panel.
## Service Status Checks
We're adding real-time health checks for:
- Redis Connection
- Database Connection
- Worker Status
- Message Sync Status
## Existing Functionality
We already have message sync and captcha counters that store aggregated
metrics in cache within a configurable time window (default: 5 minutes).
## New Endpoints
1. `/healthz` - Basic server health check for Kubernetes pod monitoring
2. `/healthz/{serviceName}` - Individual service health checks (returns
200 if healthy)
3. `/metricsz/{metricName}` - Time-windowed metrics (message sync,
captcha)
4. GraphQL resolver in admin panel for UI consumption
All endpoints use the same underlying service, with different
presentation layers for infrastructure and UI needs.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
We are using useInView to detect if a row should be rendered or not.
# Behavior issue
When browsing an Index page (let's say People), then navigating to
another one (Company), then back to People, the rows were not displayed
anymore. For some reason the inView value was set to false in this case
# Fix
- I have updated the useInView (react-intersection-observer) package but
it did not fix it
- useInView provides a ref. However, I believe this was conflicting with
the draggableRef; cecause we are in a <table> we cannot add additional
containers and are forced to apply both refs to the <tr> (draggableRef +
inViewRef). I believe this was causing the issue. I have added the
inView ref to an empty <td> within the row which is achieving the same
goal without forcing to combine refs
- Do not render a source handle for the leaf nodes
- Upgrade the `@xyflow/react` library
| Before | After |
|--------|--------|
| 
| 
|
## Other options considered
React Flow exposes a hook to get the connections of the current node. I
tried to use this hook – which makes things way simpler – but I couldn't
find a way to make it work in Storybook. I had two options: 1. Set up
React Flow to render the nodes properly, 2. Mock the hook in Storybook.
The first option was hard to achieve as the `<Reactflow />` component
renders a whole flow, and it doesn't play well with the idea of
rendering a single node in a story.
The second option seemed overkill as mocking modules with Storybook is
not straightforward. See
https://storybook.js.org/docs/writing-stories/mocking-data-and-modules/mocking-modules.
I chose to keep the initial version of my code, written before I spot a
function simplifying the code. We can give it a look another time.
One of the steps to address #8128
How to test:
Please change the locale in the settings and click on change password
button. A password reset email in the preferred locale will be sent.
Todo:
- Remove the hardcoded locales for invitation, warn suspended workspace
email, clean suspended workspace emails
- Need to test invitation, email verification, warn suspended workspace
email, clean suspended workspace emails
- The duration variable `5 minutes` is always in english. Do we need to
do something about that? It does seems odd in case of chinese
translations.
Notes:
- Only tested the password reset , password update notify templates.
- Cant test email verification due to error during sign up `Internal
server error: New workspace setup is disabled`
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
# In this PR
- Allow to register a custom domain
- Refacto subdomain generation
# In other PRs
- Add UI to deal with a custom domain
- Add logic to work with custom domain
In this PR:
- removing rootDir / baseUrl from any tsconfig.json
- we need to keep it in tsconfig.spec.json and also specify rootDir in
jest.config.ts moduleMapper because of the way nx jest executor works
(automatically moving back to root)
- we need to explictly specify the depencies to twenty-shared /
twenty-emails (built packages) in packages package.json to help nx
understand dependencies
Refers #8128
Changes Introduced:
- Added i18n configuration.
- Added a feature flag for localization.
- Enabled language switching based on the flag.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
