WorkspaceMemberId is mandatory in the jwt token generated for a given
user on a given workspace.
However, when a user signs up, it does not have a workspaceMemberId yet.
Fix
[#web](https://github.com/orgs/twentyhq/projects/1/views/3?pane=issue&itemId=75329194)
This PR does 2 things:
- migrate webhooks to TwentyORM
- Fix inversion between objectNameSingular and operation in webhook
eventName. It is stored as {objectNameSingular}.{operation} and we were
querying {operation}.{objectNameSingular}
We have found the root cause of the issue:
- when using a datasource (including the cached ones), we are fetching
ObjectMetadataCollection from cache (700kB). Datasource usage is
happening any time we are using twentyORM, which is everywhere in the
jobs and in some resolvers (including the GetCurrentUser one). This is
leading to a high load on redis and leading to the performance issues we
are seeing.
- we actually don't need to fetch this objectMetadataCollection while
using a cached datasource, only when we instantiate a new one
## Context
As we grow, the messaging scripts are experiencing performance issues
forcing us to temporarily disable them on the cloud.
While investigating the performance, I have noticed that generating the
entity schema (for twentyORM) in the repository is taking ~500ms locally
on my Mac M2 so likely more on pods. Caching the entitySchema then!
I'm also clarifying naming around schemaVersion and cacheVersions ==>
both are renamed workspaceMetadataVersion and migrated to the workspace
table (the workspaceCacheVersion table is dropped).
This PR introduces the following changes:
- Add the ability to filter webhooks by objectSingularName and Actions
- Refactor SettingsWebhookDetails edition to not use react-hook-form
(which will be deprecated on the whole project)
- Updating the tests with a complex set of mock (we just need to fix ~30
of them now :D)
<img width="1053" alt="image"
src="https://github.com/user-attachments/assets/4e56d972-f129-4789-8d1c-4b5797a8ffd7">
In this PR:
- adding Favorites to Tasks and Notes
- fixing inconsistencies between custom object creation and sync of
standard fields of custom objects
- fixing workspaceCacheVersion not used to invalidate existing
datasource
In this PR:
- refactoring auth module to extract a jwt module that can be re-used
from other part of the app (avoiding circular dependencies file module
=> auth => file (file and auth both need jwt actually)
- activating imageIdentfier on person on workspace creation (this will
put back the images on people)
- fixing picture upload (we were missing some fileToken)
In this PR, I'm:
- adding createdBy field (type ACTOR) on custom objects when created
- moving `name` and `position` default column to the set of columns
automatically creation on object creation
- fixing a bug on mutations (update / create), if the targetted object
has a 'data' custom field, it was conflicting with the payload ==> I
feel we need to refactor this part of the code but we can keep this for
a bit later as we plan to move out of pg_graphql
<img width="1198" alt="image"
src="https://github.com/user-attachments/assets/891c4a97-bab1-415c-8551-dabd5996a794">
As per our guideline to maintain a smooth migration to the new minor
versions, this command is backfilling existing workspaces with the 3
userVars used to keep track of user onboarding:
```
ONBOARDING_CONNECT_ACCOUNT_COMPLETE = 'ONBOARDING_CONNECT_ACCOUNT_COMPLETE',
ONBOARDING_INVITE_TEAM_COMPLETE = 'ONBOARDING_INVITE_TEAM_COMPLETE',
ONBOARDING_CREATE_PROFILE_COMPLETE = 'ONBOARDING_CREATE_PROFILE_COMPLETE',
```
An ACTIVE workspace is a workspace that has a complete workspaceSchema
and is authorized to be browsed by users.
In this PR, I'm:
- introducing a new activationStatus: PENDING_CREATION (existing ACTIVE
/ INACTIVE)
- removing workspaceService.isWorkspaceActivated (based on
workspaceSchema existence which is not robust and checking
activationStatus.ACTIVE instead)
- removing dynamic activationStatus field on worksapce resolver (we can
use the postgres column directly now that data has been migrated)
- on user sign up creating the workspace in PENDING_CREATION, and on
workspace activation setting it to ACTIVE
- only re-activating a workspace if the current activationStatus is
INACTIVE through billing webhooks (a PENDING_CREATION should stay
PENDING and ACTIVE should stay ACTIVE)
In this PR, I'm fixing two issues that we have faced:
- computing a rich text first line in case of the first block content is
not a text
- migrating existing timelineActivities tied to tasks / notes to
linked-tasks / linked-notes during migration command
In this PR, I am:
- removing a useEffect (we use the onCompleted prop of useFindManyQuery
hook)
- moving this logic into a RecordShowPageHeader to avoid re-renders of
the whole tree
This PR refactors the ORM-Manager to simplify and unify the datasource
creation. I'm deprecating all usages if InjectWorkspaceDatasource and
InjectWorkspaceRepository as we can't be sure they are up-to-date
## Bug Description
We are facing a bug in case recaptcha is enabled.
To reproduce:
- Create your recaptcha: https://www.google.com/recaptcha/about/
- update your server .env with the following variables:
```
CAPTCHA_SECRET_KEY=REPLACE_ME
CAPTCHA_SITE_KEY=REPLACE_ME
CAPTCHA_DRIVER=google-recaptcha
```
- Go to the login page, enter an existing user email and hit 'Reset your
password'.
- Add a console.log in emailPasswordResetLink in auth.resolver.ts to get
the token that would be sent by email if you don't have the mailer setup
- Browse: /reset-password/{passwordToken}
- Update the password:
<img width="1446" alt="image"
src="https://github.com/user-attachments/assets/dd5b077f-293e-451a-8630-22d24ac66c42">
- See that the token is invalid
You should see two calls in your developer network tab. A successful one
to update the password and another to log you in. This 2nd call
(Challenge) does not have the captcha token provided. It should be
## Fix
- Refreshing the token on page load
- providing it to the Challenge graphql call
This PR fixes a few bugs on TwentyORM:
- fix many to one relations that were not properly queries
- fix many to one relations that were not properly parsed
- compute datasource (or use from cache) at run-time and do not use
injected one that could be outdated
We still need to refactor it to simplify, I feel the API are too complex
and we have too many cache layers. Also the relation computation part is
very complex and bug prone
<img width="1512" alt="image"
src="https://github.com/user-attachments/assets/9fcdd5ca-4329-467c-ada8-4dd5d45be259">
Open questions:
- the Tag component does not match Figma in term of style and API for
"transparent" | "outline". We need to discuss with @Bonapara what is the
desired behavior here
- right now opportunity.stage is not nullable. We need to discuss with
@FelixMalfait and @Bonapara what we want here. I would advocate to make
a it nullable for now until we introduce settings on select fields.
custom select are nullable and it could be confusing for the user
Follow up:
- enhance tests on Tags
- add story to cover the No Value column on record board
Using a portal in dropdown systematically can be an issue in case we are
having dropdown within dropdown. The useClickOutside listener will be
triggered. It's easier to usePortal only in the case we really need
them, which is quite rare
Recently, we've forced all dropdown menu to be displayed in portal. This
loses the z-index hieararchy structure and forces us to specify a higher
z-index in order for dropdown menus to be displayed on top
In this PR:
- Renaming SettingsAccountsEmailBlocklist to
SettingsAccountsEmailBlocklist as the blocklist is not tied to
emails/messaging but is user level
- Changing the UI settings UI by removing /emails/{id} page and adding
tabs on /emails page
<img width="1512" alt="image"
src="https://github.com/twentyhq/twenty/assets/12035771/d215a891-fff9-477d-915d-0d7a697742e8">
We have recently deprecated our subscriptionStatus on workspace to
replace it by a check on existing subscription (+ freeAccess
featureFlag) but the logic was not properly implemented
In this PR, I'm simplifying storybook setup:
1) Remove build --test configuration that prevent autodocs. We are not
using autodocs at all (the dev experience is not good enough), so I have
completely disabled it.
2) Clarify `serve` vs `test` vs `serve-and-test` configurations
After this PR:
- you can serve storybook in two modes: `npx nx run
twenty-front:storybook:serve:dev` and `npx nx run
twenty-front:storybook:serve:static`
- you can run tests agains an already served storybook (this is useful
in dev so you don't have to rebuild everytime to run tests): `npx nx run
twenty-front:storybook:test`
- you can conbine both: `npx nx run
twenty-front:storybook:serve-and-test:static`
As per title!
Also, I'm removing an incorrect logic in the enum migration runner that
takes care of the case where we have no defaultValue but non nullable
which is not a valid business case.
We had an issue on infinite scroll on table view.
The fetch more logic was modifying isTableLastRowVisible state (which is
wrong, how could it know)? This was done to prevent loading too much
data at once. This was causing some race condition on
isTableLastRowVisible (as the table itself was also changing it
depending on the real visibility of the line)
I have remove this hacky usage of isTableLastRowVisible and replaced it
by a setTimeout to let the user some time to scroll and introduce a
throttle logic.
Our tests on FE are red, which is a threat to code quality. I'm adding a
few unit tests to improve the coverage and lowering a bit the lines
coverage threshold
In this PR, I'm doing 2 things:
- refresh connectedAccount token on message-list-fetch. It's currently
only refresh while doing the messages-import. However messages-import
stage are only triggered if new messages are detected (which could take
days or week depending of the messageChannel activity). We should also
refresh it while trying to fetch the list
- handle Unhandled Gmail error code 500 with reason "backendError".
These can occur on gmail side. In this case, we just retry later.
In this PR, I'm mainly doing two things:
- uniformizing messaging-messages-import and
messaging-message-list-fetch behaviors (cron.job and job)
- improving performances of these cron.jobs by not triggering the jobs
if the stage is not relevant
- making sure these jobs have same signature (workspaceId +
messageChannelId)
In this PR, I'm refactoring the messaging module into smaller pieces
that have **ONE** responsibility: import messages, clean messages,
handle message participant creation, instead of having ~30 modules (1
per service, jobs, cron, ...). This is mandatory to start introducing
drivers (gmails, office365, ...) IMO. It is too difficult to enforce
common interfaces as we have too many interfaces (30 modules...). All
modules should not be exposed
Right now, we have services that are almost functions:
do-that-and-this.service.ts / do-that-and-this.module.ts
I believe we should have something more organized at a high level and it
does not matter that much if we have a bit of code duplicates.
Note that the proposal is not fully implemented in the current PR that
has only focused on messaging folder (biggest part)
Here is the high level proposal:
- connected-account: token-refresher
- blocklist
- messaging: message-importer, message-cleaner, message-participants,
... (right now I'm keeping a big messaging-common but this will
disappear see below)
- calendar: calendar-importer, calendar-cleaner, ...
Consequences:
1) It's OK to re-implement several times some things. Example:
- error handling in connected-account, messaging, and calendar instead
of trying to unify. They are actually different error handling. The only
things that might be in common is the GmailError => CommonError parsing
and I'm not even sure it makes a lot of sense as these 3 apis might have
different format actually
- auto-creation. Calendar and Messaging could actually have different
rules
2) **We should not have circular dependencies:**
- I believe this was the reason why we had so many modules, to be able
to cherry pick the one we wanted to avoid circular deps. This is not the
right approach IMO, we need architect the whole messaging by defining
high level blocks that won't have circular dependencies by design. If we
encounter one, we should rethink and break the block in a way that makes
sense.
- ex: connected-account.resolver is not in the same module as
token-refresher. ==> connected-account.resolver => message-importer (as
we trigger full sync job when we connect an account) => token-refresher
(as we refresh token on message import).
connected-account.resolver and token-refresher both in connected-account
folder but should be in different modules. Otherwise it's a circular
dependency. It does not mean that we should create 1 module per service
as it was done before
In a nutshell: The code needs to be thought in term of reponsibilities
and in a way that enforce high level interfaces (and avoid circular
dependencies)
Bonus: As you can see, this code is also removing a lot of code because
of the removal of many .module.ts (also because I'm removing the sync
scripts v2 feature flag end removing old code)
Bonus: I have prefixed services name with Messaging to improve dev xp.
GmailErrorHandler could be different between MessagingGmailErrorHandler
and CalendarGmailErrorHandler for instance
RelationFieldDisplay was estabilishing a dependency on
RecordTableContext which is not right as FieldDisplay can be loaded
outside of RecordTable context
I'm using an util directly but understand this is a bit heavier than
before in term of performance. If we want to pre-compute this, we will
need to be a bit smarter.
Also the previous code based on fieldName was not right, we should check
relationObjectMetadataItem instead
Signing in is a two steps process:
- Checking if the user email is already used
- Checking if the email + password is correct
Those two steps need a captchaToken to be valid. Unfortunately, from
Cloudflare Turnstile doc:
`A token can only be validated once and cannot be consumed twice. Once a
token has been issued, it can be validated within the next 300 seconds.
After 300 seconds, the token is no longer valid and another challenge
needs to be solved`
So we need to generate a new token at each step instead of re-using the
same
In this PR, we are refactoring two things:
- leverage field.defaultValue for Select and MultiSelect settings form
(instead of option.isDefault)
- use quoted string (ex: "'USD'") for string default values to embrace
backend format
---------
Co-authored-by: Thaïs Guigon <guigon.thais@gmail.com>
A user has reported an issue with REST API.
We have recently migrated the graphql IDs from UUID to ID type. As Rest
API is leveraging the graphql API under the hood, the Rest API query
builder should be updated accordingly
In this PR, I'm refactoring the way we associate messageParticipant post
person/company creation. Instead of looking a all person without
participant, we are passing the one that were just created.
Also, I'm making sure the message and messageParticipant creation
transaction is commited before creating person/company creation (and
then messageParticipant association)
This PR fixes several issues:
- enum naming should be: {tableName}_{fieldName}_enum and respecting the
case
- defaultValue format handled in the FE should respect the one in the BE
In my opinion we should refactor the defaultValue:
- we should respect backend format: "'myDefault'" for constant default
and "0" for float, "now" for expressions, "true" for booleans. we can
rename it to defaultValueExpression if it is more clear but we should
not maintain a parallel system
- we should deprecate option: isDefaultValue which is confusing
- we should re-work backend to have a more unified approach between
fields and avoid having if everywhere about select, multiselect, and
currency cases. one unified "computeDefaultValue" function should do the
job
What is still broken:
- currency default Value on creation. I think we should do the refactor
first
- select default value edition.
These cases do not break the schema but are ignored currently
The whole viewBar component was re-rendered on view changes which was
introducing performance issue. The need was to compute page title, this
should be done in a lower level component
While using middleware (executed pre-graphql) for graphql endpoint, we
need to swallow exception and return errors with a 200. Otherwise it's
not a valid graphql response
## Query depth deprecation
I'm deprecating depth parameter in our graphql query / cache tooling.
They were obsolete since we introduce the possibility to provide
RecordGqlFields
## Refactor combinedFindManyRecordHook
The hook can now take an array of operationSignatures
## Fix tasks issues
Fix optimistic rendering issue. Note that we still haven't handle
optimisticEffect on creation properly
## Context
We recently enabled the option to bypass SSL certificate authority
validation when establishing a connection to PostgreSQL. Previously, if
this validation failed, the server would revert to unencrypted traffic.
Now, it maintains encryption even if the SSL certificate check fails. In
the process, we overlooked a few DataSource setups, prompting a review
of DataSource creation within our code.
## Current State
Our DataSource initialization is distributed as follows:
- **Database folder**: Contains 'core', 'metadata', and 'raw'
DataSources. The 'core' and 'metadata' DataSources manage migrations and
static resolver calls to the database. The 'raw' DataSource is utilized
in scripts and commands that require handling both aspects.
- **typeorm.service.ts script**: These DataSources facilitate
multi-schema connections.
## Vision for Discussion
- **SystemSchema (formerly core) DataSource**: Manages system schema
migrations and system resolvers/repos. The 'core' schema will be renamed
to 'system' as the Core API will include parts of the system and
workspace schemas.
- **MetadataSchema DataSource**: Handles metadata schema migrations and
metadata API resolvers/repos.
- **(Dynamic) WorkspaceSchema DataSource**: Will be used in the Twenty
ORM to access a specific workspace schema.
We currently do not support cross-schema joins, so maintaining these
DataSources separately should be feasible. Core API resolvers will
select the appropriate DataSource based on the field context.
- **To be discussed**: The potential need for an AdminDataSource (akin
to 'Raw'), which would be used in commands, setup scripts, and the admin
panel to connect to any database schema without loading any model. This
DataSource should be reserved for cases where utilizing metadata,
system, or workspace entities is impractical.
## In This PR
- Ensuring all existing DataSources are compliant with the SSL update.
- Introducing RawDataSource to eliminate the need for declaring new
DataSource() instances in commands.
In this PR:
- Follow up on #5170 as we did not take into account not logged in users
- only apply throttler on root fields to avoid performance overhead
An error has been recently introduced in the sync of fieldMetadata. This
PR fixes it
Additionnally, we are enabling email for trialing and past_due
workspaces. There is an ongoing work to introduce a more robust
activationStatus on workspace.
While trying to migrate a workspace from 0.3.3 to 0.10.0, we've faced an
issue with the script to migrate default-values format.
This PR fixes it.
We really need to add tests on this part ;)
## Context
For users with many records, only the first n*60 records were loaded on
board views (n being the number of visible columns). This was because of
the following behavior:
- watch for end of column visibility changes. If an end of column is
visible, try to fetch more. However, watching for visbility changes is
not reliable enough.
## What we want
If an end of column is visible, try to fetch more. If no more records is
availble in pagination, do not fetch more
## Context
We have recently added an event listener to create audit logs on objects
update. However, we have only created the structure (relations on event
standard objects) for Company, Person, Opportunity and custom objects.
There is a larger effort in #4936 to refactor this.
For now, we are disabling log auditing on all other objects
## How
Add @IsNotAuditLogged() annotation on all standard objects except
Company, Person, Opportunity
Unfortunately, it is not possible in CSS to have an overflow:visible
over x-axis while having an overflow:hidden over y-axis, leading to the
following issue:
<img width="1512" alt="image"
src="https://github.com/twentyhq/twenty/assets/12035771/9b84cbbb-c6c4-4fd6-a630-a24f01eccf73">
I'm refactoring the RecordInlineCell and RecordTableCell to use
useFloating + createPortal to open the cell.
We have recently discovered that we were using ID type in place of UUID
type in many place in the code.
We have merged #4895 but this introduced bugs as we forgot to replace it
everywhere
We have discovered that GraphQL inputs for fields of type ids in create
/ update input where using a more permissive ID type than the type used
in FilterInput in queries.
This PRs fixes it and make sure that all Input are using UUID graphql
scalar types
## Removing repetitive queries (impacting performance on page load)
We have recently introduced the capability to detect schema version
mismatch. To do that, we add a new header in all our queries. On page
load, this header is added once we know the currentUser and especially
its currentWorkspace and related schema version.
However, applying this header to apollo client will re-trigger all
queries that have been already performed (GetClientConfig and
GetCurrentUser). To avoid re-triggering them, I'm introducing two new
"isLoaded" states and skip the query if the query has already been
performed
## Fixing Relation Detail not displaying data on show page
Small bug introduced in a previous PR
In this PR:
- fix empty list placeholder positionning
- prevent user from erasing custom address field as composite types
removal is not supported yet @ijreilly FYI
- fix show page relation error
- Implement address filter
Charles Bochet