## Context
We recently introduced a change that now throws a 401 if the token is
invalid or expired.
The first implementation is using an allow list and 'IntrospectionQuery'
was missing so the playground was broken.
The check has been updated and we now only check the excludedOperations
list if a token is not present. This is because some operations can be
both used as loggedIn and loggedOut so we want to validate the token for
those sometimes (and set the workspace, user, cache version, etc). Still
not a very clean solution imho.
## Context
We have a non-nullable constraint on authorId in attachments and
documents, until we have soft-deletion we need to handle deletion of
workspace-members and their attachments/documents.
This PR introduces pre-hooks to deleteOne/deleteMany
This is called when a user deletes a workspace-member from the members
page
Next: needs to be done on user level as well. This is called when users
try to delete their own accounts. I've seen other issues such as
re-creating a user with a previously used email failing.
Adding stripe integration by making the server logic independent of the
input fields:
- query factories (remote server, foreign data wrapper, foreign table)
to loop on fields and values without hardcoding the names of the fields
- adding stripe input and type
- add the logic to handle static schema. Simply creating a big object to
store into the server
Additional work:
- rename username field to user. This is the input intended for postgres
user mapping and we now need a matching by name
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Context
Currently, this middleware validates the token and stores the user,
workspace and cacheversion in the request object.
It only does so when a token is provided and ignores the middleware
logic if not. If the token is invalid or expired, the exception is
swallowed.
This PR removes the try/catch and adds an allowlist to skip the token
validation for operations executed while not signed-in.
I don't know a better way to do that with Nestjs. We can't easily add
the middleware per resolver without refactoring the flexible schema
engine so I'm doing it the other way around.
Fixes https://github.com/twentyhq/twenty/issues/5224
## Context
#4774
## How was it tested
Locally
## In further PRs
- Update connection status upon page change
- Adapt Info banner to dark mode
- placeholders for form
## Context
Currently we have an unicity constraint in the DB but we don't return a
clear error to the frontend before reaching the DB (which then throws a
500). This PR adds a validation check similar to what we have with field
creation
Now that we have persistent cache for schemas, we want to be able to
reset its state when users run the database:reset db otherwise schemas
won't be synced with the new DB state.
Note: In an upcoming PR, we want to be able to invalidate the cache on a
workspace level when we change the metadata schema through twenty
version upgrade
We should not depend on the foreign data wrapper type to manage distant
table. The remote server should be enough to handle the table creation.
Here is the new flow to fetch available tables:
- check if the remote server have available tables already stored
- if not, import full schema in a temporary schema
- copy the tables into the available tables field
- delete the schema
Left todo:
- update remote server input for postgres so we receive the schema
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Introduction
This PR introduces "TwentyORM," a custom ORM module designed to
streamline database interactions within our workspace schema, reducing
the need for raw SQL queries. The API mirrors TypeORM's to provide a
familiar interface while integrating enhancements specific to our
project's needs.
To facilitate this integration, new decorators prefixed with `Workspace`
have been implemented. These decorators are used to define entity
metadata more explicitly and are critical in constructing our schema
dynamically.
## New Features
- **Custom ORM System**: Named "TwentyORM," which aligns closely with
TypeORM for ease of use but is tailored to our application's specific
requirements.
- **Decorator-Driven Configuration**: Entities are now configured with
`Workspace`-prefixed decorators that clearly define schema mappings and
relationships directly within the entity classes.
- **Injectable Repositories**: Repositories can be injected similarly to
TypeORM, allowing for flexible and straightforward data management.
## Example Implementations
### Decorated Entity Definitions
Entities are defined with new decorators that outline table and field
metadata, relationships, and constraints. Here are examples of these
implementations:
#### Company Metadata Object
```typescript
@WorkspaceObject({
standardId: STANDARD_OBJECT_IDS.company,
namePlural: 'companies',
labelSingular: 'Company',
labelPlural: 'Companies',
description: 'A company',
icon: 'IconBuildingSkyscraper',
})
export class CompanyObjectMetadata extends BaseObjectMetadata {
@WorkspaceField({
standardId: COMPANY_STANDARD_FIELD_IDS.name,
type: FieldMetadataType.TEXT,
label: 'Name',
description: 'The company name',
icon: 'IconBuildingSkyscraper',
})
name: string;
@WorkspaceField({
standardId: COMPANY_STANDARD_FIELD_IDS.xLink,
type: FieldMetadataType.LINK,
label: 'X',
description: 'The company Twitter/X account',
icon: 'IconBrandX',
})
@WorkspaceIsNullable()
xLink: LinkMetadata;
@WorkspaceField({
standardId: COMPANY_STANDARD_FIELD_IDS.position,
type: FieldMetadataType.POSITION,
label: 'Position',
description: 'Company record position',
icon: 'IconHierarchy2',
})
@WorkspaceIsSystem()
@WorkspaceIsNullable()
position: number;
@WorkspaceRelation({
standardId: COMPANY_STANDARD_FIELD_IDS.accountOwner,
label: 'Account Owner',
description: 'Your team member responsible for managing the company account',
type: RelationMetadataType.MANY_TO_ONE,
inverseSideTarget: () => WorkspaceMemberObjectMetadata,
inverseSideFieldKey: 'accountOwnerForCompanies',
onDelete: RelationOnDeleteAction.SET_NULL,
})
@WorkspaceIsNullable()
accountOwner: WorkspaceMemberObjectMetadata;
}
```
#### Workspace Member Metadata Object
```typescript
@WorkspaceObject({
standardId: STANDARD_OBJECT_IDS.workspaceMember,
namePlural: 'workspaceMembers',
labelSingular: 'Workspace Member',
labelPlural: 'Workspace Members',
description: 'A workspace member',
icon: 'IconUserCircle',
})
@WorkspaceIsSystem()
@WorkspaceIsNotAuditLogged()
export class WorkspaceMemberObjectMetadata extends BaseObjectMetadata {
@WorkspaceField({
standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.name,
type: FieldMetadataType.FULL_NAME,
label: 'Name',
description: 'Workspace member name',
icon: 'IconCircleUser',
})
name: FullNameMetadata;
@WorkspaceRelation({
standardId: WORKSPACE_MEMBER_STANDARD_FIELD_IDS.accountOwnerForCompanies,
label: 'Account Owner For Companies',
description: 'Account owner for companies',
icon: 'IconBriefcase',
type: RelationMetadataType.ONE_TO_MANY,
inverseSideTarget: () => CompanyObjectMetadata,
inverseSideFieldKey: 'accountOwner',
onDelete: RelationOnDeleteAction.SET_NULL,
})
accountOwnerForCompanies: Relation
<CompanyObjectMetadata[]>;
}
```
### Injectable Repository Usage
Repositories can be directly injected into services, allowing for
streamlined query operations:
```typescript
export class CompanyService {
constructor(
@InjectWorkspaceRepository(CompanyObjectMetadata)
private readonly companyObjectMetadataRepository: WorkspaceRepository<CompanyObjectMetadata>,
) {}
async companies(): Promise<CompanyObjectMetadata[]> {
// Example queries demonstrating simple and relation-loaded operations
const simpleCompanies = await this.companyObjectMetadataRepository.find({});
const companiesWithOwners = await this.companyObjectMetadataRepository.find({
relations: ['accountOwner'],
});
const companiesFilteredByLinkLabel = await this.companyObjectMetadataRepository.find({
where: { xLinkLabel: 'MyLabel' },
});
return companiesFilteredByLinkLabel;
}
}
```
## Conclusions
This PR sets the foundation for a decorator-driven ORM layer that
simplifies data interactions and supports complex entity relationships
while maintaining clean and manageable code architecture. This is not
finished yet, and should be extended.
All the standard objects needs to be migrated and all the module using
the old decorators too.
---------
Co-authored-by: Weiko <corentin@twenty.com>
## Context
@lucasbordeau introduced a new Yoga plugin that allows us to cache our
requests (👏), see https://github.com/twentyhq/twenty/pull/5189
I'm simply updating the implementation to allow us to use different
cache storage types such as redis
Also adding a check so it does not use cache for other operations than
ObjectMetadataItems
## Test
locally, first call takes 340ms, 2nd takes 30ms with 'redis' and 13ms
with 'memory'
In this PR I'm introducing a simple custom graphql-yoga plugin to create
a caching mechanism specific to our metadata.
The cache key is made of : workspace id + workspace cache version, with
this the cache is automatically invalidated each time a change is made
on the workspace metadata.
New strategy:
- add settings field on FieldMetadata. Contains a boolean isIdField and
for numbers, a precision
- if idField, the graphql scalar returned will be a GraphQL id. This
will allow the app to work even for ids that are not uuid
- remove globals dateScalar and numberScalar modes. These were not used
- set limit as Integer
- check manually in query runner mutations that we send a valid id
Todo left:
- remove WorkspaceBuildSchemaOptions since this is not used anymore.
Will do in another PR
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Weiko <corentin@twenty.com>
## Description
This PR adds recaptcha on login form. One can add any one of three
recaptcha vendor -
1. Google Recaptcha -
https://developers.google.com/recaptcha/docs/v3#programmatically_invoke_the_challenge
2. HCaptcha -
https://docs.hcaptcha.com/invisible#programmatically-invoke-the-challenge
3. Turnstile -
https://developers.cloudflare.com/turnstile/get-started/client-side-rendering/#execution-modes
### Issue
- #3546
### Environment variables -
1. `CAPTCHA_DRIVER` - `google-recaptcha` | `hcaptcha` | `turnstile`
2. `CAPTCHA_SITE_KEY` - site key
3. `CAPTCHA_SECRET_KEY` - secret key
### Engineering choices
1. If some of the above env variable provided, then, backend generates
an error -
<img width="990" alt="image"
src="https://github.com/twentyhq/twenty/assets/60139930/9fb00fab-9261-4ff3-b23e-2c2e06f1bf89">
Please note that login/signup form will keep working as expected.
2. I'm using a Captcha guard that intercepts the request. If
"captchaToken" is present in the body and all env is set, then, the
captcha token is verified by backend through the service.
3. One can use this guard on any resolver to protect it by the captcha.
4. On frontend, two hooks `useGenerateCaptchaToken` and
`useInsertCaptchaScript` is created. `useInsertCaptchaScript` adds the
respective captcha JS script on frontend. `useGenerateCaptchaToken`
returns a function that one can use to trigger captcha token generation
programatically. This allows one to generate token keeping recaptcha
invisible.
### Note
This PR contains some changes in unrelated files like indentation,
spacing, inverted comma etc. I ran "yarn nx fmt:fix twenty-front" and
"yarn nx lint twenty-front -- --fix".
### Screenshots
<img width="869" alt="image"
src="https://github.com/twentyhq/twenty/assets/60139930/a75f5677-9b66-47f7-9730-4ec916073f8c">
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR:
- Follow up on #5170 as we did not take into account not logged in users
- only apply throttler on root fields to avoid performance overhead
An error has been recently introduced in the sync of fieldMetadata. This
PR fixes it
Additionnally, we are enabling email for trialing and past_due
workspaces. There is an ongoing work to introduce a more robust
activationStatus on workspace.
While trying to migrate a workspace from 0.3.3 to 0.10.0, we've faced an
issue with the script to migrate default-values format.
This PR fixes it.
We really need to add tests on this part ;)
Previously we had to create a separate API key to give access to chrome
extension so we can make calls to the DB. This PR includes logic to
initiate a oauth flow with PKCE method which redirects to the
`Authorise` screen to give access to server tokens.
Implemented in this PR-
1. make `redirectUrl` a non-nullable parameter
2. Add `NODE_ENV` to environment variable service
3. new env variable `CHROME_EXTENSION_REDIRECT_URL` on server side
4. strict checks for redirectUrl
5. try catch blocks on utils db query methods
6. refactor Apollo Client to handle `unauthorized` condition
7. input field to enter server url (for self-hosting)
8. state to show user if its already connected
9. show error if oauth flow is cancelled by user
Follow up PR -
Renew token logic
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Refactored the code to introduce two different concepts:
- AuditLogs (immutable, raw data)
- TimelineActivities (user-friendly, transformed data)
Still some work needed:
- Add message, files, calendar events to timeline (~2 hours if done
naively)
- Refactor repository to try to abstract concept when we can (tbd, wait
for Twenty ORM)
- Introduce ability to display child timelines on parent timeline with
filtering (~2 days)
- Improve UI: add links to open note/task, improve diff display, etc
(half a day)
- Decide the path forward for Task vs Notes: either introduce a new
field type "Record Type" and start going into that direction ; or split
in two objects?
- Trigger updates when a field is changed (will be solved by real-time /
websockets: 2 weeks)
- Integrate behavioral events (1 day for POC, 1 week for
clean/documented)
<img width="1248" alt="Screenshot 2024-04-12 at 09 24 49"
src="https://github.com/twentyhq/twenty/assets/6399865/9428db1a-ab2b-492c-8b0b-d4d9a36e81fa">
When distant table does not have an id column, syncing does not work.
Today the check is only made after creating the foreign table locally.
We should do it first, so we avoid having a foreign table created and
failing right after.
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Context
We recently introduced this verification but we didn't take into account
self-hosting that might not use billing.
## Test
tested locally with
- new workspace and new account
- existing workspace with new account and billing not enabled and status
incomplete => OK
- existing workspace with new account and billing enabled and status
incomplete => NOK
- existing workspace with new account and billing enabled and status
active => OK
We will require remote table entity to map distant table name and local
foreign table name.
Introducing the entity:
- new source of truth to know if a table is sync or not
- created synchronously at the same time as metadata and foreign table
Adding a few more changes:
- exception rather than errors so the user can see these
- `pluralize` library that will allow to stop adding `Remote` suffix on
names
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
## Context
- Rename remaining V2 services.
- Delete messages in DB when gmail history tells us they've been
deleted. I removed the logic where we store those in a cache since it's
a bit overkill because we don't need to query gmail and can use those
ids directly. The strategy is to delete the message channel message
association of the current channel, not the message or the thread since
they can still be linked to other channels. However, we will need to
call the threadCleaner service on the workspace to remove orphan
threads/non-associated messages.
Note: deletion for full-sync is a bit tricky because we need the full
list of message ids to compare with the DB and make sure we don't
over-delete. Currently, to keep memory, we don't have a variable that
holds all ids as we flush it after each page. Easier solution would be
to wipe everything before each full sync but it's probably not great for
the user experience if they are currently manipulating messages since
full-sync can happen without a user intervention (if a partial sync
fails due to historyId being invalidated by google for some reason)
## Context
The full-sync job was enqueued within a transaction, which means it
could be executed before the transaction was commit and
connected-account was not created yet.
This PR re-arrange the code a bit to avoid this
cc @bosiraphael thx for flagging this!
## Context
Recent PR introduced a verifyTransientToken inside the
GoogleAPIsProviderEnabledGuard guard. This is used to extract the
workspaceId from the token. This is working fine for the first call sent
to google however the callback is calling the same guard which is
causing an issue because the transientToken is missing from the
callback.
Imho, the same guard shouldn't be used by the callback but for the time
being I'm adding a check to prevent using feature flag when
transientToken is absent. In fact, it is present in the request but not
in the same key. Because the scope is only relevant for the first call,
I'm simply adding a check there.
