Let's deprecate Sentry Release and use APP_VERSION instead.
It'll make it more clear in the interface to use named version for bug
analysis, than commit sha
Done :
- move metrics and health cache services from health module to metrics
module
- refactor metrics counter from specific method to set up from enum keys
- add OpenTelemetry (Otel) instrumentation for metrics
- set up Otel SDK to send metrics to Otel collector
To do later :
- implement Otel instrumentation for traces + plug Sentry on top
Replaced multiple environment variables for frontend URL construction
with a single FRONTEND_URL variable. This change reduces complexity and
improves clarity by consolidating frontend URL handling into one source.
Updated relevant validations and removed unused variables like
FRONT_PROTOCOL and FRONT_PORT.
Fix#10016
Introduce isCustomDomainEnabled field in Workspace entity to manage
custom domain activation. Update related services, types, and logic to
validate and toggle the custom domain's status dynamically based on its
current state. This ensures accurate domain configurations are reflected
across the system.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# In this PR
- Allow to register a custom domain
- Refacto subdomain generation
# In other PRs
- Add UI to deal with a custom domain
- Add logic to work with custom domain
Closestwentyhq/twenty#8240
This PR introduces email verification for non-Microsoft/Google Emails:
## Email Verification SignInUp Flow:
https://github.com/user-attachments/assets/740e9714-5413-4fd8-b02e-ace728ea47ef
The email verification link is sent as part of the
`SignInUpStep.EmailVerification`. The email verification token
validation is handled on a separate page (`AppPath.VerifyEmail`). A
verification email resend can be triggered from both pages.
## Email Verification Flow Screenshots (In Order):
## Sent Email Details (Subject & Template):
### Successful Email Verification Redirect:
### Unsuccessful Email Verification (invalid token, invalid email, token
expired, user does not exist, etc.):
### Force Sign In When Email Not Verified:
# TODOs:
## Sign Up Process
- [x] Introduce server-level environment variable
IS_EMAIL_VERIFICATION_REQUIRED (defaults to false)
- [x] Ensure users joining an existing workspace through an invite are
not required to validate their email
- [x] Generate an email verification token
- [x] Store the token in appToken
- [x] Send email containing the verification link
- [x] Create new email template for email verification
- [x] Create a frontend page to handle verification requests
## Sign In Process
- [x] After verifying user credentials, check if user's email is
verified and prompt to to verify
- [x] Show an option to resend the verification email
## Database
- [x] Rename the `emailVerified` colum on `user` to to `isEmailVerified`
for consistency
## During Deployment
- [x] Run a script/sql query to set `isEmailVerified` to `true` for all
users with a Google/Microsoft email and all users that show an
indication of a valid subscription (e.g. linked credit card)
- I have created a draft migration file below that shows one possible
approach to implementing this change:
```typescript
import { MigrationInterface, QueryRunner } from 'typeorm';
export class UpdateEmailVerifiedForActiveUsers1733318043628
implements MigrationInterface
{
name = 'UpdateEmailVerifiedForActiveUsers1733318043628';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`
CREATE TABLE core."user_email_verified_backup" AS
SELECT id, email, "isEmailVerified"
FROM core."user"
WHERE "deletedAt" IS NULL;
`);
await queryRunner.query(`
-- Update isEmailVerified for users who have been part of workspaces with active subscriptions
UPDATE core."user" u
SET "isEmailVerified" = true
WHERE EXISTS (
-- Check if user has been part of a workspace through userWorkspace table
SELECT 1
FROM core."userWorkspace" uw
JOIN core."workspace" w ON uw."workspaceId" = w.id
WHERE uw."userId" = u.id
-- Check for valid subscription indicators
AND (
w."activationStatus" = 'ACTIVE'
-- Add any other subscription-related conditions here
)
)
AND u."deletedAt" IS NULL;
`);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`
UPDATE core."user" u
SET "isEmailVerified" = b."isEmailVerified"
FROM core."user_email_verified_backup" b
WHERE u.id = b.id;
`);
await queryRunner.query(`DROP TABLE core."user_email_verified_backup";`);
}
}
```
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Eliminated all references to `isSSOEnabled` across the frontend,
backend, and configuration files. This change simplifies the codebase by
removing unnecessary feature flag checks, associated logic, and
environment variables. The SSO feature remains available without
reliance on this flag.
## Summary
Add support for multi-workspace feature and adjust configurations and
states accordingly.
- Introduced new state isMultiWorkspaceEnabledState.
- Updated ClientConfigProviderEffect component to handle
multi-workspace.
- Modified GraphQL schema and queries to include multi-workspace related
configurations.
- Adjusted server environment variables and their respective
documentation to support multi-workspace toggle.
- Updated server-side logic to handle new multi-workspace configurations
and conditions.
We will remove the `twenty-postgres` image that was used for local
development and only use `twenty-postgres-pilo` (which we use in prod),
bringing the development environment closer to prod and avoiding having
to maintain 2 images.
Instead of provisioning the super user after the db initialization, we
directly rely on the superuser provided by Spilo for simplicity. We also
introduce a change that tries to create the right database (`default` or
`test`) based on the context.
How to test:
```
docker build -t twentycrm/twenty-postgres-spilo:latest -f ./packages/twenty-docker/twenty-postgres-spilo/Dockerfile .
docker images --no-trunc | grep twenty-postgres-spilo
postgres-on-docker:
docker run \
--name twenty_pg \
-e PGUSER_SUPERUSER=twenty \
-e PGPASSWORD_SUPERUSER=twenty \
-e ALLOW_NOSSL=true \
-v twenty_db_data:/home/postgres/pgdata \
-p 5432:5432 \
REPLACE_WITH_IMAGE_ID
```
Implemented:
* Account Connect
* Calendar sync via delta ids then requesting single events
I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
## What it does
### Backend
- [x] Add a mutation to create OIDC and SAML configuration
- [x] Add a mutation to delete an SSO config
- [x] Add a feature flag to toggle SSO
- [x] Add a mutation to activate/deactivate an SSO config
- [x] Add a mutation to delete an SSO config
- [x] Add strategy to use OIDC or SAML
- [ ] Improve error management
### Frontend
- [x] Add section "security" in settings
- [x] Add page to list SSO configurations
- [x] Add page and forms to create OIDC or SAML configuration
- [x] Add field to "connect with SSO" in the signin/signup process
- [x] Trigger auth when a user switch to a workspace with SSO enable
- [x] Add an option on the security page to activate/deactivate the
global invitation link
- [ ] Add new Icons for SSO Identity Providers (okta, Auth0, Azure,
Microsoft)
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
Closes#7421
This pull request consolidates Redis connection parameters into a single
`REDIS_URL` environment variable across various configuration files and
code modules. The most important changes include updates to environment
variable files, Docker and Kubernetes configurations, and code modules
to utilize the new `REDIS_URL` format.
### Environment Variable Updates:
*
[`packages/twenty-docker/.env.example`](diffhunk://#diff-f4b5e7acc0dde630eafb2228390ca56bd56db0d183582be6433a9ee808088a4cL8-R8):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-server/.env.example`](diffhunk://#diff-c06e244338b33286ea90221635809785352a971de53f647ea65650124ae74ad2L53-R53):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
*
[`packages/twenty-server/.env.test`](diffhunk://#diff-def44a08e699c9deca2f72c9f87951de5d33d9ccf5621eab2f34978df8ad0954L16-R16):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
### Docker and Kubernetes Configuration Updates:
*
[`packages/twenty-docker/docker-compose.yml`](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL28-R28):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL` in multiple
service definitions.
[[1]](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL28-R28)
[[2]](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL62-R61)
*
[`packages/twenty-docker/k8s/manifests/deployment-server.yaml`](diffhunk://#diff-91623ed4e8b2088947cfa9a5dad76b6013e8db0c150d84347a215fa9ad78bf04L44-R45):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/manifests/deployment-worker.yaml`](diffhunk://#diff-8532debd131ce168a2527fa6a9be6405792178576ee47e2eef0cd9f3ff4a8f8cL43-R44):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/terraform/deployment-server.tf`](diffhunk://#diff-b4c468660ab00bd03589d0d47502c477ca83f7d876ff196534880b02ba46dce0L64-R65):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/terraform/deployment-worker.tf`](diffhunk://#diff-e5d6a1f68d5391e2120aef6261f22f905aa1ce0a567e31b21a27d44f313eaecaL62-R63):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
### Code Module Updates:
*
[`packages/twenty-server/src/engine/core-modules/cache-storage/cache-storage.module-factory.ts`](diffhunk://#diff-06e787a7c8a48022d5909b5df9b8c6ca192521cf32f51d7f561cee937bed6678L23-R35):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-server/src/engine/core-modules/environment/environment-variables.ts`](diffhunk://#diff-26ce615693b053eda02aa48aa2e30400381a2588dcb08d4a9dc3b0bf5bdd6fe7L378-R384):
Added validation for `REDIS_URL`.
*
[`packages/twenty-server/src/engine/core-modules/message-queue/message-queue.module-factory.ts`](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bR6):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
[[1]](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bR6)
[[2]](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bL35-R41)
### Documentation Updates:
*
[`packages/twenty-website/src/content/developers/self-hosting/self-hosting-var.mdx`](diffhunk://#diff-c4cc78a3ce18b6edb10f1aee8990271e1d2796a8c06c1c6ae3b68db8d52278a3L37-R37):
Updated documentation to reflect the change to `REDIS_URL`.
*
[`packages/twenty-website/src/content/developers/self-hosting/upgrade-guide.mdx`](diffhunk://#diff-c7d757829f6128e1f47aa1955bde561292dce558280511fab66281afa99042a5R102-R112):
Added upgrade instructions for the new `REDIS_URL` variable.
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
Added:
- An "Ask AI" command to the command menu.
- A simple GraphQL resolver that converts the user's question into a
relevant SQL query using an LLM, runs the query, and returns the result.
<img width="428" alt="Screenshot 2024-06-09 at 20 53 09"
src="https://github.com/twentyhq/twenty/assets/171685816/57127f37-d4a6-498d-b253-733ffa0d209f">
No security concerns have been addressed, this is only a
proof-of-concept and not intended to be enabled in production.
All changes are behind a feature flag called `IS_ASK_AI_ENABLED`.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
## Description
This PR adds recaptcha on login form. One can add any one of three
recaptcha vendor -
1. Google Recaptcha -
https://developers.google.com/recaptcha/docs/v3#programmatically_invoke_the_challenge
2. HCaptcha -
https://docs.hcaptcha.com/invisible#programmatically-invoke-the-challenge
3. Turnstile -
https://developers.cloudflare.com/turnstile/get-started/client-side-rendering/#execution-modes
### Issue
- #3546
### Environment variables -
1. `CAPTCHA_DRIVER` - `google-recaptcha` | `hcaptcha` | `turnstile`
2. `CAPTCHA_SITE_KEY` - site key
3. `CAPTCHA_SECRET_KEY` - secret key
### Engineering choices
1. If some of the above env variable provided, then, backend generates
an error -
<img width="990" alt="image"
src="https://github.com/twentyhq/twenty/assets/60139930/9fb00fab-9261-4ff3-b23e-2c2e06f1bf89">
Please note that login/signup form will keep working as expected.
2. I'm using a Captcha guard that intercepts the request. If
"captchaToken" is present in the body and all env is set, then, the
captcha token is verified by backend through the service.
3. One can use this guard on any resolver to protect it by the captcha.
4. On frontend, two hooks `useGenerateCaptchaToken` and
`useInsertCaptchaScript` is created. `useInsertCaptchaScript` adds the
respective captcha JS script on frontend. `useGenerateCaptchaToken`
returns a function that one can use to trigger captcha token generation
programatically. This allows one to generate token keeping recaptcha
invisible.
### Note
This PR contains some changes in unrelated files like indentation,
spacing, inverted comma etc. I ran "yarn nx fmt:fix twenty-front" and
"yarn nx lint twenty-front -- --fix".
### Screenshots
<img width="869" alt="image"
src="https://github.com/twentyhq/twenty/assets/60139930/a75f5677-9b66-47f7-9730-4ec916073f8c">
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
Previously we had to create a separate API key to give access to chrome
extension so we can make calls to the DB. This PR includes logic to
initiate a oauth flow with PKCE method which redirects to the
`Authorise` screen to give access to server tokens.
Implemented in this PR-
1. make `redirectUrl` a non-nullable parameter
2. Add `NODE_ENV` to environment variable service
3. new env variable `CHROME_EXTENSION_REDIRECT_URL` on server side
4. strict checks for redirectUrl
5. try catch blocks on utils db query methods
6. refactor Apollo Client to handle `unauthorized` condition
7. input field to enter server url (for self-hosting)
8. state to show user if its already connected
9. show error if oauth flow is cancelled by user
Follow up PR -
Renew token logic
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
In the previous PR #4912 it seems that I forgot to pass the environment
on the backend.
Here is a quick fix!
I also added some "doc" in the the .env.example
* calendar module
* wip
* creating a folder for common files between calendar and messages
* wip
* wip
* wip
* wip
* update calendar search filter
* wip
* working on full sync service
* reorganizing folders
* adding repositories
* fix typo
* working on full-sync service
* Add calendarQueue to MessageQueue enum and update dependencies
* start transaction
* wip
* add save and update functions for event
* wip
* save events
* improving step by step
* add calendar scope
* fix nest modules imports
* renaming
* create calendar channel
* create job for google calendar full-sync
* call GoogleCalendarFullSyncJob after connected account creation
* ask for scope conditionnally
* fixes
* create channels conditionnally
* fix
* fixes
* fix FK bug
* filter out canceled events
* create save and update functions for calendarEventAttendee repository
* saving messageParticipants is working
* save calendarEventAttendees is working
* add calendarEvent cleaner
* calendar event cleaner is working
* working on updating attendees
* wip
* reintroducing google-gmail endpoint to ensure smooth deploy
* modify callbackURL
* modify front url
* changes to be able to merge
* put back feature flag
* fixes after PR comments
* add feature flag check
* remove unused modules
* separate delete connected account associated job data in two jobs
* fix error
* rename calendar_v3 as calendarV3
* Update packages/twenty-server/src/workspace/calendar-and-messaging/utils/valueStringForBatchRawQuery.util.ts
Co-authored-by: Jérémy M <jeremy.magrin@gmail.com>
* improve readability
* renaming to remove plural
* renaming to remove plural
* don't throw if no connected account is found
* use calendar queue
* modify usage of HttpService in fetch-by-batch
* modify valuesStringForBatchRawQuery to improve api and return flattened values
* fix auth module feature flag import
* fix getFlattenedValuesAndValuesStringForBatchRawQuery
---------
Co-authored-by: Jérémy M <jeremy.magrin@gmail.com>
