**TLDR**
Added Billing Entitlement table, based on stripe
customer.ActiveEntitlements webhook event. In this table it has a key
value pair with each key being the stripe feature lookup key and the
value a boolean. We use this table in order to see if SSO or other
feaures are enabled by workspace.
**In order to test: twenty-server**
Billing:
- Set IS_BILLING_ENABLED to true
- Add your BILLING_STRIPE_SECRET and BILLING_STRIPE_API_KEY
- Add your BILLING_STRIPE_BASE_PLAN_PRODUCT_ID (use the one in testMode
> Base Plan)
Auth:
- Set AUTH_SSO_ENABLED to true
- Set your ACCESS_TOKEN_SECRET, LOGIN_TOKEN_SECRET, REFRESH_TOKEN_SECRET
and FILE_TOKEN_SECRET
- Set IS_SSO_ENABLED feature flag to true
Stripe Webhook:
- Authenticate with your account in the stripe CLI
- Run the command: stripe listen --forward-to
http://localhost:3000/billing/webhooks
Migration:
- npx nx typeorm -- migration:run -d
src/database/typeorm/core/core.datasource.ts
**In order to test: twenty site**
- Buy a subscription (you can use the card 4242...42 with expiration
date later in the future)
- Go to SSO and create an OICD subscription
- Change the value in the entitlement table in order to put it in false
- An error should occur saying that the current workspace has no
entitlement
**Considerations**
The data from the Entitlement table is updated based on the stripe
webhook responses, and we use the customerActiveEntitlemet response to
update the info on the table, however this event doesnt have the
metadata containing the workspaceId. Because we cannot control at wich
order the webhook send events, we force a server error if the
entitlements are updated before the BillingSubscription. Stripe resends
the event based on a exponential backoff (for more info see
https://docs.stripe.com/webhooks#retries ) because we are in test mode
Stripe retries three times over a few hours. So if the
BillingEntitlement is not updated it is completely normal and it will be
updated when stripe resends the event.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
## Context
Now that each operation has its own resolver, we need to make sure they
all map to query arg getters. CreateOne was not properly mapped to the
position getter which made record creation fail because "position:
first" was not properly converted to a float.
Also fixing queries with custom object where we were wrongly using the
table name instead of entity name
Fixes https://github.com/twentyhq/twenty/issues/8300
## Context
API events were created too late and were already formatted as Gql
responses (including nesting with edges/node/type + formatting that
should not exist in an event payload). This PR moves the emit logic to
the resolver where we actually do the DB query
Note: Also added RESTORED events
Introduced a custom decorator 'WarningIf' to log warnings for specific
environment variable conditions. Implemented this for
SESSION_STORE_SECRET to ensure users change it from the default value.
The `QueryResultGettersFactory` that is called on every query return to
was called only on the first level of relations because recursivity
wasn't implemented.
In this PR I implement recursivity and add some typing for the possible
forms a GraphQL query field can take.
This PR will fix any issue we have with pictures that were losing their
token (here for person.avatarUrl)
Fixes https://github.com/twentyhq/twenty/issues/8425
Fixes https://github.com/twentyhq/twenty/issues/8498
---------
Co-authored-by: Weiko <corentin@twenty.com>
**TLDR**
Refactor WebhoonAnalytics Graph to a more abstract version
AnalyticsGraph (in analytics module). Thus enabling the components to be
used on different instances (ex: new endpoint, new kind of graph).
**In order to test:**
1. Set ANALYTICS_ENABLED to true
2. Set TINYBIRD_JWT_TOKEN to the ADMIN token from the workspace
twenty_analytics_playground
3. Set TINYBIRD_JWT_TOKEN to the datasource or your admin token from the
workspace twenty_analytics_playground
4. Create a Webhook in twenty and set wich events it needs to track
5. Run twenty-worker in order to make the webhooks work.
6. Do your tasks in order to populate the data
7. Enter to settings> webhook>your webhook and the statistics section
should be displayed.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Implemented:
* Account Connect
* Calendar sync via delta ids then requesting single events
I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Run the CI integrationin sync mode
and add the option to run it without db reset
cleaning all the useless integration test
---------
Co-authored-by: guillim <guillaume@twenty.com>
- add `inputSchema` column in serverless function. This is an array of
parameters, with their name and type
- on serverless function id update, get the `inputSchema` + store empty
settings in step
- from step settings, build the form
TODO in next PR:
- use field type to decide what kind of form should be printed
- have a strategy to handle object as input
https://github.com/user-attachments/assets/ed96f919-24b5-4baf-a051-31f76f45e575
## Context
bull-mq connection was not working as intended, the connection parameter
was ignored and was falling back to localhost.
This PR should fix the issue by instantiating a IORedis client following
bullmq documentation https://docs.bullmq.io/guide/connections
I also changed cache-storage module to use IORedis client as well to be
more consistent even though it was not necessary there. We could move
that instantiation to a factory class in the future.
## Test
start server + worker with correct port and wrong port with
cache-storage-type memory/redis and queue-type sync/bull-mq
## What it does
### Backend
- [x] Add a mutation to create OIDC and SAML configuration
- [x] Add a mutation to delete an SSO config
- [x] Add a feature flag to toggle SSO
- [x] Add a mutation to activate/deactivate an SSO config
- [x] Add a mutation to delete an SSO config
- [x] Add strategy to use OIDC or SAML
- [ ] Improve error management
### Frontend
- [x] Add section "security" in settings
- [x] Add page to list SSO configurations
- [x] Add page and forms to create OIDC or SAML configuration
- [x] Add field to "connect with SSO" in the signin/signup process
- [x] Trigger auth when a user switch to a workspace with SSO enable
- [x] Add an option on the security page to activate/deactivate the
global invitation link
- [ ] Add new Icons for SSO Identity Providers (okta, Auth0, Azure,
Microsoft)
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
TLDR:
Secure connexion between tinybird and twenty using jwt when accessing
datasource from tinybird.
Solves:
https://github.com/twentyhq/private-issues/issues/73
In order to test:
1. Set ANALYTICS_ENABLED to true
2. Set TINYBIRD_JWT_TOKEN to the ADMIN token from the workspace
twenty_analytics_playground
3. Set TINYBIRD_JWT_TOKEN to the datasource or your admin token from the
workspace twenty_analytics_playground
4. Create a Webhook in twenty and set wich events it needs to track
5. Run twenty-worker in order to make the webhooks work.
6. Do your tasks in order to populate the data
7. Enter to settings> webhook>your webhook and the statistics section
should be displayed.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
**Before:**
Only last 5 days where displayed on Developers Settings Webhook Usage
Graph.
**Now**
Added component where you can select the time range where you want to
view the webhook usage. To do better the styling and content depassing .
<img width="652" alt="Screenshot 2024-10-15 at 16 56 45"
src="https://github.com/user-attachments/assets/d06e7f4c-a689-49a0-8839-f015ce36bab9">
**In order to test**
1. Set ANALYTICS_ENABLED to true
2. Set TINYBIRD_TOKEN to your token from the workspace
twenty_analytics_playground
3. Write your client tinybird token in
SettingsDeveloppersWebhookDetail.tsx in line 93
4. Create a Webhook in twenty and set wich events it needs to track
5. Run twenty-worker in order to make the webhooks work.
6. Do your tasks in order to populate the data
7. Enter to settings> webhook>your webhook and the statistics section
should be displayed.
8. Select the desired time range in the dropdown
**To do list**
- Tooltip is truncated when accessing values at the right end of the
graph
- DateTicks needs to follow a more clear standard
- Update this PR with more representative images
Closes#7421
This pull request consolidates Redis connection parameters into a single
`REDIS_URL` environment variable across various configuration files and
code modules. The most important changes include updates to environment
variable files, Docker and Kubernetes configurations, and code modules
to utilize the new `REDIS_URL` format.
### Environment Variable Updates:
*
[`packages/twenty-docker/.env.example`](diffhunk://#diff-f4b5e7acc0dde630eafb2228390ca56bd56db0d183582be6433a9ee808088a4cL8-R8):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-server/.env.example`](diffhunk://#diff-c06e244338b33286ea90221635809785352a971de53f647ea65650124ae74ad2L53-R53):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
*
[`packages/twenty-server/.env.test`](diffhunk://#diff-def44a08e699c9deca2f72c9f87951de5d33d9ccf5621eab2f34978df8ad0954L16-R16):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
### Docker and Kubernetes Configuration Updates:
*
[`packages/twenty-docker/docker-compose.yml`](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL28-R28):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL` in multiple
service definitions.
[[1]](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL28-R28)
[[2]](diffhunk://#diff-545fb51ad66f93f727bb8f8b375dc6858b33348d91d5e51ad931fb0bbc1affeeL62-R61)
*
[`packages/twenty-docker/k8s/manifests/deployment-server.yaml`](diffhunk://#diff-91623ed4e8b2088947cfa9a5dad76b6013e8db0c150d84347a215fa9ad78bf04L44-R45):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/manifests/deployment-worker.yaml`](diffhunk://#diff-8532debd131ce168a2527fa6a9be6405792178576ee47e2eef0cd9f3ff4a8f8cL43-R44):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/terraform/deployment-server.tf`](diffhunk://#diff-b4c468660ab00bd03589d0d47502c477ca83f7d876ff196534880b02ba46dce0L64-R65):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-docker/k8s/terraform/deployment-worker.tf`](diffhunk://#diff-e5d6a1f68d5391e2120aef6261f22f905aa1ce0a567e31b21a27d44f313eaecaL62-R63):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
### Code Module Updates:
*
[`packages/twenty-server/src/engine/core-modules/cache-storage/cache-storage.module-factory.ts`](diffhunk://#diff-06e787a7c8a48022d5909b5df9b8c6ca192521cf32f51d7f561cee937bed6678L23-R35):
Replaced `REDIS_HOST` and `REDIS_PORT` with `REDIS_URL`.
*
[`packages/twenty-server/src/engine/core-modules/environment/environment-variables.ts`](diffhunk://#diff-26ce615693b053eda02aa48aa2e30400381a2588dcb08d4a9dc3b0bf5bdd6fe7L378-R384):
Added validation for `REDIS_URL`.
*
[`packages/twenty-server/src/engine/core-modules/message-queue/message-queue.module-factory.ts`](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bR6):
Replaced `REDIS_HOST`, `REDIS_PORT`, `REDIS_USERNAME`, and
`REDIS_PASSWORD` with `REDIS_URL`.
[[1]](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bR6)
[[2]](diffhunk://#diff-7bd644d28bdd86c159c1d71242753df5d5acd91c73c6e60b2e35caa53ed3836bL35-R41)
### Documentation Updates:
*
[`packages/twenty-website/src/content/developers/self-hosting/self-hosting-var.mdx`](diffhunk://#diff-c4cc78a3ce18b6edb10f1aee8990271e1d2796a8c06c1c6ae3b68db8d52278a3L37-R37):
Updated documentation to reflect the change to `REDIS_URL`.
*
[`packages/twenty-website/src/content/developers/self-hosting/upgrade-guide.mdx`](diffhunk://#diff-c7d757829f6128e1f47aa1955bde561292dce558280511fab66281afa99042a5R102-R112):
Added upgrade instructions for the new `REDIS_URL` variable.
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
By default, when custom fields are created, a searchVector field is
created based on the "name" field, which is also the label identifier by
default.
When this label identifier is updated, we want to update the
searchVector field to use this field as searchable field instead, if it
is of "searchable type" (today it is only possible to select a text or
number field as label identifier, while number fields are not
searchable).
## Context
This PR removes workspace-query-runner/builder in preparation for fully
deprecating pg_graphql
next steps: Remove from the setup and make a command to remove comments
on schema/tables related to pg_graphql
Add support for indexes on composite fields and unicity constraint on
indexes
This pull request includes several changes across multiple files to
improve error handling, enforce unique constraints, and update database
migrations. The most important changes include updating error messages
for snack bars, adding a new command to enforce unique constraints, and
updating database migrations to include new fields and constraints.
### Error Handling Improvements:
*
[`packages/twenty-front/src/modules/error-handler/components/PromiseRejectionEffect.tsx`](diffhunk://#diff-e7dc05ced8e4730430f5c7fcd0c75b3aa723da438c26e0bef8130b614427dd9aL23-R23):
Updated error messages in `enqueueSnackBar` to use `error.message`
directly.
*
[`packages/twenty-front/src/modules/object-metadata/hooks/useFindManyObjectMetadataItems.ts`](diffhunk://#diff-74c126d6bc7a5ed6b63be994d298df6669058034bfbc367b11045f9f31a3abe6L44-R46):
Simplified error messages in `enqueueSnackBar`.
*
[`packages/twenty-front/src/modules/object-record/hooks/useFindDuplicateRecords.ts`](diffhunk://#diff-af23a1d99639a66c251f87473e63e2b7bceaa4ee4f70fedfa0fcffe5c7d79181L56-R58):
Simplified error messages in `enqueueSnackBar`.
*
[`packages/twenty-front/src/modules/object-record/hooks/useHandleFindManyRecordsError.ts`](diffhunk://#diff-da04296cbe280202a1eaf6b1244a30490d4f400411bee139651172c59719088eL22-R24):
Simplified error messages in `enqueueSnackBar`.
### New Command for Unique Constraints:
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-enforce-unique-constraints.command.ts`](diffhunk://#diff-8337096c8c80dd2619a5ba691ae5145101f8ae0368a75192a050047e8c6ab7cbR1-R159):
Added a new command to enforce unique constraints on company domain
names and person emails.
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.command.ts`](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR13-R14):
Integrated the new `EnforceUniqueConstraintsCommand` into the upgrade
process.
[[1]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR13-R14)
[[2]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR31)
[[3]](diffhunk://#diff-20215e9981a53c7566e9cbff96715685125878f5bcb84fe461a7440f2e68f6fcR64-R68)
*
[`packages/twenty-server/src/database/commands/upgrade-version/0-31/0-31-upgrade-version.module.ts`](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R7):
Registered the new `EnforceUniqueConstraintsCommand` in the module.
[[1]](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R7)
[[2]](diffhunk://#diff-da52814efc674c25ed55645f8ee2561013641a407f88423e705dd6c77b405527R24)
### Database Migrations:
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368824-migrationDebt.ts`](diffhunk://#diff-c450aeae7bc0ef4416a0ade2dc613ca3f688629f35d2a32f90a09c3f494febdcR1-R53):
Added a migration to update the `relationMetadata_ondeleteaction_enum`
and set default values.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726757368825-addIsUniqueToIndexMetadata.ts`](diffhunk://#diff-8f1e14bd7f6835ec2c3bb39bcc51e3c318a3008d576a981e682f4c985e746fbfR1-R19):
Added a migration to include the `isUnique` field in `indexMetadata`.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726762935841-addCompostiveColumnToIndexFieldMetadata.ts`](diffhunk://#diff-7c96b7276c7722d41ff31de23b2de4d6e09adfdc74815356ba63bc96a2669440R1-R19):
Added a migration to include the `compositeColumn` field in
`indexFieldMetadata`.
*
[`packages/twenty-server/src/database/typeorm/metadata/migrations/1726766871572-addWhereToIndexMetadata.ts`](diffhunk://#diff-26651295a975eb50e672dce0e4e274e861f66feb1b68105eee5a04df32796190R1-R14):
Added a migration to include the `indexWhereClause` field in
`indexMetadata`.
### GraphQL Exception Handling:
*
[`packages/twenty-server/src/engine/api/graphql/workspace-query-runner/utils/workspace-query-runner-graphql-api-exception-handler.util.ts`](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R1-R4):
Enhanced exception handling for `QueryFailedError` to provide more
specific error messages for unique constraint violations.
[[1]](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R1-R4)
[[2]](diffhunk://#diff-58445eb362dc89e31107777d39b592d7842d2ab09a223012ccd055da325270a8R23-R59)
*
[`packages/twenty-server/src/engine/api/graphql/workspace-resolver-builder/factories/create-many-resolver.factory.ts`](diffhunk://#diff-233d58ab2333586dd45e46e33d4f07e04a4b8adde4a11a48e25d86985e5a7943L58-R58):
Updated the `workspaceQueryRunnerGraphqlApiExceptionHandler` call to
include context.
*
[`packages/twenty-server/src/engine/api/graphql/workspace-resolver-builder/factories/create-one-resolver.factory.ts`](diffhunk://#diff-68b803f0762c407f5d2d1f5f8d389655a60654a2dd2394a81318655dcd44dc43L58-R58):
Updated the `workspaceQueryRunnerGraphqlApiExceptionHandler` call to
include context.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Flush Specific Redis Keys After Upgrade Command :----
Changes Made :----
Updated the FlushCacheCommand to allow for selective flushing of keys
that match the pattern engine:*, rather than flushing the entire cache.
Added a new optional parameter to specify the cache keys pattern.
Ensured that the cache is flushed at the end of the upgrade command.
Code Changes :----
Modified FlushCacheCommand to include a method for flushing keys by
pattern.
Added a new function in CacheStorageService to handle the pattern-based
flushing.
---------
Co-authored-by: Weiko <corentin@twenty.com>
In this PR:
- remove deprecated EMAIL, PHONE, LINK field types (except for Zapier
package as there is another work ongoing)
- remove composite currency filter on currencyCode, actor filter on name
and workspaceMember as the UX is not great yet
