Closes#9187
This pull request introduces a new feature and several enhancements for
managing webhook security by adding a secret field and enabling HMAC
signature-based authentication. Below is a detailed breakdown of the
changes made:
## Frontend Updates
### Secret Field on Webhook Edit Page
- Added a new **Secret** section on the webhook edit page.
- Includes a text input field for entering a webhook secret.
- Added a descriptive note explaining the purpose of the secret for
webhook authentication.
### State Management and Persistence
- Integrated the secret field into the Webhook type definition and state
management.
- Connected the secret field UI to the data layer, ensuring seamless
persistence of the secret field.
### Validation Improvement
- Trims leading and trailing whitespace from webhook secret inputs to
avoid potential validation issues.
## Backend Updates
### Database and Entity Changes
- Introduced a nullable `secret` field to the `WebhookWorkspaceEntity`
for securely storing webhook signing secrets.
- Field uses a standard field ID:
`20202020-97ce-410f-bff9-e9ccb038fb67`.
### Signature Generation
- Implemented HMAC-SHA256 signature generation for webhook payloads when
a secret is present:
- Signatures are added as a custom `X-Twenty-Webhook-Signature` header.
- Secret is excluded from the payload to maintain security.
### Enhanced Security Measures
- Added additional headers for enhanced security:
- **Timestamp Header**: Prevents replay attacks.
- **Nonce Header**: Mitigates duplicate requests.
- Updated the OpenAPI specification to include documentation on these
security-related headers and signature verification.
## Documentation Updates
- Updated OpenAPI documentation for webhook endpoints:
- Described security-related headers (signature, timestamp, nonce).
- Included detailed instructions for verifying HMAC signatures to assist
consumers.
## Testing and Demonstration
- [Loom Video
Link](https://www.loom.com/share/bd827e4d045f46d99f3c8186e5e5676a?sid=a5e61904-0536-4e82-8055-3d05e4598393):
Demonstrating the functionality of the secret field and webhook security
features.
- [Script Example
Link](https://runkit.com/samyakpiya/676af044040c0400086d400a): A script
showing how consumers can verify webhook authenticity using the HMAC
signature.
- [Testing Site
Instance](https://webhook.site/#!/view/3472468b-ebcd-4b7f-a083-c4ba20825bb4/6885fdce-8843-4d3f-8fe0-1d8abdd53f68/1):
Contains the logged requests sent during testing and is available for
review.
## Steps for Review
1. Verify the secret field functionality on the webhook edit page,
including state persistence and UI updates.
2. Review the security enhancements, including header additions and HMAC
signature generation.
3. Validate OpenAPI documentation changes for completeness and clarity.
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
## Context
ObjectMetadataService became quite large and handles too many
responsibilities. I'm trying to refactor a bit this part in preparation
of a larger work that will combine object-metadata services and
sync-metadata logic
- Created a STANDARD_OBJECT_ICONS that can be reused in relation
creation to refer to a standard object icon.
- Created a STANDARD_OBJECT_FIELD_IDS that can be used with an object
name to get its standard field ids.
- Moved migration, record and relation creations to dedicated services,
refactored to improve performances and readability
- Refactored some validation logic
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Fix#4244 and #4356
This pull request introduces the new "view groups" capability, enabling
the reordering, hiding, and showing of columns in Kanban mode. The core
enhancement includes the addition of a new entity named `ViewGroup`,
which manages column behaviors and interactions.
#### Key Changes:
1. **ViewGroup Entity**:
The newly added `ViewGroup` entity is responsible for handling the
organization and state of columns.
This includes:
- The ability to reorder columns.
- The option to hide or show specific columns based on user preferences.
#### Conclusion:
This PR adds a significant new feature that enhances the flexibility of
Kanban views through the `ViewGroup` entity.
We'll later add the view group logic to table view too.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
By default, when custom fields are created, a searchVector field is
created based on the "name" field, which is also the label identifier by
default.
When this label identifier is updated, we want to update the
searchVector field to use this field as searchable field instead, if it
is of "searchable type" (today it is only possible to select a text or
number field as label identifier, while number fields are not
searchable).
- update `send-email.workflow-action.ts` so it send email via the google
sdk
- remove useless `workflow-action.email.ts`
- add `send` authorization to google api scopes
- update the front workflow email step form to provide a
`connectedAccountId` from the available connected accounts
- update the permissions of connected accounts: ask users to reconnect
when selecting missing send permission
Fix syncedAt no longer been set on message sync.
Fix calendar data model:
- Add `syncedAt` to `CalendarChannelWorkspaceEntity`
- Move `recurringEventExternalId` from `CalendarEventWorkspaceEntity` to
`CalendarChannelEventAssociationWorkspaceEntity` since the id is
relative to one channel
Fix save queries on calendar sync after regression.
Enabling feature flags IsSearchEnabled and IsWorkspaceMigratedForSearch
at workspace creation to ensure workspaces have the searchVector fields
and indexes created.
For the feature to be enabled in the front-end we will also need
IsQueryRunnerTwentyORMEnabled to be enabled but that is an independent
topic.
When object is not part of the workspace favorite list, we want to show
it in the "opened section" while its record page is accessed.
This PR:
- adds a new component `NavigationDrawerOpenedSection`
- makes workflow versions and runs not system object + creates a
prefilled view index for these
- do not create workspace favorites for these so these do not appear in
the workspace section
<img width="1129" alt="Capture d’écran 2024-09-26 à 11 45 25"
src="https://github.com/user-attachments/assets/c84d773c-0bef-4dce-b66a-55d7d00b0fb6">
Steps to test
1. Run metadata migrations
2. Run sync-metadata on your workspace
3. Enable the following feature flags:
IS_SEARCH_ENABLED
IS_QUERY_RUNNER_TWENTY_ORM_ENABLED
IS_WORKSPACE_MIGRATED_FOR_SEARCH
4. Type Cmd + K and search anything
This PR was created by [GitStart](https://gitstart.com/) to address the
requirements from this ticket:
[TWNTY-6260](https://clients.gitstart.com/twenty/5449/tickets/TWNTY-6260).
This ticket was imported from:
[TWNTY-6260](https://github.com/twentyhq/twenty/issues/6260)
---
### Description
This is the second PR on TWNTY-6260 which handles data migration of
Phone field to Phones field.\
\
How to Test?\
Follow the below steps:
- On the main branch,
- go to
`packages/twenty-server/src/database/typeorm-seeds/workspace/people.ts`
and change any person's phone number to a string with characters for
example: "test invalid phone", and then reset the DB.
- reset database using `npx nx database:reset twenty-server`
- This is to make sure that invalid numbers will be handled properly. We
should use the invalid value itself to avoid removing data and see how
the behavior is on the front end. should be the same as in the main, the
display shows the invalid value, but the input is empty when you click,
and then you can update.
- Checkout to `TWNTY-6260-phone-migration` branch
- Rebuild typescript using `npx nx build twenty-server`
- Run command `yarn command:prod upgrade-0.32` to do migration
- Run both backend and frontend to see the migrated field
### Demo
- **Loom Video:**\
<https://www.loom.com/share/4b9bcb423cee447d8ad09852a83b27da?sid=ed74ecaa-0339-4575-acdc-a863e95e94fd>
### Refs
#6260
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
This is the second PR on TWNTY-6261 which handlesdata migration of Email
field to Emails field.\
\
How to Test?\
Firstly make sure that you have completed the testing steps on first PR
then follow the below steps:
- Checkout to TWNTY-6261-emails-migrations branch
- Rebuild typescript using "npx nx build twenty-server"
- Run command "yarn command:prod upgrade-0.25" to do migration\
\
Loom Video:\
<https://www.loom.com/share/f82b8d29f8f64f92abe3c59c01147b45?sid=9f8ccc05-aa38-4c49-b139-fd0823066273>
**Testing Messaging Sync functionality:**
Please watch the below video to see that the synchronization of contacts
is working fine after migrating Email field to Emails field:\
<https://www.loom.com/share/400949464b244272b78c25e338cc6ab2?sid=103f6625-5933-4b99-9825-0fed33782f36>
**Question to the client**
should we rename email to emails here? in the DomainName PR, the name
did not change.
```typescript
@WorkspaceField({
standardId: PERSON_STANDARD_FIELD_IDS.email,
type: FieldMetadataType.EMAILS,
label: 'Email',
description: 'Contact’s Email',
icon: 'IconMail',
})
email: EmailsMetadata;
```
**Test Messaging Sync**
This pr will update messaging sync files so the changes shouldn't break
existing functionality of importing people and companies in the app.\
To test messaging sync you should follow the below steps:\
1. you need to connect a google account to see the importing
functionality. For this purpose you
have to create a project inside Google Cloud. But to make things easier
you can use the below credentials of an already created project. Put
them in .env of twenty-server package:
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=951231465939-h61tg6nkpkv1821qi899fjbj9looquto.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
Alternative env
```properties
MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GOOGLE_ENABLED=true
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=622006708006-dc4n3vrtf3cs2h6k7hgbborudme7ku9l.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=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
AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/redirect
AUTH_GOOGLE_APIS_CALLBACK_URL=http://localhost:3000/auth/google-apis/get-access-token
MESSAGE_QUEUE_TYPE=bull-mq
```
1. Launch your worker with `npx nx run twenty-server:worker`
2. npx nx run twenty-server:command cron:messaging:messages-import
3. npx nx run twenty-server:command cron:messaging:message-list-fetch
4. npx nx run twenty-server:command
cron📆calendar-event-list-fetch
5. Run the app and navigate to Settings/Accounts then connect your
Google account
---------
Co-authored-by: gitstart-twenty <gitstart-twenty@users.noreply.github.com>
Co-authored-by: Marie Stoppa <marie.stoppa@essec.edu>
Co-authored-by: Weiko <corentin@twenty.com>
- make member nullable on favorites
- add potential relation with view entity
- add a new type of favorite list in front : workspace favorite
- build a new component for retrieving workspace favorite to display +
refacto the existing one
Bonus:
- removing activities seed since this is deprecated
Following figma updates
https://www.figma.com/design/PNBfTgOVraw557OXChYagk/Explo?node-id=21872-7929&t=DOUzd6rzwr6lprcs-0
- No activity targets for workflow entities for now
- Adding a direct relation between workflow run et workflow
- Adding a status on the version (draft, active, deactivated)
- Adding a list of statuses on workflow
- publishedVersionId => lastPublishedVersionId
Also adding:
- the endpoint to deactivate a version
We want to avoid the nested structure of active pieces. Steps to execute
will now be separated from the trigger. It will be an array executed
sequentially.
For now a step can only be an action. But at some point it will also be
a branch or a loop
Implement soft delete on standards and custom objects.
This is a temporary solution, when we drop `pg_graphql` we should rely
on the `softDelete` functions of TypeORM.
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
- create a workflow run every time a workflow is triggered in
not_started status. This status will be helpful later for once workflows
will be scheduled
- update run status once workflow starts running
- complete status once the workflow finished running
- add a failed status if an error occurs
In this PR:
- adding Favorites to Tasks and Notes
- fixing inconsistencies between custom object creation and sync of
standard fields of custom objects
- fixing workspaceCacheVersion not used to invalidate existing
datasource
This pull request introduces a new `FieldMetadataType` called `ACTOR`.
The primary objective of this new type is to add an extra column to the
following objects: `person`, `company`, `opportunity`, `note`, `task`,
and all custom objects.
This composite type contains three properties:
- `source`
```typescript
export enum FieldActorSource {
EMAIL = 'EMAIL',
CALENDAR = 'CALENDAR',
API = 'API',
IMPORT = 'IMPORT',
MANUAL = 'MANUAL',
}
```
- `workspaceMemberId`
- This property can be `undefined` in some cases and refers to the
member who created the record.
- `name`
- Serves as a fallback if the `workspaceMember` is deleted and is used
for other source types like `API`.
### Functionality
The pre-hook system has been updated to allow real-time argument
updates. When a record is created, a pre-hook can now compute and update
the arguments accordingly. This enhancement enables the `createdBy`
field to be populated with the correct values based on the
`authContext`.
The `authContext` now includes:
- An optional User entity
- An optional ApiKey entity
- The workspace entity
This provides access to the necessary data for the `createdBy` field.
In the GraphQL API, only the `source` can be specified in the
`createdBy` input. This allows the front-end to specify the source when
creating records from a CSV file.
### Front-End Handling
On the front-end, `orderBy` and `filter` are only applied to the name
property of the `ACTOR` composite type. Currently, we are unable to
apply these operations to the workspace member relation. This means that
if a workspace member changes their first name or last name, there may
be a mismatch because the name will differ from the new one. The name
displayed on the screen is based on the workspace member entity when
available.
### Missing Components
Currently, this PR does not include a `createdBy` value for the `MAIL`
and `CALENDAR` sources. These records are created in a job, and at
present, we only have access to the workspaceId within the job. To
address this, we should use a function similar to
`loadServiceWithContext`, which was recently removed from `TwentyORM`.
This function would allow us to pass the `authContext` to the jobs
without disrupting existing jobs.
Another PR will be created to handle these cases.
### Related Issues
Fixes issue #5155.
### Additional Notes
This PR doesn't include the migrations of the current records and views.
Everything works properly when the database is reset but this part is
still missing for now. We'll add that in another PR.
- There is a minor issue: front-end tests are broken since this commit:
[80c0fc7ff1).
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
# Feature: Email thread members visibility
For this feature we implemented a chip and a dropdown menu that allows
users to check which workspace members can see an email thread, as
depicted on issue (#4199).
## Implementations
- create a new database table (messageThreadMember)
- relations between `messageThreadMembers` and the relevant existing
tables (`MessageThread` and `WorkspaceMembers`)
- added a new column to the `MessageThread table`: `everyone` - to
indicate that all workspace members can see the email thread
- create a new repository for the new table, including new queries
- edit the queries so that the new fields could be fetched from the
frontend
- created a component `MultiChip`, that shows a group of user avatars,
instead of just one
- created a component, `ShareDropdownMenu`, that shows up once the
`EmailThreadMembersChip` is clicked. On this menu you can see which
workspace members can view the email thread.
## Screenshots
Here are some screenshots of the frontend components that were created:
Chip with everyone in the workspace being part of the message thread:
Chip with just one member of the workspace (the owner) being part of the
message thread:
Chip with some members of the workspace being part of the message
thread:
How the chip looks in a message thread:
Dropdown that opens when you click on the chip:
## Testing and Mock data
We also added mock data (TypeORM seeds), focusing on adding mock data
related to message thread members.
## Conclusion
As some of the changes that we needed to do, regarding the change of
visibility of the message thread, were not covered by the existing
documentation, we were told to open a PR and ask for feedback on this
part of the implementation. Right now, our implementation is focused on
displaying who is part of an email thread.
Feel free to let us know which steps we should follow next :)
---------
Co-authored-by: Simão Sanguinho <simao.sanguinho@tecnico.ulisboa.pt>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Implement date formatting per workspace member settings
We'll need another round to maybe initialize all workspaces on the
default settings.
For now the default behavior is to take system settings if nothing is
found in DB.
---------
Co-authored-by: Weiko <corentin@twenty.com>
Closes#5735.
The field probability on opportunity will -
- stop being created for new workspaces (after this PR is merged)
- have "isCustom" value set to true and be displayed as such in the
settings (after this PR is merged + sync-metadata is run on workspace)
- still show in the views (all the time)
This field is deprecated as a standard field but not replaced by another
one, so we are not adding the `(deprecated)` suffix in the label.
Insert inside AuditLog table are all failing due to objectMetadataId
column missing.
The FieldMetadata was sharing the same standard-id with another one
(objectName) so it was skipped during the comparison step of the
sync-metadata.
Running a sync-metadata again should fix this issue. Note that this
column is non-nullable so if the table contains existing records, it
will fail. However, since the insert was failing I'm assuming the table
is empty anyway.
Closes#5748
- Create feature flag
- Add scope `https://www.googleapis.com/auth/profile.emails.read` when
connecting an account
- Get email aliases with google people API, store them in
connectedAccount and refresh them before each message-import
- Update the contact creation logic accordingly
- Refactor
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
- Rename syncSubStatus to syncStage
- Rename ongoingSyncStartedAt to syncStageStartedAt
- Remove throttlePauseUntil from db and compute it with
syncStageStartedAt and throttleFailureCount
