Class exception for each metadata module + handler to map on graphql
error
TODO left :
- find a way to call handler on auto-resolvers nestjs query (probably
interceptors)
- discuss what should be done for pre-hooks errors
- discuss what should be done for Unauthorized exception
First step for creating credentials for database proxy.
In next PRs:
- When calling endpoint, create database Postgres on proxy server
- Setup user on database using postgresCredentials
- Build remote server on DB to access workspace data
- Removing existing listener that was backfilling created records
without position
- Switch to a job that backfill all objects within workspace
- Adapting `FIND_BY_POSITION` so it can fetch objects without position.
Currently we needed to input a number
- refactor record position factory and record position query factory
- override position if not present during createMany
To avoid overriding the same positions for all data in createMany, the
logic is:
- if inserted last, use last position + arg index + 1
- if inserted first, use first position - arg index - 1
Remote tables could be in an empty state because:
- either we do not have data, which is normal
- either the connexion is broken (issue with the server, table requires
updates...)
Apollo throws errors but these will quickly disappear and do not provide
any tips to the user on how handle those.
This PR adds a new empty state placeholder for remote objects, that will
be display when the record list is empty. It will provide a link to the
settings page.
<img width="1512" alt="Capture d’écran 2024-05-30 à 11 49 33"
src="https://github.com/twentyhq/twenty/assets/22936103/fc2dd3cc-e90b-4033-b023-83ac9ff2a70b">
Query read timeouts happen when a remote server is not available. It
breaks:
- the remote server show page
- the record table page of imported remote tables
This PR will catch the exception so it does not go to Sentry in both
cases.
Also did 2 renaming.
Remote object id columns are not removed anymore when a remote object is
unsynced.
This is because we do not use relations anymore. We only created the id
field. So the current behavior that was implemented for custom objects,
to retrieve the fields to deleted, does not work.
Since remote object relations are really different, I extracted the
logic from `objectMetadataService`. It now handles only the relations
for custom objects creation and deletion (this part should be extracted
as well).
I create a new remote table relation service that will:
- fetch objects metadata linked to remotes (favorites,
activityTargets...)
- look for columns based on remote object name
- delete the fields and columns
Stripe tables do not support `hasNextPage` and `totalCount`. This may be
because of stripe wrapper do not properly support `COUNT` request.
Waiting on pg_graphql answer
[here](https://github.com/supabase/pg_graphql/issues/519).
This PR:
- removes `totalCount` and `hasNextPage` form queries for remote
objects. Even if it works for postgres, this may really be inefficient
- adapt the `fetchMore` functions so it works despite `hasNextPage`
missing
- remove `totalCount` display for remotes
- fix `orderBy`
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
For remotes, we will only create the foreign key, without the relation
metadata. Expected behavior will be:
- possible to create an activity. But the remote object will not be
displayed in the relations of the activity
- the remote objects should not be available in the search for relations
Also switched the number settings to an enum, since we now have to
handle `BigInt` case.
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
- Check if a table with the same name already exists
- If yes, add a number suffix, and check again
Co-authored-by: Thomas Trompette <thomast@twenty.com>
We do not update the comment on the local table when a foreign table key
is deleted.
This was not breaking, which is why we did not see it. But comments
should be kept up to date.
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Adding stripe integration by making the server logic independent of the
input fields:
- query factories (remote server, foreign data wrapper, foreign table)
to loop on fields and values without hardcoding the names of the fields
- adding stripe input and type
- add the logic to handle static schema. Simply creating a big object to
store into the server
Additional work:
- rename username field to user. This is the input intended for postgres
user mapping and we now need a matching by name
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
We should not depend on the foreign data wrapper type to manage distant
table. The remote server should be enough to handle the table creation.
Here is the new flow to fetch available tables:
- check if the remote server have available tables already stored
- if not, import full schema in a temporary schema
- copy the tables into the available tables field
- delete the schema
Left todo:
- update remote server input for postgres so we receive the schema
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
New strategy:
- add settings field on FieldMetadata. Contains a boolean isIdField and
for numbers, a precision
- if idField, the graphql scalar returned will be a GraphQL id. This
will allow the app to work even for ids that are not uuid
- remove globals dateScalar and numberScalar modes. These were not used
- set limit as Integer
- check manually in query runner mutations that we send a valid id
Todo left:
- remove WorkspaceBuildSchemaOptions since this is not used anymore.
Will do in another PR
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Weiko <corentin@twenty.com>
When distant table does not have an id column, syncing does not work.
Today the check is only made after creating the foreign table locally.
We should do it first, so we avoid having a foreign table created and
failing right after.
Co-authored-by: Thomas Trompette <thomast@twenty.com>
We will require remote table entity to map distant table name and local
foreign table name.
Introducing the entity:
- new source of truth to know if a table is sync or not
- created synchronously at the same time as metadata and foreign table
Adding a few more changes:
- exception rather than errors so the user can see these
- `pluralize` library that will allow to stop adding `Remote` suffix on
names
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
1/ When the user inputs wrong connection informations, we do not inform
him. He will only see that no tables are available.
We will display a connection failed status if an error is raised testing
the connection
2/ If the connection fails, it should still be possible to delete the
server. Today, since we try first to delete the tables, the connection
failure throws an error that will prevent server deletion. Using the
foreign tables instead of calling the distant DB.
3/ Redirect to connection show page instead of connection list after
creation
4/ Today, foreign tables are fetched without the server name. This is a
mistake because we need to know which foreign table is linked with which
server. Updating the associated query.
<img width="632" alt="Capture d’écran 2024-04-12 à 10 52 49"
src="https://github.com/twentyhq/twenty/assets/22936103/9e8406b8-75d0-494c-ac1f-5e9fa7100f5c">
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
- Set `readOnly` boolean in table row context. Preventing updates and
deletion
- Show page is null for remote objects. No need for complicated design
since this is temporary?
- Relation creations are now behind a feature flag for remote objects
- Refetch objects and views after syncing objects
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
This PR:
- separates the existing updateSyncStatus endpoint into 2 endpoints
- creates mutations and hooks that will call those endpoints
- trigger the hook on toggle
- removes form logic and add a separated component for toggling
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Foreign tables should be created using migrations, as we do for standard
tables.
Since those are not really generated from the object metadata but from
the remote table, those migrations won't live in the object metadata
service.
This PR:
- creates new types of migration : create_foreign_table and
drop_foreign_table
- triggers those migrations rather than raw queries directly
- moves the logic to fetch current foreign tables into the remote table
service since this is not directly linked to postgres data wrapper
- adds logic to unsync all tables before deleting
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
This PR:
- creates the query to delete a connection
- creates the hook that triggers the query
- triggers the hook function when clicking on remove + get back to
connection page
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Endpoint is broken since we now use `Remote` as a suffix for remote
table names.
This PR:
- creates a common function to calculate the name of the remote table
- use it in the `findAvailableRemotePostgresTables` to know if a table
has been synced or not
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Backend: Adding a new util function that throw an error if the
objectMetadata is remote
Frontend: hiding the save button when remote
Also renaming `useObjectMetadataItemForSettings` since this hook is used
in other places than settings and is not in the settings repo. Name can
definitely be challenged!
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Several fixes for remote objects:
- labels are now displayed in title case. Added an util for this.
- Ids are often integers but the foreign keys on the relations were
uuid. Sending the id type to the object metadata service so it can
creates the foreign key accordingly
- Graphql comments are override when several remote objects are
imported. Building a function that fetch the existing comment and update
it
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Foreign table id cannot be a foreign key of a base table. But the
current code use foreign keys to link object metadata with activities,
events... So we will:
- create a column without creating a foreign key
- add a comment on the table schema so pg_graphql sees it as a foreign
key
This PR:
- refactor a bit object metadata service so the mutation creation is
separated into an util
- adds the mutation creation for remote object relations
- add a new type of mutation to create a comment
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Build remote server
* Add getters
* Migrate to json inputs
* Use extendable type
* Use regex validation
* Remove acronymes
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Add getter factory for attachements
* Override guard in test
* Add secret in env variables
* Return custom message on expiration
* Rename to signPayload
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Build arg setter for position
* Build separated query factory + rename existing
* Sort record by position in front
* Add tests
* Set first for type board
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Migrate command to cron
* Put back command using job as well
* Build service and module + move into folder
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Set opportunity stage as editable
* Fix comments
* Add command for migration
* Fixes
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
* Store compact view status
* Rename to isCompact
* Fixes
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
* Rename recordPosition into position
* Fix according to review
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
* Create record field on non syst standard objects + on custom objects
* Create workspace migration
* Fix naming and add seed
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Use recoil state for page info
* Remove memoization
* Remove right drawer fetch more loader
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Add record chip for sender and add receivers
* Build enum for roles
* Rename var and use string literal
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Fetch messages with hard coded thread id
* Fix test
* Use first workspace member or person names
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Adding message thread component
* Add state and mocks
* Rename components and use local state for messages
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* Trigger message thread top bar
* Rename message thread to thread
* Move all components in a directory
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
* WIP
* Poc
* Use cached root query + remove proloaded views state
* Fix storybook test + fix codegen
* Return default schema if token is absent, unauthenticated if token is invalid
* Use enum instead of bool
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
* Migrate record table to scope map
* Update record scope id to record id
* Remove todos and fix edit mode
* Fix perf
* Fix tests
* Fix tests
---------
Co-authored-by: Thomas Trompette <thomast@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
