Search vector fields based on `RICH_TEXT` fields were generated using a
treatment on `RICH_TEXT` fields's `body`
column, to only extract and index the core text.
([PR](https://github.com/twentyhq/twenty/pull/7953))
Actually our RICH_TEXT fields are of datatype `text` in our database,
allowing users to insert non-json values, which breaks the search vector
generation (as it expects json values).
Our vision is unclear for now: for instance we may want to turn
RICH_TEXT into a composite field where the plain text would be stored in
a different column.
So for now, we will (1) treat rich_text data as text, and (2) update the
search vector expressions for the existing workspaces.
Simplifying the logic around multi-object pickers and search by getting
rid of the behaviour that keeped selected elements even when they did
not match the search filter (eg keeping selected record "Brian Chesky"
in dropdown even when search input is "Qonto"). This allows us to
simplify the fetch queries around the search to only do one query.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Fixes https://github.com/twentyhq/twenty/issues/8233
Typing a search input was triggering a re-render of the whole
RecordBoardCard, resetting the search input to its initial value, an
empty string, making it impossible to actually type anything.
useRelationPicker had (legacy?) useless dependencies to recoil states
that caused the rerenders.
[This PR](https://github.com/twentyhq/twenty/pull/8210) introduced a
regression, causing noteId or taskId (respectively for noteTarget or
taskTarget creation) to be overwritten with an undefined value in the
input for noteTarget or taskTarget creation.
This is because in ActivityTargetInlineCellEditMode, in addition to the
noteId and taskId we are declaring, we are looking into the object
(noteTarget or taskTarget)'s fields and prefilling the record-to-create
with a value, potentially undefined, for all of the object fields.
So when looping over noteTarget's fields, we would find the `note`
relation field, and eventually add `note: undefined` to the
record-to-create input, in addition to the non-empty and valid existing
`noteId`.
Then in sanitizeRecordInput, from the note added right above, we add an
empty noteId to the input from node, overwriting the "good" noteId.
There are several ways to fix this, I chose to update prefillRecord not
to add an empty "note" object that makes no sense in addition to the
"noteId" we already have at this stage.
It is also possible to update `sanitizeRecordInput` not to overwrite a
value from a relation (noteId from note relation) if there is already a
value in the input.
SingleEntitySelect was wrongfully showing the empty option (eg "No
company") as selected if the selected option was not showing in the
list, because of a search filter for instance (eg selected option is
"Linkedin" but search filter is "a").
Fixing this and removing misleading prop selectedEntity from components
where it is never passed.
In the expression of our searchVector fields, we use the "simple"
configuration (over the default "english" one), to avoid picking a
language that's irrelevant to the user.
I initially forgot to add the same configuration to the query that is
being sent using ts_query.
Adding it will also allow the search to work for a single character,
while so far a single letter was most of the time considered a "stop
word" (a word with no semantic value, like "a").
In this PR
1) (FE) fixing update of name not taken into account if toggle value is
changed in the same transaction
2) (BE) moving same nameSingular and namePlural validation before
effective update
The recent addition of a "orWhere" condition to[ improve the search algo
quality](https://github.com/twentyhq/twenty/pull/7955) accidentally
broke the filter, being considered an independent "or" wondition while
we still want the filter to apply.
We were previously checking for matching with each search term
independently. Ex searching for "felix malfait" we were searching for
correspondances with "felix" and "malfait".
As a result record A with name "Marie-Claude Mala" and email
"ma.lala@email.com" had a biggest search score than record B "Felix
Malfait" with email felix@email.com for search "felix ma":
for record A we had 0 match with felix and 3 matches with "ma" ("marie",
"mala", "ma")
for record B we had 1 match with felix and 1 match with "ma" (with
"malfait").
So we want to give more weight to a row that would combine matches with
both terms, considering "felix malfait" altogether.
First step of #https://github.com/twentyhq/twenty/issues/3298.
Here we update the search endpoint to allow for a filter argument, which
we currently use in the relation pickers to restrict or exclude ids from
search.
In a future PR we will try to simplify the search logic in the FE
By default, when custom fields are created, a searchVector field is
created based on the "name" field, which is also the label identifier by
default.
When this label identifier is updated, we want to update the
searchVector field to use this field as searchable field instead, if it
is of "searchable type" (today it is only possible to select a text or
number field as label identifier, while number fields are not
searchable).
In [this](https://github.com/twentyhq/twenty/pull/7522) and
[this](https://github.com/twentyhq/twenty/pull/7543) PR we introduced
the impossibility to save a field that would be non nullable but without
a default value.
The check is actually called on the input while the defaultValue is
added by the service on a "built" fieldMetadata to create or save. So
far all fields created from the app it currently fails as both
isNullable and defaultValue are undefined so falsy at that stage.
- Fixing seeds after introducing the requirement for non-nullable fields
to have a default value (https://github.com/twentyhq/twenty/pull/7522).
- Empty string needs to be considered a valid default value
Enabling feature flags IsSearchEnabled and IsWorkspaceMigratedForSearch
at workspace creation to ensure workspaces have the searchVector fields
and indexes created.
For the feature to be enabled in the front-end we will also need
IsQueryRunnerTwentyORMEnabled to be enabled but that is an independent
topic.
Steps to test
1. Run metadata migrations
2. Run sync-metadata on your workspace
3. Enable the following feature flags:
IS_SEARCH_ENABLED
IS_QUERY_RUNNER_TWENTY_ORM_ENABLED
IS_WORKSPACE_MIGRATED_FOR_SEARCH
4. Type Cmd + K and search anything
At field creation we are checking the availability of the name by
comparing it to the other fields' names' on the object; but for
composite fields the fields' names' as indicated in the repository do
not exactly match the column names' on the tables (e.g "createdBy" field
is actually represented by columns createdByName, createdBySource etc.).
In this PR we prevent the conflict with the standard composite fields'
names.
There is still room for errors with the custom composite fields: for
example a custom composite field "address" of type address on a custom
object "listing" will introduce the columns addressAddressStreet1,
addressAddressStreet2 etc. while we won't prevent the user from later
creating a custom field named "addressAddressStreet1".
For now I decided not to tackle this as this seem extremely edgy + would
impact performance on creation of all fields while never actually useful
(I think).
Closes#6434.
We don't want to override the values of the records' address or links as
they are composite field and it is costly to loose the data.
We will need a more unified behaviour here - maybe introduce a Ctrl+Z
option.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
When migrating the option values of a select type, if the field is non
nullable (for now, only available for opportunity's "stage" standard
field), we fallback to the (potentially updated) default value instead
of nullifying the value to avoid getting a database error.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This is the first step of Link field type deprecation
(https://github.com/twentyhq/twenty/issues/5909).
Forbid creation of link field type in product and api. Update to this
type is not a concern as we do not allow the update of field type.
In the longer term, we want to improve the efficiency and reliability of
the sync-metadata command, by choosing an error handling strategy and
paying greater attention to health checks.
In the meantime, this PR adds an option to run the sync-metadata command
on all active workspaces at once.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Closes#5735.
The field probability on opportunity will -
- stop being created for new workspaces (after this PR is merged)
- have "isCustom" value set to true and be displayed as such in the
settings (after this PR is merged + sync-metadata is run on workspace)
- still show in the views (all the time)
This field is deprecated as a standard field but not replaced by another
one, so we are not adding the `(deprecated)` suffix in the label.
We have recently decided that boolean fields should only accept truthy
or falsy value, with users deciding of a default value at creation.
This command helps cleaning the existing data, by
1. updating all boolean fields default values from null to false
2. updating all boolean fields values for records from null to false
---------
Co-authored-by: Weiko <corentin@twenty.com>
Fixes#6032.
Pg has a char limit on identifiers (= table, columns, enum names) of 63
bytes.
Let's limit the metadata names that will be converted to identifiers
(objects names, fields names, relation names, enum values) to 63 chars.
For the sake of simplicity in the FE we will limit the input length of
labels.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Closes#5924.
Adding the "many" side of relations in the table view, and fixing some
issues (glitch in Multi record select, cache update after update).
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
In this PR, we implement the display and update of fields from
fromManyObjects (e.g update Employees for a Company).
Product requirement
- update should be triggered at each box check/uncheck, not at lose of
focus
Left to do in upcoming PRs
- add the column in the table views (e.g. column "Employees" on
"Companies" table view)
- add "Add new" possibility when there is no records (as is currently
exists for "one" side of relations:)
<img width="374" alt="Capture d’écran 2024-06-10 à 17 38 02"
src="https://github.com/twentyhq/twenty/assets/51697796/6f0cc494-e44f-4620-a762-d7b438951eec">
- update cache after an update affecting other records (e.g "Listings"
have one "Person"; if listing A belonged to Person A but then we
attribute listing A to Person B, Person A is no longer owner of Listing
A. For the moment that would not be reflected immediatly leading, to
potential false information if information is accessed from cache)
- try to get rid of the glitch - we also have it on the task page
example. (probably) due to the fact that we are using a recoil state to
read, update then re-read
https://github.com/twentyhq/twenty/assets/51697796/54f71674-237a-4946-866e-b8d96353c458
Update of select fields options was failing if we deleted an option that
was used for at least one row: former code would not update the value to
null but leave it to the no-longer-allowed value.
Closes#5057.
RefetchQuery is unreliable - [it won't be executed if the component is
unmounted](https://github.com/apollographql/apollo-client/issues/5419),
which is the case here because of the redirection that occurs after the
mutation.
We want to avoid using refetchQuery as much as possible, and write
directly in the cache instead.
Closes#5062.
Refactoring tables list to avoid rendering all toggles on each sync or
schema update while using fresh data:
- introducing id for RemoteTables in apollo cache
- manually updating the cache for the record that was updated after a
sync or schema update instead of fetching all tables again
Closes#5069 back-end part
And:
- do not display schemaPendingUpdates status on remote server lists as
this call will become too costly if there are dozens of servers
- (refacto) create foreignTableService
After this is merged we will be able to delete remoteTable's
availableTables column
