Closes https://github.com/twentyhq/core-team-issues/issues/861 sentries: [User workspace role map not found after recompute](https://twenty-v7.sentry.io/issues/6575092700/events/f9825338a30b470eb2345fe78c1e3479/?project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D%20not%20found%20after%20recompute&referrer=next-event&stream_index=0) (64 events in 90d), [Feature flag map not found after recompute](https://twenty-v7.sentry.io/issues/6547696076/?project=4507072499810304&query=is%3Aunresolved%20issue.priority%3A%5Bhigh%2C%20medium%5D%20not%20found%20after%20recompute&referrer=issue-stream&stream_index=1) (23 events in 90d) We have a structural issue with cached data and our locking mechanism: if a data is missing in cache and queried at the same time by two different entities, the first one will recompute the data and indicate a lock during the operation, while the second one will seek to recompute the data too but be stopped because of the ongoing lock, and be left with no data to use, condemned to throw an error. In this PR I considered that it was more important to avoid that error and chose to ignoreLock instead when the data is being queried (through getFromCacheWithRecompute), but this is maybe questionnable. An important note is that I can't figure out how users that regularly use twenty (as it has been the case since this error occured on our internal workspace) can encounter that error as once computed, **the key should always be present in the workspace**: the corresponding value it is always updated, never deleted (until this PR) and recreated. I was not able understand how this happened For our data cached without a version to refer to in the database, I also chose to ignore the lock when the recompute is triggered by a data change (eg feature flag enabling or assigning user to a role or adding an objectPermission on a role, etc.), as we never want to imped the recompute in that case to avoid potential stale data.
89 lines
2.2 KiB
TypeScript
89 lines
2.2 KiB
TypeScript
import { Logger } from '@nestjs/common';
|
|
|
|
import { isDefined } from 'twenty-shared/utils';
|
|
|
|
import {
|
|
TwentyORMException,
|
|
TwentyORMExceptionCode,
|
|
} from 'src/engine/twenty-orm/exceptions/twenty-orm.exception';
|
|
|
|
type CacheResult<T, U> = {
|
|
version: T;
|
|
data: U;
|
|
};
|
|
|
|
const getFromCacheWithRecompute = async <T, U>({
|
|
workspaceId,
|
|
getCacheData,
|
|
getCacheVersion,
|
|
recomputeCache,
|
|
cachedEntityName,
|
|
exceptionCode,
|
|
logger,
|
|
}: {
|
|
workspaceId: string;
|
|
getCacheData: (workspaceId: string) => Promise<U | undefined>;
|
|
getCacheVersion?: (workspaceId: string) => Promise<T | undefined>;
|
|
recomputeCache: (params: {
|
|
workspaceId: string;
|
|
ignoreLock?: boolean;
|
|
}) => Promise<void>;
|
|
cachedEntityName: string;
|
|
exceptionCode: TwentyORMExceptionCode;
|
|
logger: Logger;
|
|
}): Promise<CacheResult<T, U>> => {
|
|
let cachedVersion: T | undefined;
|
|
let cachedData: U | undefined;
|
|
|
|
const expectCacheVersion = isDefined(getCacheVersion);
|
|
|
|
if (expectCacheVersion) {
|
|
cachedVersion = await getCacheVersion(workspaceId);
|
|
}
|
|
|
|
cachedData = await getCacheData(workspaceId);
|
|
|
|
if (
|
|
!isDefined(cachedData) ||
|
|
(expectCacheVersion && !isDefined(cachedVersion))
|
|
) {
|
|
logger.warn(
|
|
`Triggering cache recompute for ${cachedEntityName} (workspace ${workspaceId})`,
|
|
{
|
|
cachedVersion,
|
|
cachedData,
|
|
},
|
|
);
|
|
await recomputeCache({ workspaceId, ignoreLock: true });
|
|
|
|
cachedData = await getCacheData(workspaceId);
|
|
if (expectCacheVersion) {
|
|
cachedVersion = await getCacheVersion(workspaceId);
|
|
}
|
|
|
|
if (
|
|
!isDefined(cachedData) ||
|
|
(expectCacheVersion && !isDefined(cachedVersion))
|
|
) {
|
|
logger.warn(
|
|
`Data still missing after recompute for ${cachedEntityName} (workspace ${workspaceId})`,
|
|
{
|
|
cachedVersion,
|
|
cachedData,
|
|
},
|
|
);
|
|
throw new TwentyORMException(
|
|
`${cachedEntityName} not found after recompute for workspace ${workspaceId} (missingData: ${!isDefined(cachedData)}, missingVersion: ${expectCacheVersion && !isDefined(cachedVersion)})`,
|
|
exceptionCode,
|
|
);
|
|
}
|
|
}
|
|
|
|
return {
|
|
version: cachedVersion as T,
|
|
data: cachedData,
|
|
};
|
|
};
|
|
|
|
export { CacheResult, getFromCacheWithRecompute };
|