Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions docs/contracts/storage-pagination.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Dataset and snapshot pagination

This change requires the ordered cursor contract from
[vpsfreecz/vpsadmin#44](https://github.com/vpsfreecz/vpsadmin/pull/44), currently
`320af0e152ed223bf0365e0f1cf4b38cf00d7b1d`. Do not release the UI fix as a
standalone guarantee against an older API using numeric ID comparisons.
The branch is based on UI PR496 for browser-history cursor restoration.

- Dataset index (including NAS): ascending `(full_name, id)`.
- Dataset snapshots: ascending `(created_at, id)`.
- `from_id` identifies the last displayed row in the scoped ordered list. It
is not the minimum or maximum ID and does not include the lookahead record.
- Fetch `limit + 1`, display `limit`, and enable Next only with a fresh extra row.
A historical forward cursor alone cannot prove that another page exists.
- Next replaces its visited forward edge with the current row boundary.
- Requests carry cancellation signals. URL filters changed by the UI clear the
cursor; browser navigation and reload restore the URL cursor.
- Invalid/deleted/out-of-scope anchors return an API error. Retry preserves the
cursor; the explicit first-page action preserves filters. Empty cursor results
retain a return path. Text search remains limited to the displayed API page.
- Member NAS does not submit an arbitrary owner filter; API authorization scopes
the records. These frontend checks do not replace backend authorization.

Concurrent renames, inserts, deletes or timestamp changes are not a consistent
snapshot. Restart traversal after such changes; no exactly-once promise is made.
Property history, downloads and expansion history are separate endpoints and
are not certified by this change.

## Evidence

Fixture Playwright covers three pages with non-monotonic IDs, tied snapshot
creation times, exact end, desktop/mobile, cs/en, admin owner filter/member NAS,
reload, Back, invalid cursors, empty pages, and a changed visited boundary.
Existing local text-filter and snapshot pagination tests remain covered.
The dedicated filter regression also checks reset and browser-history recovery.

API specs for PR44 and UI fixtures are separate evidence. Actual combined
API44/UI496/storage execution in the existing isolated VM cluster is still a
release gate; these tests must not be reported as live VM certification.
4 changes: 2 additions & 2 deletions e2e/specs/app/datasets_list_keyset_pagination.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ test.describe('Datasets list keyset pagination', () => {
handlers: {
'GET datasets': ({ searchParams }) => {
const fromId = searchParams.get('dataset[from_id]');
return { datasets: fromId ? page2 : page1, _meta: { total_count: 100 } };
return { datasets: fromId ? page2 : [...page1, page2[0]], _meta: { total_count: 100 } };
},
},
});
Expand Down Expand Up @@ -99,7 +99,7 @@ test.describe('Datasets list text filter contract', () => {
expect(searchParams.get('dataset[q]')).toBeNull();
const fromId = searchParams.get('dataset[from_id]');
return {
datasets: fromId ? secondPage : firstPage,
datasets: fromId ? secondPage : [...firstPage, secondPage[0]],
_meta: { total_count: 51 },
};
},
Expand Down
159 changes: 159 additions & 0 deletions e2e/specs/app/storage_ordered_cursors.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
import { expect, test } from '../../fixtures/vpsadmin-window';
import { bootstrapVpsAdminWindow } from '../../fixtures/bootstrap';
import { failEnvelope, installHaveApiMock, jsonFulfill } from '../../fixtures/haveapi';
import { setUiSettingsLocalStorage } from '../../fixtures/uiSettings';

const dataset = { id: 10, name: 'fixture', full_name: 'tank/fixture', user: { id: 7, login: 'member' }, object_state: 'active' };
const records = Array.from({ length: 75 }, (_, i) => ({
id: ((i * 29) % 75) + 100,
full_name: `tank/${String(i).padStart(3, '0')}`,
name: `snapshot-${i}`, label: `Snapshot ${i}`,
created_at: new Date(Date.UTC(2026, 0, 1, 0, Math.floor(i / 3))).toISOString(),
user: { id: 7, login: 'member' }, object_state: 'active',
}));

for (const kind of ['datasets', 'snapshots'] as const) {
for (const role of ['admin', 'member'] as const) {
const language = role === 'admin' ? 'cs' : 'en';
test(`@pr-smoke @pr-smoke-mobile ${kind} follows ordered anchors with ${role} scope (${language})`, async ({ page }, info) => {
await setUiSettingsLocalStorage(page, { language });
await bootstrapVpsAdminWindow(page);
const mobile = info.project.name === 'mobile-chrome';
const prefix = kind === 'datasets' ? 'datasets' : 'dataset.snapshots';
const pager = `${prefix}.pagination.${mobile ? 'mobile' : 'desktop'}`;
const item = (id: number) => page.getByTestId(`${prefix}.${mobile ? 'card' : 'row'}.${id}`);
const rows = [...records].sort((a, b) => kind === 'datasets'
? a.full_name.localeCompare(b.full_name) || a.id - b.id
: a.created_at.localeCompare(b.created_at) || a.id - b.id);
const ns = kind === 'datasets' ? 'dataset' : 'snapshot';
const cursors: number[] = [];
let ownerFilterCleared = false;
await installHaveApiMock(page, {
user: { id: 7, login: role, level: role === 'admin' ? 99 : 1 },
handlers: {
'GET datasets/10': () => dataset,
[kind === 'datasets' ? 'GET datasets' : 'GET datasets/10/snapshots']: ({ searchParams }) => {
const cursor = Number(searchParams.get(`${ns}[from_id]`) ?? 0);
expect(searchParams.get(`${ns}[limit]`)).toBe('26');
if (kind === 'datasets') {
expect(searchParams.get('dataset[user]')).toBe(role === 'admin' && !ownerFilterCleared ? '7' : null);
if (role === 'member') expect(searchParams.get('dataset[role]')).toBe('primary');
}
cursors.push(cursor);
const start = cursor ? rows.findIndex(row => row.id === cursor) + 1 : 0;
return { [kind]: rows.slice(start, start + 26) };
},
},
});
const base = kind === 'datasets' && role === 'member' ? '/app/nas'
: `/${role === 'admin' ? 'admin' : 'app'}/datasets${kind === 'snapshots' ? '/10/snapshots' : ''}`;
await page.goto(`${base}?limit=25${kind === 'datasets' ? (role === 'admin' ? '&user=7' : '&user=999') : ''}`);
for (let p = 0; p < 3; p++) {
await expect(item(rows[p * 25].id)).toBeVisible();
const visible = page.locator(`[data-testid^="${prefix}.${mobile ? 'card' : 'row'}."]:visible`);
// Ignore nested status-dot test IDs; compare every actual row in API order.
const ids = () => visible.evaluateAll(elements => elements.map(el => el.getAttribute('data-testid')).filter(id => /^.*\.(row|card)\.\d+$/.test(id ?? '')));
await expect.poll(ids).toEqual(rows.slice(p * 25, (p + 1) * 25).map(row => `${prefix}.${mobile ? 'card' : 'row'}.${row.id}`));
if (p < 2) await page.getByTestId(`${pager}.next`).click();
}
await expect(page.getByTestId(`${pager}.next`)).toBeDisabled();
expect(cursors).toEqual([0, rows[24].id, rows[49].id]);
await page.reload();
await expect(item(rows[50].id)).toBeVisible();
await expect(page.getByTestId(`${pager}.next`)).toBeDisabled();
await page.getByTestId(`${pager}.prev`).click();
await expect(item(rows[25].id)).toBeVisible();
await page.goBack();
await expect(item(rows[50].id)).toBeVisible();
if (kind === 'datasets' && role === 'admin') {
ownerFilterCleared = true;
await page.getByTestId('datasets.filter.clear').click();
await expect(page).not.toHaveURL(/from_id=|user=/);
await expect(item(rows[0].id)).toBeVisible();
ownerFilterCleared = false;
await page.goBack();
await expect(page).toHaveURL(/user=7/);
await expect(item(rows[50].id)).toBeVisible();
}
});
}

test(`@pr-smoke @pr-smoke-mobile ${kind} retries invalid anchors and recovers empty pages`, async ({ page }, info) => {
await setUiSettingsLocalStorage(page, { language: 'en' });
await bootstrapVpsAdminWindow(page);
let fail = true;
const mobile = info.project.name === 'mobile-chrome';
const prefix = kind === 'datasets' ? 'datasets' : 'dataset.snapshots';
const error = kind === 'datasets' ? 'datasets.list.error' : 'dataset.snapshots.error';
const base = `/admin/datasets${kind === 'snapshots' ? '/10/snapshots' : ''}`;
const url = `${base}?limit=25&from_id=999&page=2${kind === 'datasets' ? '&user=7' : ''}`;
await installHaveApiMock(page, {
user: { id: 1, login: 'admin', level: 99 },
handlers: {
'GET datasets/10': () => dataset,
[kind === 'datasets' ? 'GET datasets' : 'GET datasets/10/snapshots']: ({ searchParams }) => {
const cursor = searchParams.get(`${kind === 'datasets' ? 'dataset' : 'snapshot'}[from_id]`);
if (kind === 'datasets') expect(searchParams.get('dataset[user]')).toBe('7');
if (cursor && fail) return jsonFulfill(failEnvelope('Invalid pagination cursor'), 400);
return { [kind]: cursor ? [] : [records[0]] };
},
},
});
await page.goto(url);
await expect(page.getByTestId(error)).toBeVisible();
fail = false;
await page.getByTestId(`${error}.primary`).click();
await expect(page.getByTestId(error)).toHaveCount(0);
if (kind === 'datasets') await page.getByTestId('datasets.pagination.empty.restart').click();
else await page.getByTestId(`${prefix}.pagination.${mobile ? 'mobile' : 'desktop'}.prev`).click();
await expect(page.getByTestId(`${prefix}.${mobile ? 'card' : 'row'}.${records[0].id}`)).toBeVisible();
fail = true;
await page.goto(url);
await page.getByTestId(`${error}.secondary`).click();
await expect(page).not.toHaveURL(/from_id=/);
await expect(page.getByTestId(`${prefix}.${mobile ? 'card' : 'row'}.${records[0].id}`)).toBeVisible();
if (kind === 'datasets') await expect(page).toHaveURL(/user=7/);
});
}

for (const kind of ['datasets', 'snapshots'] as const) {
test(`@pr-smoke @pr-smoke-mobile ${kind} replaces a visited forward edge after refresh`, async ({ page }, info) => {
await bootstrapVpsAdminWindow(page);
let rows = [...records].sort((a, b) => kind === 'datasets'
? a.full_name.localeCompare(b.full_name) || a.id - b.id
: a.created_at.localeCompare(b.created_at) || a.id - b.id);
const mobile = info.project.name === 'mobile-chrome';
const prefix = kind === 'datasets' ? 'datasets' : 'dataset.snapshots';
const pager = `${prefix}.pagination.${mobile ? 'mobile' : 'desktop'}`;
const item = (id: number) => page.getByTestId(`${prefix}.${mobile ? 'card' : 'row'}.${id}`);
const cursors: number[] = [];
await installHaveApiMock(page, {
user: { id: 1, login: 'admin', level: 99 },
handlers: {
'GET datasets/10': () => dataset,
[kind === 'datasets' ? 'GET datasets' : 'GET datasets/10/snapshots']: ({ searchParams }) => {
const cursor = Number(searchParams.get(`${kind === 'datasets' ? 'dataset' : 'snapshot'}[from_id]`) ?? 0);
cursors.push(cursor);
const start = cursor ? rows.findIndex(row => row.id === cursor) + 1 : 0;
return { [kind]: rows.slice(start, start + 26) };
},
},
});
await page.goto(`/admin/datasets${kind === 'snapshots' ? '/10/snapshots' : ''}?limit=25`);
await page.getByTestId(`${pager}.next`).click();
await expect(item(rows[25].id)).toBeVisible();
await page.getByTestId(`${pager}.prev`).click();
await expect(item(rows[0].id)).toBeVisible();
rows = rows.slice(1);
await page.reload();
await expect(item(rows[24].id)).toBeVisible();
await page.getByTestId(`${pager}.next`).click();
await expect(item(rows[25].id)).toBeVisible();
expect(cursors.at(-1)).toBe(rows[24].id);
await page.getByTestId(`${pager}.prev`).click();
rows = rows.slice(0, 25);
await page.reload();
await expect(item(rows[24].id)).toBeVisible();
await expect(page.getByTestId(`${pager}.next`)).toBeDisabled();
});
}
1 change: 1 addition & 0 deletions src/i18n/locales/cs/storage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { csStorageExports } from "./storage/exports";
import { csDatasetExpansion } from './storage/dataset_expansion';
import { csBackups } from './storage/backups';
export const csStorage = {
"datasets.pagination.restart": "Zpět na první stránku",
...csBackups,
"datasets.list.title": "Datasety",
"datasets.list.description":
Expand Down
1 change: 1 addition & 0 deletions src/i18n/locales/en/storage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { enStorageExports } from "./storage/exports";
import { enDatasetExpansion } from './storage/dataset_expansion';
import { enBackups } from './storage/backups';
export const enStorage = {
"datasets.pagination.restart": "Back to first page",
...enBackups,
"datasets.list.title": "Datasets",
"datasets.list.description":
Expand Down
4 changes: 4 additions & 0 deletions src/lib/api/datasets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ export async function fetchDatasets(opts?: {
count?: boolean;
reversed?: boolean;
role?: 'primary' | 'hypervisor';
signal?: AbortSignal;
}) {
const params: Record<string, unknown> = {};
if (opts?.fromId !== undefined) params['from_id'] = opts.fromId;
Expand All @@ -154,6 +155,7 @@ export async function fetchDatasets(opts?: {
const res = await haveApiCall<Dataset[]>({
method: 'GET',
path: '/datasets',
signal: opts?.signal,
namespace: 'dataset',
params,
meta:
Expand Down Expand Up @@ -233,6 +235,7 @@ export async function fetchDatasetSnapshots(datasetId: number, opts?: {
fromId?: number;
limit?: number;
count?: boolean;
signal?: AbortSignal;
}) {
const params: Record<string, unknown> = {};
if (opts?.fromId !== undefined) params['from_id'] = opts.fromId;
Expand All @@ -241,6 +244,7 @@ export async function fetchDatasetSnapshots(datasetId: number, opts?: {
const res = await haveApiCall<Snapshot[]>({
method: 'GET',
path: `/datasets/${datasetId}/snapshots`,
signal: opts?.signal,
namespace: 'snapshot',
params,
meta: opts?.count ? { count: true } : undefined,
Expand Down
7 changes: 4 additions & 3 deletions src/pages/app/datasets/DatasetPageSearchState.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export function DatasetPageSearchEmpty(props: {
hasMore: boolean;
hasSourceRows: boolean;
onClear: () => void;
onNext: () => void;
}) {
const { t } = useI18n();
const { pagination } = props;
Expand All @@ -37,15 +38,15 @@ export function DatasetPageSearchEmpty(props: {
actionLabel={t('common.clear_filters')}
onAction={props.onClear}
/>
{props.hasSourceRows ? (
{props.hasSourceRows || pagination.canPrev ? (
<Card>
<KeysetPagination
page={pagination.page}
pageCount={pagination.stack.length}
canPrev={pagination.canPrev}
canNext={pagination.hasForward || (props.hasMore && props.pageCursor !== null)}
canNext={props.hasMore && props.pageCursor !== null}
onPrev={pagination.goPrev}
onNext={() => pagination.goNext(props.pageCursor)}
onNext={props.onNext}
onGoToPage={pagination.goToPage}
limit={pagination.limit}
allowedLimits={pagination.allowedLimits}
Expand Down
2 changes: 1 addition & 1 deletion src/pages/app/datasets/DatasetSnapshotsPage.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ describe('DatasetSnapshotsPage', () => {
await waitFor(() =>
expect(api.fetchDatasetSnapshots).toHaveBeenCalledWith(
10402,
{ limit: 51, fromId: undefined, count: true }
{ limit: 51, fromId: undefined, count: true, signal: expect.any(AbortSignal) }
)
);
expect(screen.queryByTestId('dataset.snapshots.search.input')).not.toBeInTheDocument();
Expand Down
28 changes: 21 additions & 7 deletions src/pages/app/datasets/DatasetSnapshotsPage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ import {
import { formatErrorMessage } from '../../../lib/errors';
import { formatDateTime } from '../../../lib/format';
import { useKeysetPagination } from '../../../lib/hooks/useKeysetPagination';
import { cursorFromAscendingPage } from '../../../lib/lockIndex';
import { hasActiveChains } from '../../../lib/taskStatus';

import { useDatasetContext } from './DatasetContext';
Expand Down Expand Up @@ -95,6 +94,7 @@ export function DatasetSnapshotsPage({ queryParamPrefix = '' }: DatasetSnapshots
paramPrefix: queryParamPrefix,
defaultLimit: 50,
allowedLimits: [25, 50, 100],
restoreUrlCursorOnSignatureChange: true,
});
const [createOpen, setCreateOpen] = useState(false);
const [createLabel, setCreateLabel] = useState('');
Expand All @@ -116,13 +116,14 @@ export function DatasetSnapshotsPage({ queryParamPrefix = '' }: DatasetSnapshots

const snapsQ = useQuery({
queryKey: ['datasets', dataset.id, 'snapshots', { limit: pagination.limit, fromId: pagination.fromId }],
queryFn: async () =>
queryFn: async ({ signal }) =>
fetchDatasetSnapshots(dataset.id, {
// HaveAPI's cursor does not expose an end marker. Fetch one extra row so
// Next remains correct for exact-size pages, deep links and count churn.
limit: pagination.limit + 1,
fromId: pagination.fromId,
count: true,
signal,
}),
});

Expand Down Expand Up @@ -295,8 +296,17 @@ export function DatasetSnapshotsPage({ queryParamPrefix = '' }: DatasetSnapshots
const rows = pageData.slice(0, pagination.limit);
const totalCount = reportedTotalCount ?? rows.length;

const pageCursor = useMemo(() => cursorFromAscendingPage(rows as any), [rows]);
const hasMore = pagination.hasForward || pageData.length > pagination.limit;
// The API anchors in (created_at, id) order, not numeric ID order.
const pageCursor = rows.at(-1)?.id ?? null;
const hasMore = !snapsQ.isFetching && !snapsQ.isError
&& pageData.length > pagination.limit && Number.isSafeInteger(pageCursor) && Number(pageCursor) > 0;
const restart = () => pagination.goToPageWithStack(1, [null]);
const goNext = () => {
if (!hasMore || pageCursor == null) return;
pagination.goToPageWithStack(pagination.page + 1, [
...pagination.stack.slice(0, pagination.index + 1), pageCursor,
]);
};

function requestSnapshotDownload(s: Snapshot) {
createDl.mutate(s);
Expand Down Expand Up @@ -397,7 +407,11 @@ export function DatasetSnapshotsPage({ queryParamPrefix = '' }: DatasetSnapshots
testId="dataset.snapshots.error"
title={t('dataset.snapshots.load_error.title')}
error={snapsQ.error}
onRetry={() => void snapsQ.refetch()}
actions={{
primary: { label: t('common.retry'), onClick: () => snapsQ.refetch() },
secondary: pagination.cursor != null
? { label: t('datasets.pagination.restart'), onClick: restart } : undefined,
}}
showBack={false}
detailsExtra={{ page: 'dataset.snapshots', datasetId: dataset.id }}
/>
Expand Down Expand Up @@ -543,7 +557,7 @@ export function DatasetSnapshotsPage({ queryParamPrefix = '' }: DatasetSnapshots
canPrev={pagination.canPrev}
canNext={hasMore}
onPrev={pagination.goPrev}
onNext={() => pagination.goNext(pageCursor)}
onNext={goNext}
onGoToPage={pagination.goToPage}
limit={pagination.limit}
allowedLimits={pagination.allowedLimits}
Expand All @@ -561,7 +575,7 @@ export function DatasetSnapshotsPage({ queryParamPrefix = '' }: DatasetSnapshots
canPrev={pagination.canPrev}
canNext={hasMore}
onPrev={pagination.goPrev}
onNext={() => pagination.goNext(pageCursor)}
onNext={goNext}
onGoToPage={pagination.goToPage}
limit={pagination.limit}
allowedLimits={pagination.allowedLimits}
Expand Down
Loading
Loading