Verify the imported catalog end to end #23

Closed
opened 2026-11-07 21:51:00 +00:00 by martin · 1 comment
Owner

Goal

Confirm that generated catalog files produce the expected usable catalog through the authenticated Backstage API and browser.

Acceptance criteria

  • Authenticated API checks verify representative Forgejo components, ownership, system membership, and recurring-events-api.
  • The OpenAPI definition loads and the provides-API relationship resolves.
  • Browser coverage checks repository links, service links, and API navigation.
  • Automated tests use controlled fixtures instead of depending on live Forgejo.
  • Document a separate optional live verification procedure without exposing tokens.

References

examples/services.yaml, app-config.yaml, scripts/sync-forgejo-catalog.mjs, and the catalog verification item in TODO.md.

Phase

1 — Trustworthy foundation

## Goal Confirm that generated catalog files produce the expected usable catalog through the authenticated Backstage API and browser. ## Acceptance criteria - [ ] Authenticated API checks verify representative Forgejo components, ownership, system membership, and `recurring-events-api`. - [ ] The OpenAPI definition loads and the provides-API relationship resolves. - [ ] Browser coverage checks repository links, service links, and API navigation. - [ ] Automated tests use controlled fixtures instead of depending on live Forgejo. - [ ] Document a separate optional live verification procedure without exposing tokens. ## References `examples/services.yaml`, `app-config.yaml`, `scripts/sync-forgejo-catalog.mjs`, and the catalog verification item in `TODO.md`. ## Phase 1 — Trustworthy foundation
Author
Owner

Implemented in #35 (test/verify-catalog-23), which closes this issue on merge.

What changed

  • Added packages/app/e2e-tests/catalog-api.test.ts: guest sign-in, then authenticated reads from /api/catalog/entities/by-name/... asserting project slug, user:default/martin ownership, personal-services membership, and repository/service links for toolbox, martials-no, and the system.
  • Added packages/app/e2e-tests/catalog-browser.test.ts: browser sign-in, component link checks, and navigation from the API list to the rendered OpenAPI definition.
  • Added controlled fixtures in packages/app/e2e-tests/fixtures/catalog and a CATALOG_DIR override in app-config.yaml; the Playwright webServer sets it to the fixtures, so the suite needs no Forgejo token or live data. E2E_CATALOG_DIR selects another catalog.
  • Verified recurring-events-api embeds its OpenAPI definition and resolves apiProvidedBy/providesApi against component:default/recurring-events.
  • Documented the automated suite and the optional live procedure in README.md#catalog-verification.

Catalog data fixed while verifying

  • Enrichment is keyed by repository name instead of full name, because repositories moved from martin/* to Personal/*; examples/forgejo-repos.yaml was regenerated.
  • Owners now use user:default/martin; the former owner: martin resolved to a nonexistent group.
  • OpenAPI links/definition now point at Personal/recurring-events on main.

Validation (run in nix develop)

  • yarn prettier:check — pass
  • yarn lint:all — pass
  • yarn tsc — pass
  • yarn test:ci --runInBand — 1 suite / 1 test pass
  • yarn build:backend — pass
  • yarn test:e2e --reporter=line — 5 passed (app + 2 API + 2 browser)

The Nix dev shell now provides Chromium and exports PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH, so the browser checks run on NixOS.

Implemented in #35 (`test/verify-catalog-23`), which closes this issue on merge. **What changed** - Added `packages/app/e2e-tests/catalog-api.test.ts`: guest sign-in, then authenticated reads from `/api/catalog/entities/by-name/...` asserting project slug, `user:default/martin` ownership, `personal-services` membership, and repository/service links for `toolbox`, `martials-no`, and the system. - Added `packages/app/e2e-tests/catalog-browser.test.ts`: browser sign-in, component link checks, and navigation from the API list to the rendered OpenAPI definition. - Added controlled fixtures in `packages/app/e2e-tests/fixtures/catalog` and a `CATALOG_DIR` override in `app-config.yaml`; the Playwright `webServer` sets it to the fixtures, so the suite needs no Forgejo token or live data. `E2E_CATALOG_DIR` selects another catalog. - Verified `recurring-events-api` embeds its OpenAPI definition and resolves `apiProvidedBy`/`providesApi` against `component:default/recurring-events`. - Documented the automated suite and the optional live procedure in `README.md#catalog-verification`. **Catalog data fixed while verifying** - Enrichment is keyed by repository name instead of full name, because repositories moved from `martin/*` to `Personal/*`; `examples/forgejo-repos.yaml` was regenerated. - Owners now use `user:default/martin`; the former `owner: martin` resolved to a nonexistent group. - OpenAPI links/definition now point at `Personal/recurring-events` on `main`. **Validation** (run in `nix develop`) - `yarn prettier:check` — pass - `yarn lint:all` — pass - `yarn tsc` — pass - `yarn test:ci --runInBand` — 1 suite / 1 test pass - `yarn build:backend` — pass - `yarn test:e2e --reporter=line` — 5 passed (app + 2 API + 2 browser) The Nix dev shell now provides Chromium and exports `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH`, so the browser checks run on NixOS.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
Personal/backstage#23
No description provided.