Sync architecture
The docs portal aggregates workflow documentation using a Git-backed mirror with GitLab multi-project pipeline automation.
Architectural decision: Model A (Git mirrors)
Chosen model: workflow docs are stored in the portal repository under docs-<workflow>/ after each sync.
| Model | Description | Verdict |
|---|---|---|
| A — Git mirrors | Portal repo contains mirrored Markdown; CI updates mirrors deterministically | Default |
| B — Fetch at build | Portal pipeline clones workflow docs only at build time; no mirror in Git | Rejected for this project |
Why Model A
- Reviewability: merge requests show exactly what will be published
- Traceability: Git history records each sync commit with workflow ref context
- Reproducibility: a portal commit SHA fully determines site content
- Local development: developers run the same Python sync locally without CI tokens
- Agent clarity: mirrored paths match Docusaurus content roots explicitly
Tradeoffs of Model B (rejected)
- Smaller portal repo, but MRs do not show imported doc diffs
- Harder offline development and bisect
- Build-only failures surface late without a reviewable mirror commit
Components
┌──────────────────────── workflow repo ────────────────────────┐
│ docs/ authoritative workflow Markdown │
│ docs/workflow-manifest.yaml machine-readable sync contract │
│ docs/portal-integration.md human portal integration guide │
│ docs/navigation-meta.md navigation notes (not synced) │
│ .gitlab-ci.yml validate + trigger docs-portal │
└────────────────────────────┬──────────────────────────────────┘
│ downstream pipeline (strategy: depend)
▼
┌──────────────────────── docs-portal ──────────────────────────────┐
│ workflows/registry.yaml portal registry of known workflows │
│ scripts/sync_workflow.py cross-platform Python sync CLI │
│ docs-<workflow>/ mirrored Markdown (+ portal intro) │
│ sidebars<Workflow>.ts Docusaurus navigation │
│ .gitlab/workflow-sync.yml sync + validate CI jobs │
└─────────────────────────────────────────────────────────────────┘
Ownership boundaries
| Content | Owner | Portal path |
|---|---|---|
| Workflow policy docs | Workflow repo docs/ | docs-<workflow>/ (mirrored) |
Section landing intro.md | Portal | docs-<workflow>/intro.md (never overwritten by sync) |
| Portal contract metadata | Workflow repo | Not mirrored (exclude_from_sync) |
| Common / cross-workflow docs | Portal | docs-common/ |
Companion assets (templates, scripts)
Workflow repos may ship non-page assets under directories such as templates/. These must not live under docs-<workflow>/ — Docusaurus would parse .md templates as MDX and fail on placeholder syntax.
Use manifest companion_static_dirs to mirror assets into static/<workflow>-assets/… and link_rewrites with the Docusaurus pathname:// protocol so static downloads bypass SPA route checks (for example pathname:///superbuild-assets/templates/project-migration-template.md).
Python tooling (cross-platform)
All sync automation is Python 3.11+ using pathlib, shutil, and hashlib — no cp, rsync, or shell scripts.
| Command | Purpose |
|---|---|
python scripts/sync_workflow.py <id> [--source PATH] | Sync one workflow |
python scripts/sync_all_workflows.py | Sync all active registry entries |
python scripts/validate_workflow_import.py <id> | Validate contract + mirror |
Local sibling clone resolution: if ../superbuild-workflow exists next to docs-portal, sync uses it without Git credentials.
Dependency: PyYAML only (requirements-sync.txt) for registry and manifests.
Automated commits
When PORTAL_SYNC_COMMIT=true (downstream triggers), the portal pipeline creates a commit:
chore(sync): Update mirrored docs for <workflow_id>
with footer AI: no and source ref in the body. Do not hand-edit mirrored files — changes will be overwritten on the next sync.
Related documents
- Automation model — GitLab pipeline flow
- Workflow onboarding — add a new workflow
- Content ownership — edit boundaries