ERO Operations

What was delivered, and how to run it. This covers workstation onboarding, the secrets lifecycle, local development, and identity validation.

Customer-facing Google federation onboarding is a separate workstream, documented in Customer Google Sign-In Onboarding.

1. Developer onboarding

Onboarding is script-only. A developer needs VS Code, a repository clone, and one command. There are no manual file edits.

Automation is driven by platform/automation/powershell/bootstrap.ps1, wired into VS Code build tasks.

1.1 What the bootstrap does

Host policy and toolchain. Sets the PowerShell execution policy to RemoteSigned for the current user, then installs or verifies Git, Node.js LTS, Terraform, the Google Cloud CLI, and the .NET SDK through winget. Versions are pinned in config/dependencies.json.
Environment and secrets. Runs join-project.ps1 to hydrate the environment file for the selected environment. Key Vault hydration is not currently wired, so values must come from the identity provisioning workflow output.
Database scaffolding. Restores API dependencies, applies schema.sql, and loads seed.sql with stages, bands, walking travel times, and sample preferences.
Editor provisioning. Installs recommended workspace extensions from the generated .vscode/extensions.json, produced from settings/vscode-extensions.json.
Dev services. Launches the Wrangler Worker API and the Astro dev server, with Vite proxying /api/* to the Worker, then opens the application.
MCP validation. Validates the local Azure DevOps and filesystem MCP server binaries.

1.2 Preferred environment

Use one persistent GitHub Codespace as the primary environment when moving between devices, and resume that same Codespace from either device. It retains dependencies, local data, terminals, and browser tooling between sessions.

.devcontainer/devcontainer.json restores root, application, API, and browser-test dependencies along with local HTTPS material. After the Codespace opens:

npm run ero-serve:local
npm run test:ciam:runtime:local

Open forwarded port 4321 through its GitHub HTTPS URL. Keep port 8787 internal to the Codespace. Commit and push before stopping it.

Use a local Windows workspace only when remote access is unavailable. Install mkcert and run npm run ero-https:cert once per device before starting at https://localhost:4321.

2. Secrets lifecycle

Local configuration lives only under settings/ and is never committed. .gitignore must contain settings/ and *.tfvars.

Onboarding scripts resolve every managed key through this lifecycle:

Check for the value in the local environment context.
If absent, check Azure Key Vault.
If absent from Key Vault and the key policy is generated, create the value.
Store the created value in Key Vault.
Store the resolved value in settings/.env.<environment>.local.
If the key already exists in Key Vault, use that value.

Entry points are configure-project-workspace.cmd and join-project.ps1. The Key Vault synchronisation entry point is not currently implemented, so steps 2 through 6 are manual until it is restored.

2.1 Local files

File	Holds
`settings/.env.local`	Global parameters such as Azure DevOps organisation, project, and token
`settings/.env.dev.local`, `.env.test.local`, `.env.prod.local`	Per-environment resolved values
`settings/.dev.vars`	Local Wrangler secrets

Terraform variable files carry the Cloudflare account identifier and API token, and are excluded from version control by the *.tfvars rule.

Automation must never read, display, or print a secret value. It confirms only that the expected variable names resolve.

3. Local development

npm run ero-bootstrap:inner-loop   # dependencies and browser tooling
npm run ero-https:cert             # once per device
npm run ero-serve:local            # D1 preparation, Worker API, and Astro app

ero-serve:local prepares schema.sql and seed.sql against local D1 before starting services.

Deploy Cloudflare and infrastructure changes only through the CI and CD path. Direct Wrangler or Terraform deployment from a workstation is prohibited by AG-IAC-002. Land changes on main through one pull request, then release by tag, as described in Platform Requirements.

4. Identity configuration

4.1 Provider model

The external tenant supports built-in social providers for Google, Facebook, and Apple. Microsoft account sign-in for customers is configured as an OpenID Connect provider and then enabled in the user flow. Organisational federation to an Entra ID tenant uses custom OIDC. Local account methods, email with password and email one-time passcode, remain enabled as fallback and for break-glass validation.

4.2 Delivery rules

Non-production CIAM stays in the sknxerononproduction external tenant; production stays in sknxeroproduction.
Use Synkronyx-owned Google OAuth projects for any shared environment.
Personal Google ownership is acceptable only for an isolated local proof of concept.
For live Microsoft account customer sign-in, configure the provider through OIDC and add it to the user flow.

4.3 Flow naming

This is an Entra External ID (CIAM) tenant, so flows are self-service sign-up flows read and written through beta/identity/authenticationEventsFlows. The legacy b2xUserFlows endpoint belongs to Azure AD B2C, returns 403 here, and the B2C_1_ and B2X_1_ prefixes do not apply.

Tier	User flow
Nonproduction	`ero-sign-up-sign-in-nonproduction`
Production	`ero-sign-up-sign-in-production`

A flow only takes effect for applications listed in its conditions.applications.includeApplications. An application registration must also have a service principal in the tenant before it can be added: without one, Graph rejects the association with The application id <guid> is invalid, which names the wrong cause.

Run platform/automation/powershell/test-ciam-tenant-readiness.ps1 -Tier <tier> to create the flow, create missing service principals, and associate the ERO applications. Add -Check to report without changing anything.

Tenant identifiers and client identifiers are held in ignored local settings and deployment secret stores, never in documentation.

4.4 Inner and outer loop

INNER LOOP (local)
  Browser  https://localhost:4321
  Auth     local mock JWT or development Entra app
  API      http://127.0.0.1:8787 local Wrangler Worker
  Edge     mock JWKS or development token bypass

OUTER LOOP (cloud)
  Browser  https://dev-ero.synkronyx.app
  Identity Entra CIAM tenant with Google, Facebook, and email OTP
  Edge     strict live JWKS signature verification

The client initialises MSAL from PUBLIC_SKNX_ERO_CLIENT_ID, PUBLIC_SKNX_ERO_TENANT_ID, and PUBLIC_SKNX_ERO_TENANT_DOMAIN. PUBLIC_SKNX_ERO_AUTHORITY_HOST is optional and must be set only after an Entra External ID custom auth domain exists, for example login.synkronyx.com. When it is empty, the client uses the tenant ciamlogin.com host. When PUBLIC_SKNX_ERO_AUTH_MOCK is true it returns a mock session with a local test token. The Worker accepts Bearer local-dev-token as dev-user-1 when ENVIRONMENT is dev, and otherwise verifies against https://<tenant>.ciamlogin.com/<tenant-id>/discovery/v2.0/keys.

5. Identity validation

Validation is script-only. Manual portal steps are break-glass fallback, not the primary path. Run from the repository root.

5.0 What runs where

Cloud writes belong to GitHub Actions. A developer device may read cloud state, and may run the first-run federation scripts, because those bootstrap the workload identity that Actions itself depends on.

Capability	Runs in	Notes
Documentation publish and Cloudflare provisioning	GitHub Actions	Local entry points removed. `npm run docs:preview` remains for local preview
Landing zone and subscription deployment	GitHub Actions	`deploy-landingzone.ps1` refuses to run on a device without `-BreakGlassReason`
Entra app registrations, Google provider binding	GitHub Actions	Pending the identity provisioning workflow, see open issue 6
Workload identity federation first run	Developer device	Creates the identity Actions authenticates with, so it cannot run in Actions
CIAM configuration validation	Either	Read-only, safe anywhere
Build, lint, test, inner loop	Developer device	No cloud writes

5.1 Prerequisite: workload identity

Provisioning must run in GitHub Actions, and Actions must hold an Azure identity before any of it works. This is the one step that needs a signed-in administrator, so it is a first-run script rather than a workflow.

# Plan first, then apply
./platform/automation/powershell/first-run-github-federation.ps1 -WhatIf
./platform/automation/powershell/first-run-github-federation.ps1

The script creates the sknx-github-actions application registration, adds OIDC federated credentials for main, pull requests, and each environment, creates the nonproduction subscription against the Microsoft Customer Agreement invoice section recorded in settings/.env.local, assigns Contributor and Role Based Access Control Administrator on each subscription, and publishes AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID as repository variables. It creates no secret.

The CIAM tenants are separate directories, so the workload identity that manages identity providers and user flows must be created inside each of them:

./platform/automation/powershell/first-run-ciam-federation.ps1 -Tier nonproduction -WhatIf
./platform/automation/powershell/first-run-ciam-federation.ps1 -Tier nonproduction
./platform/automation/powershell/first-run-ciam-federation.ps1 -Tier production

Tenant values come from settings/.env.dev.local and settings/.env.prod.local, so no identifier is typed by hand. The script signs in to the external tenant, creates the registration with Application.ReadWrite.All, IdentityProvider.ReadWrite.All, IdentityUserFlow.ReadWrite.All, and User.ReadWrite.All, grants admin consent, lists the real user flow identifiers held by the tenant, and restores the previous Azure CLI context on exit.

Until this has run, every azure/login step in the workflows is skipped by its own guard condition and no Azure provisioning happens at all.

# 1. Provision or reconcile CIAM app registrations
./platform/automation/powershell/deploy/deploy-ciam-app-registrations.ps1

# 2. Reconcile the Google identity provider and flow binding
./platform/automation/powershell/deploy/bootstrap-google-oauth.ps1 -Environment dev  -SkipProjectBootstrap -EnsureGoogleApiKey -UserFlows B2C_1_SUSI_SKNXR_NONPROD
./platform/automation/powershell/deploy/bootstrap-google-oauth.ps1 -Environment test -SkipProjectBootstrap -EnsureGoogleApiKey -UserFlows B2C_1_SUSI_SKNXR_NONPROD
./platform/automation/powershell/deploy/bootstrap-google-oauth.ps1 -Environment prod -SkipProjectBootstrap -EnsureGoogleApiKey -UserFlows B2C_1_SUSI_SKNXR_PROD

# 3. Validate configuration without portal testing
./platform/automation/powershell/deploy/test-ciam-auth-configuration.ps1 -Environment all

# 4. Validate local runtime with Wrangler and a headless browser
npm run test:ciam:runtime:local

Scope validation to one environment with -Environment dev or -Environment test. If a run is blocked by tenant access policy, resolve the tenant role assignment and rerun the same command set.

5.1 Evidence to capture

Take these directly from script output: the command and timestamp, the active account context including tenant, subscription, and user, the OIDC metadata check result, the Google provider presence result, the flow binding result, and the final pass or fail status.

5.2 Interpreting failures

Symptom	Cause
OIDC checks fail	Tenant domain or flow value mismatch in the local environment file
Graph checks fail with access policy errors	Operator role and policy resolution in the CIAM tenant is pending
Google binding fails	The provider exists but is not linked to the target user flow

5.3 Closure criteria

Script-only validation returns success for development, test, and production.
The Google provider exists and flow binding is verified for the target flows.
OIDC metadata checks pass for the target tenant domain and user flow.
Runtime values in environment files match Terraform output values.
Nonproduction alias recovery is complete and pinned in settings/.env.local.
No manual portal action is required for routine validation.

5.4 Open issues

CIAM tenant role assignment and policy configuration for the operator account still blocks full Graph validation in some runs.
The interaction between security defaults and conditional access in CIAM tenants needs an explicit governance decision.
The nonproduction subscription alias still needs successful creation and capture into settings/.env.local.
Subscription recovery and CIAM validation must remain separate execution tracks, because running them together masks CIAM-only blockers.
Google is not yet an identity provider on the sign-up flow. The flow carries EmailPassword-OAUTH only, because creating a Google OAuth web client has no gcloud path and remains a console step.
Provisioning still runs from local scripts. It must move into a GitHub Actions workflow once the workload identity exists, so that the inner loop consumes provisioned configuration rather than creating it.
Key Vault synchronisation for local environment files is not implemented. The previous entry point was a broken shim and has been removed.

6. Ownership

Identity engineering owns CIAM tenant access and policy resolution.
Platform engineering owns deterministic script execution and deployment orchestration.
Documentation stays docs-as-code and must carry operational updates as they happen.