Qortal-Nextcloud-Integration/README.md

# Qortal Nextcloud Integration

This repository bootstraps a local development setup for integrating Qortal identity/auth flows with Nextcloud.

## Current scope

1. Local Nextcloud development stack via Docker Compose.
2. A TypeScript broker service for:
   - Nextcloud provisioning bridge (create/link users).
   - OIDC authorization-code flow (`/authorize`, `/token`, `/userinfo`, `/jwks`).
3. Nextcloud app for:
   - admin setup + onboarding workflows,
   - user self‑service wallet import/linking,
   - Q‑Apps gateway proxy + embedded launcher,
   - Qortal Account dashboard (live auth validation).
4. Initial architecture and phased implementation notes.

## Repo layout

- `docker-compose.yml`: MySQL/MariaDB local stack.
- `docker-compose.postgres.yml`: PostgreSQL local stack.
- `.env.example`: environment variables for local stack.
- `.env.postgres.example`: environment variables for PostgreSQL stack.
- `Makefile`: helper commands for startup and Nextcloud `occ` tasks.
- `docs/concept.md`: phase plan, contracts, and milestones.
- `docs/external-auth-integration.md`: Qortal External Auth hookup and commands.
- `docs/master-goals.md`: full goals, architecture boundaries, and delivery phases.
- `docs/nextcloud-admin-app.md`: usage for the custom Nextcloud admin app.
- `docs/nextcloud-app-roadmap.md`: roadmap for the custom Nextcloud app.
- `docs/nextcloud-oidc-setup.md`: working `user_oidc` configuration + policy notes.
- `docs/nextcloud-vm-setup.md`: production-style setup for Nextcloud VM.
- `docs/devprod-ssl.md`: HTTPS-enabled dev-prod Docker stack.
- `docs/features-list.md`: full current feature inventory.
- `docs/settings-matrix.md`: admin/personal settings behavior matrix.
- `docs/scripts-reference.md`: script usage and recommended execution flows.
- `docs/testing-guide.md`: end-to-end onboarding test paths and expected results.
- `nextcloud/custom_apps/qortal_integration`: custom admin app for Qortal management inside Nextcloud.
- `nextcloud/html`: bind-mounted Nextcloud web root for Docker stacks.
- `nextcloud/data`: bind-mounted Nextcloud data directory for Docker stacks.
- `services/qortal-oidc-broker`: Node/TypeScript broker + provisioning bridge starter.
- `external-auth/data`: bind-mounted data for bundled External Auth container.
- `qortal/data`: bind-mounted data for bundled Qortal node container.

## Quickstart

1. Run the guided dev setup:
   - `./start-dev.sh`
2. Follow logs until Nextcloud is installed:
   - `make logs`
3. Open Nextcloud:
   - `http://localhost:8080`
4. Install the OIDC app in Nextcloud:
   - `make install-oidc`
5. Enable the Qortal integration app:
   - `make occ cmd="app:enable qortal_integration"`

Dev‑prod (HTTPS) quickstart:

1. Run guided dev‑prod setup:
   - `./start-devprod.sh`
2. Rebuild/recreate after env changes:
   - `./recreate-devprod.sh`
   - (this now disables/enables the app automatically)

Production-oriented Docker bootstrap:

1. Run installer baseline:
   - `./scripts/install-production-docker.sh --mode nossl --with-external-auth`
2. Verify services:
   - `docker compose -f docker-compose.devprod.nossl.yml --env-file .env.devprod ps`

PostgreSQL option:

1. (Optional) copy PG env file:
   - `cp .env.postgres.example .env.postgres`
2. Start PostgreSQL stack:
   - `make up-pg`
3. Stop PostgreSQL stack:
   - `make down-pg`

## Useful commands

- MySQL env behavior: `Makefile` uses `.env` if present, otherwise `.env.example`.
- PostgreSQL env behavior: `Makefile` uses `.env.postgres` if present, otherwise `.env.postgres.example`.
- Start: `make up`
- Stop: `make down`
- Logs: `make logs`
- List services: `make ps`
- Run Nextcloud `occ`: `make occ cmd="app:list"`
- Start (PostgreSQL): `make up-pg`
- Stop (PostgreSQL): `make down-pg`
- Logs (PostgreSQL): `make logs-pg`
- List services (PostgreSQL): `make ps-pg`
- Run Nextcloud `occ` (PostgreSQL): `make occ-pg cmd="app:list"`
- Broker logs: `docker compose logs -f broker`

## Qortal External Auth hookup

The broker now supports authenticated calls to your External Auth API.

1. Register an app in your external auth API (or reuse an existing app):
   - `POST /apps/register`
2. Set these in `.env` (or `.env.postgres`):
   - `BROKER_INTERNAL_API_TOKEN` (required for broker internal APIs)
   - `BROKER_CORS_ALLOWED_ORIGINS` (recommended: your Nextcloud public URL)
   - `QORTAL_EXTERNAL_AUTH_BASE_URL` (for Docker Desktop: usually `http://gateway.docker.internal:3191`)
   - `QORTAL_EXTERNAL_AUTH_APP_ID`
   - `QORTAL_EXTERNAL_AUTH_APP_SECRET`
   - If your node requires API keys, also set:
     - `QORTAL_AUTH_NODE_API_KEY`
     - `QORTAL_AUTH_NODE_API_KEY_MODE`
     - `QORTAL_AUTH_NODE_API_KEY_PATHS`
3. Restart broker:
   - `make up`

Useful broker endpoints:
- `GET /api/qortal/health` (connectivity check)
- `GET /api/qortal/wallets` (wallets accessible to configured app creds)
- `POST /api/qortal/wallets` (create wallet via configured app creds)
- `GET /api/oidc/allowlist` (list allowlisted Qortal addresses for auto-provision)
- `POST /api/oidc/allowlist` (add allowlisted Qortal address)
- `POST /api/oidc/allowlist/remove` (remove allowlisted Qortal address)
- `GET /api/oidc/invites` (list invite tokens)
- `POST /api/oidc/invites` (create invite token)
- `POST /api/oidc/invites/revoke` (revoke invite token)
- `POST /api/provision/upsert-from-wallet` (provision/link via walletId -> address0)
- `POST /api/provision/import-seed-link` (import existing seed + link to Nextcloud user)
- `POST /api/provision/import-backup-link` (import encrypted backup JSON + link to Nextcloud user)
- `POST /api/provision/unlink` (remove mapping)
- `GET /api/provision/mappings/by-nextcloud/:nextcloudUserId` (list mappings for one Nextcloud user)
- `GET /api/provision/mappings` (list persisted mappings)

Internal API security:
- Broker internal APIs under `/api/qortal/*`, `/api/provision/*`, and `/api/oidc/*` require `BROKER_INTERNAL_API_TOKEN`.
- Nextcloud app sends this token automatically when configured via:
  - Env in app container: `QORTAL_BROKER_INTERNAL_API_TOKEN` (recommended for docker stacks), or
  - Nextcloud Admin setting: `Broker Internal API Token`.
- In bundled devprod stacks, helper scripts auto-generate the token in `.env.devprod`.

Custom Nextcloud app:
- Open Nextcloud Admin settings and go to `Qortal Integration` section.
- Configure broker URL and test connectivity directly from Nextcloud UI.
- Run setup actions from UI:
  - wallet create/list
  - mapping link/list
  - allowlist and invite management for guarded auto-provision
- User self-service panel in Personal Settings:
  - import existing wallet by seed or backup JSON + link to current user
  - remove linked accounts
  - list linked accounts for current user
- Qortal Account dashboard:
  - `/apps/qortal_integration/account` (live auth validation)
- Q-Apps launcher:
  - `/apps/qortal_integration/qapps` (embedded gateway)

Broker persistence:
- Broker mappings are now stored in PostgreSQL (`identity_mapping` table).
- Configure via `BROKER_DATABASE_URL` and broker DB env vars in `.env`.

## Bundled Qortal node

All compose stacks now include a `qortal_node` container by default:

- Build context defaults to `QORTAL_NODE_CONTEXT=../qortal`
- Node API is published to host via:
  - `QORTAL_NODE_API_BIND_HOST` / `QORTAL_NODE_API_HOST_PORT` (default `127.0.0.1:12391`)
- Node P2P is published to host via:
  - `QORTAL_NODE_P2P_BIND_HOST` / `QORTAL_NODE_P2P_HOST_PORT` (default `0.0.0.0:12392`)
- Node QDN data is published to host via:
  - `QORTAL_NODE_QDN_BIND_HOST` / `QORTAL_NODE_QDN_HOST_PORT` (default `0.0.0.0:12394`)
- External Auth defaults to use the internal node URL:
  - `QORTAL_AUTH_NODE_URL=http://qortal_node:12391`
  - `QORTAL_AUTH_NODE_API_KEY` (set this when the node requires `X-API-KEY`)
  - `QORTAL_AUTH_NODE_API_KEY_MODE=paths`
  - `QORTAL_AUTH_NODE_API_KEY_PATHS=/` (send key header for all node API calls)
- Default node settings template:
  - `deploy/templates/qortal/default-node-settings.json`
  - `scripts/ensure-qortal-settings.sh` initializes `qortal/data/settings.json` from this template when missing (never overwrites existing settings).
- Default JVM start args:
  - `scripts/ensure-qortal-start-args.sh` initializes `qortal/data/start-arguments.txt` when missing.
  - Container startup reads that file on every restart, so you can tune memory without reinstall.
  - Default value: `-XX:MaxRAMPercentage=25 -XX:+UseG1GC -Xss1024k`
  - Optional env seed for first-run/empty file: `QORTAL_JVM_MEMORY_ARGS`

When using `./start-dev.sh`, `./start-devprod.sh`, or `./recreate-devprod.sh`,
the script auto-selects `QORTAL_NODE_API_HOST_PORT` in this order:

1. `12391`
2. `22391`, `32391`, `42391`, `52391`
3. `12291`, `22291`, `32291`, `42291`, `52291`

Then it sets:

- `QORTAL_NODE_P2P_HOST_PORT` to `QORTAL_NODE_API_HOST_PORT + 1`
- `QORTAL_NODE_QDN_HOST_PORT` to `QORTAL_NODE_API_HOST_PORT + 3`

## Qortal reverse proxy templates

Installer-ready templates for exposing a local node as both API/render and gateway:

- `deploy/templates/proxy/nginx-qortal-node.conf.template`
- `deploy/templates/proxy/apache-qortal-node.conf.template`
- `deploy/templates/proxy/README.md`

## Smoke test

Create/link a Nextcloud user from the broker:

```bash
curl -sS http://localhost:3000/api/provision/upsert \
  -H "X-Broker-Internal-Token: ${BROKER_INTERNAL_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "qortalAddress": "QTESTADDRESS1234567890ABCDEFG",
    "displayName": "Qortal Test",
    "email": "qortal-test@example.com"
  }'
```

Expected first-call behavior:
- `created: true` with a new mapping created.
- If Nextcloud user creation is needed, broker uses an internal random password that is never returned.

Expected repeated-call behavior:
- `created: false` with same `nextcloudUserId` mapping.

Provision/link from wallet (uses Qortal External Auth):

```bash
curl -sS http://localhost:3000/api/provision/upsert-from-wallet \
  -H "X-Broker-Internal-Token: ${BROKER_INTERNAL_API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "walletId": "YOUR_WALLET_ID",
    "displayName": "Qortal Wallet User",
    "email": "wallet-user@example.com"
  }'
```

## Next step focus

- Add admin onboarding campaign APIs + audit events.
- Add Nextcloud app UI for mapping management and campaign triggering.
- Add stricter signed-nonce verification once External Auth exposes a dedicated verify endpoint.
- Add Q-App launch policy + signed launch token flow.