Qortal-Nextcloud-Integration/docs/engineering/nuqloud-scrum-implementation-plan.md

# NuQloud Scrum Implementation Plan

Plan date: 2026-07-03

## 1. Direction

NuQloud Scrum is a Talk-first scrum layer for NuQloud deployments.

Canonical model:

| Nextcloud surface | NuQloud Scrum meaning |
|---|---|
| Talk conversation | Scrum workspace |
| Talk thread / reply chain | Task |
| Talk messages in thread | Task discussion/activity |
| Files shared in conversation | Workspace files |
| Files shared in thread/reply chain | Task files |
| Deck board | Workspace board mirror |
| Deck card | Task mirror |
| NuQloud Scrum DB | Mapping, index, sync state, audit |

Product rule for v1:

1. Talk is the primary UX, especially on mobile.
2. Threads/reply chains are tasks.
3. Files and Deck are mirrors.
4. NuQloud Scrum stores indexes and sync state only.
5. No custom mobile client is required.

## 2. App Boundary

Create a new app:

```text
nextcloud/custom_apps/nuqloud_scrum/
```

Do not merge this into:

1. `qortal_integration` - account, broker, OIDC, publishing controls.
2. `qortal_files_bridge` - QDN/files bridge.
3. `qortal_talk_bridge` - temporary Talk/Q-Chat bridge.
4. `chd_admin` - internal CHD/admin-only work.

Suggested app metadata:

```text
id: nuqloud_scrum
name: NuQloud Scrum
namespace: NuqloudScrum
category: integration
dependencies:
  nextcloud min-version: 32
  nextcloud max-version: 34 initially
  optional apps: spreed, deck
```

Use Powered Mode language by default. Avoid crypto-native labels in this app UI.

## 3. Compatibility Baseline

Verified documentation assumptions on 2026-07-03:

1. Talk chat API base endpoint is `/ocs/v2.php/apps/spreed/api/v1`.
2. Talk chat fetch supports `lastKnownMessageId`, pagination headers, parent message data, and reaction fields.
3. Talk send-message supports `replyTo` for replies.
4. Talk file share uses Files Sharing OCS with `shareType=10` for conversation shares.
5. Talk shared item listings support `lastKnownMessageId` pagination when the relevant capability exists.
6. Deck REST API is authenticated and exposes boards, stacks, cards, assignments, labels, comments, and attachments.
7. Nextcloud app schema changes belong in `lib/Migration`.
8. Background jobs belong in the app bootstrap/job registration path.

Implementation rule:

```text
Native Talk thread metadata available?
  Use Talk thread identifiers directly.

Only parent/reply chain available?
  Treat root message ID + parent relationship as task identity.

No stable thread creation API available?
  Bot posts an anchor task message and users reply to that message.
```

## 4. Target App Structure

```text
nextcloud/custom_apps/nuqloud_scrum/
  appinfo/
    info.xml
    routes.php

  lib/
    AppInfo/
      Application.php

    BackgroundJob/
      SyncConversationsJob.php
      SyncConversationMessagesJob.php
      SyncDeckJob.php
      SyncFilesJob.php
      SummarizeWorkspacesJob.php

    Controller/
      PageController.php
      SettingsController.php
      WorkspaceController.php
      TaskController.php

    Db/
      Workspace.php
      WorkspaceMapper.php
      ThreadTask.php
      ThreadTaskMapper.php
      ThreadMessage.php
      ThreadMessageMapper.php
      MirroredFile.php
      MirroredFileMapper.php
      DeckSync.php
      DeckSyncMapper.php
      SyncCursor.php
      SyncCursorMapper.php

    Migration/
      Version000001DateYYYYMMDDHHMMSS.php

    Service/
      ConfigService.php
      TalkApiClient.php
      TalkCapabilityService.php
      WorkspaceService.php
      ThreadTaskService.php
      FolderMirrorService.php
      FileIndexService.php
      DeckApiClient.php
      DeckMirrorService.php
      ReactionWorkflowService.php
      BotCommandService.php
      SummaryService.php
      SlugService.php
      AuditService.php

  templates/
    index.php
    admin.php

  js/
    index.js
    admin.js

  css/
    style.css
    admin.css

  img/
    app.svg

  l10n/
    .gitkeep
```

Keep the first UI thin:

1. Admin settings: enable/disable, service user config, sync intervals, workspace filters.
2. User page: workspace index, task links, sync state, Deck links, folder links.
3. No replacement scrum board in v1.

## 5. Data Model

### `nuqloud_scrum_workspaces`

```text
id bigint primary key
talk_token varchar(128) not null unique
talk_room_name varchar(255) not null
talk_room_type varchar(64) null
deck_board_id bigint null
root_folder_path varchar(1024) not null
sync_enabled smallint default 1
last_message_id bigint default 0
last_share_message_id bigint default 0
created_by varchar(128) null
created_at bigint not null
updated_at bigint not null
```

Indexes:

```text
uniq talk_token
idx sync_enabled
idx updated_at
```

### `nuqloud_scrum_threads`

```text
id bigint primary key
workspace_id bigint not null
talk_thread_id varchar(128) null
talk_root_message_id bigint not null
title varchar(255) not null
slug varchar(255) not null
folder_path varchar(1024) not null
deck_card_id bigint null
deck_stack_id bigint null
status varchar(32) default 'inbox'
priority varchar(32) default 'normal'
created_by varchar(128) null
assigned_to varchar(128) null
due_at bigint null
last_activity_message_id bigint default 0
created_at bigint not null
updated_at bigint not null
```

Indexes:

```text
uniq workspace_id + talk_root_message_id
uniq workspace_id + talk_thread_id when available
idx workspace_id + status
idx deck_card_id
```

### `nuqloud_scrum_thread_messages`

```text
id bigint primary key
thread_id bigint not null
talk_message_id bigint not null
actor_type varchar(64) not null
actor_id varchar(128) not null
actor_display_name varchar(255) null
message_text clob null
message_hash varchar(128) not null
raw_payload clob null
created_at bigint not null
updated_at bigint not null
```

Indexes:

```text
uniq thread_id + talk_message_id
idx actor_id
idx created_at
```

### `nuqloud_scrum_files`

```text
id bigint primary key
workspace_id bigint not null
thread_id bigint null
talk_message_id bigint null
nc_file_id bigint null
original_path varchar(1024) not null
workspace_path varchar(1024) not null
checksum varchar(128) null
size bigint default 0
mime_type varchar(255) null
mirror_state varchar(32) default 'pending'
sync_error clob null
created_at bigint not null
updated_at bigint not null
```

Indexes:

```text
idx workspace_id + thread_id
idx nc_file_id
idx mirror_state
```

### `nuqloud_scrum_deck_sync`

```text
id bigint primary key
workspace_id bigint not null
thread_id bigint null
deck_board_id bigint null
deck_stack_id bigint null
deck_card_id bigint null
last_sync_at bigint null
sync_state varchar(32) default 'pending'
sync_error clob null
created_at bigint not null
updated_at bigint not null
```

### `nuqloud_scrum_sync_cursors`

```text
id bigint primary key
cursor_type varchar(64) not null
scope_key varchar(255) not null
cursor_value varchar(255) not null
locked_until bigint null
updated_at bigint not null
```

Indexes:

```text
uniq cursor_type + scope_key
idx locked_until
```

## 6. Folder Layout

Root folder:

```text
/NuQloud Scrum/
```

Workspace folder:

```text
/NuQloud Scrum/{workspace-name}/
  _workspace.md
  _manifest.json
  _open-tasks.md
  _decisions.md
  Files/
  Threads/
  Summaries/
```

Task folder:

```text
/NuQloud Scrum/{workspace-name}/Threads/
  {root-message-id}-{slug}/
    _task.md
    _activity.jsonl
    _summary.md
    Files/
    Decisions/
```

Rules:

1. Prefix folders with immutable IDs.
2. Slugs are display-only and may be regenerated.
3. DB identity always wins over path names.
4. Folder moves are detected and healed through `file_id` where possible.
5. Generated markdown must escape raw user text.

## 7. Sync Flow

### Conversation discovery

`SyncConversationsJob`:

1. Fetch configured Talk conversations visible to the automation/service user.
2. Apply workspace filters:
   - all conversations
   - named allowlist
   - group/member filter
3. `ensureWorkspace(token)`.
4. Create root folder and manifest.
5. Ensure Deck board when Deck is enabled.

### Message sync

`SyncConversationMessagesJob`:

1. For each enabled workspace, fetch Talk messages after `last_message_id`.
2. Store raw payload only server-side.
3. Detect task root:
   - native `threadId` root when available
   - parent/reply root message ID fallback
   - bot anchor message fallback
4. `ensureThreadTask`.
5. Append message record if it belongs to a task.
6. Apply reactions.
7. Queue Deck and file sync.
8. Advance cursor only after successful persistence.

### File sync

`SyncFilesJob`:

1. Fetch shared items from Talk where capability allows it.
2. Resolve target:
   - thread/task folder when message has thread/root context
   - workspace `Files/YYYY-MM/` otherwise
3. Copy or link according to admin setting:
   - v1 default: copy into NuQloud Scrum folder
   - future option: retain original and write folder link only
4. Record file ID, checksum, size, MIME type.
5. Avoid duplicate copies by `nc_file_id + target_path`.

### Deck sync

`SyncDeckJob`:

1. Ensure one Deck board per workspace.
2. Ensure default stacks:
   - Inbox
   - Ready
   - In Progress
   - Blocked
   - Review
   - Done
   - Archived
3. Ensure one Deck card per thread task.
4. Mirror status to stack.
5. Update card description with:
   - Talk room link
   - thread/task folder link
   - latest summary
   - source IDs
6. Avoid syncing every Talk reply as a Deck comment in v1 unless explicitly enabled.

## 8. Reaction Workflow

Default mappings:

| Reaction | Status/action |
|---|---|
| check mark | Done |
| construction | Blocked |
| eyes | Review |
| fire | Priority |
| pushpin | Decision |
| folder | File/context marker |

Implementation notes:

1. Store reaction-derived status changes in DB.
2. Prefer latest trusted actor reaction if conflicts exist.
3. Allow admins to disable reaction mappings.
4. Do not rely on client-only state.
5. Reaction values must be normalized before matching.

## 9. Bot Commands

Use bot-style messages, not legacy slash commands.

Supported v1 commands:

```text
@NuQloud status blocked
@NuQloud status done
@NuQloud assign @username
@NuQloud due 2026-07-10
@NuQloud summarize
@NuQloud help
```

Rules:

1. Commands only work in authenticated Talk conversations.
2. Only room participants can affect that workspace.
3. Admin-configured moderators can override any task.
4. User input is parsed with strict allowlists.
5. Bot responses must not expose raw payloads, secrets, or stack traces.

## 10. Security Requirements

Public endpoints:

1. None in v1.

Authenticated endpoints:

1. Workspace/task index pages.
2. Per-user workspace/task JSON APIs.

Admin endpoints:

1. App enable/disable.
2. Service user/app-password settings.
3. Workspace sync filters.
4. Reaction/Deck/file mirroring settings.
5. Manual resync/repair actions.

Required controls:

1. Use Nextcloud controller auth for all routes.
2. Use CSRF checks for state-changing browser routes.
3. Validate all IDs, paths, statuses, dates, and command arguments.
4. Escape user message text in generated markdown and UI.
5. Never log app passwords, raw auth headers, invite tokens, or raw message bodies.
6. Store service credentials in app config with admin-only access.
7. Enforce participant/admin checks server-side.
8. Treat Deck and folder mirrors as derived state, not authority.

## 11. Admin Settings

Initial settings:

```text
nuqloud_scrum_enabled: 0|1
talk_service_user: string
talk_service_app_password: secret
workspace_filter_mode: all|allowlist|disabled
workspace_allowlist: JSON array of Talk tokens/names
deck_sync_enabled: 0|1
file_mirror_enabled: 0|1
file_mirror_mode: copy|link
reaction_workflow_enabled: 0|1
bot_commands_enabled: 0|1
summary_enabled: 0|1
sync_interval_minutes: integer
```

Admin UI should use theme variables, l10n strings, and existing Nextcloud settings patterns.

## 12. Implementation Phases

### Phase 0 - API validation spike

Goal: prove target Talk/Deck behavior on devcloud before committing to event semantics.

Tasks:

1. Confirm Talk capabilities for target Nextcloud/Talk version.
2. Confirm whether native thread identifiers appear in fetched message payloads.
3. Confirm reply-chain fallback using `parent` and `replyTo`.
4. Confirm reaction payload shape.
5. Confirm file share metadata includes usable thread/root context.
6. Confirm Deck card create/update/comment/attachment endpoints.
7. Decide service account pattern for background jobs.

Deliverable:

```text
docs/engineering/nuqloud-scrum-api-validation.md
```

### Phase 1 - app skeleton and settings

Tasks:

1. Add `nuqloud_scrum` app scaffold.
2. Add `info.xml`, bootstrap, routes, admin settings page.
3. Add config service.
4. Add app icon and minimal themed CSS.
5. Add l10n-ready strings.
6. Register background jobs disabled until config is valid.

Acceptance:

1. App installs/enables.
2. Admin can save settings.
3. Missing Talk/Deck apps show clear admin warnings.
4. No public routes exist.

### Phase 2 - migrations and persistence

Tasks:

1. Add migration for workspace/thread/message/file/deck/cursor tables.
2. Add entities/mappers.
3. Add repository-level unit tests where practical.
4. Add idempotent upsert methods.
5. Add sync cursor locking.

Acceptance:

1. Install creates tables.
2. Upgrade path is repeatable.
3. Duplicate Talk messages do not create duplicate tasks.

### Phase 3 - Talk workspace discovery

Tasks:

1. Implement `TalkApiClient`.
2. Implement `TalkCapabilityService`.
3. Implement `WorkspaceService`.
4. Implement `SyncConversationsJob`.
5. Create `/NuQloud Scrum/{conversation}/` folders.
6. Write `_workspace.md` and `_manifest.json`.

Acceptance:

1. Allowed Talk conversations become workspaces.
2. Re-running sync is idempotent.
3. Folder names are sanitized.
4. Immutable Talk token remains the workspace key.

### Phase 4 - thread/task indexing

Tasks:

1. Implement `ThreadTaskService`.
2. Implement task root detection.
3. Implement title extraction:
   - explicit thread title when available
   - first sentence/message summary fallback
   - safe length cap
4. Create task folders.
5. Write `_task.md` and `_activity.jsonl`.
6. Track `last_message_id` per workspace.

Acceptance:

1. Talk replies create/update task records.
2. Existing reply chains backfill correctly.
3. Generated files escape raw Talk content.
4. Task folders use `{root-message-id}-{slug}`.

### Phase 5 - file mirroring

Tasks:

1. Implement `FileIndexService`.
2. Implement `FolderMirrorService`.
3. Resolve root conversation files vs task files.
4. Copy files into workspace/task folders.
5. Deduplicate by source file ID/checksum.
6. Record mirror errors without exposing secrets.

Acceptance:

1. Conversation file shares appear under `Files/YYYY-MM/`.
2. Task file shares appear under task `Files/`.
3. Duplicate shares do not create uncontrolled copies.
4. Failed copies remain retryable.

### Phase 6 - Deck board/card mirror

Tasks:

1. Implement `DeckApiClient`.
2. Implement `DeckMirrorService`.
3. Create workspace board.
4. Create default stacks.
5. Create/update one card per task.
6. Sync status to stack.
7. Add card links back to Talk/folder.

Acceptance:

1. One board exists per workspace.
2. One card exists per task.
3. Moving Deck card updates local task status.
4. Talk remains canonical for task identity.

### Phase 7 - mobile status controls

Tasks:

1. Implement `ReactionWorkflowService`.
2. Implement admin-configurable mappings.
3. Implement conflict resolution.
4. Implement bot command parser.
5. Add concise bot replies.

Acceptance:

1. Mobile Talk reactions update task status.
2. Bot commands update status/assignee/due date.
3. Invalid commands fail safely.
4. Status changes are mirrored to Deck.

### Phase 8 - summaries

Tasks:

1. Implement `SummaryService`.
2. Generate `_summary.md` per task.
3. Generate daily summaries under `Summaries/`.
4. Generate `_open-tasks.md` and `_decisions.md`.
5. Keep AI extraction as optional suggestions only.

Acceptance:

1. Summaries are regenerated idempotently.
2. No raw secrets or private auth data are included.
3. Summaries link back to source task folders.

### Phase 9 - hardening and release

Tasks:

1. Add repair command/job for missing folders/cards.
2. Add admin sync status dashboard.
3. Add logging with redaction.
4. Add docs for admin setup and user workflow.
5. Add compatibility notes for Nextcloud 32-34 and Talk thread differences.
6. Add rollback notes.

Acceptance:

1. Failed syncs are visible and retryable.
2. App degrades gracefully without Deck.
3. App degrades gracefully without native Talk thread metadata.
4. Operators have devcloud verification commands.

## 13. Verification Commands

Do not run or simulate Docker/Nextcloud locally. Use devcloud.

```bash
docker compose -f docker-compose.devprod.yml --env-file .env.devprod ps
docker compose -f docker-compose.devprod.yml --env-file .env.devprod exec app php occ app:list
docker compose -f docker-compose.devprod.yml --env-file .env.devprod exec app php occ app:enable nuqloud_scrum
docker compose -f docker-compose.devprod.yml --env-file .env.devprod exec app php occ migrations:status nuqloud_scrum
docker compose -f docker-compose.devprod.yml --env-file .env.devprod exec app php occ background-job:list | grep -i nuqloud
docker compose -f docker-compose.devprod.yml --env-file .env.devprod logs --tail=200 app
```

If using the no-SSL devprod stack:

```bash
docker compose -f docker-compose.devprod.nossl.yml --env-file .env.devprod ps
docker compose -f docker-compose.devprod.nossl.yml --env-file .env.devprod exec app php occ app:enable nuqloud_scrum
docker compose -f docker-compose.devprod.nossl.yml --env-file .env.devprod exec app php occ migrations:status nuqloud_scrum
```

## 14. Risks

1. Talk native thread payloads may differ by Talk version.
   - Mitigation: implement reply-chain fallback and API validation spike.
2. Deck API authentication from background jobs needs a clear service-account pattern.
   - Mitigation: configure a dedicated automation user/app password.
3. Mirroring files can duplicate storage.
   - Mitigation: admin-selectable `copy|link`, dedupe, size limits.
4. Reaction status conflicts can be ambiguous.
   - Mitigation: deterministic precedence and admin-configurable mappings.
5. App-to-app internals may change.
   - Mitigation: isolate Talk/Deck clients behind service interfaces.

## 15. External References

1. Nextcloud Talk chat API: https://nextcloud-talk.readthedocs.io/en/latest/chat/
2. Nextcloud Deck REST API: https://deck.readthedocs.io/en/latest/API/
3. Nextcloud migrations: https://docs.nextcloud.com/server/stable/developer_manual/basics/storage/migrations.html
4. Nextcloud developer manual: https://docs.nextcloud.com/server/stable/developer_manual/