Qortal/qortal

Fork 1

mirror of https://github.com/Qortal/qortal.git synced 2026-04-26 19:49:22 +00:00

Files

T

Ice b07f6d09f6 Fix Typos

2026-02-09 07:47:32 -05:00

9.3 KiB

Raw Permalink Blame History

Qortal v6.0.0

Upcoming Qortal Changes | Network Upgrades MAJOR Qortal Networking and QDN Update Primary Developers: Phil, Ice

This is the next MAJOR version update for Qortal with substantial changes to core services.

Devnet Information: qortal://DevNet-Testing
Q-Deck for Qortal DevNet: Q-Deck Board

Repository Information

Repository: iceburst/qortal
Branch: iceburst:5.2-DataFlows → iceburst:master
Status: February 2026

Scope
Architecture Refactoring
Core Component Analysis
Network Protocol Changes
Serialization Framework
Relay & NAT Traversal
Configuration & Settings
Bug Fixes & Stability
API Changes
Performance Optimizations

Scope

This is a major refactoring focused on separating and improving the Qortal Data Network (QDN) from the blockchain network layer. The changes touch multiple architectural layers including network protocols, peer management, data serialization, and arbitrary data distribution.

Architecture Refactoring

Separation of Concerns

Before, Qortal used a monolithic network layer where blockchain and data traffic were mixed. Version 6 introduces a Dual Network Architecture.

Before: Monolithic Network Layer

Network.java (Mixed Blockchain + Data Traffic)
Single peer list
Mixed message handling
No capability negotiation
Single connection port

After: Dual Network Architecture

Nodes now run two distinct network managers that share a unified Peer.java context.

Feature	Network.java (Blockchain P2P)	NetworkData.java (QDN Layer)
Primary Tasks	Block propagation, TX broadcasting, Consensus	Arbitrary data, File distribution, Relays
Port	12392 (Default)	12394 (Default)
PeerType	`NETWORK`	`NETWORKDATA`

Key Architectural Changes:

NetworkData Class: New singleton managing QDN-specific networking.
Independent ExecutorService: Utilizing a ThreadPoolExecutor for data tasks.
Peer Context: Peers carry context (NETWORK vs NETWORKDATA). A single Peer can exist in both networks with separate message queues and behaviors.

Core Component Analysis

1. NetworkData Class

Location: src/main/java/org/qortal/network/NetworkData.java

Thread Pool Configuration: Minimum 10 threads, scaling to 20 under load. Uses an unbounded queue to prevent task rejection during bursts.
Peer Selection: Implements isolation handling. If no peers are connected, it waits for an ISOLATION_RETRY_INTERVAL (60s) before retrying backoff peers.
Capability Discovery: Only connects to peers advertising QDN support. It parses the port from the capability map.
Node Identity: Generates unique node identity using Ed25519 keys.

2. PeerCapabilities System

Location: src/main/java/org/qortal/network/helper/PeerCapabilities.java Type-safe capability storage using Map<String, Object>. This allows nodes to communicate supported features (like the QDN listen port) during the handshake without breaking the protocol.

3. PeerList Class

Location: src/main/java/org/qortal/network/PeerList.java Design: Immutable Snapshot Pattern.

Host-Based Matching: Matches by host only, ignoring the port. This prevents duplicate connections when peers connect via ephemeral ports but represent the same physical host.

Network Protocol Changes

New Handshake Protocol v2

A unified state machine for both network types.

State Flow: STARTED → HELLO_V2 → CHALLENGE → RESPONSE → COMPLETED.
Deterministic Tie-Breaking: Compares Node IDs during duplicate detection to prevent reconnect loops. If IDs conflict, the "lower" ID node disconnects its outbound attempt.
HELLO_V2 Format: Includes NTP-validated timestamp, version string, Node ID, capabilities map, and peerType.

STARTED
│
├──► HELLO ──────┐
│                │
└──► HELLO_V2 ───┤
                 │
                 ▼
              CHALLENGE
                 │
                 ├──► Self-connection check
                 ├──► Duplicate detection
                 ├──► Ghost peer cleanup
                 │
                 ▼
                 RESPONSE (RX: validate)
                 │
                 ▼
                 RESPONDING (TX: compute PoW)
                 │
                 ▼
                 COMPLETED
                 │
                 ├──► Network.onHandshakeCompleted() [NETWORK]
                 └──► NetworkData.onHandshakeCompleted() [NETWORKDATA]

Serialization Framework

Location: src/main/java/org/qortal/utils/Serialization.java

Introduces type-aware reconstruction for Map<String, Object>. This is critical for exchanging complex metadata between nodes.

Binary Format: [map_size:4][key_len:4][key:n][type_len:4][type:m][value_len:4][value:p]
Stream Agnostic: Supports both ByteArrayOutputStream and DataOutput variants for gradual migration from legacy code.

Relay & NAT Traversal

The Relay mechanism allows nodes behind NAT/Firewalls to bridge connections through well-connected public nodes.

Eligibility Detection: NetworkData determines if a peer is directly connectable based on advertised QDN capabilities.
Relay Decision: If not directly connectable, ArbitraryDataFileRequestThread initiates a relayDataRequest via known network peers.
UPnP Integration: Automatically attempts to open TCP Port 12394 on startup and closes it gracefully on shutdown.

Configuration & Settings

New Settings:

listenDataPort: 12394 (Default).
wallets: An array of individual coin flags (ARRR, BTC, DGB, DOGE, LTC, RVN) can be set to false to reduce memory and bandwidth for data-only nodes.
ourExternalIPAddress: String value to explicitly define an outside world address in which the node can be reached.

Updated Settings:

maxDataPeers: Maximum number of Data Network peers to maintain.
arbitraryIndexingPriority: Set to 0 to disable indexing for archive or non-QDN nodes.

Bug Fixes & Stability

Stuck Chain (25 Blocks): Fixed a validation error where single-node testnets halted when OnlineAccounts was empty.
External IP Handling: Corrected null handling; the system now takes the first valid response as truth before requiring consensus for changes.
Thread Counter Fixes: Replaced manual thread incrementing with ThreadPoolExecutor for reliable resource management.
FQDN vs IP Matching: Resolved lookup failures where nodes connected by FQDN failed to match subsequent IP-based requests, causing rejected duplicate connections. NodeID is now the primary mapping index.
Dependency Updates: Many dependencies defined in the pom have been updated to include Swagger, Bouncy Castle, AltCoinJ, Maven

API Changes

GET /peers/data: Returns information about connected QDN (NetworkData) peers.
POST /peers/data: Add peer to list of available QDN peers and attempt direct connection.
DELETE /peers/data: Disconnect an existing QDN data peer.
GET /peers: Enhanced to include the full capabilities map for each peer.
GET /crosschain/<coin>/status: Returns the current wallet state, number of connected Electrum servers, and the number of known Electrum servers.
POST /crosschain/<coin>/start: Allows for "late binding" and startup of a wallet that was disabled in the initial configuration.
GET /arbitrary/resource/request/peers/<service>/<name>: Retrieve information about a QDN resource that is currently being transferred.
POST /data/cache/erase: Deletes from disk the QDN temporary cache
GET /data/cache/size: Returns the size of the current QDN temporary cache.
GET /admin/status: Now includes number of QDN connects as a separate entry.

Performance Optimizations

Reduced Tidy Frequency: Network tidying now occurs every 100 blocks instead of every block, reducing CPU usage by ~99% for those operations.
Immutable Peer Snapshots: Eliminates lock contention during peer list iteration, enabling better parallel processing.
Batch Block Syncing: Peers more than 200 blocks behind use the new GetBlocksMessage to request 200 blocks at once, increasing sync performance by up to 300%.
Lazy Capability Parsing: Capabilities are only parsed when explicitly requested, reducing overhead during the handshake.
QDN Increased Transfer Speeds: Now tracks expected speeds for remote nodes based on past transfer data. This allows the core to prioritize remote nodes that meet required throughput expectations.
QDN QApp Build Speed: QApps build time was reduced by decrypting on the fly and then assembly.

Document Version: 1.1
Last Updated: 2026-02-09
Contributors: Phil, Ice, and the Qortal Development Team

9.3 KiB Raw Permalink Blame History