Added API call GET /admin/status which reports whether minting
is possible (have minting keys & up-to-date) and whether node is
currently attempting to sync.
Corresponding change to system tray mouseover text.
Corresponding text added to SysTray transaction resources.
Previous images had a small hole in the icon (probably a result of a background removal), I just filled it back in with white like it's supposed to be. Also, the previous square icons were streched into a square aspect ratio, these are unstreched.
No more "node UI". UI provided by 3rd party.
"Open UI" tray icon menu item now attempts to open UI at various
local servers (see Settings.uilocalServers) or some random
remote server (Settings.uiRemoteServers).
Default UI port now 12388 (Settings.uiPort).
Translator class no longer logs warnings for every failed translation.
Commented out unused ApiError enum entries.
Renamed some ApiError names like "_NO_EXISTS" to "_UNKNOWN".
Removed old src/main/resources/globalization/* files.
Added CheckTranslations test app.
Fixed some extraneous/missing ApiError aspects in some API-related classes.
e.g. added NAME_UNKNOWN to GET /names/{name}
Blockchain configs will need "v2Timestamp" feature trigger renaming to "qortalTimestamp"!
Due to Controller.VERSION_PREFIX changing, peer-to-peer protocol version detection has
been changed. Was previous a substring match, now we test peers's buildTimestamp is at
least Peer.V2_PROTOCOL_TIMESTAMP_THRESHOLD. Changes in Peer.java.
Also added comment and throw() to QortalATAPI.getNextTransactionTimestamp()
as this needs given the change to Qortal's block timestamps from legacy Qora.
Changes to HSQLDB data types, e.g. QoraAddress to QortalAddress, means existing
database will need to be thrown away after this commit!
Seems to produce fat JARs with the same MD5 after repeated runs of:
mvn clean package
Needs testing on other platforms, hence this commit.
Also trimmed some unneeded included JAR libraries from pom.xml,
e.g. netty, async-http-client,
and the *.js.map files from Swagger-UI, which seems to reduce
output JAR size from about 51MB to about 44MB.
These changes are for allowing devices to be shipped with an interim blockchain config & genesis block, running with "main-net" settings, prior to launch.
At launch, an "auto-update" will be broadcast to replace blockchain config & genesis block with the final, launch version.
When updated nodes restart (immediately after auto-update) they should notice their existing genesis blocks are invalid and hence wipe their blockchains,
effectively starting the final, launch blockchain.
To this end, genesis block signatures have been changed to better incorporate values that are likely to change at launch, e.g. block timestamp and transaction data.
Added Crypto.dupDigest(), which is effectively Bytes.concat(digest(msg), digest(msg)).
No need for HSQLDB repository to backup, or wipe, non-existent files for "in-memory" databases!
New interim blockchain config/genesis block.
**NOTE** These changes are being committed but require testing. However the auto-update suite requires a pushed commit in order to build an update file!
Also:
RewardShareKeys app now supports only one arg (minter private key)
in self-reward-share mode, where recipient public key is derived
from minter private key.
Added methods to Account for returning 'effective' minting level
where minting level for founders is read from blockchain config.
(Or returns zero if unable to mint).
Changed two Block constructors into static methods that return
a new Block as there was way too much work being done to really
be called a constructor, especially with all the opportunities
to throw an exception too.
Main blockchain config updated to reflect near-launch version.
Added/changed blockchain weight tests to check block winning
based on higher account levels.
Unified terminology from block "generator", "forger", etc. to "minter"
Unified terminology "proxy forger" to "reward share" as it was
incorrect, or at least highly ambiguous, which account had which role.
AccountRepository.findRewardShares() has different arg order!
Account.canMint() now returns true if account has 'founder' flag set.
Added Account.canRewardShare() which returns whether acocunt can create
a reward-share (e.g. level 5+ or founder).
Fixed HSQLDBAssetRepository.getAllAssets() which had incorrect
resultSet column indexes.
Removed old traces of EnableForging transaction.
ACCOUNT_LEVEL and ACCOUNT_FLAGS (genesis-block-only transaction types)
now set target account's last-reference. This is allow later REWARD_SHARE
transactions in genesis block and post-genesis transactions by that account.
REWARD_SHARE transactions are now FREE, but only if minter is also recipient.
If a self-reward-share already exists, then unless share-percent is zero (to
terminate reward-share), any subsequent self-reward-share is invalid.
Updated SysTray i18n properties file.
BlockChain config file requires 'minAccountLevelToRewardShare' and optional
'minAccountLevelToMint'.
Added potential, but currently unused, memory-hard PoW algorithm.
Fixed/removed/disabled some unit tests.
BlockMinter.generateTestingBlock asks Controller to pretend mintingAccount is 'online'.
More testing needed!
API call GET /addresses/online reports online accounts,
including both addresses relating to the proxy-forge public key.
New PeerChainTipData class to replace the broken "peer data lock"
that was supposed to make sure peer's last height/blockSig/timestamp
were all in sync. Now peer's chain tip data is a single object
reference that can be replaced in one go.
Removed pointless API calls /blocks/time and /blocks/{generatingbalance}.
Various changes, mostly in Block class, to do with switching to BlockTimingByHeight
from old min/max block time.
New block 'weight' based on number of online accounts
and 'distance' of perturbed generator public key from 'ideal' public key
(for that particular block's height).
New sub-chain 'weight' based on accumulating block weights,
currently by shifting previous accumulator left by 8 bits then
adding next block's weight.
More validation of BlockChain config. Helpful for debugging, probably
not very useful to end-users.
BlockGenerator now uses unified Peer predicates from Controller, like:
Controller.hasMisbehaved, Controller.hasNoRecentBlock, etc.
Controller now keeps a list of chain-tip signatures that are for inferior
chains, so it doesn't try to synchronize with peers with inferior chains.
(This list is wiped when node's blockchain changes/block is generated).
Controller now asks Gui to display error box if it can't parse Settings.
Controller.potentiallySynchronize() does more filtering of potential peers
before calling actuallySynchronize(). (Mostly moved from Synchronizer,
so now we expect actuallySynchronize() to do something rather than bail
out because it doesn't like the peer after all).
If synchronization discovers that peer has an inferior chain,
then Controller notifies that peer of our superior chain, to help keep
the network in sync.
Renamed OnlineAccount to OnlineAccountData, as it is in package org.qora.data
after all...
Synchronizer reworked to request block summaries so it can judge which chain
is better, and hence whether to sync with peer or abort.
Slight optimization of Peer.readChannel() to exit earlier if no more network
messages can be extracted from buffer.
More tests.
Improved documentation and logging.
Block data now includes number of online accounts, as encoded online account indexes can't be
validated by ConciseSet it seems.
Corresponding changes to repository, transformer, block validation, data object, block summaries...
Block timestamps are now calculated using parent block data and generator's public key,
instead of old qora1 generating balance code.
Generators are valid to forge if they have forging flag enabled. This will probably change
to an account-level check in the near future.
Added trimming of old online accounts signatures from blocks.
Tidied up SysTray/BlockGenerator generation enabled/possible flag.
Although we perform online accounts tasks (currently) every 10 seconds,
only broadcast our online accounts every 60 seconds.
In Controller.main(), if args are present then use first as a filename to
settings JSON file (overriding the default filename).
Still to do: change Block/BlockChain/Synchronizer to prefer blocks with more online accounts,
failing that use generator nearest 'ideal', etc.
Safety commit in case of data loss!
Lots of changes to do with "online accounts", including:
* networking
+ GET_ONLINE_ACCOUNTS * ONLINE_ACCOUNTS messages & handling
+ Changes to serialization of block data to include online accounts info
* block-related
+ Adding online accounts info when generating blocks
+ Validating online accounts info in block data
+ Repository changes to store online accounts info
* Controller
+ Managing in-memory cache of online accounts
+ Updating/broadcasting our online accounts
* BlockChain config
Added "account levels", so new code/changes required in the usual places, like:
* transaction data object
* repository
* transaction transformer
* transaction processing
New NTP class now runs as a simplistic NTP client, repeatedly polling
several NTP servers and maintaining a more accurate time independent
of operating system.
Several occurrences of System.currentTimeMillis() replaced with NTP.getTime()
particularly where block/transaction/networking is involved.
GET /admin/info now includes "currentTimestamp" as reported from NTP.
Added support for block timestamps determined by generator, instead of
supplied by clock. (BlockChain.newBlockTimestampHeight - not yet activated).
Incorrect timestamps will produce a TIMESTAMP_INCORRECT Block.ValidationResult.
Block.calcMinimumTimestamp repurposed as Block.calcTimestamp for above.
Block timestamps are now allowed to be max 2000ms in the future,
was previously max 500ms.
Block generation prohibited until initial NTP sync.
Instead of deleting INVALID unconfirmed transactions in BlockGenerator,
Controller now deletes EXPIRED unconfirmed transactions every so often.
This also fixes persistent expired unconfirmed transactions on nodes
that do not generate blocks, as BlockGenerator.deleteInvalidTransactions()
was never reached.
Abbreviated block sigs added to log entries declaring a new block is generated
in BlockGenerator.
Controller checks for NTP sync much faster during start-up and SysTray's
tooltip text starts as "Synchronizing clock" until NTP sync occurs.
After NTP sync, Controller logs NTP offset every so often (currently every 5 mins).
When considering synchronizing, Controller skips peers that have the same block sig
as last time when synchronization resulted in no action, e.g. INFERIOR_CHAIN,
NOTHING_TO_DO and also OK. OK is included as another sync attempt would result in
NOTHING_TO_DO.
Previously this skipping check only happened after prior INFERIOR_CHAIN.
During inbound peer handshaking, if we receive a peer ID that matches an existing inbound
peer then send peer ID of all zeros, then close connection.
Remote end should detect this and cleanly close connection instead of waiting for handshake timeout.
Randomly generated peer IDs have lowest bit set to avoid all zeros.
Might need further work.
Networking doesn't connect, or accept, until NTP has synced.
Transaction validation can fail with CLOCK_NOT_SYNCED if NTP not synced.
Controller performs NTP check on startup (and every 5 minutes)
which determines whether block generation is allowed.
System Tray tooltip updated to reflect generating status.
Plus new translations.
Improved GuiTests.
BlockGenerator fetches forging accounts first, and sleeps
if none configured, which is less work than processing peer lists.
Now uses several NTP servers to determine mean offset from
system clock to internet time.
If abs(offset) > 500ms or NTP service not running then
user is 'nagged' via system tray pop-up notification
with instructions on how to fix.
Also improved system tray translations!
SysTray pop-up menu now includes entry for launching https://time.is
so node owners can check their system clocks against internet time.
Windows installs also have additional systray menu entry which
runs ntpcfg.bat script, included in resources.
Also available as download via node-UI servlet,
e.g. http://localhost:9880/downloads/ntpcfg.bat
ntpcfg.bat reconfigures Windows Time Service with many NTP servers,
restarts the service, and also makes sure it auto-starts on boot.
Added DEBUG-level logging when rejecting nodes due to excessive
time difference (during PROOF handshake stage).
Bumped default settings values for minOutboundPeers from 10 to 20.
Bumped default settings values for maxPeers from 30 to 50.
Refactored duplicate code into Transaction.setInitialApprovalStatus().
This is make sure transactions HAVE a group-approval status
in Synchronizer before Block.isValid is called.
This wasn't a problem for new, unconfirmed, individual transactions
arriving over the wire due to Transaction.importAsUnconfirmed()
doing the right thing.
Also added a groupApprovalTimestamp to BlockChain feature-triggers
to support legacy chains.
Tested by syncing mainnet from scratch.
HTML/JS in src/test/resources/proxy-key-example.html updated accordingly.
Add handshake status to output of API call GET /peers
Add/correct @ApiErrors annotations on some API calls.
Add API call POST /admin/orphan (target height as body)
to force blockchain orphaning for when node is wildly out of sync.
Added support for above to BlockChain class.
BlockGenerator now requires a minimum number of peers
before it will generate any new blocks.
See "minBlockchainPeers" in settings.
Controller now requires a minimum number of peers
before it will consider synchronizing.
See "minBlockchainPeers" in settings.
Old "minPeers" entry in settings.json no longer valid!
Networking now allows both an outbound and inbound connection
to a peer although will use the outbound connection in preference.
Networking checks peer ID of inbound connections to detect,
and resolve, peer ID clashes/theft.
Some dev/testing API calls are now turned off by default in production mode,
see "restrictApi" settings entry, returning NON_PRODUCTION API error.
Corrections to how account's defaultGroupId works, removing "effective groupID"
which overly complicated matters.
In relation to above, DEFAULT_GROUP (0) no longer exists and NO_GROUP(-1) now has
the value 0 instead.
So transactions can no longer have txGroupId of DEFAULT_GROUP, which in turn
required all the erroneous "effective groupID" code.
API call /addresses/{address} now supplies blockchain-wide defaultGroupId if
account doesn't exist or if account's default not set and NO-GROUP not allowed.
API /transactions/pending now offloaded to repository instead of Java-based
processing and filtering.
Transaction approval checks added to Block.isValid
Groups now have min/max approval block delays.
Checks added to incoming unconfirmed, block generator, block.isValid, etc.
'needing approval' and 'meets approval threshold' now split into separate calls.
NB: settings.json no longer part of git repo
DB shape change from v29 to add peer info.
New NetworkRepository to handle above.
Peer handshaking with v2 anti-DoS PoW code.
Handshaking refactored into a state-machine-like enum.
Some peer-related API calls added.
Peers exchange pings, heights, peers.
No actual peer sync yet.
Other changes:
Peer version info taken from Maven build properties/resource file.
AnnotationPostProcessor more resilient when fetching PathItems.
Per-repository session debugging flag that can be toggled at will.
HSQLDBRepository.delete() now returns int so callers can detect
whether anything was actually deleted.
Some renaming to settings.
Invalid/expired unconfirmed transactions are cleaned during various calls,
e.g. requesting list of unconfirmed transactions,
or requesting account's last reference (including considering unconfirmed),
or generating a new block.
BlockGenerator now calls repository.discardChanges before sleep to
release any repository-level locks.
Added settings.json toggle "wipeUnconfirmedOnStart" (default: true)
to aid testing.
REMOVED API call /addresses/lastreference/{address}/unconfirmed as
/addresses/lastreference/{address} now considers unconfirmed
transactions regardless.
Added useful error to /transactions/sign if an invalid private key
is supplied.
Improved API "invalid transaction" error to include actual apsect
that caused validity check to fail (e.g. invalid reference)
Refactored to standard Maven layout:
src/main/java
src/main/resources
src/test/java
etc.
New translation code that uses locale-specific ResourceBundles
to load translations on demand.
Reworked API error/exceptions code to a shorter, simpler
@ApiErrors annotation. Processing of @ApiErrors annotations
produces an example for each possible API error and includes
API error string in HTTP response code, e.g.
400 INVALID_SIGNATURE
Missing API error cases added to each API call.
Translation of openAPI.json removed (for now).
block-explorer.html and BIP39 wordlists now read as resources
instead of direct from disk.
Java compile warnings fixed.
Some runtime warnings remain:
WARNING: A provider api.resource.ApiDefinition registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime.
WARNING: A provider api.resource.AnnotationPostProcessor registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime.
WARN org.reflections.Reflections - given scan urls are empty. set urls in the configuration
