5s is way too long, and even 2s might still be considered excessive.
However, reducing the timeout might also reduce the number of
network engine "spawn failures" due to too many threads tied up
waiting for ping responses from overloaded peers.
Does not affect peer handshaking: that has a separate timeout.
Added Ed25519 private key to public key function accessible from SQL.
Added Ed25519 public key to Qortal address function accessible from SQL.
Used above functions to store minting account public key in SQL to
reduce the number of unnecessarily repeated Ed25519 conversions.
Used above functions to store reward-share minting's accounts address
to reduce the number of unneccessarily repeated PK-to-address conversions.
Reduced the usage of PublicKeyAccount to simply Account where possible,
to reduce the number of Ed25519 conversions.
Account.canMint(), Account.canRewardShare() and Account.getEffectiveMintingLevel()
now only perform 1 repository fetch instead of potentially 2 or more.
Cleaned up NTP main thread to reduce CPU load.
A fixed offset can be applied to NTP.getTime() responses, for both
scenarios when NTP is running or not. Useful for testing or simulating
distant remote peers.
Controller.onNetworkMessage() and Network.onMessage() have both had their
complexity simplified by extracting per-case code to separate methods.
Network's EPC engine's thread pool size no longer hard-coded, but comes
from Settings.maxNetworkThreadPoolSize, which is still 10 by default,
but can be increased for high-availability nodes.
Network's EPC task-producing code streamlined to reduce CPU load.
Generally reduced calls to System.currentTimeMillis(), especially
where the value would only be used in verbose logging situations,
and especially in high-call-volume methods, like within repository.
Keep track of when EPC engine can't spawn a new thread as this
might indicate thread-pool exhaustion and cause some network
messages to be lost.
If logging level is NOT 'trace' (or 'all') then don't call
System.currentTimeMillis() as we'll never use the value.
Similarly, don't set thread names if not logging at 'trace' either.
Update EPC tests, particularly unified per-second/end-of-test stats
reporting.
Added API call GET /peers/enginestats to allow external monitoring.
Extract all engine stats in one synchronized block, instead of
separate calls, for better consistency.
Synchronizer now bails out early when trying to find common block with
a peer. There's no need to keep searching if common block is too far
behind that a TOO_DIVERGENT result would be returned.
fetchSummariesFromCommonBlock() reworked to return a useful
SynchronizationResult directly instead of caller trying to infer
what happened based on null/empty returned list!
Added API call GET /admin/status which reports whether minting
is possible (have minting keys & up-to-date) and whether node is
currently attempting to sync.
Corresponding change to system tray mouseover text.
Corresponding text added to SysTray transaction resources.
Previously BlockMinter would attempt to mint if there were at least
'minBlockchainPeers' connected peers and none of them had an
up-to-date block and we did. This was maybe useful for minting block 2
but possibly causes minting chain islands where a badly connected
node mints by itself, even though connected to not up-to-date peers.
Now BlockMinter requires 'minBlockchainPeers' up-to-date peers, not
simply just connected. This should let synchronization bring the
node up-to-date but does require the node to have better peers.
Currently, the default for minBlockchainPeers is 10. So a node
requires 10 up-to-date peers before it will consider minting. It
might be possible to reduce this in the future to lessen network load.
Although HSQLDB is happy being given unix-style path separator '/'
and converting as necessary on other platforms (e.g. Windows),
manipulation of repository pathnames in Java, outside of HSQLDB,
needs to use platform-specific path separators.
Thus, changes made to replace '/' with File.separator where
necessary.
This should fix repository rebuild errors, which then lead to odd
start-up errors like:
2020-03-11 13:55:19 INFO Controller:270 - Starting repository
2020-03-11 13:55:20 INFO Controller:287 - Validating blockchain
2020-03-11 13:55:20 INFO HSQLDBRepository:227 - Rebuilding repository from scratch
2020-03-11 13:55:20 INFO GenesisBlock:296 - Using genesis block timestamp of 1583870000000
2020-03-11 13:55:21 WARN HSQLDBRepository:720 - Uncommitted changes (882) after connection close, session [3]
java.lang.NullPointerException
at org.qortal.transform.block.BlockTransformer.decodeOnlineAccounts(BlockTransformer.java:422)
at org.qortal.block.Block.getExpandedAccounts(Block.java:546)
at org.qortal.block.Block.increaseAccountLevels(Block.java:1245)
at org.qortal.block.Block.increaseAccountLevels(Block.java:1239)
at org.qortal.block.Block.process(Block.java:1206)
at org.qortal.block.GenesisBlock.process(GenesisBlock.java:345)
at org.qortal.block.BlockChain.rebuildBlockchain(BlockChain.java:526)
at org.qortal.block.BlockChain.validate(BlockChain.java:481)
at org.qortal.controller.Controller.main(Controller.java:289)
The above happens because the old blockchain still exists when trying to process
the genesis block.
AdvancedInstaller's Java launcher EXE seems to use JNI to launch
the JAR, instead of using the command-line 'java' binary directly.
When AI's launcher does this, it adds options like "abort" and "exit",
along with corresponding hook addresses.
These options are returned by the call to
ManagementFactory.getRuntimeMXBean().getInputArguments() which is
done in AutoUpdate while building the command line for launching
ApplyUpdate.
Because command-line 'java' binary doesn't support these options,
they are now stripped out.
No more "node UI". UI provided by 3rd party.
"Open UI" tray icon menu item now attempts to open UI at various
local servers (see Settings.uilocalServers) or some random
remote server (Settings.uiRemoteServers).
Default UI port now 12388 (Settings.uiPort).
Translator class no longer logs warnings for every failed translation.
Commented out unused ApiError enum entries.
Renamed some ApiError names like "_NO_EXISTS" to "_UNKNOWN".
Removed old src/main/resources/globalization/* files.
Added CheckTranslations test app.
Fixed some extraneous/missing ApiError aspects in some API-related classes.
e.g. added NAME_UNKNOWN to GET /names/{name}
We've never needed this before but now it's fixed.
Added corresponding +ve & -ve tests just to make sure.
Only actual use-case that comes to mind is cancelling reward-share.
Instead of working on a single 'auto-update' branch,
that needed to be rebased from master branch's HEAD,
we create an orphan branch named after master:HEAD
containing on the update file.
This allows us to keep/delete updates on a per-commit
basis and shows which commit each update is based upon.
Added loads more checking.
Before:
0-fee self-share REWARD_SHARE when there is an existing self-share would result in
INSUFFICIENT_FEE from isFeeValid()
After:
isFeeValid() returns OK for above, but isValid() returns SELF_SHARE_EXISTS.
In addition, a transaction that tries to modify existing self-share, even with fee,
also returns SELF_SHARE_EXISTS.
Improved tests to double check.
NOTE: first startup after this commit can take a while due to building index!
SQL statement "DELETE FROM HistoricAccountBalances WHERE height >= ?" required
a full table scan and so was very slow on VMs/routers. This statement used by
HSQLDBAccountRepository.deleteBalancesFromHeight(), itself called during
Block.orphan().
Symptoms particularly evident during shutdown where above statement could take
upwards of 15 minutes on single-CPU, small-memory VMs!
Statement wasn't noticed before as slow-query checking wasn't involved.
Slow-query checking now applies to HSQLDBRepository.delete() and
HSQLDBRepository.exists() calls.
Added test for correct HSQLDB interrupt handling.
Fixed some typos.
