Commit Graph

113 Commits

Author SHA1 Message Date
catbref
00aee1458e Higher account levels more likely to win blocks
Also:

RewardShareKeys app now supports only one arg (minter private key)
in self-reward-share mode, where recipient public key is derived
from minter private key.

Added methods to Account for returning 'effective' minting level
where minting level for founders is read from blockchain config.
(Or returns zero if unable to mint).

Changed two Block constructors into static methods that return
a new Block as there was way too much work being done to really
be called a constructor, especially with all the opportunities
to throw an exception too.

Main blockchain config updated to reflect near-launch version.

Added/changed blockchain weight tests to check block winning
based on higher account levels.
2019-11-06 09:47:10 +00:00
catbref
ebc2ee6ea9 Rework distributing block reward to legacy qora holders
Fix related unit tests.

Fix assetID support in GENESIS transactions.
2019-11-01 10:01:31 +00:00
catbref
62e2fd759c Fixing unit tests 2019-10-30 13:19:56 +00:00
catbref
fef16e7620 Self-reward-share tests and fixes.
Added Transaction.isFeeValid() to allow transaction subclasses to override and allow zero fees, etc.

Added tests to cover self-reward-shares, including zero fee scenario.

Set 'dilbert' test account to level 8 in test genesis block.

Removed leftover mention of "previous_level" from HSQLDBAccountLevelTransactions,
and AccountLevelTransactionData. (Previous level makes no sense as ACCOUNT_LEVEL
transactions are genesis-block only).

Fixed some incorrect uses of PrivateKeyAccount.getSharedSecret() to
PrivateKeyAccount.getRewardSharePrivateKey() in tests.
2019-10-30 10:23:59 +00:00
catbref
491e79b8e6 ProxyForging->RewardShare massive refactor & more...
Unified terminology from block "generator", "forger", etc. to "minter"

Unified terminology "proxy forger" to "reward share" as it was
incorrect, or at least highly ambiguous, which account had which role.

AccountRepository.findRewardShares() has different arg order!

Account.canMint() now returns true if account has 'founder' flag set.
Added Account.canRewardShare() which returns whether acocunt can create
a reward-share (e.g. level 5+ or founder).

Fixed HSQLDBAssetRepository.getAllAssets() which had incorrect
resultSet column indexes.

Removed old traces of EnableForging transaction.

ACCOUNT_LEVEL and ACCOUNT_FLAGS (genesis-block-only transaction types)
now set target account's last-reference. This is allow later REWARD_SHARE
transactions in genesis block and post-genesis transactions by that account.

REWARD_SHARE transactions are now FREE, but only if minter is also recipient.
If a self-reward-share already exists, then unless share-percent is zero (to
terminate reward-share), any subsequent self-reward-share is invalid.

Updated SysTray i18n properties file.

BlockChain config file requires 'minAccountLevelToRewardShare' and optional
'minAccountLevelToMint'.

Added potential, but currently unused, memory-hard PoW algorithm.

Fixed/removed/disabled some unit tests.
BlockMinter.generateTestingBlock asks Controller to pretend mintingAccount is 'online'.
More testing needed!
2019-10-29 17:46:55 +00:00
catbref
5798c69449 Code clean-up thanks to SonarLint & bugfix for ByteArray::compareTo! 2019-10-15 17:10:13 +01:00
catbref
319bfc8d75 Add account level change due to blocks generated.
Removed ENABLE_FORGING and related.

Still to do: 'orphan' version of Block.increaseAccountLevels
2019-10-10 13:52:00 +01:00
catbref
54d49e0f1d More work on block rewards in relation to account level/legacy qora held.
Rename Asset.QORA to Asset.QORT so we can also have Asset.LEGACY_QORA
as another hard-coded asset.

Add "is unspendable" aspect to assets where only the asset owner can
transfer/pay asset to other people. Asset trading is barred regardless,
as is use of asset for ATs.

Added "initial level" to account data in preparation for accounts levelling
up from generating blocks.

Added distribution/removal of block reward based on legacy-QORA held.

Removed "previous level" from ACCOUNT_LEVEL transactions as they're
only ever valid in genesis block and so previous level is never needed.
2019-10-08 14:58:00 +01:00
catbref
568a1f8a30 Migration to Java 11
Updated pom.xml.
Updated dependencies, including various minor code mods, esp. bitcoin-related.

Starts up and talks to Java 8 nodes!

Dev environment should be at least Eclipse 4.11 with m2e 1.9.1+
2019-09-30 18:06:00 +01:00
catbref
305bb38446 Remove "generating balance", and all related aspects/code.
Block forging is now allowed by level 1+ accounts,
instead of accounts with "minting flag" set.
2019-09-27 15:16:12 +01:00
catbref
4c6656dd17 New synchronizer and other improvements
API call GET /addresses/online reports online accounts,
including both addresses relating to the proxy-forge public key.

New PeerChainTipData class to replace the broken "peer data lock"
that was supposed to make sure peer's last height/blockSig/timestamp
were all in sync. Now peer's chain tip data is a single object
reference that can be replaced in one go.

Removed pointless API calls /blocks/time and /blocks/{generatingbalance}.

Various changes, mostly in Block class, to do with switching to BlockTimingByHeight
from old min/max block time.

New block 'weight' based on number of online accounts
and 'distance' of perturbed generator public key from 'ideal' public key
(for that particular block's height).

New sub-chain 'weight' based on accumulating block weights,
currently by shifting previous accumulator left by 8 bits then
adding next block's weight.

More validation of BlockChain config. Helpful for debugging, probably
not very useful to end-users.

BlockGenerator now uses unified Peer predicates from Controller, like:
Controller.hasMisbehaved, Controller.hasNoRecentBlock, etc.

Controller now keeps a list of chain-tip signatures that are for inferior
chains, so it doesn't try to synchronize with peers with inferior chains.
(This list is wiped when node's blockchain changes/block is generated).

Controller now asks Gui to display error box if it can't parse Settings.

Controller.potentiallySynchronize() does more filtering of potential peers
before calling actuallySynchronize(). (Mostly moved from Synchronizer,
so now we expect actuallySynchronize() to do something rather than bail
out because it doesn't like the peer after all).

If synchronization discovers that peer has an inferior chain,
then Controller notifies that peer of our superior chain, to help keep
the network in sync.

Renamed OnlineAccount to OnlineAccountData, as it is in package org.qora.data
after all...

Synchronizer reworked to request block summaries so it can judge which chain
is better, and hence whether to sync with peer or abort.

Slight optimization of Peer.readChannel() to exit earlier if no more network
messages can be extracted from buffer.

More tests.
Improved documentation and logging.
2019-09-26 17:43:50 +01:00
catbref
c889e95da4 ByteArray tests 2019-09-26 17:25:39 +01:00
catbref
504cfc6a74 Interim commit: online accounts and account levels
Safety commit in case of data loss!

Lots of changes to do with "online accounts", including:

* networking
	+ GET_ONLINE_ACCOUNTS * ONLINE_ACCOUNTS messages & handling
	+ Changes to serialization of block data to include online accounts info
* block-related
	+ Adding online accounts info when generating blocks
	+ Validating online accounts info in block data
	+ Repository changes to store online accounts info
* Controller
	+ Managing in-memory cache of online accounts
	+ Updating/broadcasting our online accounts
* BlockChain config

Added "account levels", so new code/changes required in the usual places, like:

* transaction data object
* repository
* transaction transformer
* transaction processing
2019-09-13 14:32:32 +01:00
catbref
4330782bb7 Added Ed25519 signature verify test JavaScript
JS uses tonyg/js-nacl libsodium port.

First sig shows disagreement with WhisperSystems-based legacy crypto.
2019-08-16 12:33:21 +01:00
catbref
2889d04633 Fix incorrect orphaning of BUY_NAME transactions.
Orphaning a BUY_NAME transaction would not reinstate the
sale price. Sale price could be nullified by (e.g.) orphaning
a SELL_NAME transaction.

Also added test case to cover above and other test-related support,
e.g. test-mode in NTP.
2019-08-16 11:48:30 +01:00
catbref
f83dc26ae0 ExecuteProduceConsume and networking improvements
Added "volatile" to more fields, for thread-safety on reads.
Changes to field values are done inside synchronized blocks
so no need for AtomicInteger/AtomicBoolean. (Could be changed
in the future to show intention/readability though).

Added more statistics (tasks produced/consumed).

Limited Network's EPC executor to 10 threads max.
2019-08-16 08:28:26 +01:00
catbref
63b262a76e NTP and performance changes + fixes.
New NTP class now runs as a simplistic NTP client, repeatedly polling
several NTP servers and maintaining a more accurate time independent
of operating system.

Several occurrences of System.currentTimeMillis() replaced with NTP.getTime()
particularly where block/transaction/networking is involved.

GET /admin/info now includes "currentTimestamp" as reported from NTP.

Added support for block timestamps determined by generator, instead of
supplied by clock. (BlockChain.newBlockTimestampHeight - not yet activated).
Incorrect timestamps will produce a TIMESTAMP_INCORRECT Block.ValidationResult.

Block.calcMinimumTimestamp repurposed as Block.calcTimestamp for above.

Block timestamps are now allowed to be max 2000ms in the future,
was previously max 500ms.

Block generation prohibited until initial NTP sync.

Instead of deleting INVALID unconfirmed transactions in BlockGenerator,
Controller now deletes EXPIRED unconfirmed transactions every so often.
This also fixes persistent expired unconfirmed transactions on nodes
that do not generate blocks, as BlockGenerator.deleteInvalidTransactions()
was never reached.

Abbreviated block sigs added to log entries declaring a new block is generated
in BlockGenerator.

Controller checks for NTP sync much faster during start-up and SysTray's
tooltip text starts as "Synchronizing clock" until NTP sync occurs.
After NTP sync, Controller logs NTP offset every so often (currently every 5 mins).

When considering synchronizing, Controller skips peers that have the same block sig
as last time when synchronization resulted in no action, e.g. INFERIOR_CHAIN,
NOTHING_TO_DO and also OK. OK is included as another sync attempt would result in
NOTHING_TO_DO.
Previously this skipping check only happened after prior INFERIOR_CHAIN.

During inbound peer handshaking, if we receive a peer ID that matches an existing inbound
peer then send peer ID of all zeros, then close connection.
Remote end should detect this and cleanly close connection instead of waiting for handshake timeout.
Randomly generated peer IDs have lowest bit set to avoid all zeros.
Might need further work.

Networking doesn't connect, or accept, until NTP has synced.

Transaction validation can fail with CLOCK_NOT_SYNCED if NTP not synced.
2019-08-13 09:47:44 +01:00
catbref
671dc5995a Don't allow block generation unless system clock is accurate.
Controller performs NTP check on startup (and every 5 minutes)
which determines whether block generation is allowed.

System Tray tooltip updated to reflect generating status.
Plus new translations.

Improved GuiTests.

BlockGenerator fetches forging accounts first, and sleeps
if none configured, which is less work than processing peer lists.
2019-08-13 09:15:21 +01:00
catbref
73e53120a9 Improved detection of inaccurate system clock & nagging.
Now uses several NTP servers to determine mean offset from
system clock to internet time.

If abs(offset) > 500ms or NTP service not running then
user is 'nagged' via system tray pop-up notification
with instructions on how to fix.

Also improved system tray translations!
2019-08-13 09:14:07 +01:00
catbref
0c17f9cff6 More useful Synchronizer logging + sync report tool.
Synchronizer logging now includes abbreviated block signature.
2019-08-13 09:08:57 +01:00
catbref
964e0a02ca Fixes/improvements to networking
Reworked networking execute-produce-consume threading.
Some networking task were wrongly performed during 'produce' phase,
and some producing was happening in 'consume' phase (also corrected).

Peer connection tasks are rate-limited to 1 per second to reduce CPU thrashing.

Show P2P listen port in logs on startup.

Tests for general purpose ExecuteProduceConsume class to cover both
random task scenario and mass-ping scenario.
2019-08-13 08:40:50 +01:00
catbref
f8b496ff3c Convert SysTray to use JPopupMenu for Unicode support
Correct Systray_zh.properties to ISO 8859-1 instead of UTF-8.

Added SysTray test to GuiTests
2019-08-13 08:30:40 +01:00
catbref
82910b6524 Add periodic system-tray pop-up if Windows Time service not running 2019-08-13 08:27:43 +01:00
catbref
2f2d9a664d Replaced all NTP.getTime with System.currentTimeMillis. Clocks handled by O/S 2019-08-06 11:12:29 +01:00
catbref
9435e9576a Move getRewardByHeight from Block to BlockChain 2019-08-06 10:53:13 +01:00
catbref
e47b4dceb2 StringBuilder, and other, optimizations for repository-related classes.
Add optional "excludeZero" to API call GET /assets/balances

Added tests to call most API calls to check no exceptions are thrown.
2019-08-06 09:46:31 +01:00
catbref
48eae0cb38 Minor change to test account assertion message. 2019-08-06 09:41:52 +01:00
catbref
99ffd62a6e Change to how "best block" is determined. 2019-08-02 15:29:48 +01:00
catbref
915eebb8e5 BlockChain.isTestNet now BlockChain.isTestChain.
Added Settings.isTestNet.

Disabled ArbitraryDataManager for now.
2019-08-02 14:42:10 +01:00
catbref
2e6d33659c Fix CANCEL_ASSET_ORDER check for already closed, partially/fully matched, order.
Additional unit tests to cover above case.
2019-08-02 14:07:47 +01:00
catbref
d1d45b12f7 Added unit test to check initial proxy reward share of 0% is invalid 2019-08-02 13:06:16 +01:00
catbref
9af18aad34 Setting proxy forging reward share to 0% removes proxy forging relationship.
Included unit test to cover change.
Modified test blockchain config "test-chain-v2.json" to add maxProxyRelationships.

Added comments to some proxy-related methods in AccountRepository class.
2019-08-02 13:05:56 +01:00
catbref
03f60ef898 Fix incorrect declared transaction length for SET_GROUP and VOTE_ON_POLL transactions.
Added transaction [de]serialization test, along with corresponding random transaction generators.

Minor typo fix in Transaction.

Minor clarification in MessageTransactionTransformer.

Added debugging to Account.
2019-08-02 13:03:54 +01:00
catbref
02e8bdb034 Add support for fetching updates using a combination of hostname and IP address.
IP address used to create socket, hostname used for SNI, HTTPS, etc.

Added hostname+IP auto-update locations to Settings.
2019-08-02 12:53:49 +01:00
catbref
c2e8392f05 Synchronization improvements (again!)
Bumped version

Controller no longer uses block height to determine whether to sync
but now uses peer's latest block's timestamp and signature.

Also BlockGenerator checks whether it's generating in isolation using
the same peer info (latest block timestamp and signature).

Added API call POST /admin/forcesync peer-address to help get wayward
nodes back on track.

Unified code around, and calling, Transaction.importAsUnconfirmed().

Tidied code around somelock.tryLock() to be more readable.

Controller (post-sync) now broadcasts new chaintip info if
our latest block's signature has changed, not simply the height.

Network.broadcast() only sends out via outbound peer if node has
more than one connection from the same peer. So Controller would
only update one of the peer records with chaintip info.
Controller now updates all connected peers with the ID when it
receives a HEIGHT or HEIGHT_V2 message.

Added node1 thru node7.mcfamily.io to default peers in Network.

Network ignores first "listen port" entry when receiving peers
list from an outbound-connection peer as it already knows
by virtue of having connected to it!

More network message debug logging (hopefully never to be seen).

[some old code left in, but commented out, for a while]
2019-07-23 17:50:07 +01:00
catbref
e42c3e60bc Added mock-up of node management UI 2019-07-23 15:24:36 +01:00
catbref
4667b768df Added tests for each group-approval outcome + fixes 2019-07-23 15:13:28 +01:00
catbref
c9f226cf88 group approval tests and fixes 2019-07-23 15:12:35 +01:00
catbref
4b3f877dc0 Interim commit - refactored transaction transformers and fixed unit test compiler errors 2019-07-23 15:06:28 +01:00
catbref
06ba004238 Interim commit - refactored HSQLDBTransactionRepository, and subclasses. Also added approval_height to transactions, renaming height to block_height. 2019-07-23 14:50:56 +01:00
catbref
da1bd82c19 Minor integration progress
Remove fetching unconfirmed from Synchronizer

Add extra validity/reference/processable checks to
Transaction.isValidUnconfirmed

Update TransactionUtils to use Transaction.importAsUnconfirmed
for unit tests.
2019-07-23 14:48:49 +01:00
catbref
c9968b3dd2 Interim commit of *TransactionData classes for safety 2019-07-23 14:42:02 +01:00
catbref
c83d888a7d Demo HTML/JS page to show encrypted local storage of private key 2019-07-23 12:48:47 +01:00
catbref
f3c588d90f Fix sometimes erroneous Ed25519 public key to X25519 public key conversion.
Added mass (x1000) testing of key conversion and shared secret calculations.

Fix incorrect proxy private key test that has expected result from previous
algorithm.

Added another test HTML/JS file.
2019-07-23 11:22:23 +01:00
catbref
8f7c954f5a Proxy private keys are now SHA256(shared secret only) instead of SHA256(shared secret + public keys).
HTML/JS in src/test/resources/proxy-key-example.html updated accordingly.

Add handshake status to output of API call GET /peers

Add/correct @ApiErrors annotations on some API calls.

Add API call POST /admin/orphan (target height as body)
to force blockchain orphaning for when node is wildly out of sync.
Added support for above to BlockChain class.

BlockGenerator now requires a minimum number of peers
before it will generate any new blocks.
See "minBlockchainPeers" in settings.

Controller now requires a minimum number of peers
before it will consider synchronizing.
See "minBlockchainPeers" in settings.

Old "minPeers" entry in settings.json no longer valid!

Networking now allows both an outbound and inbound connection
to a peer although will use the outbound connection in preference.

Networking checks peer ID of inbound connections to detect,
and resolve, peer ID clashes/theft.
2019-07-23 11:09:25 +01:00
catbref
a3d4cf2900 MAJOR: Don't delete transactions when orphaning - make them unconfirmed again
Lots of edits to Transaction subclasses to change/remove 'delete'.

Corresponding extra changes to help reset some transaction fields to pre-process
state during orphaning.

Changed Block, GenesisBlock & Synchronizer to save transactions where appropriate.

Added enhanced GET_SIGNATURES_V2 network message to reduce the number of
block signatures sent over network.

Peers are now version 2 if they send a new-style build version string,
instead of using first digit from build version.
2019-07-23 10:37:37 +01:00
catbref
0259702df2 Fix generating X25519 shared secret.
X25519 shared secrets now match those generated by libsodium.

New tests show that shared secrets are the same using either set
of private+public key combinations.

Changed proxy private key generation from using simple SHA256
of shared secret to using SHA256(shared secret + both public keys).

Added a temporary "BouncyCastle25519" shim class to provide missing
key conversion from Ed25519 to X25519.
2019-07-23 10:29:18 +01:00
catbref
4279ad0673 Fix Javascript shared-secret & proxy forging private key code 2019-07-23 10:20:20 +01:00
catbref
4b02b7a14f Fix unit tests 2019-07-22 18:05:41 +01:00
catbref
57b982d2fb Block summaries (repository/data/message/synchronizer) + BlockGenerator
Also refactored some tests.

Original commit was 06fe8fc, with commit message:
Initial implementation of random block generator, etc.
2019-07-18 12:15:11 +01:00