Various issues in Jetty v9.4.22 (and some later versions too)
cause websockets to use up all available threads.
Bumped Jetty to v9.4.29 to resolve some of these issues.
Changed some Qortal-side websocket code to minimize
locking on websocket notifiers. Websocket messages now
sent async, although the returned Futures are discarded,
as it's up to the remote end to consume fast enough.
Changed Controller to only request a SysTray update before
synchronization if there's a chance node might change height.
Similarly, Controller only requests SysTray update after
synchronization if chain tip has actually changed.
Both of the above together should reduce the number of
messages sent out via the admin status websockets.
Previous version fetched all the blocks from previous 'timestamp'
to current height, checking each transaction. (very slow)
New implementation leverages repository to do the heavy lifting.
Could potentially benefit from some DB indexes in the future?
Added unit test to cover.
Bitcoin main-net ElectrumX server list added to ElectrumX class,
albeit commented out at this point until it is decided that trade-bot
is ready for production use. (Simply remove the leading //s)
More comments and documentation has been added to TradeBot class
to further describe the actions taken.
It is important to note that:
Bitcoin wallet access is required by trade-bot
and so:
A Bitcoin WALLET PRIVATE KEY is stored in the database by trade-bot
and hence, if you use trade-bot:
DO NOT DISTRIBUTE YOUR DB FILES TO ANYONE ELSE!
Furthermore it should be obvious that this functionality is provided on
a 'best effort", not guaranteed, basis, therefore:
YOUR FUNDS ARE AT RISK!
If you are unsure about any aspect, or cannot afford to lose your funds,
or it's possible that unexpected outcomes occur, then DO NOT USE.
To use trade-bot on Bitcoin TESTNET then this to your settings JSON file:
"bitcoinNet": "TEST3",
See Settings.java line 100, and BTC class for more info.
bitcoinj now uses ElectrumX as an UTXO provider in order to keep track
of coins in BIP32 deterministic wallet.
Trade responder (Alice) needs to pass a BIP32 extended private key to API
so trade-bot can create unattended spends.
Both Alice and Bob can find their final funds in accounts using the
ephemeral 'tradePrivateKey' from trade-bot state data.
Most cross-chain API calls are now only allowed from localhost.
Most Bitcoin fees pegged at 0.00001000 BTC.
More work needed to handle refunds in case of trade failures.
(See XXX comment tags in TradeBot.java)
Qortal AT now includes suggested tradeTimeout again as a constant so trade partner/recipient can use that to calculate a suitable lockTimeA. CODE_HASH changed!
Renamed some secret_hash to hash_of_secret.
Changed TradeBotStates.trade_state back to TINYINT and adjusted values in TradeBotData.State enum to suit.
Added lockTimeA to TradeBotData & repository.
Added JAXB-only extra representations of Bitcoin PKHs as addresses.
Fixed incorrect expected length in BTCACCT.extractOfferMessageData().
CrossChainTradeData.refundTimeout now only present in TRADE mode.
Added BTC.pkhToAddress().
Added initial TradeBot.handleAliceWaitingForP2shA().
Enforce only one TradeBot thread running using 'activeFlag' atomic boolean.
Replace incorrect SHA256 with HASH160 for hashOfSecretA in TradeBot.startResponse().
Controller now calls TradeBot.onChainTipChange() inside thread
started by Controller.onNewBlock(), instead of blocking
Controller.setChainTip().
DB TradeBotStates has trade_foreign_public_key changed to VARBINARY(33)
as Bitcoin pubkeys aren't uniformly 32 bytes!
Also, trade_state changed from TINYINT to SMALLINT to cover enum value range.
TradeBot.createTrade() incorrectly used Crypto.digest() to create hash-of-secret
instead of Crypto.hash160(). Also corrected tradeState to
BOB_WAITING_FOR_AT_CONFIRM. Also added missing fee calculation.
Added missing repository.saveChanges() to TradeBot methods.
Added balance check to API POST /crosschain/tradebot before passing
request to TradeBot.createTrade(), which also ensures there's a
usable account last-reference too.
BTC.getBalance() now returns Long instead of Coin.
BTC.FORMAT.format(Coin) changed to BTC.format(Coin or long).
Added BTC.deriveP2shAddress(byte[] redeemScriptBytes).
Added GET_MESSAGE_LENGTH_FROM_TX_IN_A
and PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.
Replaced AT-1.3.4 with version including bug-fix for off-by-one
data address bounds checking.
Moved long-from-bytes method to BitTwiddling class.
Renamed some methods to make it more obvious they work with
little/big endian data.
Also added Named/DaemonThreadFactory classes.
Network EPC now uses NamedThreadFactory for easier debugging.
Added settings field "networkPoWComputePoolSize", default 2, which
seems to work with both low-power ARM boards and high-power desktops.
If a node accepts a connection from an inbound peer
then remote peer will send RESPONSE first
and local node would previously change handshaking state
to COMPLETED while computing their own RESPONSE.
This meant that the local node would sometimes also start
sending post-handshake messages to the remote peer,
e.g. TRANSACTION_SIGNATURES.
Remote peer is only expecting a RESPONSE message, so would
close connection.
So we introduce an extra handshaking state "RESPONDING" for use
by local node while they compute RESPONSE in a separate thread.
Once the RESPONSE has been sent, local node moves to COMPLETED
state and called onHandshakeCompleted() as per usual.
Note that the code path when connecting outbound to a remote peer
is not changed, and the RESPONDING state is not used.
Also in this commit:
Network.onPeerReady now bypasses call to onMessage and instead
calls onHandshakingMessage() directly to avoid race condition
where peer's handshake status could change between
onPeerReady's caller and onMessage() calling peer.getHandshakeStatus()
NodeStatus contructor now fills in fields, which themselves are now 'final'.
NodeStatus also includes numberOfConnections and height as per systray.
AdminResource.status() unified with websocket version.
Incorrect column names when saving a group ban.
Missing column in LeaveGroupTransactions.
More stringent validity checks in group-kick, group-ban and remove-group-admin.
Added loads more tests to cover group actions.
Requires entries 'sslKeystorePathname' and 'sslKeystorePassword'
in settings.json.
With SSL enabled, API will auto-detect HTTP or HTTPs on the same port.
Included tools/build-keystore.sh to help build keystore from
Let's Encrypt certificates.
Renamed GET /blocks/minters to /blocks/signers
Renamed GET /blocks/minter/{address} to /blocks/signer/{address}
Changed corresponding repository methods and data classes.
Controller.onBlockMinted() now .onNewBlock(BlockData)
which saves having to fetch from repository.
Controller.onNewBlock also takes care of updating Controller's
cached chain tip, requesting SysTray refresh, broadcasting
new tip info to peers and notifying websockets.
BlockMinter and Controller.actuallySynchronize updated
to use unified .onNewBlock.
BlocksWebsocket also returns blocks on demand, given either
integer block height or base58 block signature.
Added support to return ApiError via websockets.
Unified Transaction.importAsUnconfirmed() and Controller.onNetworkTransactionMessage()
to both call Controller.onNewTransaction().
Modified Controller.onNewTransaction() to only send transaction signature to
other peers, instead of full transaction. Peers can request full transaction if they
don't have it.
Controller.onNewTransaction() also calls ChatNotifier, which in turn
notifies websocket handlers about new CHAT transactions.
Added jetty websocket dependency to pom.xml
Any reward leftover from ditributing to legacy QORA holders is reallocated to either:
founders if any online
or
account-level-based reward candidates, if no founders online
We should get pretty close to 100% block reward distribution, barring rounding artifacts.
More documentation and tests.
Removed BlockChain's founderShare as it is calculated in Block on a per-block basis instead.
Now we sum generic block reward + transaction fees before performing
distribution only once.
Added Map to collate account-balance changes during block reward
distribution so the final changes can be applied in one batch,
reducing DB load.
Some other optimizations like a faster ExpandedAccount.getShareBin().
Passes test EXCEPT RewardTests.testLegacyQoraReward(), pending decision
on how to reallocate 'unspent' block reward.
No more bitcoinj peer-group stalls, or slow startups,
or downloading tons of block headers, or checkpoint files.
Now we use ElectrumX protocol to query info from random servers.
Also:
BTC.hash160 callers now use Crypto.hash160 instead.
Added BitTwiddling.fromLEBytes() returns int.
Unit tests seem OK, but needs complete testnet ACCT walkthrough.
Old Qora v1 message types removed.
Message type values changed.
Network handshaking reworked to fix multiple-connections issue.
Instead of using some random peerID, we now use proper keypairs and a challenge-response handshake to prevent doppelgangers/ID-theft.
This results in simpler handshaking code as we don't have to perform some arcane doppelganger resolution.
Handshaking still uses proof-of-work for challenge-response, but switched to newer MemoryPoW.
API call GET /peers no longer has 'buildTimestamp' field, but does now have 'nodeId' field.
Network no longer has a whole raft of getXXXpeers() due to simplified handshaking.
Quite a few method calls changed to simply Network.getHandshakedPeers(), which is also faster.
Previously GET /chats/active/{address} would only return an active group chat
entry where 'address' was a member AND there was an existing CHAT
transaction with the same tx_group_id (and no recipient).
Now the response contains entries for ALL groups where 'address' is a member,
regardless of an existing CHAT transactions, omitting the 'timestamp' entry
if there are none.
CREATE_GROUP, ISSUE_ASSET, REGISTER_NAME and UPDATE_NAME transactions affected.
The code to actually generate 'reduced' name was called inside isValid() and
relied on setting the corresponding transaction data object field so that it would
be saved by isValid()'s caller. Although this worked, it wasn't a very clean
solution.
Now the 'reduced' name is generated by transaction data object's constructors so
it is always present.
Also removed name/group/asset reduceName(String) methods as they were all the
same single-line call to Unicode.sanitize().
Group owner now derived from CREATE_GROUP transaction creator's public key.
Added 'reduced' group name to GroupData, with corresponding change to DB.
Renamed GroupData.getIsOpen() to simply isOpen().
Tidied up CreateGroupTransactionData, adding 'reduced' group name.
Renamed getIsOpen() to simply isOpen().
Added code to generated reduced group name when building genesis block.
Added Group.MIN_NAME_SIZE of 3.
DB tables changed to add reduced_group_name where appropriate,
removing owner where necessary.
Added GroupRepository.reducedGroupNameExists(String).
Fixed up test blockchain configs in src/test/resources/test-chain-v2*.json.
This allows on-chain messages to a group, including NO_GROUP / groupID zero.
No-recipient messages cannot have an amount - where would it go?
Changed MESSAGE serialization layout to add boolean indicating
whether recipient is present.
Changed MESSAGE serialization layout so assetID is after amount,
and only present if amount is non-zero.
Changed DB table structures to cover above.
Added unit tests to cover above.
Owner now derived from issuer's public key.
Maximum asset name length reduced to 40 characters.
Repository table changes.
"owner" removed from test blockchain configs and "issuerPublicKey" used instead
where applicable.
Some getters in the form of "getIs___()" renamed to simply "is____()".
Changes include:
* Allowing renaming
* Tracking last-updated timestamps
* More stringent Unicode processing
* Way more unit tests
* Max name length reduction to 40 chars
Note: HSQLDB repository table changes
Controller no longer starts up BTC support during main startup.
This does mean that BTC startup is deferred until first BTC-related
action, and that the first BTC-related action will take much longer
to complete.
Added tests to cover startup/shutdown.
This also fixes splash logo stuck on-screen and broken Controller
shutdown when using REGTEST bitcoin network AND there is no
local regtest bitcoin server running.
