Commit Graph

395 Commits

Author SHA1 Message Date
catbref
ad827ae01d Fix Controller's processing of HEIGHT_V2 to update all peers with same ID 2019-08-06 10:59:13 +01:00
catbref
bdfbea3a53 Remove OSGi/felix from pom.xml 2019-08-06 10:57:29 +01:00
catbref
976ea97af1 Improve pruning of old peers 2019-08-06 10:57:02 +01:00
catbref
9435e9576a Move getRewardByHeight from Block to BlockChain 2019-08-06 10:53:13 +01:00
catbref
f45cedb6ff Save lastConnected for outbound peers on completed handshake 2019-08-06 09:49:44 +01:00
catbref
840f52ff90 Added API call GET /blocks/timestamp/{timestamp} and increase timeout on POST /admin/forcesync from 5s to 30s 2019-08-06 09:47:38 +01:00
catbref
2621e04025 Change Peer ping interval to below that of SOCKET_TIMEOUT, used by blocking read()s 2019-08-06 09:47:12 +01:00
catbref
e47b4dceb2 StringBuilder, and other, optimizations for repository-related classes.
Add optional "excludeZero" to API call GET /assets/balances

Added tests to call most API calls to check no exceptions are thrown.
2019-08-06 09:46:31 +01:00
catbref
48eae0cb38 Minor change to test account assertion message. 2019-08-06 09:41:52 +01:00
catbref
99ffd62a6e Change to how "best block" is determined. 2019-08-02 15:29:48 +01:00
catbref
c559c16a4a Add extra isInterrupted() quick-exits to Controller during TRANSACTION_SIGNATURES processing. 2019-08-02 15:11:04 +01:00
catbref
fc82bcaf49 Tidying up peer info.
lastHeight/blockSig/blockTimestamp, etc. moved from PeerData/repository
to Peer as it's transient so no need to store in repository.

Repository now keeps track of when/who added peer, e.g. API/INIT/another peer.

API calls DELETE /peers and DELETE /peers/known also disconnect peer
as well as deleting from repository.

Connection timestamp now reported by API call GET /peers

Some repository-updating code removed from Network/Controller as no
longer needed.

Removed obsolete Controller.hasShorterBlockchain predicate.
2019-08-02 14:56:19 +01:00
catbref
8b135eb447 Improve startup & shutdown. Improve API call POST /admin/forcesync.
Previous version was prone to throwing exceptions during shutdown
sequence, especially in Peer/Network-related threads.

Shutdown now tries to wait for Peer/Network threads to cleanly exit.

API call POST /admin/forcesync now tries to grab blockchain lock,
with timeout, and then repeatedly sync with requested peer until
either fully synced or something unexpected happens.
2019-08-02 14:54:22 +01:00
catbref
84f5935d38 Improve shutdown of Network.
Detect interrupt during peer connect so we can return without trying
to set up new peer.

Do join() after interrupt() in Network.shutdown.
2019-08-02 14:52:46 +01:00
catbref
c68e0eb6ea change auto-update download locations & logging message 2019-08-02 14:50:17 +01:00
catbref
5b70f0004d Disregard misbehaved peers before counting to see if we can synchronize.
Also added simple system-tray tooltip update to show number of peers and current height.
2019-08-02 14:47:40 +01:00
catbref
915eebb8e5 BlockChain.isTestNet now BlockChain.isTestChain.
Added Settings.isTestNet.

Disabled ArbitraryDataManager for now.
2019-08-02 14:42:10 +01:00
catbref
1d81c4db6b BlockGen / Synchronizer improvements
BlockGenerator will now attempt to generate a new block if none of its
peers have a recent block either (in case of network stall). BlockGenerator
still needs a minimum number of peers before generating though.

Reduce BlockGenerator workload and use of blockchain lock if it
can't generate a block.

Reduce Synchronizer logging output.

Unify calculating timestamp threshold for 'recent' block into Controller.
2019-08-02 14:33:41 +01:00
catbref
5acc92ef26 Improve TRANSACTION_SIGNATURES handling in Controller.
Don't disconnect peers that fail to send a requested transaction,
as they may no longer have it. e.g. transaction might have expired
or become invalid.

For some other cases, e.g. we have transaction already, move on to
requesting the next transaction instead of giving up on the list.
2019-08-02 14:16:18 +01:00
catbref
b48f671774 In AutoUpdate, pass download buffer to SHA256 digester BEFORE deXORing. 2019-08-02 14:15:56 +01:00
catbref
8727780b77 Added XorUpdate utility to help prepare auto-updates. 2019-08-02 14:15:38 +01:00
catbref
877c126ba3 Don't allow incorrect/null ARBITRARY dataType when building transactions via
API's POST /arbitrary call.
2019-08-02 14:14:56 +01:00
catbref
4cdb5e6b95 Auto-Update improvements
NOTE: Downloaded update JARs are now expected to have been XORed with 0x5A!

This is to help prevent Windows Firewall from blocking update downloads
based on deep packet inspection.

Download read timeout reduced from 5s to 3s.

Download locations reordered so github entries are at the top as they have
better CDNs.

ApplyUpdate now assumes null response from GET /admin/stop means node
is not running.

ApplyUpdate now checks replacement JAR actually exists before attempting
to overwrite previous version.

ApplyUpdate now tries to use Windows EXE launcher in preference to raw
java command line. (This should improve Windows installer behaviour
in detecting running process and possibly firewall implications too).
2019-08-02 14:13:04 +01:00
catbref
2e6d33659c Fix CANCEL_ASSET_ORDER check for already closed, partially/fully matched, order.
Additional unit tests to cover above case.
2019-08-02 14:07:47 +01:00
catbref
aa141bb97f Added settings.json to gitignore 2019-08-02 14:07:12 +01:00
catbref
d1d45b12f7 Added unit test to check initial proxy reward share of 0% is invalid 2019-08-02 13:06:16 +01:00
catbref
9af18aad34 Setting proxy forging reward share to 0% removes proxy forging relationship.
Included unit test to cover change.
Modified test blockchain config "test-chain-v2.json" to add maxProxyRelationships.

Added comments to some proxy-related methods in AccountRepository class.
2019-08-02 13:05:56 +01:00
catbref
46dc91e5a0 Bump HSQLDB from svn r5970 to release v2.5.0 2019-08-02 13:05:18 +01:00
catbref
ae6cf9fc32 Fix ordering when requesting summary of block forgers 2019-08-02 13:04:48 +01:00
catbref
4f25fffbe9 Change API transaction subsmission from single-shot to 500ms timeout for obtaining blockchain lock 2019-08-02 13:04:21 +01:00
catbref
03f60ef898 Fix incorrect declared transaction length for SET_GROUP and VOTE_ON_POLL transactions.
Added transaction [de]serialization test, along with corresponding random transaction generators.

Minor typo fix in Transaction.

Minor clarification in MessageTransactionTransformer.

Added debugging to Account.
2019-08-02 13:03:54 +01:00
catbref
02e8bdb034 Add support for fetching updates using a combination of hostname and IP address.
IP address used to create socket, hostname used for SNI, HTTPS, etc.

Added hostname+IP auto-update locations to Settings.
2019-08-02 12:53:49 +01:00
catbref
8dd4745c5c Work on auto-update
Arbitrary transactions now [de]serialize data-type (raw/hash) for v4+ transactions.
Data type also stored in repository. Very small (<=255 byte) data payloads are also stored directly in HSQLDB.

Added ArbitraryDataManager which looks for hash-only data payloads and possibly requests raw data over network
depending on 'policy' (which currently is "fetch everything").

Added networking support for finding, and transferring, arbitrary data payloads.

Minor optimization to message ID generation in Peer.

Minor optimization in Serialization.serializeSizedString()
2019-08-02 12:48:48 +01:00
catbref
4ced9cc3e2 More flexible auto-update locations 2019-08-02 12:44:52 +01:00
catbref
1d7bda5d46 Don't propagate new transactions back to sending peer. Added small inter-peer delay when doing network broadcast. 2019-08-02 12:42:30 +01:00
catbref
c2e8392f05 Synchronization improvements (again!)
Bumped version

Controller no longer uses block height to determine whether to sync
but now uses peer's latest block's timestamp and signature.

Also BlockGenerator checks whether it's generating in isolation using
the same peer info (latest block timestamp and signature).

Added API call POST /admin/forcesync peer-address to help get wayward
nodes back on track.

Unified code around, and calling, Transaction.importAsUnconfirmed().

Tidied code around somelock.tryLock() to be more readable.

Controller (post-sync) now broadcasts new chaintip info if
our latest block's signature has changed, not simply the height.

Network.broadcast() only sends out via outbound peer if node has
more than one connection from the same peer. So Controller would
only update one of the peer records with chaintip info.
Controller now updates all connected peers with the ID when it
receives a HEIGHT or HEIGHT_V2 message.

Added node1 thru node7.mcfamily.io to default peers in Network.

Network ignores first "listen port" entry when receiving peers
list from an outbound-connection peer as it already knows
by virtue of having connected to it!

More network message debug logging (hopefully never to be seen).

[some old code left in, but commented out, for a while]
2019-07-23 17:50:07 +01:00
catbref
d910cce807 Improve API peer info output 2019-07-23 17:38:56 +01:00
catbref
20200b844e Networking and synchronization improvements
Controller now sets (volatile) requestSync flag when a peer sends new height info.
This allows much quicker response to new blocks which might hopefully improve synchronization
compared with the old periodic sync method.

"Unsolicited" network messages are now added to a BlockingQueue,
and a separate unsolicited message processing thread (one per peer)
deals with this messages in turn.
This allows "reply" network messages to propagate up to the
threads that are waiting for them, preventing deadlocks and
peer disconnections due to lost pings.

Controller tries to do as much new transaction processing
outside of the blockchain lock as possible, and only
broadcasts new transaction's signature if we successfully
import transaction to our unconfirmed pile.

Synchronizer.findSignaturesFromCommonBlock now returns null
to indicate some sort of connection issue (no cool-off)
and an empty list to indicate NO COMMON BLOCK.
That method also tries to work back to genesis block
instead of giving up too early if test block height
becomes negative.

Network.createConnection additionally filters out
candidates if their addresses resolve to the same
IP+port as an existing connection. So now it won't
connect to localhost:1234 if it has an existing
connection with 127.0.0.1:1234.

Network.broadcast only considers unique peers,
i.e. prefers outbound connection if a peer has
corresponding inbound connection.

Added Thread.currentThread().setName() where possible.
2019-07-23 17:32:21 +01:00
catbref
ffffb50884 Networking/Controller changes to aid broadcast of unconfirmed transactions.
Notably: network messages passed up to Controller are now processed in their
own thread, as opposed to peer's thread.
So each message processor in Controller needs to thread-safe.

V2 network protocol asks for unconfirmed transactions, can send round lists
of transaction signatures and ask for individual transactions, to save
bandwidth/processing.
2019-07-23 17:28:20 +01:00
catbref
680361daa6 Temporary fix to Controller to only try to acquire blockchain for incoming new unconfirmed transactions 2019-07-23 16:19:01 +01:00
catbref
c03a5c3d0e Fix incorrect count of forged blocks 2019-07-23 16:18:24 +01:00
catbref
b7f53afe68 Improve API output for GET /peers 2019-07-23 16:18:01 +01:00
catbref
3a3cc5a81b Improve API output of GET /blocks/forgers 2019-07-23 16:17:42 +01:00
catbref
97a5b3ef9b Increase max proxy relationships in blockchain config 2019-07-23 16:17:19 +01:00
catbref
5429cb12dd Possible fix for single instance check by AI Windows launcher 2019-07-23 16:13:51 +01:00
catbref
2f3123a315 Add lock around some Peer.peerData actions to help sync 2019-07-23 16:06:04 +01:00
catbref
6a969f9473 Add limit to number of proxy forging relationships per generator account.
For testing, current limit is 4 as set in blockchain.json
2019-07-23 16:05:21 +01:00
catbref
de96e8347d Move handling of GET_PEERS from Controller to Network. On sync fail, only update outbound peer info in repository to help prevent useless 'known peer' entries. 2019-07-23 16:02:32 +01:00
catbref
dcb8d67236 Don't allow new transactions to be submitted if not up to date 2019-07-23 15:59:01 +01:00
catbref
491a6ecd67 Fix incorrect declared transaction length for create-group transactions 2019-07-23 15:46:42 +01:00