Commit Graph

2913 Commits

Author SHA1 Message Date
CalDescent
8b3f9db497 Safety check - also to be removed shortly. 2022-06-13 22:41:10 +01:00
CalDescent
0eebfe4a8c Temporarily limit block minter as first stage of multipart minting fix. This will be reverted almost immediately after release. 2022-06-13 22:40:54 +01:00
CalDescent
12b3fc257b Updated AdvancedInstaller project for v3.3.3 2022-06-04 19:50:43 +01:00
CalDescent
66a3322ea6 Bump version to 3.3.3 2022-06-04 19:16:01 +01:00
CalDescent
4965cb7121 Set BlockV2Message min peer version to 3.3.3 2022-06-04 15:50:17 +01:00
CalDescent
b92b1fecb0 disableReferenceTimestamp set to 1655222400000 (Tuesday, 14 June 2022 16:00:00 UTC) 2022-06-04 14:51:37 +01:00
CalDescent
43a75420d0 Merge branch 'disable-reference' 2022-06-04 14:46:51 +01:00
catbref
e85026f866 Initial work on reducing network load for transferring blocks.
Reduced AT state info from per-AT address + state hash + fees to AT count + total AT fees + hash of all AT states.
Modified Block and Controller to support above. Controller needs more work regarding CachedBlockMessages.
Note that blocks fetched from archive are in old V1 format.
Changed Triple<BlockData, List<TransactionData>, List<ATStateData>> to BlockTransformation to support both V1 and V2 forms.

Set min peer version to 3.3.203 in BlockV2Message class.
2022-06-04 14:43:46 +01:00
CalDescent
ba7b9f3ad8 Added feature to allow repository to be kept intact after running certain tests 2022-06-04 14:17:01 +01:00
catbref
4eb58d3591 BlockTimestampTests to show results from changing blockTimingsByHeight 2022-06-04 12:36:36 +01:00
catbref
8d8e58a905 Network$NetworkProcessor now has its own LOGGER 2022-06-04 12:36:36 +01:00
catbref
8f58da4f52 OnlineAccountsManager:
Bump v3 min peer version from 3.2.203 to 3.3.203
No need for toOnlineAccountTimestamp(long) as we only ever use getCurrentOnlineAccountTimestamp().
Latter now returns Long and does the call to NTP.getTime() on behalf of caller, removing duplicated NTP.getTime() calls and null checks in multiple callers.
Add aggregate-signature feature-trigger timestamp threshold checks where needed, near sign() and verify() calls.
Improve logging - but some logging will need to be removed / reduced before merging.
2022-06-04 12:36:36 +01:00
catbref
a4e2aedde1 Remove debug-hindering "final" modifier from effectively final locals 2022-06-04 12:36:36 +01:00
catbref
24d04fe928 Block.mint() always uses latest timestamped online accounts 2022-06-04 12:36:35 +01:00
catbref
0cf32f6c5e BlockMinter now only acquires repository instance as needed to prevent long HSQLDB rollbacks 2022-06-04 12:35:54 +01:00
catbref
84d850ee0b WIP: use blockchain feature-trigger "aggregateSignatureTimestamp" to determine when online-accounts sigs and block sigs switch to aggregate sigs 2022-06-04 12:35:51 +01:00
catbref
51930d3ccf Move some private key methods to Crypto class 2022-06-04 12:35:15 +01:00
catbref
c5e5316f2e Schnorr public key and signature aggregation for 'online accounts'.
Aggregated signature should reduce block payload significantly,
as well as associated network, memory & CPU loads.

org.qortal.crypto.BouncyCastle25519 renamed to Qortal25519Extras.
Our class provides additional features such as DH-based shared secret,
aggregating public keys & signatures and sign/verify for aggregate use.

BouncyCastle's Ed25519 class copied in as BouncyCastleEd25519,
but with 'private' modifiers changed to 'protected',
to allow extension by our Qortal25519Extras class,
and to avoid lots of messy reflection-based calls.
2022-06-04 12:35:15 +01:00
catbref
829ab1eb37 Cherry-pick minor fixes from another branch to resolve "No online accounts - not even our own?" issues 2022-06-04 12:35:03 +01:00
catbref
d9b330b46a OnlineAccountData no longer uses signature in equals() or hashCode() because newer aggregate signatures use random nonces and OAD class doesn't care about / verify sigs 2022-06-04 10:49:59 +01:00
catbref
c032b92d0d Logging fix: size() was called on wrong collection, leading to confusing logging output 2022-06-04 10:49:59 +01:00
catbref
ae92a6eed4 OnlineAccountsV3: slightly rework Block.mint() so it doesn't need to filter so many online accounts
Slight optimization to BlockMinter by adding OnlineAccountsManager.hasOnlineAccounts():boolean instead of returning actual data, only to call isEmpty()!
2022-06-04 10:49:59 +01:00
catbref
712c4463f7 OnlineAccountsV3:
Move online account cache code from Block into OnlineAccountsManager, simplifying Block code and removing duplicated caches from Block also.
This tidies up those remaining set-based getters in OnlineAccountsManager.
No need for currentOnlineAccountsHashes's inner Map to be sorted so addAccounts() creates new ConcurentHashMap insteaad of ConcurrentSkipListMap.

Changed GetOnlineAccountsV3Message to use a single byte for count of hashes as it can only be 1 to 256.
256 is represented by 0.

Comments tidy-up.
Change v3 broadcast interval from 10s to 15s.
2022-06-04 10:49:59 +01:00
catbref
fbdc1e1cdb OnlineAccountsV3:
Adding support for GET_ONLINE_ACCOUNTS_V3 to Controller, which calls OnlineAccountsManager.

With OnlineAccountsV3, instead of nodes sending their list of known online accounts (public keys),
nodes now send a summary which contains hashes of known online accounts, one per timestamp + leading-byte combo.
Thus outgoing messages are much smaller and scale better with more users.
Remote peers compare the hashes and send back lists of online accounts (for that timestamp + leading-byte combo) where hashes do not match.

Massive rewrite of OnlineAccountsManager to maintain online accounts.
Now there are three caches:
1. all online accounts, but split into sets by timestamp
2. 'hashes' of all online accounts, one hash per timestamp+leading-byte combination
Mainly for efficient use by GetOnlineAccountsV3 message constructor.
3. online accounts for the highest blocks on our chain to speed up block processing
Note that highest blocks might be way older than 'current' blocks if we're somewhat behind in syncing.

Other OnlineAccountsManager changes:
* Use scheduling executor service to manage subtasks
* Switch from 'synchronized' to 'concurrent' collections
* Generally switch from Lists to Sets - requires improved OnlineAccountData.hashCode() - further work needed
* Only send V3 messages to peers with version >= 3.2.203 (for testing)
* More info on which online accounts lists are returned depending on use-cases

To test, change your peer's version (in pom.xml?) to v3.2.203.
2022-06-04 10:49:59 +01:00
catbref
f2060fe7a1 Initial work on online-accounts-v3 network messages to drastically reduce network load.
Lots of TODOs to action.
2022-06-04 10:49:59 +01:00
catbref
6950c6bf69 Initial work on reducing network load for transferring blocks.
Reduced AT state info from per-AT address + state hash + fees to AT count + total AT fees + hash of all AT states.
Modified Block and Controller to support above. Controller needs more work regarding CachedBlockMessages.
Note that blocks fetched from archive are in old V1 format.
Changed Triple<BlockData, List<TransactionData>, List<ATStateData>> to BlockTransformation to support both V1 and V2 forms.

Set min peer version to 3.3.203 in BlockV2Message class.
2022-06-04 10:49:11 +01:00
catbref
8a76c6c0de Sync behaviour changes:
* Prefer longer chains
* Compare chain tips
* Skip pre-sync all-peer sorting (for now)
2022-06-04 10:23:31 +01:00
CalDescent
ef51cf5702 Added defensiveness in getOnlineTimestampModulus(), just in case NTP.getTime() returns null 2022-06-02 12:46:11 +01:00
CalDescent
0c3988202e Merge branch 'master' into increase-online-timestamp-modulus 2022-06-02 12:38:05 +01:00
CalDescent
987446cf7f Updated AdvancedInstaller project for v3.3.2 2022-06-02 11:31:56 +01:00
CalDescent
6dd44317c4 Sync pirate wallet every 30 seconds instead of 60, to match behaviour of official wallet. 2022-05-31 10:09:37 +02:00
CalDescent
d2fc705846 Merge remote-tracking branch 'catbref/UnsupportedMessage' 2022-05-31 09:09:08 +02:00
CalDescent
e393150e9c Require references to be the correct length post feature-trigger 2022-05-30 22:40:39 +02:00
CalDescent
43bfd28bcd Revert "Discard unsupported messages instead of disconnecting the peer."
This reverts commit d086ade91f.
2022-05-30 21:46:16 +02:00
catbref
ca8f8a59f4 Better forwards compatibility with newer message types so we don't disconnect newer peers 2022-05-30 20:41:45 +01:00
CalDescent
85a26ae052 Another rework of null seed wallets, to allow them to be saved and loaded.
A full sync is unavoidable for P2SH redeem/refund, so we need to be able to save our progress. Creating a new null seed wallet each time isn't an option because it relies on having a recent checkpoint to avoid having to sync large amounts of blocks every time (sync is per wallet, not per node).
2022-05-30 19:31:55 +02:00
CalDescent
c30b1145a1 Improved ensureSynchronized() as it would often not notice an unsynced wallet. 2022-05-30 18:14:26 +02:00
CalDescent
d086ade91f Discard unsupported messages instead of disconnecting the peer. 2022-05-30 15:27:42 +02:00
CalDescent
64d4c458ec Fixed logging error 2022-05-30 15:16:44 +02:00
CalDescent
2478450694 Revert "Removed "consecutive blocks" limitation in block minter."
This reverts commit f41fbb3b3d.
2022-05-30 13:49:15 +02:00
CalDescent
9f19a042e6 Added test to ensure short (1 byte) references can be imported. 2022-05-30 13:46:00 +02:00
CalDescent
922ffcc0be Modified post-trigger last reference checking, to now require a non-null value
This allows for compatibility with TRANSFER_PRIVS validation in commit 8950bb7, which treats any account with a non-null reference as "existing". It also avoids possible unknown side effects from trying to process and store transactions with a null reference - something that wouldn't have been possible until the validation was removed.
2022-05-30 13:22:15 +02:00
CalDescent
f887fcafe3 Disable last reference validation after feature trigger timestamp (not yet set).
This should prevent the failed transactions that are encountered when issuing two or more in a short space of time. Using a feature trigger (hard fork) to release this, to avoid potential consensus confusion around the time of the update (older versions could consider the main chain invalid until updating).
2022-05-30 13:06:55 +02:00
CalDescent
48b562f71b Auto update check interval slowed from 10s to 30s, to hopefully reduce the chance of encountering "repository in use by another process?" error. 2022-05-28 14:33:37 +02:00
CalDescent
5203742b05 Started work on architecture-specific lite wallet library loading. Paths are not yet correct. 2022-05-28 14:31:38 +02:00
CalDescent
f14b494bfc "Disposable" wallets renamed to "null seed" wallets, as this is a better description of what they are. 2022-05-28 14:28:42 +02:00
CalDescent
9a4ce57001 Increase blockchain lock wait time from 30 to 60 seconds in /transactions/process.
This will hopefully reduce the number of failed tradeoffer listings that result in a nonfunctional tradebot (and subsequent PENDING status shown in the UI)
2022-05-28 13:33:09 +02:00
CalDescent
10af961fdf Consider a node with a block in the last 30 mins to be "up to date" when trading. 2022-05-27 22:31:21 +02:00
CalDescent
33cffe45fd Bump version to 3.3.2 2022-05-27 17:23:53 +02:00
CalDescent
a0ce75a978 Minimum BTC order amount set to 0.001 BTC. Anything lower than that will result in greater than 10% fees. 2022-05-27 16:59:34 +02:00