Commit Graph

2281 Commits

Author SHA1 Message Date
CalDescent
84e4f9a1c1 Rework of arbitraryRelayMap to keep track of multiple responses.
Previously, only one peer's response for a hash would be remembered, even if multiple others reported back too. This would cause useful mapping to be lost.
2022-02-06 17:20:01 +00:00
CalDescent
cd5ce6dd5e Don't remove from the relay map after a file is requested, as it may be needed by other peers.
It will be cleaned up automatically after 60 seconds, so it is best to keep the data intact until then.
2022-02-06 16:04:54 +00:00
CalDescent
9ec4e24ef6 Slightly optimized logic in fetchArbitraryDataFiles() 2022-02-06 15:45:40 +00:00
CalDescent
fa447ccded Builder thread updates. 2022-02-06 15:37:21 +00:00
CalDescent
ef838627c4 Stop asking for hashes from a peer if one fails.
This fixes the request looping that occurs on when a peer is unable to serve files.
2022-02-06 15:37:08 +00:00
CalDescent
b8aaf14cdc Introduced ArbitraryDataFileRequestThread to allow for multiple concurrent file requests.
This is likely a short term solution (to allow existing code to be repurposed) until replaced with a task-based approach, as this will allow for a much greater number of threads.
2022-02-06 15:34:06 +00:00
CalDescent
2740543abf Added "async" and "attempts" parameters to GET /arbitrary/{service}/{name}* endpoints.
async = fail immediately with 404 if missing, and request in the background
attempts = the number of times to request the data (synchronous mode only for now)
2022-02-06 13:04:58 +00:00
CalDescent
3c526db52e Fixed bug in build manager which would prevent future builds until the core was restarted. 2022-02-06 13:03:01 +00:00
CalDescent
cfe0414d96 Small rework of invalidUnconfirmedTransactions to specify the expiry time instead of the time added.
This allows TIMESTAMP_TOO_OLD transactions to be tracked for a shorter time (10 minutes) than the other invalid transactions (60 minutes). Should reduce network traffic and db load around the time that transactions are expiring, as there is a lag before they are noticed and removed from each node. Due to the variance, it could cause other peers to request them again after deleting. They are now ignored for 10 minutes to avoid request spam.
2022-02-06 12:35:41 +00:00
CalDescent
08e06ba11a Fixed bugs preventing invalidUnconfirmedTransactions from working as intended. 2022-02-06 12:09:44 +00:00
CalDescent
8c03164ea5 Don't add expired transactions to invalidUnconfirmedTransactions, as there is no need to keep track of these. 2022-02-06 11:55:07 +00:00
CalDescent
0fe2f226bc Added invalidUnconfirmedTransactions map
An incoming invalid unconfirmed transaction will be added to this map if its timestamp is more than 30 minutes old. This should allow enough time and opportunities for it to be imported and included in a block (allowing for re-orgs which could switch its status from invalid to valid).

Once added, it will be removed after an hour to allow for another chance to be requested from any peers that still have it. If invalid again, it's added back to the map for another hour.

This fixes a 24 hour long loop, where invalid transactions are requested over and over from peers that have already imported them. It could be improved further by periodically removing invalid unconfirmed transactions from the database, but this will be a higher risk.

The results of this feature should be less network traffic, and less blockchain locks (which should ultimately increase the responsiveness of the synchronizer).
2022-02-06 11:23:28 +00:00
CalDescent
55b5702158 Invalidate last low weight block signature whenever the previous block data changes. 2022-02-05 17:54:36 +00:00
CalDescent
a4cbbb3868 Moved block minter sleep to later in the process, otherwise it can remain there for longer than expected. 2022-02-05 17:54:36 +00:00
CalDescent
816b01c1fc Fixed issue in rebase 2022-02-05 17:54:36 +00:00
CalDescent
483e7549f8 Revert "Moved log from INFO to DEBUG, as now the synchronizer is on its own thread it can occur more often than before."
This reverts commit e2e87766fa.
2022-02-05 17:54:35 +00:00
CalDescent
60d71863dc Allow 3 seconds for the synchronizer to obtain a blockchain lock, to reduce missed attempts. 2022-02-05 17:54:35 +00:00
CalDescent
170244e679 More work on higher weight chain detection in the block minter.
Added 30 second timeout, so that any errors should self correct.
2022-02-05 17:54:35 +00:00
CalDescent
472e1da792 Added debug level logging in higherWeightChainExists() for better visibility. 2022-02-05 17:54:35 +00:00
CalDescent
cbf03d58c8 Made synchronizer method public as it is now also used by the block minter. 2022-02-05 17:54:35 +00:00
CalDescent
ba41d84af9 Initial attempt to avoid an unnecessary block submission if one of our peers already has a higher weight chain. 2022-02-05 17:54:35 +00:00
CalDescent
98831a9449 Break out of the various loops in the cleanup manager if the thread is stopping. 2022-02-05 17:53:49 +00:00
CalDescent
9692539a3f Don't include fee in balance calculation (it looks like it could be double counting at the moment). 2022-02-05 17:24:33 +00:00
CalDescent
76df332b57 Check for null IP address before notifying of an external IP update. 2022-02-05 11:51:54 +00:00
CalDescent
c6405340bc minAccountLevelForBlockSubmissions reduced from 6 to 5 2022-02-05 11:33:28 +00:00
CalDescent
775e3c065e Invalidate ElectrumX transactions cache when switching accounts. 2022-02-05 10:23:25 +00:00
CalDescent
8937b3ec86 Don't allow duplicate transaction in the incoming transactions queue.
This should reduce database load slightly, as it won't have to check the same transaction multiple times in each batch.
2022-02-05 10:19:26 +00:00
CalDescent
3fbb86fded Added indexes, to make looking up name transactions by name around 5x faster. 2022-02-04 16:27:31 +00:00
CalDescent
0cf2f7f254 Missing import from last commit 2022-02-04 16:18:00 +00:00
CalDescent
9e571b87e8 Yet another rewrite of fetchAllTransactionsInvolvingName() - this time over 1000x faster since it doesn't involve joining the Transactions table. 2022-02-04 16:17:31 +00:00
CalDescent
23bafb6233 Removed unused methods 2022-02-04 14:00:44 +00:00
CalDescent
6dec65c5d9 Rewrite of fetchAllTransactionsInvolvingName() to avoid having to load all name transactions into memory. 2022-02-04 13:58:05 +00:00
CalDescent
4e59eb8958 Added unit test to simulate the false association between previous a UPDATE_NAME transaction, and the emoji name with a blank reducedName. 2022-02-04 12:51:22 +00:00
CalDescent
756d5e685a Added naming tests for blank new names 2022-02-04 12:50:05 +00:00
CalDescent
f52530b848 More thorough approach to fetchAllTransactionsInvolvingName(), to fix an issue found in unit testing. 2022-02-04 11:58:43 +00:00
CalDescent
c2bf37b878 Added transactionV5Timestamp featureTrigger to unit tests 2022-02-04 11:37:41 +00:00
CalDescent
98a2dd04b8 Fixed bug caused by improper handling of UPDATE_NAME transactions, similar to commit d16663f. 2022-02-04 10:28:26 +00:00
CalDescent
694ea689c8 Synchronize the loop and break out of it before fetching arbitrary data files. Hopeful fix for ConcurrentModificationException, and maybe a potential deadlock. 2022-02-03 21:14:41 +00:00
CalDescent
618aaaf243 Removed logs used for debugging only 2022-02-03 21:06:36 +00:00
CalDescent
9224ffbf73 Cache transaction list for 2 minutes, and synchronize, to prevent the balance and transactions APIs both requesting at once.
This ensures that only a single round of requests (per coin) is used for the wallettransactions and balance APIs. It also speeds up loading on subsequent requests. The 2 minute cache isn't much longer than the foreign block times, so shouldn't cause values to be too out of date.
2022-02-03 21:04:49 +00:00
CalDescent
892612c084 Calculate wallet balances from the transactions (ElectrumX) rather than using bitcoinj.
Hopeful fix for incorrect balances in wallets with large numbers of transactions. At the very least, this gives us control of the code that calculates the balance.
2022-02-03 20:22:25 +00:00
CalDescent
077165b807 Modified fetchArbitraryDataFileList() to support requesting only the missing hashes, but it is not yet used.
Once GetArbitraryDataFileListMessage is rolled out to the network we can uncomment this code. Needs testnet testing prior to that.
2022-02-03 20:01:44 +00:00
CalDescent
7994fc6407 Rework of onNetworkGetArbitraryDataFileListMessage() to support custom hashes to be optionally supplied.
Also simplified the existing logic, to make the code more readable.
2022-02-03 19:58:50 +00:00
CalDescent
d98df3e47d Added support for file hashes to optionally be included in GetArbitraryDataFileListMessage.
Needs testing on testnet, as the majority of nodes need to update before the hashes can be used.
2022-02-03 19:55:22 +00:00
CalDescent
1064b1a08b Removed duplicate metadata file checks. 2022-02-03 19:53:29 +00:00
CalDescent
0b7a7ed0f1 Added some more debug logging relating to file responses. 2022-02-03 19:52:28 +00:00
CalDescent
114b1aac76 Added arbitrary data file manager thread, which will ensure that all file list responses are tried until we receive the files.
Previously we would only try the first response and then discard the others due to being duplicates. They are now added to a queue and retried by the dedicated thread (up to the 60 second timeout).
2022-02-03 19:51:02 +00:00
CalDescent
6d06953a0e Increased RELAY_REQUEST_MAX_HOPS from 3 to 4, in an attempt to reach more peers. 2022-02-02 22:31:40 +00:00
CalDescent
0430fc8a47 Give up immediately after a synchronization if we are shutting down the synchronizer 2022-02-02 09:18:46 +00:00
CalDescent
7338f5f985 Attempt to acquire a blockchain lock (for up to 5 seconds) before shutting down the repository.
This should fix conflicts caused by the synchronizer and controller now being on separate threads. It may also reduce the chances of the database corrupting on shutdown, but this remains to be seen.
2022-02-02 09:17:24 +00:00