Compare commits

...

274 Commits

Author SHA1 Message Date
Proto
9263d74b75 Update TransactionValidity_fr.properties 2021-12-19 11:16:20 +01:00
Proto
9601bddc84 Update SysTray_fr.properties 2021-12-19 11:05:34 +01:00
Proto
e281e19052 Add files via upload 2021-12-19 11:02:00 +01:00
Proto
0238b78f45 Update ApiError_fr.properties 2021-12-19 10:57:57 +01:00
QuickMythril
0ccee4326d Added French translations
credit: stchoupi
2021-12-18 15:57:49 -05:00
CalDescent
391c3fe4c9 Added same functionality to GET /blocks/signature/{signature}
Also renamed query string parameter to "includeOnlineSignatures" to make it clearer.
2021-12-15 16:37:59 +00:00
CalDescent
3a7da9f13b Don't return online accounts signatures from GET /blocks/byheight/{height} unless requested using the includesignatures=true query string parameter.
This should fix issue where it would take up to 30 seconds to return for a recent block, and would consume masses of CPU due to having to base58 encode the online accounts signatures. Base58 is very slow and made this API endpoint almost unusable for recent blocks, due to them having untrimmed online accounts signatures.
2021-12-15 16:33:08 +00:00
CalDescent
e7fd803d19 Updated AdvancedInstaller project for v2.1.2 2021-12-11 22:16:43 +00:00
CalDescent
3b96747871 Bump version to 2.1.2 2021-12-11 21:50:54 +00:00
CalDescent
33088df07d Revert "Bump log4j-api version to 2.15.0"
This reverts commit 6a9904fd43.
2021-12-11 19:40:44 +00:00
CalDescent
a215714b6b Set log4j2.formatMsgNoLookups to true 2021-12-11 19:09:25 +00:00
CalDescent
6a9904fd43 Bump log4j-api version to 2.15.0
The main log4j version cannot be increased yet due to some incompatibilities with the Qortal code.
2021-12-10 17:54:11 +00:00
CalDescent
391d31759a Fixed small issue in GET /crosschain/trades and GET /crosschain/tradeoffers APIs where 0 was being treated as zero instead of unlimited.
In 2.1.1, unlimited results can be requested by removing the `limit` query string parameter completely, e.g:

http://127.0.0.1:12391/crosschain/trades?foreignBlockchain=LITECOIN&minimumTimestamp=1638835200000&reverse=false
2021-12-08 20:22:02 +00:00
CalDescent
ed2f2435d2 Updated AdvancedInstaller project for v2.1.1 2021-12-08 19:32:36 +00:00
CalDescent
6e6b2ccfa0 Bump version to 2.1.1 2021-12-08 18:39:30 +00:00
CalDescent
be9a73560d Added one more node 2021-12-08 17:23:42 +00:00
CalDescent
e82b5a4ecf Added 7 qortalnodes.live nodes to the default list. 2021-12-08 16:10:50 +00:00
CalDescent
a27d8ac828 Connect ACCTv3 trade bots to the ACCTv3 smart contracts 2021-12-08 12:52:22 +00:00
CalDescent
6267258189 Switch over to ACCTv3 for new listings 2021-12-08 12:22:26 +00:00
CalDescent
e7527f532e Added LitecoinACCTv3 and DogecoinACCTv3 to SupportedBlockchain 2021-12-08 12:21:52 +00:00
CalDescent
8b6e74d505 Added ACCTv3 trade bots. Identical to v1 and v2. 2021-12-08 12:18:22 +00:00
CalDescent
e6106c0c4e Added ACCTv3 tests, based on the same criteria as ACCTv1. Refunds are working as intended. 2021-12-08 12:10:36 +00:00
CalDescent
f52bafc014 Removed second "SLEEP_UNTIL_MESSAGE" function code call in LitecoinACCTv3 and DogecoinACCTv3.
It turns out that when you call SLEEP_UNTIL_MESSAGE, the AT resumes from that very same line on the next execution. The original code incorrectly assumed that it would execute from the restart position (SET_PCS).

So sleeping can be thought of as pausing one execution half way through, rather than ending it.

This caused a bug, because once the AT receives a transaction it wakes up and resumes from the SLEEP_UNTIL_MESSAGE line, which is after the refund check. Even when it loops back around again it lands on labelRedeemTxnLoop = codeByteBuffer.position(); which is again after the refund check.

For now, the simplest fix is to only sleep when listed. We could have alternatively moved the SLEEP_UNTIL_MESSAGE above GET_BLOCK_TIMESTAMP, but this would still require users to send a random transaction to the AT to trigger the refund. Given that the ATs are only "alive" for 30 minutes once the trade begins, it's simpler to just execute every block and therefore allow the refunds to happen automatically.
2021-12-08 12:08:06 +00:00
CalDescent
9e0630ea79 Added LitecoinACCTv3 and DogecoinACCTv3 - at the moment identical to ACCTv2 2021-12-08 11:55:25 +00:00
CalDescent
968bfb92d0 Fixed bugs in the GET /crosschain/tradeoffers API endpoint caused by the introduction of ACCTv2 2021-12-07 22:32:46 +00:00
CalDescent
284c9fcee2 Fixed bugs in the GET /crosschain/price API endpoint caused by the introduction of ACCTv2 2021-12-07 21:20:08 +00:00
CalDescent
5b0b939531 Fixed bugs in the GET /crosschain/trades API endpoint caused by the introduction of ACCTv2 2021-12-06 20:14:08 +00:00
CalDescent
dc52fd1dcf Only return OFFERING trades in GET /crosschain/tradeoffers 2021-12-06 19:37:30 +00:00
CalDescent
13f3aca838 Added GET /addresses/online/levels API endpoint to return the number of minters at each level 2021-12-04 19:28:28 +00:00
CalDescent
d9de27e6f2 Updated AdvancedInstaller project for v2.1.0 2021-11-24 19:39:37 +00:00
CalDescent
6930bf0200 Bump version to 2.1.0 2021-11-24 18:36:40 +00:00
CalDescent
199833bdd4 Fixed issue with GET /crosschain/trades API endpoint where the minimumTimestamp parameter returned inconsistent results. 2021-11-24 15:21:08 +00:00
CalDescent
0dcd2e6e93 Fixed bug in GET ​/crosschain​/price​/{blockchain} inverse price API endpoint when the "inverse" parameter is null. 2021-11-24 15:09:45 +00:00
CalDescent
098d7baa4d Bump version to 2.1.0-prerelease.0 2021-11-23 19:03:27 +00:00
CalDescent
59a57d3d28 Increased frequency of automatic repository maintenance attempts
repositoryMaintenanceMinInterval reduced from 7 to 3 days
repositoryMaintenanceMaxInterval reduced from 30 to 14 days
2021-11-23 18:47:27 +00:00
CalDescent
cce95e09de Default min level for block submissions increased to 3
This doesn't affect minting rewards; it is simply a means of reducing block candidates. There should be no noticeable difference other than hopefully less re-orgs. We can ultimately do a hard fork and increase Blockchain.minAccountLevelToMint but this allows us to test the approach in a lower risk way.
2021-11-23 18:45:35 +00:00
CalDescent
d018f11877 Log the initial height of the block archiver on startup 2021-11-21 09:12:32 +00:00
CalDescent
9502444bbc Prevent any kind of trading unless the blockchain is fully synced 2021-11-05 16:31:54 +00:00
CalDescent
3b914d4a7f Improved trade bot backups so that the current order being bought is included.
This should fix any key recovery issues if the node crashes or otherwise fails when buying an offer.
2021-11-03 19:27:56 +00:00
Scare Crowe
319d96f94e Add CWD bootstrap node 2021-11-03 11:45:47 +05:00
CalDescent
394ced9fb9 "Lists" feature is now generic.
This means that no additional structural code is required to add new lists. The only non-generic aspect are the API endpoints - it's best to keep these specific until we have a need for user-created lists.
2021-10-31 18:45:40 +00:00
CalDescent
314b6fc2f8 Include the initial peers when creating bootstraps 2021-10-27 08:46:52 +01:00
CalDescent
974df031a0 Added another bootstrap host 2021-10-26 21:41:22 +01:00
CalDescent
36d0292c6b Added "sleep until message" functionality to LTC ACCTv2. 2021-10-26 20:10:05 +01:00
CalDescent
7c16952c92 Added LitecoinACCTv2 and LitecoinACCTv2TradeBot 2021-10-26 19:56:33 +01:00
CalDescent
557807e3ba Initial attempt at adding "sleep until message" functionality to DOGE ACCTv2. 2021-10-26 10:59:23 +01:00
CalDescent
c1d5b2df29 Added DogecoinACCTv2 and DogecoinACCTv2TradeBot 2021-10-26 10:59:16 +01:00
CalDescent
3e0574e563 Added another missing block archive lookup relating to trade timestamps.
Note that this is unlikely to be the cause of some of the zero timestamps issue seen on a subset of nodes - there is still likely to be another problem that needs fixing.
2021-10-25 18:21:40 +01:00
CalDescent
69e557e70d Delete .sha256 file which was left lying around after running the bootstrap unit tests. 2021-10-25 18:20:58 +01:00
QuickMythril
49dd63af1e updated BTC electrum servers 2021-10-23 00:46:02 -04:00
CalDescent
18c6f0ccc3 Merge pull request #60 from Tocoolmh58/master 2021-10-22 18:05:38 +01:00
QuickMythril
55c50a4b5b add API option to return inverse price (#61) 2021-10-22 18:04:53 +01:00
CalDescent
1fbd5f7922 Fix for issue causing tradeTimestamp to report as 0 for trades in archived blocks. 2021-10-17 09:52:59 +01:00
CalDescent
bbb71083ef Updated AdvancedInstaller project for v2.0.0 2021-10-13 19:11:42 +01:00
CalDescent
e2134d76ec Bump version to 2.0.0 2021-10-13 18:16:50 +01:00
CalDescent
651372cd64 Bump version to 2.0.0-beta.7 2021-10-12 18:56:58 +01:00
CalDescent
581fe17b58 Added message to check the internet connection if the download cannot start. 2021-10-12 08:08:48 +01:00
CalDescent
af8608f302 Show full stack trace when bootstrapping fails for any reason. 2021-10-12 08:08:05 +01:00
CalDescent
290a19b6c6 Log the URL when downloading a bootstrap, to help with problem solving. 2021-10-12 08:01:47 +01:00
CalDescent
73eaa93be8 Added missing space in log entry. 2021-10-11 23:00:59 +01:00
CalDescent
7ab17383a6 Fix for NPE when serialized block bytes are unavailable. 2021-10-10 13:38:10 +01:00
CalDescent
b103c5b13f Bump version to 2.0.0-beta.6 2021-10-09 17:46:20 +01:00
CalDescent
b7d8a83017 Log "Downloading bootstrap..." as well as showing it in the splash screen. 2021-10-09 17:46:08 +01:00
CalDescent
e7bf4f455d Added missing repository.saveChanges() when reimporting data after creating a bootstrap. 2021-10-09 16:57:53 +01:00
CalDescent
a7f212c4f2 Create a .sha256 file to accompany each bootstrap
This can ultimately be validated after download, and can also be used to help coordinate updates on the various bootstrap hosts.
2021-10-09 16:57:19 +01:00
CalDescent
eb991c6026 Fixed issue causing bootstrap validation to be ignored before creation. 2021-10-09 16:29:40 +01:00
CalDescent
a78af8f248 Added SHA-256 file digest utility methods.
These read the file in small chunks, to reduce memory.
2021-10-09 16:22:21 +01:00
CalDescent
f34bdf0f58 Fixed issue causing minting accounts to be lost in some cases when auto bootstrapping. 2021-10-09 14:31:13 +01:00
CalDescent
ba272253a5 Bump version to 2.0.0-beta.5 2021-10-09 13:03:58 +01:00
CalDescent
9f488b7b77 Sleep for 5s before cleaning up temp path, in case this improves reliability on Windows. 2021-10-09 13:03:32 +01:00
CalDescent
3fb7df18a0 Delete temp directories at the beginning of the bootstrap process too, as Windows doesn't like deleting it at the end of the process. 2021-10-09 13:02:47 +01:00
CalDescent
00401080e0 Simplified cleanup process. Individual deletions aren't needed as they are all inside the main temp directory. 2021-10-09 13:02:00 +01:00
CalDescent
b265dc3bfb Don't log the complete stack trace for exceptions generated by bootstrap.checkRepositoryState(). The error message is enough in these cases. 2021-10-09 11:47:49 +01:00
CalDescent
63cabbe960 Log the full exception details and stack trace when creating bootstraps. 2021-10-09 11:39:08 +01:00
CalDescent
f6c1a7e6db Disregard exceptions in the bootstrap creation cleanup process because these don't affect the created bootstrap - instead just log the exception and full stack trace. 2021-10-09 11:38:13 +01:00
CalDescent
a3dcacade9 Now showing errors directly in the POST /bootstrap/create API response.
This avoids needing to check the log file each time.
2021-10-09 11:02:21 +01:00
CalDescent
17e65e422c Bump version to 2.0.0-beta.4 2021-10-08 19:11:25 +01:00
CalDescent
f53e2ffa47 Add initial peers on node startup if we don't have any in the repository.
This will be needed for future bootstraps, which don't contain any peers. It is also useful for those who have used the DELETE /peers/known API.
2021-10-08 19:10:02 +01:00
CalDescent
a1e4047695 Rework of bootstrap finalization process. 2021-10-08 18:06:41 +01:00
CalDescent
47ce884bbe Delete all known peers when creating a bootstrap 2021-10-08 15:24:10 +01:00
CalDescent
1b17c2613d Show "full node" or "top-only" in the "Downloading bootstrap" message. 2021-10-08 13:12:47 +01:00
CalDescent
dedc8d89c7 Handle case when attempting to load a block from the archive by reference, but the referenced block is in the main block repository, not the archive. This is the case with the genesis block.
Should fix issue where no block summaries were returned when syncing from block 1
2021-10-08 12:51:02 +01:00
CalDescent
d00fce86d2 Treat the genesis block as unpruned, as we leave this in the HSQLDB repository. 2021-10-08 12:42:23 +01:00
CalDescent
abab2d1cde Fixed issue preventing blocks from being served from the archive.
Now prefixing the byte buffer with the block height to mimic a cached block message.
2021-10-08 12:22:21 +01:00
CalDescent
33b715eb4e Merge branch 'networking' into v2.0-beta
# Conflicts:
#	src/main/java/org/qortal/settings/Settings.java
2021-10-07 18:53:49 +01:00
CalDescent
f6effbb6bb Removed unnecessary repository parameter from PruneManager.isBlockPruned() 2021-10-07 18:51:52 +01:00
CalDescent
dff9ec0704 Don't attempt to cache blocks from the archive, as they will never be recent 2021-10-07 18:50:59 +01:00
CalDescent
bfaf4c58e4 Make sure to check the archive when serving block summaries and signatures 2021-10-07 18:50:25 +01:00
CalDescent
ab7d24b637 Updated status wording 2021-10-07 09:02:28 +01:00
CalDescent
c256dae736 Ensure that the temp directory is always in the parent directory of the db folder. 2021-10-07 09:02:13 +01:00
CalDescent
5a55ef64c4 Bump version to 2.0.0-beta.3 2021-10-06 19:51:33 +01:00
CalDescent
045026431b Create a cleaner base directory path, without the "/./" 2021-10-06 19:50:32 +01:00
CalDescent
4dff91a0e5 Initial bootstrap import retry interval reduced from 5 minutes to 1 minute 2021-10-06 19:45:18 +01:00
CalDescent
7105872a37 Improved exception message 2021-10-06 19:44:30 +01:00
CalDescent
179bd8e018 Moved repository reopen to the finally {} block, so that we're never left without a repository instance. Should fix occasional "No repository available" error seen when retrying. 2021-10-06 19:44:04 +01:00
CalDescent
c82293342f Show full exception stack trace when a bootstrap import fails 2021-10-06 19:32:49 +01:00
CalDescent
81bf79e9d3 Bump version to 2.0.0-beta.2 2021-10-06 18:23:51 +01:00
CalDescent
8d6dffb3ff Added test for bootstrap random host selection. 2021-10-06 18:23:17 +01:00
CalDescent
2f6a8f793b Invert the colours in the splash screen 2021-10-06 18:22:52 +01:00
CalDescent
9bcd0bbfac Reduce log spam 2021-10-06 18:22:38 +01:00
CalDescent
cd359de7eb Scheduled maintenance now enabled by default, but uses a min and a max, to reduce the chances of multiple nodes running maintenance at the same time. Default to min: 7 days, max: 30 days. 2021-10-06 18:22:31 +01:00
Tocoolmh58
000f9ed459 Update ApiError_de.properties 2021-10-06 17:23:16 +02:00
Tocoolmh58
c5b2c0b4ec Create SysTray_de.properties 2021-10-06 17:01:52 +02:00
CalDescent
b7e9af100a Added scheduled repository maintenance feature. Currently disabled by default. 2021-10-06 08:52:27 +01:00
CalDescent
0d6409098f Added another bootstrap host 2021-10-05 22:08:18 +01:00
CalDescent
e07238ded8 Fixed variable name 2021-10-04 22:52:47 +01:00
CalDescent
27903f278d Add tmp folder to gitignore 2021-10-04 22:45:05 +01:00
CalDescent
ddf966d08c Show progress status when extracting files 2021-10-04 22:44:51 +01:00
CalDescent
65dca36ae1 Show progress status when downloading a bootstrap 2021-10-04 22:38:58 +01:00
CalDescent
289dae0780 Fixed issue causing the local repository data backup to be overwritten with an empty list. 2021-10-04 09:28:16 +01:00
CalDescent
71f802ef35 Exponentially backoff when bootstrapping fails, to reduce bandwidth
The retry interval starts at 5 minutes and doubles with each failure.
2021-10-04 09:25:23 +01:00
CalDescent
0135f25b9d Delete existing repository before extracting bootstrap
This limits the amount of additional space needed to the size of the compressed bootstrap (currently just under 4GB for full nodes, or 200MB for top-only nodes).
2021-10-04 09:15:54 +01:00
CalDescent
de3ebf664f Fixed issue with format specifier 2021-10-04 09:11:11 +01:00
CalDescent
850d879726 Use a "tmp" folder in the Qortal directory rather than a system generated temp folder.
This avoids the need to move files between partitions, and we also can't assume that the system partition has enough space to do the extraction.
2021-10-04 09:10:56 +01:00
CalDescent
5397e6c723 Bump version to 2.0.0-beta.1 2021-10-03 22:59:11 +01:00
CalDescent
889f6fc5fc Add a "testnet-" prefix in filenames when creating or importing bootstraps on testnet, so that the two databases can be kept separate. 2021-10-03 22:57:38 +01:00
CalDescent
41c2ed7c67 Fixed out of memory errors when copying AT states. 2021-10-03 22:51:15 +01:00
CalDescent
cdf47d4719 Reduce log spam. 2021-10-03 22:33:36 +01:00
CalDescent
210368bea0 Bump version to 2.0.0-beta.0 2021-10-03 19:43:28 +01:00
CalDescent
4f48751d0b Fixed issue caused when trying to update the splash frame status in a headless environment. 2021-10-03 19:43:10 +01:00
CalDescent
b6d3e82304 Update status when performing repository maintenance 2021-10-03 19:31:05 +01:00
CalDescent
3bb3528aa5 Merge branch 'master' into bootstrap 2021-10-03 18:44:13 +01:00
CalDescent
4f892835b8 Show maximum time estimations in archiving and pruning statuses 2021-10-03 18:41:47 +01:00
CalDescent
ac49221639 Show warning status on startup if the database is missing the AtStatesHeightIndex. 2021-10-03 18:34:21 +01:00
CalDescent
75ed5db3e4 Test multiple files when bulk archiving. 2021-10-03 17:33:02 +01:00
CalDescent
59c8e4e6a2 Fixed bug in earlier commit 2021-10-03 16:09:52 +01:00
CalDescent
52b322b756 Take a backup of local data before overwriting with a bootstrap.
Also moved the import phase to after the validation phase, so that the data returns after the bootstrap.
2021-10-03 16:09:40 +01:00
CalDescent
dc876d9c96 Force a bootstrap if the block archive isn't intact on launch
This allows the topOnly setting to be disabled and the node will automatically bootstrap to the archive version. A rebuild isn't attempted if bootstrapping is disabled, in order to reduce risk.
2021-10-03 16:00:11 +01:00
CalDescent
5b028428c4 Checkpoint immediately after starting/upgrading the repository
This should fix a longstanding issue where quitting the core before the first checkpoint (1-2 hours after first launch) causes the database to become corrupt.
2021-10-03 15:47:10 +01:00
CalDescent
f67a0469fc SplashFrame styling improvements 2021-10-03 15:33:53 +01:00
CalDescent
494cd0efff Added white background to splash frame - I think it looks nicer this way, and it may solve the X2Go issues too. 2021-10-03 15:27:19 +01:00
CalDescent
fc8e38e862 Show version number in the "Starting Qortal Core" status. 2021-10-03 15:22:21 +01:00
CalDescent
f09fb5a209 Run the bulk block archiver any time that it isn't reported to be up to date. This should allow it to re-run if the user quits the core before it completes. 2021-10-03 15:17:31 +01:00
CalDescent
b00c1c1575 Update splash frame statuses when reshaping, pruning, or building the block archive 2021-10-03 15:16:36 +01:00
CalDescent
7e5dd62a92 Show bootstrap statuses on splash frame. 2021-10-03 15:01:50 +01:00
CalDescent
35718f6215 Fixed issue with bootstrap retries. 2021-10-03 15:00:40 +01:00
CalDescent
a6d3891a95 Fixed bugs when importing bootstrap. 2021-10-03 09:48:14 +01:00
CalDescent
9591c4eb58 Added support for creating top-only bootstraps 2021-10-02 19:13:18 +01:00
CalDescent
8aaf720b0b Force archiveEnabled to false if we're in top only mode.
Most of the code handles this case anyway, but it's an easy place for bugs to be created. So it's safest to enforce it at the settings level.
2021-10-02 19:11:47 +01:00
CalDescent
63a35c97bc Fixed bug in path returned to POST /bootstrap/create API. 2021-10-02 19:10:23 +01:00
CalDescent
8eddaa3fac Small refactor to update wording. 2021-10-02 15:48:31 +01:00
CalDescent
1b3f37eb78 Delete the "archive" folder when in top-only mode
This allows a block archive node to be switched into top-only mode.
2021-10-02 15:37:06 +01:00
CalDescent
1f8fbfaa24 Missed a few topOnly references from the last commit. 2021-10-02 15:34:55 +01:00
CalDescent
ea92ccb4c1 "pruningEnabled" setting renamed to "topOnly"
Pruning is still a concept in the code, but since it relates to both archived and topOnly mode, it makes it cleaner to use "topOnly" to refer to the pruned db with no archive.
2021-10-02 15:13:23 +01:00
CalDescent
d25a77b633 Ignore bootstraps and other 7z archives in git. 2021-10-02 15:04:00 +01:00
CalDescent
51bb776e56 Select a random host when importing a bootstrap, and started adding support for multiple bootstrap types. 2021-10-02 15:03:31 +01:00
CalDescent
47b1b6daba Retry the entire bootstrap import process on failure, rather than just the download. 2021-10-02 14:43:26 +01:00
CalDescent
adeb654248 Rework of repository maintenance and backups
It will now attempt to wait until there are no other active transactions before starting, to avoid deadlocks. A timeout for this process is specified - generally 60 seconds - so that callers can give up or retry if something is holding a transaction open for too long. Right now we will give up in all places except for bootstrap creation, where it will keep retrying until successful.
2021-10-02 13:23:26 +01:00
CalDescent
c4d7335fdd Fixed more issues that could cause transactions to be held open. 2021-10-02 13:05:16 +01:00
CalDescent
ca7f42c409 Reduced unnecessary database queries in the block archiver. 2021-10-02 11:52:20 +01:00
CalDescent
ca02cd72ae Fixed issue in block archiver, which caused it to hold a transaction open for a very long time. This caused deadlocks when trying to create bootstraps or perform repository maintenance. 2021-10-02 11:51:53 +01:00
CalDescent
1ba542eb50 Simplified minting code in block archive tests. 2021-10-01 14:51:45 +01:00
CalDescent
53cd967541 Added defrag (repository maintenance) tests. 2021-10-01 14:51:28 +01:00
CalDescent
49749a0bc7 Added more precise checking of database states to the bulk pruning test. This highlighted a major bug in the bulk prune process whereby the recent AT states weren't being retained. 2021-10-01 11:03:56 +01:00
CalDescent
446f924380 Added bulk pruning test, which highlighted some bugs in both bulk and regular pruning. 2021-10-01 09:32:35 +01:00
CalDescent
5b231170cd Fixed small issues with block archive tests. 2021-10-01 08:10:28 +01:00
CalDescent
7375357b11 Added bootstrap tests
This involved adding a feature to the test suite in include the option of using a repository located on disk rather than in memory. Also moved the bootstrap compression/extraction working directories to temporary folders.
2021-10-01 07:44:33 +01:00
CalDescent
347d799d85 Reduce log spam. 2021-09-28 20:30:06 +01:00
CalDescent
0d17f02191 Pass a repository instance into the bulk archiving and pruning methods.
This is a better approach than opening a new session for each, and it makes it easier to write unit tests.
2021-09-28 20:29:53 +01:00
CalDescent
ce5bc80347 Increased threshold of BlockArchiveWriter.isArchiverUpToDate() from 90 to 95%.
In practice, the reading from a correctly archived chain with 550k blocks is currently around 99.5%, but it will be lower if starting with a chain that isn't fully synced.
2021-09-28 20:21:19 +01:00
CalDescent
0a4479fe9e Initial implementation of automatic bootstrapping
Currently supports block archive (i.e. full) bootstraps only. Still need to add support for top-only bootstraps.
2021-09-28 20:17:19 +01:00
CalDescent
de8e96cd75 Added Blockchain.validateAllBlocks() to check every block back to genesis.
This is extremely slow and shouldn't be needed in normal use cases. It currently checks that each block references the one before, but can ultimately be expanded to check more information about each block and its derived data.
2021-09-28 09:28:47 +01:00
CalDescent
e2a62f88a6 Modified repository backup and recovery to allow a custom filename to be specified. 2021-09-28 09:26:18 +01:00
CalDescent
8926d2a73c Rework of import/export process.
- Adds support for minting accounts as well as trade bot states
- Includes automatic import of both types on node startup, and automatic export on node shutdown
- Retains legacy trade bot states in a separate "TradeBotStatesArchive.json" file, whilst keeping the current active ones in "TradeBotStates.json". This prevents states being re-imported after they have been removed, but still keeps a copy of the data in case a key is ever needed.
- Uses indentation in the JSON files for easier readability.
2021-09-28 09:24:21 +01:00
CalDescent
114833cf8e Fixed NPE when attempting to lookup a block signature that doesn't exist. 2021-09-27 09:13:19 +01:00
CalDescent
32227436e0 Updated AdvancedInstaller project for v1.7.0 2021-09-27 08:43:44 +01:00
CalDescent
28ff5636af Bump version to 1.7.0 2021-09-26 22:20:21 +01:00
CalDescent
656896d16f Fixed issue causing base block prune/trim heights to not be updated on the final pass. 2021-09-24 21:36:05 +01:00
CalDescent
19bf8afece Fixed bug in pruning phase on node startup
This was causing very recent AT states to be deleted accidentally, because we weren't rebuilding the LatestATStates table before running the query. We should add unit tests to cover this process in case there are any other undiscovered problems.
2021-09-24 20:52:45 +01:00
CalDescent
841b6c4ddf Fixed another issue causing ATStatesHeightIndex to go missing after pruning. 2021-09-24 15:58:09 +01:00
CalDescent
4c171df848 Disable archiving and pruning if the AtStatesHeightIndex is missing, and log it so that the user knows they should bootstrap or resync. 2021-09-24 11:14:18 +01:00
CalDescent
1f79d88840 Fixed errors found in unit tests. 2021-09-24 09:35:36 +01:00
CalDescent
6ee7e9d731 Merge branch 'block-archive' of github.com:Qortal/qortal into block-archive
# Conflicts:
#	src/main/java/org/qortal/controller/Controller.java
2021-09-24 08:50:35 +01:00
CalDescent
4856223838 Fixed error in rebase. 2021-09-24 08:50:00 +01:00
CalDescent
74ea2a847d Added unit tests for trimming, pruning, and archiving. 2021-09-24 08:37:36 +01:00
CalDescent
9813dde3d9 Added importFromArchive() feature
This allows archived blocks to be imported back into HSQLDB in order to make them SQL-compatible again.
2021-09-24 08:37:36 +01:00
CalDescent
fea7b62b9c Fixed some bugs found in unit testing. 2021-09-24 08:37:36 +01:00
CalDescent
37e03bf2bb Removed BLOCK_LIMIT_REACHED result from the block archive writer.
This wasn't needed, and is now instead caught by the NOT_ENOUGH_BLOCKS result.
2021-09-24 08:37:36 +01:00
CalDescent
5656de79a2 Removed maxDuplicatedBlocksWhenArchiving setting as it's no longer needed. 2021-09-24 08:37:36 +01:00
CalDescent
70c6048cc1 Added block archive mode
This takes all trimmed blocks (which should now be all but the last 1450 or so) and moves them into flat files. Each file contains the serialized bytes of as many blocks that can fit within the file size target of 100MiB.

As a result, the HSQLDB size drops to less than 1GB, making it much faster and easier to maintain. It also significantly reduces the total size of each full node, because the data is stored in a highly optimized way.

HSQLDB then works similarly to the way it does in pruning mode - it holds all transactions, the latest state of every AT, as well as the full AT states data and hashes for the past 1450 blocks.

Each archive file contains headers and indexes in order to quickly locate blocks. When a peer requests a block that is within the archive, the serialized bytes are sent directly without the need to go via a BlockData object. Now that there are no slow queries or data serialization processes needed, it should greatly speed up the block serving.

The /block API endpoints have been modified in such a way that they will also check and retrieve blocks from the archive when needed.

A lightweight "BlockArchive" table is needed in HSQLDB to map block heights to signatures minters and timestamps. It made more sense to keep SQL support for these basic attributes of each block. These are located in a separate table from the full blocks, in order to create a clear distinction between HSQLDB blocks and archived blocks, and also to speed up query times in the Blocks table, which is the one we are using 99% of the time.

There is currently a restriction on the /admin/orphan API endpoint to prevent orphaning beyond the threshold of the block archive.
2021-09-24 08:37:36 +01:00
CalDescent
87595fd704 Synchronized LatestATStates, to make rebuildLatestAtStates() thread safe. 2021-09-24 08:37:15 +01:00
CalDescent
dc030a42bb Moved trimming and pruning classes into a single package (org.qortal.controller.repository) 2021-09-24 08:37:15 +01:00
CalDescent
89283ed179 Increased atStatesPruneBatchSize from 10 to 25. 2021-09-24 08:36:52 +01:00
CalDescent
64e8a05a9f Prune ATStatesData as well as the ATStates when switching to pruning mode. 2021-09-24 08:36:52 +01:00
CalDescent
676320586a Updated tests to use the renamed method. 2021-09-24 08:36:52 +01:00
CalDescent
734fa51806 Unified the code to build the LatestATStates table, as it's now used by more than one class.
Note - the rebuildLatestAtStates() must never be used by two different classes at the same time, or AT states could be incorrectly deleted. It is okay at the moment as we don't run the AT states trimmer and pruner in the same app session. However we should probably synchronize this method so that we don't accidentally call it from two places in the future.
2021-09-24 08:36:52 +01:00
CalDescent
f056ecc8d8 Added bulk pruning phase on node startup the first time that pruning mode is enabled.
When switching from a full node to a pruning node, we need to delete most of the database contents. If we do this entirely as a background process, it is very slow and can interfere with syncing. However, if we take the approach of transferring only the necessary rows to a new table and then deleting the original table, this makes the process much faster. It was taking several days to delete the AT states in the background, but only a couple of minutes to copy them to a new table.

The trade off is that we have to go through a form of "reshape" when starting the app for the first time after enabling pruning mode. But given that this is an opt-in mode, I don't think it will be a problem.

Once the pruning is complete, it automatically performs a CHECKPOINT DEFRAG in order to shrink the database file size down to a fraction of what it was before.

From this point, the original background process will run, but can be dialled right down so not to interfere with syncing.
2021-09-24 08:36:52 +01:00
CalDescent
1a722c1517 Break out of the AT pruning inner loops if we're stopping the app. 2021-09-24 08:36:51 +01:00
CalDescent
44607ba6a4 Fixed NPE introduced in earlier commit. 2021-09-24 08:36:51 +01:00
CalDescent
01d66212da Updated AT states pruner as it previously relied on blocks being present in the db to make decisions. As a side effect, this now prunes ATs up the the pruneBlockLimit too, rather than keeping the last 35 days or so. Will review this later but I don't think we will need the missing ones. 2021-09-24 08:36:51 +01:00
CalDescent
925e10b19b Rework of Blockchain.validate() to account for pruning mode. 2021-09-24 08:36:51 +01:00
CalDescent
1b4c75a76e Prune all blocks up until the blockPruneLimit
By default, this leaves only the last 1450 blocks in the database. Only applies when pruning mode is enabled.
2021-09-24 08:36:51 +01:00
CalDescent
3400e36ac4 Started work on pruning mode (top-only-sync)
Initially just deleting old and unused AT states, to get this table under control. I have had to delete them individually as the table can't handle complex queries due to its size.

Nodes in pruning mode will be unable to serve older blocks to peers.
2021-09-24 08:36:51 +01:00
CalDescent
78e2ae4f36 Allow trade bots in the REFUNDING state to be deleted, if the user chooses to via the DELETE /crosschain/tradebot API endpoint. 2021-09-23 17:53:57 +01:00
CalDescent
957944f6a5 Revert "original design"
This reverts commit 8c325f3a8a.
2021-09-23 17:44:57 +01:00
CalDescent
9eab500e2c atStatesMaxLifetime reduced from 14 days to 5 days
Whilst we would ultimately like to drop these to 24 hours only, for now we need some headroom to allow for orphaning in the event of a problem. Orphaning currently fails if there is no ATStatesData available (which is the case for trimmed blocks). This could ultimately be solved by retaining older unique states, which is essentially what the sleeping AT feature will do.
2021-09-23 08:42:15 +01:00
CalDescent
573f4675a1 Reduced online account signatures min and max lifetimes
onlineAccountSignaturesMinLifetime reduced from 720 hours to 12 hours
onlineAccountSignaturesMaxLifetime reduced from 888 hours to 24 hours

These were using up too much space in the database and so it makes sense to trim them more aggressively (assuming testing goes well). We will now stop validating online account signatures after 12 hours, which should be more than enough confirmations, and we will discard them after 24 hours.

Note: this will create some complexity once some of the network is running this code. It could cause out-of-sync nodes on old versions to start treating blocks as invalid from updated peers. It's likely not worth the complexity of a hard fork though, given that almost all nodes will be synced to the chain tip and will therefore be unaffected. And even with a hard fork, we'd still face this problem on out of date nodes.
2021-09-23 08:39:24 +01:00
CalDescent
e6bde3e1f4 Minimum order size set to 0.01 LTC, to avoid dust errors. 2021-09-23 08:36:55 +01:00
CalDescent
5869174021 Combined the three invalid name registration block patches into a single class. This should allow syncing from genesis again. 2021-09-23 08:28:51 +01:00
CalDescent
449761b6ca Rework of "Names" integrity check
Problem:
The "Names" table (the latest state of each name) drifts out of sync with the name-related transaction history on a subset of nodes for some unknown and seemingly difficult to find reason.

Solution:
Treat the "Names" table as a cache that can be rebuilt at any time. It now works like this:
- On node startup, rebuild the entire Names table by replaying the transaction history of all registered names. Includes registrations, updates, buys and sells.
- Add a "pre-process" stage to block/transaction processing. If the block contains a name related transaction, rebuild the Names cache for any names referenced by these transactions before validating anything.

The existing "integrity check" has been modified to just check basic attributes based on the latest transaction for a name. It will log if there are any inconsistencies found, but won't correct anything. This adds confidence that the rebuild has worked correctly.

There are also multiple unit tests to ensure that the rebuilds are coping with various different scenarios.
2021-09-22 08:15:23 +01:00
CalDescent
39d5ce19e2 Removed unused import. 2021-09-19 20:24:12 +01:00
CalDescent
3b156bc5c9 Added database integrity check for registered names
This ensures that all name-related transactions have resulted in correct entries in the Names table. A bug in the code has resulted in some nodes having missing data in their Names table. If this process finds a missing name, it will log it and add the name.

Missing names are added, but ownership issues are only logged. The known bug wasn't related to ownership, so the logging is only to alert us to any issues that may arise in the future.

In hindsight, the code could be rewritten to store all three transaction types in a single list, but this current approach has had a lot of testing, so it is best to stick with it for now.
2021-09-19 20:23:59 +01:00
CalDescent
a4f5124b61 Delete signatures from the invalidBlockSignatures array if we haven't seen them in over 1 hour
This is necessary because it's possible (in theory) for a block to be considered invalid due to an internal failure such as an SQLException. This gives them more chances to be considered valid again. 1 hour is more than enough time for the node to find an alternate valid chain if there is one available.
2021-09-19 19:46:48 +01:00
CalDescent
47a34c2f54 Validate blocks in syncToPeerChain() before orphaning
This prevents a valid block candidate being discarded in favour of an invalid one. We can't actually validate a block before orphaning (because it will fail due to various reasons such as already existing transactions, an existing block with the same height, etc) so we will instead just check the signature against the list of known invalid blocks.
2021-09-19 17:33:04 +01:00
CalDescent
8a7446fb40 Added "apiKeyDisabled" setting to bypass API key / loopback checking for those who need it.
This should only be used if all of the following conditions are true:
a) Your node is private and not shared with others
b) Port 12391 (API port) isn't forwarded
c) You have granted access to specific IP addresses using the "apiWhitelist" setting

The node will warn on startup if this setting is used without a sensible access control whitelist.
2021-09-19 09:34:48 +01:00
CalDescent
705e7d1cf1 Test name.register() and name.unregister() 2021-09-18 13:28:44 +01:00
CalDescent
44a90b4e12 Keep track of invalid block signatures and avoid peers that return them
Until now, a high weight invalid block can cause other valid, lower weight alternatives to be discarded. The solution to this problem is to track invalid blocks and quickly avoid them once discovered. This gives other valid alternative blocks the opportunity to become part of a valid chain, where they would otherwise have been discarded.

As with the block minter update, this will cause a fork when the highest weight block candidate is invalid. But it is likely that the fork would be short lived, assuming that the majority of nodes pick the valid chain.
2021-09-18 10:58:05 +01:00
CalDescent
54e5a65cf0 Allow an alternative block to be minted if the chain stalls due to an invalid block
If it has been more than 10 minutes since receiving the last valid block, but we have had at least one invalid block since then, this is indicative of a stuck chain due to no valid block candidates. In this case, we want to allow the block minter to mint an alternative candidate so that the chain can continue.

This would create a fork at the point of the invalid block, in which two chains (valid an invalid) would diverge. The valid chain could never rejoin the invalid one, however it's likely that the invalid chain would be discarded in favour of the valid one shortly after, on the assumption that the majority of nodes would have picked the valid one.
2021-09-18 10:41:58 +01:00
CalDescent
06a2c380bd Updated and added some naming tests. 2021-09-17 09:34:10 +01:00
CalDescent
33ac1fed2a Revert "Treat a REGISTER_NAME transaction as an UPDATE_NAME if the creator matches."
This reverts commit b800fb5846.
2021-09-16 19:27:17 +01:00
CalDescent
cc65a7cd11 Fixed bug which prevented the "reduced name" from being updated in UPDATE_NAME transactions.
Updating a name was incorrectly leaving the existing "reduced name" intact. Thanks to Qortal user @MyBestBet for reporting this bug.
2021-09-14 20:38:20 +01:00
CalDescent
d600a54034 Modified name update tests to check the reduced name. 2021-09-14 20:34:42 +01:00
CalDescent
ba06225b01 Merge branch 'master' into block-archive 2021-09-12 10:17:11 +01:00
CalDescent
ce60ab8e00 Updated naming unit tests
- Use the "{\"age\":30}" data to make the tests more similar to some real world data.
- Added tests to ensure that registering and orphaning works as expected.
2021-09-12 10:16:07 +01:00
CalDescent
14f6fd19ef Added unit tests for trimming, pruning, and archiving. 2021-09-12 10:13:52 +01:00
CalDescent
1d8351f921 Added importFromArchive() feature
This allows archived blocks to be imported back into HSQLDB in order to make them SQL-compatible again.
2021-09-12 10:10:25 +01:00
CalDescent
6a55b052f5 Fixed some bugs found in unit testing. 2021-09-12 09:57:12 +01:00
CalDescent
2a36b83dea Removed BLOCK_LIMIT_REACHED result from the block archive writer.
This wasn't needed, and is now instead caught by the NOT_ENOUGH_BLOCKS result.
2021-09-12 09:55:49 +01:00
CalDescent
14acc4feb9 Removed maxDuplicatedBlocksWhenArchiving setting as it's no longer needed. 2021-09-12 09:52:28 +01:00
CalDescent
0657ca2969 atStatesMaxLifetime increased to 5 days
For now, we need some headroom to allow for orphaning in the event of a problem. Orphaning currently fails if there is no ATStatesData available (which is the case for trimmed blocks). This could ultimately be solved by retaining older unique states.
2021-09-09 17:46:19 +01:00
CalDescent
e90c3a78d1 Updated default "data" field text in the API documentation, to match the value the UI uses. 2021-09-09 15:12:28 +01:00
CalDescent
63c9bc5c1c Revert "Workaround for block 535658 problem"
This reverts commit 278201e87c.
2021-09-09 12:55:21 +01:00
CalDescent
a6bbc81962 Revert "Merge pull request #58 from QuickMythril/536140-fix"
This reverts commit 6d1f7b36a7, reversing
changes made to 6b74ef77e6.

# Conflicts:
#	src/main/java/org/qortal/block/Block536140.java
2021-09-09 12:55:08 +01:00
CalDescent
b800fb5846 Treat a REGISTER_NAME transaction as an UPDATE_NAME if the creator matches.
Whilst not ideal, this is necessary to prevent the chain from getting stuck on future blocks due to duplicate name registrations. See Block535658.java for full details on this problem - this is simply a "catch-all" implementation of that class in order to futureproof this fix.

There is still a database inconsistency to be solved, as some nodes are failing to add a registered name to their Names table the first time around, but this will take some time. Once fixed, this commit could potentially be reverted.

Also added unit tests for both scenarios (same and different creator).

TLDR: this allows all past and future invalid blocks caused by NAME_ALREADY_REGISTERED (by the same creator) to now be valid.
2021-09-09 12:54:01 +01:00
CalDescent
172a629da3 Added comments 2021-09-05 23:32:11 +01:00
CalDescent
6d1f7b36a7 Merge pull request #58 from QuickMythril/536140-fix
Block 536140 fix (same situation as block 535658)
2021-09-05 23:16:08 +01:00
QuickMythril
673ee4aeed Update Block.java 2021-09-05 18:07:11 -04:00
QuickMythril
25b787f6f2 Add files via upload 2021-09-05 18:06:32 -04:00
CalDescent
6b74ef77e6 Increased log level of invalid transaction message. 2021-09-05 21:25:38 +01:00
CalDescent
278201e87c Workaround for block 535658 problem 2021-09-05 21:24:02 +01:00
CalDescent
703cdfe174 Added block archive mode
This takes all trimmed blocks (which should now be all but the last 1450 or so) and moves them into flat files. Each file contains the serialized bytes of as many blocks that can fit within the file size target of 100MiB.

As a result, the HSQLDB size drops to less than 1GB, making it much faster and easier to maintain. It also significantly reduces the total size of each full node, because the data is stored in a highly optimized way.

HSQLDB then works similarly to the way it does in pruning mode - it holds all transactions, the latest state of every AT, as well as the full AT states data and hashes for the past 1450 blocks.

Each archive file contains headers and indexes in order to quickly locate blocks. When a peer requests a block that is within the archive, the serialized bytes are sent directly without the need to go via a BlockData object. Now that there are no slow queries or data serialization processes needed, it should greatly speed up the block serving.

The /block API endpoints have been modified in such a way that they will also check and retrieve blocks from the archive when needed.

A lightweight "BlockArchive" table is needed in HSQLDB to map block heights to signatures minters and timestamps. It made more sense to keep SQL support for these basic attributes of each block. These are located in a separate table from the full blocks, in order to create a clear distinction between HSQLDB blocks and archived blocks, and also to speed up query times in the Blocks table, which is the one we are using 99% of the time.

There is currently a restriction on the /admin/orphan API endpoint to prevent orphaning beyond the threshold of the block archive.
2021-09-04 19:40:51 +01:00
CalDescent
02988989ad Reduced online account signatures min and max lifetimes
onlineAccountSignaturesMinLifetime reduced from 720 hours to 12 hours
onlineAccountSignaturesMaxLifetime reduced from 888 hours to 24 hours

These were using up too much space in the database and so it makes sense to trim them more aggressively (assuming testing goes well). We will now stop validating online account signatures after 12 hours, which should be more than enough confirmations, and we will discard them after 24 hours.

Note: this will create some complexity once some of the network is running this code. It could cause out-of-sync nodes on old versions to start treating blocks as invalid from updated peers. It's likely not worth the complexity of a hard fork though, given that almost all nodes will be synced to the chain tip and will therefore be unaffected. And even with a hard fork, we'd still face this problem on out of date nodes.
2021-09-03 10:11:02 +01:00
CalDescent
25c17d3704 atStatesMaxLifetime reduced from 14 days to 24 hours 2021-09-03 10:04:04 +01:00
CalDescent
9973fe4326 Synchronized LatestATStates, to make rebuildLatestAtStates() thread safe. 2021-08-28 11:00:49 +01:00
CalDescent
2479f2d65d Moved trimming and pruning classes into a single package (org.qortal.controller.repository) 2021-08-27 09:45:56 +01:00
CalDescent
9056cb7026 Increased atStatesPruneBatchSize from 10 to 25. 2021-08-27 09:45:56 +01:00
CalDescent
cd9d9b31ef Prune ATStatesData as well as the ATStates when switching to pruning mode. 2021-08-27 09:45:56 +01:00
CalDescent
ff841c28e3 Updated tests to use the renamed method. 2021-08-27 09:45:56 +01:00
CalDescent
ca1379d9f8 Unified the code to build the LatestATStates table, as it's now used by more than one class.
Note - the rebuildLatestAtStates() must never be used by two different classes at the same time, or AT states could be incorrectly deleted. It is okay at the moment as we don't run the AT states trimmer and pruner in the same app session. However we should probably synchronize this method so that we don't accidentally call it from two places in the future.
2021-08-27 09:45:56 +01:00
CalDescent
5127f94423 Added bulk pruning phase on node startup the first time that pruning mode is enabled.
When switching from a full node to a pruning node, we need to delete most of the database contents. If we do this entirely as a background process, it is very slow and can interfere with syncing. However, if we take the approach of transferring only the necessary rows to a new table and then deleting the original table, this makes the process much faster. It was taking several days to delete the AT states in the background, but only a couple of minutes to copy them to a new table.

The trade off is that we have to go through a form of "reshape" when starting the app for the first time after enabling pruning mode. But given that this is an opt-in mode, I don't think it will be a problem.

Once the pruning is complete, it automatically performs a CHECKPOINT DEFRAG in order to shrink the database file size down to a fraction of what it was before.

From this point, the original background process will run, but can be dialled right down so not to interfere with syncing.
2021-08-27 09:45:56 +01:00
CalDescent
f5910ab950 Break out of the AT pruning inner loops if we're stopping the app. 2021-08-27 09:45:56 +01:00
CalDescent
22efaccd4a Fixed NPE introduced in earlier commit. 2021-08-27 09:45:56 +01:00
CalDescent
c8466a2e7a Updated AT states pruner as it previously relied on blocks being present in the db to make decisions. As a side effect, this now prunes ATs up the the pruneBlockLimit too, rather than keeping the last 35 days or so. Will review this later but I don't think we will need the missing ones. 2021-08-27 09:45:56 +01:00
CalDescent
209a9fa8c3 Rework of Blockchain.validate() to account for pruning mode. 2021-08-27 09:45:56 +01:00
CalDescent
bc1af12655 Prune all blocks up until the blockPruneLimit
By default, this leaves only the last 1450 blocks in the database. Only applies when pruning mode is enabled.
2021-08-27 09:45:55 +01:00
CalDescent
e7e4cb7579 Started work on pruning mode (top-only-sync)
Initially just deleting old and unused AT states, to get this table under control. I have had to delete them individually as the table can't handle complex queries due to its size.

Nodes in pruning mode will be unable to serve older blocks to peers.
2021-08-27 09:45:55 +01:00
CalDescent
1b39db664c Added missing ATStatesHeightIndex to the reshape code.
This was accidentally missed out of the original code. Some pre-updated nodes on the network will be missing this index, but we can use the upcoming "auto-bootstrap" feature to get those back.
2021-08-27 08:54:46 +01:00
CalDescent
b4f980b349 Restrict lists API endpoints to local/apiKey requests only. 2021-08-12 19:52:49 +01:00
CalDescent
673f23b6a0 Improvement to commit f71516f
Now only skipping the HTLC redemption if the AT is finished and the balance has been redeemed by the buyer. This allows HTLCs to be refunded for ATs that have been refunded or cancelled.
2021-08-12 08:29:52 +01:00
QuickMythril
8c325f3a8a original design 2021-08-11 18:18:10 -04:00
CalDescent
f71516f36f Skip finished ATs in the refund API endpoints. 2021-08-11 21:26:29 +01:00
CalDescent
1752386a6c Fixed logging errors in previous commit. 2021-08-11 20:33:54 +01:00
CalDescent
112675c782 Better handling of RPC errors.
Previously, if an error was returned from an Electrum server (such as "server busy") it would throw a NetworkException that would be caught outside of the server loop and cause the entire request to fail.

Instead of throwing an exception, I am now logging the error and returning null, in the same way we do for IOException and NoSuchElementException further up in the same method.

This allows the caller - most likely connectedRpc() - to move on to the next server in the list and try again.

This should fix an issue seen where a "server busy" response from a single server was essentially breaking our implementation, as we would give up altogether instead of trying another server.
2021-08-11 19:22:53 +01:00
CalDescent
3b6ba7641d Updated icon for Qortal.exe 2021-08-10 19:35:03 +01:00
CalDescent
477a35a685 Fixed response schema for GET /lists/blacklist/addresses endpoint 2021-08-10 08:43:47 +01:00
CalDescent
2a0a39a95a Avoid creation of lists directory until the first item is added to a list. 2021-08-09 23:35:32 +01:00
CalDescent
dfc77db51d Merge pull request #57 from ScythianQortal/translations
Added Hungarian translations and reorganised existing translations
2021-08-09 10:05:13 +01:00
CalDescent
c9596fd8c4 Catch exceptions thrown during GUI initialization.
This is a workaround for an UnsupportedOperationException thrown when using X2Go, due to PERPIXEL_TRANSLUCENT translucency being unsupported in splashDialog.setBackground(). We could choose to use a different version of the splash screen with an opaque background in these cases, but it is low priority.
2021-08-09 10:02:24 +01:00
CalDescent
78373f3746 HTLC redeem/refund APIs switched from GET to POST. 2021-08-08 10:29:15 +01:00
CalDescent
ebc3db8aed Default file path for repository data imports set to "qortal-backup/TradeBotStates.json". This allows the trade bot backup to be imported in a single click, and can now be potentially added as a button in the UI. 2021-08-08 10:20:44 +01:00
CalDescent
756601c1ce Initialize to an empty list.
This fixes various bugs caused by the list being null when no blacklist JSON file was available.
2021-08-08 08:41:13 +01:00
CalDescent
8bb5077e76 Catch occasional NPE when setting tray icon. 2021-08-08 08:29:29 +01:00
CalDescent
5b85f01427 Added defensiveness to list management methods in ResourceList.java 2021-08-08 08:28:34 +01:00
CalDescent
a7d594e566 Log the AT states reshape progress, as it seems to be taking a very long time. 2021-08-07 19:18:20 +01:00
CalDescent
481e6671c2 Added GET /lists/blacklist/addresses API endpoint
This returns a JSON array containing the blacklisted addresses.
2021-08-07 16:27:16 +01:00
Scythian
b890e02a6a Added new TransactionValidity keys
Added ADDRESS_ABOVE_RATE_LIMIT and DUPLICATE_MESSAGE ValidationResults to localeLang translation keys
2021-08-07 15:09:48 +01:00
Scythian
4772840b4c Reorganised translations
Updated the "localeLang" files with new keys and removed old unused keys for English, German, Dutch, Italian, Finnish, Hungarian, Russian and Chinese translations
2021-08-07 14:05:10 +01:00
Scythian
76ec3473d6 Updated TransactionValidity keys
Added ADDRESS_IN_BLACKLIST ValidationResult to TransactionValidity translation keys
2021-08-07 10:47:48 +01:00
Scythian
0b53de1bb6 Added Hungarian translations 2021-08-06 19:28:56 +01:00
Scythian
746c68c9f6 Reorganised translations
Added new keys and removed old unused keys
Localised the "Build version" string in the SysTray
2021-08-06 19:27:28 +01:00
CalDescent
c3ff9e49e8 Merge pull request #40 from szisti/fixedNetwork
Support for configuration based fixed network
2021-05-29 09:51:56 +01:00
Istvan Szabo
d52875aa8f Added logs to intentional disconnects 2021-05-28 16:06:27 +01:00
Istvan Szabo
9027cd290c Filter out on demand connections when using fixed network 2021-05-28 14:47:30 +01:00
Istvan Szabo
58a7203ede Support for configuration based fixed network 2021-05-28 14:47:30 +01:00
176 changed files with 18280 additions and 1275 deletions

2
.gitignore vendored
View File

@@ -26,3 +26,5 @@
/run.pid
/run.log
/WindowsInstaller/Install Files/qortal.jar
/*.7z
/tmp

View File

@@ -17,10 +17,10 @@
<ROW Property="Manufacturer" Value="Qortal"/>
<ROW Property="MsiLogging" MultiBuildValue="DefaultBuild:vp"/>
<ROW Property="NTP_GOOD" Value="false"/>
<ROW Property="ProductCode" Value="1033:{482E9390-1005-42FD-9F3F-E160E0E6FB19} 1049:{8FE09AC2-814B-42FC-9FCE-53D45A396529} 2052:{4FABD326-8345-438B-82B8-66C2DC3676E6} 2057:{7ECFFF43-DEC7-4B7F-BC88-260A10AF132A} " Type="16"/>
<ROW Property="ProductCode" Value="1033:{D45BD9F0-1D22-46A2-9990-8BB3598D8AD3} 1049:{21BCBDEC-4BF2-41D6-9426-E906042FD61F} 2052:{95F43BE3-58B4-4422-BCCF-D77F4BBFD7A4} 2057:{4C4D611E-3AF5-4216-952A-CA9E5EC32C19} " Type="16"/>
<ROW Property="ProductLanguage" Value="2057"/>
<ROW Property="ProductName" Value="Qortal"/>
<ROW Property="ProductVersion" Value="1.6.0" Type="32"/>
<ROW Property="ProductVersion" Value="2.1.2" Type="32"/>
<ROW Property="RECONFIG_NTP" Value="true"/>
<ROW Property="REMOVE_BLOCKCHAIN" Value="YES" Type="4"/>
<ROW Property="REPAIR_BLOCKCHAIN" Value="YES" Type="4"/>
@@ -212,7 +212,7 @@
<ROW Component="ADDITIONAL_LICENSE_INFO_71" ComponentId="{12A3ADBE-BB7A-496C-8869-410681E6232F}" Directory_="jdk.zipfs_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_71" Type="0"/>
<ROW Component="ADDITIONAL_LICENSE_INFO_8" ComponentId="{D53AD95E-CF96-4999-80FC-5812277A7456}" Directory_="java.naming_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_8" Type="0"/>
<ROW Component="ADDITIONAL_LICENSE_INFO_9" ComponentId="{6B7EA9B0-5D17-47A8-B78C-FACE86D15E01}" Directory_="java.net.http_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_9" Type="0"/>
<ROW Component="AI_CustomARPName" ComponentId="{7941AD6C-7C09-48E7-93ED-0340E0F52EC0}" Directory_="APPDIR" Attributes="260" KeyPath="DisplayName" Options="1"/>
<ROW Component="AI_CustomARPName" ComponentId="{5109CB00-40C0-499E-875B-9364F2075410}" Directory_="APPDIR" Attributes="260" KeyPath="DisplayName" Options="1"/>
<ROW Component="AI_ExePath" ComponentId="{3644948D-AE0B-41BB-9FAF-A79E70490A08}" Directory_="APPDIR" Attributes="260" KeyPath="AI_ExePath"/>
<ROW Component="APPDIR" ComponentId="{680DFDDE-3FB4-47A5-8FF5-934F576C6F91}" Directory_="APPDIR" Attributes="0"/>
<ROW Component="AccessBridgeCallbacks.h" ComponentId="{288055D1-1062-47A3-AA44-5601B4E38AED}" Directory_="bridge_Dir" Attributes="0" KeyPath="AccessBridgeCallbacks.h" Type="0"/>

BIN
WindowsInstaller/qortal.ico Executable file → Normal file

Binary file not shown.

Before

Width:  |  Height:  |  Size: 250 KiB

After

Width:  |  Height:  |  Size: 42 KiB

20
pom.xml
View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.qortal</groupId>
<artifactId>qortal</artifactId>
<version>1.6.0</version>
<version>2.1.2</version>
<packaging>jar</packaging>
<properties>
<skipTests>true</skipTests>
@@ -14,6 +14,9 @@
<ciyam-at.version>1.3.8</ciyam-at.version>
<commons-net.version>3.6</commons-net.version>
<commons-text.version>1.8</commons-text.version>
<commons-io.version>2.6</commons-io.version>
<commons-compress.version>1.21</commons-compress.version>
<xz.version>1.9</xz.version>
<dagger.version>1.2.2</dagger.version>
<guava.version>28.1-jre</guava.version>
<hsqldb.version>2.5.1</hsqldb.version>
@@ -449,6 +452,21 @@
<artifactId>commons-text</artifactId>
<version>${commons-text.version}</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>${commons-io.version}</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>${commons-compress.version}</version>
</dependency>
<dependency>
<groupId>org.tukaani</groupId>
<artifactId>xz</artifactId>
<version>${xz.version}</version>
</dependency>
<!-- For bitset/bitmap compression -->
<dependency>
<groupId>io.druid</groupId>

View File

@@ -1,6 +1,7 @@
package org.qortal;
import java.security.Security;
import java.util.concurrent.TimeoutException;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -57,10 +58,10 @@ public class RepositoryMaintenance {
LOGGER.info("Starting repository periodic maintenance. This can take a while...");
try (final Repository repository = RepositoryManager.getRepository()) {
repository.performPeriodicMaintenance();
repository.performPeriodicMaintenance(null);
LOGGER.info("Repository periodic maintenance completed");
} catch (DataException e) {
} catch (DataException | TimeoutException e) {
LOGGER.error("Repository periodic maintenance failed", e);
}

View File

@@ -16,4 +16,8 @@ public enum ApiExceptionFactory {
return createException(request, apiError, null);
}
public ApiException createCustomException(HttpServletRequest request, ApiError apiError, String message) {
return new ApiException(apiError.getStatus(), apiError.getCode(), message, null);
}
}

View File

@@ -14,6 +14,8 @@ import java.security.SecureRandom;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.eclipse.jetty.http.HttpVersion;
import org.eclipse.jetty.rewrite.handler.RedirectPatternRule;
import org.eclipse.jetty.rewrite.handler.RewriteHandler;
@@ -50,6 +52,8 @@ import org.qortal.settings.Settings;
public class ApiService {
private static final Logger LOGGER = LogManager.getLogger(ApiService.class);
private static ApiService instance;
private final ResourceConfig config;
@@ -203,6 +207,9 @@ public class ApiService {
context.addServlet(TradeBotWebSocket.class, "/websockets/crosschain/tradebot");
context.addServlet(PresenceWebSocket.class, "/websockets/presence");
// Warn about API security if needed
this.checkApiSecurity();
// Start server
this.server.start();
} catch (Exception e) {
@@ -222,4 +229,23 @@ public class ApiService {
this.server = null;
}
private void checkApiSecurity() {
// Warn about API security if needed
boolean allConnectionsAllowed = false;
if (Settings.getInstance().isApiKeyDisabled()) {
for (String pattern : Settings.getInstance().getApiWhitelist()) {
if (pattern.startsWith("0.0.0.0/") || pattern.startsWith("::/") || pattern.endsWith("/0")) {
allConnectionsAllowed = true;
}
}
if (allConnectionsAllowed) {
LOGGER.warn("Warning: API key validation is currently disabled, and the API whitelist " +
"is allowing all connections. This can be a security risk.");
LOGGER.warn("To fix, set the apiKeyDisabled setting to false, or allow only specific local " +
"IP addresses using the apiWhitelist setting.");
}
}
}
}

View File

@@ -12,6 +12,11 @@ public abstract class Security {
public static final String API_KEY_HEADER = "X-API-KEY";
public static void checkApiCallAllowed(HttpServletRequest request) {
// If API key checking has been disabled, we will allow the request in all cases
boolean isApiKeyDisabled = Settings.getInstance().isApiKeyDisabled();
if (isApiKeyDisabled)
return;
String expectedApiKey = Settings.getInstance().getApiKey();
String passedApiKey = request.getHeader(API_KEY_HEADER);

View File

@@ -7,12 +7,12 @@ import javax.xml.bind.annotation.XmlAccessorType;
import java.util.List;
@XmlAccessorType(XmlAccessType.FIELD)
public class AddressListRequest {
public class ListRequest {
@Schema(description = "A list of addresses")
public List<String> addresses;
@Schema(description = "A list of items")
public List<String> items;
public AddressListRequest() {
public ListRequest() {
}
}

View File

@@ -12,6 +12,7 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
@@ -38,6 +39,7 @@ import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountData;
import org.qortal.data.account.RewardShareData;
import org.qortal.data.network.OnlineAccountData;
import org.qortal.data.network.OnlineAccountLevel;
import org.qortal.data.transaction.PublicizeTransactionData;
import org.qortal.data.transaction.RewardShareTransactionData;
import org.qortal.data.transaction.TransactionData;
@@ -180,6 +182,66 @@ public class AddressesResource {
}
}
@GET
@Path("/online/levels")
@Operation(
summary = "Return currently 'online' accounts counts, grouped by level",
responses = {
@ApiResponse(
description = "online accounts",
content = @Content(mediaType = MediaType.APPLICATION_JSON, array = @ArraySchema(schema = @Schema(implementation = ApiOnlineAccount.class)))
)
}
)
@ApiErrors({ApiError.PUBLIC_KEY_NOT_FOUND, ApiError.REPOSITORY_ISSUE})
public List<OnlineAccountLevel> getOnlineAccountsByLevel() {
List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
try (final Repository repository = RepositoryManager.getRepository()) {
List<OnlineAccountLevel> onlineAccountLevels = new ArrayList<>();
for (OnlineAccountData onlineAccountData : onlineAccounts) {
try {
final int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, onlineAccountData.getPublicKey());
OnlineAccountLevel onlineAccountLevel = onlineAccountLevels.stream()
.filter(a -> a.getLevel() == minterLevel)
.findFirst().orElse(null);
// Note: I don't think we can use the level as the List index here because there will be gaps.
// So we are forced to manually look up the existing item each time.
// There's probably a nice shorthand java way of doing this, but this approach gets the same result.
if (onlineAccountLevel == null) {
// No entry exists for this level yet, so create one
onlineAccountLevel = new OnlineAccountLevel(minterLevel, 1);
onlineAccountLevels.add(onlineAccountLevel);
}
else {
// Already exists - so increment the count
int existingCount = onlineAccountLevel.getCount();
onlineAccountLevel.setCount(++existingCount);
// Then replace the existing item
int index = onlineAccountLevels.indexOf(onlineAccountLevel);
onlineAccountLevels.set(index, onlineAccountLevel);
}
} catch (DataException e) {
continue;
}
}
// Sort by level
onlineAccountLevels.sort(Comparator.comparingInt(OnlineAccountLevel::getLevel));
return onlineAccountLevels;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@GET
@Path("/balance/{address}")
@Operation(

View File

@@ -22,6 +22,7 @@ import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.locks.ReentrantLock;
import java.util.stream.Collectors;
@@ -35,6 +36,7 @@ import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.appender.RollingFileAppender;
import org.qortal.account.Account;
@@ -67,6 +69,8 @@ import com.google.common.collect.Lists;
@Tag(name = "Admin")
public class AdminResource {
private static final Logger LOGGER = LogManager.getLogger(AdminResource.class);
private static final int MAX_LOG_LINES = 500;
@Context
@@ -459,6 +463,23 @@ public class AdminResource {
if (targetHeight <= 0 || targetHeight > Controller.getInstance().getChainHeight())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_HEIGHT);
// Make sure we're not orphaning as far back as the archived blocks
// FUTURE: we could support this by first importing earlier blocks from the archive
if (Settings.getInstance().isTopOnly() ||
Settings.getInstance().isArchiveEnabled()) {
try (final Repository repository = RepositoryManager.getRepository()) {
// Find the first unarchived block
int oldestBlock = repository.getBlockArchiveRepository().getBlockArchiveHeight();
// Add some extra blocks just in case we're currently archiving/pruning
oldestBlock += 100;
if (targetHeight <= oldestBlock) {
LOGGER.info("Unable to orphan beyond block {} because it is archived", oldestBlock);
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_HEIGHT);
}
}
}
if (BlockChain.orphan(targetHeight))
return "true";
else
@@ -553,13 +574,13 @@ public class AdminResource {
@Path("/repository/data")
@Operation(
summary = "Import data into repository.",
description = "Imports data from file on local machine. Filename is forced to 'import.json' if apiKey is not set.",
description = "Imports data from file on local machine. Filename is forced to 'qortal-backup/TradeBotStates.json' if apiKey is not set.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string", example = "MintingAccounts.script"
type = "string", example = "qortal-backup/TradeBotStates.json"
)
)
),
@@ -577,7 +598,7 @@ public class AdminResource {
// Hard-coded because it's too dangerous to allow user-supplied filenames in weaker security contexts
if (Settings.getInstance().getApiKey() == null)
filename = "import.json";
filename = "qortal-backup/TradeBotStates.json";
try (final Repository repository = RepositoryManager.getRepository()) {
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
@@ -589,6 +610,10 @@ public class AdminResource {
repository.saveChanges();
return "true";
} catch (IOException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA, e);
} finally {
blockchainLock.unlock();
}
@@ -644,14 +669,16 @@ public class AdminResource {
blockchainLock.lockInterruptibly();
try {
repository.backup(true);
// Timeout if the database isn't ready for backing up after 60 seconds
long timeout = 60 * 1000L;
repository.backup(true, "backup", timeout);
repository.saveChanges();
return "true";
} finally {
blockchainLock.unlock();
}
} catch (InterruptedException e) {
} catch (InterruptedException | TimeoutException e) {
// We couldn't lock blockchain to perform backup
return "false";
} catch (DataException e) {
@@ -676,13 +703,15 @@ public class AdminResource {
blockchainLock.lockInterruptibly();
try {
repository.performPeriodicMaintenance();
// Timeout if the database isn't ready to start after 60 seconds
long timeout = 60 * 1000L;
repository.performPeriodicMaintenance(timeout);
} finally {
blockchainLock.unlock();
}
} catch (InterruptedException e) {
// No big deal
} catch (DataException e) {
} catch (DataException | TimeoutException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}

View File

@@ -15,6 +15,8 @@ import java.math.BigDecimal;
import java.math.BigInteger;
import java.math.RoundingMode;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
@@ -33,11 +35,13 @@ import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.model.BlockMintingInfo;
import org.qortal.api.model.BlockSignerSummary;
import org.qortal.block.Block;
import org.qortal.controller.Controller;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountData;
import org.qortal.data.block.BlockData;
import org.qortal.data.block.BlockSummaryData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.repository.BlockArchiveReader;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@@ -71,7 +75,8 @@ public class BlocksResource {
@ApiErrors({
ApiError.INVALID_SIGNATURE, ApiError.BLOCK_UNKNOWN, ApiError.REPOSITORY_ISSUE
})
public BlockData getBlock(@PathParam("signature") String signature58) {
public BlockData getBlock(@PathParam("signature") String signature58,
@QueryParam("includeOnlineSignatures") Boolean includeOnlineSignatures) {
// Decode signature
byte[] signature;
try {
@@ -81,11 +86,25 @@ public class BlocksResource {
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Check the database first
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
if (blockData != null) {
if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
blockData.setOnlineAccountsSignatures(null);
}
return blockData;
}
return blockData;
// Not found, so try the block archive
blockData = repository.getBlockArchiveRepository().fromSignature(signature);
if (blockData != null) {
if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
blockData.setOnlineAccountsSignatures(null);
}
return blockData;
}
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -116,16 +135,24 @@ public class BlocksResource {
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Check the database first
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
if (blockData != null) {
Block block = new Block(repository, blockData);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
bytes.write(Ints.toByteArray(block.getBlockData().getHeight()));
bytes.write(BlockTransformer.toBytes(block));
return Base58.encode(bytes.toByteArray());
}
Block block = new Block(repository, blockData);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
bytes.write(Ints.toByteArray(block.getBlockData().getHeight()));
bytes.write(BlockTransformer.toBytes(block));
return Base58.encode(bytes.toByteArray());
// Not found, so try the block archive
byte[] bytes = BlockArchiveReader.getInstance().fetchSerializedBlockBytesForSignature(signature, false, repository);
if (bytes != null) {
return Base58.encode(bytes);
}
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
} catch (TransformationException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA, e);
} catch (DataException | IOException e) {
@@ -170,8 +197,12 @@ public class BlocksResource {
}
try (final Repository repository = RepositoryManager.getRepository()) {
if (repository.getBlockRepository().getHeightFromSignature(signature) == 0)
// Check if the block exists in either the database or archive
if (repository.getBlockRepository().getHeightFromSignature(signature) == 0 &&
repository.getBlockArchiveRepository().getHeightFromSignature(signature) == 0) {
// Not found in either the database or archive
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
}
return repository.getBlockRepository().getTransactionsFromSignature(signature, limit, offset, reverse);
} catch (DataException e) {
@@ -200,7 +231,19 @@ public class BlocksResource {
})
public BlockData getFirstBlock() {
try (final Repository repository = RepositoryManager.getRepository()) {
return repository.getBlockRepository().fromHeight(1);
// Check the database first
BlockData blockData = repository.getBlockRepository().fromHeight(1);
if (blockData != null) {
return blockData;
}
// Try the archive
blockData = repository.getBlockArchiveRepository().fromHeight(1);
if (blockData != null) {
return blockData;
}
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -262,17 +305,28 @@ public class BlocksResource {
}
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData childBlockData = null;
// Check if block exists in database
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData != null) {
return repository.getBlockRepository().fromReference(signature);
}
// Check block exists
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
BlockData childBlockData = repository.getBlockRepository().fromReference(signature);
// Not found, so try the archive
// This also checks that the parent block exists
// It will return null if either the parent or child don't exit
childBlockData = repository.getBlockArchiveRepository().fromReference(signature);
// Check child block exists
if (childBlockData == null)
if (childBlockData == null) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
}
// Check child block's reference matches the supplied signature
if (!Arrays.equals(childBlockData.getReference(), signature)) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
}
return childBlockData;
} catch (DataException e) {
@@ -338,13 +392,20 @@ public class BlocksResource {
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Firstly check the database
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData != null) {
return blockData.getHeight();
}
// Check block exists
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
// Not found, so try the archive
blockData = repository.getBlockArchiveRepository().fromSignature(signature);
if (blockData != null) {
return blockData.getHeight();
}
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
return blockData.getHeight();
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -369,13 +430,29 @@ public class BlocksResource {
@ApiErrors({
ApiError.BLOCK_UNKNOWN, ApiError.REPOSITORY_ISSUE
})
public BlockData getByHeight(@PathParam("height") int height) {
public BlockData getByHeight(@PathParam("height") int height,
@QueryParam("includeOnlineSignatures") Boolean includeOnlineSignatures) {
try (final Repository repository = RepositoryManager.getRepository()) {
// Firstly check the database
BlockData blockData = repository.getBlockRepository().fromHeight(height);
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
if (blockData != null) {
if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
blockData.setOnlineAccountsSignatures(null);
}
return blockData;
}
// Not found, so try the archive
blockData = repository.getBlockArchiveRepository().fromHeight(height);
if (blockData != null) {
if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
blockData.setOnlineAccountsSignatures(null);
}
return blockData;
}
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
return blockData;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -402,12 +479,31 @@ public class BlocksResource {
})
public BlockMintingInfo getBlockMintingInfoByHeight(@PathParam("height") int height) {
try (final Repository repository = RepositoryManager.getRepository()) {
// Try the database
BlockData blockData = repository.getBlockRepository().fromHeight(height);
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
if (blockData == null) {
// Not found, so try the archive
blockData = repository.getBlockArchiveRepository().fromHeight(height);
if (blockData == null) {
// Still not found
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
}
}
Block block = new Block(repository, blockData);
BlockData parentBlockData = repository.getBlockRepository().fromSignature(blockData.getReference());
if (parentBlockData == null) {
// Parent block not found - try the archive
parentBlockData = repository.getBlockArchiveRepository().fromSignature(blockData.getReference());
if (parentBlockData == null) {
// Still not found
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
}
}
int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, blockData.getMinterPublicKey());
if (minterLevel == 0)
// This may be unavailable when requesting a trimmed block
@@ -454,13 +550,26 @@ public class BlocksResource {
})
public BlockData getByTimestamp(@PathParam("timestamp") long timestamp) {
try (final Repository repository = RepositoryManager.getRepository()) {
int height = repository.getBlockRepository().getHeightFromTimestamp(timestamp);
if (height == 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
BlockData blockData = null;
BlockData blockData = repository.getBlockRepository().fromHeight(height);
if (blockData == null)
// Try the Blocks table
int height = repository.getBlockRepository().getHeightFromTimestamp(timestamp);
if (height > 0) {
// Found match in Blocks table
return repository.getBlockRepository().fromHeight(height);
}
// Not found in Blocks table, so try the archive
height = repository.getBlockArchiveRepository().getHeightFromTimestamp(timestamp);
if (height > 0) {
// Found match in archive
blockData = repository.getBlockArchiveRepository().fromHeight(height);
}
// Ensure block exists
if (blockData == null) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
}
return blockData;
} catch (DataException e) {
@@ -497,9 +606,14 @@ public class BlocksResource {
for (/* count already set */; count > 0; --count, ++height) {
BlockData blockData = repository.getBlockRepository().fromHeight(height);
if (blockData == null)
// Run out of blocks!
break;
if (blockData == null) {
// Not found - try the archive
blockData = repository.getBlockArchiveRepository().fromHeight(height);
if (blockData == null) {
// Run out of blocks!
break;
}
}
blocks.add(blockData);
}
@@ -544,7 +658,29 @@ public class BlocksResource {
if (accountData == null || accountData.getPublicKey() == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.PUBLIC_KEY_NOT_FOUND);
return repository.getBlockRepository().getBlockSummariesBySigner(accountData.getPublicKey(), limit, offset, reverse);
List<BlockSummaryData> summaries = repository.getBlockRepository()
.getBlockSummariesBySigner(accountData.getPublicKey(), limit, offset, reverse);
// Add any from the archive
List<BlockSummaryData> archivedSummaries = repository.getBlockArchiveRepository()
.getBlockSummariesBySigner(accountData.getPublicKey(), limit, offset, reverse);
if (archivedSummaries != null && !archivedSummaries.isEmpty()) {
summaries.addAll(archivedSummaries);
}
else {
summaries = archivedSummaries;
}
// Sort the results (because they may have been obtained from two places)
if (reverse != null && reverse) {
summaries.sort((s1, s2) -> Integer.valueOf(s2.getHeight()).compareTo(Integer.valueOf(s1.getHeight())));
}
else {
summaries.sort(Comparator.comparing(s -> Integer.valueOf(s.getHeight())));
}
return summaries;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -580,7 +716,8 @@ public class BlocksResource {
if (!Crypto.isValidAddress(address))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
return repository.getBlockRepository().getBlockSigners(addresses, limit, offset, reverse);
// This method pulls data from both Blocks and BlockArchive, so no need to query serparately
return repository.getBlockArchiveRepository().getBlockSigners(addresses, limit, offset, reverse);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -620,7 +757,76 @@ public class BlocksResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
try (final Repository repository = RepositoryManager.getRepository()) {
return repository.getBlockRepository().getBlockSummaries(startHeight, endHeight, count);
/*
* start end count result
* 10 40 null blocks 10 to 39 (excludes end block, ignore count)
*
* null null null blocks 1 to 50 (assume count=50, maybe start=1)
* 30 null null blocks 30 to 79 (assume count=50)
* 30 null 10 blocks 30 to 39
*
* null null 50 last 50 blocks? so if max(blocks.height) is 200, then blocks 151 to 200
* null 200 null blocks 150 to 199 (excludes end block, assume count=50)
* null 200 10 blocks 190 to 199 (excludes end block)
*/
List<BlockSummaryData> blockSummaries = new ArrayList<>();
// Use the latest X blocks if only a count is specified
if (startHeight == null && endHeight == null && count != null) {
BlockData chainTip = repository.getBlockRepository().getLastBlock();
startHeight = chainTip.getHeight() - count;
endHeight = chainTip.getHeight();
}
// ... otherwise default the start height to 1
if (startHeight == null && endHeight == null) {
startHeight = 1;
}
// Default the count to 50
if (count == null) {
count = 50;
}
// If both a start and end height exist, ignore the count
if (startHeight != null && endHeight != null) {
if (startHeight > 0 && endHeight > 0) {
count = Integer.MAX_VALUE;
}
}
// Derive start height from end height if missing
if (startHeight == null || startHeight == 0) {
if (endHeight != null && endHeight > 0) {
if (count != null) {
startHeight = endHeight - count;
}
}
}
for (/* count already set */; count > 0; --count, ++startHeight) {
if (endHeight != null && startHeight >= endHeight) {
break;
}
BlockData blockData = repository.getBlockRepository().fromHeight(startHeight);
if (blockData == null) {
// Not found - try the archive
blockData = repository.getBlockArchiveRepository().fromHeight(startHeight);
if (blockData == null) {
// Run out of blocks!
break;
}
}
if (blockData != null) {
BlockSummaryData blockSummaryData = new BlockSummaryData(blockData);
blockSummaries.add(blockSummaryData);
}
}
return blockSummaries;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}

View File

@@ -0,0 +1,92 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.api.ApiError;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.repository.Bootstrap;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.*;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import java.io.IOException;
@Path("/bootstrap")
@Tag(name = "Bootstrap")
public class BootstrapResource {
private static final Logger LOGGER = LogManager.getLogger(BootstrapResource.class);
@Context
HttpServletRequest request;
@POST
@Path("/create")
@Operation(
summary = "Create bootstrap",
description = "Builds a bootstrap file for distribution",
responses = {
@ApiResponse(
description = "path to file on success, an exception on failure",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
public String createBootstrap() {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
Bootstrap bootstrap = new Bootstrap(repository);
try {
bootstrap.checkRepositoryState();
} catch (DataException e) {
LOGGER.info("Not ready to create bootstrap: {}", e.getMessage());
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.REPOSITORY_ISSUE, e.getMessage());
}
bootstrap.validateBlockchain();
return bootstrap.create();
} catch (DataException | InterruptedException | IOException e) {
LOGGER.info("Unable to create bootstrap", e);
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.REPOSITORY_ISSUE, e.getMessage());
}
}
@GET
@Path("/validate")
@Operation(
summary = "Validate blockchain",
description = "Useful to check database integrity prior to creating or after installing a bootstrap. " +
"This process is intensive and can take over an hour to run.",
responses = {
@ApiResponse(
description = "true if valid, false if invalid",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
public boolean validateBootstrap() {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
Bootstrap bootstrap = new Bootstrap(repository);
return bootstrap.validateCompleteBlockchain();
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE);
}
}
}

View File

@@ -11,6 +11,7 @@ import java.util.List;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.core.Context;
@@ -173,7 +174,7 @@ public class CrossChainHtlcResource {
}
}
@GET
@POST
@Path("/redeem/{ataddress}")
@Operation(
summary = "Redeems HTLC associated with supplied AT",
@@ -231,7 +232,7 @@ public class CrossChainHtlcResource {
}
}
@GET
@POST
@Path("/redeemAll")
@Operation(
summary = "Redeems HTLC for all applicable ATs in tradebot data",
@@ -415,7 +416,7 @@ public class CrossChainHtlcResource {
return false;
}
@GET
@POST
@Path("/refund/{ataddress}")
@Operation(
summary = "Refunds HTLC associated with supplied AT",
@@ -463,7 +464,7 @@ public class CrossChainHtlcResource {
}
@GET
@POST
@Path("/refundAll")
@Operation(
summary = "Refunds HTLC for all applicable ATs in tradebot data",
@@ -478,8 +479,6 @@ public class CrossChainHtlcResource {
)
@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN})
public boolean refundAllHtlc() {
Security.checkApiCallAllowed(request);
Security.checkApiCallAllowed(request);
boolean success = false;
@@ -568,6 +567,13 @@ public class CrossChainHtlcResource {
if (crossChainTradeData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// If the AT is "finished" then it will have a zero balance
// In these cases we should avoid HTLC refunds if tbe QORT haven't been returned to the seller
if (atData.getIsFinished() && crossChainTradeData.mode != AcctMode.REFUNDED && crossChainTradeData.mode != AcctMode.CANCELLED) {
LOGGER.info(String.format("Skipping AT %s because the QORT has already been redemed", atAddress));
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
TradeBotData tradeBotData = allTradeBotData.stream().filter(tradeBotDataItem -> tradeBotDataItem.getAtAddress().equals(atAddress)).findFirst().orElse(null);
if (tradeBotData == null)

View File

@@ -1,5 +1,6 @@
package org.qortal.api.resource;
import com.google.common.primitives.Longs;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.ArraySchema;
@@ -10,12 +11,7 @@ import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.*;
import java.util.function.Supplier;
import javax.servlet.http.HttpServletRequest;
@@ -95,7 +91,7 @@ public class CrossChainResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
final boolean isExecutable = true;
List<CrossChainTradeData> crossChainTradesData = new ArrayList<>();
List<CrossChainTradeData> crossChainTrades = new ArrayList<>();
try (final Repository repository = RepositoryManager.getRepository()) {
Map<ByteArray, Supplier<ACCT>> acctsByCodeHash = SupportedBlockchain.getFilteredAcctMap(foreignBlockchain);
@@ -108,11 +104,27 @@ public class CrossChainResource {
for (ATData atData : atsData) {
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
crossChainTradesData.add(crossChainTradeData);
if (crossChainTradeData.mode == AcctMode.OFFERING) {
crossChainTrades.add(crossChainTradeData);
}
}
}
return crossChainTradesData;
// Sort the trades by timestamp
if (reverse != null && reverse) {
crossChainTrades.sort((a, b) -> Longs.compare(b.creationTimestamp, a.creationTimestamp));
}
else {
crossChainTrades.sort((a, b) -> Longs.compare(a.creationTimestamp, b.creationTimestamp));
}
if (limit != null && limit > 0) {
// Make sure to not return more than the limit
int upperLimit = Math.min(limit, crossChainTrades.size());
crossChainTrades = crossChainTrades.subList(0, upperLimit);
}
return crossChainTrades;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -195,6 +207,11 @@ public class CrossChainResource {
if (minimumTimestamp != null) {
minimumFinalHeight = repository.getBlockRepository().getHeightFromTimestamp(minimumTimestamp);
// If not found in the block repository it will return either 0 or 1
if (minimumFinalHeight == 0 || minimumFinalHeight == 1) {
// Try the archive
minimumFinalHeight = repository.getBlockArchiveRepository().getHeightFromTimestamp(minimumTimestamp);
}
if (minimumFinalHeight == 0)
// We don't have any blocks since minimumTimestamp, let alone trades, so nothing to return
@@ -222,12 +239,30 @@ public class CrossChainResource {
// We also need block timestamp for use as trade timestamp
long timestamp = repository.getBlockRepository().getTimestampFromHeight(atState.getHeight());
if (timestamp == 0) {
// Try the archive
timestamp = repository.getBlockArchiveRepository().getTimestampFromHeight(atState.getHeight());
}
CrossChainTradeSummary crossChainTradeSummary = new CrossChainTradeSummary(crossChainTradeData, timestamp);
crossChainTrades.add(crossChainTradeSummary);
}
}
// Sort the trades by timestamp
if (reverse != null && reverse) {
crossChainTrades.sort((a, b) -> Longs.compare(b.getTradeTimestamp(), a.getTradeTimestamp()));
}
else {
crossChainTrades.sort((a, b) -> Longs.compare(a.getTradeTimestamp(), b.getTradeTimestamp()));
}
if (limit != null && limit > 0) {
// Make sure to not return more than the limit
int upperLimit = Math.min(limit, crossChainTrades.size());
crossChainTrades = crossChainTrades.subList(0, upperLimit);
}
return crossChainTrades;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
@@ -260,7 +295,12 @@ public class CrossChainResource {
description = "Maximum number of trades to include in price calculation",
example = "10",
schema = @Schema(type = "integer", defaultValue = "10")
) @QueryParam("maxtrades") Integer maxtrades) {
) @QueryParam("maxtrades") Integer maxtrades,
@Parameter(
description = "Display price in terms of foreign currency per unit QORT",
example = "false",
schema = @Schema(type = "boolean", defaultValue = "false")
) @QueryParam("inverse") Boolean inverse) {
// foreignBlockchain is required
if (foreignBlockchain == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
@@ -270,6 +310,7 @@ public class CrossChainResource {
int maximumCount = maxtrades != null ? maxtrades : 10;
long minimumPeriod = 4 * 60 * 60 * 1000L; // ms
Boolean isFinished = Boolean.TRUE;
boolean useInversePrice = (inverse != null && inverse == true);
try (final Repository repository = RepositoryManager.getRepository()) {
Map<ByteArray, Supplier<ACCT>> acctsByCodeHash = SupportedBlockchain.getFilteredAcctMap(foreignBlockchain);
@@ -277,6 +318,9 @@ public class CrossChainResource {
long totalForeign = 0;
long totalQort = 0;
Map<Long, CrossChainTradeData> reverseSortedTradeData = new TreeMap<>(Collections.reverseOrder());
// Collect recent AT states for each ACCT version
for (Map.Entry<ByteArray, Supplier<ACCT>> acctInfo : acctsByCodeHash.entrySet()) {
byte[] codeHash = acctInfo.getKey().value;
ACCT acct = acctInfo.getValue().get();
@@ -285,13 +329,38 @@ public class CrossChainResource {
isFinished, acct.getModeByteOffset(), (long) AcctMode.REDEEMED.value, minimumCount, maximumCount, minimumPeriod);
for (ATStateData atState : atStates) {
// We also need block timestamp for use as trade timestamp
long timestamp = repository.getBlockRepository().getTimestampFromHeight(atState.getHeight());
if (timestamp == 0) {
// Try the archive
timestamp = repository.getBlockArchiveRepository().getTimestampFromHeight(atState.getHeight());
}
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atState);
totalForeign += crossChainTradeData.expectedForeignAmount;
totalQort += crossChainTradeData.qortAmount;
reverseSortedTradeData.put(timestamp, crossChainTradeData);
}
}
return Amounts.scaledDivide(totalQort, totalForeign);
// Loop through the sorted map and calculate the average price
// Also remove elements beyond the maxtrades limit
Set set = reverseSortedTradeData.entrySet();
Iterator i = set.iterator();
int index = 0;
while (i.hasNext()) {
Map.Entry tradeDataMap = (Map.Entry)i.next();
CrossChainTradeData crossChainTradeData = (CrossChainTradeData) tradeDataMap.getValue();
if (maxtrades != null && index >= maxtrades) {
// We've reached the limit
break;
}
totalForeign += crossChainTradeData.expectedForeignAmount;
totalQort += crossChainTradeData.qortAmount;
index++;
}
return useInversePrice ? Amounts.scaledDivide(totalForeign, totalQort) : Amounts.scaledDivide(totalQort, totalForeign);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -421,4 +490,4 @@ public class CrossChainResource {
}
}
}
}

View File

@@ -30,6 +30,7 @@ import org.qortal.api.Security;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.api.model.crosschain.TradeBotRespondRequest;
import org.qortal.asset.Asset;
import org.qortal.controller.Controller;
import org.qortal.controller.tradebot.AcctTradeBot;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.crosschain.ForeignBlockchain;
@@ -136,6 +137,9 @@ public class CrossChainTradeBotResource {
if (tradeBotCreateRequest.qortAmount <= 0 || tradeBotCreateRequest.fundingQortAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ORDER_SIZE_TOO_SMALL);
if (!Controller.getInstance().isUpToDate())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
try (final Repository repository = RepositoryManager.getRepository()) {
// Do some simple checking first
Account creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
@@ -193,6 +197,9 @@ public class CrossChainTradeBotResource {
if (tradeBotRespondRequest.receivingAddress == null || !Crypto.isValidAddress(tradeBotRespondRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!Controller.getInstance().isUpToDate())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
// Extract data from cross-chain trading AT
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = fetchAtDataWithChecking(repository, atAddress);

View File

@@ -1,6 +1,7 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
@@ -8,7 +9,7 @@ import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.qortal.api.*;
import org.qortal.api.model.AddressListRequest;
import org.qortal.api.model.ListRequest;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountData;
import org.qortal.list.ResourceListManager;
@@ -28,37 +29,9 @@ public class ListsResource {
@Context
HttpServletRequest request;
@POST
@Path("/blacklist/address/{address}")
@Operation(
summary = "Add a QORT address to the local blacklist",
responses = {
@ApiResponse(
description = "Returns true on success, or an exception on failure",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN, ApiError.REPOSITORY_ISSUE})
public String addAddressToBlacklist(@PathParam("address") String address) {
if (!Crypto.isValidAddress(address))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
AccountData accountData = repository.getAccountRepository().getAccount(address);
// Not found?
if (accountData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// Valid address, so go ahead and blacklist it
boolean success = ResourceListManager.getInstance().addAddressToBlacklist(address, true);
return success ? "true" : "false";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
/* Address blacklist */
@POST
@Path("/blacklist/addresses")
@@ -69,7 +42,7 @@ public class ListsResource {
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = AddressListRequest.class
implementation = ListRequest.class
)
)
),
@@ -83,8 +56,10 @@ public class ListsResource {
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN, ApiError.REPOSITORY_ISSUE})
public String addAddressesToBlacklist(AddressListRequest addressListRequest) {
if (addressListRequest == null || addressListRequest.addresses == null) {
public String addAddressesToBlacklist(ListRequest listRequest) {
Security.checkApiCallAllowed(request);
if (listRequest == null || listRequest.items == null) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
@@ -93,7 +68,7 @@ public class ListsResource {
try (final Repository repository = RepositoryManager.getRepository()) {
for (String address : addressListRequest.addresses) {
for (String address : listRequest.items) {
if (!Crypto.isValidAddress(address)) {
errorCount++;
@@ -108,7 +83,7 @@ public class ListsResource {
}
// Valid address, so go ahead and blacklist it
boolean success = ResourceListManager.getInstance().addAddressToBlacklist(address, false);
boolean success = ResourceListManager.getInstance().addToList("blacklist", "addresses", address, false);
if (success) {
successCount++;
}
@@ -122,48 +97,16 @@ public class ListsResource {
if (successCount > 0 && errorCount == 0) {
// All were successful, so save the blacklist
ResourceListManager.getInstance().saveBlacklist();
ResourceListManager.getInstance().saveList("blacklist", "addresses");
return "true";
}
else {
// Something went wrong, so revert
ResourceListManager.getInstance().revertBlacklist();
ResourceListManager.getInstance().revertList("blacklist", "addresses");
return "false";
}
}
@DELETE
@Path("/blacklist/address/{address}")
@Operation(
summary = "Remove a QORT address from the local blacklist",
responses = {
@ApiResponse(
description = "Returns true on success, or an exception on failure",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN, ApiError.REPOSITORY_ISSUE})
public String removeAddressFromBlacklist(@PathParam("address") String address) {
if (!Crypto.isValidAddress(address))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
AccountData accountData = repository.getAccountRepository().getAccount(address);
// Not found?
if (accountData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// Valid address, so go ahead and blacklist it
boolean success = ResourceListManager.getInstance().removeAddressFromBlacklist(address, true);
return success ? "true" : "false";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@DELETE
@Path("/blacklist/addresses")
@Operation(
@@ -173,7 +116,7 @@ public class ListsResource {
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = AddressListRequest.class
implementation = ListRequest.class
)
)
),
@@ -187,8 +130,10 @@ public class ListsResource {
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN, ApiError.REPOSITORY_ISSUE})
public String removeAddressesFromBlacklist(AddressListRequest addressListRequest) {
if (addressListRequest == null || addressListRequest.addresses == null) {
public String removeAddressesFromBlacklist(ListRequest listRequest) {
Security.checkApiCallAllowed(request);
if (listRequest == null || listRequest.items == null) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
@@ -197,7 +142,7 @@ public class ListsResource {
try (final Repository repository = RepositoryManager.getRepository()) {
for (String address : addressListRequest.addresses) {
for (String address : listRequest.items) {
if (!Crypto.isValidAddress(address)) {
errorCount++;
@@ -213,7 +158,7 @@ public class ListsResource {
// Valid address, so go ahead and blacklist it
// Don't save as we will do this at the end of the process
boolean success = ResourceListManager.getInstance().removeAddressFromBlacklist(address, false);
boolean success = ResourceListManager.getInstance().removeFromList("blacklist", "addresses", address, false);
if (success) {
successCount++;
}
@@ -227,45 +172,30 @@ public class ListsResource {
if (successCount > 0 && errorCount == 0) {
// All were successful, so save the blacklist
ResourceListManager.getInstance().saveBlacklist();
ResourceListManager.getInstance().saveList("blacklist", "addresses");
return "true";
}
else {
// Something went wrong, so revert
ResourceListManager.getInstance().revertBlacklist();
ResourceListManager.getInstance().revertList("blacklist", "addresses");
return "false";
}
}
@GET
@Path("/blacklist/address/{address}")
@Path("/blacklist/addresses")
@Operation(
summary = "Check if an address is present in the local blacklist",
summary = "Fetch the list of blacklisted addresses",
responses = {
@ApiResponse(
description = "Returns true or false if the list was queried, or an exception on failure",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
description = "A JSON array of addresses",
content = @Content(mediaType = MediaType.APPLICATION_JSON, array = @ArraySchema(schema = @Schema(implementation = String.class)))
)
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN, ApiError.REPOSITORY_ISSUE})
public String checkAddressInBlacklist(@PathParam("address") String address) {
if (!Crypto.isValidAddress(address))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
AccountData accountData = repository.getAccountRepository().getAccount(address);
// Not found?
if (accountData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// Valid address, so go ahead and blacklist it
boolean blacklisted = ResourceListManager.getInstance().isAddressInBlacklist(address);
return blacklisted ? "true" : "false";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
public String getAddressBlacklist() {
Security.checkApiCallAllowed(request);
return ResourceListManager.getInstance().getJSONStringForList("blacklist", "addresses");
}
}

View File

@@ -1092,9 +1092,14 @@ public class Block {
// Create repository savepoint here so we can rollback to it after testing transactions
repository.setSavepoint();
if (this.blockData.getHeight() == 212937)
if (this.blockData.getHeight() == 212937) {
// Apply fix for block 212937 but fix will be rolled back before we exit method
Block212937.processFix(this);
}
else if (InvalidNameRegistrationBlocks.isAffectedBlock(this.blockData.getHeight())) {
// Apply fix for affected name registration blocks, but fix will be rolled back before we exit method
InvalidNameRegistrationBlocks.processFix(this);
}
for (Transaction transaction : this.getTransactions()) {
TransactionData transactionData = transaction.getTransactionData();
@@ -1133,7 +1138,7 @@ public class Block {
// Check transaction can even be processed
validationResult = transaction.isProcessable();
if (validationResult != Transaction.ValidationResult.OK) {
LOGGER.debug(String.format("Error during transaction validation, tx %s: %s", Base58.encode(transactionData.getSignature()), validationResult.name()));
LOGGER.info(String.format("Error during transaction validation, tx %s: %s", Base58.encode(transactionData.getSignature()), validationResult.name()));
return ValidationResult.TRANSACTION_INVALID;
}
@@ -1282,6 +1287,21 @@ public class Block {
return mintingAccount.canMint();
}
/**
* Pre-process block, and its transactions.
* This allows for any database integrity checks prior to validation.
* This is called before isValid() and process()
*
* @throws DataException
*/
public void preProcess() throws DataException {
List<Transaction> blocksTransactions = this.getTransactions();
for (Transaction transaction : blocksTransactions) {
transaction.preProcess();
}
}
/**
* Process block, and its transactions, adding them to the blockchain.
*

View File

@@ -4,10 +4,7 @@ import java.io.File;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.*;
import java.util.concurrent.locks.ReentrantLock;
import javax.xml.bind.JAXBContext;
@@ -27,11 +24,9 @@ import org.eclipse.persistence.jaxb.UnmarshallerProperties;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.network.Network;
import org.qortal.repository.BlockRepository;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.repository.*;
import org.qortal.settings.Settings;
import org.qortal.utils.Base58;
import org.qortal.utils.StringLongMapXmlAdapter;
/**
@@ -506,29 +501,105 @@ public class BlockChain {
* @throws SQLException
*/
public static void validate() throws DataException {
// Check first block is Genesis Block
if (!isGenesisBlockValid())
rebuildBlockchain();
boolean isTopOnly = Settings.getInstance().isTopOnly();
boolean archiveEnabled = Settings.getInstance().isArchiveEnabled();
boolean canBootstrap = Settings.getInstance().getBootstrap();
boolean needsArchiveRebuild = false;
BlockData chainTip;
try (final Repository repository = RepositoryManager.getRepository()) {
chainTip = repository.getBlockRepository().getLastBlock();
// Ensure archive is (at least partially) intact, and force a bootstrap if it isn't
if (!isTopOnly && archiveEnabled && canBootstrap) {
needsArchiveRebuild = (repository.getBlockArchiveRepository().fromHeight(2) == null);
if (needsArchiveRebuild) {
LOGGER.info("Couldn't retrieve block 2 from archive. Bootstrapping...");
// If there are minting accounts, make sure to back them up
// Don't backup if there are no minting accounts, as this can cause problems
if (!repository.getAccountRepository().getMintingAccounts().isEmpty()) {
Controller.getInstance().exportRepositoryData();
}
}
}
}
boolean hasBlocks = (chainTip != null && chainTip.getHeight() > 1);
if (isTopOnly && hasBlocks) {
// Top-only mode is enabled and we have blocks, so it's possible that the genesis block has been pruned
// It's best not to validate it, and there's no real need to
} else {
// Check first block is Genesis Block
if (!isGenesisBlockValid() || needsArchiveRebuild) {
try {
rebuildBlockchain();
} catch (InterruptedException e) {
throw new DataException(String.format("Interrupted when trying to rebuild blockchain: %s", e.getMessage()));
}
}
}
// We need to create a new connection, as the previous repository and its connections may be been
// closed by rebuildBlockchain() if a bootstrap was applied
try (final Repository repository = RepositoryManager.getRepository()) {
repository.checkConsistency();
int startHeight = Math.max(repository.getBlockRepository().getBlockchainHeight() - 1440, 1);
// Set the number of blocks to validate based on the pruned state of the chain
// If pruned, subtract an extra 10 to allow room for error
int blocksToValidate = (isTopOnly || archiveEnabled) ? Settings.getInstance().getPruneBlockLimit() - 10 : 1440;
int startHeight = Math.max(repository.getBlockRepository().getBlockchainHeight() - blocksToValidate, 1);
BlockData detachedBlockData = repository.getBlockRepository().getDetachedBlockSignature(startHeight);
if (detachedBlockData != null) {
LOGGER.error(String.format("Block %d's reference does not match any block's signature", detachedBlockData.getHeight()));
LOGGER.error(String.format("Block %d's reference does not match any block's signature",
detachedBlockData.getHeight()));
LOGGER.error(String.format("Your chain may be invalid and you should consider bootstrapping" +
" or re-syncing from genesis."));
}
}
}
// Wait for blockchain lock (whereas orphan() only tries to get lock)
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
blockchainLock.lock();
try {
LOGGER.info(String.format("Orphaning back to block %d", detachedBlockData.getHeight() - 1));
orphan(detachedBlockData.getHeight() - 1);
} finally {
blockchainLock.unlock();
/**
* More thorough blockchain validation method. Useful for validating bootstraps.
* A DataException is thrown if anything is invalid.
*
* @throws DataException
*/
public static void validateAllBlocks() throws DataException {
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData chainTip = repository.getBlockRepository().getLastBlock();
final int chainTipHeight = chainTip.getHeight();
final int oldestBlock = 1; // TODO: increase if in pruning mode
byte[] lastReference = null;
for (int height = chainTipHeight; height > oldestBlock; height--) {
BlockData blockData = repository.getBlockRepository().fromHeight(height);
if (blockData == null) {
blockData = repository.getBlockArchiveRepository().fromHeight(height);
}
if (blockData == null) {
String error = String.format("Missing block at height %d", height);
LOGGER.error(error);
throw new DataException(error);
}
if (height != chainTipHeight) {
// Check reference
if (!Arrays.equals(blockData.getSignature(), lastReference)) {
String error = String.format("Invalid reference for block at height %d: %s (should be %s)",
height, Base58.encode(blockData.getReference()), Base58.encode(lastReference));
LOGGER.error(error);
throw new DataException(error);
}
}
lastReference = blockData.getReference();
}
}
}
@@ -551,7 +622,15 @@ public class BlockChain {
}
}
private static void rebuildBlockchain() throws DataException {
private static void rebuildBlockchain() throws DataException, InterruptedException {
boolean shouldBootstrap = Settings.getInstance().getBootstrap();
if (shouldBootstrap) {
// Settings indicate that we should apply a bootstrap rather than rebuilding and syncing from genesis
Bootstrap bootstrap = new Bootstrap();
bootstrap.startImport();
return;
}
// (Re)build repository
if (!RepositoryManager.wasPristineAtOpen())
RepositoryManager.rebuild();

View File

@@ -0,0 +1,114 @@
package org.qortal.block;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.naming.Name;
import org.qortal.repository.DataException;
import java.util.HashMap;
import java.util.Map;
/**
* Invalid Name Registration Blocks
* <p>
* A node minted a version of block 535658 that contained one transaction:
* a REGISTER_NAME transaction that attempted to register a name that was already registered.
* <p>
* This invalid transaction made block 535658 (rightly) invalid to several nodes,
* which refused to use that block.
* However, it seems there were no other nodes minting an alternative, valid block at that time
* and so the chain stalled for several nodes in the network.
* <p>
* Additionally, the invalid block 535658 affected all new installations, regardless of whether
* they synchronized from scratch (block 1) or used an 'official release' bootstrap.
* <p>
* The diagnosis found the following:
* - The original problem occurred in block 535205 where for some unknown reason many nodes didn't
* add the name from a REGISTER_NAME transaction to their Names table.
* - As a result, those nodes had a corrupt db, because they weren't holding a record of the name.
* - This invalid db then caused them to treat a candidate for block 535658 as valid when it
* should have been invalid.
* - As such, the chain continued on with a technically invalid block in it, for a subset of the network
* <p>
* As with block 212937, there were three options, but the only feasible one was to apply edits to block
* 535658 to make it valid. There were several cross-chain trades completed after this block, so doing
* any kind of rollback was out of the question.
* <p>
* To complicate things further, a custom data field was used for the first REGISTER_NAME transaction,
* and the default data field was used for the second. So it was important that all nodes ended up with
* the exact same data regardless of how they arrived there.
* <p>
* The invalid block 535658 signature is: <tt>3oiuDhok...NdXvCLEV</tt>.
* <p>
* The invalid transaction in block 212937 is:
* <p>
* <code><pre>
{
"type": "REGISTER_NAME",
"timestamp": 1630739437517,
"reference": "4peRechwSPxP6UkRj9Y8ox9YxkWb34sWk5zyMc1WyMxEsACxD4Gmm7LZVsQ6Skpze8QCSBMZasvEZg6RgdqkyADW",
"fee": "0.00100000",
"signature": "2t1CryCog8KPDBarzY5fDCKu499nfnUcGrz4Lz4w5wNb5nWqm7y126P48dChYY7huhufcBV3RJPkgKP4Ywxc1gXx",
"txGroupId": 0,
"blockHeight": 535658,
"approvalStatus": "NOT_REQUIRED",
"creatorAddress": "Qbx9ojxv7XNi1xDMWzzw7xDvd1zYW6SKFB",
"registrantPublicKey": "HJqGEf6cW695Xun4ydhkB2excGFwsDxznhNCRHZStyyx",
"name": "Qplay",
"data": "Registered Name on the Qortal Chain"
}
</pre></code>
* <p>
* Account <tt>Qbx9ojxv7XNi1xDMWzzw7xDvd1zYW6SKFB</tt> attempted to register the name <tt>Qplay</tt>
* when they had already registered it 12 hours before in block <tt>535205</tt>.
* <p>
* However, on the broken DB nodes, their Names table was missing a record for the `Qplay` name
* which was sufficient to make the transaction valid.
*
* This problem then occurred two more times, in blocks 536140 and 541334
* To reduce duplication, I have combined all three block fixes into a single class
*
*/
public final class InvalidNameRegistrationBlocks {
private static final Logger LOGGER = LogManager.getLogger(InvalidNameRegistrationBlocks.class);
public static Map<Integer, String> invalidBlocksNamesMap = new HashMap<Integer, String>()
{
{
put(535658, "Qplay");
put(536140, "Qweb");
put(541334, "Qithub");
}
};
private InvalidNameRegistrationBlocks() {
/* Do not instantiate */
}
public static boolean isAffectedBlock(int height) {
return (invalidBlocksNamesMap.containsKey(height));
}
public static void processFix(Block block) throws DataException {
Integer blockHeight = block.getBlockData().getHeight();
String invalidName = invalidBlocksNamesMap.get(blockHeight);
if (invalidName == null) {
throw new DataException(String.format("Unable to lookup invalid name for block height %d", blockHeight));
}
// Unregister the existing name record if it exists
// This ensures that the duplicate name is considered valid, and therefore
// the second (i.e. duplicate) REGISTER_NAME transaction data is applied.
// Both were issued by the same user account, so there is no conflict.
Name name = new Name(block.repository, invalidName);
name.unregister();
LOGGER.debug("Applied name registration patch for block {}", blockHeight);
}
// Note:
// There is no need to write an orphanFix() method, as we do not have
// the necessary ATStatesData to orphan back this far anyway
}

View File

@@ -14,6 +14,7 @@ import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.TimeoutException;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -215,8 +216,17 @@ public class AutoUpdate extends Thread {
}
// Give repository a chance to backup in case things go badly wrong (if enabled)
if (Settings.getInstance().getRepositoryBackupInterval() > 0)
RepositoryManager.backup(true);
if (Settings.getInstance().getRepositoryBackupInterval() > 0) {
try {
// Timeout if the database isn't ready for backing up after 60 seconds
long timeout = 60 * 1000L;
RepositoryManager.backup(true, "backup", timeout);
} catch (TimeoutException e) {
LOGGER.info("Attempt to backup repository failed due to timeout: {}", e.getMessage());
// Continue with the auto update anyway...
}
}
// Call ApplyUpdate to end this process (unlocking current JAR so it can be replaced)
String javaHome = System.getProperty("java.home");

View File

@@ -16,6 +16,7 @@ import org.qortal.account.PrivateKeyAccount;
import org.qortal.block.Block;
import org.qortal.block.Block.ValidationResult;
import org.qortal.block.BlockChain;
import org.qortal.data.account.AccountData;
import org.qortal.data.account.MintingAccountData;
import org.qortal.data.account.RewardShareData;
import org.qortal.data.block.BlockData;
@@ -44,6 +45,14 @@ public class BlockMinter extends Thread {
private static Long lastLogTimestamp;
private static Long logTimeout;
// Recovery
public static final long INVALID_BLOCK_RECOVERY_TIMEOUT = 10 * 60 * 1000L; // ms
// Min account level to submit blocks
// This is an unvalidated version of Blockchain.minAccountLevelToMint
// and exists only to reduce block candidates by default.
private static int MIN_LEVEL_FOR_BLOCK_SUBMISSION = 3;
// Constructors
public BlockMinter() {
@@ -127,6 +136,16 @@ public class BlockMinter extends Thread {
madi.remove();
continue;
}
// Optional (non-validated) prevention of block submissions below a defined level
AccountData accountData = repository.getAccountRepository().getAccount(mintingAccount.getAddress());
if (accountData != null) {
Integer level = accountData.getLevel();
if (level != null && level < MIN_LEVEL_FOR_BLOCK_SUBMISSION) {
madi.remove();
continue;
}
}
}
List<Peer> peers = Network.getInstance().getHandshakedPeers();
@@ -144,9 +163,25 @@ public class BlockMinter extends Thread {
if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
continue;
// If we are stuck on an invalid block, we should allow an alternative to be minted
boolean recoverInvalidBlock = false;
if (Synchronizer.getInstance().timeInvalidBlockLastReceived != null) {
// We've had at least one invalid block
long timeSinceLastValidBlock = NTP.getTime() - Synchronizer.getInstance().timeValidBlockLastReceived;
long timeSinceLastInvalidBlock = NTP.getTime() - Synchronizer.getInstance().timeInvalidBlockLastReceived;
if (timeSinceLastValidBlock > INVALID_BLOCK_RECOVERY_TIMEOUT) {
if (timeSinceLastInvalidBlock < INVALID_BLOCK_RECOVERY_TIMEOUT) {
// Last valid block was more than 10 mins ago, but we've had an invalid block since then
// Assume that the chain has stalled because there is no alternative valid candidate
// Enter recovery mode to allow alternative, valid candidates to be minted
recoverInvalidBlock = true;
}
}
}
// If our latest block isn't recent then we need to synchronize instead of minting, unless we're in recovery mode.
if (!peers.isEmpty() && lastBlockData.getTimestamp() < minLatestBlockTimestamp)
if (Controller.getInstance().getRecoveryMode() == false)
if (Controller.getInstance().getRecoveryMode() == false && recoverInvalidBlock == false)
continue;
// There are enough peers with a recent block and our latest block is recent
@@ -230,6 +265,8 @@ public class BlockMinter extends Thread {
if (testBlock.isTimestampValid() != ValidationResult.OK)
continue;
testBlock.preProcess();
// Is new block valid yet? (Before adding unconfirmed transactions)
ValidationResult result = testBlock.isValid();
if (result != ValidationResult.OK) {
@@ -421,7 +458,8 @@ public class BlockMinter extends Thread {
// Add to blockchain
newBlock.process();
LOGGER.info(String.format("Minted new test block: %d", newBlock.getBlockData().getHeight()));
LOGGER.info(String.format("Minted new test block: %d sig: %.8s",
newBlock.getBlockData().getHeight(), Base58.encode(newBlock.getBlockData().getSignature())));
repository.saveChanges();

View File

@@ -2,8 +2,11 @@ package org.qortal.controller;
import java.awt.TrayIcon.MessageType;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.SecureRandom;
import java.security.Security;
import java.time.LocalDateTime;
@@ -24,7 +27,7 @@ import java.util.Properties;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Function;
@@ -46,6 +49,8 @@ import org.qortal.block.Block;
import org.qortal.block.BlockChain;
import org.qortal.block.BlockChain.BlockTimingByHeight;
import org.qortal.controller.Synchronizer.SynchronizationResult;
import org.qortal.controller.repository.PruneManager;
import org.qortal.controller.repository.NamesDatabaseIntegrityCheck;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.MintingAccountData;
@@ -83,21 +88,14 @@ import org.qortal.network.message.OnlineAccountsMessage;
import org.qortal.network.message.SignaturesMessage;
import org.qortal.network.message.TransactionMessage;
import org.qortal.network.message.TransactionSignaturesMessage;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryFactory;
import org.qortal.repository.RepositoryManager;
import org.qortal.repository.*;
import org.qortal.repository.hsqldb.HSQLDBRepositoryFactory;
import org.qortal.settings.Settings;
import org.qortal.transaction.ArbitraryTransaction;
import org.qortal.transaction.Transaction;
import org.qortal.transaction.Transaction.TransactionType;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.utils.Base58;
import org.qortal.utils.ByteArray;
import org.qortal.utils.DaemonThreadFactory;
import org.qortal.utils.NTP;
import org.qortal.utils.Triple;
import org.qortal.utils.*;
import com.google.common.primitives.Longs;
@@ -105,6 +103,7 @@ public class Controller extends Thread {
static {
// This must go before any calls to LogManager/Logger
System.setProperty("log4j2.formatMsgNoLookups", "true");
System.setProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager");
}
@@ -157,6 +156,7 @@ public class Controller extends Thread {
};
private long repositoryBackupTimestamp = startTime; // ms
private long repositoryMaintenanceTimestamp = startTime; // ms
private long repositoryCheckpointTimestamp = startTime; // ms
private long ntpCheckTimestamp = startTime; // ms
private long deleteExpiredTimestamp = startTime + DELETE_EXPIRED_INTERVAL; // ms
@@ -314,6 +314,10 @@ public class Controller extends Thread {
return this.buildVersion;
}
public String getVersionStringWithoutPrefix() {
return this.buildVersion.replaceFirst(VERSION_PREFIX, "");
}
/** Returns current blockchain height, or 0 if it's not available. */
public int getChainHeight() {
synchronized (this.latestBlocks) {
@@ -357,7 +361,7 @@ public class Controller extends Thread {
return this.savedArgs;
}
/* package */ static boolean isStopping() {
public static boolean isStopping() {
return isStopping;
}
@@ -415,6 +419,12 @@ public class Controller extends Thread {
try {
RepositoryFactory repositoryFactory = new HSQLDBRepositoryFactory(getRepositoryUrl());
RepositoryManager.setRepositoryFactory(repositoryFactory);
RepositoryManager.setRequestedCheckpoint(Boolean.TRUE);
try (final Repository repository = RepositoryManager.getRepository()) {
RepositoryManager.archive(repository);
RepositoryManager.prune(repository);
}
} catch (DataException e) {
// If exception has no cause then repository is in use by some other process.
if (e.getCause() == null) {
@@ -428,6 +438,11 @@ public class Controller extends Thread {
return; // Not System.exit() so that GUI can display error
}
// Rebuild Names table and check database integrity
NamesDatabaseIntegrityCheck namesDatabaseIntegrityCheck = new NamesDatabaseIntegrityCheck();
namesDatabaseIntegrityCheck.rebuildAllNames();
namesDatabaseIntegrityCheck.runIntegrityCheck();
LOGGER.info("Validating blockchain");
try {
BlockChain.validate();
@@ -440,6 +455,12 @@ public class Controller extends Thread {
return; // Not System.exit() so that GUI can display error
}
// Import current trade bot states and minting accounts if they exist
Controller.importRepositoryData();
// Add the initial peers to the repository if we don't have any
Controller.installInitialPeers();
LOGGER.info("Starting controller");
Controller.getInstance().start();
@@ -509,10 +530,10 @@ public class Controller extends Thread {
final long repositoryBackupInterval = Settings.getInstance().getRepositoryBackupInterval();
final long repositoryCheckpointInterval = Settings.getInstance().getRepositoryCheckpointInterval();
long repositoryMaintenanceInterval = getRandomRepositoryMaintenanceInterval();
ExecutorService trimExecutor = Executors.newCachedThreadPool(new DaemonThreadFactory());
trimExecutor.execute(new AtStatesTrimmer());
trimExecutor.execute(new OnlineAccountsSignaturesTrimmer());
// Start executor service for trimming or pruning
PruneManager.getInstance().start();
try {
while (!isStopping) {
@@ -570,7 +591,39 @@ public class Controller extends Thread {
Translator.INSTANCE.translate("SysTray", "CREATING_BACKUP_OF_DB_FILES"),
MessageType.INFO);
RepositoryManager.backup(true);
try {
// Timeout if the database isn't ready for backing up after 60 seconds
long timeout = 60 * 1000L;
RepositoryManager.backup(true, "backup", timeout);
} catch (TimeoutException e) {
LOGGER.info("Attempt to backup repository failed due to timeout: {}", e.getMessage());
}
}
// Give repository a chance to perform maintenance (if enabled)
if (repositoryMaintenanceInterval > 0 && now >= repositoryMaintenanceTimestamp + repositoryMaintenanceInterval) {
repositoryMaintenanceTimestamp = now + repositoryMaintenanceInterval;
if (Settings.getInstance().getShowMaintenanceNotification())
SysTray.getInstance().showMessage(Translator.INSTANCE.translate("SysTray", "DB_MAINTENANCE"),
Translator.INSTANCE.translate("SysTray", "PERFORMING_DB_MAINTENANCE"),
MessageType.INFO);
LOGGER.info("Starting scheduled repository maintenance. This can take a while...");
try (final Repository repository = RepositoryManager.getRepository()) {
// Timeout if the database isn't ready for maintenance after 60 seconds
long timeout = 60 * 1000L;
repository.performPeriodicMaintenance(timeout);
LOGGER.info("Scheduled repository maintenance completed");
} catch (DataException | TimeoutException e) {
LOGGER.error("Scheduled repository maintenance failed", e);
}
// Get a new random interval
repositoryMaintenanceInterval = getRandomRepositoryMaintenanceInterval();
}
// Prune stuck/slow/old peers
@@ -597,13 +650,68 @@ public class Controller extends Thread {
Thread.interrupted();
// Fall-through to exit
} finally {
trimExecutor.shutdownNow();
PruneManager.getInstance().stop();
}
}
/**
* Import current trade bot states and minting accounts.
* This is needed because the user may have bootstrapped, or there could be a database inconsistency
* if the core crashed when computing the nonce during the start of the trade process.
*/
private static void importRepositoryData() {
try (final Repository repository = RepositoryManager.getRepository()) {
String exportPath = Settings.getInstance().getExportPath();
try {
Path importPath = Paths.get(exportPath, "TradeBotStates.json");
repository.importDataFromFile(importPath.toString());
} catch (FileNotFoundException e) {
// Do nothing, as the files will only exist in certain cases
}
try {
trimExecutor.awaitTermination(2L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// We tried...
Path importPath = Paths.get(exportPath, "MintingAccounts.json");
repository.importDataFromFile(importPath.toString());
} catch (FileNotFoundException e) {
// Do nothing, as the files will only exist in certain cases
}
repository.saveChanges();
}
catch (DataException | IOException e) {
LOGGER.info("Unable to import data into repository: {}", e.getMessage());
}
}
private static void installInitialPeers() {
try (final Repository repository = RepositoryManager.getRepository()) {
if (repository.getNetworkRepository().getAllPeers().isEmpty()) {
Network.installInitialPeers(repository);
}
} catch (DataException e) {
// Fail silently as this is an optional step
}
}
private long getRandomRepositoryMaintenanceInterval() {
final long minInterval = Settings.getInstance().getRepositoryMaintenanceMinInterval();
final long maxInterval = Settings.getInstance().getRepositoryMaintenanceMaxInterval();
if (maxInterval == 0) {
return 0;
}
return (new Random().nextLong() % (maxInterval - minInterval)) + minInterval;
}
/**
* Export current trade bot states and minting accounts.
*/
public void exportRepositoryData() {
try (final Repository repository = RepositoryManager.getRepository()) {
repository.exportNodeLocalData();
} catch (DataException e) {
// Fail silently as this is an optional step
}
}
@@ -886,7 +994,7 @@ public class Controller extends Thread {
}
}
String tooltip = String.format("%s - %d %s - %s %d", actionText, numberOfPeers, connectionsText, heightText, height) + "\n" + String.format("Build version: %s", this.buildVersion);
String tooltip = String.format("%s - %d %s - %s %d", actionText, numberOfPeers, connectionsText, heightText, height) + "\n" + String.format("%s: %s", Translator.INSTANCE.translate("SysTray", "BUILD_VERSION"), this.buildVersion);
SysTray.getInstance().setToolTipText(tooltip);
this.callbackExecutor.execute(() -> {
@@ -958,6 +1066,10 @@ public class Controller extends Thread {
}
}
// Export local data
LOGGER.info("Backing up local data");
this.exportRepositoryData();
LOGGER.info("Shutting down networking");
Network.getInstance().shutdown();
@@ -1286,6 +1398,34 @@ public class Controller extends Thread {
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData != null) {
if (PruneManager.getInstance().isBlockPruned(blockData.getHeight())) {
// If this is a pruned block, we likely only have partial data, so best not to sent it
blockData = null;
}
}
// If we have no block data, we should check the archive in case it's there
if (blockData == null) {
if (Settings.getInstance().isArchiveEnabled()) {
byte[] bytes = BlockArchiveReader.getInstance().fetchSerializedBlockBytesForSignature(signature, true, repository);
if (bytes != null) {
CachedBlockMessage blockMessage = new CachedBlockMessage(bytes);
blockMessage.setId(message.getId());
// This call also causes the other needed data to be pulled in from repository
if (!peer.sendMessage(blockMessage)) {
peer.disconnect("failed to send block");
// Don't fall-through to caching because failure to send might be from failure to build message
return;
}
// Sent successfully from archive, so nothing more to do
return;
}
}
}
if (blockData == null) {
// We don't have this block
this.stats.getBlockMessageStats.unknownBlocks.getAndIncrement();
@@ -1406,12 +1546,29 @@ public class Controller extends Thread {
int numberRequested = Math.min(Network.MAX_BLOCK_SUMMARIES_PER_REPLY, getBlockSummariesMessage.getNumberRequested());
BlockData blockData = repository.getBlockRepository().fromReference(parentSignature);
if (blockData == null) {
// Try the archive
blockData = repository.getBlockArchiveRepository().fromReference(parentSignature);
}
if (blockData != null) {
if (PruneManager.getInstance().isBlockPruned(blockData.getHeight())) {
// If this request contains a pruned block, we likely only have partial data, so best not to sent anything
// We always prune from the oldest first, so it's fine to just check the first block requested
blockData = null;
}
}
while (blockData != null && blockSummaries.size() < numberRequested) {
BlockSummaryData blockSummary = new BlockSummaryData(blockData);
blockSummaries.add(blockSummary);
blockData = repository.getBlockRepository().fromReference(blockData.getSignature());
byte[] previousSignature = blockData.getSignature();
blockData = repository.getBlockRepository().fromReference(previousSignature);
if (blockData == null) {
// Try the archive
blockData = repository.getBlockArchiveRepository().fromReference(previousSignature);
}
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending block summaries after %s to peer %s", Base58.encode(parentSignature), peer), e);
@@ -1460,11 +1617,20 @@ public class Controller extends Thread {
try (final Repository repository = RepositoryManager.getRepository()) {
int numberRequested = getSignaturesMessage.getNumberRequested();
BlockData blockData = repository.getBlockRepository().fromReference(parentSignature);
if (blockData == null) {
// Try the archive
blockData = repository.getBlockArchiveRepository().fromReference(parentSignature);
}
while (blockData != null && signatures.size() < numberRequested) {
signatures.add(blockData.getSignature());
blockData = repository.getBlockRepository().fromReference(blockData.getSignature());
byte[] previousSignature = blockData.getSignature();
blockData = repository.getBlockRepository().fromReference(previousSignature);
if (blockData == null) {
// Try the archive
blockData = repository.getBlockArchiveRepository().fromReference(previousSignature);
}
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending V2 signatures after %s to peer %s", Base58.encode(parentSignature), peer), e);

View File

@@ -3,12 +3,9 @@ package org.qortal.controller;
import java.math.BigInteger;
import java.text.DecimalFormat;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.*;
import java.util.concurrent.locks.ReentrantLock;
import java.util.stream.Collectors;
import java.util.Iterator;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -62,6 +59,11 @@ public class Synchronizer {
// Keep track of the size of the last re-org, so it can be logged
private int lastReorgSize;
// Keep track of invalid blocks so that we don't keep trying to sync them
private Map<String, Long> invalidBlockSignatures = Collections.synchronizedMap(new HashMap<>());
public Long timeValidBlockLastReceived = null;
public Long timeInvalidBlockLastReceived = null;
private static Synchronizer instance;
public enum SynchronizationResult {
@@ -337,6 +339,12 @@ public class Synchronizer {
}
}
// Ignore this peer if it holds an invalid block
if (this.containsInvalidBlockSummary(peer.getCommonBlockData().getBlockSummariesAfterCommonBlock())) {
LOGGER.debug("Ignoring peer %s because it holds an invalid block", peer);
peers.remove(peer);
}
// Reduce minChainLength if needed. If we don't have any blocks, this peer will be excluded from chain weight comparisons later in the process, so we shouldn't update minChainLength
List <BlockSummaryData> peerBlockSummaries = peer.getCommonBlockData().getBlockSummariesAfterCommonBlock();
if (peerBlockSummaries != null && peerBlockSummaries.size() > 0)
@@ -480,6 +488,71 @@ public class Synchronizer {
}
/* Invalid block signature tracking */
private void addInvalidBlockSignature(byte[] signature) {
Long now = NTP.getTime();
if (now == null) {
return;
}
// Add or update existing entry
String sig58 = Base58.encode(signature);
invalidBlockSignatures.put(sig58, now);
}
private void deleteOlderInvalidSignatures(Long now) {
if (now == null) {
return;
}
// Delete signatures with older timestamps
Iterator it = invalidBlockSignatures.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
Long lastSeen = (Long) pair.getValue();
// Remove signature if we haven't seen it for more than 1 hour
if (now - lastSeen > 60 * 60 * 1000L) {
it.remove();
}
}
}
private boolean containsInvalidBlockSummary(List<BlockSummaryData> blockSummaries) {
if (blockSummaries == null || invalidBlockSignatures == null) {
return false;
}
// Loop through our known invalid blocks and check each one against supplied block summaries
for (String invalidSignature58 : invalidBlockSignatures.keySet()) {
byte[] invalidSignature = Base58.decode(invalidSignature58);
for (BlockSummaryData blockSummary : blockSummaries) {
byte[] signature = blockSummary.getSignature();
if (Arrays.equals(signature, invalidSignature)) {
return true;
}
}
}
return false;
}
private boolean containsInvalidBlockSignature(List<byte[]> blockSignatures) {
if (blockSignatures == null || invalidBlockSignatures == null) {
return false;
}
// Loop through our known invalid blocks and check each one against supplied block signatures
for (String invalidSignature58 : invalidBlockSignatures.keySet()) {
byte[] invalidSignature = Base58.decode(invalidSignature58);
for (byte[] signature : blockSignatures) {
if (Arrays.equals(signature, invalidSignature)) {
return true;
}
}
}
return false;
}
/**
* Attempt to synchronize blockchain with peer.
* <p>
@@ -526,6 +599,15 @@ public class Synchronizer {
// Reset last re-org size as we are starting a new sync round
this.lastReorgSize = 0;
// Set the initial value of timeValidBlockLastReceived if it's null
Long now = NTP.getTime();
if (this.timeValidBlockLastReceived == null) {
this.timeValidBlockLastReceived = now;
}
// Delete invalid signatures with older timestamps
this.deleteOlderInvalidSignatures(now);
List<BlockSummaryData> peerBlockSummaries = new ArrayList<>();
SynchronizationResult findCommonBlockResult = fetchSummariesFromCommonBlock(repository, peer, ourInitialHeight, force, peerBlockSummaries, true);
if (findCommonBlockResult != SynchronizationResult.OK) {
@@ -874,6 +956,12 @@ public class Synchronizer {
break;
}
// Catch a block with an invalid signature before orphaning, so that we retain our existing valid candidate
if (this.containsInvalidBlockSignature(peerBlockSignatures)) {
LOGGER.info(String.format("Peer %s sent invalid block signature: %.8s", peer, Base58.encode(latestPeerSignature)));
return SynchronizationResult.INVALID_DATA;
}
byte[] nextPeerSignature = peerBlockSignatures.get(0);
int nextHeight = height + 1;
@@ -976,13 +1064,20 @@ public class Synchronizer {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
newBlock.preProcess();
ValidationResult blockResult = newBlock.isValid();
if (blockResult != ValidationResult.OK) {
LOGGER.info(String.format("Peer %s sent invalid block for height %d, sig %.8s: %s", peer,
newBlock.getBlockData().getHeight(), Base58.encode(newBlock.getSignature()), blockResult.name()));
this.addInvalidBlockSignature(newBlock.getSignature());
this.timeInvalidBlockLastReceived = NTP.getTime();
return SynchronizationResult.INVALID_DATA;
}
// Block is valid
this.timeValidBlockLastReceived = NTP.getTime();
// Save transactions attached to this block
for (Transaction transaction : newBlock.getTransactions()) {
TransactionData transactionData = transaction.getTransactionData();
@@ -1064,13 +1159,20 @@ public class Synchronizer {
for (Transaction transaction : newBlock.getTransactions())
transaction.setInitialApprovalStatus();
newBlock.preProcess();
ValidationResult blockResult = newBlock.isValid();
if (blockResult != ValidationResult.OK) {
LOGGER.info(String.format("Peer %s sent invalid block for height %d, sig %.8s: %s", peer,
ourHeight, Base58.encode(latestPeerSignature), blockResult.name()));
this.addInvalidBlockSignature(newBlock.getSignature());
this.timeInvalidBlockLastReceived = NTP.getTime();
return SynchronizationResult.INVALID_DATA;
}
// Block is valid
this.timeValidBlockLastReceived = NTP.getTime();
// Save transactions attached to this block
for (Transaction transaction : newBlock.getTransactions()) {
TransactionData transactionData = transaction.getTransactionData();

View File

@@ -0,0 +1,109 @@
package org.qortal.controller.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.NTP;
public class AtStatesPruner implements Runnable {
private static final Logger LOGGER = LogManager.getLogger(AtStatesPruner.class);
@Override
public void run() {
Thread.currentThread().setName("AT States pruner");
boolean archiveMode = false;
if (!Settings.getInstance().isTopOnly()) {
// Top-only mode isn't enabled, but we might want to prune for the purposes of archiving
if (!Settings.getInstance().isArchiveEnabled()) {
// No pruning or archiving, so we must not prune anything
return;
}
else {
// We're allowed to prune blocks that have already been archived
archiveMode = true;
}
}
try (final Repository repository = RepositoryManager.getRepository()) {
int pruneStartHeight = repository.getATRepository().getAtPruneHeight();
repository.discardChanges();
repository.getATRepository().rebuildLatestAtStates();
while (!Controller.isStopping()) {
repository.discardChanges();
Thread.sleep(Settings.getInstance().getAtStatesPruneInterval());
BlockData chainTip = Controller.getInstance().getChainTip();
if (chainTip == null || NTP.getTime() == null)
continue;
// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
if (Controller.getInstance().isSynchronizing())
continue;
// Prune AT states for all blocks up until our latest minus pruneBlockLimit
final int ourLatestHeight = chainTip.getHeight();
int upperPrunableHeight = ourLatestHeight - Settings.getInstance().getPruneBlockLimit();
// In archive mode we are only allowed to trim blocks that have already been archived
if (archiveMode) {
upperPrunableHeight = repository.getBlockArchiveRepository().getBlockArchiveHeight() - 1;
// TODO: validate that the actual archived data exists before pruning it?
}
int upperBatchHeight = pruneStartHeight + Settings.getInstance().getAtStatesPruneBatchSize();
int upperPruneHeight = Math.min(upperBatchHeight, upperPrunableHeight);
if (pruneStartHeight >= upperPruneHeight)
continue;
LOGGER.debug(String.format("Pruning AT states between blocks %d and %d...", pruneStartHeight, upperPruneHeight));
int numAtStatesPruned = repository.getATRepository().pruneAtStates(pruneStartHeight, upperPruneHeight);
repository.saveChanges();
int numAtStateDataRowsTrimmed = repository.getATRepository().trimAtStates(
pruneStartHeight, upperPruneHeight, Settings.getInstance().getAtStatesTrimLimit());
repository.saveChanges();
if (numAtStatesPruned > 0 || numAtStateDataRowsTrimmed > 0) {
final int finalPruneStartHeight = pruneStartHeight;
LOGGER.debug(() -> String.format("Pruned %d AT state%s between blocks %d and %d",
numAtStatesPruned, (numAtStatesPruned != 1 ? "s" : ""),
finalPruneStartHeight, upperPruneHeight));
} else {
// Can we move onto next batch?
if (upperPrunableHeight > upperBatchHeight) {
pruneStartHeight = upperBatchHeight;
repository.getATRepository().setAtPruneHeight(pruneStartHeight);
repository.getATRepository().rebuildLatestAtStates();
repository.saveChanges();
final int finalPruneStartHeight = pruneStartHeight;
LOGGER.debug(() -> String.format("Bumping AT state base prune height to %d", finalPruneStartHeight));
}
else {
// We've pruned up to the upper prunable height
// Back off for a while to save CPU for syncing
repository.discardChanges();
Thread.sleep(5*60*1000L);
}
}
}
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to prune AT states: %s", e.getMessage()));
} catch (InterruptedException e) {
// Time to exit
}
}
}

View File

@@ -1,7 +1,8 @@
package org.qortal.controller;
package org.qortal.controller.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
@@ -20,8 +21,8 @@ public class AtStatesTrimmer implements Runnable {
try (final Repository repository = RepositoryManager.getRepository()) {
int trimStartHeight = repository.getATRepository().getAtTrimHeight();
repository.getATRepository().prepareForAtStateTrimming();
repository.saveChanges();
repository.discardChanges();
repository.getATRepository().rebuildLatestAtStates();
while (!Controller.isStopping()) {
repository.discardChanges();
@@ -62,7 +63,7 @@ public class AtStatesTrimmer implements Runnable {
if (upperTrimmableHeight > upperBatchHeight) {
trimStartHeight = upperBatchHeight;
repository.getATRepository().setAtTrimHeight(trimStartHeight);
repository.getATRepository().prepareForAtStateTrimming();
repository.getATRepository().rebuildLatestAtStates();
repository.saveChanges();
final int finalTrimStartHeight = trimStartHeight;

View File

@@ -0,0 +1,113 @@
package org.qortal.controller.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.repository.*;
import org.qortal.settings.Settings;
import org.qortal.transform.TransformationException;
import org.qortal.utils.NTP;
import java.io.IOException;
public class BlockArchiver implements Runnable {
private static final Logger LOGGER = LogManager.getLogger(BlockArchiver.class);
private static final long INITIAL_SLEEP_PERIOD = 0L; // TODO: 5 * 60 * 1000L + 1234L; // ms
public void run() {
Thread.currentThread().setName("Block archiver");
if (!Settings.getInstance().isArchiveEnabled()) {
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Don't even start building until initial rush has ended
Thread.sleep(INITIAL_SLEEP_PERIOD);
int startHeight = repository.getBlockArchiveRepository().getBlockArchiveHeight();
// Don't attempt to archive if we have no ATStatesHeightIndex, as it will be too slow
boolean hasAtStatesHeightIndex = repository.getATRepository().hasAtStatesHeightIndex();
if (!hasAtStatesHeightIndex) {
LOGGER.info("Unable to start block archiver due to missing ATStatesHeightIndex. Bootstrapping is recommended.");
repository.discardChanges();
return;
}
LOGGER.info("Starting block archiver from height {}...", startHeight);
while (!Controller.isStopping()) {
repository.discardChanges();
Thread.sleep(Settings.getInstance().getArchiveInterval());
BlockData chainTip = Controller.getInstance().getChainTip();
if (chainTip == null || NTP.getTime() == null) {
continue;
}
// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
if (Controller.getInstance().isSynchronizing()) {
continue;
}
// Don't attempt to archive if we're not synced yet
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp == null || chainTip.getTimestamp() < minLatestBlockTimestamp) {
continue;
}
// Build cache of blocks
try {
final int maximumArchiveHeight = BlockArchiveWriter.getMaxArchiveHeight(repository);
BlockArchiveWriter writer = new BlockArchiveWriter(startHeight, maximumArchiveHeight, repository);
BlockArchiveWriter.BlockArchiveWriteResult result = writer.write();
switch (result) {
case OK:
// Increment block archive height
startHeight += writer.getWrittenCount();
repository.getBlockArchiveRepository().setBlockArchiveHeight(startHeight);
repository.saveChanges();
break;
case STOPPING:
return;
// We've reached the limit of the blocks we can archive
// Sleep for a while to allow more to become available
case NOT_ENOUGH_BLOCKS:
// We didn't reach our file size target, so that must mean that we don't have enough blocks
// yet or something went wrong. Sleep for a while and then try again.
repository.discardChanges();
Thread.sleep(60 * 60 * 1000L); // 1 hour
break;
case BLOCK_NOT_FOUND:
// We tried to archive a block that didn't exist. This is a major failure and likely means
// that a bootstrap or re-sync is needed. Try again every minute until then.
LOGGER.info("Error: block not found when building archive. If this error persists, " +
"a bootstrap or re-sync may be needed.");
repository.discardChanges();
Thread.sleep( 60 * 1000L); // 1 minute
break;
}
} catch (IOException | TransformationException e) {
LOGGER.info("Caught exception when creating block cache", e);
}
}
} catch (DataException e) {
LOGGER.info("Caught exception when creating block cache", e);
} catch (InterruptedException e) {
// Do nothing
}
}
}

View File

@@ -0,0 +1,114 @@
package org.qortal.controller.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.NTP;
public class BlockPruner implements Runnable {
private static final Logger LOGGER = LogManager.getLogger(BlockPruner.class);
@Override
public void run() {
Thread.currentThread().setName("Block pruner");
boolean archiveMode = false;
if (!Settings.getInstance().isTopOnly()) {
// Top-only mode isn't enabled, but we might want to prune for the purposes of archiving
if (!Settings.getInstance().isArchiveEnabled()) {
// No pruning or archiving, so we must not prune anything
return;
}
else {
// We're allowed to prune blocks that have already been archived
archiveMode = true;
}
}
try (final Repository repository = RepositoryManager.getRepository()) {
int pruneStartHeight = repository.getBlockRepository().getBlockPruneHeight();
// Don't attempt to prune if we have no ATStatesHeightIndex, as it will be too slow
boolean hasAtStatesHeightIndex = repository.getATRepository().hasAtStatesHeightIndex();
if (!hasAtStatesHeightIndex) {
LOGGER.info("Unable to start block pruner due to missing ATStatesHeightIndex. Bootstrapping is recommended.");
return;
}
while (!Controller.isStopping()) {
repository.discardChanges();
Thread.sleep(Settings.getInstance().getBlockPruneInterval());
BlockData chainTip = Controller.getInstance().getChainTip();
if (chainTip == null || NTP.getTime() == null)
continue;
// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
if (Controller.getInstance().isSynchronizing()) {
continue;
}
// Don't attempt to prune if we're not synced yet
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp == null || chainTip.getTimestamp() < minLatestBlockTimestamp) {
continue;
}
// Prune all blocks up until our latest minus pruneBlockLimit
final int ourLatestHeight = chainTip.getHeight();
int upperPrunableHeight = ourLatestHeight - Settings.getInstance().getPruneBlockLimit();
// In archive mode we are only allowed to trim blocks that have already been archived
if (archiveMode) {
upperPrunableHeight = repository.getBlockArchiveRepository().getBlockArchiveHeight() - 1;
}
int upperBatchHeight = pruneStartHeight + Settings.getInstance().getBlockPruneBatchSize();
int upperPruneHeight = Math.min(upperBatchHeight, upperPrunableHeight);
if (pruneStartHeight >= upperPruneHeight) {
continue;
}
LOGGER.debug(String.format("Pruning blocks between %d and %d...", pruneStartHeight, upperPruneHeight));
int numBlocksPruned = repository.getBlockRepository().pruneBlocks(pruneStartHeight, upperPruneHeight);
repository.saveChanges();
if (numBlocksPruned > 0) {
LOGGER.debug(String.format("Pruned %d block%s between %d and %d",
numBlocksPruned, (numBlocksPruned != 1 ? "s" : ""),
pruneStartHeight, upperPruneHeight));
} else {
final int nextPruneHeight = upperPruneHeight + 1;
repository.getBlockRepository().setBlockPruneHeight(nextPruneHeight);
repository.saveChanges();
LOGGER.debug(String.format("Bumping block base prune height to %d", pruneStartHeight));
// Can we move onto next batch?
if (upperPrunableHeight > nextPruneHeight) {
pruneStartHeight = nextPruneHeight;
}
else {
// We've pruned up to the upper prunable height
// Back off for a while to save CPU for syncing
repository.discardChanges();
Thread.sleep(10*60*1000L);
}
}
}
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to prune blocks: %s", e.getMessage()));
} catch (InterruptedException e) {
// Time to exit
}
}
}

View File

@@ -0,0 +1,410 @@
package org.qortal.controller.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.resource.TransactionsResource.ConfirmationStatus;
import org.qortal.data.naming.NameData;
import org.qortal.data.transaction.*;
import org.qortal.naming.Name;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transaction.Transaction.TransactionType;
import org.qortal.utils.Unicode;
import java.util.*;
public class NamesDatabaseIntegrityCheck {
private static final Logger LOGGER = LogManager.getLogger(NamesDatabaseIntegrityCheck.class);
private static final List<TransactionType> ALL_NAME_TX_TYPE = Arrays.asList(
TransactionType.REGISTER_NAME,
TransactionType.UPDATE_NAME,
TransactionType.BUY_NAME,
TransactionType.SELL_NAME
);
private List<TransactionData> nameTransactions = new ArrayList<>();
public int rebuildName(String name, Repository repository) {
int modificationCount = 0;
try {
List<TransactionData> transactions = this.fetchAllTransactionsInvolvingName(name, repository);
if (transactions.isEmpty()) {
// This name was never registered, so there's nothing to do
return modificationCount;
}
// Loop through each past transaction and re-apply it to the Names table
for (TransactionData currentTransaction : transactions) {
// Process REGISTER_NAME transactions
if (currentTransaction.getType() == TransactionType.REGISTER_NAME) {
RegisterNameTransactionData registerNameTransactionData = (RegisterNameTransactionData) currentTransaction;
Name nameObj = new Name(repository, registerNameTransactionData);
nameObj.register();
modificationCount++;
LOGGER.trace("Processed REGISTER_NAME transaction for name {}", name);
}
// Process UPDATE_NAME transactions
if (currentTransaction.getType() == TransactionType.UPDATE_NAME) {
UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) currentTransaction;
if (Objects.equals(updateNameTransactionData.getNewName(), name) &&
!Objects.equals(updateNameTransactionData.getName(), updateNameTransactionData.getNewName())) {
// This renames an existing name, so we need to process that instead
this.rebuildName(updateNameTransactionData.getName(), repository);
}
else {
Name nameObj = new Name(repository, name);
if (nameObj != null && nameObj.getNameData() != null) {
nameObj.update(updateNameTransactionData);
modificationCount++;
LOGGER.trace("Processed UPDATE_NAME transaction for name {}", name);
} else {
// Something went wrong
throw new DataException(String.format("Name data not found for name %s", updateNameTransactionData.getName()));
}
}
}
// Process SELL_NAME transactions
if (currentTransaction.getType() == TransactionType.SELL_NAME) {
SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) currentTransaction;
Name nameObj = new Name(repository, sellNameTransactionData.getName());
if (nameObj != null && nameObj.getNameData() != null) {
nameObj.sell(sellNameTransactionData);
modificationCount++;
LOGGER.trace("Processed SELL_NAME transaction for name {}", name);
}
else {
// Something went wrong
throw new DataException(String.format("Name data not found for name %s", sellNameTransactionData.getName()));
}
}
// Process BUY_NAME transactions
if (currentTransaction.getType() == TransactionType.BUY_NAME) {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) currentTransaction;
Name nameObj = new Name(repository, buyNameTransactionData.getName());
if (nameObj != null && nameObj.getNameData() != null) {
nameObj.buy(buyNameTransactionData);
modificationCount++;
LOGGER.trace("Processed BUY_NAME transaction for name {}", name);
}
else {
// Something went wrong
throw new DataException(String.format("Name data not found for name %s", buyNameTransactionData.getName()));
}
}
}
} catch (DataException e) {
LOGGER.info("Unable to run integrity check for name {}: {}", name, e.getMessage());
}
return modificationCount;
}
public int rebuildAllNames() {
int modificationCount = 0;
try (final Repository repository = RepositoryManager.getRepository()) {
List<String> names = this.fetchAllNames(repository);
for (String name : names) {
modificationCount += this.rebuildName(name, repository);
}
repository.saveChanges();
}
catch (DataException e) {
LOGGER.info("Error when running integrity check for all names: {}", e.getMessage());
}
//LOGGER.info("modificationCount: {}", modificationCount);
return modificationCount;
}
public void runIntegrityCheck() {
boolean integrityCheckFailed = false;
try (final Repository repository = RepositoryManager.getRepository()) {
// Fetch all the (confirmed) REGISTER_NAME transactions
List<RegisterNameTransactionData> registerNameTransactions = this.fetchRegisterNameTransactions();
// Loop through each REGISTER_NAME txn signature and request the full transaction data
for (RegisterNameTransactionData registerNameTransactionData : registerNameTransactions) {
String registeredName = registerNameTransactionData.getName();
NameData nameData = repository.getNameRepository().fromName(registeredName);
// Check to see if this name has been updated or bought at any point
TransactionData latestUpdate = this.fetchLatestModificationTransactionInvolvingName(registeredName, repository);
if (latestUpdate == null) {
// Name was never updated once registered
// We expect this name to still be registered to this transaction's creator
if (nameData == null) {
LOGGER.info("Error: registered name {} doesn't exist in Names table. Adding...", registeredName);
integrityCheckFailed = true;
}
else {
LOGGER.trace("Registered name {} is correctly registered", registeredName);
}
// Check the owner is correct
PublicKeyAccount creator = new PublicKeyAccount(repository, registerNameTransactionData.getCreatorPublicKey());
if (!Objects.equals(creator.getAddress(), nameData.getOwner())) {
LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
registeredName, nameData.getOwner(), creator.getAddress());
integrityCheckFailed = true;
}
else {
LOGGER.trace("Registered name {} has the correct owner", registeredName);
}
}
else {
// Check if owner is correct after update
// Check for name updates
if (latestUpdate.getType() == TransactionType.UPDATE_NAME) {
UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) latestUpdate;
PublicKeyAccount creator = new PublicKeyAccount(repository, updateNameTransactionData.getCreatorPublicKey());
// When this name is the "new name", we expect the current owner to match the txn creator
if (Objects.equals(updateNameTransactionData.getNewName(), registeredName)) {
if (!Objects.equals(creator.getAddress(), nameData.getOwner())) {
LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
registeredName, nameData.getOwner(), creator.getAddress());
integrityCheckFailed = true;
}
else {
LOGGER.trace("Registered name {} has the correct owner after being updated", registeredName);
}
}
// When this name is the old name, we expect the "new name"'s owner to match the txn creator
// The old name will then be unregistered, or re-registered.
// FUTURE: check database integrity for names that have been updated and then the original name re-registered
else if (Objects.equals(updateNameTransactionData.getName(), registeredName)) {
NameData newNameData = repository.getNameRepository().fromName(updateNameTransactionData.getNewName());
if (!Objects.equals(creator.getAddress(), newNameData.getOwner())) {
LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
updateNameTransactionData.getNewName(), newNameData.getOwner(), creator.getAddress());
integrityCheckFailed = true;
}
else {
LOGGER.trace("Registered name {} has the correct owner after being updated", updateNameTransactionData.getNewName());
}
}
else {
LOGGER.info("Unhandled update case for name {}", registeredName);
}
}
// Check for name buys
else if (latestUpdate.getType() == TransactionType.BUY_NAME) {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) latestUpdate;
PublicKeyAccount creator = new PublicKeyAccount(repository, buyNameTransactionData.getCreatorPublicKey());
if (!Objects.equals(creator.getAddress(), nameData.getOwner())) {
LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
registeredName, nameData.getOwner(), creator.getAddress());
integrityCheckFailed = true;
}
else {
LOGGER.trace("Registered name {} has the correct owner after being bought", registeredName);
}
}
// Check for name sells
else if (latestUpdate.getType() == TransactionType.SELL_NAME) {
SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) latestUpdate;
PublicKeyAccount creator = new PublicKeyAccount(repository, sellNameTransactionData.getCreatorPublicKey());
if (!Objects.equals(creator.getAddress(), nameData.getOwner())) {
LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
registeredName, nameData.getOwner(), creator.getAddress());
integrityCheckFailed = true;
}
else {
LOGGER.trace("Registered name {} has the correct owner after being listed for sale", registeredName);
}
}
else {
LOGGER.info("Unhandled case for name {}", registeredName);
}
}
}
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to trim online accounts signatures: %s", e.getMessage()));
integrityCheckFailed = true;
}
if (integrityCheckFailed) {
LOGGER.info("Registered names database integrity check failed. Bootstrapping is recommended.");
} else {
LOGGER.info("Registered names database integrity check passed.");
}
}
private List<RegisterNameTransactionData> fetchRegisterNameTransactions() {
List<RegisterNameTransactionData> registerNameTransactions = new ArrayList<>();
for (TransactionData transactionData : this.nameTransactions) {
if (transactionData.getType() == TransactionType.REGISTER_NAME) {
RegisterNameTransactionData registerNameTransactionData = (RegisterNameTransactionData) transactionData;
registerNameTransactions.add(registerNameTransactionData);
}
}
return registerNameTransactions;
}
private List<UpdateNameTransactionData> fetchUpdateNameTransactions() {
List<UpdateNameTransactionData> updateNameTransactions = new ArrayList<>();
for (TransactionData transactionData : this.nameTransactions) {
if (transactionData.getType() == TransactionType.UPDATE_NAME) {
UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) transactionData;
updateNameTransactions.add(updateNameTransactionData);
}
}
return updateNameTransactions;
}
private List<SellNameTransactionData> fetchSellNameTransactions() {
List<SellNameTransactionData> sellNameTransactions = new ArrayList<>();
for (TransactionData transactionData : this.nameTransactions) {
if (transactionData.getType() == TransactionType.SELL_NAME) {
SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) transactionData;
sellNameTransactions.add(sellNameTransactionData);
}
}
return sellNameTransactions;
}
private List<BuyNameTransactionData> fetchBuyNameTransactions() {
List<BuyNameTransactionData> buyNameTransactions = new ArrayList<>();
for (TransactionData transactionData : this.nameTransactions) {
if (transactionData.getType() == TransactionType.BUY_NAME) {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) transactionData;
buyNameTransactions.add(buyNameTransactionData);
}
}
return buyNameTransactions;
}
private void fetchAllNameTransactions(Repository repository) throws DataException {
List<TransactionData> nameTransactions = new ArrayList<>();
// Fetch all the confirmed REGISTER_NAME transaction signatures
List<byte[]> signatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(
null, null, null, ALL_NAME_TX_TYPE, null, null,
ConfirmationStatus.CONFIRMED, null, null, false);
for (byte[] signature : signatures) {
TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
nameTransactions.add(transactionData);
}
this.nameTransactions = nameTransactions;
}
private List<TransactionData> fetchAllTransactionsInvolvingName(String name, Repository repository) throws DataException {
List<TransactionData> transactions = new ArrayList<>();
String reducedName = Unicode.sanitize(name);
// Fetch all the confirmed name-modification transactions
if (this.nameTransactions.isEmpty()) {
this.fetchAllNameTransactions(repository);
}
for (TransactionData transactionData : this.nameTransactions) {
if ((transactionData instanceof RegisterNameTransactionData)) {
RegisterNameTransactionData registerNameTransactionData = (RegisterNameTransactionData) transactionData;
if (Objects.equals(registerNameTransactionData.getReducedName(), reducedName)) {
transactions.add(transactionData);
}
}
if ((transactionData instanceof UpdateNameTransactionData)) {
UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) transactionData;
if (Objects.equals(updateNameTransactionData.getName(), name) ||
Objects.equals(updateNameTransactionData.getReducedNewName(), reducedName)) {
transactions.add(transactionData);
}
}
if ((transactionData instanceof BuyNameTransactionData)) {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) transactionData;
if (Objects.equals(buyNameTransactionData.getName(), name)) {
transactions.add(transactionData);
}
}
if ((transactionData instanceof SellNameTransactionData)) {
SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) transactionData;
if (Objects.equals(sellNameTransactionData.getName(), name)) {
transactions.add(transactionData);
}
}
}
return transactions;
}
private TransactionData fetchLatestModificationTransactionInvolvingName(String registeredName, Repository repository) throws DataException {
List<TransactionData> transactionsInvolvingName = this.fetchAllTransactionsInvolvingName(registeredName, repository);
// Get the latest update for this name (excluding REGISTER_NAME transactions)
TransactionData latestUpdateToName = transactionsInvolvingName.stream()
.filter(txn -> txn.getType() != TransactionType.REGISTER_NAME)
.max(Comparator.comparing(TransactionData::getTimestamp))
.orElse(null);
return latestUpdateToName;
}
private List<String> fetchAllNames(Repository repository) throws DataException {
List<String> names = new ArrayList<>();
// Fetch all the confirmed name transactions
if (this.nameTransactions.isEmpty()) {
this.fetchAllNameTransactions(repository);
}
for (TransactionData transactionData : this.nameTransactions) {
if ((transactionData instanceof RegisterNameTransactionData)) {
RegisterNameTransactionData registerNameTransactionData = (RegisterNameTransactionData) transactionData;
if (!names.contains(registerNameTransactionData.getName())) {
names.add(registerNameTransactionData.getName());
}
}
if ((transactionData instanceof UpdateNameTransactionData)) {
UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) transactionData;
if (!names.contains(updateNameTransactionData.getName())) {
names.add(updateNameTransactionData.getName());
}
if (!names.contains(updateNameTransactionData.getNewName())) {
names.add(updateNameTransactionData.getNewName());
}
}
if ((transactionData instanceof BuyNameTransactionData)) {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) transactionData;
if (!names.contains(buyNameTransactionData.getName())) {
names.add(buyNameTransactionData.getName());
}
}
if ((transactionData instanceof SellNameTransactionData)) {
SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) transactionData;
if (!names.contains(sellNameTransactionData.getName())) {
names.add(sellNameTransactionData.getName());
}
}
}
return names;
}
}

View File

@@ -1,8 +1,9 @@
package org.qortal.controller;
package org.qortal.controller.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.block.BlockChain;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;

View File

@@ -0,0 +1,160 @@
package org.qortal.controller.repository;
import org.apache.commons.io.FileUtils;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.settings.Settings;
import org.qortal.utils.DaemonThreadFactory;
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class PruneManager {
private static final Logger LOGGER = LogManager.getLogger(PruneManager.class);
private static PruneManager instance;
private boolean isTopOnly = Settings.getInstance().isTopOnly();
private int pruneBlockLimit = Settings.getInstance().getPruneBlockLimit();
private ExecutorService executorService;
private PruneManager() {
}
public static synchronized PruneManager getInstance() {
if (instance == null)
instance = new PruneManager();
return instance;
}
public void start() {
this.executorService = Executors.newCachedThreadPool(new DaemonThreadFactory());
if (Settings.getInstance().isTopOnly()) {
// Top-only-sync
this.startTopOnlySyncMode();
}
else if (Settings.getInstance().isArchiveEnabled()) {
// Full node with block archive
this.startFullNodeWithBlockArchive();
}
else {
// Full node with full SQL support
this.startFullSQLNode();
}
}
/**
* Top-only-sync
* In this mode, we delete (prune) all blocks except
* a small number of recent ones. There is no need for
* trimming or archiving, because all relevant blocks
* are deleted.
*/
private void startTopOnlySyncMode() {
this.startPruning();
// We don't need the block archive in top-only mode
this.deleteArchive();
}
/**
* Full node with block archive
* In this mode we archive trimmed blocks, and then
* prune archived blocks to keep the database small
*/
private void startFullNodeWithBlockArchive() {
this.startTrimming();
this.startArchiving();
this.startPruning();
}
/**
* Full node with full SQL support
* In this mode we trim the database but don't prune
* or archive any data, because we want to maintain
* full SQL support of old blocks. This mode will not
* be actively maintained but can be used by those who
* need to perform SQL analysis on older blocks.
*/
private void startFullSQLNode() {
this.startTrimming();
}
private void startPruning() {
this.executorService.execute(new AtStatesPruner());
this.executorService.execute(new BlockPruner());
}
private void startTrimming() {
this.executorService.execute(new AtStatesTrimmer());
this.executorService.execute(new OnlineAccountsSignaturesTrimmer());
}
private void startArchiving() {
this.executorService.execute(new BlockArchiver());
}
private void deleteArchive() {
if (!Settings.getInstance().isTopOnly()) {
LOGGER.error("Refusing to delete archive when not in top-only mode");
}
try {
Path archivePath = Paths.get(Settings.getInstance().getRepositoryPath(), "archive");
if (archivePath.toFile().exists()) {
LOGGER.info("Deleting block archive because we are in top-only mode...");
FileUtils.deleteDirectory(archivePath.toFile());
}
} catch (IOException e) {
LOGGER.info("Couldn't delete archive: {}", e.getMessage());
}
}
public void stop() {
this.executorService.shutdownNow();
try {
this.executorService.awaitTermination(2L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// We tried...
}
}
public boolean isBlockPruned(int height) throws DataException {
if (!this.isTopOnly) {
return false;
}
BlockData chainTip = Controller.getInstance().getChainTip();
if (chainTip == null) {
throw new DataException("Unable to determine chain tip when checking if a block is pruned");
}
if (height == 1) {
// We don't prune the genesis block
return false;
}
final int ourLatestHeight = chainTip.getHeight();
final int latestUnprunedHeight = ourLatestHeight - this.pruneBlockLimit;
return (height < latestUnprunedHeight);
}
}

View File

@@ -360,6 +360,7 @@ public class BitcoinACCTv1TradeBot implements AcctTradeBot {
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:

View File

@@ -201,7 +201,7 @@ public class DogecoinACCTv1TradeBot implements AcctTradeBot {
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
@@ -276,7 +276,8 @@ public class DogecoinACCTv1TradeBot implements AcctTradeBot {
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
@@ -353,6 +354,7 @@ public class DogecoinACCTv1TradeBot implements AcctTradeBot {
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:

View File

@@ -0,0 +1,884 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class DogecoinACCTv2TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv2TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static DogecoinACCTv2TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private DogecoinACCTv2TradeBot() {
}
public static synchronized DogecoinACCTv2TradeBot getInstance() {
if (instance == null)
instance = new DogecoinACCTv2TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for DOGE.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Dogecoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Dogecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>DOGE amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Dogecoin receiving address into public key hash (we only support P2PKH at this time)
Address dogecoinReceivingAddress;
try {
dogecoinReceivingAddress = Address.fromString(Dogecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (dogecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] dogecoinReceivingAccountInfo = dogecoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/DOGE ACCT";
String description = "QORT/DOGE cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT DOGE";
byte[] creationBytes = DogecoinACCTv2.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DogecoinACCTv2.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.DOGECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, dogecoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching DOGE to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Dogecoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Dogecoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Dogecoin main-net)
* or 'tprv' for (Dogecoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Dogecoin amount expected by 'Bob'.
* <p>
* If the Dogecoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Dogecoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DogecoinACCTv2.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.DOGECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Dogecoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Dogecoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Dogecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Dogecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Dogecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = DogecoinACCTv2.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = DogecoinACCTv2.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Dogecoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Dogecoin dogecoin = Dogecoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Dogecoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
DogecoinACCTv2.OfferMessageData offerMessageData = DogecoinACCTv2.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerDogecoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = DogecoinACCTv2.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = DogecoinACCTv2.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the DOGE funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Dogecoin dogecoin = Dogecoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = DogecoinACCTv2.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = DogecoinACCTv2.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the DOGE funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the DOGE funds from P2SH-A
* to Bob's 'foreign'/Dogecoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send DOGE to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the DOGE
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = DogecoinACCTv2.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Dogecoin dogecoin = Dogecoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(dogecoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
dogecoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = dogecoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Dogecoin dogecoin = Dogecoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = dogecoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = dogecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(dogecoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(dogecoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
dogecoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,885 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class DogecoinACCTv3TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv3TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static DogecoinACCTv3TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private DogecoinACCTv3TradeBot() {
}
public static synchronized DogecoinACCTv3TradeBot getInstance() {
if (instance == null)
instance = new DogecoinACCTv3TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for DOGE.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Dogecoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Dogecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>DOGE amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Dogecoin receiving address into public key hash (we only support P2PKH at this time)
Address dogecoinReceivingAddress;
try {
dogecoinReceivingAddress = Address.fromString(Dogecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (dogecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] dogecoinReceivingAccountInfo = dogecoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/DOGE ACCT";
String description = "QORT/DOGE cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT DOGE";
byte[] creationBytes = DogecoinACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DogecoinACCTv3.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.DOGECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, dogecoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching DOGE to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Dogecoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Dogecoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Dogecoin main-net)
* or 'tprv' for (Dogecoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Dogecoin amount expected by 'Bob'.
* <p>
* If the Dogecoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Dogecoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DogecoinACCTv3.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.DOGECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Dogecoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Dogecoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Dogecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Dogecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Dogecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = DogecoinACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = DogecoinACCTv3.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Dogecoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Dogecoin dogecoin = Dogecoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Dogecoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
DogecoinACCTv3.OfferMessageData offerMessageData = DogecoinACCTv3.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerDogecoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = DogecoinACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = DogecoinACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the DOGE funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Dogecoin dogecoin = Dogecoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = DogecoinACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = DogecoinACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the DOGE funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the DOGE funds from P2SH-A
* to Bob's 'foreign'/Dogecoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send DOGE to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the DOGE
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = DogecoinACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Dogecoin dogecoin = Dogecoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(dogecoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
dogecoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = dogecoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Dogecoin dogecoin = Dogecoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = dogecoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = dogecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(dogecoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(dogecoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
dogecoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -212,7 +212,7 @@ public class LitecoinACCTv1TradeBot implements AcctTradeBot {
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
@@ -287,7 +287,8 @@ public class LitecoinACCTv1TradeBot implements AcctTradeBot {
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
@@ -364,6 +365,7 @@ public class LitecoinACCTv1TradeBot implements AcctTradeBot {
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:

View File

@@ -0,0 +1,885 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class LitecoinACCTv2TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(LitecoinACCTv2TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static LitecoinACCTv2TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private LitecoinACCTv2TradeBot() {
}
public static synchronized LitecoinACCTv2TradeBot getInstance() {
if (instance == null)
instance = new LitecoinACCTv2TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for LTC.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Litecoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Litecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>LTC amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Litecoin receiving address into public key hash (we only support P2PKH at this time)
Address litecoinReceivingAddress;
try {
litecoinReceivingAddress = Address.fromString(Litecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (litecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] litecoinReceivingAccountInfo = litecoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/LTC ACCT";
String description = "QORT/LTC cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT LTC";
byte[] creationBytes = LitecoinACCTv2.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, LitecoinACCTv2.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.LITECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, litecoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching LTC to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Litecoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Litecoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Litecoin main-net)
* or 'tprv' for (Litecoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Litecoin amount expected by 'Bob'.
* <p>
* If the Litecoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Litecoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, LitecoinACCTv2.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.LITECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Litecoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Litecoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Litecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Litecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Litecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = LitecoinACCTv2.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = LitecoinACCTv2.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Litecoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Litecoin litecoin = Litecoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Litecoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
LitecoinACCTv2.OfferMessageData offerMessageData = LitecoinACCTv2.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerLitecoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = LitecoinACCTv2.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = LitecoinACCTv2.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the LTC funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Litecoin litecoin = Litecoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = LitecoinACCTv2.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = LitecoinACCTv2.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the LTC funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the LTC funds from P2SH-A
* to Bob's 'foreign'/Litecoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send LTC to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the LTC
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = LitecoinACCTv2.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Litecoin litecoin = Litecoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(litecoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
litecoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = litecoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Litecoin litecoin = Litecoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = litecoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = litecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(litecoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(litecoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
litecoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,885 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class LitecoinACCTv3TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(LitecoinACCTv3TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static LitecoinACCTv3TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private LitecoinACCTv3TradeBot() {
}
public static synchronized LitecoinACCTv3TradeBot getInstance() {
if (instance == null)
instance = new LitecoinACCTv3TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for LTC.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Litecoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Litecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>LTC amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Litecoin receiving address into public key hash (we only support P2PKH at this time)
Address litecoinReceivingAddress;
try {
litecoinReceivingAddress = Address.fromString(Litecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (litecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] litecoinReceivingAccountInfo = litecoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/LTC ACCT";
String description = "QORT/LTC cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT LTC";
byte[] creationBytes = LitecoinACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, LitecoinACCTv3.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.LITECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, litecoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching LTC to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Litecoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Litecoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Litecoin main-net)
* or 'tprv' for (Litecoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Litecoin amount expected by 'Bob'.
* <p>
* If the Litecoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Litecoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, LitecoinACCTv3.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.LITECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Litecoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Litecoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Litecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Litecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Litecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = LitecoinACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = LitecoinACCTv3.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Litecoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Litecoin litecoin = Litecoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Litecoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
LitecoinACCTv3.OfferMessageData offerMessageData = LitecoinACCTv3.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerLitecoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = LitecoinACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = LitecoinACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the LTC funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Litecoin litecoin = Litecoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = LitecoinACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = LitecoinACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the LTC funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the LTC funds from P2SH-A
* to Bob's 'foreign'/Litecoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send LTC to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the LTC
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = LitecoinACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Litecoin litecoin = Litecoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(litecoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
litecoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = litecoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Litecoin litecoin = Litecoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = litecoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = litecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(litecoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(litecoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
litecoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -31,6 +31,7 @@ import org.qortal.gui.SysTray;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.repository.hsqldb.HSQLDBImportExport;
import org.qortal.settings.Settings;
import org.qortal.transaction.PresenceTransaction;
import org.qortal.transaction.PresenceTransaction.PresenceType;
@@ -76,7 +77,11 @@ public class TradeBot implements Listener {
static {
acctTradeBotSuppliers.put(BitcoinACCTv1.class, BitcoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv1.class, LitecoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv2.class, LitecoinACCTv2TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv3.class, LitecoinACCTv3TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv1.class, DogecoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv2.class, DogecoinACCTv2TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv3.class, DogecoinACCTv3TradeBot::getInstance);
}
private static TradeBot instance;
@@ -245,17 +250,17 @@ public class TradeBot implements Listener {
}
}
/*package*/ static byte[] generateTradePrivateKey() {
public static byte[] generateTradePrivateKey() {
// The private key is used for both Curve25519 and secp256k1 so needs to be valid for both.
// Curve25519 accepts any seed, so generate a valid secp256k1 key and use that.
return new ECKey().getPrivKeyBytes();
}
/*package*/ static byte[] deriveTradeNativePublicKey(byte[] privateKey) {
public static byte[] deriveTradeNativePublicKey(byte[] privateKey) {
return PrivateKeyAccount.toPublicKey(privateKey);
}
/*package*/ static byte[] deriveTradeForeignPublicKey(byte[] privateKey) {
public static byte[] deriveTradeForeignPublicKey(byte[] privateKey) {
return ECKey.fromPrivate(privateKey).getPubKey();
}
@@ -265,11 +270,11 @@ public class TradeBot implements Listener {
return secret;
}
/*package*/ static void backupTradeBotData(Repository repository) {
/*package*/ static void backupTradeBotData(Repository repository, List<TradeBotData> additional) {
// Attempt to backup the trade bot data. This an optional step and doesn't impact trading, so don't throw an exception on failure
try {
LOGGER.info("About to backup trade bot data...");
repository.exportNodeLocalData();
HSQLDBImportExport.backupTradeBotStates(repository, additional);
} catch (DataException e) {
LOGGER.info(String.format("Repository issue when exporting trade bot data: %s", e.getMessage()));
}

View File

@@ -42,36 +42,30 @@ public class Bitcoin extends Bitcoiny {
public Collection<ElectrumX.Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("128.0.190.26", Server.ConnectionType.SSL, 50002),
new Server("hodlers.beer", Server.ConnectionType.SSL, 50002),
new Server("electrumx.erbium.eu", Server.ConnectionType.TCP, 50001),
new Server("electrumx.erbium.eu", Server.ConnectionType.SSL, 50002),
new Server("btc.lastingcoin.net", Server.ConnectionType.SSL, 50002),
new Server("electrum.bitaroo.net", Server.ConnectionType.SSL, 50002),
new Server("bitcoin.grey.pw", Server.ConnectionType.SSL, 50002),
new Server("2electrumx.hopto.me", Server.ConnectionType.SSL, 56022),
new Server("185.64.116.15", Server.ConnectionType.SSL, 50002),
new Server("kirsche.emzy.de", Server.ConnectionType.SSL, 50002),
new Server("alviss.coinjoined.com", Server.ConnectionType.SSL, 50002),
new Server("electrum.emzy.de", Server.ConnectionType.SSL, 50002),
new Server("electrum.emzy.de", Server.ConnectionType.TCP, 50001),
new Server("vmd71287.contaboserver.net", Server.ConnectionType.SSL, 50002),
new Server("btc.litepay.ch", Server.ConnectionType.SSL, 50002),
new Server("electrum.stippy.com", Server.ConnectionType.SSL, 50002),
new Server("xtrum.com", Server.ConnectionType.SSL, 50002),
new Server("electrum.acinq.co", Server.ConnectionType.SSL, 50002),
new Server("electrum2.taborsky.cz", Server.ConnectionType.SSL, 50002),
new Server("vmd63185.contaboserver.net", Server.ConnectionType.SSL, 50002),
new Server("electrum2.privateservers.network", Server.ConnectionType.SSL, 50002),
new Server("electrumx.alexridevski.net", Server.ConnectionType.SSL, 50002),
new Server("192.166.219.200", Server.ConnectionType.SSL, 50002),
new Server("2ex.digitaleveryware.com", Server.ConnectionType.SSL, 50002),
new Server("dxm.no-ip.biz", Server.ConnectionType.SSL, 50002),
new Server("caleb.vegas", Server.ConnectionType.SSL, 50002),
new Server("ecdsa.net", Server.ConnectionType.SSL, 110),
new Server("electrum.hsmiths.com", Server.ConnectionType.SSL, 995),
new Server("elec.luggs.co", Server.ConnectionType.SSL, 443),
new Server("btc.smsys.me", Server.ConnectionType.SSL, 995));
new Server("electrum.coinext.com.br", Server.ConnectionType.TCP, 50001),
new Server("korea.electrum-server.com", Server.ConnectionType.TCP, 50001),
new Server("eai.coincited.net", Server.ConnectionType.TCP, 50001),
new Server("electrum.coinext.com.br", Server.ConnectionType.SSL, 50002),
new Server("node1.btccuracao.com", Server.ConnectionType.SSL, 50002),
new Server("korea.electrum-server.com", Server.ConnectionType.SSL, 50002),
new Server("btce.iiiiiii.biz", Server.ConnectionType.SSL, 50002),
new Server("bitcoin.lukechilds.co", Server.ConnectionType.SSL, 50002),
new Server("guichet.centure.cc", Server.ConnectionType.SSL, 50002),
new Server("electrumx.hodlwallet.com", Server.ConnectionType.SSL, 50002),
new Server("eai.coincited.net", Server.ConnectionType.SSL, 50002),
new Server("prospero.bitsrc.net", Server.ConnectionType.SSL, 50002),
new Server("gd42.org", Server.ConnectionType.SSL, 50002),
new Server("electrum.pabu.io", Server.ConnectionType.SSL, 50002));
}
@Override

View File

@@ -0,0 +1,861 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Dogecoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Dogecoin & Qortal 'trade' keys</li>
* <li>Alice funds Dogecoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Dogecoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Dogecoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Dogecoin trade key and secret-A</li>
* <li>P2SH-A DOGE funds end up at Dogecoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class DogecoinACCTv2 implements ACCT {
private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv2.class);
public static final String NAME = DogecoinACCTv2.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("6fff38d6eeb06568a9c879c5628527730319844aa0de53f5f4ffab5506efe885").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerDogecoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerDogecoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Dogecoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static DogecoinACCTv2 instance;
private DogecoinACCTv2() {
}
public static synchronized DogecoinACCTv2 getInstance() {
if (instance == null)
instance = new DogecoinACCTv2();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Dogecoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param dogecoinPublicKeyHash 20-byte HASH160 of creator's trade Dogecoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param dogecoinAmount how much DOGE the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] dogecoinPublicKeyHash, long qortAmount, long dogecoinAmount, int tradeTimeout) {
if (dogecoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Dogecoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrDogecoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrDogecoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerDogecoinPKHOffset = addrCounter++;
final int addrPartnerDogecoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerDogecoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Dogecoin public key hash
assert dataByteBuffer.position() == addrDogecoinPublicKeyHash * MachineState.VALUE_SIZE : "addrDogecoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(dogecoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Dogecoin amount
assert dataByteBuffer.position() == addrDogecoinAmount * MachineState.VALUE_SIZE : "addrDogecoinAmount incorrect";
dataByteBuffer.putLong(dogecoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Dogecoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerDogecoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerDogecoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Dogecoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerDogecoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerDogecoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerDogecoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
/* NOP - to ensure DOGECOIN ACCT is unique */
codeByteBuffer.put(OpCode.NOP.compile());
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Dogecoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerDogecoinPKHOffset));
// Store partner's Dogecoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerDogecoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile DOGE-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), DogecoinACCTv2.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.DOGECOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Dogecoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected DOGE amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Dogecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Dogecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Dogecoin PKH
byte[] partnerDogecoinPKH = new byte[20];
dataByteBuffer.get(partnerDogecoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerDogecoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerDogecoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerDogecoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -0,0 +1,858 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Dogecoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Dogecoin & Qortal 'trade' keys</li>
* <li>Alice funds Dogecoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Dogecoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Dogecoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Dogecoin trade key and secret-A</li>
* <li>P2SH-A DOGE funds end up at Dogecoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class DogecoinACCTv3 implements ACCT {
private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv3.class);
public static final String NAME = DogecoinACCTv3.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("03b44087f6325463eb745aa32d9a782add03148bcfbe73ffd8854ce55ff863d4").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerDogecoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerDogecoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Dogecoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static DogecoinACCTv3 instance;
private DogecoinACCTv3() {
}
public static synchronized DogecoinACCTv3 getInstance() {
if (instance == null)
instance = new DogecoinACCTv3();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Dogecoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param dogecoinPublicKeyHash 20-byte HASH160 of creator's trade Dogecoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param dogecoinAmount how much DOGE the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] dogecoinPublicKeyHash, long qortAmount, long dogecoinAmount, int tradeTimeout) {
if (dogecoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Dogecoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrDogecoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrDogecoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerDogecoinPKHOffset = addrCounter++;
final int addrPartnerDogecoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerDogecoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Dogecoin public key hash
assert dataByteBuffer.position() == addrDogecoinPublicKeyHash * MachineState.VALUE_SIZE : "addrDogecoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(dogecoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Dogecoin amount
assert dataByteBuffer.position() == addrDogecoinAmount * MachineState.VALUE_SIZE : "addrDogecoinAmount incorrect";
dataByteBuffer.putLong(dogecoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Dogecoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerDogecoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerDogecoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Dogecoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerDogecoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerDogecoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerDogecoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
/* NOP - to ensure DOGECOIN ACCT is unique */
codeByteBuffer.put(OpCode.NOP.compile());
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Dogecoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerDogecoinPKHOffset));
// Store partner's Dogecoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerDogecoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile DOGE-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), DogecoinACCTv3.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.DOGECOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Dogecoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected DOGE amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Dogecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Dogecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Dogecoin PKH
byte[] partnerDogecoinPKH = new byte[20];
dataByteBuffer.get(partnerDogecoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerDogecoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerDogecoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerDogecoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -653,18 +653,27 @@ public class ElectrumX extends BitcoinyBlockchainProvider {
Object errorObj = responseJson.get("error");
if (errorObj != null) {
if (errorObj instanceof String)
throw new ForeignBlockchainException.NetworkException(String.format("Unexpected error message from ElectrumX RPC %s: %s", method, (String) errorObj), this.currentServer);
if (errorObj instanceof String) {
LOGGER.debug(String.format("Unexpected error message from ElectrumX server %s for RPC method %s: %s", this.currentServer, method, (String) errorObj));
// Try another server
return null;
}
if (!(errorObj instanceof JSONObject))
throw new ForeignBlockchainException.NetworkException(String.format("Unexpected error response from ElectrumX RPC %s", method), this.currentServer);
if (!(errorObj instanceof JSONObject)) {
LOGGER.debug(String.format("Unexpected error response from ElectrumX server %s for RPC method %s", this.currentServer, method));
// Try another server
return null;
}
JSONObject errorJson = (JSONObject) errorObj;
Object messageObj = errorJson.get("message");
if (!(messageObj instanceof String))
throw new ForeignBlockchainException.NetworkException(String.format("Missing/invalid message in error response from ElectrumX RPC %s", method), this.currentServer);
if (!(messageObj instanceof String)) {
LOGGER.debug(String.format("Missing/invalid message in error response from ElectrumX server %s for RPC method %s", this.currentServer, method));
// Try another server
return null;
}
String message = (String) messageObj;

View File

@@ -21,6 +21,8 @@ public class Litecoin extends Bitcoiny {
private static final Coin DEFAULT_FEE_PER_KB = Coin.valueOf(10000); // 0.0001 LTC per 1000 bytes
private static final long MINIMUM_ORDER_AMOUNT = 1000000; // 0.01 LTC minimum order, to avoid dust errors
// Temporary values until a dynamic fee system is written.
private static final long MAINNET_FEE = 1000L;
private static final long NON_MAINNET_FEE = 1000L; // enough for TESTNET3 and should be OK for REGTEST
@@ -164,6 +166,11 @@ public class Litecoin extends Bitcoiny {
return DEFAULT_FEE_PER_KB;
}
@Override
public long getMinimumOrderAmount() {
return MINIMUM_ORDER_AMOUNT;
}
/**
* Returns estimated LTC fee, in sats per 1000bytes, optionally for historic timestamp.
*

View File

@@ -0,0 +1,854 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Litecoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Litecoin & Qortal 'trade' keys</li>
* <li>Alice funds Litecoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Litecoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Litecoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Litecoin trade key and secret-A</li>
* <li>P2SH-A LTC funds end up at Litecoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class LitecoinACCTv2 implements ACCT {
public static final String NAME = LitecoinACCTv2.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("d5ea386a41441180c854ca8d7bbc620bfd53a97df2650a2b162b52324caf6e19").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerLitecoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerLitecoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Litecoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static LitecoinACCTv2 instance;
private LitecoinACCTv2() {
}
public static synchronized LitecoinACCTv2 getInstance() {
if (instance == null)
instance = new LitecoinACCTv2();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Litecoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param litecoinPublicKeyHash 20-byte HASH160 of creator's trade Litecoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param litecoinAmount how much LTC the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] litecoinPublicKeyHash, long qortAmount, long litecoinAmount, int tradeTimeout) {
if (litecoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Litecoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrLitecoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrLitecoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerLitecoinPKHOffset = addrCounter++;
final int addrPartnerLitecoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerLitecoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Litecoin public key hash
assert dataByteBuffer.position() == addrLitecoinPublicKeyHash * MachineState.VALUE_SIZE : "addrLitecoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(litecoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Litecoin amount
assert dataByteBuffer.position() == addrLitecoinAmount * MachineState.VALUE_SIZE : "addrLitecoinAmount incorrect";
dataByteBuffer.putLong(litecoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Litecoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerLitecoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerLitecoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Litecoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerLitecoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerLitecoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerLitecoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Litecoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerLitecoinPKHOffset));
// Store partner's Litecoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerLitecoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile LTC-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), LitecoinACCTv2.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.LITECOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Litecoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected LTC amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Litecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Litecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Litecoin PKH
byte[] partnerLitecoinPKH = new byte[20];
dataByteBuffer.get(partnerLitecoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerLitecoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerLitecoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerLitecoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -0,0 +1,851 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Litecoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Litecoin & Qortal 'trade' keys</li>
* <li>Alice funds Litecoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Litecoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Litecoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Litecoin trade key and secret-A</li>
* <li>P2SH-A LTC funds end up at Litecoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class LitecoinACCTv3 implements ACCT {
public static final String NAME = LitecoinACCTv3.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("31588e7ddb95a908bce310bb6dc76f011ff4693d2f8a3741fab3c989d56ce7c7").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerLitecoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerLitecoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Litecoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static LitecoinACCTv3 instance;
private LitecoinACCTv3() {
}
public static synchronized LitecoinACCTv3 getInstance() {
if (instance == null)
instance = new LitecoinACCTv3();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Litecoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param litecoinPublicKeyHash 20-byte HASH160 of creator's trade Litecoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param litecoinAmount how much LTC the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] litecoinPublicKeyHash, long qortAmount, long litecoinAmount, int tradeTimeout) {
if (litecoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Litecoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrLitecoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrLitecoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerLitecoinPKHOffset = addrCounter++;
final int addrPartnerLitecoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerLitecoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Litecoin public key hash
assert dataByteBuffer.position() == addrLitecoinPublicKeyHash * MachineState.VALUE_SIZE : "addrLitecoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(litecoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Litecoin amount
assert dataByteBuffer.position() == addrLitecoinAmount * MachineState.VALUE_SIZE : "addrLitecoinAmount incorrect";
dataByteBuffer.putLong(litecoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Litecoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerLitecoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerLitecoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Litecoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerLitecoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerLitecoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerLitecoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Litecoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerLitecoinPKHOffset));
// Store partner's Litecoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerLitecoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile LTC-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), LitecoinACCTv3.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.LITECOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Litecoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected LTC amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Litecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Litecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Litecoin PKH
byte[] partnerLitecoinPKH = new byte[20];
dataByteBuffer.get(partnerLitecoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerLitecoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerLitecoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerLitecoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -28,7 +28,9 @@ public enum SupportedBlockchain {
},
LITECOIN(Arrays.asList(
Triple.valueOf(LitecoinACCTv1.NAME, LitecoinACCTv1.CODE_BYTES_HASH, LitecoinACCTv1::getInstance)
Triple.valueOf(LitecoinACCTv1.NAME, LitecoinACCTv1.CODE_BYTES_HASH, LitecoinACCTv1::getInstance),
Triple.valueOf(LitecoinACCTv2.NAME, LitecoinACCTv2.CODE_BYTES_HASH, LitecoinACCTv2::getInstance),
Triple.valueOf(LitecoinACCTv3.NAME, LitecoinACCTv3.CODE_BYTES_HASH, LitecoinACCTv3::getInstance)
)) {
@Override
public ForeignBlockchain getInstance() {
@@ -37,12 +39,14 @@ public enum SupportedBlockchain {
@Override
public ACCT getLatestAcct() {
return LitecoinACCTv1.getInstance();
return LitecoinACCTv3.getInstance();
}
},
DOGECOIN(Arrays.asList(
Triple.valueOf(DogecoinACCTv1.NAME, DogecoinACCTv1.CODE_BYTES_HASH, DogecoinACCTv1::getInstance)
Triple.valueOf(DogecoinACCTv1.NAME, DogecoinACCTv1.CODE_BYTES_HASH, DogecoinACCTv1::getInstance),
Triple.valueOf(DogecoinACCTv2.NAME, DogecoinACCTv2.CODE_BYTES_HASH, DogecoinACCTv2::getInstance),
Triple.valueOf(DogecoinACCTv3.NAME, DogecoinACCTv3.CODE_BYTES_HASH, DogecoinACCTv3::getInstance)
)) {
@Override
public ForeignBlockchain getInstance() {
@@ -51,7 +55,7 @@ public enum SupportedBlockchain {
@Override
public ACCT getLatestAcct() {
return DogecoinACCTv1.getInstance();
return DogecoinACCTv3.getInstance();
}
};

View File

@@ -1,5 +1,8 @@
package org.qortal.crypto;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
@@ -75,12 +78,74 @@ public abstract class Crypto {
return digest(digest(input));
}
/**
* Returns 32-byte SHA-256 digest of file passed in input.
*
* @param file
* file in which to perform digest
* @return byte[32] digest, or null if SHA-256 algorithm can't be accessed
*
* @throws IOException if the file cannot be read
*/
public static byte[] digest(File file) throws IOException {
return Crypto.digest(file, 8192);
}
/**
* Returns 32-byte SHA-256 digest of file passed in input, in hex format
*
* @param file
* file in which to perform digest
* @return String digest as a hexadecimal string, or null if SHA-256 algorithm can't be accessed
*
* @throws IOException if the file cannot be read
*/
public static String digestHexString(File file, int bufferSize) throws IOException {
byte[] digest = Crypto.digest(file, bufferSize);
// Convert to hex
StringBuilder stringBuilder = new StringBuilder();
for (byte b : digest) {
stringBuilder.append(String.format("%02x", b));
}
return stringBuilder.toString();
}
/**
* Returns 32-byte SHA-256 digest of file passed in input.
*
* @param file
* file in which to perform digest
* @param bufferSize
* the number of bytes to load into memory
* @return byte[32] digest, or null if SHA-256 algorithm can't be accessed
*
* @throws IOException if the file cannot be read
*/
public static byte[] digest(File file, int bufferSize) throws IOException {
try {
MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
FileInputStream fileInputStream = new FileInputStream(file);
byte[] bytes = new byte[bufferSize];
int count;
while ((count = fileInputStream.read(bytes)) != -1) {
sha256.update(bytes, 0, count);
}
fileInputStream.close();
return sha256.digest();
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException("SHA-256 message digest not available");
}
}
/**
* Returns 64-byte duplicated digest of message passed in input.
* <p>
* Effectively <tt>Bytes.concat(digest(input), digest(input)).
*
* @param addressVersion
*
* @param input
*/
public static byte[] dupDigest(byte[] input) {

View File

@@ -4,10 +4,12 @@ import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlTransient;
import org.json.JSONObject;
import org.qortal.crypto.Crypto;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.media.Schema.AccessMode;
import org.qortal.utils.Base58;
// All properties to be converted to JSON via JAXB
@XmlAccessorType(XmlAccessType.FIELD)
@@ -61,4 +63,21 @@ public class MintingAccountData {
return this.publicKey;
}
// JSON
public JSONObject toJson() {
JSONObject jsonObject = new JSONObject();
jsonObject.put("privateKey", Base58.encode(this.getPrivateKey()));
jsonObject.put("publicKey", Base58.encode(this.getPublicKey()));
return jsonObject;
}
public static MintingAccountData fromJson(JSONObject json) {
return new MintingAccountData(
json.isNull("privateKey") ? null : Base58.decode(json.getString("privateKey")),
json.isNull("publicKey") ? null : Base58.decode(json.getString("publicKey"))
);
}
}

View File

@@ -0,0 +1,47 @@
package org.qortal.data.block;
import org.qortal.block.Block;
public class BlockArchiveData {
// Properties
private byte[] signature;
private Integer height;
private Long timestamp;
private byte[] minterPublicKey;
// Constructors
public BlockArchiveData(byte[] signature, Integer height, long timestamp, byte[] minterPublicKey) {
this.signature = signature;
this.height = height;
this.timestamp = timestamp;
this.minterPublicKey = minterPublicKey;
}
public BlockArchiveData(BlockData blockData) {
this.signature = blockData.getSignature();
this.height = blockData.getHeight();
this.timestamp = blockData.getTimestamp();
this.minterPublicKey = blockData.getMinterPublicKey();
}
// Getters/setters
public byte[] getSignature() {
return this.signature;
}
public Integer getHeight() {
return this.height;
}
public Long getTimestamp() {
return this.timestamp;
}
public byte[] getMinterPublicKey() {
return this.minterPublicKey;
}
}

View File

@@ -204,6 +204,10 @@ public class BlockData implements Serializable {
return this.onlineAccountsSignatures;
}
public void setOnlineAccountsSignatures(byte[] onlineAccountsSignatures) {
this.onlineAccountsSignatures = onlineAccountsSignatures;
}
// JAXB special
@XmlElement(name = "minterAddress")

View File

@@ -0,0 +1,55 @@
package org.qortal.data.network;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
// All properties to be converted to JSON via JAXB
@XmlAccessorType(XmlAccessType.FIELD)
public class OnlineAccountLevel {
protected int level;
protected int count;
// Constructors
// necessary for JAXB serialization
protected OnlineAccountLevel() {
}
public OnlineAccountLevel(int level, int count) {
this.level = level;
this.count = count;
}
public int getLevel() {
return this.level;
}
public int getCount() {
return this.count;
}
public void setCount(int count) {
this.count = count;
}
// Comparison
@Override
public boolean equals(Object other) {
if (other == this)
return true;
if (!(other instanceof OnlineAccountLevel))
return false;
OnlineAccountLevel otherOnlineAccountData = (OnlineAccountLevel) other;
if (otherOnlineAccountData.level != this.level)
return false;
return true;
}
}

View File

@@ -26,7 +26,7 @@ public class RegisterNameTransactionData extends TransactionData {
@Schema(description = "requested name", example = "my-name")
private String name;
@Schema(description = "simple name-related info in JSON format", example = "{ \"age\": 30 }")
@Schema(description = "simple name-related info in JSON or text format", example = "Registered Name on the Qortal Chain")
private String data;
// For internal use

View File

@@ -26,7 +26,7 @@ public class UpdateNameTransactionData extends TransactionData {
@Schema(description = "new name", example = "my-new-name")
private String newName;
@Schema(description = "replacement simple name-related info in JSON format", example = "{ \"age\": 30 }")
@Schema(description = "replacement simple name-related info in JSON or text format", example = "Registered Name on the Qortal Chain")
private String newData;
// For internal use

View File

@@ -23,17 +23,21 @@ public class Gui {
private SysTray sysTray = null;
private Gui() {
this.isHeadless = GraphicsEnvironment.isHeadless();
try {
this.isHeadless = GraphicsEnvironment.isHeadless();
if (!this.isHeadless) {
try {
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
} catch (ClassNotFoundException | InstantiationException | IllegalAccessException
| UnsupportedLookAndFeelException e) {
// Use whatever look-and-feel comes by default then
if (!this.isHeadless) {
try {
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
} catch (ClassNotFoundException | InstantiationException | IllegalAccessException
| UnsupportedLookAndFeelException e) {
// Use whatever look-and-feel comes by default then
}
showSplash();
}
showSplash();
} catch (Exception e) {
LOGGER.info("Unable to initialize GUI: {}", e.getMessage());
}
}

View File

@@ -6,9 +6,11 @@ import java.util.List;
import java.awt.image.BufferedImage;
import javax.swing.*;
import javax.swing.border.EmptyBorder;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
public class SplashFrame {
@@ -16,6 +18,7 @@ public class SplashFrame {
private static SplashFrame instance;
private JFrame splashDialog;
private SplashPanel splashPanel;
@SuppressWarnings("serial")
public static class SplashPanel extends JPanel {
@@ -23,26 +26,53 @@ public class SplashFrame {
private String defaultSplash = "Qlogo_512.png";
private JLabel statusLabel;
public SplashPanel() {
image = Gui.loadImage(defaultSplash);
setOpaque(false);
setLayout(new GridBagLayout());
}
setOpaque(true);
setLayout(new BoxLayout(this, BoxLayout.Y_AXIS));
setBorder(new EmptyBorder(10, 10, 10, 10));
setBackground(Color.BLACK);
@Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
g.drawImage(image, 0, 0, getWidth(), getHeight(), this);
// Add logo
JLabel imageLabel = new JLabel(new ImageIcon(image));
imageLabel.setSize(new Dimension(300, 300));
add(imageLabel);
// Add spacing
add(Box.createRigidArea(new Dimension(0, 16)));
// Add status label
String text = String.format("Starting Qortal Core v%s...", Controller.getInstance().getVersionStringWithoutPrefix());
statusLabel = new JLabel(text, JLabel.CENTER);
statusLabel.setMaximumSize(new Dimension(500, 50));
statusLabel.setFont(new Font("Verdana", Font.PLAIN, 20));
statusLabel.setBackground(Color.BLACK);
statusLabel.setForeground(new Color(255, 255, 255, 255));
statusLabel.setOpaque(true);
statusLabel.setBorder(null);
add(statusLabel);
}
@Override
public Dimension getPreferredSize() {
return new Dimension(500, 500);
return new Dimension(500, 580);
}
public void updateStatus(String text) {
if (statusLabel != null) {
statusLabel.setText(text);
}
}
}
private SplashFrame() {
if (GraphicsEnvironment.isHeadless()) {
return;
}
this.splashDialog = new JFrame();
List<Image> icons = new ArrayList<>();
@@ -55,12 +85,13 @@ public class SplashFrame {
icons.add(Gui.loadImage("icons/Qlogo_128.png"));
this.splashDialog.setIconImages(icons);
this.splashDialog.getContentPane().add(new SplashPanel());
this.splashPanel = new SplashPanel();
this.splashDialog.getContentPane().add(this.splashPanel);
this.splashDialog.setDefaultCloseOperation(JFrame.DO_NOTHING_ON_CLOSE);
this.splashDialog.setUndecorated(true);
this.splashDialog.pack();
this.splashDialog.setLocationRelativeTo(null);
this.splashDialog.setBackground(new Color(0,0,0,0));
this.splashDialog.setBackground(Color.BLACK);
this.splashDialog.setVisible(true);
}
@@ -79,4 +110,10 @@ public class SplashFrame {
this.splashDialog.dispose();
}
public void updateStatus(String text) {
if (this.splashPanel != null) {
this.splashPanel.updateStatus(text);
}
}
}

View File

@@ -291,19 +291,23 @@ public class SysTray {
public void setTrayIcon(int iconid) {
if (trayIcon != null) {
switch (iconid) {
case 1:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_syncing_time-alt.png"));
break;
case 2:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_minting.png"));
break;
case 3:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_syncing.png"));
break;
case 4:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_synced.png"));
break;
try {
switch (iconid) {
case 1:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_syncing_time-alt.png"));
break;
case 2:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_minting.png"));
break;
case 3:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_syncing.png"));
break;
case 4:
this.trayIcon.setImage(Gui.loadImage("icons/qortal_ui_tray_synced.png"));
break;
}
} catch (NullPointerException e) {
LOGGER.info("Unable to set tray icon");
}
}
}

View File

@@ -21,7 +21,7 @@ public class ResourceList {
private String category;
private String resourceName;
private List<String> list;
private List<String> list = new ArrayList<>();
/**
* ResourceList
@@ -44,14 +44,8 @@ public class ResourceList {
private Path getFilePath() {
String pathString = String.format("%s%s%s_%s.json", Settings.getInstance().getListsPath(),
File.separator, this.resourceName, this.category);
Path outputFilePath = Paths.get(pathString);
try {
Files.createDirectories(outputFilePath.getParent());
} catch (IOException e) {
throw new IllegalStateException("Unable to create lists directory");
}
return outputFilePath;
File.separator, this.category, this.resourceName);
return Paths.get(pathString);
}
public void save() throws IOException {
@@ -62,8 +56,15 @@ public class ResourceList {
throw new IllegalStateException("Can't save list with missing category");
}
String jsonString = ResourceList.listToJSONString(this.list);
Path filePath = this.getFilePath();
// Create parent directory if needed
try {
Files.createDirectories(filePath.getParent());
} catch (IOException e) {
throw new IllegalStateException("Unable to create lists directory");
}
BufferedWriter writer = new BufferedWriter(new FileWriter(filePath.toString()));
writer.write(jsonString);
writer.close();
@@ -99,24 +100,35 @@ public class ResourceList {
/* List management */
public void add(String resource) {
if (resource == null || this.list == null) {
return;
}
if (!this.contains(resource)) {
this.list.add(resource);
}
}
public void remove(String resource) {
if (resource == null || this.list == null) {
return;
}
this.list.remove(resource);
}
public boolean contains(String resource) {
if (resource == null || this.list == null) {
return false;
}
return this.list.contains(resource);
}
/* Utils */
public static String listToJSONString(List<String> list) {
if (list == null) {
return null;
}
JSONArray items = new JSONArray();
for (String item : list) {
items.put(item);
@@ -125,6 +137,9 @@ public class ResourceList {
}
private static List<String> listFromJSONString(String jsonString) {
if (jsonString == null) {
return null;
}
JSONArray jsonList = new JSONArray(jsonString);
List<String> resourceList = new ArrayList<>();
for (int i=0; i<jsonList.length(); i++) {
@@ -134,4 +149,20 @@ public class ResourceList {
return resourceList;
}
public String getJSONString() {
return ResourceList.listToJSONString(this.list);
}
public String getCategory() {
return this.category;
}
public String getResourceName() {
return this.resourceName;
}
public String toString() {
return String.format("%s %s", this.category, this.resourceName);
}
}

View File

@@ -4,84 +4,125 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
public class ResourceListManager {
private static final Logger LOGGER = LogManager.getLogger(ResourceListManager.class);
private static ResourceListManager instance;
private ResourceList addressBlacklist;
private List<ResourceList> lists = new ArrayList<>();
public ResourceListManager() {
try {
this.addressBlacklist = new ResourceList("blacklist", "address");
} catch (IOException e) {
LOGGER.info("Error while loading address blacklist. Blocking is currently unavailable.");
}
}
public static synchronized ResourceListManager getInstance() {
if (instance == null) {
instance = new ResourceListManager();
}
return instance;
}
public boolean addAddressToBlacklist(String address, boolean save) {
try {
this.addressBlacklist.add(address);
if (save) {
this.addressBlacklist.save();
private ResourceList getList(String category, String resourceName) {
for (ResourceList list : this.lists) {
if (Objects.equals(list.getCategory(), category) &&
Objects.equals(list.getResourceName(), resourceName)) {
return list;
}
return true;
} catch (IllegalStateException | IOException e) {
LOGGER.info("Unable to add address to blacklist", e);
return false;
}
}
public boolean removeAddressFromBlacklist(String address, boolean save) {
// List doesn't exist in array yet, so create it
// This will load any existing data from the filesystem
try {
this.addressBlacklist.remove(address);
ResourceList list = new ResourceList(category, resourceName);
this.lists.add(list);
return list;
if (save) {
this.addressBlacklist.save();
}
return true;
} catch (IllegalStateException | IOException e) {
LOGGER.info("Unable to remove address from blacklist", e);
return false;
}
}
public boolean isAddressInBlacklist(String address) {
if (this.addressBlacklist == null) {
return false;
}
return this.addressBlacklist.contains(address);
}
public void saveBlacklist() {
if (this.addressBlacklist == null) {
return;
}
try {
this.addressBlacklist.save();
} catch (IOException e) {
LOGGER.info("Unable to save blacklist - reverting back to last saved state");
this.addressBlacklist.revert();
LOGGER.info("Unable to load or create list {} {}: {}", category, resourceName, e.getMessage());
return null;
}
}
public boolean addToList(String category, String resourceName, String item, boolean save) {
ResourceList list = this.getList(category, resourceName);
if (list == null) {
return false;
}
try {
list.add(item);
if (save) {
list.save();
}
return true;
} catch (IllegalStateException | IOException e) {
LOGGER.info(String.format("Unable to add item %s to list %s", item, list), e);
return false;
}
}
public void revertBlacklist() {
if (this.addressBlacklist == null) {
public boolean removeFromList(String category, String resourceName, String item, boolean save) {
ResourceList list = this.getList(category, resourceName);
if (list == null) {
return false;
}
try {
list.remove(item);
if (save) {
list.save();
}
return true;
} catch (IllegalStateException | IOException e) {
LOGGER.info(String.format("Unable to remove item %s from list %s", item, list), e);
return false;
}
}
public boolean listContains(String category, String resourceName, String address) {
ResourceList list = this.getList(category, resourceName);
if (list == null) {
return false;
}
return list.contains(address);
}
public void saveList(String category, String resourceName) {
ResourceList list = this.getList(category, resourceName);
if (list == null) {
return;
}
this.addressBlacklist.revert();
try {
list.save();
} catch (IOException e) {
LOGGER.info("Unable to save list {} - reverting back to last saved state", list);
list.revert();
}
}
public void revertList(String category, String resourceName) {
ResourceList list = this.getList(category, resourceName);
if (list == null) {
return;
}
list.revert();
}
public String getJSONStringForList(String category, String resourceName) {
ResourceList list = this.getList(category, resourceName);
if (list == null) {
return null;
}
return list.getJSONString();
}
}

View File

@@ -78,9 +78,10 @@ public class Name {
// Set name's last-updated timestamp
this.nameData.setUpdated(updateNameTransactionData.getTimestamp());
// Update name and data where appropriate
// Update name, reduced name, and data where appropriate
if (!updateNameTransactionData.getNewName().isEmpty()) {
this.nameData.setName(updateNameTransactionData.getNewName());
this.nameData.setReducedName(updateNameTransactionData.getReducedNewName());
// If we're changing the name, we need to delete old entry
this.repository.getNameRepository().delete(updateNameTransactionData.getName());
@@ -106,6 +107,9 @@ public class Name {
// We can find previous 'name' from update transaction
this.nameData.setName(updateNameTransactionData.getName());
// We can derive the previous 'reduced name' from the previous name
this.nameData.setReducedName(Unicode.sanitize(updateNameTransactionData.getName()));
// We might need to hunt for previous data value
if (!updateNameTransactionData.getNewData().isEmpty())
this.nameData.setData(findPreviousData(nameReference));
@@ -261,4 +265,8 @@ public class Name {
return previousTransactionData.getTimestamp();
}
public NameData getNameData() {
return this.nameData;
}
}

View File

@@ -73,7 +73,9 @@ public class Network {
"node1.qortal.org", "node2.qortal.org", "node3.qortal.org", "node4.qortal.org", "node5.qortal.org",
"node6.qortal.org", "node7.qortal.org", "node8.qortal.org", "node9.qortal.org", "node10.qortal.org",
"node.qortal.ru", "node2.qortal.ru", "node3.qortal.ru", "node.qortal.uk", "node22.qortal.org",
"cinfu1.crowetic.com", "node.cwd.systems"
"cinfu1.crowetic.com", "node.cwd.systems", "bootstrap.cwd.systems", "node1.qortalnodes.live",
"node2.qortalnodes.live", "node3.qortalnodes.live", "node4.qortalnodes.live", "node5.qortalnodes.live",
"node6.qortalnodes.live", "node7.qortalnodes.live", "node8.qortalnodes.live"
};
private static final long NETWORK_EPC_KEEPALIVE = 10L; // seconds
@@ -155,9 +157,23 @@ public class Network {
}
// Load all known peers from repository
try (Repository repository = RepositoryManager.getRepository()) {
synchronized (this.allKnownPeers) {
this.allKnownPeers.addAll(repository.getNetworkRepository().getAllPeers());
synchronized (this.allKnownPeers) { List<String> fixedNetwork = Settings.getInstance().getFixedNetwork();
if (fixedNetwork != null && !fixedNetwork.isEmpty()) {
Long addedWhen = NTP.getTime();
String addedBy = "fixedNetwork";
List<PeerAddress> peerAddresses = new ArrayList<>();
for (String address : fixedNetwork) {
PeerAddress peerAddress = PeerAddress.fromString(address);
peerAddresses.add(peerAddress);
}
List<PeerData> peers = peerAddresses.stream()
.map(peerAddress -> new PeerData(peerAddress, addedWhen, addedBy))
.collect(Collectors.toList());
this.allKnownPeers.addAll(peers);
} else {
try (Repository repository = RepositoryManager.getRepository()) {
this.allKnownPeers.addAll(repository.getNetworkRepository().getAllPeers());
}
}
}
@@ -513,14 +529,24 @@ public class Network {
if (socketChannel == null) {
return;
}
PeerAddress address = PeerAddress.fromSocket(socketChannel.socket());
List<String> fixedNetwork = Settings.getInstance().getFixedNetwork();
if (fixedNetwork != null && !fixedNetwork.isEmpty() && ipNotInFixedList(address, fixedNetwork)) {
try {
LOGGER.debug("Connection discarded from peer {} as not in the fixed network list", address);
socketChannel.close();
} catch (IOException e) {
// IGNORE
}
return;
}
final Long now = NTP.getTime();
Peer newPeer;
try {
if (now == null) {
LOGGER.debug("Connection discarded from peer {} due to lack of NTP sync",
PeerAddress.fromSocket(socketChannel.socket()));
LOGGER.debug("Connection discarded from peer {} due to lack of NTP sync", address);
socketChannel.close();
return;
}
@@ -528,12 +554,12 @@ public class Network {
synchronized (this.connectedPeers) {
if (connectedPeers.size() >= maxPeers) {
// We have enough peers
LOGGER.debug("Connection discarded from peer {}", PeerAddress.fromSocket(socketChannel.socket()));
LOGGER.debug("Connection discarded from peer {} because the server is full", address);
socketChannel.close();
return;
}
LOGGER.debug("Connection accepted from peer {}", PeerAddress.fromSocket(socketChannel.socket()));
LOGGER.debug("Connection accepted from peer {}", address);
newPeer = new Peer(socketChannel, channelSelector);
this.connectedPeers.add(newPeer);
@@ -541,6 +567,7 @@ public class Network {
} catch (IOException e) {
if (socketChannel.isOpen()) {
try {
LOGGER.debug("Connection failed from peer {} while connecting/closing", address);
socketChannel.close();
} catch (IOException ce) {
// Couldn't close?
@@ -552,6 +579,16 @@ public class Network {
this.onPeerReady(newPeer);
}
private boolean ipNotInFixedList(PeerAddress address, List<String> fixedNetwork) {
for (String ipAddress : fixedNetwork) {
String[] bits = ipAddress.split(":");
if (bits.length >= 1 && bits.length <= 2 && address.getHost().equals(bits[0])) {
return false;
}
}
return true;
}
private Peer getConnectablePeer(final Long now) throws InterruptedException {
// We can't block here so use tryRepository(). We don't NEED to connect a new peer.
try (Repository repository = RepositoryManager.tryRepository()) {
@@ -653,7 +690,7 @@ public class Network {
if (peersToDisconnect != null && peersToDisconnect.size() > 0) {
for (Peer peer : peersToDisconnect) {
LOGGER.info("Forcing disconnection of peer {} because connection age ({} ms) " +
LOGGER.debug("Forcing disconnection of peer {} because connection age ({} ms) " +
"has reached the maximum ({} ms)", peer, peer.getConnectionAge(), peer.getMaxConnectionAge());
peer.disconnect("Connection age too old");
}
@@ -1145,6 +1182,10 @@ public class Network {
private boolean mergePeers(Repository repository, String addedBy, long addedWhen, List<PeerAddress> peerAddresses)
throws DataException {
List<String> fixedNetwork = Settings.getInstance().getFixedNetwork();
if (fixedNetwork != null && !fixedNetwork.isEmpty()) {
return false;
}
List<PeerData> newPeers;
synchronized (this.allKnownPeers) {
for (PeerData knownPeerData : this.allKnownPeers) {

View File

@@ -23,7 +23,7 @@ public class CachedBlockMessage extends Message {
this.block = block;
}
private CachedBlockMessage(byte[] cachedBytes) {
public CachedBlockMessage(byte[] cachedBytes) {
super(MessageType.BLOCK);
this.block = null;

View File

@@ -1,5 +1,7 @@
package org.qortal.repository;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.List;
import java.util.Set;
@@ -112,6 +114,14 @@ public interface ATRepository {
*/
public List<ATStateData> getBlockATStatesAtHeight(int height) throws DataException;
/** Rebuild the latest AT states cache, necessary for AT state trimming/pruning.
* <p>
* NOTE: performs implicit <tt>repository.saveChanges()</tt>.
*/
public void rebuildLatestAtStates() throws DataException;
/** Returns height of first trimmable AT state. */
public int getAtTrimHeight() throws DataException;
@@ -121,12 +131,27 @@ public interface ATRepository {
*/
public void setAtTrimHeight(int trimHeight) throws DataException;
/** Hook to allow repository to prepare/cache info for AT state trimming. */
public void prepareForAtStateTrimming() throws DataException;
/** Trims full AT state data between passed heights. Returns number of trimmed rows. */
public int trimAtStates(int minHeight, int maxHeight, int limit) throws DataException;
/** Returns height of first prunable AT state. */
public int getAtPruneHeight() throws DataException;
/** Sets new base height for AT state pruning.
* <p>
* NOTE: performs implicit <tt>repository.saveChanges()</tt>.
*/
public void setAtPruneHeight(int pruneHeight) throws DataException;
/** Prunes full AT state data between passed heights. Returns number of pruned rows. */
public int pruneAtStates(int minHeight, int maxHeight) throws DataException;
/** Checks for the presence of the ATStatesHeightIndex in repository */
public boolean hasAtStatesHeightIndex() throws DataException;
/**
* Save ATStateData into repository.
* <p>

View File

@@ -191,6 +191,8 @@ public interface AccountRepository {
public List<MintingAccountData> getMintingAccounts() throws DataException;
public MintingAccountData getMintingAccount(byte[] mintingAccountKey) throws DataException;
public void save(MintingAccountData mintingAccountData) throws DataException;
/** Delete minting account info, used by BlockMinter, from repository using passed public or private key. */

View File

@@ -0,0 +1,284 @@
package org.qortal.repository;
import com.google.common.primitives.Ints;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.data.at.ATStateData;
import org.qortal.data.block.BlockArchiveData;
import org.qortal.data.block.BlockData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.settings.Settings;
import org.qortal.transform.TransformationException;
import org.qortal.transform.block.BlockTransformer;
import org.qortal.utils.Triple;
import static org.qortal.transform.Transformer.INT_LENGTH;
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.*;
public class BlockArchiveReader {
private static BlockArchiveReader instance;
private Map<String, Triple<Integer, Integer, Integer>> fileListCache = Collections.synchronizedMap(new HashMap<>());
private static final Logger LOGGER = LogManager.getLogger(BlockArchiveReader.class);
public BlockArchiveReader() {
}
public static synchronized BlockArchiveReader getInstance() {
if (instance == null) {
instance = new BlockArchiveReader();
}
return instance;
}
private void fetchFileList() {
Path archivePath = Paths.get(Settings.getInstance().getRepositoryPath(), "archive").toAbsolutePath();
File archiveDirFile = archivePath.toFile();
String[] files = archiveDirFile.list();
Map<String, Triple<Integer, Integer, Integer>> map = new HashMap<>();
if (files != null) {
for (String file : files) {
Path filePath = Paths.get(file);
String filename = filePath.getFileName().toString();
// Parse the filename
if (filename == null || !filename.contains("-") || !filename.contains(".")) {
// Not a usable file
continue;
}
// Remove the extension and split into two parts
String[] parts = filename.substring(0, filename.lastIndexOf('.')).split("-");
Integer startHeight = Integer.parseInt(parts[0]);
Integer endHeight = Integer.parseInt(parts[1]);
Integer range = endHeight - startHeight;
map.put(filename, new Triple(startHeight, endHeight, range));
}
}
this.fileListCache = map;
}
public Triple<BlockData, List<TransactionData>, List<ATStateData>> fetchBlockAtHeight(int height) {
if (this.fileListCache.isEmpty()) {
this.fetchFileList();
}
byte[] serializedBytes = this.fetchSerializedBlockBytesForHeight(height);
if (serializedBytes == null) {
return null;
}
ByteBuffer byteBuffer = ByteBuffer.wrap(serializedBytes);
Triple<BlockData, List<TransactionData>, List<ATStateData>> blockInfo = null;
try {
blockInfo = BlockTransformer.fromByteBuffer(byteBuffer);
if (blockInfo != null && blockInfo.getA() != null) {
// Block height is stored outside of the main serialized bytes, so it
// won't be set automatically.
blockInfo.getA().setHeight(height);
}
} catch (TransformationException e) {
return null;
}
return blockInfo;
}
public Triple<BlockData, List<TransactionData>, List<ATStateData>> fetchBlockWithSignature(
byte[] signature, Repository repository) {
if (this.fileListCache.isEmpty()) {
this.fetchFileList();
}
Integer height = this.fetchHeightForSignature(signature, repository);
if (height != null) {
return this.fetchBlockAtHeight(height);
}
return null;
}
public List<Triple<BlockData, List<TransactionData>, List<ATStateData>>> fetchBlocksFromRange(
int startHeight, int endHeight) {
List<Triple<BlockData, List<TransactionData>, List<ATStateData>>> blockInfoList = new ArrayList<>();
for (int height = startHeight; height <= endHeight; height++) {
Triple<BlockData, List<TransactionData>, List<ATStateData>> blockInfo = this.fetchBlockAtHeight(height);
if (blockInfo == null) {
return blockInfoList;
}
blockInfoList.add(blockInfo);
}
return blockInfoList;
}
public Integer fetchHeightForSignature(byte[] signature, Repository repository) {
// Lookup the height for the requested signature
try {
BlockArchiveData archivedBlock = repository.getBlockArchiveRepository().getBlockArchiveDataForSignature(signature);
if (archivedBlock == null) {
return null;
}
return archivedBlock.getHeight();
} catch (DataException e) {
return null;
}
}
public int fetchHeightForTimestamp(long timestamp, Repository repository) {
// Lookup the height for the requested signature
try {
return repository.getBlockArchiveRepository().getHeightFromTimestamp(timestamp);
} catch (DataException e) {
return 0;
}
}
private String getFilenameForHeight(int height) {
Iterator it = this.fileListCache.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
if (pair == null && pair.getKey() == null && pair.getValue() == null) {
continue;
}
Triple<Integer, Integer, Integer> heightInfo = (Triple<Integer, Integer, Integer>) pair.getValue();
Integer startHeight = heightInfo.getA();
Integer endHeight = heightInfo.getB();
if (height >= startHeight && height <= endHeight) {
// Found the correct file
String filename = (String) pair.getKey();
return filename;
}
}
return null;
}
public byte[] fetchSerializedBlockBytesForSignature(byte[] signature, boolean includeHeightPrefix, Repository repository) {
if (this.fileListCache.isEmpty()) {
this.fetchFileList();
}
Integer height = this.fetchHeightForSignature(signature, repository);
if (height != null) {
byte[] blockBytes = this.fetchSerializedBlockBytesForHeight(height);
if (blockBytes == null) {
return null;
}
// When responding to a peer with a BLOCK message, we must prefix the byte array with the block height
// This mimics the toData() method in BlockMessage and CachedBlockMessage
if (includeHeightPrefix) {
ByteArrayOutputStream bytes = new ByteArrayOutputStream(blockBytes.length + INT_LENGTH);
try {
bytes.write(Ints.toByteArray(height));
bytes.write(blockBytes);
return bytes.toByteArray();
} catch (IOException e) {
return null;
}
}
return blockBytes;
}
return null;
}
public byte[] fetchSerializedBlockBytesForHeight(int height) {
String filename = this.getFilenameForHeight(height);
if (filename == null) {
// We don't have this block in the archive
// Invalidate the file list cache in case it is out of date
this.invalidateFileListCache();
return null;
}
Path filePath = Paths.get(Settings.getInstance().getRepositoryPath(), "archive", filename).toAbsolutePath();
RandomAccessFile file = null;
try {
file = new RandomAccessFile(filePath.toString(), "r");
// Get info about this file (the "fixed length header")
final int version = file.readInt(); // Do not remove or comment out, as it is moving the file pointer
final int startHeight = file.readInt(); // Do not remove or comment out, as it is moving the file pointer
final int endHeight = file.readInt(); // Do not remove or comment out, as it is moving the file pointer
file.readInt(); // Block count (unused) // Do not remove or comment out, as it is moving the file pointer
final int variableHeaderLength = file.readInt(); // Do not remove or comment out, as it is moving the file pointer
final int fixedHeaderLength = (int)file.getFilePointer();
// End of fixed length header
// Make sure the version is one we recognize
if (version != 1) {
LOGGER.info("Error: unknown version in file {}: {}", filename, version);
return null;
}
// Verify that the block is within the reported range
if (height < startHeight || height > endHeight) {
LOGGER.info("Error: requested height {} but the range of file {} is {}-{}",
height, filename, startHeight, endHeight);
return null;
}
// Seek to the location of the block index in the variable length header
final int locationOfBlockIndexInVariableHeaderSegment = (height - startHeight) * INT_LENGTH;
file.seek(fixedHeaderLength + locationOfBlockIndexInVariableHeaderSegment);
// Read the value to obtain the index of this block in the data segment
int locationOfBlockInDataSegment = file.readInt();
// Now seek to the block data itself
int dataSegmentStartIndex = fixedHeaderLength + variableHeaderLength + INT_LENGTH; // Confirmed correct
file.seek(dataSegmentStartIndex + locationOfBlockInDataSegment);
// Read the block metadata
int blockHeight = file.readInt(); // Do not remove or comment out, as it is moving the file pointer
int blockLength = file.readInt(); // Do not remove or comment out, as it is moving the file pointer
// Ensure the block height matches the one requested
if (blockHeight != height) {
LOGGER.info("Error: height {} does not match requested: {}", blockHeight, height);
return null;
}
// Now retrieve the block's serialized bytes
byte[] blockBytes = new byte[blockLength];
file.read(blockBytes);
return blockBytes;
} catch (FileNotFoundException e) {
LOGGER.info("File {} not found: {}", filename, e.getMessage());
return null;
} catch (IOException e) {
LOGGER.info("Unable to read block {} from archive: {}", height, e.getMessage());
return null;
}
finally {
// Close the file
if (file != null) {
try {
file.close();
} catch (IOException e) {
// Failed to close, but no need to handle this
}
}
}
}
public void invalidateFileListCache() {
this.fileListCache.clear();
}
}

View File

@@ -0,0 +1,139 @@
package org.qortal.repository;
import org.qortal.api.model.BlockSignerSummary;
import org.qortal.data.block.BlockArchiveData;
import org.qortal.data.block.BlockData;
import org.qortal.data.block.BlockSummaryData;
import java.util.List;
public interface BlockArchiveRepository {
/**
* Returns BlockData from archive using block signature.
*
* @param signature
* @return block data, or null if not found in archive.
* @throws DataException
*/
public BlockData fromSignature(byte[] signature) throws DataException;
/**
* Return height of block in archive using block's signature.
*
* @param signature
* @return height, or 0 if not found in blockchain.
* @throws DataException
*/
public int getHeightFromSignature(byte[] signature) throws DataException;
/**
* Returns BlockData from archive using block height.
*
* @param height
* @return block data, or null if not found in blockchain.
* @throws DataException
*/
public BlockData fromHeight(int height) throws DataException;
/**
* Returns a list of BlockData objects from archive using
* block height range.
*
* @param startHeight
* @return a list of BlockData objects, or an empty list if
* not found in blockchain. It is not guaranteed that all
* requested blocks will be returned.
* @throws DataException
*/
public List<BlockData> fromRange(int startHeight, int endHeight) throws DataException;
/**
* Returns BlockData from archive using block reference.
* Currently relies on a child block being the one block
* higher than its parent. This limitation can be removed
* by storing the reference in the BlockArchive table, but
* this has been avoided to reduce space.
*
* @param reference
* @return block data, or null if either parent or child
* not found in the archive.
* @throws DataException
*/
public BlockData fromReference(byte[] reference) throws DataException;
/**
* Return height of block with timestamp just before passed timestamp.
*
* @param timestamp
* @return height, or 0 if not found in blockchain.
* @throws DataException
*/
public int getHeightFromTimestamp(long timestamp) throws DataException;
/**
* Returns block timestamp for a given height.
*
* @param height
* @return timestamp, or 0 if height is out of bounds.
* @throws DataException
*/
public long getTimestampFromHeight(int height) throws DataException;
/**
* Returns block summaries for blocks signed by passed public key, or reward-share with minter with passed public key.
*/
public List<BlockSummaryData> getBlockSummariesBySigner(byte[] signerPublicKey, Integer limit, Integer offset, Boolean reverse) throws DataException;
/**
* Returns summaries of block signers, optionally limited to passed addresses.
* This combines both the BlockArchive and the Blocks data into a single result set.
*/
public List<BlockSignerSummary> getBlockSigners(List<String> addresses, Integer limit, Integer offset, Boolean reverse) throws DataException;
/** Returns height of first unarchived block. */
public int getBlockArchiveHeight() throws DataException;
/** Sets new height for block archiving.
* <p>
* NOTE: performs implicit <tt>repository.saveChanges()</tt>.
*/
public void setBlockArchiveHeight(int archiveHeight) throws DataException;
/**
* Returns the block archive data for a given signature, from the block archive.
* <p>
* This method will return null if no block archive has been built for the
* requested signature. In those cases, the height (and other data) can be
* looked up using the Blocks table. This allows a block to be located in
* the archive when we only know its signature.
* <p>
*
* @param signature
* @throws DataException
*/
public BlockArchiveData getBlockArchiveDataForSignature(byte[] signature) throws DataException;
/**
* Saves a block archive entry into the repository.
* <p>
* This can be used to find the height of a block by its signature, without
* having access to the block data itself.
* <p>
*
* @param blockArchiveData
* @throws DataException
*/
public void save(BlockArchiveData blockArchiveData) throws DataException;
/**
* Deletes a block archive entry from the repository.
*
* @param blockArchiveData
* @throws DataException
*/
public void delete(BlockArchiveData blockArchiveData) throws DataException;
}

View File

@@ -0,0 +1,201 @@
package org.qortal.repository;
import com.google.common.primitives.Ints;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.block.Block;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockArchiveData;
import org.qortal.data.block.BlockData;
import org.qortal.settings.Settings;
import org.qortal.transform.TransformationException;
import org.qortal.transform.block.BlockTransformer;
import java.io.ByteArrayOutputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public class BlockArchiveWriter {
public enum BlockArchiveWriteResult {
OK,
STOPPING,
NOT_ENOUGH_BLOCKS,
BLOCK_NOT_FOUND
}
private static final Logger LOGGER = LogManager.getLogger(BlockArchiveWriter.class);
public static final long DEFAULT_FILE_SIZE_TARGET = 100 * 1024 * 1024; // 100MiB
private int startHeight;
private final int endHeight;
private final Repository repository;
private long fileSizeTarget = DEFAULT_FILE_SIZE_TARGET;
private boolean shouldEnforceFileSizeTarget = true;
private int writtenCount;
private int lastWrittenHeight;
private Path outputPath;
public BlockArchiveWriter(int startHeight, int endHeight, Repository repository) {
this.startHeight = startHeight;
this.endHeight = endHeight;
this.repository = repository;
}
public static int getMaxArchiveHeight(Repository repository) throws DataException {
// We must only archive trimmed blocks, or the archive will grow far too large
final int accountSignaturesTrimStartHeight = repository.getBlockRepository().getOnlineAccountsSignaturesTrimHeight();
final int atTrimStartHeight = repository.getATRepository().getAtTrimHeight();
final int trimStartHeight = Math.min(accountSignaturesTrimStartHeight, atTrimStartHeight);
return trimStartHeight - 1; // subtract 1 because these values represent the first _untrimmed_ block
}
public static boolean isArchiverUpToDate(Repository repository) throws DataException {
final int maxArchiveHeight = BlockArchiveWriter.getMaxArchiveHeight(repository);
final int actualArchiveHeight = repository.getBlockArchiveRepository().getBlockArchiveHeight();
final float progress = (float)actualArchiveHeight / (float) maxArchiveHeight;
LOGGER.debug(String.format("maxArchiveHeight: %d, actualArchiveHeight: %d, progress: %f",
maxArchiveHeight, actualArchiveHeight, progress));
// If archiver is within 95% of the maximum, treat it as up to date
// We need several percent as an allowance because the archiver will only
// save files when they reach the target size
return (progress >= 0.95);
}
public BlockArchiveWriteResult write() throws DataException, IOException, TransformationException, InterruptedException {
// Create the archive folder if it doesn't exist
// This is a subfolder of the db directory, to make bootstrapping easier
Path archivePath = Paths.get(Settings.getInstance().getRepositoryPath(), "archive").toAbsolutePath();
try {
Files.createDirectories(archivePath);
} catch (IOException e) {
LOGGER.info("Unable to create archive folder");
throw new DataException("Unable to create archive folder");
}
// Determine start height of blocks to fetch
if (startHeight <= 2) {
// Skip genesis block, as it's not designed to be transmitted, and we can build that from blockchain.json
// TODO: include genesis block if we can
startHeight = 2;
}
// Header bytes will store the block indexes
ByteArrayOutputStream headerBytes = new ByteArrayOutputStream();
// Bytes will store the actual block data
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
LOGGER.info(String.format("Fetching blocks from height %d...", startHeight));
int i = 0;
while (headerBytes.size() + bytes.size() < this.fileSizeTarget
|| this.shouldEnforceFileSizeTarget == false) {
if (Controller.isStopping()) {
return BlockArchiveWriteResult.STOPPING;
}
if (Controller.getInstance().isSynchronizing()) {
continue;
}
int currentHeight = startHeight + i;
if (currentHeight > endHeight) {
break;
}
//LOGGER.info("Fetching block {}...", currentHeight);
BlockData blockData = repository.getBlockRepository().fromHeight(currentHeight);
if (blockData == null) {
return BlockArchiveWriteResult.BLOCK_NOT_FOUND;
}
// Write the signature and height into the BlockArchive table
BlockArchiveData blockArchiveData = new BlockArchiveData(blockData);
repository.getBlockArchiveRepository().save(blockArchiveData);
repository.saveChanges();
// Write the block data to some byte buffers
Block block = new Block(repository, blockData);
int blockIndex = bytes.size();
// Write block index to header
headerBytes.write(Ints.toByteArray(blockIndex));
// Write block height
bytes.write(Ints.toByteArray(block.getBlockData().getHeight()));
byte[] blockBytes = BlockTransformer.toBytes(block);
// Write block length
bytes.write(Ints.toByteArray(blockBytes.length));
// Write block bytes
bytes.write(blockBytes);
i++;
}
int totalLength = headerBytes.size() + bytes.size();
LOGGER.info(String.format("Total length of %d blocks is %d bytes", i, totalLength));
// Validate file size, in case something went wrong
if (totalLength < fileSizeTarget && this.shouldEnforceFileSizeTarget) {
return BlockArchiveWriteResult.NOT_ENOUGH_BLOCKS;
}
// We have enough blocks to create a new file
int endHeight = startHeight + i - 1;
int version = 1;
String filePath = String.format("%s/%d-%d.dat", archivePath.toString(), startHeight, endHeight);
FileOutputStream fileOutputStream = new FileOutputStream(filePath);
// Write version number
fileOutputStream.write(Ints.toByteArray(version));
// Write start height
fileOutputStream.write(Ints.toByteArray(startHeight));
// Write end height
fileOutputStream.write(Ints.toByteArray(endHeight));
// Write total count
fileOutputStream.write(Ints.toByteArray(i));
// Write dynamic header (block indexes) segment length
fileOutputStream.write(Ints.toByteArray(headerBytes.size()));
// Write dynamic header (block indexes) data
headerBytes.writeTo(fileOutputStream);
// Write data segment (block data) length
fileOutputStream.write(Ints.toByteArray(bytes.size()));
// Write data
bytes.writeTo(fileOutputStream);
// Close the file
fileOutputStream.close();
// Invalidate cache so that the rest of the app picks up the new file
BlockArchiveReader.getInstance().invalidateFileListCache();
this.writtenCount = i;
this.lastWrittenHeight = endHeight;
this.outputPath = Paths.get(filePath);
return BlockArchiveWriteResult.OK;
}
public int getWrittenCount() {
return this.writtenCount;
}
public int getLastWrittenHeight() {
return this.lastWrittenHeight;
}
public Path getOutputPath() {
return this.outputPath;
}
public void setFileSizeTarget(long fileSizeTarget) {
this.fileSizeTarget = fileSizeTarget;
}
// For testing, to avoid having to pre-calculate file sizes
public void setShouldEnforceFileSizeTarget(boolean shouldEnforceFileSizeTarget) {
this.shouldEnforceFileSizeTarget = shouldEnforceFileSizeTarget;
}
}

View File

@@ -137,11 +137,6 @@ public interface BlockRepository {
*/
public List<BlockSummaryData> getBlockSummaries(int firstBlockHeight, int lastBlockHeight) throws DataException;
/**
* Returns block summaries for the passed height range, for API use.
*/
public List<BlockSummaryData> getBlockSummaries(Integer startHeight, Integer endHeight, Integer count) throws DataException;
/** Returns height of first trimmable online accounts signatures. */
public int getOnlineAccountsSignaturesTrimHeight() throws DataException;
@@ -166,6 +161,20 @@ public interface BlockRepository {
*/
public BlockData getDetachedBlockSignature(int startHeight) throws DataException;
/** Returns height of first prunable block. */
public int getBlockPruneHeight() throws DataException;
/** Sets new base height for block pruning.
* <p>
* NOTE: performs implicit <tt>repository.saveChanges()</tt>.
*/
public void setBlockPruneHeight(int pruneHeight) throws DataException;
/** Prunes full block data between passed heights. Returns number of pruned rows. */
public int pruneBlocks(int minHeight, int maxHeight) throws DataException;
/**
* Saves block into repository.
*

View File

@@ -0,0 +1,513 @@
package org.qortal.repository;
import org.apache.commons.io.FileUtils;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.block.BlockChain;
import org.qortal.controller.Controller;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.MintingAccountData;
import org.qortal.data.block.BlockData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.gui.SplashFrame;
import org.qortal.network.Network;
import org.qortal.repository.hsqldb.HSQLDBImportExport;
import org.qortal.repository.hsqldb.HSQLDBRepositoryFactory;
import org.qortal.settings.Settings;
import org.qortal.utils.NTP;
import org.qortal.utils.SevenZ;
import java.io.BufferedInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.file.*;
import java.security.SecureRandom;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.locks.ReentrantLock;
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
public class Bootstrap {
private Repository repository;
private int retryMinutes = 1;
private static final Logger LOGGER = LogManager.getLogger(Bootstrap.class);
/** The maximum number of untrimmed blocks allowed to be included in a bootstrap, beyond the trim threshold */
private static final int MAXIMUM_UNTRIMMED_BLOCKS = 100;
/** The maximum number of unpruned blocks allowed to be included in a bootstrap, beyond the prune threshold */
private static final int MAXIMUM_UNPRUNED_BLOCKS = 100;
public Bootstrap() {
}
public Bootstrap(Repository repository) {
this.repository = repository;
}
/**
* canCreateBootstrap()
* Performs basic initial checks to ensure everything is in order
* @return true if ready for bootstrap creation, or an exception if not
* All failure reasons are logged and included in the exception
* @throws DataException
*/
public boolean checkRepositoryState() throws DataException {
LOGGER.info("Checking repository state...");
final boolean isTopOnly = Settings.getInstance().isTopOnly();
final boolean archiveEnabled = Settings.getInstance().isArchiveEnabled();
// Make sure we have a repository instance
if (repository == null) {
throw new DataException("Repository instance required to check if we can create a bootstrap.");
}
// Require that a block archive has been built
if (!isTopOnly && !archiveEnabled) {
throw new DataException("Unable to create bootstrap because the block archive isn't enabled. " +
"Set {\"archivedEnabled\": true} in settings.json to fix.");
}
// Make sure that the block archiver is up to date
boolean upToDate = BlockArchiveWriter.isArchiverUpToDate(repository);
if (!upToDate) {
throw new DataException("Unable to create bootstrap because the block archive isn't fully built yet.");
}
// Ensure that this database contains the ATStatesHeightIndex which was missing in some cases
boolean hasAtStatesHeightIndex = repository.getATRepository().hasAtStatesHeightIndex();
if (!hasAtStatesHeightIndex) {
throw new DataException("Unable to create bootstrap due to missing ATStatesHeightIndex. A re-sync from genesis is needed.");
}
// Ensure we have synced NTP time
if (NTP.getTime() == null) {
throw new DataException("Unable to create bootstrap because the node hasn't synced its time yet.");
}
// Ensure the chain is synced
final BlockData chainTip = Controller.getInstance().getChainTip();
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp == null || chainTip.getTimestamp() < minLatestBlockTimestamp) {
throw new DataException("Unable to create bootstrap because the blockchain isn't fully synced.");
}
// FUTURE: ensure trim and prune settings are using default values
if (!isTopOnly) {
// We don't trim in top-only mode because we prune the blocks instead
// If we're not in top-only mode we should make sure that trimming is up to date
// Ensure that the online account signatures have been fully trimmed
final int accountsTrimStartHeight = repository.getBlockRepository().getOnlineAccountsSignaturesTrimHeight();
final long accountsUpperTrimmableTimestamp = NTP.getTime() - BlockChain.getInstance().getOnlineAccountSignaturesMaxLifetime();
final int accountsUpperTrimmableHeight = repository.getBlockRepository().getHeightFromTimestamp(accountsUpperTrimmableTimestamp);
final int accountsBlocksRemaining = accountsUpperTrimmableHeight - accountsTrimStartHeight;
if (accountsBlocksRemaining > MAXIMUM_UNTRIMMED_BLOCKS) {
throw new DataException(String.format("Blockchain is not fully trimmed. Please allow the node to run for longer, " +
"then try again. Blocks remaining (online accounts signatures): %d", accountsBlocksRemaining));
}
// Ensure that the AT states data has been fully trimmed
final int atTrimStartHeight = repository.getATRepository().getAtTrimHeight();
final long atUpperTrimmableTimestamp = chainTip.getTimestamp() - Settings.getInstance().getAtStatesMaxLifetime();
final int atUpperTrimmableHeight = repository.getBlockRepository().getHeightFromTimestamp(atUpperTrimmableTimestamp);
final int atBlocksRemaining = atUpperTrimmableHeight - atTrimStartHeight;
if (atBlocksRemaining > MAXIMUM_UNTRIMMED_BLOCKS) {
throw new DataException(String.format("Blockchain is not fully trimmed. Please allow the node to run " +
"for longer, then try again. Blocks remaining (AT states): %d", atBlocksRemaining));
}
}
// Ensure that blocks have been fully pruned
final int blockPruneStartHeight = repository.getBlockRepository().getBlockPruneHeight();
int blockUpperPrunableHeight = chainTip.getHeight() - Settings.getInstance().getPruneBlockLimit();
if (archiveEnabled) {
blockUpperPrunableHeight = repository.getBlockArchiveRepository().getBlockArchiveHeight() - 1;
}
final int blocksPruneRemaining = blockUpperPrunableHeight - blockPruneStartHeight;
if (blocksPruneRemaining > MAXIMUM_UNPRUNED_BLOCKS) {
throw new DataException(String.format("Blockchain is not fully pruned. Please allow the node to run " +
"for longer, then try again. Blocks remaining: %d", blocksPruneRemaining));
}
// Ensure that AT states have been fully pruned
final int atPruneStartHeight = repository.getATRepository().getAtPruneHeight();
int atUpperPrunableHeight = chainTip.getHeight() - Settings.getInstance().getPruneBlockLimit();
if (archiveEnabled) {
atUpperPrunableHeight = repository.getBlockArchiveRepository().getBlockArchiveHeight() - 1;
}
final int atPruneRemaining = atUpperPrunableHeight - atPruneStartHeight;
if (atPruneRemaining > MAXIMUM_UNPRUNED_BLOCKS) {
throw new DataException(String.format("Blockchain is not fully pruned. Please allow the node to run " +
"for longer, then try again. Blocks remaining (AT states): %d", atPruneRemaining));
}
LOGGER.info("Repository state checks passed");
return true;
}
/**
* validateBlockchain
* Performs quick validation of recent blocks in blockchain, prior to creating a bootstrap
* @return true if valid, an exception if not
* @throws DataException
*/
public boolean validateBlockchain() throws DataException {
LOGGER.info("Validating blockchain...");
try {
BlockChain.validate();
LOGGER.info("Blockchain is valid");
return true;
} catch (DataException e) {
throw new DataException(String.format("Blockchain validation failed: %s", e.getMessage()));
}
}
/**
* validateCompleteBlockchain
* Performs intensive validation of all blocks in blockchain
* @return true if valid, false if not
*/
public boolean validateCompleteBlockchain() {
LOGGER.info("Validating blockchain...");
try {
// Perform basic startup validation
BlockChain.validate();
// Perform more intensive full-chain validation
BlockChain.validateAllBlocks();
LOGGER.info("Blockchain is valid");
return true;
} catch (DataException e) {
LOGGER.info("Blockchain validation failed: {}", e.getMessage());
return false;
}
}
public String create() throws DataException, InterruptedException, IOException {
// Make sure we have a repository instance
if (repository == null) {
throw new DataException("Repository instance required in order to create a boostrap");
}
LOGGER.info("Deleting temp directory if it exists...");
this.deleteAllTempDirectories();
LOGGER.info("Acquiring blockchain lock...");
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
blockchainLock.lockInterruptibly();
Path inputPath = null;
Path outputPath = null;
try {
LOGGER.info("Exporting local data...");
repository.exportNodeLocalData();
LOGGER.info("Deleting trade bot states...");
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
for (TradeBotData tradeBotData : allTradeBotData) {
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
}
LOGGER.info("Deleting minting accounts...");
List<MintingAccountData> mintingAccounts = repository.getAccountRepository().getMintingAccounts();
for (MintingAccountData mintingAccount : mintingAccounts) {
repository.getAccountRepository().delete(mintingAccount.getPrivateKey());
}
repository.saveChanges();
LOGGER.info("Deleting peers list...");
repository.getNetworkRepository().deleteAllPeers();
repository.saveChanges();
LOGGER.info("Adding initial peers...");
Network.installInitialPeers(repository);
LOGGER.info("Creating bootstrap...");
// Timeout if the database isn't ready for backing up after 10 seconds
long timeout = 10 * 1000L;
repository.backup(false, "bootstrap", timeout);
LOGGER.info("Moving files to output directory...");
inputPath = Paths.get(Settings.getInstance().getRepositoryPath(), "bootstrap");
outputPath = Paths.get(this.createTempDirectory().toString(), "bootstrap");
// Move the db backup to a "bootstrap" folder in the root directory
Files.move(inputPath, outputPath, REPLACE_EXISTING);
// If in archive mode, copy the archive folder to inside the bootstrap folder
if (!Settings.getInstance().isTopOnly() && Settings.getInstance().isArchiveEnabled()) {
FileUtils.copyDirectory(
Paths.get(Settings.getInstance().getRepositoryPath(), "archive").toFile(),
Paths.get(outputPath.toString(), "archive").toFile()
);
}
LOGGER.info("Preparing output path...");
Path compressedOutputPath = this.getBootstrapOutputPath();
try {
Files.delete(compressedOutputPath);
} catch (NoSuchFileException e) {
// Doesn't exist, so no need to delete
}
LOGGER.info("Compressing...");
SevenZ.compress(compressedOutputPath.toString(), outputPath.toFile());
LOGGER.info("Generating checksum file...");
String checksum = Crypto.digestHexString(compressedOutputPath.toFile(), 1024*1024);
Path checksumPath = Paths.get(String.format("%s.sha256", compressedOutputPath.toString()));
Files.writeString(checksumPath, checksum, StandardOpenOption.CREATE);
// Return the path to the compressed bootstrap file
LOGGER.info("Bootstrap creation complete. Output file: {}", compressedOutputPath.toAbsolutePath().toString());
return compressedOutputPath.toAbsolutePath().toString();
}
catch (TimeoutException e) {
throw new DataException(String.format("Unable to create bootstrap due to timeout: %s", e.getMessage()));
}
finally {
try {
LOGGER.info("Re-importing local data...");
Path exportPath = HSQLDBImportExport.getExportDirectory(false);
repository.importDataFromFile(Paths.get(exportPath.toString(), "TradeBotStates.json").toString());
repository.importDataFromFile(Paths.get(exportPath.toString(), "MintingAccounts.json").toString());
repository.saveChanges();
} catch (IOException e) {
LOGGER.info("Unable to re-import local data, but created bootstrap is still valid. {}", e);
}
LOGGER.info("Unlocking blockchain...");
blockchainLock.unlock();
// Cleanup
LOGGER.info("Cleaning up...");
Thread.sleep(5000L);
this.deleteAllTempDirectories();
}
}
public void startImport() throws InterruptedException {
while (!Controller.isStopping()) {
try (final Repository repository = RepositoryManager.getRepository()) {
this.repository = repository;
this.updateStatus("Starting import of bootstrap...");
this.doImport();
break;
} catch (DataException e) {
LOGGER.info("Bootstrap import failed", e);
this.updateStatus(String.format("Bootstrapping failed. Retrying in %d minutes...", retryMinutes));
Thread.sleep(retryMinutes * 60 * 1000L);
retryMinutes *= 2;
}
}
}
private void doImport() throws DataException {
Path path = null;
try {
Path tempDir = this.createTempDirectory();
String filename = String.format("%s%s", Settings.getInstance().getBootstrapFilenamePrefix(), this.getFilename());
path = Paths.get(tempDir.toString(), filename);
this.downloadToPath(path);
this.importFromPath(path);
} catch (InterruptedException | DataException | IOException e) {
throw new DataException("Unable to import bootstrap", e);
}
finally {
if (path != null) {
try {
Files.delete(path);
} catch (IOException e) {
// Temp folder will be cleaned up below, so ignore this failure
}
}
this.deleteAllTempDirectories();
}
}
private String getFilename() {
boolean isTopOnly = Settings.getInstance().isTopOnly();
boolean archiveEnabled = Settings.getInstance().isArchiveEnabled();
boolean isTestnet = Settings.getInstance().isTestNet();
String prefix = isTestnet ? "testnet-" : "";
if (isTopOnly) {
return prefix.concat("bootstrap-toponly.7z");
}
else if (archiveEnabled) {
return prefix.concat("bootstrap-archive.7z");
}
else {
return prefix.concat("bootstrap-full.7z");
}
}
private void downloadToPath(Path path) throws DataException {
String bootstrapHost = this.getRandomHost();
String bootstrapFilename = this.getFilename();
String bootstrapUrl = String.format("%s/%s", bootstrapHost, bootstrapFilename);
String type = Settings.getInstance().isTopOnly() ? "top-only" : "full node";
SplashFrame.getInstance().updateStatus(String.format("Downloading %s bootstrap...", type));
LOGGER.info(String.format("Downloading %s bootstrap from %s ...", type, bootstrapUrl));
// Delete an existing file if it exists
try {
Files.delete(path);
} catch (IOException e) {
// No need to do anything
}
// Get the total file size
URL url;
long fileSize;
try {
url = new URL(bootstrapUrl);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("HEAD");
connection.connect();
fileSize = connection.getContentLengthLong();
connection.disconnect();
} catch (MalformedURLException e) {
throw new DataException(String.format("Malformed URL when downloading bootstrap: %s", e.getMessage()));
} catch (IOException e) {
throw new DataException(String.format("Unable to get bootstrap file size from %s. " +
"Please check your internet connection.", e.getMessage()));
}
// Download the file and update the status with progress
try (BufferedInputStream in = new BufferedInputStream(url.openStream());
FileOutputStream fileOutputStream = new FileOutputStream(path.toFile())) {
byte[] buffer = new byte[1024 * 1024];
long downloaded = 0;
int bytesRead;
while ((bytesRead = in.read(buffer, 0, 1024)) != -1) {
fileOutputStream.write(buffer, 0, bytesRead);
downloaded += bytesRead;
if (fileSize > 0) {
int progress = (int)((double)downloaded / (double)fileSize * 100);
SplashFrame.getInstance().updateStatus(String.format("Downloading %s bootstrap... (%d%%)", type, progress));
}
}
} catch (IOException e) {
throw new DataException(String.format("Unable to download bootstrap: %s", e.getMessage()));
}
}
public String getRandomHost() {
// Select a random host from bootstrapHosts
String[] hosts = Settings.getInstance().getBootstrapHosts();
int index = new SecureRandom().nextInt(hosts.length);
String bootstrapHost = hosts[index];
return bootstrapHost;
}
public void importFromPath(Path path) throws InterruptedException, DataException, IOException {
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
blockchainLock.lockInterruptibly();
try {
this.updateStatus("Stopping repository...");
// Close the repository while we are still able to
// Otherwise, the caller will run into difficulties when it tries to close it
repository.discardChanges();
repository.close();
// Now close the repository factory so that we can swap out the database files
RepositoryManager.closeRepositoryFactory();
this.updateStatus("Deleting existing repository...");
Path input = path.toAbsolutePath();
Path output = path.toAbsolutePath().getParent().toAbsolutePath();
Path inputPath = Paths.get(output.toString(), "bootstrap");
Path outputPath = Paths.get(Settings.getInstance().getRepositoryPath());
FileUtils.deleteDirectory(outputPath.toFile());
this.updateStatus("Extracting bootstrap...");
SevenZ.decompress(input.toString(), output.toFile());
if (!inputPath.toFile().exists()) {
throw new DataException("Extracted bootstrap doesn't exist");
}
// Move the "bootstrap" folder in place of the "db" folder
this.updateStatus("Moving files to output directory...");
Files.move(inputPath, outputPath);
this.updateStatus("Starting repository from bootstrap...");
}
finally {
RepositoryFactory repositoryFactory = new HSQLDBRepositoryFactory(Controller.getRepositoryUrl());
RepositoryManager.setRepositoryFactory(repositoryFactory);
blockchainLock.unlock();
}
}
private Path createTempDirectory() throws IOException {
Path initialPath = Paths.get(Settings.getInstance().getRepositoryPath()).toAbsolutePath().getParent();
String baseDir = Paths.get(initialPath.toString(), "tmp").toFile().getCanonicalPath();
String identifier = UUID.randomUUID().toString();
Path tempDir = Paths.get(baseDir, identifier);
Files.createDirectories(tempDir);
return tempDir;
}
private void deleteAllTempDirectories() {
Path initialPath = Paths.get(Settings.getInstance().getRepositoryPath()).toAbsolutePath().getParent();
Path path = Paths.get(initialPath.toString(), "tmp");
try {
FileUtils.deleteDirectory(path.toFile());
} catch (IOException e) {
LOGGER.info("Unable to delete temp directory path: {}", path.toString(), e);
}
}
public Path getBootstrapOutputPath() {
Path initialPath = Paths.get(Settings.getInstance().getRepositoryPath()).toAbsolutePath().getParent();
String compressedFilename = String.format("%s%s", Settings.getInstance().getBootstrapFilenamePrefix(), this.getFilename());
Path compressedOutputPath = Paths.get(initialPath.toString(), compressedFilename);
return compressedOutputPath;
}
private void updateStatus(String text) {
LOGGER.info(text);
SplashFrame.getInstance().updateStatus(text);
}
}

View File

@@ -1,5 +1,8 @@
package org.qortal.repository;
import java.io.IOException;
import java.util.concurrent.TimeoutException;
public interface Repository extends AutoCloseable {
public ATRepository getATRepository();
@@ -12,6 +15,8 @@ public interface Repository extends AutoCloseable {
public BlockRepository getBlockRepository();
public BlockArchiveRepository getBlockArchiveRepository();
public ChatRepository getChatRepository();
public CrossChainRepository getCrossChainRepository();
@@ -45,14 +50,16 @@ public interface Repository extends AutoCloseable {
public void setDebug(boolean debugState);
public void backup(boolean quick) throws DataException;
public void backup(boolean quick, String name, Long timeout) throws DataException, TimeoutException;
public void performPeriodicMaintenance() throws DataException;
public void performPeriodicMaintenance(Long timeout) throws DataException, TimeoutException;
public void exportNodeLocalData() throws DataException;
public void importDataFromFile(String filename) throws DataException;
public void importDataFromFile(String filename) throws DataException, IOException;
public void checkConsistency() throws DataException;
public static void attemptRecovery(String connectionUrl, String name) throws DataException {}
}

View File

@@ -1,8 +1,18 @@
package org.qortal.repository;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.gui.SplashFrame;
import org.qortal.repository.hsqldb.HSQLDBDatabaseArchiving;
import org.qortal.repository.hsqldb.HSQLDBDatabasePruning;
import org.qortal.repository.hsqldb.HSQLDBRepository;
import org.qortal.settings.Settings;
import java.sql.SQLException;
import java.util.concurrent.TimeoutException;
public abstract class RepositoryManager {
private static final Logger LOGGER = LogManager.getLogger(RepositoryManager.class);
private static RepositoryFactory repositoryFactory = null;
@@ -43,14 +53,60 @@ public abstract class RepositoryManager {
repositoryFactory = null;
}
public static void backup(boolean quick) {
public static void backup(boolean quick, String name, Long timeout) throws TimeoutException {
try (final Repository repository = getRepository()) {
repository.backup(quick);
repository.backup(quick, name, timeout);
} catch (DataException e) {
// Backup is best-effort so don't complain
}
}
public static boolean archive(Repository repository) {
// Bulk archive the database the first time we use archive mode
if (Settings.getInstance().isArchiveEnabled()) {
if (RepositoryManager.canArchiveOrPrune()) {
try {
return HSQLDBDatabaseArchiving.buildBlockArchive(repository, BlockArchiveWriter.DEFAULT_FILE_SIZE_TARGET);
} catch (DataException e) {
LOGGER.info("Unable to build block archive. The database may have been left in an inconsistent state.");
}
}
else {
LOGGER.info("Unable to build block archive due to missing ATStatesHeightIndex. Bootstrapping is recommended.");
LOGGER.info("To bootstrap, stop the core and delete the db folder, then start the core again.");
SplashFrame.getInstance().updateStatus("Missing index. Bootstrapping is recommended.");
}
}
return false;
}
public static boolean prune(Repository repository) {
// Bulk prune the database the first time we use top-only or block archive mode
if (Settings.getInstance().isTopOnly() ||
Settings.getInstance().isArchiveEnabled()) {
if (RepositoryManager.canArchiveOrPrune()) {
try {
boolean prunedATStates = HSQLDBDatabasePruning.pruneATStates((HSQLDBRepository) repository);
boolean prunedBlocks = HSQLDBDatabasePruning.pruneBlocks((HSQLDBRepository) repository);
// Perform repository maintenance to shrink the db size down
if (prunedATStates && prunedBlocks) {
HSQLDBDatabasePruning.performMaintenance(repository);
return true;
}
} catch (SQLException | DataException e) {
LOGGER.info("Unable to bulk prune AT states. The database may have been left in an inconsistent state.");
}
}
else {
LOGGER.info("Unable to prune blocks due to missing ATStatesHeightIndex. Bootstrapping is recommended.");
}
}
return false;
}
public static void setRequestedCheckpoint(Boolean quick) {
quickCheckpointRequested = quick;
}
@@ -77,4 +133,12 @@ public abstract class RepositoryManager {
return SQLException.class.isInstance(cause) && repositoryFactory.isDeadlockException((SQLException) cause);
}
public static boolean canArchiveOrPrune() {
try (final Repository repository = getRepository()) {
return repository.getATRepository().hasAtStatesHeightIndex();
} catch (DataException e) {
return false;
}
}
}

View File

@@ -8,6 +8,7 @@ import java.util.Set;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.repository.ATRepository;
@@ -600,6 +601,44 @@ public class HSQLDBATRepository implements ATRepository {
return atStates;
}
@Override
public void rebuildLatestAtStates() throws DataException {
// latestATStatesLock is to prevent concurrent updates on LatestATStates
// that could result in one process using a partial or empty dataset
// because it was in the process of being rebuilt by another thread
synchronized (this.repository.latestATStatesLock) {
LOGGER.trace("Rebuilding latest AT states...");
// Rebuild cache of latest AT states that we can't trim
String deleteSql = "DELETE FROM LatestATStates";
try {
this.repository.executeCheckedUpdate(deleteSql);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to delete temporary latest AT states cache from repository", e);
}
String insertSql = "INSERT INTO LatestATStates ("
+ "SELECT AT_address, height FROM ATs "
+ "CROSS JOIN LATERAL("
+ "SELECT height FROM ATStates "
+ "WHERE ATStates.AT_address = ATs.AT_address "
+ "ORDER BY AT_address DESC, height DESC LIMIT 1"
+ ") "
+ ")";
try {
this.repository.executeCheckedUpdate(insertSql);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to populate temporary latest AT states cache in repository", e);
}
this.repository.saveChanges();
LOGGER.trace("Rebuilt latest AT states");
}
}
@Override
public int getAtTrimHeight() throws DataException {
String sql = "SELECT AT_trim_height FROM DatabaseInfo";
@@ -625,63 +664,153 @@ public class HSQLDBATRepository implements ATRepository {
this.repository.executeCheckedUpdate(updateSql, trimHeight);
this.repository.saveChanges();
} catch (SQLException e) {
repository.examineException(e);
this.repository.examineException(e);
throw new DataException("Unable to set AT state trim height in repository", e);
}
}
}
@Override
public void prepareForAtStateTrimming() throws DataException {
// Rebuild cache of latest AT states that we can't trim
String deleteSql = "DELETE FROM LatestATStates";
try {
this.repository.executeCheckedUpdate(deleteSql);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to delete temporary latest AT states cache from repository", e);
}
String insertSql = "INSERT INTO LatestATStates ("
+ "SELECT AT_address, height FROM ATs "
+ "CROSS JOIN LATERAL("
+ "SELECT height FROM ATStates "
+ "WHERE ATStates.AT_address = ATs.AT_address "
+ "ORDER BY AT_address DESC, height DESC LIMIT 1"
+ ") "
+ ")";
try {
this.repository.executeCheckedUpdate(insertSql);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to populate temporary latest AT states cache in repository", e);
}
}
@Override
public int trimAtStates(int minHeight, int maxHeight, int limit) throws DataException {
if (minHeight >= maxHeight)
return 0;
// We're often called so no need to trim all states in one go.
// Limit updates to reduce CPU and memory load.
String sql = "DELETE FROM ATStatesData "
+ "WHERE height BETWEEN ? AND ? "
+ "AND NOT EXISTS("
// latestATStatesLock is to prevent concurrent updates on LatestATStates
// that could result in one process using a partial or empty dataset
// because it was in the process of being rebuilt by another thread
synchronized (this.repository.latestATStatesLock) {
// We're often called so no need to trim all states in one go.
// Limit updates to reduce CPU and memory load.
String sql = "DELETE FROM ATStatesData "
+ "WHERE height BETWEEN ? AND ? "
+ "AND NOT EXISTS("
+ "SELECT TRUE FROM LatestATStates "
+ "WHERE LatestATStates.AT_address = ATStatesData.AT_address "
+ "AND LatestATStates.height = ATStatesData.height"
+ ") "
+ "LIMIT ?";
+ ") "
+ "LIMIT ?";
try {
return this.repository.executeCheckedUpdate(sql, minHeight, maxHeight, limit);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to trim AT states in repository", e);
try {
int modifiedRows = this.repository.executeCheckedUpdate(sql, minHeight, maxHeight, limit);
this.repository.saveChanges();
return modifiedRows;
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to trim AT states in repository", e);
}
}
}
@Override
public int getAtPruneHeight() throws DataException {
String sql = "SELECT AT_prune_height FROM DatabaseInfo";
try (ResultSet resultSet = this.repository.checkedExecute(sql)) {
if (resultSet == null)
return 0;
return resultSet.getInt(1);
} catch (SQLException e) {
throw new DataException("Unable to fetch AT state prune height from repository", e);
}
}
@Override
public void setAtPruneHeight(int pruneHeight) throws DataException {
// trimHeightsLock is to prevent concurrent update on DatabaseInfo
// that could result in "transaction rollback: serialization failure"
synchronized (this.repository.trimHeightsLock) {
String updateSql = "UPDATE DatabaseInfo SET AT_prune_height = ?";
try {
this.repository.executeCheckedUpdate(updateSql, pruneHeight);
this.repository.saveChanges();
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to set AT state prune height in repository", e);
}
}
}
@Override
public int pruneAtStates(int minHeight, int maxHeight) throws DataException {
// latestATStatesLock is to prevent concurrent updates on LatestATStates
// that could result in one process using a partial or empty dataset
// because it was in the process of being rebuilt by another thread
synchronized (this.repository.latestATStatesLock) {
int deletedCount = 0;
for (int height = minHeight; height <= maxHeight; height++) {
// Give up if we're stopping
if (Controller.isStopping()) {
return deletedCount;
}
// Get latest AT states for this height
List<String> atAddresses = new ArrayList<>();
String updateSql = "SELECT AT_address FROM LatestATStates WHERE height = ?";
try (ResultSet resultSet = this.repository.checkedExecute(updateSql, height)) {
if (resultSet != null) {
do {
String atAddress = resultSet.getString(1);
atAddresses.add(atAddress);
} while (resultSet.next());
}
} catch (SQLException e) {
throw new DataException("Unable to fetch latest AT states from repository", e);
}
List<ATStateData> atStates = this.getBlockATStatesAtHeight(height);
for (ATStateData atState : atStates) {
//LOGGER.info("Found atState {} at height {}", atState.getATAddress(), atState.getHeight());
// Give up if we're stopping
if (Controller.isStopping()) {
return deletedCount;
}
if (atAddresses.contains(atState.getATAddress())) {
// We don't want to delete this AT state because it is still active
LOGGER.trace("Skipping atState {} at height {}", atState.getATAddress(), atState.getHeight());
continue;
}
// Safe to delete everything else for this height
try {
this.repository.delete("ATStates", "AT_address = ? AND height = ?",
atState.getATAddress(), atState.getHeight());
deletedCount++;
} catch (SQLException e) {
throw new DataException("Unable to delete AT state data from repository", e);
}
}
}
this.repository.saveChanges();
return deletedCount;
}
}
@Override
public boolean hasAtStatesHeightIndex() throws DataException {
String sql = "SELECT INDEX_NAME FROM INFORMATION_SCHEMA.SYSTEM_INDEXINFO where INDEX_NAME='ATSTATESHEIGHTINDEX'";
try (ResultSet resultSet = this.repository.checkedExecute(sql)) {
return resultSet != null;
} catch (SQLException e) {
throw new DataException("Unable to check for ATStatesHeightIndex in repository", e);
}
}
@Override
public void save(ATStateData atStateData) throws DataException {
// We shouldn't ever save partial ATStateData

View File

@@ -904,6 +904,25 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public MintingAccountData getMintingAccount(byte[] mintingAccountKey) throws DataException {
try (ResultSet resultSet = this.repository.checkedExecute("SELECT minter_private_key, minter_public_key " +
"FROM MintingAccounts WHERE minter_private_key = ? OR minter_public_key = ?",
mintingAccountKey, mintingAccountKey)) {
if (resultSet == null)
return null;
byte[] minterPrivateKey = resultSet.getBytes(1);
byte[] minterPublicKey = resultSet.getBytes(2);
return new MintingAccountData(minterPrivateKey, minterPublicKey);
} catch (SQLException e) {
throw new DataException("Unable to fetch minting accounts from repository", e);
}
}
@Override
public void save(MintingAccountData mintingAccountData) throws DataException {
HSQLDBSaver saveHelper = new HSQLDBSaver("MintingAccounts");

View File

@@ -0,0 +1,310 @@
package org.qortal.repository.hsqldb;
import org.qortal.api.ApiError;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.model.BlockSignerSummary;
import org.qortal.block.Block;
import org.qortal.data.block.BlockArchiveData;
import org.qortal.data.block.BlockData;
import org.qortal.data.block.BlockSummaryData;
import org.qortal.repository.BlockArchiveReader;
import org.qortal.repository.BlockArchiveRepository;
import org.qortal.repository.DataException;
import org.qortal.utils.Triple;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class HSQLDBBlockArchiveRepository implements BlockArchiveRepository {
protected HSQLDBRepository repository;
public HSQLDBBlockArchiveRepository(HSQLDBRepository repository) {
this.repository = repository;
}
@Override
public BlockData fromSignature(byte[] signature) throws DataException {
Triple blockInfo = BlockArchiveReader.getInstance().fetchBlockWithSignature(signature, this.repository);
if (blockInfo != null) {
return (BlockData) blockInfo.getA();
}
return null;
}
@Override
public int getHeightFromSignature(byte[] signature) throws DataException {
Integer height = BlockArchiveReader.getInstance().fetchHeightForSignature(signature, this.repository);
if (height == null || height == 0) {
return 0;
}
return height;
}
@Override
public BlockData fromHeight(int height) throws DataException {
Triple blockInfo = BlockArchiveReader.getInstance().fetchBlockAtHeight(height);
if (blockInfo != null) {
return (BlockData) blockInfo.getA();
}
return null;
}
@Override
public List<BlockData> fromRange(int startHeight, int endHeight) throws DataException {
List<BlockData> blocks = new ArrayList<>();
for (int height = startHeight; height < endHeight; height++) {
BlockData blockData = this.fromHeight(height);
if (blockData == null) {
return blocks;
}
blocks.add(blockData);
}
return blocks;
}
@Override
public BlockData fromReference(byte[] reference) throws DataException {
BlockData referenceBlock = this.repository.getBlockArchiveRepository().fromSignature(reference);
if (referenceBlock == null) {
// Try the main block repository. Needed for genesis block.
referenceBlock = this.repository.getBlockRepository().fromSignature(reference);
}
if (referenceBlock != null) {
int height = referenceBlock.getHeight();
if (height > 0) {
// Request the block at height + 1
Triple blockInfo = BlockArchiveReader.getInstance().fetchBlockAtHeight(height + 1);
if (blockInfo != null) {
return (BlockData) blockInfo.getA();
}
}
}
return null;
}
@Override
public int getHeightFromTimestamp(long timestamp) throws DataException {
String sql = "SELECT height FROM BlockArchive WHERE minted_when <= ? ORDER BY minted_when DESC, height DESC LIMIT 1";
try (ResultSet resultSet = this.repository.checkedExecute(sql, timestamp)) {
if (resultSet == null) {
return 0;
}
return resultSet.getInt(1);
} catch (SQLException e) {
throw new DataException("Error fetching height from BlockArchive repository", e);
}
}
@Override
public long getTimestampFromHeight(int height) throws DataException {
String sql = "SELECT minted_when FROM BlockArchive WHERE height = ?";
try (ResultSet resultSet = this.repository.checkedExecute(sql, height)) {
if (resultSet == null)
return 0;
return resultSet.getLong(1);
} catch (SQLException e) {
throw new DataException("Error obtaining block timestamp by height from BlockArchive repository", e);
}
}
@Override
public List<BlockSummaryData> getBlockSummariesBySigner(byte[] signerPublicKey, Integer limit, Integer offset, Boolean reverse) throws DataException {
StringBuilder sql = new StringBuilder(512);
sql.append("SELECT signature, height, BlockArchive.minter FROM ");
// List of minter account's public key and reward-share public keys with minter's public key
sql.append("(SELECT * FROM (VALUES (CAST(? AS QortalPublicKey))) UNION (SELECT reward_share_public_key FROM RewardShares WHERE minter_public_key = ?)) AS PublicKeys (public_key) ");
// Match BlockArchive blocks signed with public key from above list
sql.append("JOIN BlockArchive ON BlockArchive.minter = public_key ");
sql.append("ORDER BY BlockArchive.height ");
if (reverse != null && reverse)
sql.append("DESC ");
HSQLDBRepository.limitOffsetSql(sql, limit, offset);
List<BlockSummaryData> blockSummaries = new ArrayList<>();
try (ResultSet resultSet = this.repository.checkedExecute(sql.toString(), signerPublicKey, signerPublicKey)) {
if (resultSet == null)
return blockSummaries;
do {
byte[] signature = resultSet.getBytes(1);
int height = resultSet.getInt(2);
byte[] blockMinterPublicKey = resultSet.getBytes(3);
// Fetch additional info from the archive itself
int onlineAccountsCount = 0;
BlockData blockData = this.fromSignature(signature);
if (blockData != null) {
onlineAccountsCount = blockData.getOnlineAccountsCount();
}
BlockSummaryData blockSummary = new BlockSummaryData(height, signature, blockMinterPublicKey, onlineAccountsCount);
blockSummaries.add(blockSummary);
} while (resultSet.next());
return blockSummaries;
} catch (SQLException e) {
throw new DataException("Unable to fetch minter's block summaries from repository", e);
}
}
@Override
public List<BlockSignerSummary> getBlockSigners(List<String> addresses, Integer limit, Integer offset, Boolean reverse) throws DataException {
String subquerySql = "SELECT minter, COUNT(signature) FROM (" +
"(SELECT minter, signature FROM Blocks) UNION ALL (SELECT minter, signature FROM BlockArchive)" +
") GROUP BY minter";
StringBuilder sql = new StringBuilder(1024);
sql.append("SELECT DISTINCT block_minter, n_blocks, minter_public_key, minter, recipient FROM (");
sql.append(subquerySql);
sql.append(") AS Minters (block_minter, n_blocks) LEFT OUTER JOIN RewardShares ON reward_share_public_key = block_minter ");
if (addresses != null && !addresses.isEmpty()) {
sql.append(" LEFT OUTER JOIN Accounts AS BlockMinterAccounts ON BlockMinterAccounts.public_key = block_minter ");
sql.append(" LEFT OUTER JOIN Accounts AS RewardShareMinterAccounts ON RewardShareMinterAccounts.public_key = minter_public_key ");
sql.append(" JOIN (VALUES ");
final int addressesSize = addresses.size();
for (int ai = 0; ai < addressesSize; ++ai) {
if (ai != 0)
sql.append(", ");
sql.append("(?)");
}
sql.append(") AS FilterAccounts (account) ");
sql.append(" ON FilterAccounts.account IN (recipient, BlockMinterAccounts.account, RewardShareMinterAccounts.account) ");
} else {
addresses = Collections.emptyList();
}
sql.append("ORDER BY n_blocks ");
if (reverse != null && reverse)
sql.append("DESC ");
HSQLDBRepository.limitOffsetSql(sql, limit, offset);
List<BlockSignerSummary> summaries = new ArrayList<>();
try (ResultSet resultSet = this.repository.checkedExecute(sql.toString(), addresses.toArray())) {
if (resultSet == null)
return summaries;
do {
byte[] blockMinterPublicKey = resultSet.getBytes(1);
int nBlocks = resultSet.getInt(2);
// May not be present if no reward-share:
byte[] mintingAccountPublicKey = resultSet.getBytes(3);
String minterAccount = resultSet.getString(4);
String recipientAccount = resultSet.getString(5);
BlockSignerSummary blockSignerSummary;
if (recipientAccount == null)
blockSignerSummary = new BlockSignerSummary(blockMinterPublicKey, nBlocks);
else
blockSignerSummary = new BlockSignerSummary(blockMinterPublicKey, nBlocks, mintingAccountPublicKey, minterAccount, recipientAccount);
summaries.add(blockSignerSummary);
} while (resultSet.next());
return summaries;
} catch (SQLException e) {
throw new DataException("Unable to fetch block minters from repository", e);
}
}
@Override
public int getBlockArchiveHeight() throws DataException {
String sql = "SELECT block_archive_height FROM DatabaseInfo";
try (ResultSet resultSet = this.repository.checkedExecute(sql)) {
if (resultSet == null)
return 0;
return resultSet.getInt(1);
} catch (SQLException e) {
throw new DataException("Unable to fetch block archive height from repository", e);
}
}
@Override
public void setBlockArchiveHeight(int archiveHeight) throws DataException {
// trimHeightsLock is to prevent concurrent update on DatabaseInfo
// that could result in "transaction rollback: serialization failure"
synchronized (this.repository.trimHeightsLock) {
String updateSql = "UPDATE DatabaseInfo SET block_archive_height = ?";
try {
this.repository.executeCheckedUpdate(updateSql, archiveHeight);
this.repository.saveChanges();
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to set block archive height in repository", e);
}
}
}
@Override
public BlockArchiveData getBlockArchiveDataForSignature(byte[] signature) throws DataException {
String sql = "SELECT height, signature, minted_when, minter FROM BlockArchive WHERE signature = ? LIMIT 1";
try (ResultSet resultSet = this.repository.checkedExecute(sql, signature)) {
if (resultSet == null) {
return null;
}
int height = resultSet.getInt(1);
byte[] sig = resultSet.getBytes(2);
long timestamp = resultSet.getLong(3);
byte[] minterPublicKey = resultSet.getBytes(4);
return new BlockArchiveData(sig, height, timestamp, minterPublicKey);
} catch (SQLException e) {
throw new DataException("Error fetching height from BlockArchive repository", e);
}
}
@Override
public void save(BlockArchiveData blockArchiveData) throws DataException {
HSQLDBSaver saveHelper = new HSQLDBSaver("BlockArchive");
saveHelper.bind("signature", blockArchiveData.getSignature())
.bind("height", blockArchiveData.getHeight())
.bind("minted_when", blockArchiveData.getTimestamp())
.bind("minter", blockArchiveData.getMinterPublicKey());
try {
saveHelper.execute(this.repository);
} catch (SQLException e) {
throw new DataException("Unable to save SimpleBlockData into BlockArchive repository", e);
}
}
@Override
public void delete(BlockArchiveData blockArchiveData) throws DataException {
try {
this.repository.delete("BlockArchive",
"block_signature = ?", blockArchiveData.getSignature());
} catch (SQLException e) {
throw new DataException("Unable to delete SimpleBlockData from BlockArchive repository", e);
}
}
}

View File

@@ -10,6 +10,7 @@ import org.qortal.api.model.BlockSignerSummary;
import org.qortal.data.block.BlockData;
import org.qortal.data.block.BlockSummaryData;
import org.qortal.data.block.BlockTransactionData;
import org.qortal.data.block.BlockArchiveData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.repository.BlockRepository;
import org.qortal.repository.DataException;
@@ -382,86 +383,6 @@ public class HSQLDBBlockRepository implements BlockRepository {
}
}
@Override
public List<BlockSummaryData> getBlockSummaries(Integer startHeight, Integer endHeight, Integer count) throws DataException {
StringBuilder sql = new StringBuilder(512);
List<Object> bindParams = new ArrayList<>();
sql.append("SELECT signature, height, minter, online_accounts_count, minted_when, transaction_count ");
/*
* start end count result
* 10 40 null blocks 10 to 39 (excludes end block, ignore count)
*
* null null null blocks 1 to 50 (assume count=50, maybe start=1)
* 30 null null blocks 30 to 79 (assume count=50)
* 30 null 10 blocks 30 to 39
*
* null null 50 last 50 blocks? so if max(blocks.height) is 200, then blocks 151 to 200
* null 200 null blocks 150 to 199 (excludes end block, assume count=50)
* null 200 10 blocks 190 to 199 (excludes end block)
*/
if (startHeight != null && endHeight != null) {
sql.append("FROM Blocks ");
sql.append("WHERE height BETWEEN ? AND ?");
bindParams.add(startHeight);
bindParams.add(Integer.valueOf(endHeight - 1));
} else if (endHeight != null || (startHeight == null && count != null)) {
// we are going to return blocks from the end of the chain
if (count == null)
count = 50;
if (endHeight == null) {
sql.append("FROM (SELECT height FROM Blocks ORDER BY height DESC LIMIT 1) AS MaxHeights (max_height) ");
sql.append("JOIN Blocks ON height BETWEEN (max_height - ? + 1) AND max_height ");
bindParams.add(count);
} else {
sql.append("FROM Blocks ");
sql.append("WHERE height BETWEEN ? AND ?");
bindParams.add(Integer.valueOf(endHeight - count));
bindParams.add(Integer.valueOf(endHeight - 1));
}
} else {
// we are going to return blocks from the start of the chain
if (startHeight == null)
startHeight = 1;
if (count == null)
count = 50;
sql.append("FROM Blocks ");
sql.append("WHERE height BETWEEN ? AND ?");
bindParams.add(startHeight);
bindParams.add(Integer.valueOf(startHeight + count - 1));
}
List<BlockSummaryData> blockSummaries = new ArrayList<>();
try (ResultSet resultSet = this.repository.checkedExecute(sql.toString(), bindParams.toArray())) {
if (resultSet == null)
return blockSummaries;
do {
byte[] signature = resultSet.getBytes(1);
int height = resultSet.getInt(2);
byte[] minterPublicKey = resultSet.getBytes(3);
int onlineAccountsCount = resultSet.getInt(4);
long timestamp = resultSet.getLong(5);
int transactionCount = resultSet.getInt(6);
BlockSummaryData blockSummary = new BlockSummaryData(height, signature, minterPublicKey, onlineAccountsCount,
timestamp, transactionCount);
blockSummaries.add(blockSummary);
} while (resultSet.next());
return blockSummaries;
} catch (SQLException e) {
throw new DataException("Unable to fetch height-ranged block summaries from repository", e);
}
}
@Override
public int getOnlineAccountsSignaturesTrimHeight() throws DataException {
String sql = "SELECT online_signatures_trim_height FROM DatabaseInfo";
@@ -509,6 +430,53 @@ public class HSQLDBBlockRepository implements BlockRepository {
}
}
@Override
public int getBlockPruneHeight() throws DataException {
String sql = "SELECT block_prune_height FROM DatabaseInfo";
try (ResultSet resultSet = this.repository.checkedExecute(sql)) {
if (resultSet == null)
return 0;
return resultSet.getInt(1);
} catch (SQLException e) {
throw new DataException("Unable to fetch block prune height from repository", e);
}
}
@Override
public void setBlockPruneHeight(int pruneHeight) throws DataException {
// trimHeightsLock is to prevent concurrent update on DatabaseInfo
// that could result in "transaction rollback: serialization failure"
synchronized (this.repository.trimHeightsLock) {
String updateSql = "UPDATE DatabaseInfo SET block_prune_height = ?";
try {
this.repository.executeCheckedUpdate(updateSql, pruneHeight);
this.repository.saveChanges();
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to set block prune height in repository", e);
}
}
}
@Override
public int pruneBlocks(int minHeight, int maxHeight) throws DataException {
// Don't prune the genesis block
if (minHeight <= 1) {
minHeight = 2;
}
try {
return this.repository.delete("Blocks", "height BETWEEN ? AND ?", minHeight, maxHeight);
} catch (SQLException e) {
throw new DataException("Unable to prune blocks from repository", e);
}
}
@Override
public BlockData getDetachedBlockSignature(int startHeight) throws DataException {
String sql = "SELECT " + BLOCK_DB_COLUMNS + " FROM Blocks "

View File

@@ -0,0 +1,88 @@
package org.qortal.repository.hsqldb;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.gui.SplashFrame;
import org.qortal.repository.BlockArchiveWriter;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transform.TransformationException;
import java.io.IOException;
/**
*
* When switching to an archiving node, we need to archive most of the database contents.
* This involves copying its data into flat files.
* If we do this entirely as a background process, it is very slow and can interfere with syncing.
* However, if we take the approach of doing this in bulk, before starting up the rest of the
* processes, this makes it much faster and less invasive.
*
* From that point, the original background archiving process will run, but can be dialled right down
* so not to interfere with syncing.
*
*/
public class HSQLDBDatabaseArchiving {
private static final Logger LOGGER = LogManager.getLogger(HSQLDBDatabaseArchiving.class);
public static boolean buildBlockArchive(Repository repository, long fileSizeTarget) throws DataException {
// Only build the archive if we haven't already got one that is up to date
boolean upToDate = BlockArchiveWriter.isArchiverUpToDate(repository);
if (upToDate) {
// Already archived
return false;
}
LOGGER.info("Building block archive - this process could take a while... (approx. 15 mins on high spec)");
SplashFrame.getInstance().updateStatus("Building block archive (takes 60+ mins)...");
final int maximumArchiveHeight = BlockArchiveWriter.getMaxArchiveHeight(repository);
int startHeight = 0;
while (!Controller.isStopping()) {
try {
BlockArchiveWriter writer = new BlockArchiveWriter(startHeight, maximumArchiveHeight, repository);
writer.setFileSizeTarget(fileSizeTarget);
BlockArchiveWriter.BlockArchiveWriteResult result = writer.write();
switch (result) {
case OK:
// Increment block archive height
startHeight = writer.getLastWrittenHeight() + 1;
repository.getBlockArchiveRepository().setBlockArchiveHeight(startHeight);
repository.saveChanges();
break;
case STOPPING:
return false;
case NOT_ENOUGH_BLOCKS:
// We've reached the limit of the blocks we can archive
// Return from the whole method
return true;
case BLOCK_NOT_FOUND:
// We tried to archive a block that didn't exist. This is a major failure and likely means
// that a bootstrap or re-sync is needed. Return rom the method
LOGGER.info("Error: block not found when building archive. If this error persists, " +
"a bootstrap or re-sync may be needed.");
return false;
}
} catch (IOException | TransformationException | InterruptedException e) {
LOGGER.info("Caught exception when creating block cache", e);
return false;
}
}
// If we got this far then something went wrong (most likely the app is stopping)
return false;
}
}

View File

@@ -0,0 +1,332 @@
package org.qortal.repository.hsqldb;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.gui.SplashFrame;
import org.qortal.repository.BlockArchiveWriter;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.settings.Settings;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.concurrent.TimeoutException;
/**
*
* When switching from a full node to a pruning node, we need to delete most of the database contents.
* If we do this entirely as a background process, it is very slow and can interfere with syncing.
* However, if we take the approach of transferring only the necessary rows to a new table and then
* deleting the original table, this makes the process much faster. It was taking several days to
* delete the AT states in the background, but only a couple of minutes to copy them to a new table.
*
* The trade off is that we have to go through a form of "reshape" when starting the app for the first
* time after enabling pruning mode. But given that this is an opt-in mode, I don't think it will be
* a problem.
*
* Once the pruning is complete, it automatically performs a CHECKPOINT DEFRAG in order to
* shrink the database file size down to a fraction of what it was before.
*
* From this point, the original background process will run, but can be dialled right down so not
* to interfere with syncing.
*
*/
public class HSQLDBDatabasePruning {
private static final Logger LOGGER = LogManager.getLogger(HSQLDBDatabasePruning.class);
public static boolean pruneATStates(HSQLDBRepository repository) throws SQLException, DataException {
// Only bulk prune AT states if we have never done so before
int pruneHeight = repository.getATRepository().getAtPruneHeight();
if (pruneHeight > 0) {
// Already pruned AT states
return false;
}
if (Settings.getInstance().isArchiveEnabled()) {
// Only proceed if we can see that the archiver has already finished
// This way, if the archiver failed for any reason, we can prune once it has had
// some opportunities to try again
boolean upToDate = BlockArchiveWriter.isArchiverUpToDate(repository);
if (!upToDate) {
return false;
}
}
LOGGER.info("Starting bulk prune of AT states - this process could take a while... " +
"(approx. 2 mins on high spec, or upwards of 30 mins in some cases)");
SplashFrame.getInstance().updateStatus("Pruning database (takes up to 30 mins)...");
// Create new AT-states table to hold smaller dataset
repository.executeCheckedUpdate("DROP TABLE IF EXISTS ATStatesNew");
repository.executeCheckedUpdate("CREATE TABLE ATStatesNew ("
+ "AT_address QortalAddress, height INTEGER NOT NULL, state_hash ATStateHash NOT NULL, "
+ "fees QortalAmount NOT NULL, is_initial BOOLEAN NOT NULL, sleep_until_message_timestamp BIGINT, "
+ "PRIMARY KEY (AT_address, height), "
+ "FOREIGN KEY (AT_address) REFERENCES ATs (AT_address) ON DELETE CASCADE)");
repository.executeCheckedUpdate("SET TABLE ATStatesNew NEW SPACE");
repository.executeCheckedUpdate("CHECKPOINT");
// Add a height index
LOGGER.info("Adding index to AT states table...");
repository.executeCheckedUpdate("CREATE INDEX IF NOT EXISTS ATStatesNewHeightIndex ON ATStatesNew (height)");
repository.executeCheckedUpdate("CHECKPOINT");
// Find our latest block
BlockData latestBlock = repository.getBlockRepository().getLastBlock();
if (latestBlock == null) {
LOGGER.info("Unable to determine blockchain height, necessary for bulk block pruning");
return false;
}
// Calculate some constants for later use
final int blockchainHeight = latestBlock.getHeight();
int maximumBlockToTrim = blockchainHeight - Settings.getInstance().getPruneBlockLimit();
if (Settings.getInstance().isArchiveEnabled()) {
// Archive mode - don't prune anything that hasn't been archived yet
maximumBlockToTrim = Math.min(maximumBlockToTrim, repository.getBlockArchiveRepository().getBlockArchiveHeight() - 1);
}
final int endHeight = blockchainHeight;
final int blockStep = 10000;
// It's essential that we rebuild the latest AT states here, as we are using this data in the next query.
// Failing to do this will result in important AT states being deleted, rendering the database unusable.
repository.getATRepository().rebuildLatestAtStates();
// Loop through all the LatestATStates and copy them to the new table
LOGGER.info("Copying AT states...");
for (int height = 0; height < endHeight; height += blockStep) {
final int batchEndHeight = height + blockStep - 1;
//LOGGER.info(String.format("Copying AT states between %d and %d...", height, batchEndHeight));
String sql = "SELECT height, AT_address FROM LatestATStates WHERE height BETWEEN ? AND ?";
try (ResultSet latestAtStatesResultSet = repository.checkedExecute(sql, height, batchEndHeight)) {
if (latestAtStatesResultSet != null) {
do {
int latestAtHeight = latestAtStatesResultSet.getInt(1);
String latestAtAddress = latestAtStatesResultSet.getString(2);
// Copy this latest ATState to the new table
//LOGGER.info(String.format("Copying AT %s at height %d...", latestAtAddress, latestAtHeight));
try {
String updateSql = "INSERT INTO ATStatesNew ("
+ "SELECT AT_address, height, state_hash, fees, is_initial, sleep_until_message_timestamp "
+ "FROM ATStates "
+ "WHERE height = ? AND AT_address = ?)";
repository.executeCheckedUpdate(updateSql, latestAtHeight, latestAtAddress);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to copy ATStates", e);
}
// If this batch includes blocks after the maximum block to trim, we will need to copy
// each of its AT states above maximumBlockToTrim as they are considered "recent". We
// need to do this for _all_ AT states in these blocks, regardless of their latest state.
if (batchEndHeight >= maximumBlockToTrim) {
// Now copy this AT's states for each recent block they are present in
for (int i = maximumBlockToTrim; i < endHeight; i++) {
if (latestAtHeight < i) {
// This AT finished before this block so there is nothing to copy
continue;
}
//LOGGER.info(String.format("Copying recent AT %s at height %d...", latestAtAddress, i));
try {
// Copy each LatestATState to the new table
String updateSql = "INSERT IGNORE INTO ATStatesNew ("
+ "SELECT AT_address, height, state_hash, fees, is_initial, sleep_until_message_timestamp "
+ "FROM ATStates "
+ "WHERE height = ? AND AT_address = ?)";
repository.executeCheckedUpdate(updateSql, i, latestAtAddress);
} catch (SQLException e) {
repository.examineException(e);
throw new DataException("Unable to copy ATStates", e);
}
}
}
repository.saveChanges();
} while (latestAtStatesResultSet.next());
}
} catch (SQLException e) {
throw new DataException("Unable to copy AT states", e);
}
}
// Finally, drop the original table and rename
LOGGER.info("Deleting old AT states...");
repository.executeCheckedUpdate("DROP TABLE ATStates");
repository.executeCheckedUpdate("ALTER TABLE ATStatesNew RENAME TO ATStates");
repository.executeCheckedUpdate("ALTER INDEX ATStatesNewHeightIndex RENAME TO ATStatesHeightIndex");
repository.executeCheckedUpdate("CHECKPOINT");
// Update the prune height
int nextPruneHeight = maximumBlockToTrim + 1;
repository.getATRepository().setAtPruneHeight(nextPruneHeight);
repository.saveChanges();
repository.executeCheckedUpdate("CHECKPOINT");
// Now prune/trim the ATStatesData, as this currently goes back over a month
return HSQLDBDatabasePruning.pruneATStateData(repository);
}
/*
* Bulk prune ATStatesData to catch up with the now pruned ATStates table
* This uses the existing AT States trimming code but with a much higher end block
*/
private static boolean pruneATStateData(Repository repository) throws DataException {
if (Settings.getInstance().isArchiveEnabled()) {
// Don't prune ATStatesData in archive mode
return true;
}
BlockData latestBlock = repository.getBlockRepository().getLastBlock();
if (latestBlock == null) {
LOGGER.info("Unable to determine blockchain height, necessary for bulk ATStatesData pruning");
return false;
}
final int blockchainHeight = latestBlock.getHeight();
int upperPrunableHeight = blockchainHeight - Settings.getInstance().getPruneBlockLimit();
// ATStateData is already trimmed - so carry on from where we left off in the past
int pruneStartHeight = repository.getATRepository().getAtTrimHeight();
LOGGER.info("Starting bulk prune of AT states data - this process could take a while... (approx. 3 mins on high spec)");
while (pruneStartHeight < upperPrunableHeight) {
// Prune all AT state data up until our latest minus pruneBlockLimit (or our archive height)
if (Controller.isStopping()) {
return false;
}
// Override batch size in the settings because this is a one-off process
final int batchSize = 1000;
final int rowLimitPerBatch = 50000;
int upperBatchHeight = pruneStartHeight + batchSize;
int upperPruneHeight = Math.min(upperBatchHeight, upperPrunableHeight);
LOGGER.trace(String.format("Pruning AT states data between %d and %d...", pruneStartHeight, upperPruneHeight));
int numATStatesPruned = repository.getATRepository().trimAtStates(pruneStartHeight, upperPruneHeight, rowLimitPerBatch);
repository.saveChanges();
if (numATStatesPruned > 0) {
LOGGER.trace(String.format("Pruned %d AT states data rows between blocks %d and %d",
numATStatesPruned, pruneStartHeight, upperPruneHeight));
} else {
repository.getATRepository().setAtTrimHeight(upperBatchHeight);
// No need to rebuild the latest AT states as we aren't currently synchronizing
repository.saveChanges();
LOGGER.debug(String.format("Bumping AT states trim height to %d", upperBatchHeight));
// Can we move onto next batch?
if (upperPrunableHeight > upperBatchHeight) {
pruneStartHeight = upperBatchHeight;
}
else {
// We've finished pruning
break;
}
}
}
return true;
}
public static boolean pruneBlocks(Repository repository) throws SQLException, DataException {
// Only bulk prune AT states if we have never done so before
int pruneHeight = repository.getBlockRepository().getBlockPruneHeight();
if (pruneHeight > 0) {
// Already pruned blocks
return false;
}
if (Settings.getInstance().isArchiveEnabled()) {
// Only proceed if we can see that the archiver has already finished
// This way, if the archiver failed for any reason, we can prune once it has had
// some opportunities to try again
boolean upToDate = BlockArchiveWriter.isArchiverUpToDate(repository);
if (!upToDate) {
return false;
}
}
BlockData latestBlock = repository.getBlockRepository().getLastBlock();
if (latestBlock == null) {
LOGGER.info("Unable to determine blockchain height, necessary for bulk block pruning");
return false;
}
final int blockchainHeight = latestBlock.getHeight();
int upperPrunableHeight = blockchainHeight - Settings.getInstance().getPruneBlockLimit();
int pruneStartHeight = 0;
if (Settings.getInstance().isArchiveEnabled()) {
// Archive mode - don't prune anything that hasn't been archived yet
upperPrunableHeight = Math.min(upperPrunableHeight, repository.getBlockArchiveRepository().getBlockArchiveHeight() - 1);
}
LOGGER.info("Starting bulk prune of blocks - this process could take a while... (approx. 5 mins on high spec)");
while (pruneStartHeight < upperPrunableHeight) {
// Prune all blocks up until our latest minus pruneBlockLimit
int upperBatchHeight = pruneStartHeight + Settings.getInstance().getBlockPruneBatchSize();
int upperPruneHeight = Math.min(upperBatchHeight, upperPrunableHeight);
LOGGER.info(String.format("Pruning blocks between %d and %d...", pruneStartHeight, upperPruneHeight));
int numBlocksPruned = repository.getBlockRepository().pruneBlocks(pruneStartHeight, upperPruneHeight);
repository.saveChanges();
if (numBlocksPruned > 0) {
LOGGER.info(String.format("Pruned %d block%s between %d and %d",
numBlocksPruned, (numBlocksPruned != 1 ? "s" : ""),
pruneStartHeight, upperPruneHeight));
} else {
final int nextPruneHeight = upperPruneHeight + 1;
repository.getBlockRepository().setBlockPruneHeight(nextPruneHeight);
repository.saveChanges();
LOGGER.debug(String.format("Bumping block base prune height to %d", nextPruneHeight));
// Can we move onto next batch?
if (upperPrunableHeight > nextPruneHeight) {
pruneStartHeight = nextPruneHeight;
}
else {
// We've finished pruning
break;
}
}
}
return true;
}
public static void performMaintenance(Repository repository) throws SQLException, DataException {
try {
SplashFrame.getInstance().updateStatus("Performing maintenance...");
// Timeout if the database isn't ready for backing up after 5 minutes
// Nothing else should be using the db at this point, so a timeout shouldn't happen
long timeout = 5 * 60 * 1000L;
repository.performPeriodicMaintenance(timeout);
} catch (TimeoutException e) {
LOGGER.info("Attempt to perform maintenance failed due to timeout: {}", e.getMessage());
}
}
}

View File

@@ -9,7 +9,9 @@ import java.util.stream.Collectors;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.controller.tradebot.BitcoinACCTv1TradeBot;
import org.qortal.gui.SplashFrame;
public class HSQLDBDatabaseUpdates {
@@ -27,9 +29,14 @@ public class HSQLDBDatabaseUpdates {
public static boolean updateDatabase(Connection connection) throws SQLException {
final boolean wasPristine = fetchDatabaseVersion(connection) == 0;
SplashFrame.getInstance().updateStatus("Upgrading database, please wait...");
while (databaseUpdating(connection, wasPristine))
incrementDatabaseVersion(connection);
String text = String.format("Starting Qortal Core v%s...", Controller.getInstance().getVersionStringWithoutPrefix());
SplashFrame.getInstance().updateStatus(text);
return wasPristine;
}
@@ -837,6 +844,11 @@ public class HSQLDBDatabaseUpdates {
stmt.execute("SET TABLE ATStatesNew NEW SPACE");
stmt.execute("CHECKPOINT");
// Add the height index
LOGGER.info("Adding index to AT states table...");
stmt.execute("CREATE INDEX ATStatesNewHeightIndex ON ATStatesNew (height)");
stmt.execute("CHECKPOINT");
ResultSet resultSet = stmt.executeQuery("SELECT height FROM Blocks ORDER BY height DESC LIMIT 1");
final int blockchainHeight = resultSet.next() ? resultSet.getInt(1) : 0;
final int heightStep = 100;
@@ -849,14 +861,43 @@ public class HSQLDBDatabaseUpdates {
+ "WHERE height BETWEEN " + minHeight + " AND " + (minHeight + heightStep - 1)
+ ")");
stmt.execute("COMMIT");
int processed = Math.min(minHeight + heightStep - 1, blockchainHeight);
double percentage = (double)processed / (double)blockchainHeight * 100.0f;
LOGGER.info(String.format("Processed %d of %d blocks (%.1f%%)", processed, blockchainHeight, percentage));
}
stmt.execute("CHECKPOINT");
stmt.execute("DROP TABLE ATStates");
stmt.execute("ALTER TABLE ATStatesNew RENAME TO ATStates");
stmt.execute("ALTER INDEX ATStatesNewHeightIndex RENAME TO ATStatesHeightIndex");
stmt.execute("CHECKPOINT");
break;
}
case 35:
// Support for pruning
stmt.execute("ALTER TABLE DatabaseInfo ADD AT_prune_height INT NOT NULL DEFAULT 0");
stmt.execute("ALTER TABLE DatabaseInfo ADD block_prune_height INT NOT NULL DEFAULT 0");
break;
case 36:
// Block archive support
stmt.execute("ALTER TABLE DatabaseInfo ADD block_archive_height INT NOT NULL DEFAULT 0");
// Block archive (lookup table to map signature to height)
// Actual data is stored in archive files outside of the database
stmt.execute("CREATE TABLE BlockArchive (signature BlockSignature, height INTEGER NOT NULL, "
+ "minted_when EpochMillis NOT NULL, minter QortalPublicKey NOT NULL, "
+ "PRIMARY KEY (signature))");
// For finding blocks by height.
stmt.execute("CREATE INDEX BlockArchiveHeightIndex ON BlockArchive (height)");
// For finding blocks by the account that minted them.
stmt.execute("CREATE INDEX BlockArchiveMinterIndex ON BlockArchive (minter)");
// For finding blocks by timestamp or finding height of latest block immediately before timestamp, etc.
stmt.execute("CREATE INDEX BlockArchiveTimestampHeightIndex ON BlockArchive (minted_when, height)");
// Use a separate table space as this table will be very large.
stmt.execute("SET TABLE BlockArchive NEW SPACE");
break;
default:
// nothing to do

View File

@@ -0,0 +1,315 @@
package org.qortal.repository.hsqldb;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import org.qortal.data.account.MintingAccountData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.repository.Bootstrap;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.settings.Settings;
import org.qortal.utils.Base58;
import org.qortal.utils.Triple;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Iterator;
import java.util.List;
public class HSQLDBImportExport {
private static final Logger LOGGER = LogManager.getLogger(Bootstrap.class);
public static void backupTradeBotStates(Repository repository, List<TradeBotData> additional) throws DataException {
HSQLDBImportExport.backupCurrentTradeBotStates(repository, additional);
HSQLDBImportExport.backupArchivedTradeBotStates(repository, additional);
LOGGER.info("Exported sensitive/node-local data: trade bot states");
}
public static void backupMintingAccounts(Repository repository) throws DataException {
HSQLDBImportExport.backupCurrentMintingAccounts(repository);
LOGGER.info("Exported sensitive/node-local data: minting accounts");
}
/* Trade bot states */
/**
* Backs up the trade bot states currently in the repository, without combining them with past ones
* @param repository
* @param additional - any optional extra trade bot states to include in the backup
* @throws DataException
*/
private static void backupCurrentTradeBotStates(Repository repository, List<TradeBotData> additional) throws DataException {
try {
Path backupDirectory = HSQLDBImportExport.getExportDirectory(true);
// Load current trade bot data
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
// Add any additional entries if specified
if (additional != null && !additional.isEmpty()) {
allTradeBotData.addAll(additional);
}
// Convert them to JSON objects
JSONArray currentTradeBotDataJson = new JSONArray();
for (TradeBotData tradeBotData : allTradeBotData) {
JSONObject tradeBotDataJson = tradeBotData.toJson();
currentTradeBotDataJson.put(tradeBotDataJson);
}
// Wrap current trade bot data in an object to indicate the type
JSONObject currentTradeBotDataJsonWrapper = new JSONObject();
currentTradeBotDataJsonWrapper.put("type", "tradeBotStates");
currentTradeBotDataJsonWrapper.put("dataset", "current");
currentTradeBotDataJsonWrapper.put("data", currentTradeBotDataJson);
// Write current trade bot data (just the ones currently in the database)
String fileName = Paths.get(backupDirectory.toString(), "TradeBotStates.json").toString();
FileWriter writer = new FileWriter(fileName);
writer.write(currentTradeBotDataJsonWrapper.toString(2));
writer.close();
} catch (DataException | IOException e) {
throw new DataException("Unable to export trade bot states from repository");
}
}
/**
* Backs up the trade bot states currently in the repository to a separate "archive" file,
* making sure to combine them with any unique states already present in the archive.
* @param repository
* @param additional - any optional extra trade bot states to include in the backup
* @throws DataException
*/
private static void backupArchivedTradeBotStates(Repository repository, List<TradeBotData> additional) throws DataException {
try {
Path backupDirectory = HSQLDBImportExport.getExportDirectory(true);
// Load current trade bot data
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
// Add any additional entries if specified
if (additional != null && !additional.isEmpty()) {
allTradeBotData.addAll(additional);
}
// Convert them to JSON objects
JSONArray allTradeBotDataJson = new JSONArray();
for (TradeBotData tradeBotData : allTradeBotData) {
JSONObject tradeBotDataJson = tradeBotData.toJson();
allTradeBotDataJson.put(tradeBotDataJson);
}
// We need to combine existing archived TradeBotStates data before overwriting
String fileName = Paths.get(backupDirectory.toString(), "TradeBotStatesArchive.json").toString();
File tradeBotStatesBackupFile = new File(fileName);
if (tradeBotStatesBackupFile.exists()) {
String jsonString = new String(Files.readAllBytes(Paths.get(fileName)));
Triple<String, String, JSONArray> parsedJSON = HSQLDBImportExport.parseJSONString(jsonString);
if (parsedJSON.getA() == null || parsedJSON.getC() == null) {
throw new DataException("Missing data when exporting archived trade bot states");
}
String type = parsedJSON.getA();
String dataset = parsedJSON.getB();
JSONArray data = parsedJSON.getC();
if (!type.equals("tradeBotStates") || !dataset.equals("archive")) {
throw new DataException("Format mismatch when exporting archived trade bot states");
}
Iterator<Object> iterator = data.iterator();
while(iterator.hasNext()) {
JSONObject existingTradeBotDataItem = (JSONObject)iterator.next();
String existingTradePrivateKey = (String) existingTradeBotDataItem.get("tradePrivateKey");
// Check if we already have an entry for this trade
boolean found = allTradeBotData.stream().anyMatch(tradeBotData -> Base58.encode(tradeBotData.getTradePrivateKey()).equals(existingTradePrivateKey));
if (found == false)
// Add the data from the backup file to our "allTradeBotDataJson" array as it's not currently in the db
allTradeBotDataJson.put(existingTradeBotDataItem);
}
}
// Wrap all trade bot data in an object to indicate the type
JSONObject allTradeBotDataJsonWrapper = new JSONObject();
allTradeBotDataJsonWrapper.put("type", "tradeBotStates");
allTradeBotDataJsonWrapper.put("dataset", "archive");
allTradeBotDataJsonWrapper.put("data", allTradeBotDataJson);
// Write ALL trade bot data to archive (current plus states that are no longer in the database)
FileWriter writer = new FileWriter(fileName);
writer.write(allTradeBotDataJsonWrapper.toString(2));
writer.close();
} catch (DataException | IOException e) {
throw new DataException("Unable to export trade bot states from repository");
}
}
/* Minting accounts */
/**
* Backs up the minting accounts currently in the repository, without combining them with past ones
* @param repository
* @throws DataException
*/
private static void backupCurrentMintingAccounts(Repository repository) throws DataException {
try {
Path backupDirectory = HSQLDBImportExport.getExportDirectory(true);
// Load current trade bot data
List<MintingAccountData> allMintingAccountData = repository.getAccountRepository().getMintingAccounts();
JSONArray currentMintingAccountJson = new JSONArray();
for (MintingAccountData mintingAccountData : allMintingAccountData) {
JSONObject mintingAccountDataJson = mintingAccountData.toJson();
currentMintingAccountJson.put(mintingAccountDataJson);
}
// Wrap current trade bot data in an object to indicate the type
JSONObject currentMintingAccountDataJsonWrapper = new JSONObject();
currentMintingAccountDataJsonWrapper.put("type", "mintingAccounts");
currentMintingAccountDataJsonWrapper.put("dataset", "current");
currentMintingAccountDataJsonWrapper.put("data", currentMintingAccountJson);
// Write current trade bot data (just the ones currently in the database)
String fileName = Paths.get(backupDirectory.toString(), "MintingAccounts.json").toString();
FileWriter writer = new FileWriter(fileName);
writer.write(currentMintingAccountDataJsonWrapper.toString(2));
writer.close();
} catch (DataException | IOException e) {
throw new DataException("Unable to export minting accounts from repository");
}
}
/* Utils */
/**
* Imports data from supplied file
* Data type is loaded from the file itself, and if missing, TradeBotStates is assumed
*
* @param filename
* @param repository
* @throws DataException
* @throws IOException
*/
public static void importDataFromFile(String filename, Repository repository) throws DataException, IOException {
Path path = Paths.get(filename);
if (!path.toFile().exists()) {
throw new FileNotFoundException(String.format("File doesn't exist: %s", filename));
}
byte[] fileContents = Files.readAllBytes(path);
if (fileContents == null) {
throw new FileNotFoundException(String.format("Unable to read file contents: %s", filename));
}
LOGGER.info(String.format("Importing %s into repository ...", filename));
String jsonString = new String(fileContents);
Triple<String, String, JSONArray> parsedJSON = HSQLDBImportExport.parseJSONString(jsonString);
if (parsedJSON.getA() == null || parsedJSON.getC() == null) {
throw new DataException(String.format("Missing data when importing %s into repository", filename));
}
String type = parsedJSON.getA();
JSONArray data = parsedJSON.getC();
Iterator<Object> iterator = data.iterator();
while(iterator.hasNext()) {
JSONObject dataJsonObject = (JSONObject)iterator.next();
if (type.equals("tradeBotStates")) {
HSQLDBImportExport.importTradeBotDataJSON(dataJsonObject, repository);
}
else if (type.equals("mintingAccounts")) {
HSQLDBImportExport.importMintingAccountDataJSON(dataJsonObject, repository);
}
else {
throw new DataException(String.format("Unrecognized data type when importing %s into repository", filename));
}
}
LOGGER.info(String.format("Imported %s into repository from %s", type, filename));
}
private static void importTradeBotDataJSON(JSONObject tradeBotDataJson, Repository repository) throws DataException {
TradeBotData tradeBotData = TradeBotData.fromJson(tradeBotDataJson);
repository.getCrossChainRepository().save(tradeBotData);
}
private static void importMintingAccountDataJSON(JSONObject mintingAccountDataJson, Repository repository) throws DataException {
MintingAccountData mintingAccountData = MintingAccountData.fromJson(mintingAccountDataJson);
repository.getAccountRepository().save(mintingAccountData);
}
public static Path getExportDirectory(boolean createIfNotExists) throws DataException {
Path backupPath = Paths.get(Settings.getInstance().getExportPath());
if (createIfNotExists) {
// Create the qortal-backup folder if it doesn't exist
try {
Files.createDirectories(backupPath);
} catch (IOException e) {
LOGGER.info(String.format("Unable to create %s folder", backupPath.toString()));
throw new DataException(String.format("Unable to create %s folder", backupPath.toString()));
}
}
return backupPath;
}
/**
* Parses a JSON string and returns "data", "type", and "dataset" fields.
* In the case of legacy JSON files with no type, they are assumed to be TradeBotStates archives,
* as we had never implemented this for any other types.
*
* @param jsonString
* @return Triple<String, String, JSONArray> (type, dataset, data)
*/
public static Triple<String, String, JSONArray> parseJSONString(String jsonString) throws DataException {
String type = null;
String dataset = null;
JSONArray data = null;
try {
// Firstly try importing the new format
JSONObject jsonData = new JSONObject(jsonString);
if (jsonData != null && jsonData.getString("type") != null) {
type = jsonData.getString("type");
dataset = jsonData.getString("dataset");
data = jsonData.getJSONArray("data");
}
} catch (JSONException e) {
// Could be a legacy format which didn't contain a type or any other outer keys, so try importing that
// Treat these as TradeBotStates archives, given that this was the only type previously implemented
try {
type = "tradeBotStates";
dataset = "archive";
data = new JSONArray(jsonString);
} catch (JSONException e2) {
// Still failed, so give up
throw new DataException("Couldn't import JSON file");
}
}
return new Triple(type, dataset, data);
}
}

View File

@@ -2,7 +2,6 @@ package org.qortal.repository.hsqldb;
import java.awt.TrayIcon.MessageType;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.math.BigDecimal;
import java.nio.file.Files;
@@ -17,39 +16,20 @@ import java.sql.SQLException;
import java.sql.Savepoint;
import java.sql.Statement;
import java.util.*;
import java.util.concurrent.TimeoutException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Stream;
import org.json.JSONArray;
import org.json.JSONObject;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.crypto.Crypto;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.globalization.Translator;
import org.qortal.gui.SysTray;
import org.qortal.repository.ATRepository;
import org.qortal.repository.AccountRepository;
import org.qortal.repository.ArbitraryRepository;
import org.qortal.repository.AssetRepository;
import org.qortal.repository.BlockRepository;
import org.qortal.repository.ChatRepository;
import org.qortal.repository.CrossChainRepository;
import org.qortal.repository.DataException;
import org.qortal.repository.GroupRepository;
import org.qortal.repository.MessageRepository;
import org.qortal.repository.NameRepository;
import org.qortal.repository.NetworkRepository;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.repository.TransactionRepository;
import org.qortal.repository.VotingRepository;
import org.qortal.repository.*;
import org.qortal.repository.hsqldb.transaction.HSQLDBTransactionRepository;
import org.qortal.settings.Settings;
import org.qortal.utils.Base58;
public class HSQLDBRepository implements Repository {
@@ -69,12 +49,14 @@ public class HSQLDBRepository implements Repository {
protected final Map<String, PreparedStatement> preparedStatementCache = new HashMap<>();
// We want the same object corresponding to the actual DB
protected final Object trimHeightsLock = RepositoryManager.getRepositoryFactory();
protected final Object latestATStatesLock = RepositoryManager.getRepositoryFactory();
private final ATRepository atRepository = new HSQLDBATRepository(this);
private final AccountRepository accountRepository = new HSQLDBAccountRepository(this);
private final ArbitraryRepository arbitraryRepository = new HSQLDBArbitraryRepository(this);
private final AssetRepository assetRepository = new HSQLDBAssetRepository(this);
private final BlockRepository blockRepository = new HSQLDBBlockRepository(this);
private final BlockArchiveRepository blockArchiveRepository = new HSQLDBBlockArchiveRepository(this);
private final ChatRepository chatRepository = new HSQLDBChatRepository(this);
private final CrossChainRepository crossChainRepository = new HSQLDBCrossChainRepository(this);
private final GroupRepository groupRepository = new HSQLDBGroupRepository(this);
@@ -142,6 +124,11 @@ public class HSQLDBRepository implements Repository {
return this.blockRepository;
}
@Override
public BlockArchiveRepository getBlockArchiveRepository() {
return this.blockArchiveRepository;
}
@Override
public ChatRepository getChatRepository() {
return this.chatRepository;
@@ -281,7 +268,7 @@ public class HSQLDBRepository implements Repository {
public void close() throws DataException {
// Already closed? No need to do anything but maybe report double-call
if (this.connection == null) {
LOGGER.warn("HSQLDBRepository.close() called when repository already closed", new Exception("Repository already closed"));
LOGGER.warn("HSQLDBRepository.close() called when repository already closed. This is expected when bootstrapping.");
return;
}
@@ -393,133 +380,104 @@ public class HSQLDBRepository implements Repository {
}
@Override
public void backup(boolean quick) throws DataException {
if (!quick)
// First perform a CHECKPOINT
public void backup(boolean quick, String name, Long timeout) throws DataException, TimeoutException {
synchronized (CHECKPOINT_LOCK) {
// We can only perform a CHECKPOINT if no other HSQLDB session is mid-transaction,
// otherwise the CHECKPOINT blocks for COMMITs and other threads can't open HSQLDB sessions
// due to HSQLDB blocking until CHECKPOINT finishes - i.e. deadlock.
// Since we don't want to give up too easily, it's best to wait until the other transaction
// count reaches zero, and then continue.
this.blockUntilNoOtherTransactions(timeout);
if (!quick)
// First perform a CHECKPOINT
try (Statement stmt = this.connection.createStatement()) {
LOGGER.info("Performing maintenance - this will take a while...");
stmt.execute("CHECKPOINT");
stmt.execute("CHECKPOINT DEFRAG");
LOGGER.info("Maintenance completed");
} catch (SQLException e) {
throw new DataException("Unable to prepare repository for backup");
}
// Clean out any previous backup
try {
String connectionUrl = this.connection.getMetaData().getURL();
String dbPathname = getDbPathname(connectionUrl);
if (dbPathname == null)
throw new DataException("Unable to locate repository for backup?");
// Doesn't really make sense to backup an in-memory database...
if (dbPathname.equals("mem")) {
LOGGER.debug("Ignoring request to backup in-memory repository!");
return;
}
String backupUrl = buildBackupUrl(dbPathname, name);
String backupPathname = getDbPathname(backupUrl);
if (backupPathname == null)
throw new DataException("Unable to determine location for repository backup?");
Path backupDirPath = Paths.get(backupPathname).getParent();
String backupDirPathname = backupDirPath.toString();
try (Stream<Path> paths = Files.walk(backupDirPath)) {
paths.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.filter(file -> file.getPath().startsWith(backupDirPathname))
.forEach(File::delete);
}
} catch (NoSuchFileException e) {
// Nothing to remove
} catch (SQLException | IOException e) {
throw new DataException("Unable to remove previous repository backup");
}
// Actually create backup
try (Statement stmt = this.connection.createStatement()) {
stmt.execute("CHECKPOINT DEFRAG");
LOGGER.info("Backing up repository...");
stmt.execute(String.format("BACKUP DATABASE TO '%s/' BLOCKING AS FILES", name));
LOGGER.info("Backup completed");
} catch (SQLException e) {
throw new DataException("Unable to prepare repository for backup");
throw new DataException("Unable to backup repository");
}
// Clean out any previous backup
try {
String connectionUrl = this.connection.getMetaData().getURL();
String dbPathname = getDbPathname(connectionUrl);
if (dbPathname == null)
throw new DataException("Unable to locate repository for backup?");
// Doesn't really make sense to backup an in-memory database...
if (dbPathname.equals("mem")) {
LOGGER.debug("Ignoring request to backup in-memory repository!");
return;
}
String backupUrl = buildBackupUrl(dbPathname);
String backupPathname = getDbPathname(backupUrl);
if (backupPathname == null)
throw new DataException("Unable to determine location for repository backup?");
Path backupDirPath = Paths.get(backupPathname).getParent();
String backupDirPathname = backupDirPath.toString();
try (Stream<Path> paths = Files.walk(backupDirPath)) {
paths.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.filter(file -> file.getPath().startsWith(backupDirPathname))
.forEach(File::delete);
}
} catch (NoSuchFileException e) {
// Nothing to remove
} catch (SQLException | IOException e) {
throw new DataException("Unable to remove previous repository backup");
}
// Actually create backup
try (Statement stmt = this.connection.createStatement()) {
stmt.execute("BACKUP DATABASE TO 'backup/' BLOCKING AS FILES");
} catch (SQLException e) {
throw new DataException("Unable to backup repository");
}
}
@Override
public void performPeriodicMaintenance() throws DataException {
// Defrag DB - takes a while!
try (Statement stmt = this.connection.createStatement()) {
LOGGER.info("performing maintenance - this will take a while");
stmt.execute("CHECKPOINT");
stmt.execute("CHECKPOINT DEFRAG");
LOGGER.info("maintenance completed");
} catch (SQLException e) {
throw new DataException("Unable to defrag repository");
public void performPeriodicMaintenance(Long timeout) throws DataException, TimeoutException {
synchronized (CHECKPOINT_LOCK) {
// We can only perform a CHECKPOINT if no other HSQLDB session is mid-transaction,
// otherwise the CHECKPOINT blocks for COMMITs and other threads can't open HSQLDB sessions
// due to HSQLDB blocking until CHECKPOINT finishes - i.e. deadlock.
// Since we don't want to give up too easily, it's best to wait until the other transaction
// count reaches zero, and then continue.
this.blockUntilNoOtherTransactions(timeout);
// Defrag DB - takes a while!
try (Statement stmt = this.connection.createStatement()) {
LOGGER.info("performing maintenance - this will take a while");
stmt.execute("CHECKPOINT");
stmt.execute("CHECKPOINT DEFRAG");
LOGGER.info("maintenance completed");
} catch (SQLException e) {
throw new DataException("Unable to defrag repository");
}
}
}
@Override
public void exportNodeLocalData() throws DataException {
// Create the qortal-backup folder if it doesn't exist
Path backupPath = Paths.get("qortal-backup");
try {
Files.createDirectories(backupPath);
} catch (IOException e) {
LOGGER.info("Unable to create backup folder");
throw new DataException("Unable to create backup folder");
}
try {
// Load trade bot data
List<TradeBotData> allTradeBotData = this.getCrossChainRepository().getAllTradeBotData();
JSONArray allTradeBotDataJson = new JSONArray();
for (TradeBotData tradeBotData : allTradeBotData) {
JSONObject tradeBotDataJson = tradeBotData.toJson();
allTradeBotDataJson.put(tradeBotDataJson);
}
// We need to combine existing TradeBotStates data before overwriting
String fileName = "qortal-backup/TradeBotStates.json";
File tradeBotStatesBackupFile = new File(fileName);
if (tradeBotStatesBackupFile.exists()) {
String jsonString = new String(Files.readAllBytes(Paths.get(fileName)));
JSONArray allExistingTradeBotData = new JSONArray(jsonString);
Iterator<Object> iterator = allExistingTradeBotData.iterator();
while(iterator.hasNext()) {
JSONObject existingTradeBotData = (JSONObject)iterator.next();
String existingTradePrivateKey = (String) existingTradeBotData.get("tradePrivateKey");
// Check if we already have an entry for this trade
boolean found = allTradeBotData.stream().anyMatch(tradeBotData -> Base58.encode(tradeBotData.getTradePrivateKey()).equals(existingTradePrivateKey));
if (found == false)
// We need to add this to our list
allTradeBotDataJson.put(existingTradeBotData);
}
}
FileWriter writer = new FileWriter(fileName);
writer.write(allTradeBotDataJson.toString());
writer.close();
LOGGER.info("Exported sensitive/node-local data: trade bot states");
} catch (DataException | IOException e) {
throw new DataException("Unable to export trade bot states from repository");
}
HSQLDBImportExport.backupTradeBotStates(this, null);
HSQLDBImportExport.backupMintingAccounts(this);
}
@Override
public void importDataFromFile(String filename) throws DataException {
LOGGER.info(() -> String.format("Importing data into repository from %s", filename));
try {
String jsonString = new String(Files.readAllBytes(Paths.get(filename)));
JSONArray tradeBotDataToImport = new JSONArray(jsonString);
Iterator<Object> iterator = tradeBotDataToImport.iterator();
while(iterator.hasNext()) {
JSONObject tradeBotDataJson = (JSONObject)iterator.next();
TradeBotData tradeBotData = TradeBotData.fromJson(tradeBotDataJson);
this.getCrossChainRepository().save(tradeBotData);
}
} catch (IOException e) {
throw new DataException("Unable to import sensitive/node-local trade bot states to repository: " + e.getMessage());
}
LOGGER.info(() -> String.format("Imported trade bot states into repository from %s", filename));
public void importDataFromFile(String filename) throws DataException, IOException {
HSQLDBImportExport.importDataFromFile(filename, this);
}
@Override
@@ -541,22 +499,22 @@ public class HSQLDBRepository implements Repository {
return matcher.group(2);
}
private static String buildBackupUrl(String dbPathname) {
private static String buildBackupUrl(String dbPathname, String backupName) {
Path oldRepoPath = Paths.get(dbPathname);
Path oldRepoDirPath = oldRepoPath.getParent();
Path oldRepoFilePath = oldRepoPath.getFileName();
// Try to open backup. We need to remove "create=true" and insert "backup" dir before final filename.
String backupUrlTemplate = "jdbc:hsqldb:file:%s%sbackup%s%s;create=false;hsqldb.full_log_replay=true";
return String.format(backupUrlTemplate, oldRepoDirPath.toString(), File.separator, File.separator, oldRepoFilePath.toString());
String backupUrlTemplate = "jdbc:hsqldb:file:%s%s%s%s%s;create=false;hsqldb.full_log_replay=true";
return String.format(backupUrlTemplate, oldRepoDirPath.toString(), File.separator, backupName, File.separator, oldRepoFilePath.toString());
}
/* package */ static void attemptRecovery(String connectionUrl) throws DataException {
/* package */ static void attemptRecovery(String connectionUrl, String name) throws DataException {
String dbPathname = getDbPathname(connectionUrl);
if (dbPathname == null)
throw new DataException("Unable to locate repository for backup?");
String backupUrl = buildBackupUrl(dbPathname);
String backupUrl = buildBackupUrl(dbPathname, name);
Path oldRepoDirPath = Paths.get(dbPathname).getParent();
// Attempt connection to backup to see if it is viable
@@ -1059,4 +1017,51 @@ public class HSQLDBRepository implements Repository {
return DEADLOCK_ERROR_CODE.equals(e.getErrorCode());
}
private int otherTransactionsCount() throws DataException {
// We can only perform a CHECKPOINT if no other HSQLDB session is mid-transaction,
// otherwise the CHECKPOINT blocks for COMMITs and other threads can't open HSQLDB sessions
// due to HSQLDB blocking until CHECKPOINT finishes - i.e. deadlock
String sql = "SELECT COUNT(*) "
+ "FROM Information_schema.system_sessions "
+ "WHERE transaction = TRUE AND session_id != ?";
try {
PreparedStatement pstmt = this.cachePreparedStatement(sql);
pstmt.setLong(1, this.sessionId);
if (!pstmt.execute())
throw new DataException("Unable to check repository session status");
try (ResultSet resultSet = pstmt.getResultSet()) {
if (resultSet == null || !resultSet.next())
// Failed to even find HSQLDB session info!
throw new DataException("No results when checking repository session status");
int transactionCount = resultSet.getInt(1);
return transactionCount;
}
} catch (SQLException e) {
throw new DataException("Unable to check repository session status", e);
}
}
private void blockUntilNoOtherTransactions(Long timeout) throws DataException, TimeoutException {
try {
long startTime = System.currentTimeMillis();
while (this.otherTransactionsCount() > 0) {
// Wait and try again
LOGGER.debug("Waiting for repository...");
Thread.sleep(1000L);
if (timeout != null) {
if (System.currentTimeMillis() - startTime >= timeout) {
throw new TimeoutException("Timed out waiting for repository to become available");
}
}
}
} catch (InterruptedException e) {
throw new DataException("Interrupted before repository became available");
}
}
}

View File

@@ -54,7 +54,7 @@ public class HSQLDBRepositoryFactory implements RepositoryFactory {
throw new DataException("Unable to read repository: " + e.getMessage(), e);
// Attempt recovery?
HSQLDBRepository.attemptRecovery(connectionUrl);
HSQLDBRepository.attemptRecovery(connectionUrl, "backup");
}
this.connectionPool = new HSQLDBPool(Settings.getInstance().getRepositoryConnectionPoolSize());

View File

@@ -5,6 +5,7 @@ import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.Reader;
import java.util.List;
import java.util.Locale;
import javax.xml.bind.JAXBContext;
@@ -68,6 +69,9 @@ public class Settings {
};
private Boolean apiRestricted;
private String apiKey = null;
/** Whether to disable API key or loopback address checking
* IMPORTANT: do not disable for shared nodes or low-security local networks */
private boolean apiKeyDisabled = false;
private boolean apiLoggingEnabled = false;
private boolean apiDocumentationEnabled = false;
// Both of these need to be set for API to use SSL
@@ -86,6 +90,12 @@ public class Settings {
private long repositoryBackupInterval = 0; // ms
/** Whether to show a notification when we backup repository. */
private boolean showBackupNotification = false;
/** Minimum time between repository maintenance attempts (ms) */
private long repositoryMaintenanceMinInterval = 3 * 24 * 60 * 60 * 1000L; // 3 days (ms) default
/** Maximum time between repository maintenance attempts (ms) (0 if disabled). */
private long repositoryMaintenanceMaxInterval = 14 * 24 * 60 * 60 * 1000L; // 14 days (ms) default
/** Whether to show a notification when we run scheduled maintenance. */
private boolean showMaintenanceNotification = false;
/** How long between repository checkpoints (ms). */
private long repositoryCheckpointInterval = 60 * 60 * 1000L; // 1 hour (ms) default
/** Whether to show a notification when we perform repository 'checkpoint'. */
@@ -94,7 +104,7 @@ public class Settings {
private int blockCacheSize = 10;
/** How long to keep old, full, AT state data (ms). */
private long atStatesMaxLifetime = 2 * 7 * 24 * 60 * 60 * 1000L; // milliseconds
private long atStatesMaxLifetime = 5 * 24 * 60 * 60 * 1000L; // milliseconds
/** How often to attempt AT state trimming (ms). */
private long atStatesTrimInterval = 5678L; // milliseconds
/** Block height range to scan for trimmable AT states.<br>
@@ -109,6 +119,36 @@ public class Settings {
* This has a significant effect on execution time. */
private int onlineSignaturesTrimBatchSize = 100; // blocks
/** Whether we should prune old data to reduce database size
* This prevents the node from being able to serve older blocks */
private boolean topOnly = false;
/** The amount of recent blocks we should keep when pruning */
private int pruneBlockLimit = 1450;
/** How often to attempt AT state pruning (ms). */
private long atStatesPruneInterval = 3219L; // milliseconds
/** Block height range to scan for prunable AT states.<br>
* This has a significant effect on execution time. */
private int atStatesPruneBatchSize = 25; // blocks
/** How often to attempt block pruning (ms). */
private long blockPruneInterval = 3219L; // milliseconds
/** Block height range to scan for prunable blocks.<br>
* This has a significant effect on execution time. */
private int blockPruneBatchSize = 10000; // blocks
/** Whether we should archive old data to reduce the database size */
private boolean archiveEnabled = true;
/** How often to attempt archiving (ms). */
private long archiveInterval = 7171L; // milliseconds
/** Whether to automatically bootstrap instead of syncing from genesis */
private boolean bootstrap = true;
// Peer-to-peer related
private boolean isTestNet = false;
/** Port number for inbound peer-to-peer connections. */
@@ -154,6 +194,20 @@ public class Settings {
private String repositoryPath = "db";
/** Repository connection pool size. Needs to be a bit bigger than maxNetworkThreadPoolSize */
private int repositoryConnectionPoolSize = 100;
private List<String> fixedNetwork;
// Export/import
private String exportPath = "qortal-backup";
// Bootstrap
private String bootstrapFilenamePrefix = "";
// Bootstrap sources
private String[] bootstrapHosts = new String[] {
"http://bootstrap.qortal.org",
"http://bootstrap2.qortal.org",
"http://cinfu1.crowetic.com"
};
// Auto-update sources
private String[] autoUpdateRepos = new String[] {
@@ -356,6 +410,10 @@ public class Settings {
return this.apiKey;
}
public boolean isApiKeyDisabled() {
return this.apiKeyDisabled;
}
public boolean isApiLoggingEnabled() {
return this.apiLoggingEnabled;
}
@@ -469,6 +527,14 @@ public class Settings {
return this.repositoryConnectionPoolSize;
}
public String getExportPath() {
return this.exportPath;
}
public String getBootstrapFilenamePrefix() {
return this.bootstrapFilenamePrefix;
}
public boolean isAutoUpdateEnabled() {
return this.autoUpdateEnabled;
}
@@ -477,6 +543,10 @@ public class Settings {
return this.autoUpdateRepos;
}
public String[] getBootstrapHosts() {
return this.bootstrapHosts;
}
public String getListsPath() {
return this.listsPath;
}
@@ -497,6 +567,18 @@ public class Settings {
return this.showBackupNotification;
}
public long getRepositoryMaintenanceMinInterval() {
return this.repositoryMaintenanceMinInterval;
}
public long getRepositoryMaintenanceMaxInterval() {
return this.repositoryMaintenanceMaxInterval;
}
public boolean getShowMaintenanceNotification() {
return this.showMaintenanceNotification;
}
public long getRepositoryCheckpointInterval() {
return this.repositoryCheckpointInterval;
}
@@ -505,6 +587,10 @@ public class Settings {
return this.showCheckpointNotification;
}
public List<String> getFixedNetwork() {
return fixedNetwork;
}
public long getAtStatesMaxLifetime() {
return this.atStatesMaxLifetime;
}
@@ -529,4 +615,45 @@ public class Settings {
return this.onlineSignaturesTrimBatchSize;
}
public boolean isTopOnly() {
return this.topOnly;
}
public int getPruneBlockLimit() {
return this.pruneBlockLimit;
}
public long getAtStatesPruneInterval() {
return this.atStatesPruneInterval;
}
public int getAtStatesPruneBatchSize() {
return this.atStatesPruneBatchSize;
}
public long getBlockPruneInterval() {
return this.blockPruneInterval;
}
public int getBlockPruneBatchSize() {
return this.blockPruneBatchSize;
}
public boolean isArchiveEnabled() {
if (this.topOnly) {
return false;
}
return this.archiveEnabled;
}
public long getArchiveInterval() {
return this.archiveInterval;
}
public boolean getBootstrap() {
return this.bootstrap;
}
}

View File

@@ -48,6 +48,11 @@ public class AccountFlagsTransaction extends Transaction {
return ValidationResult.NO_FLAG_PERMISSION;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
Account target = this.getTarget();

View File

@@ -49,6 +49,11 @@ public class AccountLevelTransaction extends Transaction {
return ValidationResult.NO_FLAG_PERMISSION;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
Account target = getTarget();

View File

@@ -84,6 +84,11 @@ public class AddGroupAdminTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Update Group adminship
@@ -98,4 +103,4 @@ public class AddGroupAdminTransaction extends Transaction {
group.unpromoteToAdmin(this.addGroupAdminTransactionData);
}
}
}

View File

@@ -60,6 +60,11 @@ public class ArbitraryTransaction extends Transaction {
arbitraryTransactionData.getFee());
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Wrap and delegate payment processing to Payment class.

View File

@@ -80,6 +80,11 @@ public class AtTransaction extends Transaction {
return Arrays.equals(atAccount.getLastReference(), atTransactionData.getReference());
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public ValidationResult isValid() throws DataException {
// Check recipient address is valid

View File

@@ -6,6 +6,7 @@ import java.util.List;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.block.BlockChain;
import org.qortal.controller.repository.NamesDatabaseIntegrityCheck;
import org.qortal.crypto.Crypto;
import org.qortal.data.naming.NameData;
import org.qortal.data.transaction.BuyNameTransactionData;
@@ -98,6 +99,17 @@ public class BuyNameTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) transactionData;
// Rebuild this name in the Names table from the transaction history
// This is necessary because in some rare cases names can be missing from the Names table after registration
// but we have been unable to reproduce the issue and track down the root cause
NamesDatabaseIntegrityCheck namesDatabaseIntegrityCheck = new NamesDatabaseIntegrityCheck();
namesDatabaseIntegrityCheck.rebuildName(buyNameTransactionData.getName(), this.repository);
}
@Override
public void process() throws DataException {
// Buy Name

View File

@@ -62,6 +62,11 @@ public class CancelAssetOrderTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Mark Order as completed so no more trades can happen

View File

@@ -83,6 +83,11 @@ public class CancelGroupBanTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Update Group Membership

View File

@@ -83,6 +83,11 @@ public class CancelGroupInviteTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Update Group Membership

View File

@@ -79,6 +79,11 @@ public class CancelSellNameTransaction extends Transaction {
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Update Name

View File

@@ -135,13 +135,18 @@ public class ChatTransaction extends Transaction {
return true;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public ValidationResult isValid() throws DataException {
// Nonce checking is done via isSignatureValid() as that method is only called once per import
// Check for blacklisted author by address
ResourceListManager listManager = ResourceListManager.getInstance();
if (listManager.isAddressInBlacklist(this.chatTransactionData.getSender())) {
if (listManager.listContains("blacklist", "address", this.chatTransactionData.getSender())) {
return ValidationResult.ADDRESS_IN_BLACKLIST;
}

View File

@@ -135,6 +135,11 @@ public class CreateAssetOrderTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Order Id is transaction's signature

View File

@@ -92,6 +92,11 @@ public class CreateGroupTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Create Group

View File

@@ -106,6 +106,11 @@ public class CreatePollTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Publish poll to allow voting

View File

@@ -203,6 +203,11 @@ public class DeployAtTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
ensureATAddress(this.deployAtTransactionData);

View File

@@ -100,6 +100,11 @@ public class GenesisTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
Account recipient = new Account(repository, this.genesisTransactionData.getRecipient());

View File

@@ -66,6 +66,11 @@ public class GroupApprovalTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Find previous approval decision (if any) by this admin for pending transaction

View File

@@ -87,6 +87,11 @@ public class GroupBanTransaction extends Transaction {
return ValidationResult.OK;
}
@Override
public void preProcess() throws DataException {
// Nothing to do
}
@Override
public void process() throws DataException {
// Update Group Membership

Some files were not shown because too many files have changed in this diff Show More