Compare commits

...

428 Commits

Author SHA1 Message Date
CalDescent
96a82381d1 Bump version to 1.6.0 2021-08-01 18:19:59 +01:00
CalDescent
3c6d9a4b8e Merge branch 'new-coins' 2021-07-31 21:27:40 +01:00
CalDescent
3073388403 Fix for missing transactions in getWalletTransactions() response
The previous criteria was to stop searching for more leaf keys as soon as we found a batch of keys with no transactions, but it seems that there are occasions when subsequent batches do actually contain transactions. The solution/workaround is to require 5 consecutive empty batches before giving up. There may be ways to improve this further by copying approaches from other BIP32 implementations, but this is a quick fix that should solve the problem for now.
2021-07-31 18:10:24 +01:00
CalDescent
67f856c997 Increased minimum order size to 3 DOGE, as some transactions for 2.1 DOGE were being rejected due to the "dust" error. 2021-07-31 12:20:08 +01:00
CalDescent
742fd0b444 Added /crosschain/htlc/refundAll API endpoint
This is the equivalent of the redeemAll API but can be used from the buyer's side to get their LTC or DOGE back in the event of a problem.
2021-07-31 12:19:12 +01:00
CalDescent
e1d69c0eae Removed /refund/{ataddress}/{receivingAddress} API
This was causing confusion and isn't generally needed.
2021-07-31 11:55:48 +01:00
CalDescent
49d4190615 Support multiple blockchains in the /crosschain/htlc APIs.
This involved a small refactor of the ACCT code to expose findSecretA() in a more generic way. Bitcoin is disabled for refunding and redeeming as it uses a legacy approach that we no longer support. The {blockchain} URL parameter has also been removed from the redeem and refund APIs, because it can be obtained from the ACCT via the code hash in the AT.
2021-07-31 11:11:30 +01:00
CalDescent
64d39765ca Merge pull request #56 from DrewMPeacock/update-splash
Update task tray icon files. Now with transparency™
2021-07-30 10:51:44 +01:00
Sir.Galahad
aca8f64415 Update task tray icon files. Now with transparency™ 2021-07-30 03:43:31 -06:00
CalDescent
855b600268 testBlockHeightSpeed() reduced from 30k to 10k blocks.
This was intermittently failing on github and I suspect it may have been hitting memory limits.
2021-07-30 09:02:15 +01:00
CalDescent
476d613e20 Merge branch 'master' of github.com:Qortal/qortal 2021-07-30 09:00:54 +01:00
CalDescent
fb8a4d0a41 Merge pull request #55 from DrewMPeacock/update-splash
Update Qortal node graphics.
2021-07-30 08:30:15 +01:00
CalDescent
130f3f6d41 Added code to parse variable length block headers, necessary for DOGE. 2021-07-29 09:00:55 +01:00
CalDescent
ed997af043 Limit order size to a minimum of 2 DOGE.
The "dust" threshold is around 1 DOGE - meaning orders below this size cannot be refunded or redeemed. The simplest solution is to prevent orders of this size being placed to begin with.
2021-07-29 08:47:45 +01:00
Sir.Galahad
3c47f6917a Merge remote-tracking branch 'qm/status-on-icon' into update-splash 2021-07-27 19:46:34 -06:00
Sir.Galahad
e32a486493 Update splash screen image. 2021-07-27 19:42:00 -06:00
QuickMythril
208da935a1 updating icons
design credit: Haoshiro
2021-07-27 07:20:15 -04:00
QuickMythril
1dda9a875e updating icons
design credit: Haoshiro
2021-07-27 07:19:58 -04:00
CalDescent
b26175b7c6 Set DOGE fees to sensible values for mainnet. These can probably be optimized. 2021-07-26 19:03:33 +01:00
CalDescent
ffc6befb38 Fixed some missing DOGECOIN references 2021-07-26 09:21:56 +01:00
CalDescent
9df7c96d08 Added Dogecoin TradeBot and ACCT 2021-07-25 18:45:10 +01:00
CalDescent
32fa66f0a2 Merge pull request #54 from DrewMPeacock/master
Include AT address in /trades API call.
2021-07-23 08:50:27 +01:00
Sir.Galahad
7153ed022c Include AT address in /trades API call.
Include Seller Qortal address in /trades API call.
Include Buyer Qortal Receiving address in /trades API call.
2021-07-22 22:08:13 -06:00
CalDescent
50e4e71abb Added DOGE wallet. 2021-07-18 16:55:39 +01:00
CalDescent
d6e65a3d63 Removed version number from qortal.exe and qortal.zip
Recently we have stopped including the version number in the zip and exe files uploaded to github, as this allows us to use the "https://github.com/Qortal/*/releases/latest/download/*" redirect for all 3 files when linking from the qortal.org website. Previously, it could only be done for the JAR since this was the only file that didn't contain a version number. This avoids having to update the website every time we distribute a new release.

Note that this currently requires that the Qortal-x.y.z.exe file created by AdvancedInstaller is renamed to qortal.exe before running ./build-release.sh. If you forget to rename, the script will exit with a warning that the file couldn't be found.
2021-07-18 10:36:04 +01:00
CalDescent
79691541ae Merge pull request #51 from JaymenChou/patch-2
Update and rename SysTray_zh.properties to SysTray_zh_SC.properties
2021-07-08 23:06:19 +01:00
CalDescent
05d0542875 Merge pull request #50 from JaymenChou/patch-1
Create SysTray_zh_TC.properties
2021-07-08 23:06:03 +01:00
CalDescent
1d22b39a1d Merge pull request #52 from marcomoesman/master
Create Dutch (nl_NL) translations
2021-07-08 23:05:40 +01:00
CalDescent
549b68cf71 Merge pull request #53 from DrewMPeacock/master
Removes the flawed BIP-39 implementation in the core.
2021-07-08 23:05:15 +01:00
CalDescent
55f87de2e0 Updated AdvancedInstaller project for v1.5.6 2021-07-08 19:19:35 +01:00
CalDescent
b8424e20aa Bump version to 1.5.6 2021-07-08 18:24:11 +01:00
Sir.Galahad
bbe3a30e77 Removes the flawed BIP-39 implementation in the core.
Removes API calls for /mnemonic.

Readout for VanityGen.java now excludes a BIP-39 seed-phrase and only gives a raw private key.
2021-07-08 02:24:31 -06:00
Marco Moesman
39d8750ef9 Merge branch 'Qortal:master' into master 2021-07-01 10:24:03 +02:00
CalDescent
52b0c244a8 Extend CHECKPOINT_LOCK to HSQLDBSaver.execute()
This is used when saving new data to the db, so also needs to be blocked if we are checkpointing or deciding whether to checkpoint.
2021-06-28 19:24:53 +01:00
CalDescent
ee95a00ce2 Hopeful fix for "Performing repository CHECKPOINT..." deadlock.
This is probably our number one reliability issue at the moment, and has been a problem for a very long time.

The existing CHECKPOINT_LOCK would prevent new connections being created when we are checkpointing or about to checkpoint. However, in many cases we obtain the db connection early on and then don't perform any queries until later. An example would be in synchronization, where the connection is obtained at the start of the process and then retained throughout the sync round. My suspicion is that we were encountering this series of events:
1. Open connection to database
2. Call maybeCheckpoint() and confirm there are no active transactions
3. An existing connection starts a new transaction
4. Checkpointing is performed, but deadlocks due to the in-progress transaction

This potential fix includes preparedStatement.execute() in the CHECKPOINT_LOCK, to block any new transactions being started when we are locked for checkpointing. It is fairly high risk so we need to build some confidence in this before releasing it.
2021-06-28 09:13:36 +01:00
QuickMythril
11566ec923 set icon on status change 2021-06-27 03:45:15 -04:00
QuickMythril
a78ff08202 add setTrayIcon function 2021-06-27 03:44:29 -04:00
QuickMythril
ceb3969c8b load icons into gui 2021-06-27 03:44:25 -04:00
QuickMythril
6f048ef40e add status icons 2021-06-27 03:41:49 -04:00
Marco Moesman
aff4f6c859 Create TransactionValidity_nl.properties 2021-06-24 19:13:55 +02:00
Marco Moesman
1f8f73fa30 Create ApiError_nl.properties 2021-06-24 18:44:53 +02:00
Marco Moesman
620d6624a9 Create SysTray_nl.properties 2021-06-24 18:30:50 +02:00
JaymenChou
287f42ae64 Update and rename SysTray_zh.properties to SysTray_zh_SC.properties
Rename to zh_SC for better distinguish between zh_SC (Simple Chinese)and zh_TC(Traditional Chinese)
Rephrase some of the words for better understanding.
2021-06-24 11:42:48 +08:00
JaymenChou
d976c97d13 Create SysTray_zh_TC.properties 2021-06-24 11:31:46 +08:00
CalDescent
6d549b0754 Updated AdvancedInstaller project for v1.5.5 2021-06-23 19:57:03 +01:00
CalDescent
02dd64558f Bump version to 1.5.5 2021-06-23 18:21:55 +01:00
CalDescent
d25e98d9c4 Include peer connection ID in recently created log message. 2021-06-20 10:26:14 +01:00
CalDescent
227cdc1ec8 Log each sync attempt when our blockchain isn't up to date
Without this it can appear as though nothing is happening for a while after the app launches.
2021-06-20 07:25:25 +01:00
CalDescent
2c585a9328 Upper connection time limit reduced from 60 mins to 20 mins.
Now that we aren't disconnecting mid sync, we can get away with more frequent disconnections. This brings the average connection length to around 9 mins.
2021-06-19 17:25:50 +01:00
CalDescent
45b0d9e19b Added more initial peers, submitted by CWDSYSTEMS 2021-06-19 16:19:53 +01:00
CalDescent
026a4b896c Added BTC and LTC electrum nodes submitted by CWDSYSTEMS 2021-06-19 14:50:18 +01:00
CalDescent
78237fcd11 Don't intentionally disconnect peers if we are currently syncing with them. 2021-06-19 13:12:16 +01:00
CalDescent
73cc3dcb92 Force a disconnect of each peer when the connection age reaches the maximum allowed time.
Connection limits are defined in settings (denominated in seconds):
"minPeerConnectionTime": 120,
"maxPeerConnectionTime": 3600

Peers will disconnect after a randomly chosen amount of time between the min and the max. The default range is 2 minutes to 1 hour, as above.

This encourages nodes to connect to a wider range of peers across the course of each day, rather than staying connected to an "island" of peers for an extended period of time. Hopefully this will reduce the amount of parallel chains that can form due to permanently connected clusters of peers.

We may find that we need to reduce the defaults to get optimal results, however it is best to do this incrementally, with the option for reducing further via each node's settings. Being too aggressive here could cause some of the earlier problems (e.g. 20% missing blocks minted) to reappear. We can re-evaluate this in the next version. Note that if defaults are reduced significantly, we may need to add code to prevent this from happening mid-sync. With higher defaults, this is less of an issue.

Thanks to @szisti for supplying some base code for this commit, and also to @CWDSYSTEMS for diagnosing the original problem.
2021-06-19 13:03:46 +01:00
CalDescent
4cff03e7fe Include "size" value in the "Synchronized with peer" logs.
This indicates the size of the re-org/rollback that was required in order to perform this sync operation. It is only included if it's greater than 0 blocks.
2021-06-19 09:04:14 +01:00
CalDescent
280f7814aa Merge branch 'master' of github.com:Qortal/qortal 2021-06-14 18:49:55 +01:00
CalDescent
3174681bd8 Removed /src/main/resources/log*.properties from .gitignore
We must be careful not to add files to the resources folder accidentally, given that a bundled log4j2.properties file is used in preference to the user's copy. By keeping this out of gitignore, it becomes more obvious if a file is added, and it can then be caught and removed before a release.
2021-06-14 18:49:24 +01:00
CalDescent
853f80b928 Updates to build-zip.sh and build-release.sh
There were necessary for these scripts to function in my build environment (Mac OSX). This may give errors when running in other environments, but we can deal with that in future, when others need to use these scripts.
2021-06-14 18:46:21 +01:00
CalDescent
8bdad377d7 Removed outdated qortal.jar from "WindowsInstaller/Install Files" directory.
Including an older JAR in the source code only leads to confusion, because a zip of the source code is automatically included with each github release. From what I can see, there is no need for it to be here. Added to .gitignore so we have the option of keeping a local copy.
2021-06-14 18:41:27 +01:00
CalDescent
9e1c2a5bd1 Updated AdvancedInstaller project for v1.5.4 2021-06-14 18:39:46 +01:00
CalDescent
b1777b6011 Bump version to 1.5.4 2021-06-13 19:22:16 +01:00
CalDescent
e3923b7b22 Fixed issue causing frequent disconnects (found by szisti)
When sending or requesting more than 1000 online accounts, peers would be disconnected with an EOF or connection reset error due to an intentional null response. This response has been removed and it will instead now only send the first 1000 accounts, which prevents the disconnections from occurring.

In theory, these accounts should be in a different order on each node, so the 1000 limit should still result in a fairly even propagation of accounts. However, we may want to consider increasing this limit, to maximise the propagation speed.

Thanks to szisti for tracking this one down.
2021-06-11 19:10:04 +01:00
CalDescent
a43993e3ec Use placeholder build timestamp and build version when building without mvn package (e.g. from within an IDE)
This fixes an exception thrown when running directly in IntelliJ, as previously we relied on mvn package to parse the commit hash and timestamp.
2021-06-11 19:01:35 +01:00
CalDescent
319e64bacc Defend against an edge case NPE in the chat messages websocket. 2021-06-06 10:34:20 +01:00
CalDescent
ecf044bed1 Removed qortal-backup folder from git 2021-06-06 10:29:58 +01:00
CalDescent
76e1de38e8 Workaround for issue where sometimes an AT stays in "TRADING" mode even after it is marked as finished. This caused Bob's tradebot to enter BOB_REFUNDED mode instead of redeeming the LTC. This workaround treats "TRADING" as "REDEEMED" as long as the AT is finished. It will still enter the BOB_REFUNDED state if the AT's trade state is "REFUNDED" or "CANCELLED", to prevent it trying to redeem LTC without the secret. Longer term we need to prevent the AT itself from getting in this state to begin with, but this should at least solve the LTC redemption problem that occurs as a result. 2021-06-05 12:31:49 +01:00
CalDescent
1648a74ed7 Removed code which auto deletes trade bot data if it can't locate the AT after 24 hours. It's not a good idea to ever delete trade bot data, since it can contain private keys necessary to redeem or refund LTC. We have seen at least one instance of this where the trade bot data was deleted for an active trade. We still have the auto backup in these cases, so the keys are recoverable, but it's safest to avoid any auto deletions. 2021-06-05 11:25:05 +01:00
CalDescent
c443187d0b Reduce log spam by logging the total number of expired unconfirmed transactions that are deleted, rather than each one individually. The individual deletion logs have been moved from INFO to DEBUG. 2021-06-01 20:06:37 +01:00
CalDescent
8c305d8390 Reduced log levels of recent synchronizer / controller log additions from INFO to DEBUG.
These are no longer necessary now that we have achieved good stability in the network.
2021-06-01 08:39:38 +01:00
CalDescent
0345c5c03b Updated AdvancedInstaller project for v1.5.3 2021-05-31 21:27:42 +01:00
CalDescent
cc6ac4c9d9 Bump version to 1.5.3 2021-05-31 17:36:21 +01:00
CalDescent
815934ff5c Added GET /crosschain/htlc/redeemAll/LITECOIN API
This loops through all sell orders and attempts to redeem the LTC from each one. It will return true if at least one was redeemed, or false if none are available to be redeemed. Details are logged to the log.txt file rather than returned in the API response.
2021-05-29 19:43:08 +01:00
CalDescent
5a84016a91 Merge pull request #39 from szisti/networking
Code formatting, connection age, and logging changes for networking
2021-05-28 10:09:07 +01:00
Istvan Szabo
bb0269f484 Converted time format 2021-05-28 08:53:01 +01:00
Istvan Szabo
1adc9349fc Added connection age to connected peers dto 2021-05-28 08:04:57 +01:00
Istvan Szabo
06215c83f2 Reduced log levels 2021-05-27 10:48:17 +01:00
Istvan Szabo
8a828137ee Removed code coverage report as it seems to conflict with tests randomly 2021-05-27 09:52:33 +01:00
Istvan Szabo
de4b1c8f09 Removed missed functional change 2021-05-27 09:15:32 +01:00
Istvan Szabo
265d40f04a Code formatting and logging changes for networking 2021-05-27 09:03:18 +01:00
szisti
b64e52c0c0 Automated testing (#38)
* added basic workflow

* Testing workflow

* renamed workflow file

* Disabled extremely slow test

* Disabled currently failing tests

* Added jacoco and updated workflow

* We cannot run gui tests headless

* Fixed jacoco configuration

* Updated job name in the workflow

* Adjusting workflow

* Testing maven caching

* Added logging for one of the jacoco related issues

* Updated coverage logging

Co-authored-by: Istvan Szabo <istvan.szabo@betvictor.com>
2021-05-26 11:27:46 +01:00
CalDescent
ac02e5c0a6 Merge pull request #37 from szisti/fix-cross-transaction-display
Fix cross chain transaction display
2021-05-26 08:39:51 +01:00
Istvan Szabo
427a415fbf Adjusted bitcoiny to convert transaction info into the new DTO 2021-05-25 23:57:54 +01:00
Istvan Szabo
9a3414aaa7 Added new DTO to store the data 2021-05-25 23:55:12 +01:00
CalDescent
c8897ecf9b Rewrite of HSQLDBATRepository.getBlockATStatesAtHeight() SQL query
The previous query was taking almost half a second to run each time, whereas the new version runs 10-100x faster. This was the main bottleneck with block serialization and should therefore allow for much faster syncing once rolled out to the network. Tested several thousand blocks to ensure that the results returned by the new query match those returned by the old one.
2021-05-24 19:52:20 +01:00
CalDescent
2c8b94d469 Always use the org.qortal.utils.Base58 implementation
A couple of classes were using the bitcoinj alternative, which is twice as slow. This mostly affected the API on port 12392, as byte arrays were automatically encoded as base58 strings via the Base58TypeAdapter / JAXB package-info.
2021-05-24 19:38:01 +01:00
CalDescent
36c1cfae51 Log the P2SH address when redeeming or refunding LTC via the API. 2021-05-24 19:00:04 +01:00
CalDescent
41ad78750e Don't allow QORT addresses to be used as the receiving address when redeeming LTC
This is probably more validation than is actually needed, but given that we use the same field for LTC and QORT receiving addresses in the database, it is best to be extra careful.
2021-05-24 18:59:41 +01:00
CalDescent
3eaa4d5b38 Added /crosschain/htlc/refund/LITECOIN/{ataddress}/{receivingAddress} API
This is the same as the /crosschain/htlc/refund/LITECOIN/{ataddress} API, but allows a custom destination address to be specified.
2021-05-23 18:52:03 +01:00
CalDescent
35176f9550 Added other files to .gitignore 2021-05-23 16:57:09 +01:00
CalDescent
eb2c7268ea Removed .DS_Store files. 2021-05-23 15:31:26 +01:00
CalDescent
80311355ae Added /blocks/signature/{signature}/data API
This returns serialized, base58 encoded data for the entire block. It is the same format as the data sent between nodes when synchronizing, with base58 encoding added so that it can be outputted cleanly in the API response.
2021-05-23 13:10:47 +01:00
CalDescent
39d1590ace Improved descriptions of the new API endpoints. 2021-05-22 14:16:14 +01:00
CalDescent
0b36b650a4 Added /redeem/LITECOIN/{ataddress} API
This is the equivalent of the refund API but can be used by the seller to redeem LTC from a stuck transaction, by supplying the associated AT address, There are no lockTime requirements; it is redeemable as soon as the buyer has redeemed the QORT and sent the secret to the seller.
2021-05-22 13:59:00 +01:00
CalDescent
39575e8542 Added /refund/LITECOIN/{ataddress} API
This is designed to be called by the buyer, and will force refund their P2SH transaction associated with the supplied AT. The tradebot responsible for this trade must be present in the user's db for this API access the necessary data. It must be called after lockTime has passed, which for LTC is currently 60 minutes from the time that the P2SH was funded. Trying to refund before this time will result in a FOREIGN_BLOCKCHAIN_TOO_SOON error.
2021-05-22 10:09:28 +01:00
CalDescent
326ef498b0 Added /crosschain/htlc/redeem/LITECOIN/{ataddress}/{tradePrivateKey}/{secret}/{receivingAddress} API
This can currently be used by either the buyer or the seller, but it requires the seller's trade private key & receiving address to be specified, along with the buyer's secret. Currently hardcoded to LITECOIN but I will aim to make this generic as we start adding more coins.
2021-05-22 09:51:57 +01:00
CalDescent
5148bad82e /crosschain/htlc APIs now take base58 encoded params instead of hex.
This makes them more compatible with the output of the /crosschain/tradebot and /crosschain/trade/{ataddress} APIs which is likely where most people will be retrieving data from, rather than the database itself.
2021-05-20 09:20:14 +01:00
CalDescent
518f02472f Added POST /crosschain/LitecoinACCTv1/redeemmessage API
This is similar to the BTC equivalent, but removes secretB as an input parameter. It also signs and broadcasts the transaction, because the wallet isn't needed for this. These transactions have to be signed using the tradePrivateKey from the tradebot data rather than any of the wallet's keys.

There are two other LitecoinACCTv1 APIs still to implement, but I will leave these until they are needed.
2021-05-20 07:59:19 +01:00
CalDescent
ee5a132eb2 Updated AdvancedInstaller project for v1.5.2 2021-05-17 20:31:28 +01:00
CalDescent
654dc5bff3 Bump version to 1.5.2 2021-05-17 17:02:38 +01:00
CalDescent
13dcf7f72a Added/updated some comments relating to a possible future optimization. 2021-05-16 11:03:11 +01:00
CalDescent
65c26f17df Reduced "Error while trying to find common block with peer" log from INFO to DEBUG when determining which peer to sync with. When performing the actual synchronization, use INFO logging as this is a more serious error. 2021-05-16 10:45:40 +01:00
CalDescent
3bedba71d5 Reduced frequency and level of some synchronizer logs. 2021-05-16 10:36:41 +01:00
CalDescent
1ba64d9745 Bumped bitcoinj version from 0.15.6 to 0.15.10 2021-05-16 10:00:28 +01:00
CalDescent
84bf570243 Added optional "maxtrades" parameter to /crosschain/price/{blockchain} API
This specifies the maximum number of trades to be used when calculating the price. Default: 10
2021-05-16 09:51:11 +01:00
CalDescent
28d50bccf9 Exclude peers if we don't have a complete set of their block summaries.
This tightens up the decision making by adding two requirements:

1. The peer must return the same number of summaries to the ones requested.
2. The peer must return a summary that matches its latest reported signature.

This ensures we are always making sync decisions based on accurate data, and removes peers that are currently mid re-org. This is probably more validation than is actually necessary, but it's best to be really thorough here so it is as optimized as possible.
2021-05-16 09:15:37 +01:00
CalDescent
66711c2e9d Require a complete sync in syncToPeerChain()
We have gone backwards and forwards on this one a lot recently, but now that stability has returned, it is best to tighten this up. Previously it was loosened to help reduce network load, but that is no longer a problem. With this stricter approach, it should prevent a node ending up in an incomplete state after syncing, which is the main cause of the shorter re-orgs we are seeing.
2021-05-16 08:45:23 +01:00
CalDescent
92d8c37d7d Added AT count to block debug logs. 2021-05-15 12:54:46 +01:00
CalDescent
5824f75669 Rework of the repository export and import functions.
The existing HSQL export/import (PERFORM EXPORT SCRIPT and PERFORM IMPORT SCRIPT) have been replaced with a custom JSON import and export. Whilst this is less generic, it has some significant advantages:

- When exporting data, it is now able to combine the exported data with any data that already exists in the backup file. This prevents a backup after a bootstrap from overwriting data from before the bootstrap, and removes the need for all of the "archive" files that we currently create.
- Adds support for partial imports, and updates. Previously an import would fail if any of the data being imported already existed in the db. It will now add new rows and update existing ones.
- The format and contents of the exported trade bot data now matches the output of the /crosschain/tradebot API.
- Data is retrieved without the need for a database lock, and therefore the export process is much faster and less invasive. This should prevent the lockups and other problems seen when using the trade portal.

For now, there are a couple of trade-offs to using this new approach:
- The minting key import/export has been temporarily removed until there is more time to transition it to this new format.
- Existing .script backups can no longer be imported using versions higher than 1.5.1.

Both of these can be solved by temporarily running version 1.5.1, performing the necessary imports/exports, then returning to the latest version. Longer term the minting keys export/import will be reimplemented using the JSON format.
2021-05-15 12:19:15 +01:00
CalDescent
deb8adafc9 Added org.json dependency.
The com.googlecode.json-simple dependency we use in other parts of the project isn't ideal for some of the more complex parsing.
2021-05-15 09:15:29 +01:00
CalDescent
d2649b237c Moved chain weight calculation log from DEBUG to TRACE. 2021-05-11 19:01:23 +01:00
CalDescent
6532c258f6 Reduced log spam. 2021-05-10 09:10:14 +01:00
CalDescent
83e2b10904 Merge branch 'ignore-old-versions' 2021-05-10 09:01:04 +01:00
CalDescent
26c1793d85 Added "allowConnectionsWithOlderPeerVersions" setting (default: true)
This controls whether to allow connections with peers below minPeerVersion.

If true, we won't sync with them but they can still sync with us, and will show in the peers list. This is the default, which allows older nodes to continue functioning, but prevents them from interfering with the sync behaviour of updated nodes.

If false, sync will be blocked both ways, and they will not appear in the peers list at all.
2021-05-10 09:00:42 +01:00
CalDescent
23a9eea26b Merge branch 'ignore-old-versions' 2021-05-09 23:02:35 +01:00
CalDescent
af9b536dd9 Moved version check above getMinBlockchainPeers() check, so that nodes with old versions aren't counted. 2021-05-09 23:00:51 +01:00
CalDescent
e4874f86f9 Merge branch 'block-timings' of github.com:Qortal/qortal into block-timings
# Conflicts:
#	src/main/java/org/qortal/api/model/BlockMintingInfo.java
#	src/main/java/org/qortal/api/resource/BlocksResource.java
#	tools/block-timings.sh
2021-05-09 19:25:33 +01:00
CalDescent
e300a957e4 Added online accounts count to /blocks/byheight/{height}/mintinginfo API and block-timings.sh script. 2021-05-09 19:25:05 +01:00
CalDescent
1c38afcd25 Slight reordering of vars. 2021-05-09 19:24:25 +01:00
CalDescent
a06faa7685 Updated usage info to reflect the fact that the "count" parameter is optional.
Usage:

block-timings.sh <startheight> [count] [target] [deviation] [power]
2021-05-09 19:24:25 +01:00
CalDescent
019ab2b21d Added tools/block-timings-sh which can be used to test out new block timings (specified in blockchain.json).
The script will fetch a set of blocks and then backtest the specified blockTimings settings (target, deviation, and power) against those real life blocks. This allows configurations to be fine tuned to tighten up block times, and to adjust the timestamp variance between levels.

Usage:
block-timings.sh <startheight> <count> [target] [deviation] [power]

startheight: a block height, preferably within the untrimmed range, to avoid data gaps
count: the number of blocks to request and analyse after the start height. Default: 100
target: the target block time in milliseconds. Originates from blockchain.json. Default: 60000
deviation: the allowed block time deviation in milliseconds. Originates from blockchain.json. Default: 30000
power: used when transforming key distance to a time offset. Originates from blockchain.json. Default: 0.2
2021-05-09 19:24:25 +01:00
CalDescent
f6ba5f5d51 Added /blocks/byheight/{height}/mintinginfo API, which returns info on the minter level, key distance, and block timings. 2021-05-09 19:24:25 +01:00
CalDescent
c4cbb64643 Added "minPeerVersion" setting, and avoid syncing with peers on lower versions. 2021-05-09 17:38:07 +01:00
CalDescent
8260cec713 Added "maximumCount" parameter to HSQLDBATRepository.getMatchingFinalATStatesQuorum() and use it to limit the number of ATs being returned in the query.
Initially set to 10 when used by the /crosschain/price/{blockchain} API, so that the price is based on the last 10 trades rather than every trade that has ever taken place.
2021-05-09 15:56:15 +01:00
CalDescent
f4520e2752 Skip Block.logDebugInfo() altogether if the log level is more specific than DEBUG, to avoid wasting resources. 2021-05-09 09:00:53 +01:00
CalDescent
475802afbc Fixed divide by zero exception.
Block.calcKeyDistance() cannot be called on some trimmed blocks, because the minter level is unable to be inferred in some cases. This generally hasn't been an issue, but the new Block.logDebugInfo() method is invoking it for all blocks. For now I am adding defensiveness to the debug method, but longer term we might want to add defensiveness to Block.calcKeyDistance() itself, if we ever encounter this issue again. I will leave it alone for now, to reduce risk.
2021-05-09 08:25:24 +01:00
Tom
a170668d9d Updated AdvancedInstaller project for v1.5.1 2021-05-07 09:58:15 +01:00
Tom
f8dac39076 Updated AdvancedInstaller project for v1.5.0
This includes updating AdoptOpenJDK to version 11.0.11.9, because 11.0.6.10 is no longer recommended or available in their archive. It also looks like I am using a newer version of AdvancedInstaller itself.
2021-05-07 09:40:38 +01:00
CalDescent
fe4ae61552 Added "maxRetries" setting.
This controls the maximum number of retry attempts if a peer fails to respond with the requested data.
2021-05-06 17:49:45 +01:00
CalDescent
0c3597f757 Bump version to 1.5.1 2021-05-05 18:41:05 +01:00
CalDescent
6109bdeafe Set go-live timestamp for same-length chain weight consensus: 1620579600000 2021-05-05 18:40:07 +01:00
CalDescent
6e9a61c4e5 Fixed logging issue where it would underreport the number of common blocks found when loading some from the cache. 2021-05-02 20:51:53 +01:00
CalDescent
8e244fd956 Fixed yet another bug with minChainLength. 2021-05-02 20:45:20 +01:00
CalDescent
2eb6771963 Adapted logging in comparePeers() to report correct values for both chain weight algorithms. 2021-05-02 20:26:51 +01:00
CalDescent
db77108054 Log the number of blocks used in Block.calcChainWeight()
This makes it easier to check that the new consensus code is being used, and that it is working correctly.
2021-05-02 19:59:32 +01:00
CalDescent
241e2bef85 Merge branch 'master' into chain-weight-consensus
# Conflicts:
#	src/main/java/org/qortal/block/BlockChain.java
#	src/main/resources/blockchain.json
#	src/test/resources/test-chain-v2-founder-rewards.json
#	src/test/resources/test-chain-v2-leftover-reward.json
#	src/test/resources/test-chain-v2-minting.json
#	src/test/resources/test-chain-v2-qora-holder-extremes.json
#	src/test/resources/test-chain-v2-qora-holder.json
#	src/test/resources/test-chain-v2-reward-scaling.json
#	src/test/resources/test-chain-v2.json
2021-05-02 18:18:20 +01:00
CalDescent
fac02dbc7d Fixed bug in maxHeight parameter passed to Block.calcChainWeight()
Like the others, this one is only relevant after switching to same-length chain weight comparisons.
2021-05-02 15:56:13 +01:00
CalDescent
9ebcd55ff5 Fixed calculation error in existing chain weight code, which would have caused the last block to be missed out of the comparison after switching to same-length chain comparisons. 2021-05-01 13:34:13 +01:00
CalDescent
50244c1c40 Fixed bug which would cause other peers to not be compared against each other, if we had no blocks ourselves.
Again, this wouldn't have affected anything in 1.5.0 or before, but it will become more significant if we switch to same-length chain weight comparisons.
2021-05-01 13:32:16 +01:00
CalDescent
b4395fdad1 Fixed bug which could cause minChainLength to report a higher value.
This wouldn't have affected anything in 1.5.0, but it will become more significant if we switch to same-length chain weight comparisons.
2021-05-01 10:57:24 +01:00
CalDescent
1da8994be7 Log the block timestamp, minter level, online accounts, key distance, and weight, when orphaning or processing.
This gives an insight into the contents of each chain when doing a re-org. To enable this logging, add the following to log4j2.properties:

logger.block.name = org.qortal.block.Block
logger.block.level = debug
2021-05-01 10:24:50 +01:00
QuickMythril
55ff1e2bb1 updated and tested BTC electrum servers (#36)
* updated electrum servers

mainnet list: https://1209k.com/bitcoin-eye/ele.php?chain=btc
testnet list: https://1209k.com/bitcoin-eye/ele.php?chain=tbtc

* removed servers

tested each mainnet server individually and removed those that did not respond
2021-05-01 09:18:46 +01:00
CalDescent
5fd8528c49 Small refactor for code readability, and added some defensiveness to avoid possible NPEs. 2021-04-29 09:04:59 +01:00
CalDescent
26d8ed783a Same as commit c0c5bf1, but for blocks as well as block summaries. 2021-04-29 08:55:16 +01:00
CalDescent
c0c5bf1591 Apply blocks in syncToPeerChain() if the latest received block is newer than our latest, and we started from an out of date chain.
This solves a common problem that is mostly seen when starting a node that has been switched off for some time, or when starting from a bootstrap. In these cases, it can be difficult get synced to the latest if you are starting from a small fork. This is because it required that the node was brought up to date via a single peer, and there wasn't much room for error if it failed to retrieve a block a couple of times. This generally caused the blocks to be thrown away and it would try the same process over and over.

The solution is to apply new blocks if the most recently received block is newer than our current latest block. This gets the node back on to the main fork where it can then sync using the regular applyNewBlocks() method.
2021-04-28 22:03:13 +01:00
CalDescent
c17a481b74 Bump version to 1.5.0 2021-04-26 18:34:01 +01:00
CalDescent
a9a0e69ec0 Set go-live block height for share bin fix: block 399000 2021-04-26 17:19:39 +01:00
CalDescent
ea1fed2fd3 Merge branch 'block-reward-distribution-fix' 2021-04-26 17:16:14 +01:00
CalDescent
b37f2c7d7f MAXIMUM_RETRIES set to 2, as 3 retries may have been slightly too many. 2021-04-26 17:08:21 +01:00
CalDescent
0c0c5ff077 Invalidate our block summaries cache for a peer if it fails to respond with signatures when synchronizing. 2021-04-25 12:50:40 +01:00
CalDescent
e12b99d17e Invalidate our common block cache for a peer if we can't find a common block when synchronizing. 2021-04-25 09:37:32 +01:00
CalDescent
d599146c3a Cache peer block summaries to avoid duplicate requests when comparing peers. 2021-04-24 22:10:40 +01:00
CalDescent
476731a2c3 In syncToPeerChain(), only apply a partial set of peer's blocks if they are recent.
If a peer fails to reply with all requested blocks, we will now only apply the blocks we have received so far if at least one of them is recent. This should prevent or greatly reduce the scenario where our chain is taken from a recent to an outdated state due to only partially syncing with a peer. It is best to keep our chain "recent" if possible, as this ensures that the peer selection code always runs, and therefore avoids unnecessarily syncing to a random peer on an inferior chain.
2021-04-24 20:12:11 +01:00
CalDescent
1e491dd8fb MAXIMUM_RETRIES increased from 1 to 3.
Now that we are spending a lot of time to carefully select a peer to sync with, it makes sense to retry a couple more times before giving up and starting the peer selection process all over again.
2021-04-24 19:45:53 +01:00
CalDescent
ba6397b963 Improved logging, to give a clearer picture of the peer selection decisions. 2021-04-24 19:23:09 +01:00
CalDescent
3146da6aec Don't add to the inferior chain signatures list when comparing peers against each other.
In these comparisons it's easy to incorrectly identify a bad chain, as we aren't comparing the same number of blocks. It's quite common for one peer to fail to return all blocks and be marked as an inferior chain, yet we have other "good" peers on that exact same chain. In those cases we would have stopped talking to the good peers again until they received another block.

Instead of complicating the logic and keeping track of the various good chain tip signatures, it is simpler to just remove the inferior peers from this round of syncing, and re-test them in the next round, in case they are in fact superior or equal.
2021-04-24 16:43:29 +01:00
CalDescent
5643e57ede Fixed string formatting error. 2021-04-24 16:21:04 +01:00
CalDescent
f532dbe7b4 Optimized code in Synchronizer.uniqueCommonBlocks() 2021-04-24 15:22:29 +01:00
CalDescent
ec2af62b4d Fix for bug which failed to remove peers without block summaries.
The iterator was removing the peer from the "peersSharingCommonBlock" array, when it should have been removing it from the "peers" array. The result was that the bad peer would end up in the final list of good peers, and we could then sync with it when we shouldn't have.
2021-04-24 15:21:30 +01:00
CalDescent
423142d730 Tidied up RECOVERY_MODE_TIMEOUT constant, and made checkRecoveryModeForPeers() private. 2021-04-24 10:35:01 +01:00
CalDescent
bdddb526da Added recovery mode, which is designed to automatically bring back a stalled network.
The existing system was unable to resume without manual intervention if it stalled for more than 7.5 minutes. After this time, no peers would have "recent' blocks, which are prerequisites for synchronization and minting.

This new code monitors for such a situation, and enters "recovery mode" if there are no peers with recent blocks for at least 10 minutes. It also requires that there is at least one connected peer, to reduce false positives due to bad network connectivity.

Once in recovery mode, peers with no recent blocks are added back into the pool of available peers to sync with, and restrictions on minting are lifted. This should allow for peers to collaborate to bring the chain back to a "recent" block height. Once we have a peer with a recent block, the node will exit recovery mode and sync as normal.

Previously, lifting minting restrictions could have increased the risk of extra forks, however it is much less risky now that nodes no longer mint multiple blocks in a row.

In all cases, minBlockchainPeers is used, so a minimum number of connected peers is required for syncing and minting in recovery mode, too.
2021-04-23 09:21:15 +01:00
CalDescent
dbf1ed40b3 Log the parent block's signature when minting a new block, to help us keep track of the chain it's being minted on. 2021-04-19 09:33:24 +01:00
CalDescent
02ace06526 Revert "When syncing to a peer on a different fork, ensure that all blocks are obtained before applying them."
This reverts commit c919797553.
2021-04-18 13:03:04 +01:00
CalDescent
2d2bfc0a4c Log the number of common blocks found in each search. 2021-04-18 13:02:38 +01:00
CalDescent
3c22a12cbb Experimental idea to prevent a single node signing more than one block in a row.
This could drastically reduce the number of forks being created. Currently, if a node is having problems syncing, it will continue adding to its own fork, which adds confusion to the network. With this new idea, the node would be prevented from adding to its own chain and is instead forced to wait until it has retrieved the next block from the network.

We will need to test this on the testnet very carefully. My worry is that, because all minters submit blocks, it could create a situation where the first block is submitted by everyone, and the second block is submitted by no-one, until a different candidate for the first block has been obtained from a peer. This may not be a problem at all, and could actually improve stability in a huge way, but at the same time it has the potential to introduce serious network problems if we are not careful.
2021-04-18 10:26:36 +01:00
CalDescent
3071ef2f36 Removed redundant uiLocalServers 2021-04-17 20:55:30 +01:00
CalDescent
3022cb22d6 Merge branch 'master' into prioritize-peers 2021-04-17 20:51:35 +01:00
CalDescent
e9b4a3f6b3 Automatically backup trade bot data when starting a new trade (from either side). 2021-04-17 20:45:35 +01:00
CalDescent
4312ebfcc3 Adapted the HSQLDBRepository.exportNodeLocalData() method
It now has a new parameter - keepArchivedCopy - which when set to true will cause it to rename an existing TradeBotStates.script to TradeBotStates-archive-<timestamp>.script before creating a new backup. This should avoid keys being lost if a new backup is taken after replacing the db.

In a future version we can improve this in such a way that it combines existing and new backups into a single file. This is just a "quick fix" to increase the chances of keys being recoverable after accidentally bootstrapping without a backup.
2021-04-17 20:44:57 +01:00
CalDescent
2c0e099d1c Removed wildcard import that was automatically introduced by Intellij. 2021-04-17 14:36:24 +01:00
CalDescent
b1eb02eb1d Merge pull request #33 from QuickMythril/version-on-tooltip
add version on tooltip
2021-04-17 13:21:20 +01:00
CalDescent
c919797553 When syncing to a peer on a different fork, ensure that all blocks are obtained before applying them.
In version 1.4.6, we would still sync with a peer even if we only received a partial number of the requested blocks/summaries. This could create a new problem, because the BlockMinter would often try and make up the difference by minting a new fork of up to 5 blocks in quick succession. This could have added to network confusion.

Longer term we may want to adjust the BlockMinter code to prevent this from taking place altogether, but in the short term I will revert this change from 1.4.6 until we have a better way.
2021-04-17 13:09:52 +01:00
CalDescent
08dacab05c Make sure to give up if we are requesting block summaries when the core needs to shut down. 2021-04-17 12:57:28 +01:00
CalDescent
2efc9218df Improved the process of selecting the next peer to sync with
Added a new step, which attempts to filter out peers that are on inferior chains, by comparing them against each other and our chain. The basic logic is as follows:

1. Take the list of peers that we'd previously have chosen from randomly.
2. Figure out our common block with each of those peers (if its within 240 blocks), using cached data if possible.
3. Remove peers with no common block.
4. Find the earliest common block, and compare all peers with that common block against each other (and against our chain) using the chain weight method. This involves fetching (up to 200) summaries from each peer after the common block, and (up to 200) summaries from our own chain after the common block.
5. If our chain was superior, remove all peers with this common block, then move up to the next common block (in ascending order), and repeat from step 4.
6. If our chain was inferior, remove any peers with lower weights, then remove all peers with higher common blocks.
7. We end up with a reduced list of peers, that should in theory be on superior or equal chains to us. Pick one of those at random and sync to it.

This is a high risk feature - we don't yet know the impact on network load. Nor do we know whether it will cause issues due to prioritising longer chains, since the chain weight algorithm currently prefers them.
2021-04-17 12:52:19 +01:00
CalDescent
41505dae11 Treat two block summaries as equal if they have matching signatures 2021-04-16 09:40:22 +01:00
CalDescent
45efe7cd56 Slight reordering of vars. 2021-04-10 18:24:33 +01:00
CalDescent
78cac7f0e6 Updated usage info to reflect the fact that the "count" parameter is optional.
Usage:

block-timings.sh <startheight> [count] [target] [deviation] [power]
2021-04-10 18:12:09 +01:00
CalDescent
a1a1b8e94a Added tools/block-timings-sh which can be used to test out new block timings (specified in blockchain.json).
The script will fetch a set of blocks and then backtest the specified blockTimings settings (target, deviation, and power) against those real life blocks. This allows configurations to be fine tuned to tighten up block times, and to adjust the timestamp variance between levels.

Usage:
block-timings.sh <startheight> <count> [target] [deviation] [power]

startheight: a block height, preferably within the untrimmed range, to avoid data gaps
count: the number of blocks to request and analyse after the start height. Default: 100
target: the target block time in milliseconds. Originates from blockchain.json. Default: 60000
deviation: the allowed block time deviation in milliseconds. Originates from blockchain.json. Default: 30000
power: used when transforming key distance to a time offset. Originates from blockchain.json. Default: 0.2
2021-04-10 17:57:28 +01:00
CalDescent
641a658059 Added /blocks/byheight/{height}/mintinginfo API, which returns info on the minter level, key distance, and block timings. 2021-04-10 17:49:04 +01:00
CalDescent
44ec447014 Show an error in publish-auto-update.pl if both sha256sum and sha256 aren't found in PATH. 2021-04-01 08:27:56 +01:00
CalDescent
98308ecf98 Bump version to 1.4.6 2021-04-01 08:09:50 +01:00
CalDescent
8d613a6472 MAXIMUM_RETRIES reduced from 3 to 1 2021-03-30 13:07:34 +01:00
CalDescent
c3e5298ecd Added a few checks for Controller.isStopping() in synchronizer loops, to try and speed up the shutdown time. 2021-03-30 13:05:43 +01:00
CalDescent
e89d31eb5a Rewrite of Synchronizer.syncToPeerChain(), this time borrowing ideas from Synchronizer.applyNewBlocks().
Main differences / improvements:
- Only request a single batch of signatures upfront, instead of the entire peer's chain. There is no point in requesting them all, as the later ones may not be valid by the time we have finished requesting all the blocks before them.
- If we fail to fetch a block, clear any queued signatures that are in memory and re-fetch signatures after the last block received. This allows us to cope with peers that re-org whilst we are syncing with them.
- If we can't find any more block signatures, or the peer fails to respond to a block, apply our progress anyway. This should reduce wasted work and network congestion, and helps cope with larger peer re-orgs.
- The retry mechanism remains in place, but instead of fetching the same incorrect block over and over, it will attempt to locate a new block signature each time, as described above. To help reduce code complexity, block signature requests are no longer retried.
2021-03-30 12:29:27 +01:00
CalDescent
30160e2843 Fixes to allow publish-auto-update.sh to work with sha256sum versions that add trailing characters. 2021-03-21 18:15:29 +00:00
catbref
503d22e4d0 Updated Qortal.aip for WindowsInstaller for v1.4.5 2021-03-21 18:05:38 +00:00
CalDescent
b9a0d489d7 Bump version to 1.4.5 2021-03-21 17:06:10 +00:00
catbref
d9d4c4c302 Bump Peer response timeout from 2s to 3s 2021-03-21 16:17:40 +00:00
catbref
81c6d75d62 Adjust Synchronizer.MAXIMUM_BLOCK_STEP to 128, which means final summaries request will have enough to cover MAXIMUM_COMMON_DELTA (8+16+32+64+128 = 248, which is >240) 2021-03-21 16:12:41 +00:00
catbref
d1419bdfbd Minor comments, adjust max step size when searching for common block 2021-03-21 15:57:00 +00:00
CalDescent
8566d9b7e5 Merge branch 'master' into synchronization-improvements 2021-03-21 15:04:43 +00:00
catbref
b319d6db6b Rework BlockMessage caching with new pseudo outgoing-only message that only caches raw bytes 2021-03-21 14:14:15 +00:00
CalDescent
35fd1d8455 Base58 encode signatures in recently added logs. 2021-03-21 14:12:04 +00:00
CalDescent
be21771e49 Use SYNC_BATCH_SIZE instead of MAXIMUM_BLOCK_SIGNATURES_PER_REQUEST. 2021-03-21 13:58:42 +00:00
catbref
745528a9b1 Peer.sendMessage() should return false when it can't send because it can't build the message 2021-03-21 13:19:59 +00:00
CalDescent
f1422af95b Added retry mechanisms in Synchronizer.syncToPeerChain()
Until now, we required a perfect success rate when syncing with a peer via Synchronizer.syncToPeerChain(). Blocks were requested individually, but the node would give up and lose all progress if a single request failed. In practice, this happened very regularly, and it was difficult to succeed when there were a large number of blocks (e.g. 20+) that needed to be requested.

This commit adds two retry mechanisms, causing each of the two request types (block sigs and blocks) to retry 3 times before giving up, potentially avoiding a lot of wasted work. The number of retries is configurable in the MAXIMUM_RETRIES constant, which we could move to settings at some point if this feature proves useful.

The original issue seemed to result in a few side effects:

1. Nodes would spend a large amount of time requesting blocks from peers, only to throw it all away afterwards. This potentially added to network congestion, as nodes were using unnecessary network time to unproductively serve peers.

2. A large number of sync attempts were failing, particularly when a fork emerged with a significant number of divergent blocks (20+). This issue reduced the ability for nodes to sync to the correct chain while they still had time to do so. With every block that passed, it became made it more and more difficult to switch to the correct chain. Eventually, the correct chain would become TOO_DIVERGENT at which point there is no way to automatically switch without manual intervention. I hope that this retry mechanism will increase the chances of nodes automatically moving onto the right chain quickly, avoiding the need for a user to intervene.

3. The POST /admin/forcesync API was unlikely to succeed when the peer's chain had started to diverge from the user's chain. This should increase the success rate.

Also included in this commit is a MAXIMUM_BLOCK_SIGNATURES_PER_REQUEST constant. This limits the number of block sigs requested in each batch (default 200). Without this, we are unable to increase MAXIMUM_COMMON_DELTA because it can try and request thousands of block sigs at once, which unsurprisingly doesn't succeed.
2021-03-21 09:41:36 +00:00
CalDescent
f92f4dc1e2 Fixed some log entries in Controller.syncToPeerChain() which were incorrectly reporting our height instead of the height of block(s) being requested from the peer. Now reporting the height of the block (or block sigs) being retrieved, which should make it easier to interpret the logs. 2021-03-20 16:18:25 +00:00
catbref
019cfdc1db Minor comment re-org 2021-03-20 11:45:11 +00:00
CalDescent
e694a51cdd Fix for "numberSignaturesRequired" calculation error in Synchronizer.syncToPeerChain()
This bug often prevented the correct amount of block signatures (and blocks) from being requested from a peer, when trying to sync to it.

It could result in quite serious consequences, as it would trigger orphaning back to the common block without first requesting all of the necessary blocks from the peer's chain. Rather than applying a complete copy of the peer's chain, it could orphan back to the common block and then only apply a few blocks beyond that, leaving the node in an unexpected state, potentially hundreds of blocks behind the peer's current height, which it then has to try and obtain from other peers.

When there are forks present, this could result in it hopping from chain to chain, each time being unable to fully synchronise with the peer. Given that we currently discard our chain if it is deemed that our latest block isn't "recent", it is very important that nodes are brought up to the latest block when synchronising with a peer, to avoid constantly triggering discards.

The severity of this bug increased when there was a large disparity between the peer's latest block and the common block height, and prevented us from being able to increase MAXIMUM_COMMON_DELTA.
2021-03-20 10:33:23 +00:00
CalDescent
16453ed602 Added unit tests for level 3+4, 5+6, 7+8, and 9+10 rewards.
These are simpler than the level 1+2 tests; they only test that the rewards are correct for each level post-shareBinFix. I don't think we need multiple instances of the pre-shareBinFix or block orphaning tests. There are a few subtle differences between each test, such as the online status of Bob, in order the make the tests slightly more comprehensive.
2021-03-17 08:50:53 +00:00
CalDescent
fde68dc598 Added unit test to test level 1 and 2 rewards.
1. Assign 3 minters (one founder, one level 1, one level 2)
2. Mint a block after the shareBinFix, ensuring that level 1 and 2 are being rewarded evenly from the same share bin.
3. Orphan the block and ensure the rewards are reversed.
4. Orphan two more blocks, each time checking that the balances are being reduced in accordance with the pre-shareBinFix mapping.
2021-03-16 09:11:49 +00:00
QuickMythril
22e3140ff0 add version on tooltip
add Version Number on Qortal Core tooltip.

https://i.imgur.com/eLnLnQ5.png
2021-03-16 03:00:55 -04:00
catbref
4824c4198b Bump version to 1.4.4 2021-03-15 11:00:20 +00:00
catbref
ec7d4f4498 Changed "too busy" logging from debug to trace 2021-03-13 18:30:43 +00:00
catbref
d635de44a8 Added TODO in HSQLDBRepository about deadlock log spam 2021-03-13 18:29:31 +00:00
catbref
bce66bf57f Move HSQLDBRepositoryFactory.POOL_SIZE into Settings as "repositoryConnectionPoolSize" 2021-03-13 18:14:11 +00:00
catbref
0fc5153f9b Merge 'trade-bot-timeout-fix' into master 2021-03-13 17:13:40 +00:00
catbref
0398c2fae1 Try to avoid clogging up network threads by discarding incoming TRANSACTION messages if we're too busy
As importing a transaction requires blockchain lock, all the network threads
can be used up blocking for that lock, especially if Synchronizer is active.

So we simply discard incoming TRANSACTION messages if we can't immediately
obtain the blockchain lock. Some other peer will probably attempt to
send the transaction soon again anyway.

Plus we swap transaction lists after connection handshake.
2021-03-13 17:03:38 +00:00
CalDescent
5fc495eb6a Fix for possible logic bug introduced in commit 33a8f31. 2021-03-12 22:05:38 +00:00
CalDescent
847e81e95c Fixed a mapping issue in Block->getShareBins(), to take effect at some future (undecided) height.
Post trigger, account levels will map correctly to share bins, subtracting 1 to account for the 0th element of the shareBinsByLevel array.
Pre-trigger, the legacy mapping will remain in effect.
2021-03-12 19:48:49 +00:00
CalDescent
7918622e2e Merge pull request #31 from sakumatto/master
Initial Italian translation by Pabs 2021
2021-03-11 11:06:03 +00:00
CalDescent
427fa1816d "blockCacheSize" can now be configured via settings.json. 2021-03-07 10:00:49 +00:00
catbref
0c7e388463 Bump to v1.4.3 2021-02-27 18:24:09 +00:00
catbref
be3af53011 Set new block sig go-live block height: block 320000 2021-02-27 18:23:49 +00:00
catbref
414399b2a0 Merge branch 'blocksig' into master 2021-02-27 18:20:13 +00:00
catbref
c592051a80 Speed up BlockMinter by filtering out 'unconfirmable' transaction types like CHAT & PRESENCE 2021-02-27 17:29:19 +00:00
catbref
33a8f311e5 Reduce logging noise from lost trade-bot ATs and self-clean if AT does not exist after 24 hours 2021-02-24 21:00:52 +00:00
catbref
018c3cdcd4 Allow users to delete trade-bot entries in any state if corresponding AT does not exist 2021-02-24 20:46:47 +00:00
sakumatto
384dffbf9a Initial Italian translation by Pabs 2021
UI localized to Italian by @Pabs
2021-02-22 20:03:11 +02:00
catbref
0306ecb03d AdvancedInstaller updates for v1.4.2 2021-02-21 17:26:32 +00:00
catbref
e5ce732557 More detail in AutoUpdates.md 2021-02-21 17:12:02 +00:00
catbref
f19e0498bf Bump to v1.4.2 2021-02-21 17:06:23 +00:00
CalDescent
32ec02225a Added optional "--testnet" or "-t" argument to stop.sh. When passing this argument, it will attempt to stop the core using the default testnet port (62391) rather than the default mainnet port (12391). 2021-02-21 11:50:02 +00:00
catbref
3920933fc7 Add block fetch TRACE logging to Synchronizer 2021-02-20 13:36:54 +00:00
catbref
1fdd7f156c Reduced logging noise from deleteExpiredTransactions but increased detection & logging on "serilization failures" from HSQLDB 2021-02-20 13:25:53 +00:00
catbref
91925cf931 Change block "minter" signature code, to take effect at some future (undecided) height.
Post trigger, this change will use all 128 bytes of previous block's signature when
calculating/validating next block's "minter" signature (itself the first 64 bytes of a block signature).

Prior to trigger, current behaviour is to only use first 64 bytes of previous block's
signature, which doesn't encompass transactions signature.

New block sig code should help reduce forking and help improve transactional
security.

Added "newBlockSigHeight" to blockchain.json but initially set to block 999999
pending decision on when to merge, auto-update, go-live, etc.
2021-02-20 12:08:51 +00:00
catbref
30e58f1c19 Merge branch 'hsqldb-checkpoint' into master 2021-02-20 10:56:22 +00:00
CalDescent
8d5c6db39f Exclude IntelliJ IDEA files from git. 2021-02-14 18:17:22 +00:00
catbref
3453f0efaf Rework HSQLDB CHECKPOINTing to defer until there are no ongoing SQL transactions, in order to prevent DB deadlocks.
Symptoms of a CHECKPOINT-related DB deadlock:

On Controller thread:
"Controller" #20 prio=5 os_prio=31 cpu=1577665.56ms elapsed=17666.97s allocated=475G defined_classes=412 tid=0x00007fe99f97b000 nid=0x1644b waiting on condition  [0x0000700009a21000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@14.0.2/Native Method)
	- parking to wait for  <0x0000000602f2a6f8> (a org.hsqldb.lib.CountUpDownLatch$Sync)
[...some more lines...]
[this next line is the best indicator: ]
	at org.qortal.repository.hsqldb.HSQLDBRepository.checkpoint(HSQLDBRepository.java:385)
	at org.qortal.repository.RepositoryManager.checkpoint(RepositoryManager.java:51)
	at org.qortal.controller.Controller.run(Controller.java:544)

Other threads stuck at:
	- parking to wait for  <0x00000007ff09f0b0> (a org.hsqldb.lib.CountUpDownLatch$Sync)
	at java.util.concurrent.locks.LockSupport.park(java.base@14.0.2/LockSupport.java:211)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@14.0.2/AbstractQueuedSynchronizer.java:714)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(java.base@14.0.2/AbstractQueuedSynchronizer.java:1046)
	at org.hsqldb.lib.CountUpDownLatch.await(Unknown Source)
	at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
2021-02-13 17:32:09 +00:00
catbref
eb23940996 Fix potential NPE when trying to obtain opportunistic database connection.
Could have affected:
Controller.deleteExpiredTransactions()
Network.getConnectablePeer()
Network.opportunisticMergePeers()
Network.prunePeers()

Symptoms:

2021-02-12 16:46:06 WARN  NetworkProcessor:152 - [1556] exception while trying to produce task
java.lang.NullPointerException: null
        at org.qortal.repository.hsqldb.HSQLDBRepository.<init>(HSQLDBRepository.java:92) ~[qortal.jar:1.4.1]
        at org.qortal.repository.hsqldb.HSQLDBRepositoryFactory.tryRepository(HSQLDBRepositoryFactory.java:97) ~[qortal.jar:1.4.1]
        at org.qortal.repository.RepositoryManager.tryRepository(RepositoryManager.java:33) ~[qortal.jar:1.4.1]
        at org.qortal.network.Network.getConnectablePeer(Network.java:525) ~[qortal.jar:1.4.1]
2021-02-13 11:24:26 +00:00
catbref
6cd86d86a6 Add "atFindNextTransactionFix" feature-trigger to all test blockchain configs 2021-02-13 11:24:26 +00:00
CalDescent
c3fa34f5b9 Fixed .gitignore to ensure that all .DS_Store files (autogenerated by macOS) are excluded. 2021-02-13 11:17:10 +00:00
CalDescent
0af0aaaa21 Merge pull request #29 from sakumatto/master
Initial translations of UI terms into Finnish
2021-02-13 11:12:24 +00:00
CalDescent
02100c502b Exit from stop.sh with an error if curl isn't installed. Based on code submitted by TRM13 in issue #28. 2021-02-12 16:13:01 +00:00
sakumatto
b55154cd3c Initial translations of UI terms into Finnish
Initial translations of UI terms into Finnish language fi_ by Saku Mättö
2021-02-12 17:02:31 +02:00
catbref
1e6e5e66da Fix trailing comma on blockchain.json! 2021-02-06 12:09:24 +00:00
catbref
9b0e88ca87 Only compare same number of blocks when comparing peer chains 2021-02-06 11:40:29 +00:00
catbref
3acc0babb7 More chain-weight tests 2021-02-06 11:19:39 +00:00
catbref
dc6eda1355 Added tool to help build release notes 2021-02-06 11:02:36 +00:00
catbref
6224bc3bca Updated AdvancedInstaller config file to latest Qortal v1.4.1 2021-02-06 11:00:32 +00:00
catbref
9ceac8c991 Documentation updates 2021-02-06 10:59:31 +00:00
catbref
834fcd80d7 Bump version to v1.4.1 2021-01-18 08:17:09 +00:00
catbref
20e4a79130 Reduce logging level for deleting older PRESENCE transactions 2021-01-17 16:49:47 +00:00
catbref
d336200d75 Fix for off-by-one bug when ATs look for next transaction. Currently configured to take effect block 275,000 2021-01-17 16:01:36 +00:00
catbref
e5bb3e2f0a Added defensive try-catch around network engine calls (actually ExecuteProduceConsume) 2021-01-17 15:34:50 +00:00
catbref
5b2b2bab46 Two-pronged fix for HSQLDB 'serialization failure' errors when receiving multiple PRESENCE transactions -- reported by marracc 2021-01-17 15:33:59 +00:00
catbref
c17eea3ed9 Added timeout to Peer sendMessage() - same timeout as for awaiting incoming responses 2021-01-17 15:32:39 +00:00
catbref
83f4e2f5bf Added API call to view single trade's detailed info 2021-01-17 15:31:44 +00:00
catbref
c8e7a00c08 Remove unused JDBC statement 2021-01-16 13:20:58 +00:00
catbref
190014cf96 Fix minor NPE during shutdown 2021-01-16 13:20:20 +00:00
catbref
385064e324 Return foreign-chain wallet transactions in newest-timestamp-first order 2021-01-08 17:37:12 +00:00
catbref
f3e1f088f8 Bump to v1.4.0 2021-01-07 07:49:13 +00:00
catbref
6eb9447bb9 Merge branch 'LTCv3-with-presence' into master 2021-01-07 07:45:47 +00:00
catbref
0ee8d7da0f Improve removal of expired PRESENCE txns in websocket cache 2020-12-31 11:26:13 +00:00
catbref
918a331609 Transaction.importAsUnconfirmed() now BLOCKS for blockchain lock, instead of non-blocking try()
This is a serious change as it affects many callers.

Controller.onNetworkTransactionMessager()
-- should be OK to block as it's only one EPC thread

TransactionsResource.processTransaction()
-- should be OK to block as it's only one Jetty thread
AND has its own 30s timeout wrapper anyway

Implementations of AcctTradeBot.progress()
e.g. BitcoinACCTv1TradeBot.progress() & LitecoinACCTv1TradeBot.progress()
TradeBot.updatePresence()
-- these are called via BlockMinter/Synchronizer
when blockchain lock is already held, so will are unaffected

AcctTradeBot.createTrade() and AcctTradeBot.startResponse()
-- these are called via API
and previously would only perform non-blocking blockchainLock.try()
but now perform blocking blockchainLock.lock()
thus preventing NO_BLOCKCHAIN_LOCK when creating/responding to trades
especially after a (wasted) MESSAGE PoW compute
but with potential downside that API response might be delayed
e.g. by a very slow sync round

Future work could look into removing the need for blockchain lock
when calling Transaction.importAsUnconfirmed().
2020-12-30 12:43:41 +00:00
catbref
9bc395d36f Bump to v1.3.9 2020-12-29 17:59:02 +00:00
catbref
41453f5bd1 Add primary key index to latest AT state cache for extra speed! 2020-12-29 17:57:20 +00:00
catbref
78f62751e5 TESTNET ONLY: Correct "LitcoinACCTv1" TradeBotStates.acct_name values in DB 2020-12-29 14:16:47 +00:00
catbref
d59c30757c Fix conversion of double in ElectrumX JSON to long 2020-12-29 12:17:55 +00:00
catbref
30d2e4fdac Correct logging message to blockchain-agnostic text 2020-12-29 12:16:40 +00:00
catbref
d43a074cc1 Use *ACCT*.class.getSimpleName() for less error-prone ACCT.NAME 2020-12-29 11:00:43 +00:00
catbref
27783dc6de Add WARNING on start-up if repository is missing latest AT state data 2020-12-29 10:34:25 +00:00
catbref
2a789a9a9b Add WARNING on start-up if repository is missing latest AT state data 2020-12-29 10:15:56 +00:00
catbref
a66dba767e Fix incorrect LatestATStates SQL table definition
This table was previously defined using the TEMPORARY keyword as the rows were used as a cached/index to speed up AT trimming SQL statements.

However, the table definition was lacking the "ON COMMIT PRESERVE ROWS" clause, and possibly the "GLOBAL" keyword, which caused the table contents to be emptied, immediately after being filled, when <tt>repository.saveChanges()</tt> was called (i.e. "COMMIT").

This caused latest AT states to be trimmed in error.

AtRepositoryTests.testGetLatestATStatePostTrimming also fixed so that it fails with the previous, broken code.
2020-12-29 09:19:03 +00:00
catbref
e00579e1a2 Fix incorrect LatestATStates SQL table definition
This table was previously defined using the TEMPORARY keyword as the rows were used as a cached/index to speed up AT trimming SQL statements.

However, the table definition was lacking the "ON COMMIT PRESERVE ROWS" clause, and possibly the "GLOBAL" keyword, which caused the table contents to be emptied, immediately after being filled, when <tt>repository.saveChanges()</tt> was called (i.e. "COMMIT").

This caused latest AT states to be trimmed in error.

AtRepositoryTests.testGetLatestATStatePostTrimming also fixed so that it fails with the previous, broken code.
2020-12-29 09:11:11 +00:00
catbref
99f1a55de2 Improve logging for case where trade offer is locked to someone else 2020-12-29 08:56:26 +00:00
catbref
3ec307a2a1 Don't allow more than one (active) trade-bot entry per trade-offer 2020-12-28 15:55:14 +00:00
catbref
3fdef9ea6d Fix P2SH refund "non-final" error issue
According to Bitcoin source, CheckFinalTx() in validation.cpp ~line 223,
we need to make sure median blocktime has passed P2SH refund transaction's
nLockTime.

Previously we were erroneously checking that median blocktime was in the past.

This should fix issues where refunding P2SH results in a "non-final" error
from the ElectrumX server network.
2020-12-28 15:44:45 +00:00
catbref
332c917c94 Fix ALICE-based PRESENCE transactions.
PRESENCE transactions were previously validated using Bob's trade key (in address form).
But as PRESENCE transactions are already emitted by Alice, her trade key is also used
(if present in trade data by virtue of AT being locked to Alice).

Similarly, Alice's trade-bot won't even try to build PRESENCE transactions if her
trade key isn't publicly visible to other peers, i.e. after AT is locked to Alice.
2020-12-28 12:00:11 +00:00
catbref
35b0ac78b8 Bump ElectrumX transaction cache size from 100 to 200 2020-12-24 16:52:58 +00:00
catbref
047627a6e5 Force bitcoinj keychain lookaheadThreshold to zero so we always generate more keys 2020-12-24 16:48:47 +00:00
catbref
e4e775a107 Bump to v1.3.8 2020-12-24 12:09:30 +00:00
catbref
5d6811bd50 Workaround for block 212937 issue 2020-12-23 15:02:14 +00:00
catbref
688f215dfd Allow PresenceType filtering on presence websocket via presenceType query param 2020-12-21 12:25:58 +00:00
catbref
7cbdbbcc8d Allow exception-free conversion from String to PresenceType 2020-12-21 12:25:11 +00:00
catbref
4e89b8fbac No need to create a map entry for null foreignBlockchain in TradeOffersWebSocket 2020-12-21 12:24:44 +00:00
catbref
70ec8cb11f Add public key in Qortal address form to tradeoffers and presence websockets
This aids matching PRESENCE to corresponding trade offers for use in UI.

Also tighten up visibility of some fields in ChainChainOfferSummary and
PresenceInfo to private.

PresenceInfo.address should map to CrossChainOfferSummary.qortalCreatorTradeAddress
which is "AT creator's ephemeral trading key-pair represented as Qortal address"
2020-12-18 11:42:53 +00:00
catbref
0f0266609f Add trading price estimate API call GET /crosschain/price/{blockchain} where blockchain is something like LITECOIN 2020-12-18 11:42:32 +00:00
catbref
ecfa6e994e Add API call POST /admin/repository/backup to trigger immediate backup 2020-12-16 12:52:19 +00:00
catbref
ed4a45f214 Force blocking DB backup to improve integrity of backup files 2020-12-16 12:51:30 +00:00
catbref
e953be6e4a In pom.xml, have Maven surefire plugin skip tests by default 2020-12-14 15:25:53 +00:00
catbref
bd51806a0d Improve Block.getBytesForMinterSignature() 2020-12-14 15:05:31 +00:00
catbref
625dbfbbd7 Unify BlockInfo into BlockSummaryData, removing minterAddress (unused) and extra repository calls 2020-12-14 13:13:44 +00:00
catbref
1c6ea0a860 Improvements to ElectrumX, Bitcoin-y aspects, etc.
Add caching of transactions fetched via ElectrumX to reduce network load and speed up API response.

Fix handling ElectrumX servers that don't want to supply verbose transaction JSON.

Hide lots of data in BitcoinyTransaction that isn't needed by current API users.
2020-12-11 17:30:15 +00:00
catbref
3706cd5ff7 Return list, not set, of wallet transactions via API and provide better (usable) examples 2020-12-11 17:29:00 +00:00
catbref
934cd1d511 Add support for preferred blockchain to /websockets/crosschain/tradebot via foreignBlockchain query param 2020-12-10 17:08:32 +00:00
catbref
68e3d3b989 Add preferred-blockchain filtering to /websockets/crosschain/tradeoffers via foreignBlockchain query param 2020-12-10 16:00:01 +00:00
catbref
31fa916156 Add filtering by foreign blockchain to API crosschain calls 2020-12-10 14:52:06 +00:00
catbref
456bb3ca63 Include output addresses, if present, in BitcoinyTransaction 2020-12-10 14:01:40 +00:00
catbref
b07ad094c1 Fix DELETE /crosschain/tradebot by adding missing repository.saveChanges() 2020-12-10 13:11:48 +00:00
catbref
d766cfaa67 Fix API/websockets that were still BitcoinACCTv1-only 2020-12-10 12:48:46 +00:00
catbref
acc616c204 Add defensive code to EventBus to catch unexpected exceptions 2020-12-10 11:55:06 +00:00
catbref
8707f154ee Add support to ElectrumX for barring servers that don't give us the data we need 2020-12-10 11:32:09 +00:00
catbref
992427f0e0 Fix NPE due to unboxing null/no entry from PREVIOUS_STATES.get() in TradeBotWebSocket. Usually triggered when creating new trade-bot entry while having open websocket connection. 2020-12-10 11:31:37 +00:00
catbref
2c84add935 Bitcoiny improvements 2020-12-09 13:12:15 +00:00
catbref
e8fc91fd34 Minor work on ByteArray and associated tests 2020-12-08 15:19:12 +00:00
catbref
8c9cf4a02d Add API support for listing Bitcoin/Litecoin wallet transactions 2020-12-07 16:40:12 +00:00
catbref
23f0969b2d Requesting BTC/LTC wallet balance now accepts public key xpub/tpub too 2020-12-07 15:40:05 +00:00
catbref
cf82813280 Report foreignBlockchain and acctName in results for API call GET /crosschain/tradeoffers 2020-12-02 14:37:50 +00:00
catbref
753fa4dfa9 Remove extraneous boxing/unboxing from PresenceWebSocket 2020-12-02 13:20:37 +00:00
catbref
58ff338ab3 Now Transaction.importAsConfirmed() calls Controller.onNewTransaction(), removing call from API POST /transactions/process and Controller.onNetworkTransactionMessage() 2020-12-02 13:19:53 +00:00
catbref
064e12a57b Improve documentation regarding deadlock in EventBus 2020-12-02 13:16:18 +00:00
catbref
54bb8ed817 New PRESENCE-related API websocket at /websockets/presence 2020-12-01 17:24:02 +00:00
catbref
b651eae258 Notify EventBus with NewTransactionEvent in Controller.onNewTransaction 2020-12-01 17:23:10 +00:00
catbref
7562d9bbf8 Fix potential NPE when closing a websocket that failed to open properly 2020-12-01 17:22:40 +00:00
catbref
1b50dd5adf Modify API TransactionsResource to notify Controller after closing repository handle/session 2020-12-01 17:22:03 +00:00
catbref
c10a5db280 Fix PresenceTransaction to work with any ACCT, not only BitcoinACCTv1 2020-12-01 15:06:07 +00:00
catbref
500690be49 Add ATRepository.getAllATsByFunctionality() to fetch ATs matching selection of code hashes 2020-12-01 15:05:40 +00:00
catbref
778ac35ee6 Improve HSQLDBRepository.temporaryValuesTableSql to work with Collection, not just List 2020-12-01 15:04:50 +00:00
catbref
c16a664a78 Add toString() to ByteArray 2020-12-01 15:03:23 +00:00
catbref
75a265f89a Improve SupportedBlockchain enum 2020-12-01 15:02:58 +00:00
catbref
ddb55210b4 Merge branch 'presence-txn' into LTCv3-with-presence 2020-11-30 15:10:03 +00:00
catbref
e093520696 WIP: PRESENCE - TradeBot support, moved PresenceType enum, added Presence-only transaction deadline override 2020-11-27 18:06:24 +00:00
catbref
cfacddcb36 Change Controller.deleteExpiredTransactions to use Transaction-subclass .getDeadline() instance method call instead of static Transaction.getDeadline(transactionData) which allows Transaction subclasses to override with custom deadlines/expiry periods 2020-11-27 18:05:03 +00:00
catbref
d2dea3ff35 BlockMinter now notifies Controller of new block while still holding blockchain lock, bringing it in line with Synchronizer 2020-11-27 18:02:05 +00:00
catbref
865fcb95bf In pom.xml, have Maven surefire plugin skip tests by default 2020-11-27 14:26:01 +00:00
catbref
a52c089728 WIP: PRESENCE transactions - support only TRADE_BOT type and restrict to known trades 2020-11-26 16:33:21 +00:00
catbref
15d25649b2 WIP: PRESENCE transactions - repository support, removing older versions, tests 2020-11-26 16:11:11 +00:00
catbref
e0210635e3 Add repository support for fetching subset of unconfirmed transactions by type and/or creator 2020-11-26 16:10:13 +00:00
catbref
90b993e234 Refactor post-importAsUnconfirmed as method to be overridden by Transaction subclasses, e.g. CHAT 2020-11-26 16:08:56 +00:00
catbref
9b7c2c50fb Initial commit with PRESENCE transaction type -- untested 2020-11-24 17:02:05 +00:00
catbref
fc7a7a1549 Bump Maven surefire plugin to v2.22.2 for better test/CI support 2020-11-24 15:11:39 +00:00
catbref
a12045c19e Fix repository race condition from using wrong synchronization object
Previous fixes for "transaction rollback: serialization failure" when updating trim heights
in commits 16397852 and 58ed7205 had the right idea but were broken due to being synchronized
on different objects.

this.repository.trimHeightsLock would be a new Object() for each repository connection/session
and so not actually synchronize concurrent updates.

Implicit saveChanges()/COMMIT is still needed.

Fix is to use a repository-wide object for synchronization - in this case the repositoryFactory
object as held by RepositoryManager.

Added test to cover.

Also reduced DB trim height read to one call at start of thread for both trimming threads.
2020-11-17 14:53:39 +00:00
catbref
fccb3a3f0c Merge branch 'master' into LTCv3-merge-test 2020-11-13 10:46:45 +00:00
catbref
62ae49b639 ApiError_de.properties & SysTray_zh.properties also converted to UTF8 2020-11-12 12:18:26 +00:00
catbref
2e8f58bb2f ApiError_ru.properties & TransactionValidity_ru.properties converted to UTF8 for easier management 2020-11-12 11:34:25 +00:00
catbref
9e98ce220f Revert SysTray_ru.properties back to UTF8 form 2020-11-12 11:28:01 +00:00
catbref
10c3a0c056 Update AdvancedInstaller config file with v1.3.7 values 2020-11-12 11:27:36 +00:00
catbref
69ec654e4a Bump to version 1.3.7 2020-11-11 09:27:25 +00:00
catbref
a310e751bb Fix slow SQL query in HSQLDBATRepository.getBlockATStatesAtHeight() - mostly used during orphaning 2020-11-10 16:58:24 +00:00
catbref
3ef8b81e51 Fix incorrect column indexes when fetching frozen AT data 2020-11-10 15:38:15 +00:00
catbref
1f409235e4 Don't rebuild repository or export node-local data during repository build if repository was 'pristine'.
Under certain conditions, e.g. non-existent database files, the repository would be created
and then immediately be re-created.

Not only was this unnecessary, but HSQLDBDatabaseUpdates would attempt to export the node-local
data twice, which would cause an error due to existing .script files.

The fix is three-pronged:

1. Don't immediately rebuild the repository if it's only just been built
2. Don't export the empty node-local data if repository has only just been built
3. Don't export the node-local data if it's empty
2020-11-09 10:31:21 +00:00
catbref
806baa6ae4 Fix API call referenced in DB reshape 2020-11-06 11:23:35 +00:00
catbref
58ed72058f Another attempt to prevent "serialization failure" during trimming 2020-11-05 14:36:14 +00:00
catbref
253a994438 Add API POST /repository/checkpoint call. Renamed GET/POST /admin/repository calls to /admin/repository/data 2020-11-05 11:08:54 +00:00
catbref
5549eded38 Improve/fix use of latest block cache, for more cache hits, faster chain-tip response, etc. 2020-11-05 09:34:57 +00:00
catbref
20777363cf Split AT state storage into two HSQLDB table for better management
This involves a database reshape, but before this happens the node-local
data is exported to local files, giving the user the option to use a
bootstrap file instead of waiting.
2020-11-04 20:07:30 +00:00
catbref
b3f859f290 Don't use WITH COLUMN NAMES when exporting data from repository into local file 2020-11-04 15:48:52 +00:00
catbref
8c9f68a9c3 Add API calls for exporting node-local repository data & corresponding import to/from local files 2020-11-04 15:35:42 +00:00
catbref
41f178bf59 Add support for API key security, where X-API-KEY header must match apiKey from settings
apiKey in settings is null by default at this point, for backwards compatibility.
In the future, the Windows installer could generate a UUID for apiKey.
apiKey in settings needs to be at least 8 characters.

API calls in the documentation engine are now marked with an open/closed padlock
to show where API key might be required.
Add support for API key security, where X-API-KEY header must match apiKey from settings

apiKey in settings is null by default at this point, for backwards compatibility.
In the future, the Windows installer could generate a UUID for apiKey.
apiKey in settings needs to be at least 8 characters.

API calls in the documentation engine are now marked with an open/closed padlock
to show where API key might be required.
2020-11-04 15:35:20 +00:00
catbref
ad5050f92e Add support for exporting node-local repository data to .script files and corresponding import function 2020-11-04 15:29:10 +00:00
catbref
16397852ae Add synchronization around updating trim heights to prevent deadlock/rollback 2020-11-04 10:01:20 +00:00
catbref
c125a53655 More (optionally) logging when comparing chains with peers. Support for potential future minor consensus change. 2020-11-02 11:52:06 +00:00
catbref
7b056a832f Turn off HSQLDB redo-log "blockchain.log" and periodically call "CHECKPOINT" instead.
Checkpointing interval is 1 hour by default, changable in settings via
"repositoryCheckpointInterval"
plus corresponding "showCheckpointNotifications" SysTray flags (off by default).

Added entries to SysTray_en i18n properties, and converted SysTray_ru to ISO-8559-1.
2020-11-02 11:49:21 +00:00
catbref
6c40727027 More reporting for slow HSQLDB queries/commits 2020-11-02 11:16:40 +00:00
catbref
8f06765caf Networking performance improvements and message sending bugfix 2020-11-02 11:15:53 +00:00
catbref
de2fc78ad1 Add support for SHA256 digest of ByteBuffer in Crypto class 2020-11-02 10:47:27 +00:00
catbref
ee08410260 More trace-level debugging in Synchronizer to help diagnose chain reorg issues 2020-11-02 10:46:51 +00:00
catbref
88da8d949f Don't allow latest blocks cache to be empty 2020-11-02 10:45:21 +00:00
catbref
d2a92db921 More caching for GetBlockMessage. Added API call GET /admin/enginestats to monitor cache usage 2020-10-29 11:02:02 +00:00
catbref
9c18a33d7f Improve tools/build-auto-update.sh when working on detached HEAD 2020-10-28 09:08:16 +00:00
catbref
f3b8258067 Add more latest block caching to reduce repository accesses, especially for requests from remote peers 2020-10-28 08:46:30 +00:00
catbref
da78c73485 Remove extraneous call to Controller.onNewBlock() after synchronization, as this call is performed per-block inside Synchronizer 2020-10-28 08:42:59 +00:00
catbref
cec25ce279 Add API call POST /peers/commonblock <connected-peer> as debugging aid 2020-10-28 08:41:23 +00:00
catbref
0389007491 Skip trimming while performing synchronization 2020-10-28 08:38:11 +00:00
catbref
38a64bdd9e Prevent HSQLDB prepared statement cache invalidation when rebuilding latest AT states cache 2020-10-12 14:35:10 +01:00
catbref
6a24f787c4 Bump to v1.3.6 2020-10-07 14:56:58 +01:00
catbref
98564aa8bf Fix SQL logic error when fetching trade offers 2020-10-07 14:56:05 +01:00
catbref
9ceff90f42 Upgrade to CIYAM-AT v1.3.8 with slight performance improvements 2020-10-07 10:31:18 +01:00
catbref
6a4388fecc Use cached PreparedStatement for HSQLDB.assertEmptyTransaction + other minor HSQLDB fixes 2020-10-07 09:45:44 +01:00
catbref
1958444bc4 Add recipient indexes for payment/AT transactions to speed up AT processing 2020-10-06 14:09:42 +01:00
catbref
a2038274e1 Keep latest AT state, even if "finished", so we can produce historical trade data 2020-10-05 15:18:41 +01:00
catbref
532c697026 Moved AT State & online signatures trimming intervals, batch sizes, limits, etc. to Settings 2020-10-02 12:58:23 +01:00
catbref
5cf5c1e1f7 Take pressure off GC by not creating/destroying HSQLDB sub-repositories all the time 2020-10-01 13:18:24 +01:00
catbref
60621e8b81 Reworked AT-states and online signatures trimming
Instead of searching from block 0, we now keep a record of
base trim height in the DB itself.

Also, we no longer trim the latest AT state for non-finished ATs
in case they are in deep sleeping and we need their state for when
they awaken.
2020-10-01 13:17:00 +01:00
catbref
a6a1f65d3e Reduce block search size in AT state trimmer to reduce load 2020-09-29 11:41:30 +01:00
catbref
a681f741dd Add initial delay before trimming online accounts signatures 2020-09-29 11:40:41 +01:00
catbref
bed9837967 Added settings entry "localeLang" for controlling core language (not-API) 2020-09-29 10:56:27 +01:00
catbref
855cb2226a Aggressively trim old AT state data and online accounts signatures.
Two new classes/threads made to quickly find first trimmable row
then repeatedly trim rows in small batches after that.
2020-09-28 14:34:00 +01:00
catbref
d85a3d17c8 Fix for HSQLDB deadlock during CHECKPOINT.
Symptoms are:

* db/blockchain.log is pretty much exactly 50MB - the checkpoint-triggering size.

* Loads of threads are stuck waiting for HSQLDB's CountUpDownLatch$Sync.await()

* Synchronizer, or some other thread, possibly orphaning blocks.

The cause seems to be method A, which has a repository session,
calls EventBus.INSTANCE.notify() and one of the event listeners
then obtains their own repository session to do repository 'work'.

In the meantime, the HSQLDB log has reached 50MB, triggering auto-checkpoint.

HSQLDB attempts to CHECKPOINT, but waits for existing transactions
to complete, and also blocks starting new transactions.

Thus, one of the event listeners is blocked when they try to obtain
a new repository session, but HSQLDB never performs CHECKPOINT
because the event notifier (method A) still has an unfinished
transaction - hence deadlock.
2020-09-28 14:22:18 +01:00
catbref
81a5b154c2 Add API call DELETE /admin/repository which actually performs repository maintenance (takes several minutes) 2020-09-25 17:06:06 +01:00
catbref
a6f42df9d6 Add isTestNet to API call GET /admin/info 2020-09-25 16:35:53 +01:00
catbref
17ae7acc6d Reduce DB storage of AT states
Drop created_when column from ATStates as it never changes
and can be fetched from ATs table.
This takes about 50s on a fast machine.

Correspondingly rebuild height-based index on ATStates.
This takes about 3 minutes on a fast machine.

Modify AT-related repository methods and callers.

Aggressively remove 'old' (> 2 weeks) actual AT
state binary data, leaving only the hash in DB
(for syncing purposes). Seems to keep up with
syncing from another node on localhost.
2020-09-25 15:25:57 +01:00
catbref
3d5fec3c30 Bump to HSQLDB v2.5.1 as we seem clear of OOM issue 2020-09-25 15:25:15 +01:00
catbref
21f48fba5f HSQLDB PreparedStatement caching improvements 2020-09-24 12:46:30 +01:00
catbref
d0da5d7c48 ATs: only call MachineState.getCodeBytes() once in preparation for using newer AT lib 2020-09-24 12:46:17 +01:00
catbref
4209cc6ee4 Improve SQL prepared statement caching in HSQLDBATRepository, plus missing space in SQL in getMatchingFinalATStates 2020-09-23 09:41:24 +01:00
catbref
f3e1092dd5 Improve SQL prepared statement caching in HSQLDBBlockRepository.getBlockInfos & test to cover 2020-09-23 09:40:41 +01:00
catbref
43055b666f Improve SQL prepared statement caching in HSQLDBAccountRepository.getEligibleLegacyQoraHolders() 2020-09-23 09:38:08 +01:00
catbref
7cd8ed6e23 HTLC secret and status caching to reduce repeated calls to ElectrumX network
Additional benefit is to speed up syncing if node has trade-bot entries,
as a batch of blocks processed within 30s will have the same HTLC responses
as neither Bitcoin nor Litecoin blocks are that fast. Once synced, the next
Qortal block (~60s) should pick up any new changes. Generally users will be
synced when using trade-bot anyway.

Also moved bitcoiny.getAddressTransactions(p2shAddress)
into BitcoinyHTLC.findHtlcSecret() so only called if secret,
or lack thereof, is not cached.

Added tests to cover caching.
2020-09-22 12:13:22 +01:00
catbref
4bc0edeeca Litecoin trade-bot added!
Several cross-chain API calls moved into separate classes,
although most of the URLs remain roughly the same to provide
backwards compatibility.

API /crosschain/at/build moved into /crosschain/BitcoinACCTv1

Converted DELETE /crosschain/tradeoffer to be ACCT-agnostic.
Changes to ACCT interface, etc. to support above.

Changes applied to other crosschain API calls to make them
independent of Bitcoin/Litecoin.

Corrections to fee calculations and usage in BitcoinACCTv1.

Added new LitecoinACCTv1 trade-bot, using LitecoinACCTv1.

Some minor typo corrections, rename of secretHash to hashOfSecret.

Some more Bitcoin-specific fields deprecated, but values duplicated
from newly-named fields for now.

Lower default fee (10sats/byte) for Litecoin spending transactions.
(Not P2SH fees which are 1000sats).

Changed ApiError INSUFFICIENT_BALANCE HTTP status from 422 to 402
as 422 isn't supported by Jetty?

CrossChainTradeSummary.btcAmount deprecated, use: foreignAmount

Modified pom.xml to generated package-info.java files for classes
inside org.qortal.api.model.** subdirectories.
2020-09-21 16:54:43 +01:00
catbref
7a06df6ccd Improve SQL prepared statement caching in HSQLDBATRepository, plus missing space in SQL in getMatchingFinalATStates 2020-09-21 16:42:43 +01:00
catbref
d9164a32e5 Improve SQL prepared statement caching in HSQLDBBlockRepository.getBlockInfos & test to cover 2020-09-21 16:38:45 +01:00
catbref
a8fbf32a88 Improve SQL prepared statement caching in HSQLDBAccountRepository.getEligibleLegacyQoraHolders() 2020-09-21 16:29:34 +01:00
catbref
514689d2f4 WIP: refactoring to support multiple foreign blockchains
API support for Litecoin wallet balance and sending LTC.

TradeBotCreateRequest rejigged to use blockchain-agnostic
field names, e.g. bitcoinAmount now foreignAmount,
and added foreignBlockchain field.

The massive API CrossChainResource class has been split into:

CrossChainAtResource: for building TRADE/REDEEM/CANCEL messages
(OFFER missing?)

CrossChainBitcoinResource: for Bitcoin wallet balance/spend
CrossChainLitecoinResource: ditto for Litecoin

CrossChainHtlcResource: for Bitcoiny-HTLC actions like:
deriving P2SH address
checking HTLC status
eventually: building refund/redeem transactions

CrossChainResource: for creating/cancelling/listing trade offers.

CrossChainTradeBotResource: for creating/cancelling trade-bot
entries, including responding to trade offers.

---

Other general trading changes:

TradeBot states are now specific to each individual trade-bot,
e.g. BitcoinACCTv1TradeBot or LitecoinACCTv1TradeBot, etc.

TradeBot states now a combination of int & String, instead of
enums due to above.

Extra columns added to DB TradeBotStates to store
blockchain, which ACCT in use, etc.

---

UNTESTED at this point!
2020-09-18 17:07:49 +01:00
catbref
76a15bb026 Initial implementation of Litecoin ACCT
Extracted AcctMode from BitcoinACCTv1.Mode as the values are
common to both Bitcoin/Litecoin ACCTs.

Added test apps for deploy, cancel, trade and redeem of LitecoinACCTv1.
2020-09-17 11:07:07 +01:00
catbref
b061f188f9 Update Bitcoinv1 DeployAT test app 2020-09-16 11:11:18 +01:00
catbref
af7d7d0966 Unified Bitcoin/Litecoin test apps 2020-09-16 10:57:45 +01:00
catbref
2ffd0770c6 Initial Litecoin support
Added altcoinj library as Maven dependency.

Added new Litecoin subclass of Bitcoiny,
with mainnet and testnet ElectrumX server lists.

Added litecoinNet settings variable and getter.

Added LitecoinTests.
Most tests work but testFindHtclSecret()
needs a redeemed HTLC on chain (not yet done).

Added litecoinNet to some test settings files
in resources.

Added Litecoin BuildHTLC, Refund test apps.

Added SendLTC app as Electrum-LTC seems a bit flaky?

So far managed to build HTLC P2SH, fund it and then
refund it!

---

As Bitcoin and Litecoin are both subclasses of Bitcoiny,
could unify some test apps with added Bitcoin/Litecoin
switch as first arg?
2020-09-15 17:53:54 +01:00
catbref
e3abeafc6b Refactoring to allow many foreign blockchains and multiple ACCTs
Bitcoin/Litecoin common aspects extracted in a "Bitcoiny" common class.
So:

Bitcoin (was BTC) extends Bitcoiny
Litecoin (future code) will also extend Bitcoiny

ElectrumX is now a BitcoinyBlockchainProvider
to allow easier future replacement and also tidier integration.

BTCP2SH is now BitcoinyHTLC as they are generic hash time-locked contracts,
probably Bitcoin/Litecoin agnostic.

BTCACCT is now BitcoinACCTv1, allowing for v2+ and also LitecoinACCTv1, etc.

BitcoinTransaction is now BitcoinyTransaction
as they are pretty much the same in Litecoin.

BitcoinException is now a more generic ForeignBlockchainException.

---

Bitcoiny subclasses instantiate a new BitcoinyBlockchainProvider
when creating their singleton instance. They pass relevant network
details to the BBP, like server lists, genesis block hash, etc.

Bitcoiny.WalletAwareUTXOProvider now only has the one key search mode
that is equivalent to the old REQUEST_MORE_IF_ANY_SPENT.

Tests tidied up.

---

Still to do:

Modifying TradeBot to handle multiple types of ACCTs,
like BitcoinACCTv2, LitecoinACCTv1...

Modifying API to support multiple types of ACCTs.

Actually add Litecoin support.

Build new ACCT without needing P2SH-B if possible.
2020-09-15 14:56:43 +01:00
catbref
1720582f33 Remove obsolete NextBlockHeight table and corresponding triggers
Also fix typo in Block.online_accounts varbinary size.
2020-09-11 16:06:53 +01:00
catbref
d93e9d570f Trimming old block online accounts signatures
There's still an existing issue where log entries like this appear:

  Unable to trim old online accounts signatures in repository

which is actually caused by:

  integrity constraint violation: unique constraint or index violation; SYS_PK_10092 table: BLOCKS

which seems to be a bug in the version of HSQLDB we use.
(Tested using synced-from-scratch DB).
It's not clear what the actual problem is at this point.

It might be possible to switch to v2.5.1 if our recent HSQLDB-related
commits have fixed/worked-around the OOM issues.

Move the inner method from BlockChain to Controller.
Remove blockchain lock as it's not needed because it's not an
HSQLDB "serialization failure" but constraint violation.

Trimming old online accounts signatures limited to batches of 1440
rows to reduce CPU and memory load.
2020-09-11 15:57:12 +01:00
catbref
5ea90f2fdd Speed up fetching transactions using block signature 2020-09-11 15:56:19 +01:00
catbref
c628f97d8c Speed up fetching block height based on timestamp 2020-09-11 15:55:54 +01:00
catbref
8a1e2f4111 Reduce HSQLDBRepository log noise by omitting idle session info 2020-09-11 15:54:32 +01:00
catbref
41f244d549 Add bitcoinj Context propagation 2020-09-11 15:52:59 +01:00
catbref
79641efa87 Tighten up trade-bot, ElectrumX
Added separate method to determine status of P2SH transactions,
returning UNFUNDED, FUNDING_IN_PROGRESS, REDEEMED, etc.

Added code to trade-bot to increase robustness. Lots more
changes including unified state change/logging, checking
for existing MESSAGEs, etc.

Added missing websocket methods to silence log noise.

Trade-bot now called per block during synchronization,
instead of per batch, to pick up edge cases where some
potential trade-bot transitions were missed, resulting
in failed trades.

Corresponding changes in Controller, such as notifying
event bus of new block in same thread (thus blocking)
instead of using executor.

Added slightly more robust common block determination
to Synchronizer.

Refactored code in BTC class to use new BitcoinException
rather than simply returning null, with added sub-classes
allowing differentiation between network issues or fund
issues.

Changed BTC.buildSpend to try harder to find UXTOs to
address false "insufficient funds" issues.

Repository change to add index on MessageTransactions
for quicker look-up of trade-related messages.

Reduced reliance on bitcoinj library in BTCP2SH.

Reworked ElectrumX to better detect errors rather than
continuously try more servers to no avail.
Also added genesis block check in case of servers on
different Bitcoin networks.
Now tries to extract upstream bitcoind error codes
and pass those up to caller via exceptions.
Updated list of testnet servers.

MemoryPoW now detects thread interrupt and exits fast.

Moved some non-generic transaction-related repository
methods to their own subclass. For example:
moved TransactionRepository.getMessagesByRecipient
to MessageRepository.getMessagesByParticipants

Updated and added more tests.
2020-09-10 12:03:37 +01:00
catbref
ca3fcc3c67 Tighten up sync status reporting, especially when using forcesync 2020-09-03 12:57:29 +01:00
catbref
de8e5ec920 Updated AdvancedInstaller project file based on v1.3.5 release 2020-09-01 11:08:37 +01:00
catbref
f833e44bd5 Update tools/build-zip.sh to reflect updated start.sh, and also to take optional git tag 2020-09-01 10:41:52 +01:00
catbref
8b0b1db5a4 Improved start-up shell script "start.sh"
Was "run.sh" but renamed to "start.sh" to better complement "stop.sh".
"run.sh" is now a symbolic link to "start.sh"

Reworked Java version check to remove dependency on "bc" tool which
seems not to be installed on some Ubuntu distributions?

Removed -XX:NativeMemoryTracking flag from JVM args.

Fixed incorrect comment regarding java.net.preferIpV4Stack.
Fixed typo in comment.
2020-09-01 10:25:48 +01:00
catbref
5b95f3af02 Bump to v1.3.5 2020-08-31 11:53:12 +01:00
catbref
3cc66609e8 Trial implementation of offline repository periodic maintenance.
Requires node shutdown, lots of time (10s of minutes), spare storage space.
Called via: java -cp qortal.jar org.qortal.RepositoryMaintenance
Not (yet) for general consumption.
2020-08-31 11:51:38 +01:00
catbref
ce468d22dd Fix updating of current tradeoffers list as used by tradeoffers websocket 2020-08-31 11:25:21 +01:00
catbref
3e19516f62 Correct poor synchronization on NTP offset, potentially fixing issue #22 2020-08-31 10:24:10 +01:00
catbref
84dba739d9 Give up on cross-chain trade if initial AT doesn't confirm within 24 hours 2020-08-31 09:21:15 +01:00
catbref
99315c7378 Correct wrong source for lockTimeA when Bob waiting for P2SH-B. Spotted by tcallahan14. In lieu of PR #23 2020-08-31 09:14:15 +01:00
catbref
1ca5b864a9 Repository optimizations!
Added Qortal-side HSQLDB PreparedStatement cache, hashed
by SQL query string, to reduce re-preparing statements.
(HSQLDB actually does the work in avoiding re-preparing
by comparing its own query-to-statement cache map, but we
need to keep an 'open' statement on our side for this to
happen).

Support added for batched INSERT/UPDATE SQL statements to
update many rows in one call.

Several specific repository calls, e.g. modifyMintedBlockCount
or modifyAssetBalance, now have batch versions that allow
many rows to be updated in one call.

In Block, when distributing block rewards, although we still
build a map of balance changes to apply after all calculations,
this map is now handed off wholesale to the repository to
apply in one (or two) queries, instead of a repository call
per account. The balanceChanges map is now keyed by account
address, as opposed to actual Account.

Also in Block, we try to cache the fetched online reward-shares
(typically in Block.isValid et al) to avoid re-fetching them
later when calculating block rewards.

In addition, actually fetching online reward-shares is no longer
done index-by-index, but the whole array of indexes is passed
wholesale to the repository which then returns the corresponding
reward-shares as a list.

In Block.increaseAccountLevels, blocks minted counts are also
updated in one single repository call, rather than one
repository call per account.

When distributing Block rewards to legacy QORA holders,
all necessary info is fetched from the repository in one hit
instead of two-phases of: 1. fetching eligible QORA holders,
and 2. fetching extra data for that QORA holder as needed.

In addition, updated QORT_FROM_QORA asset balances are done
via one batch repository call, rather than per update.
2020-08-26 17:16:45 +01:00
catbref
96eb60dca3 More HSQLDB tests to cover fixes for various HSQLDB issues, especially when using custom HSQLDB build 2020-08-25 17:02:14 +01:00
catbref
c67fcb0034 Updated AdvancedInstaller project file based on v1.3.4 release 2020-08-24 15:40:29 +01:00
catbref
273dfe2365 Bump to v1.3.4 2020-08-24 15:05:51 +01:00
catbref
5952ea4b54 RU translations thanks to Alexander45 2020-08-24 15:04:33 +01:00
catbref
1708ba077c Actually define static constants for BTC fees until dynamic fees happen 2020-08-24 14:28:54 +01:00
catbref
b4301f125d Potential fix for issue #22 2020-08-24 14:27:03 +01:00
catbref
9e52f20f71 Revert back to HSQLDB v2.5.0-fixed until out-of-memory issue located 2020-08-24 14:07:36 +01:00
catbref
31bf388cab BlockMinter (now under org.qortal.controller package) doesn't need full previous block, only previous block data 2020-08-24 14:04:11 +01:00
catbref
276c479a5f Refactor to allow better Bitcoin fee estimation in the future. 2020-08-21 17:37:04 +01:00
catbref
9393689037 Send BTCACCT refunds to first unused received address instead of address derived from tradePrivateKey.
Added BTC.getUnusedReceiveAddress() to support above.
2020-08-21 17:35:33 +01:00
catbref
76485010ad Merge pull request #16 from tcallahan14/feature/electrum_nodes
Updated Electrum nodes list
2020-08-21 13:34:49 +01:00
catbref
b8ac128d5c Improve comparing chains where some blocks signed with cancelled reward-share
Symptoms include this in logs:

Unexpected zero effective minter level for reward-share %s - using 1 instead!

This occurs when Synchronizer compares two sub-chains from a common block,
and one of the blocks is signed by a reward-share key that has
subsequently been cancelled.

Although this is catered for, excessive log-spam is emited.

So in addition to demoting the log level from WARN to DEBUG,
more code has been added to try harder to find the actual data needed,
thus preventing the logging in the first place.

New repository transaction search method added to support above,
along with corresponding tests.
2020-08-21 12:27:06 +01:00
CalDescent
06c75310a1 Updated Electrum nodes list. All nodes have been tested to ensure they respond to jsonrpc calls. 2020-08-15 16:53:09 +01:00
catbref
b9d819220d Bumped HSQLDB to v2.5.1 and AT/cross-chain SQL speed-ups! 2020-08-15 11:12:10 +01:00
catbref
7a569f342f Reduce confusing BlockMinter log spam - issue #9 2020-08-15 10:52:48 +01:00
catbref
f1efae79c8 Speed-ups for some AT-related SQL queries 2020-08-14 11:54:33 +01:00
catbref
1cd4bbc078 Refactored various websockets to event bus from old BlockNotifier/StatusNotifier 2020-08-14 10:03:51 +01:00
catbref
0b5e5832c4 Added another repository deadlock test while investigating a deadlock case 2020-08-14 09:57:08 +01:00
catbref
7db96c672f Bump to v1.3.3 2020-08-13 14:16:50 +01:00
catbref
f8725d6313 Modify ApplyUpdate to pass JVM options to Windows launcher EXE
ApplyUpdate is the 2nd-stage of the auto-update system, called
after core has downloaded the update.

As old versions of the Windows launcher EXE selects a 'client'
JVM mode, heap memory could be limited to only 256MB.

Until users upgrade via Windows installer, which replaces the EXE
with 'server' JVM mode baked-in, then a work-around is to
pass -XX:MaxRAMFraction=4 to the new JVM in order to emulate
heap size in 'server' JVM mode.
2020-08-13 14:08:47 +01:00
catbref
2165c87b9d Fix race condition between Network.start() and Controller calling Network.prunePeers()
Modified synchronized Lists to be final.
Moved some initializers out of constructor.
2020-08-13 13:45:06 +01:00
catbref
f61e320230 Fix API call GET /crosschain/trades (get completed trades) due to poorly performing SQL query.
Added "minimumTimestamp" param to same API call to allow fetching results for scenarios like:
* completed trades since midnight
* completed trades within last 24 hours

Added corresponding tests for above API call, including checking call response times.
2020-08-13 11:56:08 +01:00
264 changed files with 27534 additions and 10516 deletions

33
.github/workflows/pr-testing.yml vendored Normal file
View File

@@ -0,0 +1,33 @@
name: PR testing
on:
pull_request:
branches: [ master ]
jobs:
mavenTesting:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Cache local Maven repository
uses: actions/cache@v2
with:
path: ~/.m2/repository
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- name: Set up the Java JDK
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Run all tests
run: |
mvn -B clean test -DskipTests=false --file pom.xml
if [ -f "target/site/jacoco/index.html" ]; then echo "Total coverage: $(cat target/site/jacoco/index.html | grep -o 'Total[^%]*%' | grep -o '[0-9]*%')"; fi
- name: Log coverage percentage
run: |
if [ ! -f "target/site/jacoco/index.html" ]; then echo "No coverage information available"; fi
if [ -f "target/site/jacoco/index.html" ]; then echo "Total coverage: $(cat target/site/jacoco/index.html | grep -o 'Total[^%]*%' | grep -o '[0-9]*%')"; fi

9
.gitignore vendored
View File

@@ -1,6 +1,7 @@
/db*
/bin/
/target/
/qortal-backup/
/log.txt.*
/arbitrary*
/Qortal-BTC*
@@ -16,3 +17,11 @@
/settings*.json
/testchain.json
/run-testnet.sh
/.idea
/qortal.iml
.DS_Store
/src/main/resources/resources
/*.jar
/run.pid
/run.log
/WindowsInstaller/Install Files/qortal.jar

View File

@@ -1,5 +1,20 @@
# Auto Updates
## TL;DR: how-to
* Prepare new release version (see way below for details)
* Assuming you are in git 'master' branch, at HEAD
* Shutdown local node if running
* Build auto-update download: `tools/build-auto-update.sh` - uploads auto-update file into new git branch
* Restart local node
* Publish auto-update transaction using *private key* for **non-admin** member of "dev" group:
`tools/publish-auto-update.sh non-admin-dev-member-private-key-in-base58`
* Wait for auto-update `ARBITRARY` transaction to be confirmed into a block
* Have "dev" group admins 'approve' auto-update using `tools/approve-auto-update.sh`
This tool will prompt for *private key* of **admin** of "dev" group
* A minimum number of admins are required for approval, and a minimum number of blocks must pass also.
* Nodes will start to download, and apply, the update over the next 20 minutes or so (see CHECK_INTERVAL in AutoUpdate.java)
## Theory
* Using a specific git commit (e.g. abcdef123) we produce a determinstic JAR with consistent hash.
* To avoid issues with over-eager anti-virus / firewalls we obfuscate JAR using very simplistic XOR-based method.
@@ -25,8 +40,8 @@ The same method is used to obfuscate and de-obfuscate:
## Typical download locations
The git SHA1 commit hash is used to replace `%s` in various download locations, e.g.:
* https://github.com/QORT/qortal/raw/%s/qortal.update
* https://raw.githubusercontent.com@151.101.16.133/QORT/qortal/%s/qortal.update
* https://github.com/Qortal/qortal/raw/%s/qortal.update
* https://raw.githubusercontent.com@151.101.16.133/Qortal/qortal/%s/qortal.update
These locations are part of the org.qortal.settings.Settings class and can be overriden in settings.json like:
```
@@ -45,4 +60,12 @@ $ java -cp qortal.jar org.qortal.XorUpdate
usage: XorUpdate <input-file> <output-file>
$ java -cp qortal.jar org.qortal.XorUpdate qortal.jar qortal.update
$
```
```
## Preparing new release version
* Shutdown local node
* Modify `pom.xml` and increase version inside `<version>` tag
* Commit new `pom.xml` and push to github, e.g. `git commit -m 'Bumped to v1.4.2' -- pom.xml; git push`
* Tag this new commit with same version: `git tag v1.4.2`
* Push tag up to github: `git push origin v1.4.2`

View File

@@ -4,10 +4,10 @@ You can examine your node's database using [HSQLDB's "sqltool"](http://www.hsqld
It's a good idea to install "rlwrap" (ReadLine wrapper) too as sqltool doesn't support command history/editing.
Typical command line for sqltool would be:
`rlwrap java -cp ${HSQLDB_JAR}:${SQLTOOL_JAR} org.hsqldb.cmdline.SqlTool --rcFile=${SQLTOOL_RC} qora`
`rlwrap java -cp ${HSQLDB_JAR}:${SQLTOOL_JAR} org.hsqldb.cmdline.SqlTool --rcFile=${SQLTOOL_RC} qortal`
`${HSQLDB_JAR}` should be set with pathname where Maven downloaded hsqldb,
typically `${HOME}/.m2/repository/org/hsqldb/hsqldb/2.5.0/hsqldb-2.5.0.jar`
typically `${HOME}/.m2/repository/org/hsqldb/hsqldb/2.5.1/hsqldb-2.5.1.jar`
`${SQLTOOL_JAR}` should be set with pathname where Maven downloaded sqltool,
typically `${HOME}/.m2/repository/org/hsqldb/sqltool/2.5.0/sqltool-2.5.0.jar`
@@ -25,10 +25,16 @@ Above `url` component `file:db/blockchain` assumes you will call `sqltool` from
Another idea is to assign a shell alias in your `.bashrc` like:
```
export HSQLDB_JAR=${HOME}/.m2/repository/org/hsqldb/hsqldb/2.5.0/hsqldb-2.5.0.jar
export HSQLDB_JAR=${HOME}/.m2/repository/org/hsqldb/hsqldb/2.5.1/hsqldb-2.5.1.jar
export SQLTOOL_JAR=${HOME}/.m2/repository/org/hsqldb/sqltool/2.5.0/sqltool-2.5.0.jar
alias sqltool='rlwrap java -cp ${HSQLDB_JAR}:${SQLTOOL_JAR} org.hsqldb.cmdline.SqlTool --rcFile=${SQLTOOL_RC}'
```
So you can simply type: `sqltool qortal`
Don't forget to use `SHUTDOWN;` before exiting sqltool so that database files are closed cleanly.
## Quick and dirty version
With `sqltool-2.5.0.jar` and `qortal.jar` in current directory, and database in `db/`
`java -cp qortal.jar:sqltool-2.5.0.jar org.hsqldb.cmdline.SqlTool --inlineRc=url=jdbc:hsqldb:file:db/blockchain`

View File

@@ -9,4 +9,4 @@
- Create basic *settings.json* file: `echo '{}' > settings.json`
- Run JAR in same working directory as *settings.json*: `java -jar target/qortal-1.0.jar`
- Wrap in shell script, add JVM flags, redirection, backgrounding, etc. as necessary.
- Or use supplied example shell script: *run.sh*
- Or use supplied example shell script: *start.sh*

69
TestNets.md Normal file
View File

@@ -0,0 +1,69 @@
# How to build a testnet
## Create testnet blockchain config
- You can begin by copying the mainnet blockchain config `src/main/resources/blockchain.json`
- Insert `"isTestChain": true,` after the opening `{`
- Modify testnet genesis block
### Testnet genesis block
- Set `timestamp` to a nearby future value, e.g. 15 mins from 'now'
This is to give yourself enough time to set up other testnet nodes
- Retain the initial `ISSUE_ASSET` transactions!
- Add `ACCOUNT_FLAGS` transactions with `"andMask": -1, "orMask": 1, "xorMask": 0` to create founders
- Add at least one `REWARD_SHARE` transaction otherwise no-one can mint initial blocks!
You will need to calculate `rewardSharePublicKey` (and private key),
or make a new account on mainnet and use self-share key values
- Add `ACCOUNT_LEVEL` transactions to set initial level of accounts as needed
- Add `GENESIS` transactions to add QORT/LEGACY_QORA funds to accounts as needed
## Testnet `settings.json`
- Create a new `settings-test.json`
- Make sure to add `"isTestNet": true,`
- Make sure to reference testnet blockchain config file: `"blockchainConfig": "testchain.json",`
- It is a good idea to use a separate database: `"repositoryPath": "db-testnet",`
- You might also need to add `"bitcoinNet": "TEST3",` and `"litecoinNet": "TEST3",`
## Other nodes
- Copy `testchain.json` and `settings-test.json` to other nodes
- Alternatively, you can run multiple nodes on the same machine by:
* Copying `settings-test.json` to `settings-test-1.json`
* Configure different `repositoryPath`
* Configure use of different ports:
+ `"apiPort": 22391,`
+ `"listenPort": 22392,`
## Starting-up
- Start up at least as many nodes as `minBlockchainPeers` (or adjust this value instead)
- Probably best to perform API call `DELETE /peers/known`
- Add other nodes via API call `POST /peers <peer-hostname-or-IP>`
- Add minting private key to node(s) via API call `POST /admin/mintingaccounts <minting-private-key>`
This key must have corresponding `REWARD_SHARE` transaction in testnet genesis block
- Wait for genesis block timestamp to pass
- A node should mint block 2 approximately 60 seconds after genesis block timestamp
- Other testnet nodes will sync *as long as there is at least `minBlockchainPeers` peers with an "up-to-date" chain`
- You can also use API call `POST /admin/forcesync <connected-peer-IP-and-port>` on stuck nodes
## Dealing with stuck chain
Maybe your nodes have been offline and no-one has minted a recent testnet block.
Your options are:
- Start a new testnet from scratch
- Fire up your testnet node(s)
- Force one of your nodes to mint by:
+ Set a debugger breakpoint on Settings.getMinBlockchainPeers()
+ When breakpoint is hit, change `this.minBlockchainPeers` to zero, then continue
- Once one of your nodes has minted blocks up to 'now', you can use "forcesync" on the other nodes
## Tools
- `qort` tool, but use `-t` option for default testnet API port (62391)
- `qort` tool, but first set shell variable: `export BASE_URL=some-node-hostname-or-ip:port`
- `qort` tool, but prepend with one-time shell variable: `BASE_URL=some-node-hostname-or-ip:port qort ......`
- `peer-heights`, but use `-t` option, or `BASE_URL` shell variable as above

File diff suppressed because it is too large Load Diff

View File

@@ -12,7 +12,7 @@ configured paths, or create a dummy `D:` drive with the expected layout.
Typical build procedure:
* Overwrite the `qortal.jar` file in `Install-Files\`
* Place the `qortal.jar` file in `Install-Files\`
* Open AdvancedInstaller with qortal.aip file
* If releasing a new version, change version number in:
+ "Product Information" side menu

20
WindowsInstaller/dictionary.ail Executable file → Normal file
View File

@@ -3,95 +3,115 @@
<!-- Control table -->
<ENTRY id="Control.Text.CustomizeDataPathDlg#Description">
<STRING lang="en" value="Do you want to store the blockchain, and other data, in a specific folder?"/>
<STRING lang="ru" value="Вы можете выбрать место хранения блокчейна и других данных."/>
<STRING lang="zh" value="你想把区块链数据存放在一个特定的文件夹吗?"/>
<STRING lang="zh_TW" value="你想把区块链数据存放在一个特定的文件夹吗?"/>
</ENTRY>
<ENTRY id="Control.Text.CustomizeDataPathDlg#Text">
<STRING lang="en" value="Select one of the options below, then click &quot;Next&quot;."/>
<STRING lang="ru" value="Выберите один из вариантов ниже, затем нажмите Далее"/>
<STRING lang="zh" value="请选择,然后“下一步”"/>
<STRING lang="zh_TW" value="请选择,然后“下一步”"/>
</ENTRY>
<ENTRY id="Control.Text.CustomizeDataPathDlg#Title">
<STRING lang="en" value="Choose Custom Data Storage Folder?"/>
<STRING lang="ru" value="Выберите место хранения данных."/>
<STRING lang="zh" value="选择数据保存的文件夹?"/>
<STRING lang="zh_TW" value="选择数据保存的文件夹?"/>
</ENTRY>
<ENTRY id="Control.Text.CustomizeDbDlg#Description">
<STRING lang="en" value="Do you want to store the blockchain, and other data, in a specific folder?"/>
<STRING lang="ru" value="Вы можете выбрать место хранения блокчейна и других данных."/>
<STRING lang="zh" value="Do you want to store the blockchain, and other data, in a specific folder?"/>
<STRING lang="zh_TW" value="Do you want to store the blockchain, and other data, in a specific folder?"/>
</ENTRY>
<ENTRY id="Control.Text.CustomizeDbDlg#Title">
<STRING lang="en" value="Choose Custom Data Storage Folder?"/>
<STRING lang="ru" value="Выберите место хранения данных."/>
<STRING lang="zh" value="Choose Custom Data Storage Folder?"/>
<STRING lang="zh_TW" value="Choose Custom Data Storage Folder?"/>
</ENTRY>
<ENTRY id="Control.Text.DataFolderDlg#Description">
<STRING lang="en" value="This is the folder where the blockchain, and other data, will be stored."/>
<STRING lang="ru" value="Это папка, в которой будет храниться блокчейн и другие данные."/>
<STRING lang="zh" value="这里是区块链及其它数据存放的文件夹"/>
<STRING lang="zh_TW" value="这里是区块链及其它数据存放的文件夹"/>
</ENTRY>
<ENTRY id="Control.Text.DataFolderDlg#Text">
<STRING lang="en" value="To store data in this folder, click &quot;[Text_Next]&quot;. To store data in a different folder, enter it below or click &quot;Browse&quot;."/>
<STRING lang="ru" value="Чтобы сохранить данные в этой папке, нажмите &quot;[Text_Next]&quot;. Чтобы сохранить данные в другой папке, введите ее ниже или нажмите &quot;Обзор&quot;."/>
<STRING lang="zh" value="如果存放在这个文件夹,点 “下一步”。如果存放在其它位置,请选择“浏览”。"/>
<STRING lang="zh_TW" value="如果存放在这个文件夹,点 “下一步”。如果存放在其它位置,请选择“浏览”。"/>
</ENTRY>
<ENTRY id="Control.Text.DataFolderDlg#Title">
<STRING lang="en" value="Select Data Storage Folder"/>
<STRING lang="ru" value="Выберите папку для хранения данных"/>
<STRING lang="zh" value="请选择文件存储地方"/>
<STRING lang="zh_TW" value="请选择文件存储地方"/>
</ENTRY>
<ENTRY id="Control.Text.DbFolderDlg#Description">
<STRING lang="en" value="This is the folder where the blockchain, and other data, will be stored."/>
<STRING lang="ru" value="Это папка, в которой будет храниться блокчейн и другие данные."/>
<STRING lang="zh" value="This is the folder where the blockchain, and other data, will be stored."/>
<STRING lang="zh_TW" value="This is the folder where the blockchain, and other data, will be stored."/>
</ENTRY>
<ENTRY id="Control.Text.DbFolderDlg#Title">
<STRING lang="en" value="Select Data Storage Folder"/>
<STRING lang="ru" value="Выберите папку для хранения данных"/>
<STRING lang="zh" value="请选择文件存储地方"/>
<STRING lang="zh_TW" value="请选择文件存储地方"/>
</ENTRY>
<ENTRY id="Control.Text.NTPDialog#Description">
<STRING lang="en" value="Reconfigure Windows for more accurate time?"/>
<STRING lang="ru" value="Настроить синхронизацию времени системы Windows?"/>
<STRING lang="zh" value="重新配置Windows以获得更准确的时间"/>
</ENTRY>
<ENTRY id="Control.Text.NTPDialog#Text_1">
<STRING lang="en" value="An accurate Windows clock is required to connect to the [ProductName] network and make transactions."/>
<STRING lang="ru" value="Для подключения к сети Qortal и совершения транзакций требуется точная настройка времени Windows"/>
<STRING lang="zh" value="需要准确的Windows时钟才能连接到[ProductName]网络并进行交易。"/>
</ENTRY>
<ENTRY id="Control.Text.NTPDialog#Text_2">
<STRING lang="en" value="Select one of the options below, then click &quot;Next&quot;."/>
<STRING lang="ru" value="Выберите один из вариантов ниже, затем нажмите"/>
<STRING lang="zh" value="请选择,然后“下一步”"/>
</ENTRY>
<ENTRY id="Control.Text.NTPDialog#Text_3">
<STRING lang="en" value="Your computer&apos;s clock needs to be accurate to within 0.5 seconds."/>
<STRING lang="ru" value="Точность времени вашего компьютера должна составлять 0.5 секунд."/>
<STRING lang="zh" value="您的计算机时钟需要准确到0.5秒内。"/>
</ENTRY>
<ENTRY id="Control.Text.NTPDialog#Title">
<STRING lang="en" value="Windows clock accuracy"/>
<STRING lang="ru" value="Настройка времени системы Windows"/>
<STRING lang="zh" value="Windows 时钟精度"/>
</ENTRY>
<ENTRY id="Control.Text.VerifyRemoveDlg#RemoveBlockchainCheckbox">
<STRING lang="en" value="Remove downloaded blockchain and other data"/>
<STRING lang="ru" value="Удалить загруженный блокчейн и другие данные"/>
<STRING lang="zh" value="删除您下载的区块链"/>
</ENTRY>
<!-- RadioButton table -->
<ENTRY id="RadioButton.Text.CUSTOM_DB_BOOL#choose">
<STRING lang="en" value="Choose custom data storage folder..."/>
<STRING lang="ru" value="Выбрать папку для хранения данных..."/>
<STRING lang="zh" value="选择特定的文件夹存储"/>
<STRING lang="zh_TW" value="选择特定的文件夹存储"/>
</ENTRY>
<ENTRY id="RadioButton.Text.CUSTOM_DB_BOOL#default">
<STRING lang="en" value="Use default location "/>
<STRING lang="ru" value="Использовать папку по умолчанию"/>
<STRING lang="zh" value="使用默认存储地点"/>
<STRING lang="zh_TW" value="使用默认存储地点"/>
</ENTRY>
<ENTRY id="RadioButton.Text.RECONFIG_NTP#1">
<STRING lang="en" value="Yes, configure Windows to use internet time servers (Recommended)"/>
<STRING lang="ru" value="Да, настроить синхронизацию времени Windows (Рекомендуется)"/>
<STRING lang="zh" value="是将Windows配置为使用多个Internet时间服务器 (推荐的)"/>
</ENTRY>
<ENTRY id="RadioButton.Text.RECONFIG_NTP#2">
<STRING lang="en" value="No, I will manage clock accuracy myself"/>
<STRING lang="ru" value="Нет, я сам буду управлять настройками часов"/>
<STRING lang="zh" value="不,我会自己管理时钟精度。"/>
</ENTRY>
</DICTIONARY>

Binary file not shown.

View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>org.ciyam</groupId>
<artifactId>AT</artifactId>
<version>1.3.8</version>
<description>POM was created from install:install-file</description>
</project>

View File

@@ -3,13 +3,14 @@
<groupId>org.ciyam</groupId>
<artifactId>AT</artifactId>
<versioning>
<release>1.3.7</release>
<release>1.3.8</release>
<versions>
<version>1.3.4</version>
<version>1.3.5</version>
<version>1.3.6</version>
<version>1.3.7</version>
<version>1.3.8</version>
</versions>
<lastUpdated>20200812131412</lastUpdated>
<lastUpdated>20200925114415</lastUpdated>
</versioning>
</metadata>

45
pom.xml
View File

@@ -3,19 +3,20 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.qortal</groupId>
<artifactId>qortal</artifactId>
<version>1.3.2</version>
<version>1.6.0</version>
<packaging>jar</packaging>
<properties>
<bitcoinj.version>0.15.5</bitcoinj.version>
<skipTests>true</skipTests>
<altcoinj.version>bf9fb80</altcoinj.version>
<bitcoinj.version>0.15.10</bitcoinj.version>
<bouncycastle.version>1.64</bouncycastle.version>
<build.timestamp>${maven.build.timestamp}</build.timestamp>
<ciyam-at.version>1.3.7</ciyam-at.version>
<ciyam-at.version>1.3.8</ciyam-at.version>
<commons-net.version>3.6</commons-net.version>
<commons-text.version>1.8</commons-text.version>
<dagger.version>1.2.2</dagger.version>
<guava.version>28.1-jre</guava.version>
<hsqldb.version>2.5.0-fixed</hsqldb.version>
<hsqldb-sqltool.version>2.5.0</hsqldb-sqltool.version>
<hsqldb.version>2.5.1</hsqldb.version>
<jersey.version>2.29.1</jersey.version>
<jetty.version>9.4.29.v20200521</jetty.version>
<log4j.version>2.12.1</log4j.version>
@@ -199,6 +200,10 @@
<pattern>org.qortal.api.model**</pattern>
<template>${project.build.sourceDirectory}/org/qortal/data/package-info.java</template>
</package>
<package>
<pattern>org.qortal.api.model.**</pattern>
<template>${project.build.sourceDirectory}/org/qortal/data/package-info.java</template>
</package>
</packages>
<outputDirectory>${project.build.directory}/generated-sources/package-info</outputDirectory>
</configuration>
@@ -313,6 +318,14 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<skipTests>${skipTests}</skipTests>
</configuration>
</plugin>
</plugins>
<pluginManagement>
<plugins>
@@ -375,6 +388,11 @@
<name>project</name>
<url>file:${project.basedir}/lib</url>
</repository>
<!-- jitpack for build-on-demand of altcoinj -->
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
</repositories>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.codehaus.mojo/build-helper-maven-plugin -->
@@ -397,12 +415,6 @@
<artifactId>hsqldb</artifactId>
<version>${hsqldb.version}</version>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>sqltool</artifactId>
<version>${hsqldb-sqltool.version}</version>
<scope>test</scope>
</dependency>
<!-- CIYAM AT (automated transactions) -->
<dependency>
<groupId>org.ciyam</groupId>
@@ -415,12 +427,23 @@
<artifactId>bitcoinj-core</artifactId>
<version>${bitcoinj.version}</version>
</dependency>
<!-- For Litecoin, etc. support, requires bitcoinj -->
<dependency>
<groupId>com.github.jjos2372</groupId>
<artifactId>altcoinj</artifactId>
<version>${altcoinj.version}</version>
</dependency>
<!-- Utilities -->
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20210307</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId>

51
run.sh
View File

@@ -1,51 +0,0 @@
#!/bin/sh
# There's no need to run as root, so don't allow it, for security reasons
if [ "$USER" = "root" ]; then
echo "Please su to non-root user before running"
exit
fi
# Validate Java is installed and the minimum version is available
MIN_JAVA_VER='11'
if command -v java > /dev/null 2>&1; then
version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo $version | cut -d'.' -f1,2)
if [ `echo "${version}>=${MIN_JAVA_VER}" | bc` -eq 1 ]; then
echo 'Passed Java version check'
else
echo 'Please upgrade your Java to version 11 or greater'
exit 1
fi
else
echo 'Java is not available, please install Java 11 or greater'
exit 1
fi
# No qortal.jar but we have a Maven built one?
# Be helpful and copy across to correct location
if [ ! -e qortal.jar -a -f target/qortal*.jar ]; then
echo "Copying Maven-built Qortal JAR to correct pathname"
cp target/qortal*.jar qortal.jar
fi
# Limits Java JVM stack size and maximum heap usage.
# Comment out for bigger systems, e.g. non-routers
# or when API documentation is enabled
# JVM_MEMORY_ARGS="-Xss256k -Xmx128m"
# Although java.net.preferIPv4Stack is supposed to be false
# by default in Java 11, on some platforms (e.g. FreeBSD 12),
# it is overriden to be true by default. Hence we explicitly
# set it to true to obtain desired behaviour.
nohup nice -n 20 java \
-Djava.net.preferIPv4Stack=false \
-XX:NativeMemoryTracking=summary \
${JVM_MEMORY_ARGS} \
-jar qortal.jar \
1>run.log 2>&1 &
# Save backgrounded process's PID
echo $! > run.pid
echo qortal running as pid $!

1
run.sh Symbolic link
View File

@@ -0,0 +1 @@
start.sh

View File

@@ -21,18 +21,28 @@ public class HSQLDBPool extends JDBCPool {
public Connection tryConnection() throws SQLException {
for (int i = 0; i < states.length(); i++) {
if (states.compareAndSet(i, RefState.available, RefState.allocated)) {
return connections[i].getConnection();
JDBCPooledConnection pooledConnection = connections[i];
if (pooledConnection == null)
// Probably shutdown situation
return null;
return pooledConnection.getConnection();
}
if (states.compareAndSet(i, RefState.empty, RefState.allocated)) {
try {
JDBCPooledConnection connection = (JDBCPooledConnection) source.getPooledConnection();
JDBCPooledConnection pooledConnection = (JDBCPooledConnection) source.getPooledConnection();
connection.addConnectionEventListener(this);
connection.addStatementEventListener(this);
connections[i] = connection;
if (pooledConnection == null)
// Probably shutdown situation
return null;
return connections[i].getConnection();
pooledConnection.addConnectionEventListener(this);
pooledConnection.addStatementEventListener(this);
connections[i] = pooledConnection;
return pooledConnection.getConnection();
} catch (SQLException e) {
states.set(i, RefState.empty);
}

View File

@@ -35,6 +35,8 @@ public class ApplyUpdate {
private static final String JAR_FILENAME = AutoUpdate.JAR_FILENAME;
private static final String NEW_JAR_FILENAME = AutoUpdate.NEW_JAR_FILENAME;
private static final String WINDOWS_EXE_LAUNCHER = "qortal.exe";
private static final String JAVA_TOOL_OPTIONS_NAME = "JAVA_TOOL_OPTIONS";
private static final String JAVA_TOOL_OPTIONS_VALUE = "-XX:MaxRAMFraction=4";
private static final long CHECK_INTERVAL = 10 * 1000L; // ms
private static final int MAX_ATTEMPTS = 12;
@@ -65,17 +67,19 @@ public class ApplyUpdate {
}
private static boolean shutdownNode() {
String BASE_URI = "http://localhost:" + Settings.getInstance().getApiPort() + "/";
LOGGER.info(String.format("Shutting down node using API via %s", BASE_URI));
String baseUri = "http://localhost:" + Settings.getInstance().getApiPort() + "/";
LOGGER.info(() -> String.format("Shutting down node using API via %s", baseUri));
int attempt;
for (attempt = 0; attempt < MAX_ATTEMPTS; ++attempt) {
LOGGER.info(String.format("Attempt #%d out of %d to shutdown node", attempt + 1, MAX_ATTEMPTS));
String response = ApiRequest.perform(BASE_URI + "admin/stop", null);
final int attemptForLogging = attempt;
LOGGER.info(() -> String.format("Attempt #%d out of %d to shutdown node", attemptForLogging + 1, MAX_ATTEMPTS));
String response = ApiRequest.perform(baseUri + "admin/stop", null);
if (response == null)
break;
// No response - consider node shut down
return true;
LOGGER.info(String.format("Response from API: %s", response));
LOGGER.info(() -> String.format("Response from API: %s", response));
try {
Thread.sleep(CHECK_INTERVAL);
@@ -99,19 +103,20 @@ public class ApplyUpdate {
Path newJar = Paths.get(NEW_JAR_FILENAME);
if (!Files.exists(newJar)) {
LOGGER.warn(String.format("Replacement JAR '%s' not found?", newJar));
LOGGER.warn(() -> String.format("Replacement JAR '%s' not found?", newJar));
return;
}
int attempt;
for (attempt = 0; attempt < MAX_ATTEMPTS; ++attempt) {
LOGGER.info(String.format("Attempt #%d out of %d to replace JAR", attempt + 1, MAX_ATTEMPTS));
final int attemptForLogging = attempt;
LOGGER.info(() -> String.format("Attempt #%d out of %d to replace JAR", attemptForLogging + 1, MAX_ATTEMPTS));
try {
Files.copy(newJar, realJar, StandardCopyOption.REPLACE_EXISTING);
break;
} catch (IOException e) {
LOGGER.info(String.format("Unable to replace JAR: %s", e.getMessage()));
LOGGER.info(() -> String.format("Unable to replace JAR: %s", e.getMessage()));
// Try again
}
@@ -119,6 +124,7 @@ public class ApplyUpdate {
try {
Thread.sleep(CHECK_INTERVAL);
} catch (InterruptedException e) {
LOGGER.warn("Ignoring interrupt...");
// Doggedly retry
}
}
@@ -129,13 +135,13 @@ public class ApplyUpdate {
private static void restartNode(String[] args) {
String javaHome = System.getProperty("java.home");
LOGGER.info(String.format("Java home: %s", javaHome));
LOGGER.info(() -> String.format("Java home: %s", javaHome));
Path javaBinary = Paths.get(javaHome, "bin", "java");
LOGGER.info(String.format("Java binary: %s", javaBinary));
LOGGER.info(() -> String.format("Java binary: %s", javaBinary));
Path exeLauncher = Paths.get(WINDOWS_EXE_LAUNCHER);
LOGGER.info(String.format("Windows EXE launcher: %s", exeLauncher));
LOGGER.info(() -> String.format("Windows EXE launcher: %s", exeLauncher));
List<String> javaCmd;
if (Files.exists(exeLauncher)) {
@@ -156,9 +162,16 @@ public class ApplyUpdate {
}
try {
LOGGER.info(String.format("Restarting node with: %s", String.join(" ", javaCmd)));
LOGGER.info(() -> String.format("Restarting node with: %s", String.join(" ", javaCmd)));
new ProcessBuilder(javaCmd).start();
ProcessBuilder processBuilder = new ProcessBuilder(javaCmd);
if (Files.exists(exeLauncher)) {
LOGGER.info(() -> String.format("Setting env %s to %s", JAVA_TOOL_OPTIONS_NAME, JAVA_TOOL_OPTIONS_VALUE));
processBuilder.environment().put(JAVA_TOOL_OPTIONS_NAME, JAVA_TOOL_OPTIONS_VALUE);
}
processBuilder.start();
} catch (IOException e) {
LOGGER.error(String.format("Failed to restart node (BAD): %s", e.getMessage()));
}

View File

@@ -0,0 +1,75 @@
package org.qortal;
import java.security.Security;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.jsse.provider.BouncyCastleJsseProvider;
import org.qortal.controller.Controller;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryFactory;
import org.qortal.repository.RepositoryManager;
import org.qortal.repository.hsqldb.HSQLDBRepositoryFactory;
import org.qortal.settings.Settings;
public class RepositoryMaintenance {
static {
// This must go before any calls to LogManager/Logger
System.setProperty("java.util.logging.manager", "org.apache.logging.log4j.jul.LogManager");
}
private static final Logger LOGGER = LogManager.getLogger(RepositoryMaintenance.class);
public static void main(String[] args) {
LOGGER.info("Repository maintenance starting up...");
Security.insertProviderAt(new BouncyCastleProvider(), 0);
Security.insertProviderAt(new BouncyCastleJsseProvider(), 1);
// Load/check settings, which potentially sets up blockchain config, etc.
try {
if (args.length > 0)
Settings.fileInstance(args[0]);
else
Settings.getInstance();
} catch (Throwable t) {
LOGGER.error("Settings file error: " + t.getMessage());
System.exit(2);
}
LOGGER.info("Opening repository");
try {
RepositoryFactory repositoryFactory = new HSQLDBRepositoryFactory(Controller.getRepositoryUrl());
RepositoryManager.setRepositoryFactory(repositoryFactory);
} catch (DataException e) {
// If exception has no cause then repository is in use by some other process.
if (e.getCause() == null) {
LOGGER.info("Repository in use by another process?");
} else {
LOGGER.error("Unable to start repository", e);
}
System.exit(1);
}
LOGGER.info("Starting repository periodic maintenance. This can take a while...");
try (final Repository repository = RepositoryManager.getRepository()) {
repository.performPeriodicMaintenance();
LOGGER.info("Repository periodic maintenance completed");
} catch (DataException e) {
LOGGER.error("Repository periodic maintenance failed", e);
}
try {
LOGGER.info("Shutting down repository");
RepositoryManager.closeRepositoryFactory();
} catch (DataException e) {
LOGGER.error("Error occurred while shutting down repository", e);
}
}
}

View File

@@ -15,7 +15,7 @@ public enum ApiError {
// COMMON
// UNKNOWN(0, 500),
JSON(1, 400),
// NO_BALANCE(2, 422),
INSUFFICIENT_BALANCE(2, 402),
// NOT_YET_RELEASED(3, 422),
UNAUTHORIZED(4, 403),
REPOSITORY_ISSUE(5, 500),
@@ -126,10 +126,13 @@ public enum ApiError {
// Groups
GROUP_UNKNOWN(1101, 404),
// Bitcoin
BTC_NETWORK_ISSUE(1201, 500),
BTC_BALANCE_ISSUE(1202, 402),
BTC_TOO_SOON(1203, 408);
// Foreign blockchain
FOREIGN_BLOCKCHAIN_NETWORK_ISSUE(1201, 500),
FOREIGN_BLOCKCHAIN_BALANCE_ISSUE(1202, 402),
FOREIGN_BLOCKCHAIN_TOO_SOON(1203, 408),
// Trade portal
ORDER_SIZE_TOO_SMALL(1300, 402);
private static final Map<Integer, ApiError> map = stream(ApiError.values()).collect(toMap(apiError -> apiError.code, apiError -> apiError));
@@ -157,4 +160,4 @@ public enum ApiError {
return this.status;
}
}
}

View File

@@ -43,6 +43,7 @@ import org.qortal.api.websocket.ActiveChatsWebSocket;
import org.qortal.api.websocket.AdminStatusWebSocket;
import org.qortal.api.websocket.BlocksWebSocket;
import org.qortal.api.websocket.ChatMessagesWebSocket;
import org.qortal.api.websocket.PresenceWebSocket;
import org.qortal.api.websocket.TradeBotWebSocket;
import org.qortal.api.websocket.TradeOffersWebSocket;
import org.qortal.settings.Settings;
@@ -200,6 +201,7 @@ public class ApiService {
context.addServlet(ChatMessagesWebSocket.class, "/websockets/chat/messages");
context.addServlet(TradeOffersWebSocket.class, "/websockets/crosschain/tradeoffers");
context.addServlet(TradeBotWebSocket.class, "/websockets/crosschain/tradebot");
context.addServlet(PresenceWebSocket.class, "/websockets/presence");
// Start server
this.server.start();

View File

@@ -2,7 +2,7 @@ package org.qortal.api;
import javax.xml.bind.annotation.adapters.XmlAdapter;
import org.bitcoinj.core.Base58;
import org.qortal.utils.Base58;
public class Base58TypeAdapter extends XmlAdapter<String, byte[]> {

View File

@@ -5,10 +5,20 @@ import java.net.UnknownHostException;
import javax.servlet.http.HttpServletRequest;
public class Security {
import org.qortal.settings.Settings;
public abstract class Security {
public static final String API_KEY_HEADER = "X-API-KEY";
// TODO: replace with proper authentication
public static void checkApiCallAllowed(HttpServletRequest request) {
String expectedApiKey = Settings.getInstance().getApiKey();
String passedApiKey = request.getHeader(API_KEY_HEADER);
if ((expectedApiKey != null && !expectedApiKey.equals(passedApiKey)) ||
(passedApiKey != null && !passedApiKey.equals(expectedApiKey)))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.UNAUTHORIZED);
InetAddress remoteAddr;
try {
remoteAddr = InetAddress.getByName(request.getRemoteAddr());
@@ -19,4 +29,5 @@ public class Security {
if (!remoteAddr.isLoopbackAddress())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.UNAUTHORIZED);
}
}

View File

@@ -1,5 +1,6 @@
package org.qortal.api.model;
import java.util.Collections;
import java.util.EnumMap;
import java.util.Map;
@@ -13,17 +14,61 @@ import org.qortal.transaction.Transaction.TransactionType;
@XmlAccessorType(XmlAccessType.FIELD)
public class ActivitySummary {
public int blockCount;
public int transactionCount;
public int assetsIssued;
public int namesRegistered;
private int blockCount;
private int assetsIssued;
private int namesRegistered;
// Assuming TransactionType values are contiguous so 'length' equals count
@XmlJavaTypeAdapter(TransactionCountMapXmlAdapter.class)
public Map<TransactionType, Integer> transactionCountByType = new EnumMap<>(TransactionType.class);
private Map<TransactionType, Integer> transactionCountByType = new EnumMap<>(TransactionType.class);
private int totalTransactionCount = 0;
public ActivitySummary() {
// Needed for JAXB
}
public int getBlockCount() {
return this.blockCount;
}
public void setBlockCount(int blockCount) {
this.blockCount = blockCount;
}
public int getTotalTransactionCount() {
return this.totalTransactionCount;
}
public int getAssetsIssued() {
return this.assetsIssued;
}
public void setAssetsIssued(int assetsIssued) {
this.assetsIssued = assetsIssued;
}
public int getNamesRegistered() {
return this.namesRegistered;
}
public void setNamesRegistered(int namesRegistered) {
this.namesRegistered = namesRegistered;
}
public Map<TransactionType, Integer> getTransactionCountByType() {
return Collections.unmodifiableMap(this.transactionCountByType);
}
public void setTransactionCountByType(TransactionType transactionType, int transactionCount) {
this.transactionCountByType.put(transactionType, transactionCount);
this.totalTransactionCount = this.transactionCountByType.values().stream().mapToInt(Integer::intValue).sum();
}
public void setTransactionCountByType(Map<TransactionType, Integer> transactionCountByType) {
this.transactionCountByType = new EnumMap<>(transactionCountByType);
this.totalTransactionCount = this.transactionCountByType.values().stream().mapToInt(Integer::intValue).sum();
}
}

View File

@@ -1,71 +0,0 @@
package org.qortal.api.model;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import org.qortal.data.account.RewardShareData;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@XmlAccessorType(XmlAccessType.FIELD)
public class BlockInfo {
private byte[] signature;
private int height;
private long timestamp;
private int transactionCount;
private String minterAddress;
protected BlockInfo() {
/* For JAXB */
}
public BlockInfo(byte[] signature, int height, long timestamp, int transactionCount, String minterAddress) {
this.signature = signature;
this.height = height;
this.timestamp = timestamp;
this.transactionCount = transactionCount;
this.minterAddress = minterAddress;
}
public BlockInfo(BlockData blockData) {
// Convert BlockData to BlockInfo, using additional data
this.minterAddress = "unknown?";
try (final Repository repository = RepositoryManager.getRepository()) {
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(blockData.getMinterPublicKey());
if (rewardShareData != null)
this.minterAddress = rewardShareData.getMintingAccount();
} catch (DataException e) {
// We'll carry on with placeholder minterAddress then...
}
this.signature = blockData.getSignature();
this.height = blockData.getHeight();
this.timestamp = blockData.getTimestamp();
this.transactionCount = blockData.getTransactionCount();
}
public byte[] getSignature() {
return this.signature;
}
public int getHeight() {
return this.height;
}
public long getTimestamp() {
return this.timestamp;
}
public int getTransactionCount() {
return this.transactionCount;
}
public String getMinterAddress() {
return this.minterAddress;
}
}

View File

@@ -0,0 +1,23 @@
package org.qortal.api.model;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import java.math.BigDecimal;
import java.math.BigInteger;
@XmlAccessorType(XmlAccessType.FIELD)
public class BlockMintingInfo {
public byte[] minterPublicKey;
public int minterLevel;
public int onlineAccountsCount;
public BigDecimal maxDistance;
public BigInteger keyDistance;
public double keyDistanceRatio;
public long timestamp;
public long timeDelta;
public BlockMintingInfo() {
}
}

View File

@@ -1,61 +1,74 @@
package org.qortal.api.model;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import io.swagger.v3.oas.annotations.media.Schema;
import org.qortal.data.network.PeerChainTipData;
import org.qortal.data.network.PeerData;
import org.qortal.network.Handshake;
import org.qortal.network.Peer;
import io.swagger.v3.oas.annotations.media.Schema;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
@XmlAccessorType(XmlAccessType.FIELD)
public class ConnectedPeer {
public enum Direction {
INBOUND,
OUTBOUND;
}
public Direction direction;
public Handshake handshakeStatus;
public Long lastPing;
public Long connectedWhen;
public Long peersConnectedWhen;
public enum Direction {
INBOUND,
OUTBOUND;
}
public String address;
public String version;
public Direction direction;
public Handshake handshakeStatus;
public Long lastPing;
public Long connectedWhen;
public Long peersConnectedWhen;
public String nodeId;
public String address;
public String version;
public Integer lastHeight;
@Schema(example = "base58")
public byte[] lastBlockSignature;
public Long lastBlockTimestamp;
public String nodeId;
protected ConnectedPeer() {
}
public Integer lastHeight;
@Schema(example = "base58")
public byte[] lastBlockSignature;
public Long lastBlockTimestamp;
public UUID connectionId;
public String age;
public ConnectedPeer(Peer peer) {
this.direction = peer.isOutbound() ? Direction.OUTBOUND : Direction.INBOUND;
this.handshakeStatus = peer.getHandshakeStatus();
this.lastPing = peer.getLastPing();
protected ConnectedPeer() {
}
PeerData peerData = peer.getPeerData();
this.connectedWhen = peer.getConnectionTimestamp();
this.peersConnectedWhen = peer.getPeersConnectionTimestamp();
public ConnectedPeer(Peer peer) {
this.direction = peer.isOutbound() ? Direction.OUTBOUND : Direction.INBOUND;
this.handshakeStatus = peer.getHandshakeStatus();
this.lastPing = peer.getLastPing();
this.address = peerData.getAddress().toString();
PeerData peerData = peer.getPeerData();
this.connectedWhen = peer.getConnectionTimestamp();
this.peersConnectedWhen = peer.getPeersConnectionTimestamp();
this.version = peer.getPeersVersionString();
this.nodeId = peer.getPeersNodeId();
this.address = peerData.getAddress().toString();
PeerChainTipData peerChainTipData = peer.getChainTipData();
if (peerChainTipData != null) {
this.lastHeight = peerChainTipData.getLastHeight();
this.lastBlockSignature = peerChainTipData.getLastBlockSignature();
this.lastBlockTimestamp = peerChainTipData.getLastBlockTimestamp();
}
}
this.version = peer.getPeersVersionString();
this.nodeId = peer.getPeersNodeId();
this.connectionId = peer.getPeerConnectionId();
if (peer.getConnectionEstablishedTime() > 0) {
long age = (System.currentTimeMillis() - peer.getConnectionEstablishedTime());
long minutes = TimeUnit.MILLISECONDS.toMinutes(age);
long seconds = TimeUnit.MILLISECONDS.toSeconds(age) - TimeUnit.MINUTES.toSeconds(minutes);
this.age = String.format("%dm %ds", minutes, seconds);
} else {
this.age = "connecting...";
}
PeerChainTipData peerChainTipData = peer.getChainTipData();
if (peerChainTipData != null) {
this.lastHeight = peerChainTipData.getLastHeight();
this.lastBlockSignature = peerChainTipData.getLastBlockSignature();
this.lastBlockTimestamp = peerChainTipData.getLastBlockTimestamp();
}
}
}

View File

@@ -1,31 +0,0 @@
package org.qortal.api.model;
import java.math.BigDecimal;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class CrossChainBitcoinP2SHStatus {
@Schema(description = "Bitcoin P2SH address", example = "3CdH27kTpV8dcFHVRYjQ8EEV5FJg9X8pSJ (mainnet), 2fMiRRXVsxhZeyfum9ifybZvaMHbQTmwdZw (testnet)")
public String bitcoinP2shAddress;
@Schema(description = "Bitcoin P2SH balance")
public BigDecimal bitcoinP2shBalance;
@Schema(description = "Can P2SH redeem yet?")
public boolean canRedeem;
@Schema(description = "Can P2SH refund yet?")
public boolean canRefund;
@Schema(description = "Secret extracted by P2SH redeemer")
public byte[] secret;
public CrossChainBitcoinP2SHStatus() {
}
}

View File

@@ -22,6 +22,9 @@ public class CrossChainBitcoinRefundRequest {
@Schema(description = "Bitcoin miner fee", example = "0.00001000")
public BigDecimal bitcoinMinerFee;
@Schema(description = "Bitcoin HASH160(public key) for receiving funds, or omit to derive from private key", example = "u17kBVKkKSp12oUzaxFwNnq1JZf")
public byte[] receivingAccountInfo;
public CrossChainBitcoinRefundRequest() {
}

View File

@@ -0,0 +1,31 @@
package org.qortal.api.model;
import java.math.BigDecimal;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class CrossChainBitcoinyHTLCStatus {
@Schema(description = "P2SH address", example = "3CdH27kTpV8dcFHVRYjQ8EEV5FJg9X8pSJ (mainnet), 2fMiRRXVsxhZeyfum9ifybZvaMHbQTmwdZw (testnet)")
public String bitcoinP2shAddress;
@Schema(description = "P2SH balance")
public BigDecimal bitcoinP2shBalance;
@Schema(description = "Can HTLC redeem yet?")
public boolean canRedeem;
@Schema(description = "Can HTLC refund yet?")
public boolean canRefund;
@Schema(description = "Secret used by HTLC redeemer")
public byte[] secret;
public CrossChainBitcoinyHTLCStatus() {
}
}

View File

@@ -0,0 +1,29 @@
package org.qortal.api.model;
import io.swagger.v3.oas.annotations.media.Schema;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
@XmlAccessorType(XmlAccessType.FIELD)
public class CrossChainDualSecretRequest {
@Schema(description = "Public key to match AT's trade 'partner'", example = "C6wuddsBV3HzRrXUtezE7P5MoRXp5m3mEDokRDGZB6ry")
public byte[] partnerPublicKey;
@Schema(description = "Qortal AT address")
public String atAddress;
@Schema(description = "secret-A (32 bytes)", example = "FHMzten4he9jZ4HGb4297Utj6F5g2w7serjq2EnAg2s1")
public byte[] secretA;
@Schema(description = "secret-B (32 bytes)", example = "EN2Bgx3BcEMtxFCewmCVSMkfZjVKYhx3KEXC5A21KBGx")
public byte[] secretB;
@Schema(description = "Qortal address for receiving QORT from AT")
public String receivingAddress;
public CrossChainDualSecretRequest() {
}
}

View File

@@ -4,7 +4,7 @@ import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import org.qortal.crosschain.BTCACCT;
import org.qortal.crosschain.AcctMode;
import org.qortal.data.crosschain.CrossChainTradeData;
import io.swagger.v3.oas.annotations.media.Schema;
@@ -16,26 +16,41 @@ public class CrossChainOfferSummary {
// Properties
@Schema(description = "AT's Qortal address")
public String qortalAtAddress;
private String qortalAtAddress;
@Schema(description = "AT creator's Qortal address")
public String qortalCreator;
private String qortalCreator;
@Schema(description = "AT creator's ephemeral trading key-pair represented as Qortal address")
private String qortalCreatorTradeAddress;
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
private long qortAmount;
@Schema(description = "Bitcoin amount - DEPRECATED: use foreignAmount")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
@Deprecated
private long btcAmount;
@Schema(description = "Foreign blockchain amount")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
private long foreignAmount;
@Schema(description = "Suggested trade timeout (minutes)", example = "10080")
private int tradeTimeout;
private BTCACCT.Mode mode;
@Schema(description = "Current AT execution mode")
private AcctMode mode;
private long timestamp;
@Schema(description = "Trade partner's Qortal receiving address")
private String partnerQortalReceivingAddress;
private String foreignBlockchain;
private String acctName;
protected CrossChainOfferSummary() {
/* For JAXB */
}
@@ -43,12 +58,16 @@ public class CrossChainOfferSummary {
public CrossChainOfferSummary(CrossChainTradeData crossChainTradeData, long timestamp) {
this.qortalAtAddress = crossChainTradeData.qortalAtAddress;
this.qortalCreator = crossChainTradeData.qortalCreator;
this.qortalCreatorTradeAddress = crossChainTradeData.qortalCreatorTradeAddress;
this.qortAmount = crossChainTradeData.qortAmount;
this.btcAmount = crossChainTradeData.expectedBitcoin;
this.foreignAmount = crossChainTradeData.expectedForeignAmount;
this.btcAmount = this.foreignAmount; // Duplicate for deprecated field
this.tradeTimeout = crossChainTradeData.tradeTimeout;
this.mode = crossChainTradeData.mode;
this.timestamp = timestamp;
this.partnerQortalReceivingAddress = crossChainTradeData.qortalPartnerReceivingAddress;
this.foreignBlockchain = crossChainTradeData.foreignBlockchain;
this.acctName = crossChainTradeData.acctName;
}
public String getQortalAtAddress() {
@@ -59,6 +78,10 @@ public class CrossChainOfferSummary {
return this.qortalCreator;
}
public String getQortalCreatorTradeAddress() {
return this.qortalCreatorTradeAddress;
}
public long getQortAmount() {
return this.qortAmount;
}
@@ -67,11 +90,15 @@ public class CrossChainOfferSummary {
return this.btcAmount;
}
public long getForeignAmount() {
return this.foreignAmount;
}
public int getTradeTimeout() {
return this.tradeTimeout;
}
public BTCACCT.Mode getMode() {
public AcctMode getMode() {
return this.mode;
}
@@ -83,4 +110,18 @@ public class CrossChainOfferSummary {
return this.partnerQortalReceivingAddress;
}
public String getForeignBlockchain() {
return this.foreignBlockchain;
}
public String getAcctName() {
return this.acctName;
}
// For debugging mostly
public String toString() {
return String.format("%s: %s", this.qortalAtAddress, this.mode);
}
}

View File

@@ -8,17 +8,14 @@ import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class CrossChainSecretRequest {
@Schema(description = "Public key to match AT's trade 'partner'", example = "C6wuddsBV3HzRrXUtezE7P5MoRXp5m3mEDokRDGZB6ry")
public byte[] partnerPublicKey;
@Schema(description = "Private key to match AT's trade 'partner'", example = "C6wuddsBV3HzRrXUtezE7P5MoRXp5m3mEDokRDGZB6ry")
public byte[] partnerPrivateKey;
@Schema(description = "Qortal AT address")
public String atAddress;
@Schema(description = "secret-A (32 bytes)", example = "FHMzten4he9jZ4HGb4297Utj6F5g2w7serjq2EnAg2s1")
public byte[] secretA;
@Schema(description = "secret-B (32 bytes)", example = "EN2Bgx3BcEMtxFCewmCVSMkfZjVKYhx3KEXC5A21KBGx")
public byte[] secretB;
@Schema(description = "Secret (32 bytes)", example = "FHMzten4he9jZ4HGb4297Utj6F5g2w7serjq2EnAg2s1")
public byte[] secret;
@Schema(description = "Qortal address for receiving QORT from AT")
public String receivingAddress;

View File

@@ -6,6 +6,8 @@ import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import org.qortal.data.crosschain.CrossChainTradeData;
import io.swagger.v3.oas.annotations.media.Schema;
// All properties to be converted to JSON via JAXB
@XmlAccessorType(XmlAccessType.FIELD)
public class CrossChainTradeSummary {
@@ -15,9 +17,20 @@ public class CrossChainTradeSummary {
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
private long qortAmount;
@Deprecated
@Schema(description = "DEPRECATED: use foreignAmount instead")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
private long btcAmount;
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
private long foreignAmount;
private String atAddress;
private String sellerAddress;
private String buyerReceivingAddress;
protected CrossChainTradeSummary() {
/* For JAXB */
}
@@ -25,7 +38,11 @@ public class CrossChainTradeSummary {
public CrossChainTradeSummary(CrossChainTradeData crossChainTradeData, long timestamp) {
this.tradeTimestamp = timestamp;
this.qortAmount = crossChainTradeData.qortAmount;
this.btcAmount = crossChainTradeData.expectedBitcoin;
this.foreignAmount = crossChainTradeData.expectedForeignAmount;
this.btcAmount = this.foreignAmount;
this.sellerAddress = crossChainTradeData.qortalCreator;
this.buyerReceivingAddress = crossChainTradeData.qortalPartnerReceivingAddress;
this.atAddress = crossChainTradeData.qortalAtAddress;
}
public long getTradeTimestamp() {
@@ -40,4 +57,11 @@ public class CrossChainTradeSummary {
return this.btcAmount;
}
public long getForeignAmount() { return this.foreignAmount; }
public String getAtAddress() { return this.atAddress; }
public String getSellerAddress() { return this.sellerAddress; }
public String getBuyerReceivingAddressAddress() { return this.buyerReceivingAddress; }
}

View File

@@ -11,6 +11,7 @@ public class NodeInfo {
public String buildVersion;
public long buildTimestamp;
public String nodeId;
public boolean isTestNet;
public NodeInfo() {
}

View File

@@ -20,17 +20,14 @@ public class NodeStatus {
public final int height;
public NodeStatus() {
isMintingPossible = Controller.getInstance().isMintingPossible();
isSynchronizing = Controller.getInstance().isSynchronizing();
this.isMintingPossible = Controller.getInstance().isMintingPossible();
if (isSynchronizing)
syncPercent = Controller.getInstance().getSyncPercent();
else
syncPercent = null;
this.syncPercent = Controller.getInstance().getSyncPercent();
this.isSynchronizing = this.syncPercent != null;
numberOfConnections = Network.getInstance().getHandshakedPeers().size();
this.numberOfConnections = Network.getInstance().getHandshakedPeers().size();
height = Controller.getInstance().getChainHeight();
this.height = Controller.getInstance().getChainHeight();
}
}

View File

@@ -0,0 +1,157 @@
package org.qortal.api.model;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Objects;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
@XmlAccessorType(XmlAccessType.FIELD)
public class SimpleForeignTransaction {
public static class AddressAmount {
public final String address;
public final long amount;
protected AddressAmount() {
/* For JAXB */
this.address = null;
this.amount = 0;
}
public AddressAmount(String address, long amount) {
this.address = address;
this.amount = amount;
}
}
private String txHash;
private long timestamp;
private List<AddressAmount> inputs;
public static class Output {
public final List<String> addresses;
public final long amount;
protected Output() {
/* For JAXB */
this.addresses = null;
this.amount = 0;
}
public Output(List<String> addresses, long amount) {
this.addresses = addresses;
this.amount = amount;
}
}
private List<Output> outputs;
private long totalAmount;
private long fees;
private Boolean isSentNotReceived;
protected SimpleForeignTransaction() {
/* For JAXB */
}
private SimpleForeignTransaction(Builder builder) {
this.txHash = builder.txHash;
this.timestamp = builder.timestamp;
this.inputs = Collections.unmodifiableList(builder.inputs);
this.outputs = Collections.unmodifiableList(builder.outputs);
Objects.requireNonNull(this.txHash);
if (timestamp <= 0)
throw new IllegalArgumentException("timestamp must be positive");
long totalGrossAmount = this.inputs.stream().map(addressAmount -> addressAmount.amount).reduce(0L, Long::sum);
this.totalAmount = this.outputs.stream().map(addressAmount -> addressAmount.amount).reduce(0L, Long::sum);
this.fees = totalGrossAmount - this.totalAmount;
this.isSentNotReceived = builder.isSentNotReceived;
}
public String getTxHash() {
return this.txHash;
}
public long getTimestamp() {
return this.timestamp;
}
public List<AddressAmount> getInputs() {
return this.inputs;
}
public List<Output> getOutputs() {
return this.outputs;
}
public long getTotalAmount() {
return this.totalAmount;
}
public long getFees() {
return this.fees;
}
public Boolean isSentNotReceived() {
return this.isSentNotReceived;
}
public static class Builder {
private String txHash;
private long timestamp;
private List<AddressAmount> inputs = new ArrayList<>();
private List<Output> outputs = new ArrayList<>();
private Boolean isSentNotReceived;
public Builder txHash(String txHash) {
this.txHash = Objects.requireNonNull(txHash);
return this;
}
public Builder timestamp(long timestamp) {
if (timestamp <= 0)
throw new IllegalArgumentException("timestamp must be positive");
this.timestamp = timestamp;
return this;
}
public Builder input(String address, long amount) {
Objects.requireNonNull(address);
if (amount < 0)
throw new IllegalArgumentException("amount must be zero or positive");
AddressAmount input = new AddressAmount(address, amount);
inputs.add(input);
return this;
}
public Builder output(List<String> addresses, long amount) {
Objects.requireNonNull(addresses);
if (amount < 0)
throw new IllegalArgumentException("amount must be zero or positive");
Output output = new Output(addresses, amount);
outputs.add(output);
return this;
}
public Builder isSentNotReceived(Boolean isSentNotReceived) {
this.isSentNotReceived = isSentNotReceived;
return this;
}
public SimpleForeignTransaction build() {
return new SimpleForeignTransaction(this);
}
}
}

View File

@@ -1,23 +0,0 @@
package org.qortal.api.model;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class TradeBotRespondRequest {
@Schema(description = "Qortal AT address", example = "AH3e3jHEsGHPVQPDiJx4pYqgVi72auxgVy")
public String atAddress;
@Schema(description = "Bitcoin BIP32 extended private key", example = "tprv8ZgxMBicQKsPdahhFSrCdvC1bsWyzHHZfTneTVqUXN6s1wEtZLwAkZXzFP6TbTVGajEB55L1HYLg2aQMecZLXLre5YJcawpdFG66STVAWPJ")
public String xprv58;
@Schema(description = "Qortal address for receiving QORT from AT")
public String receivingAddress;
public TradeBotRespondRequest() {
}
}

View File

@@ -1,4 +1,4 @@
package org.qortal.api.model;
package org.qortal.api.model.crosschain;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
@@ -9,16 +9,20 @@ import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class BitcoinSendRequest {
@Schema(description = "Bitcoin BIP32 extended private key", example = "tprv8ZgxMBicQKsPdahhFSrCdvC1bsWyzHHZfTneTVqUXN6s1wEtZLwAkZXzFP6TbTVGajEB55L1HYLg2aQMecZLXLre5YJcawpdFG66STVAWPJ")
@Schema(description = "Bitcoin BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
public String xprv58;
@Schema(description = "Recipient's Bitcoin address ('legacy' P2PKH only)", example = "1BitcoinEaterAddressDontSendf59kuE")
public String receivingAddress;
@Schema(description = "Amount of BTC to send")
@Schema(description = "Amount of BTC to send", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long bitcoinAmount;
@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 BTC (100 sats) per byte", example = "0.00000100", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public Long feePerByte;
public BitcoinSendRequest() {
}

View File

@@ -0,0 +1,29 @@
package org.qortal.api.model.crosschain;
import io.swagger.v3.oas.annotations.media.Schema;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
@XmlAccessorType(XmlAccessType.FIELD)
public class DogecoinSendRequest {
@Schema(description = "Dogecoin BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
public String xprv58;
@Schema(description = "Recipient's Dogecoin address ('legacy' P2PKH only)", example = "DoGecoinEaterAddressDontSendhLfzKD")
public String receivingAddress;
@Schema(description = "Amount of DOGE to send", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long dogecoinAmount;
@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 DOGE (100 sats) per byte", example = "0.00000100", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public Long feePerByte;
public DogecoinSendRequest() {
}
}

View File

@@ -0,0 +1,29 @@
package org.qortal.api.model.crosschain;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class LitecoinSendRequest {
@Schema(description = "Litecoin BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
public String xprv58;
@Schema(description = "Recipient's Litecoin address ('legacy' P2PKH only)", example = "LiTecoinEaterAddressDontSendhLfzKD")
public String receivingAddress;
@Schema(description = "Amount of LTC to send", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long litecoinAmount;
@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 LTC (100 sats) per byte", example = "0.00000100", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public Long feePerByte;
public LitecoinSendRequest() {
}
}

View File

@@ -1,9 +1,11 @@
package org.qortal.api.model;
package org.qortal.api.model.crosschain;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import org.qortal.crosschain.SupportedBlockchain;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
@@ -12,22 +14,30 @@ public class TradeBotCreateRequest {
@Schema(description = "Trade creator's public key", example = "2zR1WFsbM7akHghqSCYKBPk6LDP8aKiQSRS1FrwoLvoB")
public byte[] creatorPublicKey;
@Schema(description = "QORT amount paid out on successful trade", example = "80.40200000")
@Schema(description = "QORT amount paid out on successful trade", example = "80.40000000", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long qortAmount;
@Schema(description = "QORT amount funding AT, including covering AT execution fees", example = "81")
@Schema(description = "QORT amount funding AT, including covering AT execution fees", example = "80.50000000", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long fundingQortAmount;
@Schema(description = "Bitcoin amount wanted in return", example = "0.00864200")
@Deprecated
@Schema(description = "Bitcoin amount wanted in return. DEPRECATED: use foreignAmount instead", example = "0.00864200", type = "number", hidden = true)
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long bitcoinAmount;
public Long bitcoinAmount;
@Schema(description = "Foreign blockchain. Note: default (BITCOIN) to be removed in the future", example = "BITCOIN", implementation = SupportedBlockchain.class)
public SupportedBlockchain foreignBlockchain;
@Schema(description = "Foreign blockchain amount wanted in return", example = "0.00864200", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public Long foreignAmount;
@Schema(description = "Suggested trade timeout (minutes)", example = "10080")
public int tradeTimeout;
@Schema(description = "Bitcoin address for receiving", example = "1BitcoinEaterAddressDontSendf59kuE")
@Schema(description = "Foreign blockchain address for receiving", example = "1BitcoinEaterAddressDontSendf59kuE")
public String receivingAddress;
public TradeBotCreateRequest() {

View File

@@ -0,0 +1,29 @@
package org.qortal.api.model.crosschain;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class TradeBotRespondRequest {
@Schema(description = "Qortal AT address", example = "Aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
public String atAddress;
@Deprecated
@Schema(description = "Bitcoin BIP32 extended private key. DEPRECATED: use foreignKey instead", hidden = true,
example = "xprv___________________________________________________________________________________________________________")
public String xprv58;
@Schema(description = "Foreign blockchain private key, e.g. BIP32 'm' key for Bitcoin/Litecoin starting with 'xprv'",
example = "xprv___________________________________________________________________________________________________________")
public String foreignKey;
@Schema(description = "Qortal address for receiving QORT from AT", example = "Qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq")
public String receivingAddress;
public TradeBotRespondRequest() {
}
}

View File

@@ -7,6 +7,7 @@ import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.math.BigDecimal;
@@ -473,6 +474,7 @@ public class AddressesResource {
}
)
@ApiErrors({ApiError.TRANSACTION_INVALID, ApiError.INVALID_DATA, ApiError.TRANSFORMATION_ERROR, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String computePublicize(String rawBytes58) {
Security.checkApiCallAllowed(request);

View File

@@ -8,6 +8,7 @@ import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.io.IOException;
@@ -40,7 +41,6 @@ import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiException;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.ActivitySummary;
@@ -57,6 +57,7 @@ import org.qortal.network.PeerAddress;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
@@ -118,6 +119,7 @@ public class AdminResource {
nodeInfo.buildVersion = Controller.getInstance().getVersionString();
nodeInfo.buildTimestamp = Controller.getInstance().getBuildTimestamp();
nodeInfo.nodeId = Network.getInstance().getOurNodeId();
nodeInfo.isTestNet = Settings.getInstance().isTestNet();
return nodeInfo;
}
@@ -132,6 +134,7 @@ public class AdminResource {
)
}
)
@SecurityRequirement(name = "apiKey")
public NodeStatus status() {
Security.checkApiCallAllowed(request);
@@ -152,6 +155,7 @@ public class AdminResource {
)
}
)
@SecurityRequirement(name = "apiKey")
public String shutdown() {
Security.checkApiCallAllowed(request);
@@ -180,7 +184,10 @@ public class AdminResource {
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public ActivitySummary summary() {
Security.checkApiCallAllowed(request);
ActivitySummary summary = new ActivitySummary();
LocalDate date = LocalDate.now();
@@ -192,16 +199,13 @@ public class AdminResource {
int startHeight = repository.getBlockRepository().getHeightFromTimestamp(start);
int endHeight = repository.getBlockRepository().getBlockchainHeight();
summary.blockCount = endHeight - startHeight;
summary.setBlockCount(endHeight - startHeight);
summary.transactionCountByType = repository.getTransactionRepository().getTransactionSummary(startHeight + 1, endHeight);
summary.setTransactionCountByType(repository.getTransactionRepository().getTransactionSummary(startHeight + 1, endHeight));
for (Integer count : summary.transactionCountByType.values())
summary.transactionCount += count;
summary.setAssetsIssued(repository.getAssetRepository().getRecentAssetIds(start).size());
summary.assetsIssued = repository.getAssetRepository().getRecentAssetIds(start).size();
summary.namesRegistered = repository.getNameRepository().getRecentNames(start).size();
summary.setNamesRegistered (repository.getNameRepository().getRecentNames(start).size());
return summary;
} catch (DataException e) {
@@ -209,6 +213,30 @@ public class AdminResource {
}
}
@GET
@Path("/enginestats")
@Operation(
summary = "Fetch statistics snapshot for core engine",
responses = {
@ApiResponse(
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
array = @ArraySchema(
schema = @Schema(
implementation = Controller.StatsSnapshot.class
)
)
)
)
}
)
@SecurityRequirement(name = "apiKey")
public Controller.StatsSnapshot getEngineStats() {
Security.checkApiCallAllowed(request);
return Controller.getInstance().getStatsSnapshot();
}
@GET
@Path("/mintingaccounts")
@Operation(
@@ -221,6 +249,7 @@ public class AdminResource {
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public List<MintingAccountData> getMintingAccounts() {
Security.checkApiCallAllowed(request);
@@ -267,6 +296,7 @@ public class AdminResource {
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.REPOSITORY_ISSUE, ApiError.CANNOT_MINT})
@SecurityRequirement(name = "apiKey")
public String addMintingAccount(String seed58) {
Security.checkApiCallAllowed(request);
@@ -319,6 +349,7 @@ public class AdminResource {
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String deleteMintingAccount(String key58) {
Security.checkApiCallAllowed(request);
@@ -418,6 +449,7 @@ public class AdminResource {
}
)
@ApiErrors({ApiError.INVALID_HEIGHT, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String orphan(String targetHeightString) {
Security.checkApiCallAllowed(request);
@@ -435,8 +467,6 @@ public class AdminResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
} catch (NumberFormatException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_HEIGHT);
} catch (ApiException e) {
throw e;
}
}
@@ -461,6 +491,7 @@ public class AdminResource {
}
)
@ApiErrors({ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String forceSync(String targetPeerAddress) {
Security.checkApiCallAllowed(request);
@@ -492,8 +523,6 @@ public class AdminResource {
return syncResult.name();
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
} catch (ApiException e) {
throw e;
} catch (UnknownHostException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
} catch (InterruptedException e) {
@@ -501,4 +530,161 @@ public class AdminResource {
}
}
@GET
@Path("/repository/data")
@Operation(
summary = "Export sensitive/node-local data from repository.",
description = "Exports data to .script files on local machine"
)
@ApiErrors({ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String exportRepository() {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
repository.exportNodeLocalData();
return "true";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@POST
@Path("/repository/data")
@Operation(
summary = "Import data into repository.",
description = "Imports data from file on local machine. Filename is forced to 'import.json' if apiKey is not set.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string", example = "MintingAccounts.script"
)
)
),
responses = {
@ApiResponse(
description = "\"true\"",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String importRepository(String filename) {
Security.checkApiCallAllowed(request);
// Hard-coded because it's too dangerous to allow user-supplied filenames in weaker security contexts
if (Settings.getInstance().getApiKey() == null)
filename = "import.json";
try (final Repository repository = RepositoryManager.getRepository()) {
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
blockchainLock.lockInterruptibly();
try {
repository.importDataFromFile(filename);
repository.saveChanges();
return "true";
} finally {
blockchainLock.unlock();
}
} catch (InterruptedException e) {
// We couldn't lock blockchain to perform import
return "false";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@POST
@Path("/repository/checkpoint")
@Operation(
summary = "Checkpoint data in repository.",
description = "Forces repository to checkpoint uncommitted writes.",
responses = {
@ApiResponse(
description = "\"true\"",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String checkpointRepository() {
Security.checkApiCallAllowed(request);
RepositoryManager.setRequestedCheckpoint(Boolean.TRUE);
return "true";
}
@POST
@Path("/repository/backup")
@Operation(
summary = "Perform online backup of repository.",
responses = {
@ApiResponse(
description = "\"true\"",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String backupRepository() {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
blockchainLock.lockInterruptibly();
try {
repository.backup(true);
repository.saveChanges();
return "true";
} finally {
blockchainLock.unlock();
}
} catch (InterruptedException e) {
// We couldn't lock blockchain to perform backup
return "false";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@DELETE
@Path("/repository")
@Operation(
summary = "Perform maintenance on repository.",
description = "Requires enough free space to rebuild repository. This will pause your node for a while."
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public void performRepositoryMaintenance() {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
blockchainLock.lockInterruptibly();
try {
repository.performPeriodicMaintenance();
} finally {
blockchainLock.unlock();
}
} catch (InterruptedException e) {
// No big deal
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
}

View File

@@ -1,11 +1,17 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.OpenAPIDefinition;
import io.swagger.v3.oas.annotations.enums.SecuritySchemeIn;
import io.swagger.v3.oas.annotations.enums.SecuritySchemeType;
import io.swagger.v3.oas.annotations.extensions.Extension;
import io.swagger.v3.oas.annotations.extensions.ExtensionProperty;
import io.swagger.v3.oas.annotations.info.Info;
import io.swagger.v3.oas.annotations.security.SecurityScheme;
import io.swagger.v3.oas.annotations.security.SecuritySchemes;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.qortal.api.Security;
@OpenAPIDefinition(
info = @Info( title = "Qortal API", description = "NOTE: byte-arrays are encoded in Base58" ),
tags = {
@@ -30,5 +36,9 @@ import io.swagger.v3.oas.annotations.tags.Tag;
})
}
)
@SecuritySchemes({
@SecurityScheme(name = "basicAuth", type = SecuritySchemeType.HTTP, scheme = "basic"),
@SecurityScheme(name = "apiKey", type = SecuritySchemeType.APIKEY, in = SecuritySchemeIn.HEADER, paramName = Security.API_KEY_HEADER)
})
public class ApiDefinition {
}

View File

@@ -1,5 +1,6 @@
package org.qortal.api.resource;
import com.google.common.primitives.Ints;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.ArraySchema;
@@ -8,6 +9,11 @@ import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.math.RoundingMode;
import java.util.ArrayList;
import java.util.List;
@@ -20,11 +26,13 @@ import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.qortal.account.Account;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.model.BlockInfo;
import org.qortal.api.model.BlockMintingInfo;
import org.qortal.api.model.BlockSignerSummary;
import org.qortal.block.Block;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountData;
import org.qortal.data.block.BlockData;
@@ -33,6 +41,8 @@ import org.qortal.data.transaction.TransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transform.TransformationException;
import org.qortal.transform.block.BlockTransformer;
import org.qortal.utils.Base58;
@Path("/blocks")
@@ -81,6 +91,48 @@ public class BlocksResource {
}
}
@GET
@Path("/signature/{signature}/data")
@Operation(
summary = "Fetch serialized, base58 encoded block data using base58 signature",
description = "Returns serialized data for the block that matches the given signature",
responses = {
@ApiResponse(
description = "the block data",
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({
ApiError.INVALID_SIGNATURE, ApiError.BLOCK_UNKNOWN, ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE
})
public String getSerializedBlockData(@PathParam("signature") String signature58) {
// Decode signature
byte[] signature;
try {
signature = Base58.decode(signature58);
} catch (NumberFormatException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_SIGNATURE, e);
}
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
Block block = new Block(repository, blockData);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
bytes.write(Ints.toByteArray(block.getBlockData().getHeight()));
bytes.write(BlockTransformer.toBytes(block));
return Base58.encode(bytes.toByteArray());
} catch (TransformationException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA, e);
} catch (DataException | IOException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@GET
@Path("/signature/{signature}/transactions")
@Operation(
@@ -329,6 +381,59 @@ public class BlocksResource {
}
}
@GET
@Path("/byheight/{height}/mintinginfo")
@Operation(
summary = "Fetch block minter info using block height",
description = "Returns the minter info for the block with given height",
responses = {
@ApiResponse(
description = "the block",
content = @Content(
schema = @Schema(
implementation = BlockData.class
)
)
)
}
)
@ApiErrors({
ApiError.BLOCK_UNKNOWN, ApiError.REPOSITORY_ISSUE
})
public BlockMintingInfo getBlockMintingInfoByHeight(@PathParam("height") int height) {
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData blockData = repository.getBlockRepository().fromHeight(height);
if (blockData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
Block block = new Block(repository, blockData);
BlockData parentBlockData = repository.getBlockRepository().fromSignature(blockData.getReference());
int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, blockData.getMinterPublicKey());
if (minterLevel == 0)
// This may be unavailable when requesting a trimmed block
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
BigInteger distance = block.calcKeyDistance(parentBlockData.getHeight(), parentBlockData.getSignature(), blockData.getMinterPublicKey(), minterLevel);
double ratio = new BigDecimal(distance).divide(new BigDecimal(block.MAX_DISTANCE), 40, RoundingMode.DOWN).doubleValue();
long timestamp = block.calcTimestamp(parentBlockData, blockData.getMinterPublicKey(), minterLevel);
long timeDelta = timestamp - parentBlockData.getTimestamp();
BlockMintingInfo blockMintingInfo = new BlockMintingInfo();
blockMintingInfo.minterPublicKey = blockData.getMinterPublicKey();
blockMintingInfo.minterLevel = minterLevel;
blockMintingInfo.onlineAccountsCount = blockData.getOnlineAccountsCount();
blockMintingInfo.maxDistance = new BigDecimal(block.MAX_DISTANCE);
blockMintingInfo.keyDistance = distance;
blockMintingInfo.keyDistanceRatio = ratio;
blockMintingInfo.timestamp = timestamp;
blockMintingInfo.timeDelta = timeDelta;
return blockMintingInfo;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@GET
@Path("/timestamp/{timestamp}")
@Operation(
@@ -492,7 +597,7 @@ public class BlocksResource {
content = @Content(
array = @ArraySchema(
schema = @Schema(
implementation = BlockInfo.class
implementation = BlockSummaryData.class
)
)
)
@@ -502,7 +607,7 @@ public class BlocksResource {
@ApiErrors({
ApiError.REPOSITORY_ISSUE
})
public List<BlockInfo> getBlockRange(
public List<BlockSummaryData> getBlockSummaries(
@QueryParam("start") Integer startHeight,
@QueryParam("end") Integer endHeight,
@Parameter(ref = "count") @QueryParam("count") Integer count) {
@@ -515,7 +620,7 @@ public class BlocksResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
try (final Repository repository = RepositoryManager.getRepository()) {
return repository.getBlockRepository().getBlockInfos(startHeight, endHeight, count);
return repository.getBlockRepository().getBlockSummaries(startHeight, endHeight, count);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}

View File

@@ -7,6 +7,7 @@ import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.List;
@@ -156,6 +157,7 @@ public class ChatResource {
}
)
@ApiErrors({ApiError.TRANSACTION_INVALID, ApiError.TRANSFORMATION_ERROR, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String buildChat(ChatTransactionData transactionData) {
Security.checkApiCallAllowed(request);
@@ -203,6 +205,7 @@ public class ChatResource {
}
)
@ApiErrors({ApiError.TRANSACTION_INVALID, ApiError.INVALID_DATA, ApiError.TRANSFORMATION_ERROR, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public String buildChat(String rawBytes58) {
Security.checkApiCallAllowed(request);

View File

@@ -0,0 +1,363 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.Arrays;
import java.util.Random;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.CrossChainBuildRequest;
import org.qortal.api.model.CrossChainDualSecretRequest;
import org.qortal.api.model.CrossChainTradeRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.BitcoinACCTv1;
import org.qortal.crosschain.Bitcoiny;
import org.qortal.crosschain.AcctMode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction;
import org.qortal.transaction.Transaction.TransactionType;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.Transformer;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.transform.transaction.MessageTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
@Path("/crosschain/BitcoinACCTv1")
@Tag(name = "Cross-Chain (BitcoinACCTv1)")
public class CrossChainBitcoinACCTv1Resource {
@Context
HttpServletRequest request;
@POST
@Path("/build")
@Operation(
summary = "Build Bitcoin cross-chain trading AT",
description = "Returns raw, unsigned DEPLOY_AT transaction",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = CrossChainBuildRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_DATA, ApiError.INVALID_REFERENCE, ApiError.TRANSFORMATION_ERROR, ApiError.REPOSITORY_ISSUE})
public String buildTrade(CrossChainBuildRequest tradeRequest) {
Security.checkApiCallAllowed(request);
byte[] creatorPublicKey = tradeRequest.creatorPublicKey;
if (creatorPublicKey == null || creatorPublicKey.length != Transformer.PUBLIC_KEY_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
if (tradeRequest.hashOfSecretB == null || tradeRequest.hashOfSecretB.length != Bitcoiny.HASH160_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (tradeRequest.tradeTimeout == null)
tradeRequest.tradeTimeout = 7 * 24 * 60; // 7 days
else
if (tradeRequest.tradeTimeout < 10 || tradeRequest.tradeTimeout > 50000)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (tradeRequest.qortAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (tradeRequest.fundingQortAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
// funding amount must exceed initial + final
if (tradeRequest.fundingQortAmount <= tradeRequest.qortAmount)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (tradeRequest.bitcoinAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
try (final Repository repository = RepositoryManager.getRepository()) {
PublicKeyAccount creatorAccount = new PublicKeyAccount(repository, creatorPublicKey);
byte[] creationBytes = BitcoinACCTv1.buildQortalAT(creatorAccount.getAddress(), tradeRequest.bitcoinPublicKeyHash, tradeRequest.hashOfSecretB,
tradeRequest.qortAmount, tradeRequest.bitcoinAmount, tradeRequest.tradeTimeout);
long txTimestamp = NTP.getTime();
byte[] lastReference = creatorAccount.getLastReference();
if (lastReference == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_REFERENCE);
long fee = 0;
String name = "QORT-BTC cross-chain trade";
String description = "Qortal-Bitcoin cross-chain trade";
String atType = "ACCT";
String tags = "QORT-BTC ACCT";
BaseTransactionData baseTransactionData = new BaseTransactionData(txTimestamp, Group.NO_GROUP, lastReference, creatorAccount.getPublicKey(), fee, null);
TransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, atType, tags, creationBytes, tradeRequest.fundingQortAmount, Asset.QORT);
Transaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
ValidationResult result = deployAtTransaction.isValidUnconfirmed();
if (result != ValidationResult.OK)
throw TransactionsResource.createTransactionInvalidException(request, result);
byte[] bytes = DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
return Base58.encode(bytes);
} catch (TransformationException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSFORMATION_ERROR, e);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@POST
@Path("/trademessage")
@Operation(
summary = "Builds raw, unsigned 'trade' MESSAGE transaction that sends cross-chain trade recipient address, triggering 'trade' mode",
description = "Specify address of cross-chain AT that needs to be messaged, and signature of 'offer' MESSAGE from trade partner.<br>"
+ "AT needs to be in 'offer' mode. Messages sent to an AT in any other mode will be ignored, but still cost fees to send!<br>"
+ "You need to sign output with trade private key otherwise the MESSAGE transaction will be invalid.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = CrossChainTradeRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(
schema = @Schema(
type = "string"
)
)
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_ADDRESS, ApiError.INVALID_CRITERIA, ApiError.REPOSITORY_ISSUE})
public String buildTradeMessage(CrossChainTradeRequest tradeRequest) {
Security.checkApiCallAllowed(request);
byte[] tradePublicKey = tradeRequest.tradePublicKey;
if (tradePublicKey == null || tradePublicKey.length != Transformer.PUBLIC_KEY_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
if (tradeRequest.atAddress == null || !Crypto.isValidAtAddress(tradeRequest.atAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (tradeRequest.messageTransactionSignature == null || !Crypto.isValidAddress(tradeRequest.messageTransactionSignature))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = fetchAtDataWithChecking(repository, tradeRequest.atAddress);
CrossChainTradeData crossChainTradeData = BitcoinACCTv1.getInstance().populateTradeData(repository, atData);
if (crossChainTradeData.mode != AcctMode.OFFERING)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Does supplied public key match trade public key?
if (!Crypto.toAddress(tradePublicKey).equals(crossChainTradeData.qortalCreatorTradeAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
TransactionData transactionData = repository.getTransactionRepository().fromSignature(tradeRequest.messageTransactionSignature);
if (transactionData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_UNKNOWN);
if (transactionData.getType() != TransactionType.MESSAGE)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_INVALID);
MessageTransactionData messageTransactionData = (MessageTransactionData) transactionData;
byte[] messageData = messageTransactionData.getData();
BitcoinACCTv1.OfferMessageData offerMessageData = BitcoinACCTv1.extractOfferMessageData(messageData);
if (offerMessageData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_INVALID);
// Good to make MESSAGE
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerBitcoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
int lockTimeB = BitcoinACCTv1.calcLockTimeB(messageTransactionData.getTimestamp(), lockTimeA);
byte[] outgoingMessageData = BitcoinACCTv1.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, lockTimeB);
byte[] messageTransactionBytes = buildAtMessage(repository, tradePublicKey, tradeRequest.atAddress, outgoingMessageData);
return Base58.encode(messageTransactionBytes);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@POST
@Path("/redeemmessage")
@Operation(
summary = "Builds raw, unsigned 'redeem' MESSAGE transaction that sends secrets to AT, releasing funds to partner",
description = "Specify address of cross-chain AT that needs to be messaged, both 32-byte secrets and an address for receiving QORT from AT.<br>"
+ "AT needs to be in 'trade' mode. Messages sent to an AT in any other mode will be ignored, but still cost fees to send!<br>"
+ "You need to sign output with account the AT considers the trade 'partner' otherwise the MESSAGE transaction will be invalid.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = CrossChainDualSecretRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(
schema = @Schema(
type = "string"
)
)
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_ADDRESS, ApiError.INVALID_DATA, ApiError.INVALID_CRITERIA, ApiError.REPOSITORY_ISSUE})
public String buildRedeemMessage(CrossChainDualSecretRequest secretRequest) {
Security.checkApiCallAllowed(request);
byte[] partnerPublicKey = secretRequest.partnerPublicKey;
if (partnerPublicKey == null || partnerPublicKey.length != Transformer.PUBLIC_KEY_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
if (secretRequest.atAddress == null || !Crypto.isValidAtAddress(secretRequest.atAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (secretRequest.secretA == null || secretRequest.secretA.length != BitcoinACCTv1.SECRET_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (secretRequest.secretB == null || secretRequest.secretB.length != BitcoinACCTv1.SECRET_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (secretRequest.receivingAddress == null || !Crypto.isValidAddress(secretRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = fetchAtDataWithChecking(repository, secretRequest.atAddress);
CrossChainTradeData crossChainTradeData = BitcoinACCTv1.getInstance().populateTradeData(repository, atData);
if (crossChainTradeData.mode != AcctMode.TRADING)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
String partnerAddress = Crypto.toAddress(partnerPublicKey);
// MESSAGE must come from address that AT considers trade partner
if (!crossChainTradeData.qortalPartnerAddress.equals(partnerAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
// Good to make MESSAGE
byte[] messageData = BitcoinACCTv1.buildRedeemMessage(secretRequest.secretA, secretRequest.secretB, secretRequest.receivingAddress);
byte[] messageTransactionBytes = buildAtMessage(repository, partnerPublicKey, secretRequest.atAddress, messageData);
return Base58.encode(messageTransactionBytes);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
private ATData fetchAtDataWithChecking(Repository repository, String atAddress) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// Must be correct AT - check functionality using code hash
if (!Arrays.equals(atData.getCodeHash(), BitcoinACCTv1.CODE_BYTES_HASH))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// No point sending message to AT that's finished
if (atData.getIsFinished())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
return atData;
}
private byte[] buildAtMessage(Repository repository, byte[] senderPublicKey, String atAddress, byte[] messageData) throws DataException {
long txTimestamp = NTP.getTime();
// senderPublicKey could be ephemeral trade public key where there is no corresponding account and hence no reference
String senderAddress = Crypto.toAddress(senderPublicKey);
byte[] lastReference = repository.getAccountRepository().getLastReference(senderAddress);
final boolean requiresPoW = lastReference == null;
if (requiresPoW) {
Random random = new Random();
lastReference = new byte[Transformer.SIGNATURE_LENGTH];
random.nextBytes(lastReference);
}
int version = 4;
int nonce = 0;
long amount = 0L;
Long assetId = null; // no assetId as amount is zero
Long fee = 0L;
BaseTransactionData baseTransactionData = new BaseTransactionData(txTimestamp, Group.NO_GROUP, lastReference, senderPublicKey, fee, null);
TransactionData messageTransactionData = new MessageTransactionData(baseTransactionData, version, nonce, atAddress, amount, assetId, messageData, false, false);
MessageTransaction messageTransaction = new MessageTransaction(repository, messageTransactionData);
if (requiresPoW) {
messageTransaction.computeNonce();
} else {
fee = messageTransaction.calcRecommendedFee();
messageTransactionData.setFee(fee);
}
ValidationResult result = messageTransaction.isValidUnconfirmed();
if (result != ValidationResult.OK)
throw TransactionsResource.createTransactionInvalidException(request, result);
try {
return MessageTransactionTransformer.toBytes(messageTransactionData);
} catch (TransformationException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSFORMATION_ERROR, e);
}
}
}

View File

@@ -0,0 +1,167 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.bitcoinj.core.Transaction;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.crosschain.BitcoinSendRequest;
import org.qortal.crosschain.Bitcoin;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.crosschain.SimpleTransaction;
@Path("/crosschain/btc")
@Tag(name = "Cross-Chain (Bitcoin)")
public class CrossChainBitcoinResource {
@Context
HttpServletRequest request;
@POST
@Path("/walletbalance")
@Operation(
summary = "Returns BTC balance for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public String getBitcoinWalletBalance(String key58) {
Security.checkApiCallAllowed(request);
Bitcoin bitcoin = Bitcoin.getInstance();
if (!bitcoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Long balance = bitcoin.getWalletBalance(key58);
if (balance == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
return balance.toString();
}
@POST
@Path("/wallettransactions")
@Operation(
summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public List<SimpleTransaction> getBitcoinWalletTransactions(String key58) {
Security.checkApiCallAllowed(request);
Bitcoin bitcoin = Bitcoin.getInstance();
if (!bitcoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
return bitcoin.getWalletTransactions(key58);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/send")
@Operation(
summary = "Sends BTC from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently only supports 'legacy' P2PKH Bitcoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = BitcoinSendRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public String sendBitcoin(BitcoinSendRequest bitcoinSendRequest) {
Security.checkApiCallAllowed(request);
if (bitcoinSendRequest.bitcoinAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (bitcoinSendRequest.feePerByte != null && bitcoinSendRequest.feePerByte <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Bitcoin bitcoin = Bitcoin.getInstance();
if (!bitcoin.isValidAddress(bitcoinSendRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!bitcoin.isValidDeterministicKey(bitcoinSendRequest.xprv58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Transaction spendTransaction = bitcoin.buildSpend(bitcoinSendRequest.xprv58,
bitcoinSendRequest.receivingAddress,
bitcoinSendRequest.bitcoinAmount,
bitcoinSendRequest.feePerByte);
if (spendTransaction == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
try {
bitcoin.broadcastTransaction(spendTransaction);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
return spendTransaction.getTxId().toString();
}
}

View File

@@ -0,0 +1,140 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.CrossChainSecretRequest;
import org.qortal.crosschain.AcctMode;
import org.qortal.crosschain.DogecoinACCTv1;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.Transformer;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import java.util.Arrays;
@Path("/crosschain/DogecoinACCTv1")
@Tag(name = "Cross-Chain (DogecoinACCTv1)")
public class CrossChainDogecoinACCTv1Resource {
@Context
HttpServletRequest request;
@POST
@Path("/redeemmessage")
@Operation(
summary = "Signs and broadcasts a 'redeem' MESSAGE transaction that sends secrets to AT, releasing funds to partner",
description = "Specify address of cross-chain AT that needs to be messaged, Alice's trade private key, the 32-byte secret,<br>"
+ "and an address for receiving QORT from AT. All of these can be found in Alice's trade bot data.<br>"
+ "AT needs to be in 'trade' mode. Messages sent to an AT in any other mode will be ignored, but still cost fees to send!<br>"
+ "You need to use the private key that the AT considers the trade 'partner' otherwise the MESSAGE transaction will be invalid.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = CrossChainSecretRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(
schema = @Schema(
type = "string"
)
)
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_ADDRESS, ApiError.INVALID_DATA, ApiError.INVALID_CRITERIA, ApiError.REPOSITORY_ISSUE})
public boolean buildRedeemMessage(CrossChainSecretRequest secretRequest) {
Security.checkApiCallAllowed(request);
byte[] partnerPrivateKey = secretRequest.partnerPrivateKey;
if (partnerPrivateKey == null || partnerPrivateKey.length != Transformer.PRIVATE_KEY_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
if (secretRequest.atAddress == null || !Crypto.isValidAtAddress(secretRequest.atAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (secretRequest.secret == null || secretRequest.secret.length != DogecoinACCTv1.SECRET_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (secretRequest.receivingAddress == null || !Crypto.isValidAddress(secretRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = fetchAtDataWithChecking(repository, secretRequest.atAddress);
CrossChainTradeData crossChainTradeData = DogecoinACCTv1.getInstance().populateTradeData(repository, atData);
if (crossChainTradeData.mode != AcctMode.TRADING)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
byte[] partnerPublicKey = new PrivateKeyAccount(null, partnerPrivateKey).getPublicKey();
String partnerAddress = Crypto.toAddress(partnerPublicKey);
// MESSAGE must come from address that AT considers trade partner
if (!crossChainTradeData.qortalPartnerAddress.equals(partnerAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
// Good to make MESSAGE
byte[] messageData = DogecoinACCTv1.buildRedeemMessage(secretRequest.secret, secretRequest.receivingAddress);
PrivateKeyAccount sender = new PrivateKeyAccount(repository, partnerPrivateKey);
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, secretRequest.atAddress, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_INVALID);
return true;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
private ATData fetchAtDataWithChecking(Repository repository, String atAddress) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// Must be correct AT - check functionality using code hash
if (!Arrays.equals(atData.getCodeHash(), DogecoinACCTv1.CODE_BYTES_HASH))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// No point sending message to AT that's finished
if (atData.getIsFinished())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
return atData;
}
}

View File

@@ -0,0 +1,165 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.bitcoinj.core.Transaction;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.crosschain.DogecoinSendRequest;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.crosschain.Dogecoin;
import org.qortal.crosschain.SimpleTransaction;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import java.util.List;
@Path("/crosschain/doge")
@Tag(name = "Cross-Chain (Dogecoin)")
public class CrossChainDogecoinResource {
@Context
HttpServletRequest request;
@POST
@Path("/walletbalance")
@Operation(
summary = "Returns DOGE balance for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public String getDogecoinWalletBalance(String key58) {
Security.checkApiCallAllowed(request);
Dogecoin dogecoin = Dogecoin.getInstance();
if (!dogecoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Long balance = dogecoin.getWalletBalance(key58);
if (balance == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
return balance.toString();
}
@POST
@Path("/wallettransactions")
@Operation(
summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public List<SimpleTransaction> getDogecoinWalletTransactions(String key58) {
Security.checkApiCallAllowed(request);
Dogecoin dogecoin = Dogecoin.getInstance();
if (!dogecoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
return dogecoin.getWalletTransactions(key58);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/send")
@Operation(
summary = "Sends DOGE from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently only supports 'legacy' P2PKH Dogecoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = DogecoinSendRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public String sendBitcoin(DogecoinSendRequest dogecoinSendRequest) {
Security.checkApiCallAllowed(request);
if (dogecoinSendRequest.dogecoinAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (dogecoinSendRequest.feePerByte != null && dogecoinSendRequest.feePerByte <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Dogecoin dogecoin = Dogecoin.getInstance();
if (!dogecoin.isValidAddress(dogecoinSendRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!dogecoin.isValidDeterministicKey(dogecoinSendRequest.xprv58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Transaction spendTransaction = dogecoin.buildSpend(dogecoinSendRequest.xprv58,
dogecoinSendRequest.receivingAddress,
dogecoinSendRequest.dogecoinAmount,
dogecoinSendRequest.feePerByte);
if (spendTransaction == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
try {
dogecoin.broadcastTransaction(spendTransaction);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
return spendTransaction.getTxId().toString();
}
}

View File

@@ -0,0 +1,647 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.math.BigDecimal;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script;
import org.qortal.api.*;
import org.qortal.api.model.CrossChainBitcoinyHTLCStatus;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
@Path("/crosschain/htlc")
@Tag(name = "Cross-Chain (Hash time-locked contracts)")
public class CrossChainHtlcResource {
private static final Logger LOGGER = LogManager.getLogger(CrossChainHtlcResource.class);
@Context
HttpServletRequest request;
@GET
@Path("/address/{blockchain}/{refundPKH}/{locktime}/{redeemPKH}/{hashOfSecret}")
@Operation(
summary = "Returns HTLC address based on trade info",
description = "Public key hashes (PKH) and hash of secret should be 20 bytes (base58 encoded). Locktime is seconds since epoch.",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_CRITERIA})
public String deriveHtlcAddress(@PathParam("blockchain") String blockchainName,
@PathParam("refundPKH") String refundPKH,
@PathParam("locktime") int lockTime,
@PathParam("redeemPKH") String redeemPKH,
@PathParam("hashOfSecret") String hashOfSecret) {
SupportedBlockchain blockchain = SupportedBlockchain.valueOf(blockchainName);
if (blockchain == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
byte[] refunderPubKeyHash;
byte[] redeemerPubKeyHash;
byte[] decodedHashOfSecret;
try {
refunderPubKeyHash = Base58.decode(refundPKH);
redeemerPubKeyHash = Base58.decode(redeemPKH);
if (refunderPubKeyHash.length != 20 || redeemerPubKeyHash.length != 20)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
}
try {
decodedHashOfSecret = Base58.decode(hashOfSecret);
if (decodedHashOfSecret.length != 20)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
byte[] redeemScript = BitcoinyHTLC.buildScript(refunderPubKeyHash, lockTime, redeemerPubKeyHash, decodedHashOfSecret);
Bitcoiny bitcoiny = (Bitcoiny) blockchain.getInstance();
return bitcoiny.deriveP2shAddress(redeemScript);
}
@GET
@Path("/status/{blockchain}/{refundPKH}/{locktime}/{redeemPKH}/{hashOfSecret}")
@Operation(
summary = "Checks HTLC status",
description = "Public key hashes (PKH) and hash of secret should be 20 bytes (base58 encoded). Locktime is seconds since epoch.",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.APPLICATION_JSON, schema = @Schema(implementation = CrossChainBitcoinyHTLCStatus.class))
)
}
)
@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN})
public CrossChainBitcoinyHTLCStatus checkHtlcStatus(@PathParam("blockchain") String blockchainName,
@PathParam("refundPKH") String refundPKH,
@PathParam("locktime") int lockTime,
@PathParam("redeemPKH") String redeemPKH,
@PathParam("hashOfSecret") String hashOfSecret) {
Security.checkApiCallAllowed(request);
SupportedBlockchain blockchain = SupportedBlockchain.valueOf(blockchainName);
if (blockchain == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
byte[] refunderPubKeyHash;
byte[] redeemerPubKeyHash;
byte[] decodedHashOfSecret;
try {
refunderPubKeyHash = Base58.decode(refundPKH);
redeemerPubKeyHash = Base58.decode(redeemPKH);
if (refunderPubKeyHash.length != 20 || redeemerPubKeyHash.length != 20)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY);
}
try {
decodedHashOfSecret = Base58.decode(hashOfSecret);
if (decodedHashOfSecret.length != 20)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
byte[] redeemScript = BitcoinyHTLC.buildScript(refunderPubKeyHash, lockTime, redeemerPubKeyHash, decodedHashOfSecret);
Bitcoiny bitcoiny = (Bitcoiny) blockchain.getInstance();
String p2shAddress = bitcoiny.deriveP2shAddress(redeemScript);
long now = NTP.getTime();
try {
int medianBlockTime = bitcoiny.getMedianBlockTime();
// Check P2SH is funded
long p2shBalance = bitcoiny.getConfirmedBalance(p2shAddress.toString());
CrossChainBitcoinyHTLCStatus htlcStatus = new CrossChainBitcoinyHTLCStatus();
htlcStatus.bitcoinP2shAddress = p2shAddress;
htlcStatus.bitcoinP2shBalance = BigDecimal.valueOf(p2shBalance, 8);
List<TransactionOutput> fundingOutputs = bitcoiny.getUnspentOutputs(p2shAddress.toString());
if (p2shBalance > 0L && !fundingOutputs.isEmpty()) {
htlcStatus.canRedeem = now >= medianBlockTime * 1000L;
htlcStatus.canRefund = now >= lockTime * 1000L;
}
if (now >= medianBlockTime * 1000L) {
// See if we can extract secret
htlcStatus.secret = BitcoinyHTLC.findHtlcSecret(bitcoiny, htlcStatus.bitcoinP2shAddress);
}
return htlcStatus;
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@GET
@Path("/redeem/{ataddress}")
@Operation(
summary = "Redeems HTLC associated with supplied AT",
description = "To be used by a QORT seller (Bob) who needs to redeem LTC/DOGE/etc proceeds that are stuck in a P2SH.<br>" +
"This requires Bob's trade bot data to be present in the database for this AT.<br>" +
"It will fail if the buyer has yet to redeem the QORT held in the AT.",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN})
public boolean redeemHtlc(@PathParam("ataddress") String atAddress) {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
ACCT acct = SupportedBlockchain.getAcctByCodeHash(atData.getCodeHash());
if (acct == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
if (crossChainTradeData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Attempt to find secret from the buyer's message to AT
byte[] decodedSecret = acct.findSecretA(repository, crossChainTradeData);
if (decodedSecret == null) {
LOGGER.info(() -> String.format("Unable to find secret-A from redeem message to AT %s", atAddress));
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
TradeBotData tradeBotData = allTradeBotData.stream().filter(tradeBotDataItem -> tradeBotDataItem.getAtAddress().equals(atAddress)).findFirst().orElse(null);
// Search for the tradePrivateKey in the tradebot data
byte[] decodedPrivateKey = null;
if (tradeBotData != null)
decodedPrivateKey = tradeBotData.getTradePrivateKey();
// Search for the foreign blockchain receiving address in the tradebot data
byte[] foreignBlockchainReceivingAccountInfo = null;
if (tradeBotData != null)
// Use receiving address PKH from tradebot data
foreignBlockchainReceivingAccountInfo = tradeBotData.getReceivingAccountInfo();
return this.doRedeemHtlc(atAddress, decodedPrivateKey, decodedSecret, foreignBlockchainReceivingAccountInfo);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@GET
@Path("/redeemAll")
@Operation(
summary = "Redeems HTLC for all applicable ATs in tradebot data",
description = "To be used by a QORT seller (Bob) who needs to redeem LTC/DOGE/etc proceeds that are stuck in P2SH transactions.<br>" +
"This requires Bob's trade bot data to be present in the database for any ATs that need redeeming.<br>" +
"Returns true if at least one trade is redeemed. More detail is available in the log.txt.* file.",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN})
public boolean redeemAllHtlc() {
Security.checkApiCallAllowed(request);
boolean success = false;
try (final Repository repository = RepositoryManager.getRepository()) {
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
for (TradeBotData tradeBotData : allTradeBotData) {
String atAddress = tradeBotData.getAtAddress();
if (atAddress == null) {
LOGGER.info("Missing AT address in tradebot data", atAddress);
continue;
}
String tradeState = tradeBotData.getState();
if (tradeState == null) {
LOGGER.info("Missing trade state for AT {}", atAddress);
continue;
}
if (tradeState.startsWith("ALICE")) {
LOGGER.info("AT {} isn't redeemable because it is a buy order", atAddress);
continue;
}
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null) {
LOGGER.info("Couldn't find AT with address {}", atAddress);
continue;
}
ACCT acct = SupportedBlockchain.getAcctByCodeHash(atData.getCodeHash());
if (acct == null) {
continue;
}
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
if (crossChainTradeData == null) {
LOGGER.info("Couldn't find crosschain trade data for AT {}", atAddress);
continue;
}
// Attempt to find secret from the buyer's message to AT
byte[] decodedSecret = acct.findSecretA(repository, crossChainTradeData);
if (decodedSecret == null) {
LOGGER.info("Unable to find secret-A from redeem message to AT {}", atAddress);
continue;
}
// Search for the tradePrivateKey in the tradebot data
byte[] decodedPrivateKey = tradeBotData.getTradePrivateKey();
// Search for the foreign blockchain receiving address PKH in the tradebot data
byte[] foreignBlockchainReceivingAccountInfo = tradeBotData.getReceivingAccountInfo();
try {
LOGGER.info("Attempting to redeem P2SH balance associated with AT {}...", atAddress);
boolean redeemed = this.doRedeemHtlc(atAddress, decodedPrivateKey, decodedSecret, foreignBlockchainReceivingAccountInfo);
if (redeemed) {
LOGGER.info("Redeemed P2SH balance associated with AT {}", atAddress);
success = true;
}
else {
LOGGER.info("Couldn't redeem P2SH balance associated with AT {}. Already redeemed?", atAddress);
}
} catch (ApiException e) {
LOGGER.info("Couldn't redeem P2SH balance associated with AT {}. Missing data?", atAddress);
}
}
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
return success;
}
private boolean doRedeemHtlc(String atAddress, byte[] decodedTradePrivateKey, byte[] decodedSecret,
byte[] foreignBlockchainReceivingAccountInfo) {
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
ACCT acct = SupportedBlockchain.getAcctByCodeHash(atData.getCodeHash());
if (acct == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
if (crossChainTradeData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Validate trade private key
if (decodedTradePrivateKey == null || decodedTradePrivateKey.length != 32)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Validate secret
if (decodedSecret == null || decodedSecret.length != 32)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Validate receiving address
if (foreignBlockchainReceivingAccountInfo == null || foreignBlockchainReceivingAccountInfo.length != 20)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Make sure the receiving address isn't a QORT address, given that we can share the same field for both QORT and foreign blockchains
if (Crypto.isValidAddress(foreignBlockchainReceivingAccountInfo))
if (Base58.encode(foreignBlockchainReceivingAccountInfo).startsWith("Q"))
// This is likely a QORT address, not a foreign blockchain
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Use secret-A to redeem P2SH-A
Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
if (bitcoiny.getClass() == Bitcoin.class) {
LOGGER.info("Redeeming a Bitcoin HTLC is not yet supported");
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
int lockTime = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTime, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = bitcoiny.deriveP2shAddress(redeemScriptA);
LOGGER.info(String.format("Redeeming P2SH address: %s", p2shAddressA));
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTime, crossChainTradeData.tradeTimeout);
long p2shFee = bitcoiny.getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoiny.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return false;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
return false;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return false;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(decodedTradePrivateKey);
List<TransactionOutput> fundingOutputs = bitcoiny.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(bitcoiny.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, decodedSecret, foreignBlockchainReceivingAccountInfo);
bitcoiny.broadcastTransaction(p2shRedeemTransaction);
LOGGER.info(String.format("P2SH address %s redeemed!", p2shAddressA));
return true;
}
}
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, e);
}
return false;
}
@GET
@Path("/refund/{ataddress}")
@Operation(
summary = "Refunds HTLC associated with supplied AT",
description = "To be used by a QORT buyer (Alice) who needs to refund their LTC/DOGE/etc that is stuck in a P2SH.<br>" +
"This requires Alice's trade bot data to be present in the database for this AT.<br>" +
"It will fail if it's already redeemed by the seller, or if the lockTime (60 minutes) hasn't passed yet.",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN})
public boolean refundHtlc(@PathParam("ataddress") String atAddress) {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
TradeBotData tradeBotData = allTradeBotData.stream().filter(tradeBotDataItem -> tradeBotDataItem.getAtAddress().equals(atAddress)).findFirst().orElse(null);
if (tradeBotData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (tradeBotData.getForeignKey() == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
ACCT acct = SupportedBlockchain.getAcctByCodeHash(atData.getCodeHash());
if (acct == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// Determine foreign blockchain receive address for refund
Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
String receiveAddress = bitcoiny.getUnusedReceiveAddress(tradeBotData.getForeignKey());
return this.doRefundHtlc(atAddress, receiveAddress);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, e);
}
}
@GET
@Path("/refundAll")
@Operation(
summary = "Refunds HTLC for all applicable ATs in tradebot data",
description = "To be used by a QORT buyer (Alice) who needs to refund their LTC/DOGE/etc proceeds that are stuck in P2SH transactions.<br>" +
"This requires Alice's trade bot data to be present in the database for this AT.<br>" +
"It will fail if it's already redeemed by the seller, or if the lockTime (60 minutes) hasn't passed yet.",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
)
}
)
@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.ADDRESS_UNKNOWN})
public boolean refundAllHtlc() {
Security.checkApiCallAllowed(request);
Security.checkApiCallAllowed(request);
boolean success = false;
try (final Repository repository = RepositoryManager.getRepository()) {
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
for (TradeBotData tradeBotData : allTradeBotData) {
String atAddress = tradeBotData.getAtAddress();
if (atAddress == null) {
LOGGER.info("Missing AT address in tradebot data", atAddress);
continue;
}
String tradeState = tradeBotData.getState();
if (tradeState == null) {
LOGGER.info("Missing trade state for AT {}", atAddress);
continue;
}
if (tradeState.startsWith("BOB")) {
LOGGER.info("AT {} isn't refundable because it is a sell order", atAddress);
continue;
}
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null) {
LOGGER.info("Couldn't find AT with address {}", atAddress);
continue;
}
ACCT acct = SupportedBlockchain.getAcctByCodeHash(atData.getCodeHash());
if (acct == null) {
continue;
}
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
if (crossChainTradeData == null) {
LOGGER.info("Couldn't find crosschain trade data for AT {}", atAddress);
continue;
}
if (tradeBotData.getForeignKey() == null) {
LOGGER.info("Couldn't find foreign key for AT {}", atAddress);
continue;
}
try {
// Determine foreign blockchain receive address for refund
Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
String receivingAddress = bitcoiny.getUnusedReceiveAddress(tradeBotData.getForeignKey());
LOGGER.info("Attempting to refund P2SH balance associated with AT {}...", atAddress);
boolean refunded = this.doRefundHtlc(atAddress, receivingAddress);
if (refunded) {
LOGGER.info("Refunded P2SH balance associated with AT {}", atAddress);
success = true;
}
else {
LOGGER.info("Couldn't refund P2SH balance associated with AT {}. Already redeemed?", atAddress);
}
} catch (ApiException | ForeignBlockchainException e) {
LOGGER.info("Couldn't refund P2SH balance associated with AT {}. Missing data?", atAddress);
}
}
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
return success;
}
private boolean doRefundHtlc(String atAddress, String receiveAddress) {
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
ACCT acct = SupportedBlockchain.getAcctByCodeHash(atData.getCodeHash());
if (acct == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
if (crossChainTradeData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
TradeBotData tradeBotData = allTradeBotData.stream().filter(tradeBotDataItem -> tradeBotDataItem.getAtAddress().equals(atAddress)).findFirst().orElse(null);
if (tradeBotData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
if (bitcoiny.getClass() == Bitcoin.class) {
LOGGER.info("Refunding a Bitcoin HTLC is not yet supported");
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
}
int lockTime = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTime * 1000L)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_TOO_SOON);
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = bitcoiny.getMedianBlockTime();
if (medianBlockTime <= lockTime)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_TOO_SOON);
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTime, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = bitcoiny.deriveP2shAddress(redeemScriptA);
LOGGER.info(String.format("Refunding P2SH address: %s", p2shAddressA));
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTime, crossChainTradeData.tradeTimeout);
long p2shFee = bitcoiny.getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoiny.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_TOO_SOON);
case REDEEM_IN_PROGRESS:
case REDEEMED:
case REFUND_IN_PROGRESS:
case REFUNDED:
// Too late!
return false;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = bitcoiny.getUnspentOutputs(p2shAddressA);
// Validate the destination foreign blockchain address
Address receiving = Address.fromString(bitcoiny.getNetworkParameters(), receiveAddress);
if (receiving.getOutputScriptType() != Script.ScriptType.P2PKH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(bitcoiny.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTime, receiving.getHash());
bitcoiny.broadcastTransaction(p2shRefundTransaction);
return true;
}
}
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, e);
}
return false;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,145 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.CrossChainSecretRequest;
import org.qortal.crosschain.AcctMode;
import org.qortal.crosschain.LitecoinACCTv1;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.Transformer;
import org.qortal.transform.transaction.MessageTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import java.util.Arrays;
import java.util.Random;
@Path("/crosschain/LitecoinACCTv1")
@Tag(name = "Cross-Chain (LitecoinACCTv1)")
public class CrossChainLitecoinACCTv1Resource {
@Context
HttpServletRequest request;
@POST
@Path("/redeemmessage")
@Operation(
summary = "Signs and broadcasts a 'redeem' MESSAGE transaction that sends secrets to AT, releasing funds to partner",
description = "Specify address of cross-chain AT that needs to be messaged, Alice's trade private key, the 32-byte secret,<br>"
+ "and an address for receiving QORT from AT. All of these can be found in Alice's trade bot data.<br>"
+ "AT needs to be in 'trade' mode. Messages sent to an AT in any other mode will be ignored, but still cost fees to send!<br>"
+ "You need to use the private key that the AT considers the trade 'partner' otherwise the MESSAGE transaction will be invalid.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = CrossChainSecretRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(
schema = @Schema(
type = "string"
)
)
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_ADDRESS, ApiError.INVALID_DATA, ApiError.INVALID_CRITERIA, ApiError.REPOSITORY_ISSUE})
public boolean buildRedeemMessage(CrossChainSecretRequest secretRequest) {
Security.checkApiCallAllowed(request);
byte[] partnerPrivateKey = secretRequest.partnerPrivateKey;
if (partnerPrivateKey == null || partnerPrivateKey.length != Transformer.PRIVATE_KEY_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
if (secretRequest.atAddress == null || !Crypto.isValidAtAddress(secretRequest.atAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (secretRequest.secret == null || secretRequest.secret.length != LitecoinACCTv1.SECRET_LENGTH)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
if (secretRequest.receivingAddress == null || !Crypto.isValidAddress(secretRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = fetchAtDataWithChecking(repository, secretRequest.atAddress);
CrossChainTradeData crossChainTradeData = LitecoinACCTv1.getInstance().populateTradeData(repository, atData);
if (crossChainTradeData.mode != AcctMode.TRADING)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
byte[] partnerPublicKey = new PrivateKeyAccount(null, partnerPrivateKey).getPublicKey();
String partnerAddress = Crypto.toAddress(partnerPublicKey);
// MESSAGE must come from address that AT considers trade partner
if (!crossChainTradeData.qortalPartnerAddress.equals(partnerAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
// Good to make MESSAGE
byte[] messageData = LitecoinACCTv1.buildRedeemMessage(secretRequest.secret, secretRequest.receivingAddress);
PrivateKeyAccount sender = new PrivateKeyAccount(repository, partnerPrivateKey);
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, secretRequest.atAddress, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_INVALID);
return true;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
private ATData fetchAtDataWithChecking(Repository repository, String atAddress) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// Must be correct AT - check functionality using code hash
if (!Arrays.equals(atData.getCodeHash(), LitecoinACCTv1.CODE_BYTES_HASH))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
// No point sending message to AT that's finished
if (atData.getIsFinished())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
return atData;
}
}

View File

@@ -0,0 +1,167 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.bitcoinj.core.Transaction;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.crosschain.LitecoinSendRequest;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.crosschain.Litecoin;
import org.qortal.crosschain.SimpleTransaction;
@Path("/crosschain/ltc")
@Tag(name = "Cross-Chain (Litecoin)")
public class CrossChainLitecoinResource {
@Context
HttpServletRequest request;
@POST
@Path("/walletbalance")
@Operation(
summary = "Returns LTC balance for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public String getLitecoinWalletBalance(String key58) {
Security.checkApiCallAllowed(request);
Litecoin litecoin = Litecoin.getInstance();
if (!litecoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Long balance = litecoin.getWalletBalance(key58);
if (balance == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
return balance.toString();
}
@POST
@Path("/wallettransactions")
@Operation(
summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public List<SimpleTransaction> getLitecoinWalletTransactions(String key58) {
Security.checkApiCallAllowed(request);
Litecoin litecoin = Litecoin.getInstance();
if (!litecoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
return litecoin.getWalletTransactions(key58);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/send")
@Operation(
summary = "Sends LTC from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently only supports 'legacy' P2PKH Litecoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = LitecoinSendRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
public String sendBitcoin(LitecoinSendRequest litecoinSendRequest) {
Security.checkApiCallAllowed(request);
if (litecoinSendRequest.litecoinAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (litecoinSendRequest.feePerByte != null && litecoinSendRequest.feePerByte <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Litecoin litecoin = Litecoin.getInstance();
if (!litecoin.isValidAddress(litecoinSendRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!litecoin.isValidDeterministicKey(litecoinSendRequest.xprv58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Transaction spendTransaction = litecoin.buildSpend(litecoinSendRequest.xprv58,
litecoinSendRequest.receivingAddress,
litecoinSendRequest.litecoinAmount,
litecoinSendRequest.feePerByte);
if (spendTransaction == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
try {
litecoin.broadcastTransaction(spendTransaction);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
return spendTransaction.getTxId().toString();
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,289 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.List;
import java.util.stream.Collectors;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.qortal.account.Account;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.api.model.crosschain.TradeBotRespondRequest;
import org.qortal.asset.Asset;
import org.qortal.controller.tradebot.AcctTradeBot;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.crosschain.ForeignBlockchain;
import org.qortal.crosschain.SupportedBlockchain;
import org.qortal.crosschain.ACCT;
import org.qortal.crosschain.AcctMode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.utils.Base58;
@Path("/crosschain/tradebot")
@Tag(name = "Cross-Chain (Trade-Bot)")
public class CrossChainTradeBotResource {
@Context
HttpServletRequest request;
@GET
@Operation(
summary = "List current trade-bot states",
responses = {
@ApiResponse(
content = @Content(
array = @ArraySchema(
schema = @Schema(
implementation = TradeBotData.class
)
)
)
)
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
public List<TradeBotData> getTradeBotStates(
@Parameter(
description = "Limit to specific blockchain",
example = "LITECOIN",
schema = @Schema(implementation = SupportedBlockchain.class)
) @QueryParam("foreignBlockchain") SupportedBlockchain foreignBlockchain) {
Security.checkApiCallAllowed(request);
try (final Repository repository = RepositoryManager.getRepository()) {
List<TradeBotData> allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
if (foreignBlockchain == null)
return allTradeBotData;
return allTradeBotData.stream().filter(tradeBotData -> tradeBotData.getForeignBlockchain().equals(foreignBlockchain.name())).collect(Collectors.toList());
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@POST
@Path("/create")
@Operation(
summary = "Create a trade offer (trade-bot entry)",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = TradeBotCreateRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.INVALID_PUBLIC_KEY, ApiError.INVALID_ADDRESS, ApiError.INVALID_CRITERIA, ApiError.INSUFFICIENT_BALANCE, ApiError.REPOSITORY_ISSUE, ApiError.ORDER_SIZE_TOO_SMALL})
@SuppressWarnings("deprecation")
public String tradeBotCreator(TradeBotCreateRequest tradeBotCreateRequest) {
Security.checkApiCallAllowed(request);
if (tradeBotCreateRequest.foreignBlockchain == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
ForeignBlockchain foreignBlockchain = tradeBotCreateRequest.foreignBlockchain.getInstance();
// We prefer foreignAmount to deprecated bitcoinAmount
if (tradeBotCreateRequest.foreignAmount == null)
tradeBotCreateRequest.foreignAmount = tradeBotCreateRequest.bitcoinAmount;
if (!foreignBlockchain.isValidAddress(tradeBotCreateRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (tradeBotCreateRequest.tradeTimeout < 60)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (tradeBotCreateRequest.foreignAmount == null || tradeBotCreateRequest.foreignAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ORDER_SIZE_TOO_SMALL);
if (tradeBotCreateRequest.foreignAmount < foreignBlockchain.getMinimumOrderAmount())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ORDER_SIZE_TOO_SMALL);
if (tradeBotCreateRequest.qortAmount <= 0 || tradeBotCreateRequest.fundingQortAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ORDER_SIZE_TOO_SMALL);
try (final Repository repository = RepositoryManager.getRepository()) {
// Do some simple checking first
Account creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
if (creator.getConfirmedBalance(Asset.QORT) < tradeBotCreateRequest.fundingQortAmount)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INSUFFICIENT_BALANCE);
byte[] unsignedBytes = TradeBot.getInstance().createTrade(repository, tradeBotCreateRequest);
if (unsignedBytes == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
return Base58.encode(unsignedBytes);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@POST
@Path("/respond")
@Operation(
summary = "Respond to a trade offer. NOTE: WILL SPEND FUNDS!)",
description = "Start a new trade-bot entry to respond to chosen trade offer.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = TradeBotRespondRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_ADDRESS, ApiError.INVALID_CRITERIA, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE, ApiError.REPOSITORY_ISSUE})
@SuppressWarnings("deprecation")
public String tradeBotResponder(TradeBotRespondRequest tradeBotRespondRequest) {
Security.checkApiCallAllowed(request);
final String atAddress = tradeBotRespondRequest.atAddress;
// We prefer foreignKey to deprecated xprv58
if (tradeBotRespondRequest.foreignKey == null)
tradeBotRespondRequest.foreignKey = tradeBotRespondRequest.xprv58;
if (tradeBotRespondRequest.foreignKey == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
if (atAddress == null || !Crypto.isValidAtAddress(atAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (tradeBotRespondRequest.receivingAddress == null || !Crypto.isValidAddress(tradeBotRespondRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
// Extract data from cross-chain trading AT
try (final Repository repository = RepositoryManager.getRepository()) {
ATData atData = fetchAtDataWithChecking(repository, atAddress);
// TradeBot uses AT's code hash to map to ACCT
ACCT acct = TradeBot.getInstance().getAcctUsingAtData(atData);
if (acct == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!acct.getBlockchain().isValidWalletKey(tradeBotRespondRequest.foreignKey))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
if (crossChainTradeData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (crossChainTradeData.mode != AcctMode.OFFERING)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
AcctTradeBot.ResponseResult result = TradeBot.getInstance().startResponse(repository, atData, acct, crossChainTradeData,
tradeBotRespondRequest.foreignKey, tradeBotRespondRequest.receivingAddress);
switch (result) {
case OK:
return "true";
case BALANCE_ISSUE:
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
case NETWORK_ISSUE:
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
default:
return "false";
}
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
@DELETE
@Operation(
summary = "Delete completed trade",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
example = "93MB2qRDNVLxbmmPuYpLdAqn3u2x9ZhaVZK5wELHueP8"
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string"))
)
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.REPOSITORY_ISSUE})
public String tradeBotDelete(String tradePrivateKey58) {
Security.checkApiCallAllowed(request);
final byte[] tradePrivateKey;
try {
tradePrivateKey = Base58.decode(tradePrivateKey58);
if (tradePrivateKey.length != 32)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
} catch (NumberFormatException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Handed off to TradeBot
return TradeBot.getInstance().deleteEntry(repository, tradePrivateKey) ? "true" : "false";
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
private ATData fetchAtDataWithChecking(Repository repository, String atAddress) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atAddress);
if (atData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
// No point sending message to AT that's finished
if (atData.getIsFinished())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
return atData;
}
}

View File

@@ -6,8 +6,11 @@ import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.net.InetSocketAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
@@ -26,10 +29,17 @@ import org.qortal.api.ApiException;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.ConnectedPeer;
import org.qortal.controller.Controller;
import org.qortal.controller.Synchronizer;
import org.qortal.controller.Synchronizer.SynchronizationResult;
import org.qortal.data.block.BlockSummaryData;
import org.qortal.data.network.PeerData;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.PeerAddress;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.utils.ExecuteProduceConsume;
import org.qortal.utils.NTP;
@@ -122,6 +132,7 @@ public class PeersResource {
)
}
)
@SecurityRequirement(name = "apiKey")
public ExecuteProduceConsume.StatsSnapshot getEngineStats() {
Security.checkApiCallAllowed(request);
@@ -159,6 +170,7 @@ public class PeersResource {
@ApiErrors({
ApiError.INVALID_NETWORK_ADDRESS, ApiError.REPOSITORY_ISSUE
})
@SecurityRequirement(name = "apiKey")
public String addPeer(String address) {
Security.checkApiCallAllowed(request);
@@ -213,6 +225,7 @@ public class PeersResource {
@ApiErrors({
ApiError.INVALID_NETWORK_ADDRESS, ApiError.REPOSITORY_ISSUE
})
@SecurityRequirement(name = "apiKey")
public String removePeer(String address) {
Security.checkApiCallAllowed(request);
@@ -248,6 +261,7 @@ public class PeersResource {
@ApiErrors({
ApiError.REPOSITORY_ISSUE
})
@SecurityRequirement(name = "apiKey")
public String removeKnownPeers(String address) {
Security.checkApiCallAllowed(request);
@@ -260,4 +274,68 @@ public class PeersResource {
}
}
@POST
@Path("/commonblock")
@Operation(
summary = "Report common block with given peer.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string", example = "node2.qortal.org"
)
)
),
responses = {
@ApiResponse(
description = "the block",
content = @Content(
array = @ArraySchema(
schema = @Schema(
implementation = BlockSummaryData.class
)
)
)
)
}
)
@ApiErrors({ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
public List<BlockSummaryData> commonBlock(String targetPeerAddress) {
Security.checkApiCallAllowed(request);
try {
// Try to resolve passed address to make things easier
PeerAddress peerAddress = PeerAddress.fromString(targetPeerAddress);
InetSocketAddress resolvedAddress = peerAddress.toSocketAddress();
List<Peer> peers = Network.getInstance().getHandshakedPeers();
Peer targetPeer = peers.stream().filter(peer -> peer.getResolvedAddress().equals(resolvedAddress)).findFirst().orElse(null);
if (targetPeer == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
try (final Repository repository = RepositoryManager.getRepository()) {
int ourInitialHeight = Controller.getInstance().getChainHeight();
boolean force = true;
List<BlockSummaryData> peerBlockSummaries = new ArrayList<>();
SynchronizationResult findCommonBlockResult = Synchronizer.getInstance().fetchSummariesFromCommonBlock(repository, targetPeer, ourInitialHeight, force, peerBlockSummaries, true);
if (findCommonBlockResult != SynchronizationResult.OK)
return null;
return peerBlockSummaries;
}
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
} catch (UnknownHostException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
} catch (InterruptedException e) {
return null;
}
}
}

View File

@@ -510,14 +510,19 @@ public class TransactionsResource {
if (!Controller.getInstance().isUpToDate())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
byte[] rawBytes = Base58.decode(rawBytes58);
TransactionData transactionData;
try {
transactionData = TransactionTransformer.fromBytes(rawBytes);
} catch (TransformationException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSFORMATION_ERROR, e);
}
if (transactionData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
try (final Repository repository = RepositoryManager.getRepository()) {
byte[] rawBytes = Base58.decode(rawBytes58);
TransactionData transactionData = TransactionTransformer.fromBytes(rawBytes);
if (transactionData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
Transaction transaction = Transaction.fromData(repository, transactionData);
if (!transaction.isSignatureValid())
@@ -535,16 +540,9 @@ public class TransactionsResource {
blockchainLock.unlock();
}
// Notify controller of new transaction
Controller.getInstance().onNewTransaction(transactionData, null);
return "true";
} catch (NumberFormatException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA, e);
} catch (TransformationException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSFORMATION_ERROR, e);
} catch (ApiException e) {
throw e;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
} catch (InterruptedException e) {

View File

@@ -33,7 +33,6 @@ import org.qortal.transaction.Transaction.TransactionType;
import org.qortal.transform.Transformer;
import org.qortal.transform.transaction.TransactionTransformer;
import org.qortal.transform.transaction.TransactionTransformer.Transformation;
import org.qortal.utils.BIP39;
import org.qortal.utils.Base58;
import com.google.common.hash.HashCode;
@@ -195,123 +194,6 @@ public class UtilsResource {
return Base58.encode(random);
}
@GET
@Path("/mnemonic")
@Operation(
summary = "Generate 12-word BIP39 mnemonic",
description = "Optionally pass 16-byte, base58-encoded entropy or entropy will be internally generated.<br>"
+ "Example entropy input: YcVfxkQb6JRzqk5kF2tNLv",
responses = {
@ApiResponse(
description = "mnemonic",
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string"
)
)
)
}
)
@ApiErrors({ApiError.NON_PRODUCTION, ApiError.INVALID_DATA})
public String getMnemonic(@QueryParam("entropy") String suppliedEntropy) {
if (Settings.getInstance().isApiRestricted())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.NON_PRODUCTION);
/*
* BIP39 word lists have 2048 entries so can be represented by 11 bits.
* UUID (128bits) and another 4 bits gives 132 bits.
* 132 bits, divided by 11, gives 12 words.
*/
byte[] entropy;
if (suppliedEntropy != null) {
// Use caller-supplied entropy input
try {
entropy = Base58.decode(suppliedEntropy);
} catch (NumberFormatException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
}
// Must be 16-bytes
if (entropy.length != 16)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
} else {
// Generate entropy internally
UUID uuid = UUID.randomUUID();
byte[] uuidMSB = Longs.toByteArray(uuid.getMostSignificantBits());
byte[] uuidLSB = Longs.toByteArray(uuid.getLeastSignificantBits());
entropy = Bytes.concat(uuidMSB, uuidLSB);
}
// Use SHA256 to generate more bits
byte[] hash = Crypto.digest(entropy);
// Append first 4 bits from hash to end. (Actually 8 bits but we only use 4).
byte checksum = (byte) (hash[0] & 0xf0);
entropy = Bytes.concat(entropy, new byte[] {
checksum
});
return BIP39.encode(entropy, "en");
}
@POST
@Path("/mnemonic")
@Operation(
summary = "Calculate binary entropy from 12-word BIP39 mnemonic",
description = "Returns the base58-encoded binary form, or \"false\" if mnemonic is invalid.",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string"
)
)
),
responses = {
@ApiResponse(
description = "entropy in base58",
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string"
)
)
)
}
)
@ApiErrors({ApiError.NON_PRODUCTION})
public String fromMnemonic(String mnemonic) {
if (Settings.getInstance().isApiRestricted())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.NON_PRODUCTION);
if (mnemonic.isEmpty())
return "false";
// Strip leading/trailing whitespace if any
mnemonic = mnemonic.trim();
String[] phraseWords = mnemonic.split(" ");
if (phraseWords.length != 12)
return "false";
// Convert BIP39 mnemonic to binary
byte[] binary = BIP39.decode(phraseWords, "en");
if (binary == null)
return "false";
byte[] entropy = Arrays.copyOf(binary, 16); // 132 bits is 16.5 bytes, but we're discarding checksum nybble
byte checksumNybble = (byte) (binary[16] & 0xf0);
byte[] checksum = Crypto.digest(entropy);
if (checksumNybble != (byte) (checksum[0] & 0xf0))
return "false";
return Base58.encode(entropy);
}
@POST
@Path("/privatekey")
@Operation(

View File

@@ -31,6 +31,7 @@ public class ActiveChatsWebSocket extends ApiWebSocket {
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
Map<String, String> pathParams = getPathParams(session, "/{address}");
@@ -49,16 +50,19 @@ public class ActiveChatsWebSocket extends ApiWebSocket {
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
ChatNotifier.getInstance().deregister(session);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* ignored */
}
@OnWebSocketMessage
public void onWebSocketMessage(Session session, String message) {
/* ignored */
}
private void onNotify(Session session, ChatTransactionData chatTransactionData, String ourAddress, AtomicReference<String> previousOutput) {

View File

@@ -13,59 +13,87 @@ import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.api.model.NodeStatus;
import org.qortal.controller.StatusNotifier;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.controller.Controller;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
@WebSocket
@SuppressWarnings("serial")
public class AdminStatusWebSocket extends ApiWebSocket {
public class AdminStatusWebSocket extends ApiWebSocket implements Listener {
private static final AtomicReference<String> previousOutput = new AtomicReference<>(null);
@Override
public void configure(WebSocketServletFactory factory) {
factory.register(AdminStatusWebSocket.class);
try {
previousOutput.set(buildStatusString());
} catch (IOException e) {
// How to fail properly?
return;
}
EventBus.INSTANCE.addListener(this::listen);
}
@Override
public void listen(Event event) {
if (!(event instanceof Controller.StatusChangeEvent))
return;
String newOutput;
try {
newOutput = buildStatusString();
} catch (IOException e) {
// Ignore this time?
return;
}
if (previousOutput.getAndUpdate(currentValue -> newOutput).equals(newOutput))
// Output hasn't changed, so don't send anything
return;
for (Session session : getSessions())
this.sendStatus(session, newOutput);
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
AtomicReference<String> previousOutput = new AtomicReference<>(null);
this.sendStatus(session, previousOutput.get());
StatusNotifier.Listener listener = timestamp -> onNotify(session, previousOutput);
StatusNotifier.getInstance().register(session, listener);
this.onNotify(session, previousOutput);
super.onWebSocketConnect(session);
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
StatusNotifier.getInstance().deregister(session);
super.onWebSocketClose(session, statusCode, reason);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* We ignore errors for now, but method here to silence log spam */
}
@OnWebSocketMessage
public void onWebSocketMessage(Session session, String message) {
/* ignored */
}
private void onNotify(Session session,AtomicReference<String> previousOutput) {
try (final Repository repository = RepositoryManager.getRepository()) {
NodeStatus nodeStatus = new NodeStatus();
private static String buildStatusString() throws IOException {
NodeStatus nodeStatus = new NodeStatus();
StringWriter stringWriter = new StringWriter();
marshall(stringWriter, nodeStatus);
return stringWriter.toString();
}
StringWriter stringWriter = new StringWriter();
marshall(stringWriter, nodeStatus);
// Only output if something has changed
String output = stringWriter.toString();
if (output.equals(previousOutput.get()))
return;
previousOutput.set(output);
session.getRemote().sendStringByFuture(output);
} catch (DataException | IOException | WebSocketException e) {
private void sendStatus(Session session, String status) {
try {
session.getRemote().sendStringByFuture(status);
} catch (WebSocketException e) {
// No output this time?
}
}

View File

@@ -107,7 +107,9 @@ abstract class ApiWebSocket extends WebSocketServlet {
public void onWebSocketClose(Session session, int statusCode, String reason) {
synchronized (SESSIONS_BY_CLASS) {
SESSIONS_BY_CLASS.get(this.getClass()).remove(session);
List<Session> sessions = SESSIONS_BY_CLASS.get(this.getClass());
if (sessions != null)
sessions.remove(session);
}
}

View File

@@ -13,8 +13,12 @@ import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.api.ApiError;
import org.qortal.api.model.BlockInfo;
import org.qortal.controller.BlockNotifier;
import org.qortal.controller.Controller;
import org.qortal.data.block.BlockData;
import org.qortal.data.block.BlockSummaryData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@@ -22,26 +26,42 @@ import org.qortal.utils.Base58;
@WebSocket
@SuppressWarnings("serial")
public class BlocksWebSocket extends ApiWebSocket {
public class BlocksWebSocket extends ApiWebSocket implements Listener {
@Override
public void configure(WebSocketServletFactory factory) {
factory.register(BlocksWebSocket.class);
EventBus.INSTANCE.addListener(this::listen);
}
@Override
public void listen(Event event) {
if (!(event instanceof Controller.NewBlockEvent))
return;
BlockData blockData = ((Controller.NewBlockEvent) event).getBlockData();
BlockSummaryData blockSummary = new BlockSummaryData(blockData);
for (Session session : getSessions())
sendBlockSummary(session, blockSummary);
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
BlockNotifier.Listener listener = blockInfo -> onNotify(session, blockInfo);
BlockNotifier.getInstance().register(session, listener);
super.onWebSocketConnect(session);
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
BlockNotifier.getInstance().deregister(session);
super.onWebSocketClose(session, statusCode, reason);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* We ignore errors for now, but method here to silence log spam */
}
@OnWebSocketMessage
@@ -65,13 +85,13 @@ public class BlocksWebSocket extends ApiWebSocket {
return;
}
List<BlockInfo> blockInfos = repository.getBlockRepository().getBlockInfos(height, null, 1);
if (blockInfos == null || blockInfos.isEmpty()) {
List<BlockSummaryData> blockSummaries = repository.getBlockRepository().getBlockSummaries(height, height);
if (blockSummaries == null || blockSummaries.isEmpty()) {
sendError(session, ApiError.BLOCK_UNKNOWN);
return;
}
onNotify(session, blockInfos.get(0));
sendBlockSummary(session, blockSummaries.get(0));
} catch (DataException e) {
sendError(session, ApiError.REPOSITORY_ISSUE);
}
@@ -94,23 +114,23 @@ public class BlocksWebSocket extends ApiWebSocket {
}
try (final Repository repository = RepositoryManager.getRepository()) {
List<BlockInfo> blockInfos = repository.getBlockRepository().getBlockInfos(height, null, 1);
if (blockInfos == null || blockInfos.isEmpty()) {
List<BlockSummaryData> blockSummaries = repository.getBlockRepository().getBlockSummaries(height, height);
if (blockSummaries == null || blockSummaries.isEmpty()) {
sendError(session, ApiError.BLOCK_UNKNOWN);
return;
}
onNotify(session, blockInfos.get(0));
sendBlockSummary(session, blockSummaries.get(0));
} catch (DataException e) {
sendError(session, ApiError.REPOSITORY_ISSUE);
}
}
private void onNotify(Session session, BlockInfo blockInfo) {
private void sendBlockSummary(Session session, BlockSummaryData blockSummary) {
StringWriter stringWriter = new StringWriter();
try {
marshall(stringWriter, blockInfo);
marshall(stringWriter, blockSummary);
session.getRemote().sendStringByFuture(stringWriter.toString());
} catch (IOException | WebSocketException e) {

View File

@@ -32,6 +32,7 @@ public class ChatMessagesWebSocket extends ApiWebSocket {
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
@@ -86,16 +87,19 @@ public class ChatMessagesWebSocket extends ApiWebSocket {
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
ChatNotifier.getInstance().deregister(session);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* ignored */
}
@OnWebSocketMessage
public void onWebSocketMessage(Session session, String message) {
/* ignored */
}
private void onNotify(Session session, ChatTransactionData chatTransactionData, int txGroupId) {
@@ -111,6 +115,9 @@ public class ChatMessagesWebSocket extends ApiWebSocket {
}
private void onNotify(Session session, ChatTransactionData chatTransactionData, List<String> involvingAddresses) {
if (chatTransactionData == null)
return;
// We only want direct/non-group messages where sender/recipient match our addresses
String recipient = chatTransactionData.getRecipient();
if (recipient == null)

View File

@@ -0,0 +1,244 @@
package org.qortal.api.websocket;
import java.io.IOException;
import java.io.StringWriter;
import java.util.Collections;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketError;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.controller.Controller;
import org.qortal.crypto.Crypto;
import org.qortal.data.transaction.PresenceTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.transaction.PresenceTransaction.PresenceType;
import org.qortal.transaction.Transaction.TransactionType;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
@WebSocket
@SuppressWarnings("serial")
public class PresenceWebSocket extends ApiWebSocket implements Listener {
@XmlAccessorType(XmlAccessType.FIELD)
@SuppressWarnings("unused")
private static class PresenceInfo {
private final PresenceType presenceType;
private final String publicKey;
private final long timestamp;
private final String address;
protected PresenceInfo() {
this.presenceType = null;
this.publicKey = null;
this.timestamp = 0L;
this.address = null;
}
public PresenceInfo(PresenceType presenceType, String pubKey58, long timestamp) {
this.presenceType = presenceType;
this.publicKey = pubKey58;
this.timestamp = timestamp;
this.address = Crypto.toAddress(Base58.decode(this.publicKey));
}
public PresenceType getPresenceType() {
return this.presenceType;
}
public String getPublicKey() {
return this.publicKey;
}
public long getTimestamp() {
return this.timestamp;
}
public String getAddress() {
return this.address;
}
}
/** Outer map key is PresenceType (enum), inner map key is public key in base58, inner map value is timestamp */
private static final Map<PresenceType, Map<String, Long>> currentEntries = Collections.synchronizedMap(new EnumMap<>(PresenceType.class));
/** (Optional) PresenceType used for filtering by that Session. */
private static final Map<Session, PresenceType> sessionPresenceTypes = Collections.synchronizedMap(new HashMap<>());
@Override
public void configure(WebSocketServletFactory factory) {
factory.register(PresenceWebSocket.class);
try (final Repository repository = RepositoryManager.getRepository()) {
populateCurrentInfo(repository);
} catch (DataException e) {
// How to fail properly?
return;
}
EventBus.INSTANCE.addListener(this::listen);
}
@Override
public void listen(Event event) {
// We use NewBlockEvent as a proxy for 1-minute timer
if (!(event instanceof Controller.NewTransactionEvent) && !(event instanceof Controller.NewBlockEvent))
return;
removeOldEntries();
if (event instanceof Controller.NewBlockEvent)
// We only wanted a chance to cull old entries
return;
TransactionData transactionData = ((Controller.NewTransactionEvent) event).getTransactionData();
if (transactionData.getType() != TransactionType.PRESENCE)
return;
PresenceTransactionData presenceData = (PresenceTransactionData) transactionData;
PresenceType presenceType = presenceData.getPresenceType();
// Put/replace for this publickey making sure we keep newest timestamp
String pubKey58 = Base58.encode(presenceData.getCreatorPublicKey());
long ourTimestamp = presenceData.getTimestamp();
long computedTimestamp = mergePresence(presenceType, pubKey58, ourTimestamp);
if (computedTimestamp != ourTimestamp)
// nothing changed
return;
List<PresenceInfo> presenceInfo = Collections.singletonList(new PresenceInfo(presenceType, pubKey58, computedTimestamp));
// Notify sessions
for (Session session : getSessions()) {
PresenceType sessionPresenceType = sessionPresenceTypes.get(session);
if (sessionPresenceType == null || sessionPresenceType == presenceType)
sendPresenceInfo(session, presenceInfo);
}
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
List<String> presenceTypes = queryParams.get("presenceType");
// We only support ONE presenceType
String presenceTypeName = presenceTypes == null || presenceTypes.isEmpty() ? null : presenceTypes.get(0);
PresenceType presenceType = presenceTypeName == null ? null : PresenceType.fromString(presenceTypeName);
// Make sure that if caller does give a presenceType, that it is a valid/known one.
if (presenceTypeName != null && presenceType == null) {
session.close(4003, "unknown presenceType: " + presenceTypeName);
return;
}
// Save session's requested PresenceType, if given
if (presenceType != null)
sessionPresenceTypes.put(session, presenceType);
List<PresenceInfo> presenceInfo;
synchronized (currentEntries) {
presenceInfo = currentEntries.entrySet().stream()
.filter(entry -> presenceType == null ? true : entry.getKey() == presenceType)
.flatMap(entry -> entry.getValue().entrySet().stream().map(innerEntry -> new PresenceInfo(entry.getKey(), innerEntry.getKey(), innerEntry.getValue())))
.collect(Collectors.toList());
}
if (!sendPresenceInfo(session, presenceInfo)) {
session.close(4002, "websocket issue");
return;
}
super.onWebSocketConnect(session);
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
// clean up
sessionPresenceTypes.remove(session);
super.onWebSocketClose(session, statusCode, reason);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* ignored */
}
@OnWebSocketMessage
public void onWebSocketMessage(Session session, String message) {
/* ignored */
}
private boolean sendPresenceInfo(Session session, List<PresenceInfo> presenceInfo) {
try {
StringWriter stringWriter = new StringWriter();
marshall(stringWriter, presenceInfo);
String output = stringWriter.toString();
session.getRemote().sendStringByFuture(output);
} catch (IOException e) {
// No output this time?
return false;
}
return true;
}
private static void populateCurrentInfo(Repository repository) throws DataException {
// We want ALL PRESENCE transactions
List<TransactionData> presenceTransactionsData = repository.getTransactionRepository().getUnconfirmedTransactions(TransactionType.PRESENCE, null);
for (TransactionData transactionData : presenceTransactionsData) {
PresenceTransactionData presenceData = (PresenceTransactionData) transactionData;
PresenceType presenceType = presenceData.getPresenceType();
// Put/replace for this publickey making sure we keep newest timestamp
String pubKey58 = Base58.encode(presenceData.getCreatorPublicKey());
long ourTimestamp = presenceData.getTimestamp();
mergePresence(presenceType, pubKey58, ourTimestamp);
}
}
private static long mergePresence(PresenceType presenceType, String pubKey58, long ourTimestamp) {
Map<String, Long> typedPubkeyTimestamps = currentEntries.computeIfAbsent(presenceType, someType -> Collections.synchronizedMap(new HashMap<>()));
return typedPubkeyTimestamps.compute(pubKey58, (somePubKey58, currentTimestamp) -> (currentTimestamp == null || currentTimestamp < ourTimestamp) ? ourTimestamp : currentTimestamp);
}
private static void removeOldEntries() {
long now = NTP.getTime();
currentEntries.entrySet().forEach(entry -> {
long expiryThreshold = now - entry.getKey().getLifetime();
entry.getValue().entrySet().removeIf(pubkeyTimestamp -> pubkeyTimestamp.getValue() < expiryThreshold);
});
}
}

View File

@@ -11,10 +11,12 @@ import java.util.stream.Collectors;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketError;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.controller.TradeBot;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.crosschain.SupportedBlockchain;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
@@ -29,7 +31,9 @@ import org.qortal.utils.Base58;
public class TradeBotWebSocket extends ApiWebSocket implements Listener {
/** Cache of trade-bot entry states, keyed by trade-bot entry's "trade private key" (base58) */
private static final Map<String, TradeBotData.State> PREVIOUS_STATES = new HashMap<>();
private static final Map<String, Integer> PREVIOUS_STATES = new HashMap<>();
private static final Map<Session, String> sessionBlockchain = Collections.synchronizedMap(new HashMap<>());
@Override
public void configure(WebSocketServletFactory factory) {
@@ -41,7 +45,7 @@ public class TradeBotWebSocket extends ApiWebSocket implements Listener {
// How do we properly fail here?
return;
PREVIOUS_STATES.putAll(tradeBotEntries.stream().collect(Collectors.toMap(entry -> Base58.encode(entry.getTradePrivateKey()), TradeBotData::getState)));
PREVIOUS_STATES.putAll(tradeBotEntries.stream().collect(Collectors.toMap(entry -> Base58.encode(entry.getTradePrivateKey()), TradeBotData::getStateValue)));
} catch (DataException e) {
// No output this time
}
@@ -58,44 +62,78 @@ public class TradeBotWebSocket extends ApiWebSocket implements Listener {
String tradePrivateKey58 = Base58.encode(tradeBotData.getTradePrivateKey());
synchronized (PREVIOUS_STATES) {
if (PREVIOUS_STATES.get(tradePrivateKey58) == tradeBotData.getState())
Integer previousStateValue = PREVIOUS_STATES.get(tradePrivateKey58);
if (previousStateValue != null && previousStateValue == tradeBotData.getStateValue())
// Not changed
return;
PREVIOUS_STATES.put(tradePrivateKey58, tradeBotData.getState());
PREVIOUS_STATES.put(tradePrivateKey58, tradeBotData.getStateValue());
}
List<TradeBotData> tradeBotEntries = Collections.singletonList(tradeBotData);
for (Session session : getSessions())
sendEntries(session, tradeBotEntries);
for (Session session : getSessions()) {
// Only send if this session has this/no preferred blockchain
String preferredBlockchain = sessionBlockchain.get(session);
if (preferredBlockchain == null || preferredBlockchain.equals(tradeBotData.getForeignBlockchain()))
sendEntries(session, tradeBotEntries);
}
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
List<String> foreignBlockchains = queryParams.get("foreignBlockchain");
final String foreignBlockchain = foreignBlockchains == null ? null : foreignBlockchains.get(0);
// Make sure blockchain (if any) is valid
if (foreignBlockchain != null && SupportedBlockchain.fromString(foreignBlockchain) == null) {
session.close(4003, "unknown blockchain: " + foreignBlockchain);
return;
}
// save session's preferred blockchain (if any)
sessionBlockchain.put(session, foreignBlockchain);
// Send all known trade-bot entries
try (final Repository repository = RepositoryManager.getRepository()) {
List<TradeBotData> tradeBotEntries = repository.getCrossChainRepository().getAllTradeBotData();
if (tradeBotEntries == null) {
session.close(4001, "repository issue fetching trade-bot entries");
return;
}
// Optional filtering
if (foreignBlockchain != null)
tradeBotEntries = tradeBotEntries.stream()
.filter(tradeBotData -> tradeBotData.getForeignBlockchain().equals(foreignBlockchain))
.collect(Collectors.toList());
if (!sendEntries(session, tradeBotEntries)) {
session.close(4002, "websocket issue");
return;
}
} catch (DataException e) {
// No output this time
session.close(4001, "repository issue fetching trade-bot entries");
return;
}
super.onWebSocketConnect(session);
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
// clean up
sessionBlockchain.remove(session);
super.onWebSocketClose(session, statusCode, reason);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* ignored */
}
@OnWebSocketMessage
public void onWebSocketMessage(Session session, String message) {
/* ignored */

View File

@@ -3,128 +3,228 @@ package org.qortal.api.websocket;
import java.io.IOException;
import java.io.StringWriter;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Predicate;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketClose;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketError;
import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.api.model.BlockInfo;
import org.qortal.api.model.CrossChainOfferSummary;
import org.qortal.controller.BlockNotifier;
import org.qortal.crosschain.BTCACCT;
import org.qortal.controller.Controller;
import org.qortal.crosschain.SupportedBlockchain;
import org.qortal.crosschain.ACCT;
import org.qortal.crosschain.AcctMode;
import org.qortal.data.at.ATStateData;
import org.qortal.data.block.BlockData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.utils.ByteArray;
import org.qortal.utils.NTP;
@WebSocket
@SuppressWarnings("serial")
public class TradeOffersWebSocket extends ApiWebSocket {
public class TradeOffersWebSocket extends ApiWebSocket implements Listener {
private static final Logger LOGGER = LogManager.getLogger(TradeOffersWebSocket.class);
private static class CachedOfferInfo {
public final Map<String, AcctMode> previousAtModes = new HashMap<>();
// OFFERING
public final Map<String, CrossChainOfferSummary> currentSummaries = new HashMap<>();
// REDEEMED/REFUNDED/CANCELLED
public final Map<String, CrossChainOfferSummary> historicSummaries = new HashMap<>();
}
// Manual synchronization
private static final Map<String, CachedOfferInfo> cachedInfoByBlockchain = new HashMap<>();
private static final Predicate<CrossChainOfferSummary> isHistoric = offerSummary
-> offerSummary.getMode() == AcctMode.REDEEMED
|| offerSummary.getMode() == AcctMode.REFUNDED
|| offerSummary.getMode() == AcctMode.CANCELLED;
private static final Map<Session, String> sessionBlockchain = Collections.synchronizedMap(new HashMap<>());
@Override
public void configure(WebSocketServletFactory factory) {
factory.register(TradeOffersWebSocket.class);
}
@OnWebSocketConnect
public void onWebSocketConnect(Session session) {
Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
final boolean includeHistoric = queryParams.get("includeHistoric") != null;
final Map<String, BTCACCT.Mode> previousAtModes = new HashMap<>();
List<CrossChainOfferSummary> crossChainOfferSummaries;
try (final Repository repository = RepositoryManager.getRepository()) {
List<ATStateData> initialAtStates;
populateCurrentSummaries(repository);
// We want ALL OFFERING trades
Boolean isFinished = Boolean.FALSE;
Integer dataByteOffset = BTCACCT.MODE_BYTE_OFFSET;
Long expectedValue = (long) BTCACCT.Mode.OFFERING.value;
Integer minimumFinalHeight = null;
populateHistoricSummaries(repository);
} catch (DataException e) {
// How to fail properly?
return;
}
initialAtStates = repository.getATRepository().getMatchingFinalATStates(BTCACCT.CODE_BYTES_HASH,
isFinished, dataByteOffset, expectedValue, minimumFinalHeight,
null, null, null);
EventBus.INSTANCE.addListener(this::listen);
}
if (initialAtStates == null) {
session.close(4001, "repository issue fetching OFFERING trades");
return;
}
@Override
public void listen(Event event) {
if (!(event instanceof Controller.NewBlockEvent))
return;
// Save initial AT modes
previousAtModes.putAll(initialAtStates.stream().collect(Collectors.toMap(ATStateData::getATAddress, atState -> BTCACCT.Mode.OFFERING)));
BlockData blockData = ((Controller.NewBlockEvent) event).getBlockData();
// Convert to offer summaries
crossChainOfferSummaries = produceSummaries(repository, initialAtStates, null);
// Process any new info
if (includeHistoric) {
// We also want REDEEMED/REFUNDED/CANCELLED trades over the last 24 hours
long timestamp = NTP.getTime() - 24 * 60 * 60 * 1000L;
minimumFinalHeight = repository.getBlockRepository().getHeightFromTimestamp(timestamp);
try (final Repository repository = RepositoryManager.getRepository()) {
// Find any new/changed trade ATs since this block
final Boolean isFinished = null;
final Integer dataByteOffset = null;
final Long expectedValue = null;
final Integer minimumFinalHeight = blockData.getHeight();
if (minimumFinalHeight != 0) {
isFinished = Boolean.TRUE;
dataByteOffset = null;
expectedValue = null;
++minimumFinalHeight; // because height is just *before* timestamp
for (SupportedBlockchain blockchain : SupportedBlockchain.values()) {
Map<ByteArray, Supplier<ACCT>> acctsByCodeHash = SupportedBlockchain.getFilteredAcctMap(blockchain);
List<ATStateData> historicAtStates = repository.getATRepository().getMatchingFinalATStates(BTCACCT.CODE_BYTES_HASH,
isFinished, dataByteOffset, expectedValue, minimumFinalHeight,
null, null, null);
List<CrossChainOfferSummary> crossChainOfferSummaries = new ArrayList<>();
if (historicAtStates == null) {
session.close(4002, "repository issue fetching historic trades");
return;
synchronized (cachedInfoByBlockchain) {
CachedOfferInfo cachedInfo = cachedInfoByBlockchain.computeIfAbsent(blockchain.name(), k -> new CachedOfferInfo());
for (Map.Entry<ByteArray, Supplier<ACCT>> acctInfo : acctsByCodeHash.entrySet()) {
byte[] codeHash = acctInfo.getKey().value;
ACCT acct = acctInfo.getValue().get();
List<ATStateData> atStates = repository.getATRepository().getMatchingFinalATStates(codeHash,
isFinished, dataByteOffset, expectedValue, minimumFinalHeight,
null, null, null);
crossChainOfferSummaries.addAll(produceSummaries(repository, acct, atStates, blockData.getTimestamp()));
}
for (ATStateData historicAtState : historicAtStates) {
CrossChainOfferSummary historicOfferSummary = produceSummary(repository, historicAtState, null);
// Remove any entries unchanged from last time
crossChainOfferSummaries.removeIf(offerSummary -> cachedInfo.previousAtModes.get(offerSummary.getQortalAtAddress()) == offerSummary.getMode());
// Skip to next blockchain if nothing has changed (for this blockchain)
if (crossChainOfferSummaries.isEmpty())
continue;
// Update
for (CrossChainOfferSummary offerSummary : crossChainOfferSummaries) {
String offerAtAddress = offerSummary.getQortalAtAddress();
cachedInfo.previousAtModes.put(offerAtAddress, offerSummary.getMode());
LOGGER.trace(() -> String.format("Block height: %d, AT: %s, mode: %s", blockData.getHeight(), offerAtAddress, offerSummary.getMode().name()));
switch (offerSummary.getMode()) {
case OFFERING:
cachedInfo.currentSummaries.put(offerAtAddress, offerSummary);
cachedInfo.historicSummaries.remove(offerAtAddress);
break;
switch (historicOfferSummary.getMode()) {
case REDEEMED:
case REFUNDED:
case CANCELLED:
cachedInfo.currentSummaries.remove(offerAtAddress);
cachedInfo.historicSummaries.put(offerAtAddress, offerSummary);
break;
default:
continue;
case TRADING:
cachedInfo.currentSummaries.remove(offerAtAddress);
cachedInfo.historicSummaries.remove(offerAtAddress);
break;
}
// Add summary to initial burst
crossChainOfferSummaries.add(historicOfferSummary);
// Save initial AT mode
previousAtModes.put(historicAtState.getATAddress(), historicOfferSummary.getMode());
}
}
}
// Remove any historic offers that are over 24 hours old
final long tooOldTimestamp = NTP.getTime() - 24 * 60 * 60 * 1000L;
cachedInfo.historicSummaries.values().removeIf(historicSummary -> historicSummary.getTimestamp() < tooOldTimestamp);
}
// Notify sessions
for (Session session : getSessions()) {
// Only send if this session has this/no preferred blockchain
String preferredBlockchain = sessionBlockchain.get(session);
if (preferredBlockchain == null || preferredBlockchain.equals(blockchain.name()))
sendOfferSummaries(session, crossChainOfferSummaries);
}
}
} catch (DataException e) {
session.close(4003, "generic repository issue");
// No output this time
}
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
final boolean includeHistoric = queryParams.get("includeHistoric") != null;
List<String> foreignBlockchains = queryParams.get("foreignBlockchain");
final String foreignBlockchain = foreignBlockchains == null ? null : foreignBlockchains.get(0);
// Make sure blockchain (if any) is valid
if (foreignBlockchain != null && SupportedBlockchain.fromString(foreignBlockchain) == null) {
session.close(4003, "unknown blockchain: " + foreignBlockchain);
return;
}
// Save session's preferred blockchain, if given
if (foreignBlockchain != null)
sessionBlockchain.put(session, foreignBlockchain);
List<CrossChainOfferSummary> crossChainOfferSummaries = new ArrayList<>();
synchronized (cachedInfoByBlockchain) {
Collection<CachedOfferInfo> cachedInfos;
if (foreignBlockchain == null)
// No preferred blockchain, so iterate through all of them
cachedInfos = cachedInfoByBlockchain.values();
else
cachedInfos = Collections.singleton(cachedInfoByBlockchain.computeIfAbsent(foreignBlockchain, k -> new CachedOfferInfo()));
for (CachedOfferInfo cachedInfo : cachedInfos) {
crossChainOfferSummaries.addAll(cachedInfo.currentSummaries.values());
if (includeHistoric)
crossChainOfferSummaries.addAll(cachedInfo.historicSummaries.values());
}
}
if (!sendOfferSummaries(session, crossChainOfferSummaries)) {
session.close(4004, "websocket issue");
session.close(4002, "websocket issue");
return;
}
BlockNotifier.Listener listener = blockInfo -> onNotify(session, blockInfo, previousAtModes);
BlockNotifier.getInstance().register(session, listener);
super.onWebSocketConnect(session);
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
BlockNotifier.getInstance().deregister(session);
// clean up
sessionBlockchain.remove(session);
super.onWebSocketClose(session, statusCode, reason);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* ignored */
}
@OnWebSocketMessage
@@ -132,45 +232,6 @@ public class TradeOffersWebSocket extends ApiWebSocket {
/* ignored */
}
private void onNotify(Session session, BlockInfo blockInfo, final Map<String, BTCACCT.Mode> previousAtModes) {
List<CrossChainOfferSummary> crossChainOfferSummaries = null;
try (final Repository repository = RepositoryManager.getRepository()) {
// Find any new trade ATs since this block
final Boolean isFinished = null;
final Integer dataByteOffset = null;
final Long expectedValue = null;
final Integer minimumFinalHeight = blockInfo.getHeight();
List<ATStateData> atStates = repository.getATRepository().getMatchingFinalATStates(BTCACCT.CODE_BYTES_HASH,
isFinished, dataByteOffset, expectedValue, minimumFinalHeight,
null, null, null);
if (atStates == null)
return;
crossChainOfferSummaries = produceSummaries(repository, atStates, blockInfo.getTimestamp());
} catch (DataException e) {
// No output this time
}
synchronized (previousAtModes) { //NOSONAR squid:S2445 suppressed because previousAtModes is final and curried in lambda
// Remove any entries unchanged from last time
crossChainOfferSummaries.removeIf(offerSummary -> previousAtModes.get(offerSummary.getQortalAtAddress()) == offerSummary.getMode());
// Don't send anything if no results
if (crossChainOfferSummaries.isEmpty())
return;
final boolean wasSent = sendOfferSummaries(session, crossChainOfferSummaries);
if (!wasSent)
return;
previousAtModes.putAll(crossChainOfferSummaries.stream().collect(Collectors.toMap(CrossChainOfferSummary::getQortalAtAddress, CrossChainOfferSummary::getMode)));
}
}
private boolean sendOfferSummaries(Session session, List<CrossChainOfferSummary> crossChainOfferSummaries) {
try {
StringWriter stringWriter = new StringWriter();
@@ -186,25 +247,103 @@ public class TradeOffersWebSocket extends ApiWebSocket {
return true;
}
private static CrossChainOfferSummary produceSummary(Repository repository, ATStateData atState, Long timestamp) throws DataException {
CrossChainTradeData crossChainTradeData = BTCACCT.populateTradeData(repository, atState);
private static void populateCurrentSummaries(Repository repository) throws DataException {
// We want ALL OFFERING trades
Boolean isFinished = Boolean.FALSE;
Long expectedValue = (long) AcctMode.OFFERING.value;
Integer minimumFinalHeight = null;
for (SupportedBlockchain blockchain : SupportedBlockchain.values()) {
Map<ByteArray, Supplier<ACCT>> acctsByCodeHash = SupportedBlockchain.getFilteredAcctMap(blockchain);
CachedOfferInfo cachedInfo = cachedInfoByBlockchain.computeIfAbsent(blockchain.name(), k -> new CachedOfferInfo());
for (Map.Entry<ByteArray, Supplier<ACCT>> acctInfo : acctsByCodeHash.entrySet()) {
byte[] codeHash = acctInfo.getKey().value;
ACCT acct = acctInfo.getValue().get();
Integer dataByteOffset = acct.getModeByteOffset();
List<ATStateData> initialAtStates = repository.getATRepository().getMatchingFinalATStates(codeHash,
isFinished, dataByteOffset, expectedValue, minimumFinalHeight,
null, null, null);
if (initialAtStates == null)
throw new DataException("Couldn't fetch current trades from repository");
// Save initial AT modes
cachedInfo.previousAtModes.putAll(initialAtStates.stream().collect(Collectors.toMap(ATStateData::getATAddress, atState -> AcctMode.OFFERING)));
// Convert to offer summaries
cachedInfo.currentSummaries.putAll(produceSummaries(repository, acct, initialAtStates, null).stream()
.collect(Collectors.toMap(CrossChainOfferSummary::getQortalAtAddress, offerSummary -> offerSummary)));
}
}
}
private static void populateHistoricSummaries(Repository repository) throws DataException {
// We want REDEEMED/REFUNDED/CANCELLED trades over the last 24 hours
long timestamp = System.currentTimeMillis() - 24 * 60 * 60 * 1000L;
int minimumFinalHeight = repository.getBlockRepository().getHeightFromTimestamp(timestamp);
if (minimumFinalHeight == 0)
throw new DataException("Couldn't fetch block timestamp from repository");
Boolean isFinished = Boolean.TRUE;
Integer dataByteOffset = null;
Long expectedValue = null;
++minimumFinalHeight; // because height is just *before* timestamp
for (SupportedBlockchain blockchain : SupportedBlockchain.values()) {
Map<ByteArray, Supplier<ACCT>> acctsByCodeHash = SupportedBlockchain.getFilteredAcctMap(blockchain);
CachedOfferInfo cachedInfo = cachedInfoByBlockchain.computeIfAbsent(blockchain.name(), k -> new CachedOfferInfo());
for (Map.Entry<ByteArray, Supplier<ACCT>> acctInfo : acctsByCodeHash.entrySet()) {
byte[] codeHash = acctInfo.getKey().value;
ACCT acct = acctInfo.getValue().get();
List<ATStateData> historicAtStates = repository.getATRepository().getMatchingFinalATStates(codeHash,
isFinished, dataByteOffset, expectedValue, minimumFinalHeight,
null, null, null);
if (historicAtStates == null)
throw new DataException("Couldn't fetch historic trades from repository");
for (ATStateData historicAtState : historicAtStates) {
CrossChainOfferSummary historicOfferSummary = produceSummary(repository, acct, historicAtState, null);
if (!isHistoric.test(historicOfferSummary))
continue;
// Add summary to initial burst
cachedInfo.historicSummaries.put(historicOfferSummary.getQortalAtAddress(), historicOfferSummary);
// Save initial AT mode
cachedInfo.previousAtModes.put(historicOfferSummary.getQortalAtAddress(), historicOfferSummary.getMode());
}
}
}
}
private static CrossChainOfferSummary produceSummary(Repository repository, ACCT acct, ATStateData atState, Long timestamp) throws DataException {
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atState);
long atStateTimestamp;
if (crossChainTradeData.mode == BTCACCT.Mode.OFFERING)
if (crossChainTradeData.mode == AcctMode.OFFERING)
// We want when trade was created, not when it was last updated
atStateTimestamp = atState.getCreation();
atStateTimestamp = crossChainTradeData.creationTimestamp;
else
atStateTimestamp = timestamp != null ? timestamp : repository.getBlockRepository().getTimestampFromHeight(atState.getHeight());
return new CrossChainOfferSummary(crossChainTradeData, atStateTimestamp);
}
private static List<CrossChainOfferSummary> produceSummaries(Repository repository, List<ATStateData> atStates, Long timestamp) throws DataException {
private static List<CrossChainOfferSummary> produceSummaries(Repository repository, ACCT acct, List<ATStateData> atStates, Long timestamp) throws DataException {
List<CrossChainOfferSummary> offerSummaries = new ArrayList<>();
for (ATStateData atState : atStates)
offerSummaries.add(produceSummary(repository, atState, timestamp));
offerSummaries.add(produceSummary(repository, acct, atState, timestamp));
return offerSummaries;
}

View File

@@ -51,16 +51,17 @@ public class AT {
MachineState machineState = new MachineState(api, loggerFactory, deployATTransactionData.getCreationBytes());
byte[] codeHash = Crypto.digest(machineState.getCodeBytes());
byte[] codeBytes = machineState.getCodeBytes();
byte[] codeHash = Crypto.digest(codeBytes);
this.atData = new ATData(atAddress, creatorPublicKey, creation, machineState.version, assetId, machineState.getCodeBytes(), codeHash,
this.atData = new ATData(atAddress, creatorPublicKey, creation, machineState.version, assetId, codeBytes, codeHash,
machineState.isSleeping(), machineState.getSleepUntilHeight(), machineState.isFinished(), machineState.hadFatalError(),
machineState.isFrozen(), machineState.getFrozenBalance());
byte[] stateData = machineState.toBytes();
byte[] stateHash = Crypto.digest(stateData);
this.atStateData = new ATStateData(atAddress, height, creation, stateData, stateHash, 0L, true);
this.atStateData = new ATStateData(atAddress, height, stateData, stateHash, 0L, true);
}
// Getters / setters
@@ -106,12 +107,11 @@ public class AT {
throw new DataException(String.format("Uncaught exception while running AT '%s'", atAddress), e);
}
long creation = this.atData.getCreation();
byte[] stateData = state.toBytes();
byte[] stateHash = Crypto.digest(stateData);
long atFees = api.calcFinalFees(state);
this.atStateData = new ATStateData(atAddress, blockHeight, creation, stateData, stateHash, atFees, false);
this.atStateData = new ATStateData(atAddress, blockHeight, stateData, stateHash, atFees, false);
return api.getTransactions();
}

View File

@@ -146,7 +146,11 @@ public class QortalATAPI extends API {
String atAddress = this.atData.getATAddress();
int height = timestamp.blockHeight;
int sequence = timestamp.transactionSequence + 1;
int sequence = timestamp.transactionSequence;
if (state.getCurrentBlockHeight() < BlockChain.getInstance().getAtFindNextTransactionFixHeight())
// Off-by-one bug still in effect
sequence += 1;
ATRepository.NextTransactionInfo nextTransactionInfo;
try {

View File

@@ -10,7 +10,7 @@ import org.ciyam.at.ExecutionException;
import org.ciyam.at.FunctionData;
import org.ciyam.at.IllegalFunctionCodeException;
import org.ciyam.at.MachineState;
import org.qortal.crosschain.BTC;
import org.qortal.crosschain.Bitcoin;
import org.qortal.crypto.Crypto;
import org.qortal.data.transaction.TransactionData;
import org.qortal.settings.Settings;
@@ -108,7 +108,7 @@ public enum QortalFunctionCode {
CONVERT_B_TO_P2SH(0x0511, 0, false) {
@Override
protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
byte addressPrefix = Settings.getInstance().getBitcoinNet() == BTC.BitcoinNet.MAIN ? 0x05 : (byte) 0xc4;
byte addressPrefix = Settings.getInstance().getBitcoinNet() == Bitcoin.BitcoinNet.MAIN ? 0x05 : (byte) 0xc4;
convertAddressInB(addressPrefix, state);
}

View File

@@ -6,6 +6,8 @@ import static java.util.stream.Collectors.toMap;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.math.RoundingMode;
import java.text.DecimalFormat;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
@@ -15,6 +17,7 @@ import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.Account;
@@ -29,6 +32,7 @@ import org.qortal.controller.Controller;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.data.account.AccountData;
import org.qortal.data.account.EligibleQoraHolderData;
import org.qortal.data.account.QortFromQoraData;
import org.qortal.data.account.RewardShareData;
import org.qortal.data.at.ATData;
@@ -53,7 +57,6 @@ import org.qortal.transform.transaction.TransactionTransformer;
import org.qortal.utils.Amounts;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import org.roaringbitmap.IntIterator;
import com.google.common.primitives.Bytes;
import com.google.common.primitives.Longs;
@@ -128,7 +131,7 @@ public class Block {
@FunctionalInterface
private interface BlockRewardDistributor {
long distribute(long amount, Map<Account, Long> balanceChanges) throws DataException;
long distribute(long amount, Map<String, Long> balanceChanges) throws DataException;
}
/** Lazy-instantiated expanded info on block's online accounts. */
@@ -144,8 +147,8 @@ public class Block {
private final Account recipientAccount;
private final AccountData recipientAccountData;
ExpandedAccount(Repository repository, int accountIndex) throws DataException {
this.rewardShareData = repository.getAccountRepository().getRewardShareByIndex(accountIndex);
ExpandedAccount(Repository repository, RewardShareData rewardShareData) throws DataException {
this.rewardShareData = rewardShareData;
this.sharePercent = this.rewardShareData.getSharePercent();
this.mintingAccount = new Account(repository, this.rewardShareData.getMinter());
@@ -173,27 +176,34 @@ public class Block {
*
* @return account-level share "bin" from blockchain config, or null if founder / none found
*/
public AccountLevelShareBin getShareBin() {
public AccountLevelShareBin getShareBin(int blockHeight) {
if (this.isMinterFounder)
return null;
final int accountLevel = this.mintingAccountData.getLevel();
if (accountLevel <= 0)
return null;
return null; // level 0 isn't included in any share bins
final AccountLevelShareBin[] shareBinsByLevel = BlockChain.getInstance().getShareBinsByAccountLevel();
final BlockChain blockChain = BlockChain.getInstance();
final AccountLevelShareBin[] shareBinsByLevel = blockChain.getShareBinsByAccountLevel();
if (accountLevel > shareBinsByLevel.length)
return null;
return shareBinsByLevel[accountLevel];
if (blockHeight < blockChain.getShareBinFixHeight())
// Off-by-one bug still in effect
return shareBinsByLevel[accountLevel];
// level 1 stored at index 0, level 2 stored at index 1, etc.
return shareBinsByLevel[accountLevel-1];
}
public long distribute(long accountAmount, Map<Account, Long> balanceChanges) {
public long distribute(long accountAmount, Map<String, Long> balanceChanges) {
if (this.isRecipientAlsoMinter) {
// minter & recipient the same - simpler case
LOGGER.trace(() -> String.format("Minter/recipient account %s share: %s", this.mintingAccount.getAddress(), Amounts.prettyAmount(accountAmount)));
if (accountAmount != 0)
balanceChanges.merge(this.mintingAccount, accountAmount, Long::sum);
balanceChanges.merge(this.mintingAccount.getAddress(), accountAmount, Long::sum);
} else {
// minter & recipient different - extra work needed
long recipientAmount = (accountAmount * this.sharePercent) / 100L / 100L; // because scaled by 2dp and 'percent' means "per 100"
@@ -201,11 +211,11 @@ public class Block {
LOGGER.trace(() -> String.format("Minter account %s share: %s", this.mintingAccount.getAddress(), Amounts.prettyAmount(minterAmount)));
if (minterAmount != 0)
balanceChanges.merge(this.mintingAccount, minterAmount, Long::sum);
balanceChanges.merge(this.mintingAccount.getAddress(), minterAmount, Long::sum);
LOGGER.trace(() -> String.format("Recipient account %s share: %s", this.recipientAccount.getAddress(), Amounts.prettyAmount(recipientAmount)));
if (recipientAmount != 0)
balanceChanges.merge(this.recipientAccount, recipientAmount, Long::sum);
balanceChanges.merge(this.recipientAccount.getAddress(), recipientAmount, Long::sum);
}
// We always distribute all of the amount
@@ -217,10 +227,12 @@ public class Block {
/** Opportunistic cache of this block's valid online accounts. Only created by call to isValid(). */
private List<OnlineAccountData> cachedValidOnlineAccounts = null;
/** Opportunistic cache of this block's valid online reward-shares. Only created by call to isValid(). */
private List<RewardShareData> cachedOnlineRewardShares = null;
// Other useful constants
private static final BigInteger MAX_DISTANCE;
public static final BigInteger MAX_DISTANCE;
static {
byte[] maxValue = new byte[Transformer.PUBLIC_KEY_LENGTH];
Arrays.fill(maxValue, (byte) 0xFF);
@@ -352,12 +364,8 @@ public class Block {
System.arraycopy(onlineAccountData.getSignature(), 0, onlineAccountsSignatures, i * Transformer.SIGNATURE_LENGTH, Transformer.SIGNATURE_LENGTH);
}
byte[] minterSignature;
try {
minterSignature = minter.sign(BlockTransformer.getBytesForMinterSignature(parentBlockData.getMinterSignature(), minter, encodedOnlineAccounts));
} catch (TransformationException e) {
throw new DataException("Unable to calculate next block minter signature", e);
}
byte[] minterSignature = minter.sign(BlockTransformer.getBytesForMinterSignature(parentBlockData,
minter.getPublicKey(), encodedOnlineAccounts));
// Qortal: minter is always a reward-share, so find actual minter and get their effective minting level
int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, minter.getPublicKey());
@@ -423,12 +431,8 @@ public class Block {
int version = this.blockData.getVersion();
byte[] reference = this.blockData.getReference();
byte[] minterSignature;
try {
minterSignature = minter.sign(BlockTransformer.getBytesForMinterSignature(parentBlockData.getMinterSignature(), minter, this.blockData.getEncodedOnlineAccounts()));
} catch (TransformationException e) {
throw new DataException("Unable to calculate next block's minter signature", e);
}
byte[] minterSignature = minter.sign(BlockTransformer.getBytesForMinterSignature(parentBlockData,
minter.getPublicKey(), this.blockData.getEncodedOnlineAccounts()));
// Qortal: minter is always a reward-share, so find actual minter and get their effective minting level
int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, minter.getPublicKey());
@@ -567,22 +571,28 @@ public class Block {
/**
* Return expanded info on block's online accounts.
* <p>
* Typically called as part of Block.process() or Block.orphan()
* so ideally after any calls to Block.isValid().
*
* @throws DataException
*/
public List<ExpandedAccount> getExpandedAccounts() throws DataException {
if (this.cachedExpandedAccounts != null)
return this.cachedExpandedAccounts;
ConciseSet accountIndexes = BlockTransformer.decodeOnlineAccounts(this.blockData.getEncodedOnlineAccounts());
// We might already have a cache of online, reward-shares thanks to isValid()
if (this.cachedOnlineRewardShares == null) {
ConciseSet accountIndexes = BlockTransformer.decodeOnlineAccounts(this.blockData.getEncodedOnlineAccounts());
this.cachedOnlineRewardShares = repository.getAccountRepository().getRewardSharesByIndexes(accountIndexes.toArray());
if (this.cachedOnlineRewardShares == null)
throw new DataException("Online accounts invalid?");
}
List<ExpandedAccount> expandedAccounts = new ArrayList<>();
IntIterator iterator = accountIndexes.iterator();
while (iterator.hasNext()) {
int accountIndex = iterator.next();
ExpandedAccount accountInfo = new ExpandedAccount(repository, accountIndex);
expandedAccounts.add(accountInfo);
}
for (RewardShareData rewardShare : this.cachedOnlineRewardShares)
expandedAccounts.add(new ExpandedAccount(repository, rewardShare));
this.cachedExpandedAccounts = expandedAccounts;
@@ -735,11 +745,7 @@ public class Block {
if (!(this.minter instanceof PrivateKeyAccount))
throw new IllegalStateException("Block's minter is not a PrivateKeyAccount - can't sign!");
try {
this.blockData.setMinterSignature(((PrivateKeyAccount) this.minter).sign(BlockTransformer.getBytesForMinterSignature(this.blockData)));
} catch (TransformationException e) {
throw new RuntimeException("Unable to calculate block's minter signature", e);
}
this.blockData.setMinterSignature(((PrivateKeyAccount) this.minter).sign(BlockTransformer.getBytesForMinterSignature(this.blockData)));
}
/**
@@ -783,16 +789,49 @@ public class Block {
return BigInteger.valueOf(blockSummaryData.getOnlineAccountsCount()).shiftLeft(ACCOUNTS_COUNT_SHIFT).add(keyDistance);
}
public static BigInteger calcChainWeight(int commonBlockHeight, byte[] commonBlockSignature, List<BlockSummaryData> blockSummaries) {
public static BigInteger calcChainWeight(int commonBlockHeight, byte[] commonBlockSignature, List<BlockSummaryData> blockSummaries, int maxHeight) {
BigInteger cumulativeWeight = BigInteger.ZERO;
int parentHeight = commonBlockHeight;
byte[] parentBlockSignature = commonBlockSignature;
NumberFormat formatter = new DecimalFormat("0.###E0");
boolean isLogging = LOGGER.getLevel().isLessSpecificThan(Level.TRACE);
int blockCount = 0;
for (BlockSummaryData blockSummaryData : blockSummaries) {
cumulativeWeight = cumulativeWeight.shiftLeft(CHAIN_WEIGHT_SHIFT).add(calcBlockWeight(parentHeight, parentBlockSignature, blockSummaryData));
blockCount++;
StringBuilder stringBuilder = isLogging ? new StringBuilder(512) : null;
if (isLogging)
stringBuilder.append(formatter.format(cumulativeWeight)).append(" -> ");
cumulativeWeight = cumulativeWeight.shiftLeft(CHAIN_WEIGHT_SHIFT);
if (isLogging)
stringBuilder.append(formatter.format(cumulativeWeight)).append(" + ");
BigInteger blockWeight = calcBlockWeight(parentHeight, parentBlockSignature, blockSummaryData);
if (isLogging)
stringBuilder.append("(height: ")
.append(parentHeight + 1)
.append(", online: ")
.append(blockSummaryData.getOnlineAccountsCount())
.append(") ")
.append(formatter.format(blockWeight));
cumulativeWeight = cumulativeWeight.add(blockWeight);
if (isLogging)
stringBuilder.append(" -> ").append(formatter.format(cumulativeWeight));
if (isLogging && blockSummaries.size() > 1)
LOGGER.debug(() -> stringBuilder.toString()); //NOSONAR S1612 (false positive?)
parentHeight = blockSummaryData.getHeight();
parentBlockSignature = blockSummaryData.getSignature();
// After this timestamp, we only compare the same number of blocks
if (NTP.getTime() >= BlockChain.getInstance().getCalcChainWeightTimestamp() && parentHeight >= maxHeight)
break;
}
LOGGER.trace(String.format("Chain weight calculation was based on %d blocks", blockCount));
return cumulativeWeight;
}
@@ -917,19 +956,9 @@ public class Block {
if (accountIndexes.size() != this.blockData.getOnlineAccountsCount())
return ValidationResult.ONLINE_ACCOUNTS_INVALID;
List<RewardShareData> expandedAccounts = new ArrayList<>();
IntIterator iterator = accountIndexes.iterator();
while (iterator.hasNext()) {
int accountIndex = iterator.next();
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShareByIndex(accountIndex);
// Check that claimed online account actually exists
if (rewardShareData == null)
return ValidationResult.ONLINE_ACCOUNT_UNKNOWN;
expandedAccounts.add(rewardShareData);
}
List<RewardShareData> onlineRewardShares = repository.getAccountRepository().getRewardSharesByIndexes(accountIndexes.toArray());
if (onlineRewardShares == null)
return ValidationResult.ONLINE_ACCOUNT_UNKNOWN;
// If block is past a certain age then we simply assume the signatures were correct
long signatureRequirementThreshold = NTP.getTime() - BlockChain.getInstance().getOnlineAccountSignaturesMinLifetime();
@@ -939,7 +968,7 @@ public class Block {
if (this.blockData.getOnlineAccountsSignatures() == null || this.blockData.getOnlineAccountsSignatures().length == 0)
return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MISSING;
if (this.blockData.getOnlineAccountsSignatures().length != expandedAccounts.size() * Transformer.SIGNATURE_LENGTH)
if (this.blockData.getOnlineAccountsSignatures().length != onlineRewardShares.size() * Transformer.SIGNATURE_LENGTH)
return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MALFORMED;
// Check signatures
@@ -961,7 +990,7 @@ public class Block {
for (int i = 0; i < onlineAccountsSignatures.size(); ++i) {
byte[] signature = onlineAccountsSignatures.get(i);
byte[] publicKey = expandedAccounts.get(i).getRewardSharePublicKey();
byte[] publicKey = onlineRewardShares.get(i).getRewardSharePublicKey();
OnlineAccountData onlineAccountData = new OnlineAccountData(onlineTimestamp, signature, publicKey);
ourOnlineAccounts.add(onlineAccountData);
@@ -982,6 +1011,7 @@ public class Block {
// All online accounts valid, so save our list of online accounts for potential later use
this.cachedValidOnlineAccounts = ourOnlineAccounts;
this.cachedOnlineRewardShares = onlineRewardShares;
return ValidationResult.OK;
}
@@ -1062,6 +1092,10 @@ public class Block {
// Create repository savepoint here so we can rollback to it after testing transactions
repository.setSavepoint();
if (this.blockData.getHeight() == 212937)
// Apply fix for block 212937 but fix will be rolled back before we exit method
Block212937.processFix(this);
for (Transaction transaction : this.getTransactions()) {
TransactionData transactionData = transaction.getTransactionData();
@@ -1265,6 +1299,10 @@ public class Block {
// Distribute block rewards, including transaction fees, before transactions processed
processBlockRewards();
if (this.blockData.getHeight() == 212937)
// Apply fix for block 212937
Block212937.processFix(this);
}
// We're about to (test-)process a batch of transactions,
@@ -1299,6 +1337,9 @@ public class Block {
// Give Controller our cached, valid online accounts data (if any) to help reduce CPU load for next block
Controller.getInstance().pushLatestBlocksOnlineAccounts(this.cachedValidOnlineAccounts);
// Log some debugging info relating to the block weight calculation
this.logDebugInfo();
}
protected void increaseAccountLevels() throws DataException {
@@ -1316,13 +1357,16 @@ public class Block {
allUniqueExpandedAccounts.add(expandedAccount.recipientAccountData);
}
// Decrease blocks minted count for all accounts
// Increase blocks minted count for all accounts
// Batch update in repository
repository.getAccountRepository().modifyMintedBlockCounts(allUniqueExpandedAccounts.stream().map(AccountData::getAddress).collect(Collectors.toList()), +1);
// Local changes and also checks for level bump
for (AccountData accountData : allUniqueExpandedAccounts) {
// Adjust count locally (in Java)
accountData.setBlocksMinted(accountData.getBlocksMinted() + 1);
int rowCount = repository.getAccountRepository().modifyMintedBlockCount(accountData.getAddress(), +1);
LOGGER.trace(() -> String.format("Block minter %s up to %d minted block%s (rowCount: %d)", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : ""), rowCount));
LOGGER.trace(() -> String.format("Block minter %s up to %d minted block%s", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : "")));
final int effectiveBlocksMinted = accountData.getBlocksMinted() + accountData.getBlocksMintedAdjustment();
@@ -1477,6 +1521,9 @@ public class Block {
public void orphan() throws DataException {
LOGGER.trace(() -> String.format("Orphaning block %d", this.blockData.getHeight()));
// Log some debugging info relating to the block weight calculation
this.logDebugInfo();
// Return AT fees and delete AT states from repository
orphanAtFeesAndStates();
@@ -1490,6 +1537,10 @@ public class Block {
// Invalidate expandedAccounts as they may have changed due to orphaning TRANSFER_PRIVS transactions, etc.
this.cachedExpandedAccounts = null;
if (this.blockData.getHeight() == 212937)
// Revert fix for block 212937
Block212937.orphanFix(this);
// Block rewards, including transaction fees, removed after transactions undone
orphanBlockRewards();
@@ -1612,12 +1663,14 @@ public class Block {
}
// Decrease blocks minted count for all accounts
// Batch update in repository
repository.getAccountRepository().modifyMintedBlockCounts(allUniqueExpandedAccounts.stream().map(AccountData::getAddress).collect(Collectors.toList()), -1);
for (AccountData accountData : allUniqueExpandedAccounts) {
// Adjust count locally (in Java)
accountData.setBlocksMinted(accountData.getBlocksMinted() - 1);
int rowCount = repository.getAccountRepository().modifyMintedBlockCount(accountData.getAddress(), -1);
LOGGER.trace(() -> String.format("Block minter %s down to %d minted block%s (rowCount: %d)", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : ""), rowCount));
LOGGER.trace(() -> String.format("Block minter %s down to %d minted block%s", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : "")));
final int effectiveBlocksMinted = accountData.getBlocksMinted() + accountData.getBlocksMintedAdjustment();
@@ -1646,7 +1699,7 @@ public class Block {
this.distributionMethod = distributionMethod;
}
public long distribute(long distibutionAmount, Map<Account, Long> balanceChanges) throws DataException {
public long distribute(long distibutionAmount, Map<String, Long> balanceChanges) throws DataException {
return this.distributionMethod.distribute(distibutionAmount, balanceChanges);
}
}
@@ -1663,7 +1716,7 @@ public class Block {
// Now distribute to candidates
// Collate all balance changes and then apply in one final step
Map<Account, Long> balanceChanges = new HashMap<>();
Map<String, Long> balanceChanges = new HashMap<>();
long remainingAmount = totalAmount;
for (int r = 0; r < rewardCandidates.size(); ++r) {
@@ -1688,8 +1741,10 @@ public class Block {
}
// Apply balance changes
for (Map.Entry<Account, Long> balanceChange : balanceChanges.entrySet())
balanceChange.getKey().modifyAssetBalance(Asset.QORT, balanceChange.getValue());
List<AccountBalanceData> accountBalanceDeltas = balanceChanges.entrySet().stream()
.map(entry -> new AccountBalanceData(entry.getKey(), Asset.QORT, entry.getValue()))
.collect(Collectors.toList());
this.repository.getAccountRepository().modifyAssetBalances(accountBalanceDeltas);
}
protected List<BlockRewardCandidate> determineBlockRewardCandidates(boolean isProcessingNotOrphaning) throws DataException {
@@ -1743,7 +1798,7 @@ public class Block {
// Find all accounts in share bin. getShareBin() returns null for minter accounts that are also founders, so they are effectively filtered out.
AccountLevelShareBin accountLevelShareBin = accountLevelShareBins.get(binIndex);
// Object reference compare is OK as all references are read-only from blockchain config.
List<ExpandedAccount> binnedAccounts = expandedAccounts.stream().filter(accountInfo -> accountInfo.getShareBin() == accountLevelShareBin).collect(Collectors.toList());
List<ExpandedAccount> binnedAccounts = expandedAccounts.stream().filter(accountInfo -> accountInfo.getShareBin(this.blockData.getHeight()) == accountLevelShareBin).collect(Collectors.toList());
// No online accounts in this bin? Skip to next one
if (binnedAccounts.isEmpty())
@@ -1759,7 +1814,7 @@ public class Block {
}
// Fetch list of legacy QORA holders who haven't reached their cap of QORT reward.
List<AccountBalanceData> qoraHolders = this.repository.getAccountRepository().getEligibleLegacyQoraHolders(isProcessingNotOrphaning ? null : this.blockData.getHeight());
List<EligibleQoraHolderData> qoraHolders = this.repository.getAccountRepository().getEligibleLegacyQoraHolders(isProcessingNotOrphaning ? null : this.blockData.getHeight());
final boolean haveQoraHolders = !qoraHolders.isEmpty();
final long qoraHoldersShare = BlockChain.getInstance().getQoraHoldersShare();
@@ -1812,7 +1867,7 @@ public class Block {
return rewardCandidates;
}
private static long distributeBlockRewardShare(long distributionAmount, List<ExpandedAccount> accounts, Map<Account, Long> balanceChanges) {
private static long distributeBlockRewardShare(long distributionAmount, List<ExpandedAccount> accounts, Map<String, Long> balanceChanges) {
// Collate all expanded accounts by minting account
Map<String, List<ExpandedAccount>> accountsByMinter = new HashMap<>();
@@ -1841,7 +1896,7 @@ public class Block {
return sharedAmount;
}
private static long distributeBlockRewardToQoraHolders(long qoraHoldersAmount, List<AccountBalanceData> qoraHolders, Map<Account, Long> balanceChanges, Block block) throws DataException {
private static long distributeBlockRewardToQoraHolders(long qoraHoldersAmount, List<EligibleQoraHolderData> qoraHolders, Map<String, Long> balanceChanges, Block block) throws DataException {
final boolean isProcessingNotOrphaning = qoraHoldersAmount >= 0;
long qoraPerQortReward = BlockChain.getInstance().getQoraPerQortReward();
@@ -1849,7 +1904,7 @@ public class Block {
long totalQoraHeld = 0;
for (int i = 0; i < qoraHolders.size(); ++i)
totalQoraHeld += qoraHolders.get(i).getBalance();
totalQoraHeld += qoraHolders.get(i).getQoraBalance();
long finalTotalQoraHeld = totalQoraHeld;
LOGGER.trace(() -> String.format("Total legacy QORA held: %s", Amounts.prettyAmount(finalTotalQoraHeld)));
@@ -1862,9 +1917,13 @@ public class Block {
BigInteger totalQoraHeldBI = BigInteger.valueOf(totalQoraHeld);
long sharedAmount = 0;
// For batched update of QORT_FROM_QORA balances
List<AccountBalanceData> newQortFromQoraBalances = new ArrayList<>();
for (int h = 0; h < qoraHolders.size(); ++h) {
AccountBalanceData qoraHolder = qoraHolders.get(h);
BigInteger qoraHolderBalanceBI = BigInteger.valueOf(qoraHolder.getBalance());
EligibleQoraHolderData qoraHolder = qoraHolders.get(h);
BigInteger qoraHolderBalanceBI = BigInteger.valueOf(qoraHolder.getQoraBalance());
String qoraHolderAddress = qoraHolder.getAddress();
// This is where a 128bit integer library could help:
// long holderReward = (qoraHoldersAmount * qoraHolder.getBalance()) / totalQoraHeld;
@@ -1872,15 +1931,13 @@ public class Block {
final long holderRewardForLogging = holderReward;
LOGGER.trace(() -> String.format("QORA holder %s has %s / %s QORA so share: %s",
qoraHolder.getAddress(), Amounts.prettyAmount(qoraHolder.getBalance()), finalTotalQoraHeld, Amounts.prettyAmount(holderRewardForLogging)));
qoraHolderAddress, Amounts.prettyAmount(qoraHolder.getQoraBalance()), finalTotalQoraHeld, Amounts.prettyAmount(holderRewardForLogging)));
// Too small to register this time?
if (holderReward == 0)
continue;
Account qoraHolderAccount = new Account(block.repository, qoraHolder.getAddress());
long newQortFromQoraBalance = qoraHolderAccount.getConfirmedBalance(Asset.QORT_FROM_QORA) + holderReward;
long newQortFromQoraBalance = qoraHolder.getQortFromQoraBalance() + holderReward;
// If processing, make sure we don't overpay
if (isProcessingNotOrphaning) {
@@ -1894,44 +1951,43 @@ public class Block {
newQortFromQoraBalance -= adjustment;
// This is also the QORA holder's final QORT-from-QORA block
QortFromQoraData qortFromQoraData = new QortFromQoraData(qoraHolder.getAddress(), holderReward, block.blockData.getHeight());
QortFromQoraData qortFromQoraData = new QortFromQoraData(qoraHolderAddress, holderReward, block.blockData.getHeight());
block.repository.getAccountRepository().save(qortFromQoraData);
long finalAdjustedHolderReward = holderReward;
LOGGER.trace(() -> String.format("QORA holder %s final share %s at height %d",
qoraHolder.getAddress(), Amounts.prettyAmount(finalAdjustedHolderReward), block.blockData.getHeight()));
qoraHolderAddress, Amounts.prettyAmount(finalAdjustedHolderReward), block.blockData.getHeight()));
}
} else {
// Orphaning
QortFromQoraData qortFromQoraData = block.repository.getAccountRepository().getQortFromQoraInfo(qoraHolder.getAddress());
if (qortFromQoraData != null) {
if (qoraHolder.getFinalBlockHeight() != null) {
// Final QORT-from-QORA amount from repository was stored during processing, and hence positive.
// So we use + here as qortFromQora is negative during orphaning.
// More efficient than "holderReward - (0 - final-qort-from-qora)"
long adjustment = holderReward + qortFromQoraData.getFinalQortFromQora();
long adjustment = holderReward + qoraHolder.getFinalQortFromQora();
holderReward -= adjustment;
newQortFromQoraBalance -= adjustment;
block.repository.getAccountRepository().deleteQortFromQoraInfo(qoraHolder.getAddress());
block.repository.getAccountRepository().deleteQortFromQoraInfo(qoraHolderAddress);
long finalAdjustedHolderReward = holderReward;
LOGGER.trace(() -> String.format("QORA holder %s final share %s was at height %d",
qoraHolder.getAddress(), Amounts.prettyAmount(finalAdjustedHolderReward), block.blockData.getHeight()));
qoraHolderAddress, Amounts.prettyAmount(finalAdjustedHolderReward), block.blockData.getHeight()));
}
}
balanceChanges.merge(qoraHolderAccount, holderReward, Long::sum);
balanceChanges.merge(qoraHolderAddress, holderReward, Long::sum);
if (newQortFromQoraBalance > 0)
qoraHolderAccount.setConfirmedBalance(Asset.QORT_FROM_QORA, newQortFromQoraBalance);
else
// Remove QORT_FROM_QORA balance as it's zero
qoraHolderAccount.deleteBalance(Asset.QORT_FROM_QORA);
// Add to batched QORT_FROM_QORA balance update list
newQortFromQoraBalances.add(new AccountBalanceData(qoraHolderAddress, Asset.QORT_FROM_QORA, newQortFromQoraBalance));
sharedAmount += holderReward;
}
// Perform batched update of QORT_FROM_QORA balances
block.repository.getAccountRepository().setAssetBalances(newQortFromQoraBalances);
return sharedAmount;
}
@@ -1940,4 +1996,38 @@ public class Block {
this.repository.getAccountRepository().tidy();
}
private void logDebugInfo() {
try {
// Avoid calculations if possible. We have to check against INFO here, since Level.isMoreSpecificThan() confusingly uses <= rather than just <
if (LOGGER.getLevel().isMoreSpecificThan(Level.INFO))
return;
if (this.repository == null || this.getMinter() == null || this.getBlockData() == null)
return;
int minterLevel = Account.getRewardShareEffectiveMintingLevel(this.repository, this.getMinter().getPublicKey());
LOGGER.debug(String.format("======= BLOCK %d (%.8s) =======", this.getBlockData().getHeight(), Base58.encode(this.getSignature())));
LOGGER.debug(String.format("Timestamp: %d", this.getBlockData().getTimestamp()));
LOGGER.debug(String.format("Minter level: %d", minterLevel));
LOGGER.debug(String.format("Online accounts: %d", this.getBlockData().getOnlineAccountsCount()));
LOGGER.debug(String.format("AT count: %d", this.getBlockData().getATCount()));
BlockSummaryData blockSummaryData = new BlockSummaryData(this.getBlockData());
if (this.getParent() == null || this.getParent().getSignature() == null || blockSummaryData == null || minterLevel == 0)
return;
blockSummaryData.setMinterLevel(minterLevel);
BigInteger blockWeight = calcBlockWeight(this.getParent().getHeight(), this.getParent().getSignature(), blockSummaryData);
BigInteger keyDistance = calcKeyDistance(this.getParent().getHeight(), this.getParent().getSignature(), blockSummaryData.getMinterPublicKey(), blockSummaryData.getMinterLevel());
NumberFormat formatter = new DecimalFormat("0.###E0");
LOGGER.debug(String.format("Key distance: %s", formatter.format(keyDistance)));
LOGGER.debug(String.format("Weight: %s", formatter.format(blockWeight)));
} catch (DataException e) {
LOGGER.info(() -> String.format("Unable to log block debugging info: %s", e.getMessage()));
}
}
}

View File

@@ -0,0 +1,153 @@
package org.qortal.block;
import java.io.InputStream;
import java.util.List;
import java.util.stream.Collectors;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.UnmarshalException;
import javax.xml.bind.Unmarshaller;
import javax.xml.transform.stream.StreamSource;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.eclipse.persistence.jaxb.JAXBContextFactory;
import org.eclipse.persistence.jaxb.UnmarshallerProperties;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.repository.DataException;
/**
* Block 212937
* <p>
* Somehow a node minted a version of block 212937 that contained one transaction:
* a PAYMENT transaction that attempted to spend more QORT than that account had as QORT balance.
* <p>
* This invalid transaction made block 212937 (rightly) invalid to several nodes,
* which refused to use that block.
* However, it seems there were no other nodes minting an alternative, valid block at that time
* and so the chain stalled for several nodes in the network.
* <p>
* Additionally, the invalid block 212937 affected all new installations, regardless of whether
* they synchronized from scratch (block 1) or used an 'official release' bootstrap.
* <p>
* After lengthy diagnosis, it was discovered that
* the invalid transaction seemed to rely on incorrect balances in a corrupted database.
* Copies of DB files containing the broken chain were also shared around, exacerbating the problem.
* <p>
* There were three options:
* <ol>
* <li>roll back the chain to last known valid block 212936 and re-mint empty blocks to current height</li>
* <li>keep existing chain, but apply database edits at block 212937 to allow current chain to be valid</li>
* <li>attempt to mint an alternative chain, retaining as many valid transactions as possible</li>
* </ol>
* <p>
* Option 1 was highly undesirable due to knock-on effects from wiping 700+ transactions, some of which
* might have affect cross-chain trades, although there were no cross-chain trade completed during
* the decision period.
* <p>
* Option 3 was essentially a slightly better version of option 1 and rejected for similar reasons.
* Attempts at option 3 also rapidly hit cumulative problems with every replacement block due to
* differing block timestamps making some transactions, and then even some blocks themselves, invalid.
* <p>
* This class is the implementation of option 2.
* <p>
* The change in account balances are relatively small, see <tt>block-212937-deltas.json</tt> resource
* for actual values. These values were obtained by exporting the <tt>AccountBalances</tt> table from
* both versions of the database with chain at block 212936, and then comparing. The values were also
* tested by syncing both databases up to block 225500, re-exporting and re-comparing.
* <p>
* The invalid block 212937 signature is: <tt>2J3GVJjv...qavh6KkQ</tt>.
* <p>
* The invalid transaction in block 212937 is:
* <p>
* <code><pre>
{
"amount" : "0.10788294",
"approvalStatus" : "NOT_REQUIRED",
"blockHeight" : 212937,
"creatorAddress" : "QLdw5uabviLJgRGkRiydAFmAtZzxHfNXSs",
"fee" : "0.00100000",
"recipient" : "QZi1mNHDbiLvsytxTgxDr9nhJe4pNZaWpw",
"reference" : "J6JukdTVuXZ3JYbHatfZzwxG2vSiZwVCPDzW5K7PsVQKRj8XZeDtqnkGCGGjaSQZ9bQMtV44ky88NnGM4YBQKU6",
"senderPublicKey" : "DBFfbD2M3uh4jPE5PaUcZVvNPfrrJzVB7seeEtBn5SPs",
"signature" : "qkitxdCEEnKt8w6wRfFixtErbXsxWE6zG2ESNhpqBdScikV1WxeA6WZTTMJVV4tCeZdBFXw3V1X5NVztv6LirWK",
"timestamp" : 1607863074904,
"txGroupId" : 0,
"type" : "PAYMENT"
}
</pre></code>
* <p>
* Account <tt>QLdw5uabviLJgRGkRiydAFmAtZzxHfNXSs</tt> attempted to spend <tt>0.10888294</tt> (including fees)
* when their QORT balance was really only <tt>0.10886665</tt>.
* <p>
* However, on the broken DB nodes, their balance
* seemed to be <tt>0.10890293</tt> which was sufficient to make the transaction valid.
*/
public final class Block212937 {
private static final Logger LOGGER = LogManager.getLogger(Block212937.class);
private static final String ACCOUNT_DELTAS_SOURCE = "block-212937-deltas.json";
private static final List<AccountBalanceData> accountDeltas = readAccountDeltas();
private Block212937() {
/* Do not instantiate */
}
@SuppressWarnings("unchecked")
private static List<AccountBalanceData> readAccountDeltas() {
Unmarshaller unmarshaller;
try {
// Create JAXB context aware of classes we need to unmarshal
JAXBContext jc = JAXBContextFactory.createContext(new Class[] {
AccountBalanceData.class
}, null);
// Create unmarshaller
unmarshaller = jc.createUnmarshaller();
// Set the unmarshaller media type to JSON
unmarshaller.setProperty(UnmarshallerProperties.MEDIA_TYPE, "application/json");
// Tell unmarshaller that there's no JSON root element in the JSON input
unmarshaller.setProperty(UnmarshallerProperties.JSON_INCLUDE_ROOT, false);
} catch (JAXBException e) {
String message = "Failed to setup unmarshaller to read block 212937 deltas";
LOGGER.error(message, e);
throw new RuntimeException(message, e);
}
ClassLoader classLoader = BlockChain.class.getClassLoader();
InputStream in = classLoader.getResourceAsStream(ACCOUNT_DELTAS_SOURCE);
StreamSource jsonSource = new StreamSource(in);
try {
// Attempt to unmarshal JSON stream to BlockChain config
return (List<AccountBalanceData>) unmarshaller.unmarshal(jsonSource, AccountBalanceData.class).getValue();
} catch (UnmarshalException e) {
String message = "Failed to parse block 212937 deltas";
LOGGER.error(message, e);
throw new RuntimeException(message, e);
} catch (JAXBException e) {
String message = "Unexpected JAXB issue while processing block 212937 deltas";
LOGGER.error(message, e);
throw new RuntimeException(message, e);
}
}
public static void processFix(Block block) throws DataException {
block.repository.getAccountRepository().modifyAssetBalances(accountDeltas);
}
public static void orphanFix(Block block) throws DataException {
// Create inverse deltas
List<AccountBalanceData> inverseDeltas = accountDeltas.stream()
.map(delta -> new AccountBalanceData(delta.getAddress(), delta.getAssetId(), 0 - delta.getBalance()))
.collect(Collectors.toList());
block.repository.getAccountRepository().modifyAssetBalances(inverseDeltas);
}
}

View File

@@ -32,7 +32,6 @@ import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.NTP;
import org.qortal.utils.StringLongMapXmlAdapter;
/**
@@ -71,6 +70,10 @@ public class BlockChain {
private GenesisBlock.GenesisInfo genesisInfo;
public enum FeatureTrigger {
atFindNextTransactionFix,
newBlockSigHeight,
shareBinFix,
calcChainWeightTimestamp;
}
/** Map of which blockchain features are enabled when (height/timestamp) */
@@ -372,6 +375,22 @@ public class BlockChain {
// Convenience methods for specific blockchain feature triggers
public int getAtFindNextTransactionFixHeight() {
return this.featureTriggers.get(FeatureTrigger.atFindNextTransactionFix.name()).intValue();
}
public int getNewBlockSigHeight() {
return this.featureTriggers.get(FeatureTrigger.newBlockSigHeight.name()).intValue();
}
public int getShareBinFixHeight() {
return this.featureTriggers.get(FeatureTrigger.shareBinFix.name()).intValue();
}
public long getCalcChainWeightTimestamp() {
return this.featureTriggers.get(FeatureTrigger.calcChainWeightTimestamp.name()).longValue();
}
// More complex getters for aspects that change by height or timestamp
public long getRewardAtHeight(int ourHeight) {
@@ -492,6 +511,8 @@ public class BlockChain {
rebuildBlockchain();
try (final Repository repository = RepositoryManager.getRepository()) {
repository.checkConsistency();
int startHeight = Math.max(repository.getBlockRepository().getBlockchainHeight() - 1440, 1);
BlockData detachedBlockData = repository.getBlockRepository().getDetachedBlockSignature(startHeight);
@@ -532,7 +553,8 @@ public class BlockChain {
private static void rebuildBlockchain() throws DataException {
// (Re)build repository
RepositoryManager.rebuild();
if (!RepositoryManager.wasPristineAtOpen())
RepositoryManager.rebuild();
try (final Repository repository = RepositoryManager.getRepository()) {
GenesisBlock genesisBlock = GenesisBlock.getInstance(repository);
@@ -554,17 +576,23 @@ public class BlockChain {
try {
try (final Repository repository = RepositoryManager.getRepository()) {
for (int height = repository.getBlockRepository().getBlockchainHeight(); height > targetHeight; --height) {
int height = repository.getBlockRepository().getBlockchainHeight();
BlockData orphanBlockData = repository.getBlockRepository().fromHeight(height);
while (height > targetHeight) {
LOGGER.info(String.format("Forcably orphaning block %d", height));
BlockData blockData = repository.getBlockRepository().fromHeight(height);
Block block = new Block(repository, blockData);
Block block = new Block(repository, orphanBlockData);
block.orphan();
repository.saveChanges();
}
BlockData lastBlockData = repository.getBlockRepository().getLastBlock();
Controller.getInstance().setChainTip(lastBlockData);
repository.saveChanges();
--height;
orphanBlockData = repository.getBlockRepository().fromHeight(height);
repository.discardChanges(); // clear transaction status to prevent deadlocks
Controller.getInstance().onOrphanedBlock(orphanBlockData);
}
return true;
}
@@ -573,33 +601,4 @@ public class BlockChain {
}
}
public static void trimOldOnlineAccountsSignatures() {
final Long now = NTP.getTime();
if (now == null)
return;
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
if (!blockchainLock.tryLock())
// Too busy to trim right now, try again later
return;
try {
try (final Repository repository = RepositoryManager.tryRepository()) {
if (repository == null)
return;
int numBlocksTrimmed = repository.getBlockRepository().trimOldOnlineAccountsSignatures(now - BlockChain.getInstance().getOnlineAccountSignaturesMaxLifetime());
if (numBlocksTrimmed > 0)
LOGGER.debug(String.format("Trimmed old online accounts signatures from %d block%s", numBlocksTrimmed, (numBlocksTrimmed != 1 ? "s" : "")));
repository.saveChanges();
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to trim old online accounts signatures: %s", e.getMessage()));
}
} finally {
blockchainLock.unlock();
}
}
}

View File

@@ -0,0 +1,80 @@
package org.qortal.controller;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.NTP;
public class AtStatesTrimmer implements Runnable {
private static final Logger LOGGER = LogManager.getLogger(AtStatesTrimmer.class);
@Override
public void run() {
Thread.currentThread().setName("AT States trimmer");
try (final Repository repository = RepositoryManager.getRepository()) {
int trimStartHeight = repository.getATRepository().getAtTrimHeight();
repository.getATRepository().prepareForAtStateTrimming();
repository.saveChanges();
while (!Controller.isStopping()) {
repository.discardChanges();
Thread.sleep(Settings.getInstance().getAtStatesTrimInterval());
BlockData chainTip = Controller.getInstance().getChainTip();
if (chainTip == null || NTP.getTime() == null)
continue;
// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
if (Controller.getInstance().isSynchronizing())
continue;
long currentTrimmableTimestamp = NTP.getTime() - Settings.getInstance().getAtStatesMaxLifetime();
// We want to keep AT states near the tip of our copy of blockchain so we can process/orphan nearby blocks
long chainTrimmableTimestamp = chainTip.getTimestamp() - Settings.getInstance().getAtStatesMaxLifetime();
long upperTrimmableTimestamp = Math.min(currentTrimmableTimestamp, chainTrimmableTimestamp);
int upperTrimmableHeight = repository.getBlockRepository().getHeightFromTimestamp(upperTrimmableTimestamp);
int upperBatchHeight = trimStartHeight + Settings.getInstance().getAtStatesTrimBatchSize();
int upperTrimHeight = Math.min(upperBatchHeight, upperTrimmableHeight);
if (trimStartHeight >= upperTrimHeight)
continue;
int numAtStatesTrimmed = repository.getATRepository().trimAtStates(trimStartHeight, upperTrimHeight, Settings.getInstance().getAtStatesTrimLimit());
repository.saveChanges();
if (numAtStatesTrimmed > 0) {
final int finalTrimStartHeight = trimStartHeight;
LOGGER.debug(() -> String.format("Trimmed %d AT state%s between blocks %d and %d",
numAtStatesTrimmed, (numAtStatesTrimmed != 1 ? "s" : ""),
finalTrimStartHeight, upperTrimHeight));
} else {
// Can we move onto next batch?
if (upperTrimmableHeight > upperBatchHeight) {
trimStartHeight = upperBatchHeight;
repository.getATRepository().setAtTrimHeight(trimStartHeight);
repository.getATRepository().prepareForAtStateTrimming();
repository.saveChanges();
final int finalTrimStartHeight = trimStartHeight;
LOGGER.debug(() -> String.format("Bumping AT state base trim height to %d", finalTrimStartHeight));
}
}
}
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to trim AT states: %s", e.getMessage()));
} catch (InterruptedException e) {
// Time to exit
}
}
}

View File

@@ -1,4 +1,4 @@
package org.qortal.block;
package org.qortal.controller;
import java.math.BigInteger;
import java.util.ArrayList;
@@ -13,8 +13,9 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.block.Block;
import org.qortal.block.Block.ValidationResult;
import org.qortal.controller.Controller;
import org.qortal.block.BlockChain;
import org.qortal.data.account.MintingAccountData;
import org.qortal.data.account.RewardShareData;
import org.qortal.data.block.BlockData;
@@ -60,7 +61,7 @@ public class BlockMinter extends Thread {
List<TransactionData> unconfirmedTransactions = repository.getTransactionRepository().getUnconfirmedTransactions();
for (TransactionData transactionData : unconfirmedTransactions) {
LOGGER.trace(String.format("Deleting unconfirmed transaction %s", Base58.encode(transactionData.getSignature())));
LOGGER.trace(() -> String.format("Deleting unconfirmed transaction %s", Base58.encode(transactionData.getSignature())));
repository.getTransactionRepository().delete(transactionData);
}
@@ -69,7 +70,7 @@ public class BlockMinter extends Thread {
// Going to need this a lot...
BlockRepository blockRepository = repository.getBlockRepository();
Block previousBlock = null;
BlockData previousBlockData = null;
List<Block> newBlocks = new ArrayList<>();
@@ -115,7 +116,7 @@ public class BlockMinter extends Thread {
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(mintingAccountData.getPublicKey());
if (rewardShareData == null) {
// Reward-share doesn't even exist - probably not a good sign
// Reward-share doesn't exist - probably cancelled but not yet removed from node's list of minting accounts
madi.remove();
continue;
}
@@ -134,24 +135,27 @@ public class BlockMinter extends Thread {
// Disregard peers that have "misbehaved" recently
peers.removeIf(Controller.hasMisbehaved);
// Disregard peers that don't have a recent block
peers.removeIf(Controller.hasNoRecentBlock);
// Disregard peers that don't have a recent block, but only if we're not in recovery mode.
// In that mode, we want to allow minting on top of older blocks, to recover stalled networks.
if (Controller.getInstance().getRecoveryMode() == false)
peers.removeIf(Controller.hasNoRecentBlock);
// Don't mint if we don't have enough up-to-date peers as where would the transactions/consensus come from?
if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
continue;
// If our latest block isn't recent then we need to synchronize instead of minting.
// If our latest block isn't recent then we need to synchronize instead of minting, unless we're in recovery mode.
if (!peers.isEmpty() && lastBlockData.getTimestamp() < minLatestBlockTimestamp)
continue;
if (Controller.getInstance().getRecoveryMode() == false)
continue;
// There are enough peers with a recent block and our latest block is recent
// so go ahead and mint a block if possible.
isMintingPossible = true;
// Check blockchain hasn't changed
if (previousBlock == null || !Arrays.equals(previousBlock.getSignature(), lastBlockData.getSignature())) {
previousBlock = new Block(repository, lastBlockData);
if (previousBlockData == null || !Arrays.equals(previousBlockData.getSignature(), lastBlockData.getSignature())) {
previousBlockData = lastBlockData;
newBlocks.clear();
// Reduce log timeout
@@ -162,12 +166,20 @@ public class BlockMinter extends Thread {
mintingAccountsData.removeIf(mintingAccountData -> newBlocks.stream().anyMatch(newBlock -> Arrays.equals(newBlock.getBlockData().getMinterPublicKey(), mintingAccountData.getPublicKey())));
// Do we need to build any potential new blocks?
List<PrivateKeyAccount> mintingAccounts = mintingAccountsData.stream().map(accountData -> new PrivateKeyAccount(repository, accountData.getPrivateKey())).collect(Collectors.toList());
List<PrivateKeyAccount> newBlocksMintingAccounts = mintingAccountsData.stream().map(accountData -> new PrivateKeyAccount(repository, accountData.getPrivateKey())).collect(Collectors.toList());
for (PrivateKeyAccount mintingAccount : mintingAccounts) {
// We might need to sit the next block out, if one of our minting accounts signed the previous one
final byte[] previousBlockMinter = previousBlockData.getMinterPublicKey();
final boolean mintedLastBlock = mintingAccountsData.stream().anyMatch(mintingAccount -> Arrays.equals(mintingAccount.getPublicKey(), previousBlockMinter));
if (mintedLastBlock) {
LOGGER.trace(String.format("One of our keys signed the last block, so we won't sign the next one"));
continue;
}
for (PrivateKeyAccount mintingAccount : newBlocksMintingAccounts) {
// First block does the AT heavy-lifting
if (newBlocks.isEmpty()) {
Block newBlock = Block.mint(repository, previousBlock.getBlockData(), mintingAccount);
Block newBlock = Block.mint(repository, previousBlockData, mintingAccount);
if (newBlock == null) {
// For some reason we can't mint right now
moderatedLog(() -> LOGGER.error("Couldn't build a to-be-minted block"));
@@ -195,7 +207,7 @@ public class BlockMinter extends Thread {
// Make sure we're the only thread modifying the blockchain
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
if (!blockchainLock.tryLock(30, TimeUnit.SECONDS)) {
LOGGER.warn("Couldn't acquire blockchain lock even after waiting 30 seconds");
LOGGER.debug("Couldn't acquire blockchain lock even after waiting 30 seconds");
continue;
}
@@ -233,8 +245,8 @@ public class BlockMinter extends Thread {
continue;
// Pick best block
final int parentHeight = previousBlock.getBlockData().getHeight();
final byte[] parentBlockSignature = previousBlock.getSignature();
final int parentHeight = previousBlockData.getHeight();
final byte[] parentBlockSignature = previousBlockData.getSignature();
BigInteger bestWeight = null;
@@ -274,28 +286,33 @@ public class BlockMinter extends Thread {
try {
newBlock.process();
LOGGER.info(String.format("Minted new block: %d", newBlock.getBlockData().getHeight()));
repository.saveChanges();
LOGGER.info(String.format("Minted new block: %d", newBlock.getBlockData().getHeight()));
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(newBlock.getBlockData().getMinterPublicKey());
if (rewardShareData != null) {
LOGGER.info(String.format("Minted block %d, sig %.8s by %s on behalf of %s",
LOGGER.info(String.format("Minted block %d, sig %.8s, parent sig: %.8s by %s on behalf of %s",
newBlock.getBlockData().getHeight(),
Base58.encode(newBlock.getBlockData().getSignature()),
Base58.encode(newBlock.getParent().getSignature()),
rewardShareData.getMinter(),
rewardShareData.getRecipient()));
} else {
LOGGER.info(String.format("Minted block %d, sig %.8s by %s",
LOGGER.info(String.format("Minted block %d, sig %.8s, parent sig: %.8s by %s",
newBlock.getBlockData().getHeight(),
Base58.encode(newBlock.getBlockData().getSignature()),
Base58.encode(newBlock.getParent().getSignature()),
newBlock.getMinter().getAddress()));
}
repository.saveChanges();
// Notify controller
// Notify network after we're released blockchain lock
newBlockMinted = true;
// Notify Controller
repository.discardChanges(); // clear transaction status to prevent deadlocks
Controller.getInstance().onNewBlock(newBlock.getBlockData());
} catch (DataException e) {
// Unable to process block - report and discard
LOGGER.error("Unable to process newly minted block?", e);
@@ -305,8 +322,13 @@ public class BlockMinter extends Thread {
blockchainLock.unlock();
}
if (newBlockMinted)
Controller.getInstance().onNewBlock(newBlock.getBlockData());
if (newBlockMinted) {
// Broadcast our new chain to network
BlockData newBlockData = newBlock.getBlockData();
Network network = Network.getInstance();
network.broadcast(broadcastPeer -> network.buildHeightMessage(broadcastPeer, newBlockData));
}
}
} catch (DataException e) {
LOGGER.warn("Repository issue while running block minter", e);

View File

@@ -1,61 +0,0 @@
package org.qortal.controller;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import org.eclipse.jetty.websocket.api.Session;
import org.qortal.api.model.BlockInfo;
import org.qortal.data.block.BlockData;
public class BlockNotifier {
private static BlockNotifier instance;
@FunctionalInterface
public interface Listener {
void notify(BlockInfo blockInfo);
}
private Map<Session, Listener> listenersBySession = new HashMap<>();
private BlockNotifier() {
}
public static synchronized BlockNotifier getInstance() {
if (instance == null)
instance = new BlockNotifier();
return instance;
}
public void register(Session session, Listener listener) {
synchronized (this.listenersBySession) {
this.listenersBySession.put(session, listener);
}
}
public void deregister(Session session) {
synchronized (this.listenersBySession) {
this.listenersBySession.remove(session);
}
}
public void onNewBlock(BlockData blockData) {
// Convert BlockData to BlockInfo
BlockInfo blockInfo = new BlockInfo(blockData);
for (Listener listener : getAllListeners())
listener.notify(blockInfo);
}
private Collection<Listener> getAllListeners() {
// Make a copy of listeners to both avoid concurrent modification
// and reduce synchronization time
synchronized (this.listenersBySession) {
return new ArrayList<>(this.listenersBySession.values());
}
}
}

View File

@@ -16,16 +16,24 @@ import java.util.Collections;
import java.util.Deque;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
@@ -36,9 +44,9 @@ import org.qortal.account.PublicKeyAccount;
import org.qortal.api.ApiService;
import org.qortal.block.Block;
import org.qortal.block.BlockChain;
import org.qortal.block.BlockMinter;
import org.qortal.block.BlockChain.BlockTimingByHeight;
import org.qortal.controller.Synchronizer.SynchronizationResult;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.MintingAccountData;
import org.qortal.data.account.RewardShareData;
@@ -50,6 +58,8 @@ import org.qortal.data.network.PeerData;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.data.transaction.ArbitraryTransactionData.DataType;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.data.transaction.ChatTransactionData;
import org.qortal.globalization.Translator;
import org.qortal.gui.Gui;
@@ -57,8 +67,8 @@ import org.qortal.gui.SysTray;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.ArbitraryDataMessage;
import org.qortal.network.message.BlockMessage;
import org.qortal.network.message.BlockSummariesMessage;
import org.qortal.network.message.CachedBlockMessage;
import org.qortal.network.message.GetArbitraryDataMessage;
import org.qortal.network.message.GetBlockMessage;
import org.qortal.network.message.GetBlockSummariesMessage;
@@ -85,6 +95,7 @@ import org.qortal.transaction.Transaction.TransactionType;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.utils.Base58;
import org.qortal.utils.ByteArray;
import org.qortal.utils.DaemonThreadFactory;
import org.qortal.utils.NTP;
import org.qortal.utils.Triple;
@@ -110,6 +121,7 @@ public class Controller extends Thread {
private static final long NTP_PRE_SYNC_CHECK_PERIOD = 5 * 1000L; // ms
private static final long NTP_POST_SYNC_CHECK_PERIOD = 5 * 60 * 1000L; // ms
private static final long DELETE_EXPIRED_INTERVAL = 5 * 60 * 1000L; // ms
private static final long RECOVERY_MODE_TIMEOUT = 10 * 60 * 1000L; // ms
// To do with online accounts list
private static final long ONLINE_ACCOUNTS_TASKS_INTERVAL = 10 * 1000L; // ms
@@ -132,16 +144,30 @@ public class Controller extends Thread {
private ExecutorService callbackExecutor = Executors.newFixedThreadPool(3);
private volatile boolean notifyGroupMembershipChange = false;
private volatile BlockData chainTip = null;
/** Latest blocks on our chain. Note: tail/last is the latest block. */
private final Deque<BlockData> latestBlocks = new LinkedList<>();
/** Cache of BlockMessages, indexed by block signature */
@SuppressWarnings("serial")
private final LinkedHashMap<ByteArray, CachedBlockMessage> blockMessageCache = new LinkedHashMap<>() {
@Override
protected boolean removeEldestEntry(Map.Entry<ByteArray, CachedBlockMessage> eldest) {
return this.size() > Settings.getInstance().getBlockCacheSize();
}
};
private long repositoryBackupTimestamp = startTime; // ms
private long repositoryCheckpointTimestamp = startTime; // ms
private long ntpCheckTimestamp = startTime; // ms
private long deleteExpiredTimestamp = startTime + DELETE_EXPIRED_INTERVAL; // ms
private long onlineAccountsTasksTimestamp = startTime + ONLINE_ACCOUNTS_TASKS_INTERVAL; // ms
/** Whether we can mint new blocks, as reported by BlockMinter. */
private volatile boolean isMintingPossible = false;
/** Synchronization object for sync variables below */
private final Object syncLock = new Object();
/** Whether we are attempting to synchronize. */
private volatile boolean isSynchronizing = false;
/** Temporary estimate of synchronization progress for SysTray use. */
@@ -150,6 +176,11 @@ public class Controller extends Thread {
/** Latest block signatures from other peers that we know are on inferior chains. */
List<ByteArray> inferiorChainSignatures = new ArrayList<>();
/** Recovery mode, which is used to bring back a stalled network */
private boolean recoveryMode = false;
private boolean peersAvailable = true; // peersAvailable must default to true
private long timePeersLastAvailable = 0;
/**
* Map of recent requests for ARBITRARY transaction data payloads.
* <p>
@@ -177,6 +208,47 @@ public class Controller extends Thread {
/** Cache of latest blocks' online accounts */
Deque<List<OnlineAccountData>> latestBlocksOnlineAccounts = new ArrayDeque<>(MAX_BLOCKS_CACHED_ONLINE_ACCOUNTS);
// Stats
@XmlAccessorType(XmlAccessType.FIELD)
public static class StatsSnapshot {
public static class GetBlockMessageStats {
public AtomicLong requests = new AtomicLong();
public AtomicLong cacheHits = new AtomicLong();
public AtomicLong unknownBlocks = new AtomicLong();
public AtomicLong cacheFills = new AtomicLong();
public GetBlockMessageStats() {
}
}
public GetBlockMessageStats getBlockMessageStats = new GetBlockMessageStats();
public static class GetBlockSummariesStats {
public AtomicLong requests = new AtomicLong();
public AtomicLong cacheHits = new AtomicLong();
public AtomicLong fullyFromCache = new AtomicLong();
public GetBlockSummariesStats() {
}
}
public GetBlockSummariesStats getBlockSummariesStats = new GetBlockSummariesStats();
public static class GetBlockSignaturesV2Stats {
public AtomicLong requests = new AtomicLong();
public AtomicLong cacheHits = new AtomicLong();
public AtomicLong fullyFromCache = new AtomicLong();
public GetBlockSignaturesV2Stats() {
}
}
public GetBlockSignaturesV2Stats getBlockSignaturesV2Stats = new GetBlockSignaturesV2Stats();
public AtomicLong latestBlocksCacheRefills = new AtomicLong();
public StatsSnapshot() {
}
}
private final StatsSnapshot stats = new StatsSnapshot();
// Constructors
private Controller(String[] args) {
@@ -187,17 +259,29 @@ public class Controller extends Thread {
throw new RuntimeException("Can't read build.properties resource", e);
}
// Determine build timestamp
String buildTimestampProperty = properties.getProperty("build.timestamp");
if (buildTimestampProperty == null)
if (buildTimestampProperty == null) {
throw new RuntimeException("Can't read build.timestamp from build.properties resource");
this.buildTimestamp = LocalDateTime.parse(buildTimestampProperty, DateTimeFormatter.ofPattern("yyyyMMddHHmmss")).toEpochSecond(ZoneOffset.UTC);
}
if (buildTimestampProperty.startsWith("$")) {
// Maven vars haven't been replaced - this was most likely built using an IDE, not via mvn package
this.buildTimestamp = System.currentTimeMillis();
buildTimestampProperty = "unknown";
} else {
this.buildTimestamp = LocalDateTime.parse(buildTimestampProperty, DateTimeFormatter.ofPattern("yyyyMMddHHmmss")).toEpochSecond(ZoneOffset.UTC);
}
LOGGER.info(String.format("Build timestamp: %s", buildTimestampProperty));
// Determine build version
String buildVersionProperty = properties.getProperty("build.version");
if (buildVersionProperty == null)
if (buildVersionProperty == null) {
throw new RuntimeException("Can't read build.version from build.properties resource");
}
if (buildVersionProperty.contains("${git.commit.id.abbrev}")) {
// Maven vars haven't been replaced - this was most likely built using an IDE, not via mvn package
buildVersionProperty = buildVersionProperty.replace("${git.commit.id.abbrev}", "debug");
}
this.buildVersion = VERSION_PREFIX + buildVersionProperty;
LOGGER.info(String.format("Build version: %s", this.buildVersion));
@@ -232,21 +316,37 @@ public class Controller extends Thread {
/** Returns current blockchain height, or 0 if it's not available. */
public int getChainHeight() {
BlockData blockData = this.chainTip;
if (blockData == null)
return 0;
synchronized (this.latestBlocks) {
BlockData blockData = this.latestBlocks.peekLast();
if (blockData == null)
return 0;
return blockData.getHeight();
return blockData.getHeight();
}
}
/** Returns highest block, or null if it's not available. */
public BlockData getChainTip() {
return this.chainTip;
synchronized (this.latestBlocks) {
return this.latestBlocks.peekLast();
}
}
/** Cache new blockchain tip. */
public void setChainTip(BlockData blockData) {
this.chainTip = blockData;
public void refillLatestBlocksCache() throws DataException {
// Set initial chain height/tip
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData blockData = repository.getBlockRepository().getLastBlock();
int blockCacheSize = Settings.getInstance().getBlockCacheSize();
synchronized (this.latestBlocks) {
this.latestBlocks.clear();
for (int i = 0; i < blockCacheSize && blockData != null; ++i) {
this.latestBlocks.addFirst(blockData);
blockData = repository.getBlockRepository().fromHeight(blockData.getHeight() - 1);
}
}
}
}
public ReentrantLock getBlockchainLock() {
@@ -271,7 +371,13 @@ public class Controller extends Thread {
}
public Integer getSyncPercent() {
return this.isSynchronizing ? this.syncPercent : null;
synchronized (this.syncLock) {
return this.isSynchronizing ? this.syncPercent : null;
}
}
public boolean getRecoveryMode() {
return this.recoveryMode;
}
// Entry point
@@ -326,13 +432,8 @@ public class Controller extends Thread {
try {
BlockChain.validate();
// Set initial chain height/tip
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData blockData = repository.getBlockRepository().getLastBlock();
Controller.getInstance().setChainTip(blockData);
LOGGER.info(String.format("Our chain height at start-up: %d", blockData.getHeight()));
}
Controller.getInstance().refillLatestBlocksCache();
LOGGER.info(String.format("Our chain height at start-up: %d", Controller.getInstance().getChainHeight()));
} catch (DataException e) {
LOGGER.error("Couldn't validate blockchain", e);
Gui.getInstance().fatalError("Blockchain validation issue", e);
@@ -366,6 +467,9 @@ public class Controller extends Thread {
blockMinter = new BlockMinter();
blockMinter.start();
LOGGER.info("Starting trade-bot");
TradeBot.getInstance();
// Arbitrary transaction data manager
// LOGGER.info("Starting arbitrary-transaction data manager");
// ArbitraryDataManager.getInstance().start();
@@ -404,6 +508,11 @@ public class Controller extends Thread {
Thread.currentThread().setName("Controller");
final long repositoryBackupInterval = Settings.getInstance().getRepositoryBackupInterval();
final long repositoryCheckpointInterval = Settings.getInstance().getRepositoryCheckpointInterval();
ExecutorService trimExecutor = Executors.newCachedThreadPool(new DaemonThreadFactory());
trimExecutor.execute(new AtStatesTrimmer());
trimExecutor.execute(new OnlineAccountsSignaturesTrimmer());
try {
while (!isStopping) {
@@ -445,6 +554,13 @@ public class Controller extends Thread {
final long requestMinimumTimestamp = now - ARBITRARY_REQUEST_TIMEOUT;
arbitraryDataRequests.entrySet().removeIf(entry -> entry.getValue().getC() < requestMinimumTimestamp);
// Time to 'checkpoint' uncommitted repository writes?
if (now >= repositoryCheckpointTimestamp + repositoryCheckpointInterval) {
repositoryCheckpointTimestamp = now + repositoryCheckpointInterval;
RepositoryManager.setRequestedCheckpoint(Boolean.TRUE);
}
// Give repository a chance to backup (if enabled)
if (repositoryBackupInterval > 0 && now >= repositoryBackupTimestamp + repositoryBackupInterval) {
repositoryBackupTimestamp = now + repositoryBackupInterval;
@@ -477,7 +593,17 @@ public class Controller extends Thread {
}
}
} catch (InterruptedException e) {
// Clear interrupted flag so we can shutdown trim threads
Thread.interrupted();
// Fall-through to exit
} finally {
trimExecutor.shutdownNow();
try {
trimExecutor.awaitTermination(2L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// We tried...
}
}
}
@@ -509,7 +635,16 @@ public class Controller extends Thread {
return peerChainTipData == null || peerChainTipData.getLastBlockSignature() == null || inferiorChainTips.contains(new ByteArray(peerChainTipData.getLastBlockSignature()));
};
public static final Predicate<Peer> hasOldVersion = peer -> {
final String minPeerVersion = Settings.getInstance().getMinPeerVersion();
return peer.isAtLeastVersion(minPeerVersion) == false;
};
private void potentiallySynchronize() throws InterruptedException {
// Already synchronizing via another thread?
if (this.isSynchronizing)
return;
List<Peer> peers = Network.getInstance().getHandshakedPeers();
// Disregard peers that have "misbehaved" recently
@@ -521,6 +656,17 @@ public class Controller extends Thread {
// Disregard peers that don't have a recent block
peers.removeIf(hasNoRecentBlock);
// Disregard peers that are on an old version
peers.removeIf(hasOldVersion);
checkRecoveryModeForPeers(peers);
if (recoveryMode) {
peers = Network.getInstance().getHandshakedPeers();
peers.removeIf(hasOnlyGenesisBlock);
peers.removeIf(hasMisbehaved);
peers.removeIf(hasOldVersion);
}
// Check we have enough peers to potentially synchronize
if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
return;
@@ -531,9 +677,31 @@ public class Controller extends Thread {
// Disregard peers that are on the same block as last sync attempt and we didn't like their chain
peers.removeIf(hasInferiorChainTip);
final int peersBeforeComparison = peers.size();
// Request recent block summaries from the remaining peers, and locate our common block with each
Synchronizer.getInstance().findCommonBlocksWithPeers(peers);
// Compare the peers against each other, and against our chain, which will return an updated list excluding those without common blocks
peers = Synchronizer.getInstance().comparePeers(peers);
// We may have added more inferior chain tips when comparing peers, so remove any peers that are currently on those chains
peers.removeIf(hasInferiorChainTip);
final int peersRemoved = peersBeforeComparison - peers.size();
if (peersRemoved > 0 && peers.size() > 0)
LOGGER.debug(String.format("Ignoring %d peers on inferior chains. Peers remaining: %d", peersRemoved, peers.size()));
if (peers.isEmpty())
return;
if (peers.size() > 1) {
StringBuilder finalPeersString = new StringBuilder();
for (Peer peer : peers)
finalPeersString = finalPeersString.length() > 0 ? finalPeersString.append(", ").append(peer) : finalPeersString.append(peer);
LOGGER.debug(String.format("Choosing random peer from: [%s]", finalPeersString.toString()));
}
// Pick random peer to sync with
int index = new SecureRandom().nextInt(peers.size());
Peer peer = peers.get(index);
@@ -542,14 +710,22 @@ public class Controller extends Thread {
}
public SynchronizationResult actuallySynchronize(Peer peer, boolean force) throws InterruptedException {
syncPercent = (this.chainTip.getHeight() * 100) / peer.getChainTipData().getLastHeight();
// Only update SysTray if we're potentially changing height
if (syncPercent < 100) {
isSynchronizing = true;
updateSysTray();
}
boolean hasStatusChanged = false;
BlockData priorChainTip = this.getChainTip();
BlockData priorChainTip = this.chainTip;
synchronized (this.syncLock) {
this.syncPercent = (priorChainTip.getHeight() * 100) / peer.getChainTipData().getLastHeight();
// Only update SysTray if we're potentially changing height
if (this.syncPercent < 100) {
this.isSynchronizing = true;
hasStatusChanged = true;
}
}
peer.setSyncInProgress(true);
if (hasStatusChanged)
updateSysTray();
try {
SynchronizationResult syncResult = Synchronizer.getInstance().synchronize(peer, force);
@@ -619,19 +795,66 @@ public class Controller extends Thread {
// Reset our cache of inferior chains
inferiorChainSignatures.clear();
// Update chain-tip, systray, notify peers, websockets, etc.
this.onNewBlock(newChainTip);
Network network = Network.getInstance();
network.broadcast(broadcastPeer -> network.buildHeightMessage(broadcastPeer, newChainTip));
}
return syncResult;
} finally {
isSynchronizing = false;
peer.setSyncInProgress(false);
}
}
private boolean checkRecoveryModeForPeers(List<Peer> qualifiedPeers) {
List<Peer> handshakedPeers = Network.getInstance().getHandshakedPeers();
if (handshakedPeers.size() > 0) {
// There is at least one handshaked peer
if (qualifiedPeers.isEmpty()) {
// There are no 'qualified' peers - i.e. peers that have a recent block we can sync to
boolean werePeersAvailable = peersAvailable;
peersAvailable = false;
// If peers only just became unavailable, update our record of the time they were last available
if (werePeersAvailable)
timePeersLastAvailable = NTP.getTime();
// If enough time has passed, enter recovery mode, which lifts some restrictions on who we can sync with and when we can mint
if (NTP.getTime() - timePeersLastAvailable > RECOVERY_MODE_TIMEOUT) {
if (recoveryMode == false) {
LOGGER.info(String.format("Peers have been unavailable for %d minutes. Entering recovery mode...", RECOVERY_MODE_TIMEOUT/60/1000));
recoveryMode = true;
}
}
} else {
// We now have at least one peer with a recent block, so we can exit recovery mode and sync normally
peersAvailable = true;
if (recoveryMode) {
LOGGER.info("Peers have become available again. Exiting recovery mode...");
recoveryMode = false;
}
}
}
return recoveryMode;
}
public void addInferiorChainSignature(byte[] inferiorSignature) {
// Update our list of inferior chain tips
ByteArray inferiorChainSignature = new ByteArray(inferiorSignature);
if (!inferiorChainSignatures.contains(inferiorChainSignature))
inferiorChainSignatures.add(inferiorChainSignature);
}
public static class StatusChangeEvent implements Event {
public StatusChangeEvent() {
}
}
private void updateSysTray() {
if (NTP.getTime() == null) {
SysTray.getInstance().setToolTipText(Translator.INSTANCE.translate("SysTray", "SYNCHRONIZING_CLOCK"));
SysTray.getInstance().setTrayIcon(1);
return;
}
@@ -643,20 +866,31 @@ public class Controller extends Thread {
String heightText = Translator.INSTANCE.translate("SysTray", "BLOCK_HEIGHT");
String actionText;
if (isMintingPossible)
actionText = Translator.INSTANCE.translate("SysTray", "MINTING_ENABLED");
else if (isSynchronizing)
actionText = String.format("%s - %d%%", Translator.INSTANCE.translate("SysTray", "SYNCHRONIZING_BLOCKCHAIN"), syncPercent);
else if (numberOfPeers < Settings.getInstance().getMinBlockchainPeers())
actionText = Translator.INSTANCE.translate("SysTray", "CONNECTING");
else
actionText = Translator.INSTANCE.translate("SysTray", "MINTING_DISABLED");
String tooltip = String.format("%s - %d %s - %s %d", actionText, numberOfPeers, connectionsText, heightText, height);
synchronized (this.syncLock) {
if (this.isMintingPossible) {
actionText = Translator.INSTANCE.translate("SysTray", "MINTING_ENABLED");
SysTray.getInstance().setTrayIcon(2);
}
else if (this.isSynchronizing) {
actionText = String.format("%s - %d%%", Translator.INSTANCE.translate("SysTray", "SYNCHRONIZING_BLOCKCHAIN"), this.syncPercent);
SysTray.getInstance().setTrayIcon(3);
}
else if (numberOfPeers < Settings.getInstance().getMinBlockchainPeers()) {
actionText = Translator.INSTANCE.translate("SysTray", "CONNECTING");
SysTray.getInstance().setTrayIcon(3);
}
else {
actionText = Translator.INSTANCE.translate("SysTray", "MINTING_DISABLED");
SysTray.getInstance().setTrayIcon(4);
}
}
String tooltip = String.format("%s - %d %s - %s %d", actionText, numberOfPeers, connectionsText, heightText, height) + "\n" + String.format("Build version: %s", this.buildVersion);
SysTray.getInstance().setToolTipText(tooltip);
this.callbackExecutor.execute(() -> {
StatusNotifier.getInstance().onStatusChange(NTP.getTime());
EventBus.INSTANCE.notify(new StatusChangeEvent());
});
}
@@ -672,15 +906,26 @@ public class Controller extends Thread {
List<TransactionData> transactions = repository.getTransactionRepository().getUnconfirmedTransactions();
for (TransactionData transactionData : transactions)
if (now >= Transaction.getDeadline(transactionData)) {
LOGGER.info(String.format("Deleting expired, unconfirmed transaction %s", Base58.encode(transactionData.getSignature())));
int deletedCount = 0;
for (TransactionData transactionData : transactions) {
Transaction transaction = Transaction.fromData(repository, transactionData);
if (now >= transaction.getDeadline()) {
LOGGER.debug(() -> String.format("Deleting expired, unconfirmed transaction %s", Base58.encode(transactionData.getSignature())));
repository.getTransactionRepository().delete(transactionData);
deletedCount++;
}
}
if (deletedCount > 0) {
LOGGER.info(String.format("Deleted %d expired, unconfirmed transaction%s", deletedCount, (deletedCount == 1 ? "" : "s")));
}
repository.saveChanges();
} catch (DataException e) {
LOGGER.error("Repository issue while deleting expired unconfirmed transactions", e);
if (RepositoryManager.isDeadlockRelated(e))
LOGGER.info("Couldn't delete some expired, unconfirmed transactions this round");
else
LOGGER.error("Repository issue while deleting expired unconfirmed transactions", e);
}
}
@@ -783,32 +1028,154 @@ public class Controller extends Thread {
requestSysTrayUpdate = true;
}
public void onNewBlock(BlockData latestBlockData) {
this.setChainTip(latestBlockData);
requestSysTrayUpdate = true;
public static class NewBlockEvent implements Event {
private final BlockData blockData;
// Broadcast our new height info and notify websocket listeners
this.callbackExecutor.execute(() -> {
Network network = Network.getInstance();
network.broadcast(peer -> network.buildHeightMessage(peer, latestBlockData));
public NewBlockEvent(BlockData blockData) {
this.blockData = blockData;
}
BlockNotifier.getInstance().onNewBlock(latestBlockData);
if (this.notifyGroupMembershipChange) {
this.notifyGroupMembershipChange = false;
ChatNotifier.getInstance().onGroupMembershipChange();
}
// Trade-bot might want to perform some actions too
TradeBot.getInstance().onChainTipChange();
});
public BlockData getBlockData() {
return this.blockData;
}
}
/** Callback for when we've received a new transaction via API or peer. */
public void onNewTransaction(TransactionData transactionData, Peer peer) {
/**
* Callback for when we've received a new block.
* <p>
* See <b>WARNING</b> for {@link EventBus#notify(Event)}
* to prevent deadlocks.
*/
public void onNewBlock(BlockData latestBlockData) {
// Protective copy
BlockData blockDataCopy = new BlockData(latestBlockData);
int blockCacheSize = Settings.getInstance().getBlockCacheSize();
synchronized (this.latestBlocks) {
BlockData cachedChainTip = this.latestBlocks.peekLast();
if (cachedChainTip != null && Arrays.equals(cachedChainTip.getSignature(), blockDataCopy.getReference())) {
// Chain tip is parent for new latest block, so we can safely add new latest block
this.latestBlocks.addLast(latestBlockData);
// Trim if necessary
if (this.latestBlocks.size() >= blockCacheSize)
this.latestBlocks.pollFirst();
} else {
if (cachedChainTip != null)
// Chain tip didn't match - potentially abnormal behaviour?
LOGGER.debug(() -> String.format("Cached chain tip %.8s not parent for new latest block %.8s (reference %.8s)",
Base58.encode(cachedChainTip.getSignature()),
Base58.encode(blockDataCopy.getSignature()),
Base58.encode(blockDataCopy.getReference())));
// Defensively rebuild cache
try {
this.stats.latestBlocksCacheRefills.incrementAndGet();
this.refillLatestBlocksCache();
} catch (DataException e) {
LOGGER.warn(() -> "Couldn't refill latest blocks cache?", e);
}
}
}
this.onNewOrOrphanedBlock(blockDataCopy, NewBlockEvent::new);
}
public static class OrphanedBlockEvent implements Event {
private final BlockData blockData;
public OrphanedBlockEvent(BlockData blockData) {
this.blockData = blockData;
}
public BlockData getBlockData() {
return this.blockData;
}
}
/**
* Callback for when we've orphaned a block.
* <p>
* See <b>WARNING</b> for {@link EventBus#notify(Event)}
* to prevent deadlocks.
*/
public void onOrphanedBlock(BlockData latestBlockData) {
// Protective copy
BlockData blockDataCopy = new BlockData(latestBlockData);
synchronized (this.latestBlocks) {
BlockData cachedChainTip = this.latestBlocks.pollLast();
boolean refillNeeded = false;
if (cachedChainTip != null && Arrays.equals(cachedChainTip.getReference(), blockDataCopy.getSignature())) {
// Chain tip was parent for new latest block that has been orphaned, so we're good
// However, if we've emptied the cache then we will need to refill it
refillNeeded = this.latestBlocks.isEmpty();
} else {
if (cachedChainTip != null)
// Chain tip didn't match - potentially abnormal behaviour?
LOGGER.debug(() -> String.format("Cached chain tip %.8s (reference %.8s) was not parent for new latest block %.8s",
Base58.encode(cachedChainTip.getSignature()),
Base58.encode(cachedChainTip.getReference()),
Base58.encode(blockDataCopy.getSignature())));
// Defensively rebuild cache
refillNeeded = true;
}
if (refillNeeded)
try {
this.stats.latestBlocksCacheRefills.incrementAndGet();
this.refillLatestBlocksCache();
} catch (DataException e) {
LOGGER.warn(() -> "Couldn't refill latest blocks cache?", e);
}
}
this.onNewOrOrphanedBlock(blockDataCopy, OrphanedBlockEvent::new);
}
private void onNewOrOrphanedBlock(BlockData blockDataCopy, Function<BlockData, Event> eventConstructor) {
requestSysTrayUpdate = true;
// Notify listeners, trade-bot, etc.
EventBus.INSTANCE.notify(eventConstructor.apply(blockDataCopy));
if (this.notifyGroupMembershipChange) {
this.notifyGroupMembershipChange = false;
ChatNotifier.getInstance().onGroupMembershipChange();
}
}
public static class NewTransactionEvent implements Event {
private final TransactionData transactionData;
public NewTransactionEvent(TransactionData transactionData) {
this.transactionData = transactionData;
}
public TransactionData getTransactionData() {
return this.transactionData;
}
}
/**
* Callback for when we've received a new transaction via API or peer.
* <p>
* @implSpec performs actions in a new thread
*/
public void onNewTransaction(TransactionData transactionData) {
this.callbackExecutor.execute(() -> {
// Notify all peers (except maybe peer that sent it to us if applicable)
Network.getInstance().broadcast(broadcastPeer -> broadcastPeer == peer ? null : new TransactionSignaturesMessage(Arrays.asList(transactionData.getSignature())));
// Notify all peers
Message newTransactionSignatureMessage = new TransactionSignaturesMessage(Arrays.asList(transactionData.getSignature()));
Network.getInstance().broadcast(broadcastPeer -> newTransactionSignatureMessage);
// Notify listeners
EventBus.INSTANCE.notify(new NewTransactionEvent(transactionData));
// If this is a CHAT transaction, there may be extra listeners to notify
if (transactionData.getType() == TransactionType.CHAT)
@@ -896,11 +1263,32 @@ public class Controller extends Thread {
private void onNetworkGetBlockMessage(Peer peer, Message message) {
GetBlockMessage getBlockMessage = (GetBlockMessage) message;
byte[] signature = getBlockMessage.getSignature();
this.stats.getBlockMessageStats.requests.incrementAndGet();
ByteArray signatureAsByteArray = new ByteArray(signature);
CachedBlockMessage cachedBlockMessage = this.blockMessageCache.get(signatureAsByteArray);
int blockCacheSize = Settings.getInstance().getBlockCacheSize();
// Check cached latest block message
if (cachedBlockMessage != null) {
this.stats.getBlockMessageStats.cacheHits.incrementAndGet();
// We need to duplicate it to prevent multiple threads setting ID on the same message
CachedBlockMessage clonedBlockMessage = cachedBlockMessage.cloneWithNewId(message.getId());
if (!peer.sendMessage(clonedBlockMessage))
peer.disconnect("failed to send block");
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
BlockData blockData = repository.getBlockRepository().fromSignature(signature);
if (blockData == null) {
// We don't have this block
this.stats.getBlockMessageStats.unknownBlocks.getAndIncrement();
// Send valid, yet unexpected message type in response, so peer's synchronizer doesn't have to wait for timeout
LOGGER.debug(() -> String.format("Sending 'block unknown' response to peer %s for GET_BLOCK request for unknown block %s", peer, Base58.encode(signature)));
@@ -915,10 +1303,22 @@ public class Controller extends Thread {
Block block = new Block(repository, blockData);
Message blockMessage = new BlockMessage(block);
CachedBlockMessage blockMessage = new CachedBlockMessage(block);
blockMessage.setId(message.getId());
if (!peer.sendMessage(blockMessage))
// This call also causes the other needed data to be pulled in from repository
if (!peer.sendMessage(blockMessage)) {
peer.disconnect("failed to send block");
// Don't fall-through to caching because failure to send might be from failure to build message
return;
}
// If request is for a recent block, cache it
if (getChainHeight() - blockData.getHeight() <= blockCacheSize) {
this.stats.getBlockMessageStats.cacheFills.incrementAndGet();
this.blockMessageCache.put(new ByteArray(blockData.getSignature()), blockMessage);
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while send block %s to peer %s", Base58.encode(signature), peer), e);
}
@@ -928,6 +1328,18 @@ public class Controller extends Thread {
TransactionMessage transactionMessage = (TransactionMessage) message;
TransactionData transactionData = transactionMessage.getTransactionData();
/*
* If we can't obtain blockchain lock immediately,
* e.g. Synchronizer is active, or another transaction is taking a while to validate,
* then we're using up a network thread for ages and clogging things up
* so bail out early
*/
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
if (!blockchainLock.tryLock()) {
LOGGER.trace(() -> String.format("Too busy to import %s transaction %s from peer %s", transactionData.getType().name(), Base58.encode(transactionData.getSignature()), peer));
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
Transaction transaction = Transaction.fromData(repository, transactionData);
@@ -957,67 +1369,117 @@ public class Controller extends Thread {
LOGGER.debug(() -> String.format("Imported %s transaction %s from peer %s", transactionData.getType().name(), Base58.encode(transactionData.getSignature()), peer));
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while processing transaction %s from peer %s", Base58.encode(transactionData.getSignature()), peer), e);
} finally {
blockchainLock.unlock();
}
// Notify controller so it can notify other peers, etc.
Controller.getInstance().onNewTransaction(transactionData, peer);
}
private void onNetworkGetBlockSummariesMessage(Peer peer, Message message) {
GetBlockSummariesMessage getBlockSummariesMessage = (GetBlockSummariesMessage) message;
byte[] parentSignature = getBlockSummariesMessage.getParentSignature();
final byte[] parentSignature = getBlockSummariesMessage.getParentSignature();
this.stats.getBlockSummariesStats.requests.incrementAndGet();
try (final Repository repository = RepositoryManager.getRepository()) {
List<BlockSummaryData> blockSummaries = new ArrayList<>();
int numberRequested = Math.min(Network.MAX_BLOCK_SUMMARIES_PER_REPLY, getBlockSummariesMessage.getNumberRequested());
do {
BlockData blockData = repository.getBlockRepository().fromReference(parentSignature);
if (blockData == null)
// No more blocks to send to peer
break;
BlockSummaryData blockSummary = new BlockSummaryData(blockData);
blockSummaries.add(blockSummary);
parentSignature = blockData.getSignature();
} while (blockSummaries.size() < numberRequested);
Message blockSummariesMessage = new BlockSummariesMessage(blockSummaries);
// If peer's parent signature matches our latest block signature
// then we can short-circuit with an empty response
BlockData chainTip = getChainTip();
if (chainTip != null && Arrays.equals(parentSignature, chainTip.getSignature())) {
Message blockSummariesMessage = new BlockSummariesMessage(Collections.emptyList());
blockSummariesMessage.setId(message.getId());
if (!peer.sendMessage(blockSummariesMessage))
peer.disconnect("failed to send block summaries");
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending block summaries after %s to peer %s", Base58.encode(parentSignature), peer), e);
return;
}
List<BlockSummaryData> blockSummaries = new ArrayList<>();
// Attempt to serve from our cache of latest blocks
synchronized (this.latestBlocks) {
blockSummaries = this.latestBlocks.stream()
.dropWhile(cachedBlockData -> !Arrays.equals(cachedBlockData.getReference(), parentSignature))
.map(BlockSummaryData::new)
.collect(Collectors.toList());
}
if (blockSummaries.isEmpty()) {
try (final Repository repository = RepositoryManager.getRepository()) {
int numberRequested = Math.min(Network.MAX_BLOCK_SUMMARIES_PER_REPLY, getBlockSummariesMessage.getNumberRequested());
BlockData blockData = repository.getBlockRepository().fromReference(parentSignature);
while (blockData != null && blockSummaries.size() < numberRequested) {
BlockSummaryData blockSummary = new BlockSummaryData(blockData);
blockSummaries.add(blockSummary);
blockData = repository.getBlockRepository().fromReference(blockData.getSignature());
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending block summaries after %s to peer %s", Base58.encode(parentSignature), peer), e);
}
} else {
this.stats.getBlockSummariesStats.cacheHits.incrementAndGet();
if (blockSummaries.size() >= getBlockSummariesMessage.getNumberRequested())
this.stats.getBlockSummariesStats.fullyFromCache.incrementAndGet();
}
Message blockSummariesMessage = new BlockSummariesMessage(blockSummaries);
blockSummariesMessage.setId(message.getId());
if (!peer.sendMessage(blockSummariesMessage))
peer.disconnect("failed to send block summaries");
}
private void onNetworkGetSignaturesV2Message(Peer peer, Message message) {
GetSignaturesV2Message getSignaturesMessage = (GetSignaturesV2Message) message;
byte[] parentSignature = getSignaturesMessage.getParentSignature();
final byte[] parentSignature = getSignaturesMessage.getParentSignature();
this.stats.getBlockSignaturesV2Stats.requests.incrementAndGet();
try (final Repository repository = RepositoryManager.getRepository()) {
List<byte[]> signatures = new ArrayList<>();
do {
BlockData blockData = repository.getBlockRepository().fromReference(parentSignature);
if (blockData == null)
// No more signatures to send to peer
break;
parentSignature = blockData.getSignature();
signatures.add(parentSignature);
} while (signatures.size() < getSignaturesMessage.getNumberRequested());
Message signaturesMessage = new SignaturesMessage(signatures);
// If peer's parent signature matches our latest block signature
// then we can short-circuit with an empty response
BlockData chainTip = getChainTip();
if (chainTip != null && Arrays.equals(parentSignature, chainTip.getSignature())) {
Message signaturesMessage = new SignaturesMessage(Collections.emptyList());
signaturesMessage.setId(message.getId());
if (!peer.sendMessage(signaturesMessage))
peer.disconnect("failed to send signatures (v2)");
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending V2 signatures after %s to peer %s", Base58.encode(parentSignature), peer), e);
return;
}
List<byte[]> signatures = new ArrayList<>();
// Attempt to serve from our cache of latest blocks
synchronized (this.latestBlocks) {
signatures = this.latestBlocks.stream()
.dropWhile(cachedBlockData -> !Arrays.equals(cachedBlockData.getReference(), parentSignature))
.map(BlockData::getSignature)
.collect(Collectors.toList());
}
if (signatures.isEmpty()) {
try (final Repository repository = RepositoryManager.getRepository()) {
int numberRequested = getSignaturesMessage.getNumberRequested();
BlockData blockData = repository.getBlockRepository().fromReference(parentSignature);
while (blockData != null && signatures.size() < numberRequested) {
signatures.add(blockData.getSignature());
blockData = repository.getBlockRepository().fromReference(blockData.getSignature());
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending V2 signatures after %s to peer %s", Base58.encode(parentSignature), peer), e);
}
} else {
this.stats.getBlockSignaturesV2Stats.cacheHits.incrementAndGet();
if (signatures.size() >= getSignaturesMessage.getNumberRequested())
this.stats.getBlockSignaturesV2Stats.fullyFromCache.incrementAndGet();
}
Message signaturesMessage = new SignaturesMessage(signatures);
signaturesMessage.setId(message.getId());
if (!peer.sendMessage(signaturesMessage))
peer.disconnect("failed to send signatures (v2)");
}
private void onNetworkHeightV2Message(Peer peer, Message message) {
@@ -1367,9 +1829,6 @@ public class Controller extends Thread {
// Refresh our online accounts signatures?
sendOurOnlineAccountsInfo();
// Trim blockchain by removing 'old' online accounts signatures
BlockChain.trimOldOnlineAccountsSignatures();
}
private void sendOurOnlineAccountsInfo() {
@@ -1631,4 +2090,8 @@ public class Controller extends Thread {
return now - offset;
}
public StatsSnapshot getStatsSnapshot() {
return this.stats;
}
}

View File

@@ -0,0 +1,79 @@
package org.qortal.controller;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.block.BlockChain;
import org.qortal.data.block.BlockData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.NTP;
public class OnlineAccountsSignaturesTrimmer implements Runnable {
private static final Logger LOGGER = LogManager.getLogger(OnlineAccountsSignaturesTrimmer.class);
private static final long INITIAL_SLEEP_PERIOD = 5 * 60 * 1000L + 1234L; // ms
public void run() {
Thread.currentThread().setName("Online Accounts trimmer");
try (final Repository repository = RepositoryManager.getRepository()) {
// Don't even start trimming until initial rush has ended
Thread.sleep(INITIAL_SLEEP_PERIOD);
int trimStartHeight = repository.getBlockRepository().getOnlineAccountsSignaturesTrimHeight();
while (!Controller.isStopping()) {
repository.discardChanges();
Thread.sleep(Settings.getInstance().getOnlineSignaturesTrimInterval());
BlockData chainTip = Controller.getInstance().getChainTip();
if (chainTip == null || NTP.getTime() == null)
continue;
// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
if (Controller.getInstance().isSynchronizing())
continue;
// Trim blockchain by removing 'old' online accounts signatures
long upperTrimmableTimestamp = NTP.getTime() - BlockChain.getInstance().getOnlineAccountSignaturesMaxLifetime();
int upperTrimmableHeight = repository.getBlockRepository().getHeightFromTimestamp(upperTrimmableTimestamp);
int upperBatchHeight = trimStartHeight + Settings.getInstance().getOnlineSignaturesTrimBatchSize();
int upperTrimHeight = Math.min(upperBatchHeight, upperTrimmableHeight);
if (trimStartHeight >= upperTrimHeight)
continue;
int numSigsTrimmed = repository.getBlockRepository().trimOldOnlineAccountsSignatures(trimStartHeight, upperTrimHeight);
repository.saveChanges();
if (numSigsTrimmed > 0) {
final int finalTrimStartHeight = trimStartHeight;
LOGGER.debug(() -> String.format("Trimmed %d online accounts signature%s between blocks %d and %d",
numSigsTrimmed, (numSigsTrimmed != 1 ? "s" : ""),
finalTrimStartHeight, upperTrimHeight));
} else {
// Can we move onto next batch?
if (upperTrimmableHeight > upperBatchHeight) {
trimStartHeight = upperBatchHeight;
repository.getBlockRepository().setOnlineAccountsSignaturesTrimHeight(trimStartHeight);
repository.saveChanges();
final int finalTrimStartHeight = trimStartHeight;
LOGGER.debug(() -> String.format("Bumping online accounts signatures base trim height to %d", finalTrimStartHeight));
}
}
}
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to trim online accounts signatures: %s", e.getMessage()));
} catch (InterruptedException e) {
// Time to exit
}
}
}

View File

@@ -1,56 +0,0 @@
package org.qortal.controller;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import org.eclipse.jetty.websocket.api.Session;
public class StatusNotifier {
private static StatusNotifier instance;
@FunctionalInterface
public interface Listener {
void notify(long timestamp);
}
private Map<Session, Listener> listenersBySession = new HashMap<>();
private StatusNotifier() {
}
public static synchronized StatusNotifier getInstance() {
if (instance == null)
instance = new StatusNotifier();
return instance;
}
public void register(Session session, Listener listener) {
synchronized (this.listenersBySession) {
this.listenersBySession.put(session, listener);
}
}
public void deregister(Session session) {
synchronized (this.listenersBySession) {
this.listenersBySession.remove(session);
}
}
public void onStatusChange(long now) {
for (Listener listener : getAllListeners())
listener.notify(now);
}
private Collection<Listener> getAllListeners() {
// Make a copy of listeners to both avoid concurrent modification
// and reduce synchronization time
synchronized (this.listenersBySession) {
return new ArrayList<>(this.listenersBySession.values());
}
}
}

View File

@@ -4,18 +4,24 @@ import java.math.BigInteger;
import java.text.DecimalFormat;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.locks.ReentrantLock;
import java.util.stream.Collectors;
import java.util.Iterator;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.Account;
import org.qortal.account.PublicKeyAccount;
import org.qortal.block.Block;
import org.qortal.block.Block.ValidationResult;
import org.qortal.block.BlockChain;
import org.qortal.data.block.BlockData;
import org.qortal.data.block.BlockSummaryData;
import org.qortal.data.block.CommonBlockData;
import org.qortal.data.network.PeerChainTipData;
import org.qortal.data.transaction.RewardShareTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.network.Peer;
import org.qortal.network.message.BlockMessage;
@@ -29,17 +35,32 @@ import org.qortal.network.message.Message.MessageType;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.transaction.Transaction;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
public class Synchronizer {
private static final Logger LOGGER = LogManager.getLogger(Synchronizer.class);
/** Max number of new blocks we aim to add to chain tip in each sync round */
private static final int SYNC_BATCH_SIZE = 200; // XXX move to Settings?
/** Initial jump back of block height when searching for common block with peer */
private static final int INITIAL_BLOCK_STEP = 8;
private static final int MAXIMUM_BLOCK_STEP = 500;
/** Maximum jump back of block height when searching for common block with peer */
private static final int MAXIMUM_BLOCK_STEP = 128;
/** Maximum difference in block height between tip and peer's common block before peer is considered TOO DIVERGENT */
private static final int MAXIMUM_COMMON_DELTA = 240; // XXX move to Settings?
private static final int SYNC_BATCH_SIZE = 200;
/** Maximum number of block signatures we ask from peer in one go */
private static final int MAXIMUM_REQUEST_SIZE = 200; // XXX move to Settings?
// Keep track of the size of the last re-org, so it can be logged
private int lastReorgSize;
private static Synchronizer instance;
@@ -59,6 +80,406 @@ public class Synchronizer {
return instance;
}
/**
* Iterate through a list of supplied peers, and attempt to find our common block with each.
* If a common block is found, its summary will be retained in the peer's commonBlockSummary property, for processing later.
* <p>
* Will return <tt>SynchronizationResult.OK</tt> on success.
* <p>
* @param peers
* @return SynchronizationResult.OK if the process completed successfully, or a different SynchronizationResult if something went wrong.
* @throws InterruptedException
*/
public SynchronizationResult findCommonBlocksWithPeers(List<Peer> peers) throws InterruptedException {
try (final Repository repository = RepositoryManager.getRepository()) {
try {
if (peers.size() == 0)
return SynchronizationResult.NOTHING_TO_DO;
// If our latest block is very old, it's best that we don't try and determine the best peers to sync to.
// This is because it can involve very large chain comparisons, which is too intensive.
// In reality, most forking problems occur near the chain tips, so we will reserve this functionality for those situations.
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp == null)
return SynchronizationResult.REPOSITORY_ISSUE;
final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
if (ourLatestBlockData.getTimestamp() < minLatestBlockTimestamp) {
LOGGER.debug(String.format("Our latest block is very old, so we won't collect common block info from peers"));
return SynchronizationResult.NOTHING_TO_DO;
}
LOGGER.debug(String.format("Searching for common blocks with %d peers...", peers.size()));
final long startTime = System.currentTimeMillis();
int commonBlocksFound = 0;
boolean wereNewRequestsMade = false;
for (Peer peer : peers) {
// Are we shutting down?
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
// Check if we can use the cached common block data, by comparing the peer's current chain tip against the peer's chain tip when we last found our common block
if (peer.canUseCachedCommonBlockData()) {
LOGGER.debug(String.format("Skipping peer %s because we already have the latest common block data in our cache. Cached common block sig is %.08s", peer, Base58.encode(peer.getCommonBlockData().getCommonBlockSummary().getSignature())));
commonBlocksFound++;
continue;
}
// Cached data is stale, so clear it and repopulate
peer.setCommonBlockData(null);
// Search for the common block
Synchronizer.getInstance().findCommonBlockWithPeer(peer, repository);
if (peer.getCommonBlockData() != null)
commonBlocksFound++;
// This round wasn't served entirely from the cache, so we may want to log the results
wereNewRequestsMade = true;
}
if (wereNewRequestsMade) {
final long totalTimeTaken = System.currentTimeMillis() - startTime;
LOGGER.debug(String.format("Finished searching for common blocks with %d peer%s. Found: %d. Total time taken: %d ms", peers.size(), (peers.size() != 1 ? "s" : ""), commonBlocksFound, totalTimeTaken));
}
return SynchronizationResult.OK;
} finally {
repository.discardChanges(); // Free repository locks, if any, also in case anything went wrong
}
} catch (DataException e) {
LOGGER.error("Repository issue during synchronization with peer", e);
return SynchronizationResult.REPOSITORY_ISSUE;
}
}
/**
* Attempt to find the find our common block with supplied peer.
* If a common block is found, its summary will be retained in the peer's commonBlockSummary property, for processing later.
* <p>
* Will return <tt>SynchronizationResult.OK</tt> on success.
* <p>
* @param peer
* @param repository
* @return SynchronizationResult.OK if the process completed successfully, or a different SynchronizationResult if something went wrong.
* @throws InterruptedException
*/
public SynchronizationResult findCommonBlockWithPeer(Peer peer, Repository repository) throws InterruptedException {
try {
final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
final int ourInitialHeight = ourLatestBlockData.getHeight();
PeerChainTipData peerChainTipData = peer.getChainTipData();
int peerHeight = peerChainTipData.getLastHeight();
byte[] peersLastBlockSignature = peerChainTipData.getLastBlockSignature();
byte[] ourLastBlockSignature = ourLatestBlockData.getSignature();
LOGGER.debug(String.format("Fetching summaries from peer %s at height %d, sig %.8s, ts %d; our height %d, sig %.8s, ts %d", peer,
peerHeight, Base58.encode(peersLastBlockSignature), peer.getChainTipData().getLastBlockTimestamp(),
ourInitialHeight, Base58.encode(ourLastBlockSignature), ourLatestBlockData.getTimestamp()));
List<BlockSummaryData> peerBlockSummaries = new ArrayList<>();
SynchronizationResult findCommonBlockResult = fetchSummariesFromCommonBlock(repository, peer, ourInitialHeight, false, peerBlockSummaries, false);
if (findCommonBlockResult != SynchronizationResult.OK) {
// Logging performed by fetchSummariesFromCommonBlock() above
peer.setCommonBlockData(null);
return findCommonBlockResult;
}
// First summary is common block
final BlockData commonBlockData = repository.getBlockRepository().fromSignature(peerBlockSummaries.get(0).getSignature());
final BlockSummaryData commonBlockSummary = new BlockSummaryData(commonBlockData);
final int commonBlockHeight = commonBlockData.getHeight();
final byte[] commonBlockSig = commonBlockData.getSignature();
final String commonBlockSig58 = Base58.encode(commonBlockSig);
LOGGER.debug(String.format("Common block with peer %s is at height %d, sig %.8s, ts %d", peer,
commonBlockHeight, commonBlockSig58, commonBlockData.getTimestamp()));
peerBlockSummaries.remove(0);
// Store the common block summary against the peer, and the current chain tip (for caching)
peer.setCommonBlockData(new CommonBlockData(commonBlockSummary, peerChainTipData));
return SynchronizationResult.OK;
} catch (DataException e) {
LOGGER.error("Repository issue during synchronization with peer", e);
return SynchronizationResult.REPOSITORY_ISSUE;
}
}
/**
* Compare a list of peers to determine the best peer(s) to sync to next.
* <p>
* Will return a filtered list of peers on success, or an identical list of peers on failure.
* This allows us to fall back to legacy behaviour (random selection from the entire list of peers), if we are unable to make the comparison.
* <p>
* @param peers
* @return a list of peers, possibly filtered.
* @throws InterruptedException
*/
public List<Peer> comparePeers(List<Peer> peers) throws InterruptedException {
try (final Repository repository = RepositoryManager.getRepository()) {
try {
// If our latest block is very old, it's best that we don't try and determine the best peers to sync to.
// This is because it can involve very large chain comparisons, which is too intensive.
// In reality, most forking problems occur near the chain tips, so we will reserve this functionality for those situations.
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp == null)
return peers;
final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
if (ourLatestBlockData.getTimestamp() < minLatestBlockTimestamp) {
LOGGER.debug(String.format("Our latest block is very old, so we won't filter the peers list"));
return peers;
}
// We will switch to a new chain weight consensus algorithm at a hard fork, so determine if this has happened yet
boolean usingSameLengthChainWeight = (NTP.getTime() >= BlockChain.getInstance().getCalcChainWeightTimestamp());
LOGGER.debug(String.format("Using %s chain weight consensus algorithm", (usingSameLengthChainWeight ? "same-length" : "variable-length")));
// Retrieve a list of unique common blocks from this list of peers
List<BlockSummaryData> commonBlocks = this.uniqueCommonBlocks(peers);
// Order common blocks by height, in ascending order
// This is essential for the logic below to make the correct decisions when discarding chains - do not remove
commonBlocks.sort((b1, b2) -> Integer.valueOf(b1.getHeight()).compareTo(Integer.valueOf(b2.getHeight())));
// Get our latest height
final int ourHeight = ourLatestBlockData.getHeight();
// Create a placeholder to track of common blocks that we can discard due to being inferior chains
int dropPeersAfterCommonBlockHeight = 0;
// Remove peers with no common block data
Iterator iterator = peers.iterator();
while (iterator.hasNext()) {
Peer peer = (Peer) iterator.next();
if (peer.getCommonBlockData() == null) {
LOGGER.debug(String.format("Removed peer %s because it has no common block data", peer));
iterator.remove();
}
}
// Loop through each group of common blocks
for (BlockSummaryData commonBlockSummary : commonBlocks) {
List<Peer> peersSharingCommonBlock = peers.stream().filter(peer -> peer.getCommonBlockData().getCommonBlockSummary().equals(commonBlockSummary)).collect(Collectors.toList());
// Check if we need to discard this group of peers
if (dropPeersAfterCommonBlockHeight > 0) {
if (commonBlockSummary.getHeight() > dropPeersAfterCommonBlockHeight) {
// We have already determined that the correct chain diverged from a lower height. We are safe to skip these peers.
for (Peer peer : peersSharingCommonBlock) {
LOGGER.debug(String.format("Peer %s has common block at height %d but the superior chain is at height %d. Removing it from this round.", peer, commonBlockSummary.getHeight(), dropPeersAfterCommonBlockHeight));
Controller.getInstance().addInferiorChainSignature(peer.getChainTipData().getLastBlockSignature());
}
continue;
}
}
// Calculate the length of the shortest peer chain sharing this common block, including our chain
final int ourAdditionalBlocksAfterCommonBlock = ourHeight - commonBlockSummary.getHeight();
int minChainLength = this.calculateMinChainLengthOfPeers(peersSharingCommonBlock, commonBlockSummary);
// Fetch block summaries from each peer
for (Peer peer : peersSharingCommonBlock) {
// If we're shutting down, just return the latest peer list
if (Controller.isStopping())
return peers;
// Count the number of blocks this peer has beyond our common block
final PeerChainTipData peerChainTipData = peer.getChainTipData();
final int peerHeight = peerChainTipData.getLastHeight();
final byte[] peerLastBlockSignature = peerChainTipData.getLastBlockSignature();
final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
// Limit the number of blocks we are comparing. FUTURE: we could request more in batches, but there may not be a case when this is needed
int summariesRequired = Math.min(peerAdditionalBlocksAfterCommonBlock, MAXIMUM_REQUEST_SIZE);
// Check if we can use the cached common block summaries, by comparing the peer's current chain tip against the peer's chain tip when we last found our common block
boolean useCachedSummaries = false;
if (peer.canUseCachedCommonBlockData()) {
if (peer.getCommonBlockData().getBlockSummariesAfterCommonBlock() != null) {
if (peer.getCommonBlockData().getBlockSummariesAfterCommonBlock().size() == summariesRequired) {
LOGGER.trace(String.format("Using cached block summaries for peer %s", peer));
useCachedSummaries = true;
}
}
}
if (useCachedSummaries == false) {
if (summariesRequired > 0) {
LOGGER.trace(String.format("Requesting %d block summar%s from peer %s after common block %.8s. Peer height: %d", summariesRequired, (summariesRequired != 1 ? "ies" : "y"), peer, Base58.encode(commonBlockSummary.getSignature()), peerHeight));
// Forget any cached summaries
peer.getCommonBlockData().setBlockSummariesAfterCommonBlock(null);
// Request new block summaries
List<BlockSummaryData> blockSummaries = this.getBlockSummaries(peer, commonBlockSummary.getSignature(), summariesRequired);
if (blockSummaries != null) {
LOGGER.trace(String.format("Peer %s returned %d block summar%s", peer, blockSummaries.size(), (blockSummaries.size() != 1 ? "ies" : "y")));
if (blockSummaries.size() < summariesRequired)
// This could mean that the peer has re-orged. Exclude this peer until they return the summaries we expect.
LOGGER.debug(String.format("Peer %s returned %d block summar%s instead of expected %d - excluding them from this round", peer, blockSummaries.size(), (blockSummaries.size() != 1 ? "ies" : "y"), summariesRequired));
else if (blockSummaryWithSignature(peerLastBlockSignature, blockSummaries) == null)
// We don't have a block summary for the peer's reported chain tip, so should exclude it
LOGGER.debug(String.format("Peer %s didn't return a block summary with signature %.8s - excluding them from this round", peer, Base58.encode(peerLastBlockSignature)));
else
// All looks good, so store the retrieved block summaries in the peer's cache
peer.getCommonBlockData().setBlockSummariesAfterCommonBlock(blockSummaries);
}
} else {
// There are no block summaries after this common block
peer.getCommonBlockData().setBlockSummariesAfterCommonBlock(null);
}
}
// Reduce minChainLength if needed. If we don't have any blocks, this peer will be excluded from chain weight comparisons later in the process, so we shouldn't update minChainLength
List <BlockSummaryData> peerBlockSummaries = peer.getCommonBlockData().getBlockSummariesAfterCommonBlock();
if (peerBlockSummaries != null && peerBlockSummaries.size() > 0)
if (peerBlockSummaries.size() < minChainLength)
minChainLength = peerBlockSummaries.size();
}
// Fetch our corresponding block summaries. Limit to MAXIMUM_REQUEST_SIZE, in order to make the comparison fairer, as peers have been limited too
final int ourSummariesRequired = Math.min(ourAdditionalBlocksAfterCommonBlock, MAXIMUM_REQUEST_SIZE);
LOGGER.trace(String.format("About to fetch our block summaries from %d to %d. Our height: %d", commonBlockSummary.getHeight() + 1, commonBlockSummary.getHeight() + ourSummariesRequired, ourHeight));
List<BlockSummaryData> ourBlockSummaries = repository.getBlockRepository().getBlockSummaries(commonBlockSummary.getHeight() + 1, commonBlockSummary.getHeight() + ourSummariesRequired);
if (ourBlockSummaries.isEmpty()) {
LOGGER.debug(String.format("We don't have any block summaries so can't compare our chain against peers with this common block. We can still compare them against each other."));
}
else {
populateBlockSummariesMinterLevels(repository, ourBlockSummaries);
// Reduce minChainLength if we have less summaries
if (ourBlockSummaries.size() < minChainLength)
minChainLength = ourBlockSummaries.size();
}
// Create array to hold peers for comparison
List<Peer> superiorPeersForComparison = new ArrayList<>();
// Calculate max height for chain weight comparisons
int maxHeightForChainWeightComparisons = commonBlockSummary.getHeight() + minChainLength;
// Calculate our chain weight
BigInteger ourChainWeight = BigInteger.valueOf(0);
if (ourBlockSummaries.size() > 0)
ourChainWeight = Block.calcChainWeight(commonBlockSummary.getHeight(), commonBlockSummary.getSignature(), ourBlockSummaries, maxHeightForChainWeightComparisons);
NumberFormat formatter = new DecimalFormat("0.###E0");
NumberFormat accurateFormatter = new DecimalFormat("0.################E0");
LOGGER.debug(String.format("Our chain weight based on %d blocks is %s", (usingSameLengthChainWeight ? minChainLength : ourBlockSummaries.size()), formatter.format(ourChainWeight)));
LOGGER.debug(String.format("Listing peers with common block %.8s...", Base58.encode(commonBlockSummary.getSignature())));
for (Peer peer : peersSharingCommonBlock) {
final int peerHeight = peer.getChainTipData().getLastHeight();
final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
final CommonBlockData peerCommonBlockData = peer.getCommonBlockData();
if (peerCommonBlockData == null || peerCommonBlockData.getBlockSummariesAfterCommonBlock() == null || peerCommonBlockData.getBlockSummariesAfterCommonBlock().isEmpty()) {
// No response - remove this peer for now
LOGGER.debug(String.format("Peer %s doesn't have any block summaries - removing it from this round", peer));
peers.remove(peer);
continue;
}
final List<BlockSummaryData> peerBlockSummariesAfterCommonBlock = peerCommonBlockData.getBlockSummariesAfterCommonBlock();
populateBlockSummariesMinterLevels(repository, peerBlockSummariesAfterCommonBlock);
// Calculate cumulative chain weight of this blockchain subset, from common block to highest mutual block held by all peers in this group.
LOGGER.debug(String.format("About to calculate chain weight based on %d blocks for peer %s with common block %.8s (peer has %d blocks after common block)", (usingSameLengthChainWeight ? minChainLength : peerBlockSummariesAfterCommonBlock.size()), peer, Base58.encode(commonBlockSummary.getSignature()), peerAdditionalBlocksAfterCommonBlock));
BigInteger peerChainWeight = Block.calcChainWeight(commonBlockSummary.getHeight(), commonBlockSummary.getSignature(), peerBlockSummariesAfterCommonBlock, maxHeightForChainWeightComparisons);
peer.getCommonBlockData().setChainWeight(peerChainWeight);
LOGGER.debug(String.format("Chain weight of peer %s based on %d blocks (%d - %d) is %s", peer, (usingSameLengthChainWeight ? minChainLength : peerBlockSummariesAfterCommonBlock.size()), peerBlockSummariesAfterCommonBlock.get(0).getHeight(), peerBlockSummariesAfterCommonBlock.get(peerBlockSummariesAfterCommonBlock.size()-1).getHeight(), formatter.format(peerChainWeight)));
// Compare against our chain - if our blockchain has greater weight then don't synchronize with peer (or any others in this group)
if (ourChainWeight.compareTo(peerChainWeight) > 0) {
// This peer is on an inferior chain - remove it
LOGGER.debug(String.format("Peer %s is on an inferior chain to us - removing it from this round", peer));
peers.remove(peer);
}
else {
// Our chain is inferior or equal
LOGGER.debug(String.format("Peer %s is on an equal or better chain to us. We will compare the other peers sharing this common block against each other, and drop all peers sharing higher common blocks.", peer));
dropPeersAfterCommonBlockHeight = commonBlockSummary.getHeight();
superiorPeersForComparison.add(peer);
}
}
// Now that we have selected the best peers, compare them against each other and remove any with lower weights
if (superiorPeersForComparison.size() > 0) {
BigInteger bestChainWeight = null;
for (Peer peer : superiorPeersForComparison) {
// Increase bestChainWeight if needed
if (bestChainWeight == null || peer.getCommonBlockData().getChainWeight().compareTo(bestChainWeight) >= 0)
bestChainWeight = peer.getCommonBlockData().getChainWeight();
}
for (Peer peer : superiorPeersForComparison) {
// Check if we should discard an inferior peer
if (peer.getCommonBlockData().getChainWeight().compareTo(bestChainWeight) < 0) {
BigInteger difference = bestChainWeight.subtract(peer.getCommonBlockData().getChainWeight());
LOGGER.debug(String.format("Peer %s has a lower chain weight (difference: %s) than other peer(s) in this group - removing it from this round.", peer, accurateFormatter.format(difference)));
peers.remove(peer);
}
}
// FUTURE: we may want to prefer peers with additional blocks, and compare the additional blocks against each other.
// This would fast track us to the best candidate for the latest block.
// Right now, peers with the exact same chain as us are treated equally to those with an additional block.
}
}
return peers;
} finally {
repository.discardChanges(); // Free repository locks, if any, also in case anything went wrong
}
} catch (DataException e) {
LOGGER.error("Repository issue during peer comparison", e);
return peers;
}
}
private List<BlockSummaryData> uniqueCommonBlocks(List<Peer> peers) {
List<BlockSummaryData> commonBlocks = new ArrayList<>();
for (Peer peer : peers) {
if (peer.getCommonBlockData() != null && peer.getCommonBlockData().getCommonBlockSummary() != null) {
LOGGER.trace(String.format("Peer %s has common block %.8s", peer, Base58.encode(peer.getCommonBlockData().getCommonBlockSummary().getSignature())));
BlockSummaryData commonBlockSummary = peer.getCommonBlockData().getCommonBlockSummary();
if (!commonBlocks.contains(commonBlockSummary))
commonBlocks.add(commonBlockSummary);
}
else {
LOGGER.trace(String.format("Peer %s has no common block data. Skipping...", peer));
}
}
return commonBlocks;
}
private int calculateMinChainLengthOfPeers(List<Peer> peersSharingCommonBlock, BlockSummaryData commonBlockSummary) {
// Calculate the length of the shortest peer chain sharing this common block
int minChainLength = 0;
for (Peer peer : peersSharingCommonBlock) {
final int peerHeight = peer.getChainTipData().getLastHeight();
final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
if (peerAdditionalBlocksAfterCommonBlock < minChainLength || minChainLength == 0)
minChainLength = peerAdditionalBlocksAfterCommonBlock;
}
return minChainLength;
}
private BlockSummaryData blockSummaryWithSignature(byte[] signature, List<BlockSummaryData> blockSummaries) {
if (blockSummaries != null)
return blockSummaries.stream().filter(blockSummary -> Arrays.equals(blockSummary.getSignature(), signature)).findAny().orElse(null);
return null;
}
/**
* Attempt to synchronize blockchain with peer.
* <p>
@@ -88,15 +509,31 @@ public class Synchronizer {
byte[] peersLastBlockSignature = peerChainTipData.getLastBlockSignature();
byte[] ourLastBlockSignature = ourLatestBlockData.getSignature();
LOGGER.debug(String.format("Synchronizing with peer %s at height %d, sig %.8s, ts %d; our height %d, sig %.8s, ts %d", peer,
String syncString = String.format("Synchronizing with peer %s at height %d, sig %.8s, ts %d; our height %d, sig %.8s, ts %d", peer,
peerHeight, Base58.encode(peersLastBlockSignature), peer.getChainTipData().getLastBlockTimestamp(),
ourInitialHeight, Base58.encode(ourLastBlockSignature), ourLatestBlockData.getTimestamp()));
ourInitialHeight, Base58.encode(ourLastBlockSignature), ourLatestBlockData.getTimestamp());
// If our latest block is very old, we should log that we're attempting to sync with a peer
// Otherwise, it can appear as though nothing is happening for a while after launch
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp != null && ourLatestBlockData.getTimestamp() < minLatestBlockTimestamp) {
LOGGER.info(syncString);
}
else {
LOGGER.debug(syncString);
}
// Reset last re-org size as we are starting a new sync round
this.lastReorgSize = 0;
List<BlockSummaryData> peerBlockSummaries = new ArrayList<>();
SynchronizationResult findCommonBlockResult = fetchSummariesFromCommonBlock(repository, peer, ourInitialHeight, force, peerBlockSummaries);
if (findCommonBlockResult != SynchronizationResult.OK)
SynchronizationResult findCommonBlockResult = fetchSummariesFromCommonBlock(repository, peer, ourInitialHeight, force, peerBlockSummaries, true);
if (findCommonBlockResult != SynchronizationResult.OK) {
// Logging performed by fetchSummariesFromCommonBlock() above
// Clear our common block cache for this peer
peer.setCommonBlockData(null);
return findCommonBlockResult;
}
// First summary is common block
final BlockData commonBlockData = repository.getBlockRepository().fromSignature(peerBlockSummaries.get(0).getSignature());
@@ -146,10 +583,19 @@ public class Synchronizer {
// Commit
repository.saveChanges();
// Create string for logging
final BlockData newLatestBlockData = repository.getBlockRepository().getLastBlock();
LOGGER.info(String.format("Synchronized with peer %s to height %d, sig %.8s, ts: %d", peer,
String syncLog = String.format("Synchronized with peer %s to height %d, sig %.8s, ts: %d", peer,
newLatestBlockData.getHeight(), Base58.encode(newLatestBlockData.getSignature()),
newLatestBlockData.getTimestamp()));
newLatestBlockData.getTimestamp());
// Append re-org info
if (this.lastReorgSize > 0) {
syncLog = syncLog.concat(String.format(", size: %d", this.lastReorgSize));
}
// Log sync info
LOGGER.info(syncLog);
return SynchronizationResult.OK;
} finally {
@@ -172,7 +618,7 @@ public class Synchronizer {
* @throws DataException
* @throws InterruptedException
*/
private SynchronizationResult fetchSummariesFromCommonBlock(Repository repository, Peer peer, int ourHeight, boolean force, List<BlockSummaryData> blockSummariesFromCommon) throws DataException, InterruptedException {
public SynchronizationResult fetchSummariesFromCommonBlock(Repository repository, Peer peer, int ourHeight, boolean force, List<BlockSummaryData> blockSummariesFromCommon, boolean infoLogWhenNotFound) throws DataException, InterruptedException {
// Start by asking for a few recent block hashes as this will cover a majority of reorgs
// Failing that, back off exponentially
int step = INITIAL_BLOCK_STEP;
@@ -201,8 +647,12 @@ public class Synchronizer {
blockSummariesBatch = this.getBlockSummaries(peer, testSignature, step);
if (blockSummariesBatch == null) {
if (infoLogWhenNotFound)
LOGGER.info(String.format("Error while trying to find common block with peer %s", peer));
else
LOGGER.debug(String.format("Error while trying to find common block with peer %s", peer));
// No response - give up this time
LOGGER.info(String.format("Error while trying to find common block with peer %s", peer));
return SynchronizationResult.NO_REPLY;
}
@@ -238,16 +688,20 @@ public class Synchronizer {
blockSummariesFromCommon.addAll(blockSummariesBatch);
// Trim summaries so that first summary is common block.
// Currently we work back from the end until we hit a block we also have.
// Currently we work forward from common block until we hit a block we don't have
// TODO: rewrite as modified binary search!
for (int i = blockSummariesFromCommon.size() - 1; i > 0; --i) {
if (repository.getBlockRepository().exists(blockSummariesFromCommon.get(i).getSignature())) {
// Note: index i isn't cleared: List.subList is fromIndex inclusive to toIndex exclusive
blockSummariesFromCommon.subList(0, i).clear();
int i;
for (i = 1; i < blockSummariesFromCommon.size(); ++i) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
if (!repository.getBlockRepository().exists(blockSummariesFromCommon.get(i).getSignature()))
break;
}
}
// Note: index i - 1 isn't cleared: List.subList is fromIndex inclusive to toIndex exclusive
blockSummariesFromCommon.subList(0, i - 1).clear();
return SynchronizationResult.OK;
}
@@ -292,6 +746,9 @@ public class Synchronizer {
// Check peer sent valid heights
for (int i = 0; i < moreBlockSummaries.size(); ++i) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
++lastSummaryHeight;
BlockSummaryData blockSummary = moreBlockSummaries.get(i);
@@ -310,18 +767,21 @@ public class Synchronizer {
List<BlockSummaryData> ourBlockSummaries = repository.getBlockRepository().getBlockSummaries(commonBlockHeight + 1, ourLatestBlockData.getHeight());
// Populate minter account levels for both lists of block summaries
populateBlockSummariesMinterLevels(repository, peerBlockSummaries);
populateBlockSummariesMinterLevels(repository, ourBlockSummaries);
populateBlockSummariesMinterLevels(repository, peerBlockSummaries);
final int mutualHeight = commonBlockHeight + Math.min(ourBlockSummaries.size(), peerBlockSummaries.size());
// Calculate cumulative chain weights of both blockchain subsets, from common block to highest mutual block.
BigInteger ourChainWeight = Block.calcChainWeight(commonBlockHeight, commonBlockSig, ourBlockSummaries);
BigInteger peerChainWeight = Block.calcChainWeight(commonBlockHeight, commonBlockSig, peerBlockSummaries);
BigInteger ourChainWeight = Block.calcChainWeight(commonBlockHeight, commonBlockSig, ourBlockSummaries, mutualHeight);
BigInteger peerChainWeight = Block.calcChainWeight(commonBlockHeight, commonBlockSig, peerBlockSummaries, mutualHeight);
NumberFormat formatter = new DecimalFormat("0.###E0");
LOGGER.debug(String.format("Our chain weight: %s, peer's chain weight: %s (higher is better)", formatter.format(ourChainWeight), formatter.format(peerChainWeight)));
// If our blockchain has greater weight then don't synchronize with peer
if (ourChainWeight.compareTo(peerChainWeight) >= 0) {
LOGGER.debug(String.format("Not synchronizing with peer %s as we have better blockchain", peer));
NumberFormat formatter = new DecimalFormat("0.###E0");
LOGGER.debug(String.format("Our chain weight: %s, peer's chain weight: %s (higher is better)", formatter.format(ourChainWeight), formatter.format(peerChainWeight)));
return SynchronizationResult.INFERIOR_CHAIN;
}
}
@@ -335,52 +795,142 @@ public class Synchronizer {
final byte[] commonBlockSig = commonBlockData.getSignature();
String commonBlockSig58 = Base58.encode(commonBlockSig);
byte[] latestPeerSignature = commonBlockSig;
int height = commonBlockHeight;
LOGGER.debug(() -> String.format("Fetching peer %s chain from height %d, sig %.8s", peer, commonBlockHeight, commonBlockSig58));
int ourHeight = ourInitialHeight;
final int maxRetries = Settings.getInstance().getMaxRetries();
// Overall plan: fetch peer's blocks first, then orphan, then apply
// Convert any leftover (post-common) block summaries into signatures to request from peer
List<byte[]> peerBlockSignatures = peerBlockSummaries.stream().map(BlockSummaryData::getSignature).collect(Collectors.toList());
// Fetch remaining block signatures, if needed
int numberSignaturesRequired = peerBlockSignatures.size() - (peerHeight - commonBlockHeight);
if (numberSignaturesRequired > 0) {
byte[] latestPeerSignature = peerBlockSignatures.isEmpty() ? commonBlockSig : peerBlockSignatures.get(peerBlockSignatures.size() - 1);
LOGGER.trace(String.format("Requesting %d signature%s after height %d, sig %.8s",
numberSignaturesRequired, (numberSignaturesRequired != 1 ? "s": ""), ourHeight, Base58.encode(latestPeerSignature)));
List<byte[]> moreBlockSignatures = this.getBlockSignatures(peer, latestPeerSignature, numberSignaturesRequired);
if (moreBlockSignatures == null || moreBlockSignatures.isEmpty()) {
LOGGER.info(String.format("Peer %s failed to respond with more block signatures after height %d, sig %.8s", peer,
ourHeight, Base58.encode(latestPeerSignature)));
return SynchronizationResult.NO_REPLY;
}
LOGGER.trace(String.format("Received %s signature%s", peerBlockSignatures.size(), (peerBlockSignatures.size() != 1 ? "s" : "")));
peerBlockSignatures.addAll(moreBlockSignatures);
}
// Fetch blocks using signatures
LOGGER.debug(String.format("Fetching new blocks from peer %s", peer));
// Keep a list of blocks received so far
List<Block> peerBlocks = new ArrayList<>();
for (byte[] blockSignature : peerBlockSignatures) {
Block newBlock = this.fetchBlock(repository, peer, blockSignature);
// Calculate the total number of additional blocks this peer has beyond the common block
int additionalPeerBlocksAfterCommonBlock = peerHeight - commonBlockHeight;
// Subtract the number of signatures that we already have, as we don't need to request them again
int numberSignaturesRequired = additionalPeerBlocksAfterCommonBlock - peerBlockSignatures.size();
int retryCount = 0;
while (height < peerHeight) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
// Ensure we don't request more than MAXIMUM_REQUEST_SIZE
int numberRequested = Math.min(numberSignaturesRequired, MAXIMUM_REQUEST_SIZE);
// Do we need more signatures?
if (peerBlockSignatures.isEmpty() && numberRequested > 0) {
LOGGER.trace(String.format("Requesting %d signature%s after height %d, sig %.8s",
numberRequested, (numberRequested != 1 ? "s" : ""), height, Base58.encode(latestPeerSignature)));
peerBlockSignatures = this.getBlockSignatures(peer, latestPeerSignature, numberRequested);
if (peerBlockSignatures == null || peerBlockSignatures.isEmpty()) {
LOGGER.info(String.format("Peer %s failed to respond with more block signatures after height %d, sig %.8s", peer,
height, Base58.encode(latestPeerSignature)));
// Clear our cache of common block summaries for this peer, as they are likely to be invalid
CommonBlockData cachedCommonBlockData = peer.getCommonBlockData();
if (cachedCommonBlockData != null)
cachedCommonBlockData.setBlockSummariesAfterCommonBlock(null);
// If we have already received newer blocks from this peer that what we have already, go ahead and apply them
if (peerBlocks.size() > 0) {
final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
final Block peerLatestBlock = peerBlocks.get(peerBlocks.size() - 1);
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (ourLatestBlockData != null && peerLatestBlock != null && minLatestBlockTimestamp != null) {
// If our latest block is very old....
if (ourLatestBlockData.getTimestamp() < minLatestBlockTimestamp) {
// ... and we have received a block that is more recent than our latest block ...
if (peerLatestBlock.getBlockData().getTimestamp() > ourLatestBlockData.getTimestamp()) {
// ... then apply the blocks, as it takes us a step forward.
// This is particularly useful when starting up a node that was on a small fork when it was last shut down.
// In these cases, we now allow the node to sync forward, and get onto the main chain again.
// Without this, we would require that the node syncs ENTIRELY with this peer,
// and any problems downloading a block would cause all progress to be lost.
LOGGER.debug(String.format("Newly received blocks are %d ms newer than our latest block - so we will apply them", peerLatestBlock.getBlockData().getTimestamp() - ourLatestBlockData.getTimestamp()));
break;
}
}
}
}
// Otherwise, give up and move on to the next peer, to avoid putting our chain into an outdated or incomplete state
return SynchronizationResult.NO_REPLY;
}
numberSignaturesRequired = peerHeight - height - peerBlockSignatures.size();
LOGGER.trace(String.format("Received %s signature%s", peerBlockSignatures.size(), (peerBlockSignatures.size() != 1 ? "s" : "")));
}
if (peerBlockSignatures.isEmpty()) {
LOGGER.trace(String.format("No more signatures or blocks to request from peer %s", peer));
break;
}
byte[] nextPeerSignature = peerBlockSignatures.get(0);
int nextHeight = height + 1;
LOGGER.trace(String.format("Fetching block %d, sig %.8s from %s", nextHeight, Base58.encode(nextPeerSignature), peer));
Block newBlock = this.fetchBlock(repository, peer, nextPeerSignature);
if (newBlock == null) {
LOGGER.info(String.format("Peer %s failed to respond with block for height %d, sig %.8s", peer,
ourHeight, Base58.encode(blockSignature)));
return SynchronizationResult.NO_REPLY;
nextHeight, Base58.encode(nextPeerSignature)));
if (retryCount >= maxRetries) {
// If we have already received newer blocks from this peer that what we have already, go ahead and apply them
if (peerBlocks.size() > 0) {
final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
final Block peerLatestBlock = peerBlocks.get(peerBlocks.size() - 1);
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (ourLatestBlockData != null && peerLatestBlock != null && minLatestBlockTimestamp != null) {
// If our latest block is very old....
if (ourLatestBlockData.getTimestamp() < minLatestBlockTimestamp) {
// ... and we have received a block that is more recent than our latest block ...
if (peerLatestBlock.getBlockData().getTimestamp() > ourLatestBlockData.getTimestamp()) {
// ... then apply the blocks, as it takes us a step forward.
// This is particularly useful when starting up a node that was on a small fork when it was last shut down.
// In these cases, we now allow the node to sync forward, and get onto the main chain again.
// Without this, we would require that the node syncs ENTIRELY with this peer,
// and any problems downloading a block would cause all progress to be lost.
LOGGER.debug(String.format("Newly received blocks are %d ms newer than our latest block - so we will apply them", peerLatestBlock.getBlockData().getTimestamp() - ourLatestBlockData.getTimestamp()));
break;
}
}
}
}
// Otherwise, give up and move on to the next peer, to avoid putting our chain into an outdated or incomplete state
return SynchronizationResult.NO_REPLY;
} else {
// Re-fetch signatures, in case the peer is now on a different fork
peerBlockSignatures.clear();
numberSignaturesRequired = peerHeight - height;
// Retry until retryCount reaches maxRetries
retryCount++;
int triesRemaining = maxRetries - retryCount;
LOGGER.info(String.format("Re-issuing request to peer %s (%d attempt%s remaining)", peer, triesRemaining, (triesRemaining != 1 ? "s" : "")));
continue;
}
}
// Reset retryCount because the last request succeeded
retryCount = 0;
LOGGER.trace(String.format("Fetched block %d, sig %.8s from %s", nextHeight, Base58.encode(latestPeerSignature), peer));
if (!newBlock.isSignatureValid()) {
LOGGER.info(String.format("Peer %s sent block with invalid signature for height %d, sig %.8s", peer,
ourHeight, Base58.encode(blockSignature)));
nextHeight, Base58.encode(latestPeerSignature)));
return SynchronizationResult.INVALID_DATA;
}
@@ -389,29 +939,47 @@ public class Synchronizer {
transaction.setInitialApprovalStatus();
peerBlocks.add(newBlock);
// Now that we've received this block, we can increase our height and move on to the next one
latestPeerSignature = nextPeerSignature;
peerBlockSignatures.remove(0);
++height;
}
// Unwind to common block (unless common block is our latest block)
LOGGER.debug(String.format("Orphaning blocks back to common block height %d, sig %.8s", commonBlockHeight, commonBlockSig58));
int ourHeight = ourInitialHeight;
LOGGER.debug(String.format("Orphaning blocks back to common block height %d, sig %.8s. Our height: %d", commonBlockHeight, commonBlockSig58, ourHeight));
int reorgSize = ourHeight - commonBlockHeight;
BlockData orphanBlockData = repository.getBlockRepository().fromHeight(ourInitialHeight);
while (ourHeight > commonBlockHeight) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
BlockData blockData = repository.getBlockRepository().fromHeight(ourHeight);
Block block = new Block(repository, blockData);
Block block = new Block(repository, orphanBlockData);
block.orphan();
LOGGER.trace(String.format("Orphaned block height %d, sig %.8s", ourHeight, Base58.encode(orphanBlockData.getSignature())));
repository.saveChanges();
--ourHeight;
orphanBlockData = repository.getBlockRepository().fromHeight(ourHeight);
repository.discardChanges(); // clear transaction status to prevent deadlocks
Controller.getInstance().onOrphanedBlock(orphanBlockData);
}
LOGGER.debug(String.format("Orphaned blocks back to height %d, sig %.8s - applying new blocks from peer %s", commonBlockHeight, commonBlockSig58, peer));
for (Block newBlock : peerBlocks) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
ValidationResult blockResult = newBlock.isValid();
if (blockResult != ValidationResult.OK) {
LOGGER.info(String.format("Peer %s sent invalid block for height %d, sig %.8s: %s", peer,
ourHeight, Base58.encode(newBlock.getSignature()), blockResult.name()));
newBlock.getBlockData().getHeight(), Base58.encode(newBlock.getSignature()), blockResult.name()));
return SynchronizationResult.INVALID_DATA;
}
@@ -423,11 +991,14 @@ public class Synchronizer {
newBlock.process();
// If we've grown our blockchain then at least save progress so far
if (ourHeight > ourInitialHeight)
repository.saveChanges();
LOGGER.trace(String.format("Processed block height %d, sig %.8s", newBlock.getBlockData().getHeight(), Base58.encode(newBlock.getBlockData().getSignature())));
repository.saveChanges();
Controller.getInstance().onNewBlock(newBlock.getBlockData());
}
this.lastReorgSize = reorgSize;
return SynchronizationResult.OK;
}
@@ -453,7 +1024,8 @@ public class Synchronizer {
// Do we need more signatures?
if (peerBlockSignatures.isEmpty()) {
int numberRequested = maxBatchHeight - ourHeight;
int numberRequested = Math.min(maxBatchHeight - ourHeight, MAXIMUM_REQUEST_SIZE);
LOGGER.trace(String.format("Requesting %d signature%s after height %d, sig %.8s",
numberRequested, (numberRequested != 1 ? "s": ""), ourHeight, Base58.encode(latestPeerSignature)));
@@ -472,7 +1044,9 @@ public class Synchronizer {
peerBlockSignatures.remove(0);
++ourHeight;
LOGGER.trace(String.format("Fetching block %d, sig %.8s from %s", ourHeight, Base58.encode(latestPeerSignature), peer));
Block newBlock = this.fetchBlock(repository, peer, latestPeerSignature);
LOGGER.trace(String.format("Fetched block %d, sig %.8s from %s", ourHeight, Base58.encode(latestPeerSignature), peer));
if (newBlock == null) {
LOGGER.info(String.format("Peer %s failed to respond with block for height %d, sig %.8s", peer,
@@ -505,9 +1079,11 @@ public class Synchronizer {
newBlock.process();
// If we've grown our blockchain then at least save progress so far
if (ourHeight > ourInitialHeight)
repository.saveChanges();
LOGGER.trace(String.format("Processed block height %d, sig %.8s", newBlock.getBlockData().getHeight(), Base58.encode(newBlock.getBlockData().getSignature())));
repository.saveChanges();
Controller.getInstance().onNewBlock(newBlock.getBlockData());
}
return SynchronizationResult.OK;
@@ -550,16 +1126,37 @@ public class Synchronizer {
}
private void populateBlockSummariesMinterLevels(Repository repository, List<BlockSummaryData> blockSummaries) throws DataException {
final int firstBlockHeight = blockSummaries.get(0).getHeight();
for (int i = 0; i < blockSummaries.size(); ++i) {
if (Controller.isStopping())
return;
BlockSummaryData blockSummary = blockSummaries.get(i);
// Qortal: minter is always a reward-share, so find actual minter and get their effective minting level
int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, blockSummary.getMinterPublicKey());
if (minterLevel == 0) {
// We don't want to throw, or use zero, as this will kill Controller thread and make client unstable.
// So we log this but use 1 instead
LOGGER.warn(String.format("Unexpected zero effective minter level for reward-share %s - using 1 instead!", Base58.encode(blockSummary.getMinterPublicKey())));
minterLevel = 1;
// It looks like this block's minter's reward-share has been cancelled.
// So search for REWARD_SHARE transactions since common block to find missing minter info
List<byte[]> transactionSignatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(Transaction.TransactionType.REWARD_SHARE, null, firstBlockHeight, null);
for (byte[] transactionSignature : transactionSignatures) {
RewardShareTransactionData transactionData = (RewardShareTransactionData) repository.getTransactionRepository().fromSignature(transactionSignature);
if (transactionData != null && Arrays.equals(transactionData.getRewardSharePublicKey(), blockSummary.getMinterPublicKey())) {
Account rewardShareMinter = new PublicKeyAccount(repository, transactionData.getMinterPublicKey());
minterLevel = rewardShareMinter.getEffectiveMintingLevel();
break;
}
}
if (minterLevel == 0) {
// We don't want to throw, or use zero, as this will kill Controller thread and make client unstable.
// So we log this but use 1 instead
LOGGER.debug(() -> String.format("Unexpected zero effective minter level for reward-share %s - using 1 instead!", Base58.encode(blockSummary.getMinterPublicKey())));
minterLevel = 1;
}
}
blockSummary.setMinterLevel(minterLevel);

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,30 @@
package org.qortal.controller.tradebot;
import java.util.List;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.crosschain.ACCT;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
public interface AcctTradeBot {
public enum ResponseResult { OK, BALANCE_ISSUE, NETWORK_ISSUE, TRADE_ALREADY_EXISTS }
/** Returns list of state names for trade-bot entries that have ended, e.g. redeemed, refunded or cancelled. */
public List<String> getEndStates();
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException;
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct,
CrossChainTradeData crossChainTradeData, String foreignKey, String receivingAddress) throws DataException;
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException;
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException;
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,883 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class DogecoinACCTv1TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv1TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static DogecoinACCTv1TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private DogecoinACCTv1TradeBot() {
}
public static synchronized DogecoinACCTv1TradeBot getInstance() {
if (instance == null)
instance = new DogecoinACCTv1TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for DOGE.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Dogecoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Dogecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>DOGE amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Dogecoin receiving address into public key hash (we only support P2PKH at this time)
Address dogecoinReceivingAddress;
try {
dogecoinReceivingAddress = Address.fromString(Dogecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (dogecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] dogecoinReceivingAccountInfo = dogecoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/DOGE ACCT";
String description = "QORT/DOGE cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT DOGE";
byte[] creationBytes = DogecoinACCTv1.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DogecoinACCTv1.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.DOGECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, dogecoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching DOGE to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Dogecoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Dogecoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Dogecoin main-net)
* or 'tprv' for (Dogecoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Dogecoin amount expected by 'Bob'.
* <p>
* If the Dogecoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Dogecoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DogecoinACCTv1.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.DOGECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Dogecoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Dogecoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Dogecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Dogecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Dogecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = DogecoinACCTv1.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = DogecoinACCTv1.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Dogecoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Dogecoin dogecoin = Dogecoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Dogecoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
DogecoinACCTv1.OfferMessageData offerMessageData = DogecoinACCTv1.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerDogecoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = DogecoinACCTv1.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = DogecoinACCTv1.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the DOGE funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Dogecoin dogecoin = Dogecoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = DogecoinACCTv1.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = DogecoinACCTv1.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the DOGE funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the DOGE funds from P2SH-A
* to Bob's 'foreign'/Dogecoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send DOGE to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the DOGE
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = DogecoinACCTv1.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Dogecoin dogecoin = Dogecoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(dogecoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
dogecoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = dogecoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Dogecoin dogecoin = Dogecoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = dogecoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = dogecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(dogecoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(dogecoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
dogecoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,894 @@
package org.qortal.controller.tradebot;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.Address;
import org.bitcoinj.core.AddressFormatException;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.ECKey;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.core.TransactionOutput;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.ACCT;
import org.qortal.crosschain.AcctMode;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.crosschain.Litecoin;
import org.qortal.crosschain.LitecoinACCTv1;
import org.qortal.crosschain.SupportedBlockchain;
import org.qortal.crosschain.BitcoinyHTLC;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class LitecoinACCTv1TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(LitecoinACCTv1TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static LitecoinACCTv1TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private LitecoinACCTv1TradeBot() {
}
public static synchronized LitecoinACCTv1TradeBot getInstance() {
if (instance == null)
instance = new LitecoinACCTv1TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for LTC.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Litecoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Litecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>LTC amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Litecoin receiving address into public key hash (we only support P2PKH at this time)
Address litecoinReceivingAddress;
try {
litecoinReceivingAddress = Address.fromString(Litecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (litecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] litecoinReceivingAccountInfo = litecoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/LTC ACCT";
String description = "QORT/LTC cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT LTC";
byte[] creationBytes = LitecoinACCTv1.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, LitecoinACCTv1.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.LITECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, litecoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching LTC to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Litecoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Litecoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Litecoin main-net)
* or 'tprv' for (Litecoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Litecoin amount expected by 'Bob'.
* <p>
* If the Litecoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Litecoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, LitecoinACCTv1.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.LITECOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository);
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Litecoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Litecoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Litecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Litecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Litecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = LitecoinACCTv1.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = LitecoinACCTv1.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Litecoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Litecoin litecoin = Litecoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Litecoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
LitecoinACCTv1.OfferMessageData offerMessageData = LitecoinACCTv1.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerLitecoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = LitecoinACCTv1.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = LitecoinACCTv1.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the LTC funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Litecoin litecoin = Litecoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = LitecoinACCTv1.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = LitecoinACCTv1.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the LTC funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the LTC funds from P2SH-A
* to Bob's 'foreign'/Litecoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send LTC to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the LTC
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = LitecoinACCTv1.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Litecoin litecoin = Litecoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(litecoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
litecoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = litecoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Litecoin litecoin = Litecoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = litecoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = litecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(litecoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(litecoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
litecoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,370 @@
package org.qortal.controller.tradebot;
import java.awt.TrayIcon.MessageType;
import java.security.SecureRandom;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.concurrent.locks.ReentrantLock;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.util.Supplier;
import org.bitcoinj.core.ECKey;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.controller.Controller;
import org.qortal.controller.tradebot.AcctTradeBot.ResponseResult;
import org.qortal.crosschain.*;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.PresenceTransactionData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
import org.qortal.group.Group;
import org.qortal.gui.SysTray;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.transaction.PresenceTransaction;
import org.qortal.transaction.PresenceTransaction.PresenceType;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.transaction.TransactionTransformer;
import org.qortal.utils.NTP;
import com.google.common.primitives.Longs;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class TradeBot implements Listener {
private static final Logger LOGGER = LogManager.getLogger(TradeBot.class);
private static final Random RANDOM = new SecureRandom();
public interface StateNameAndValueSupplier {
public String getState();
public int getStateValue();
}
public static class StateChangeEvent implements Event {
private final TradeBotData tradeBotData;
public StateChangeEvent(TradeBotData tradeBotData) {
this.tradeBotData = tradeBotData;
}
public TradeBotData getTradeBotData() {
return this.tradeBotData;
}
}
private static final Map<Class<? extends ACCT>, Supplier<AcctTradeBot>> acctTradeBotSuppliers = new HashMap<>();
static {
acctTradeBotSuppliers.put(BitcoinACCTv1.class, BitcoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv1.class, LitecoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv1.class, DogecoinACCTv1TradeBot::getInstance);
}
private static TradeBot instance;
private final Map<String, Long> presenceTimestampsByAtAddress = Collections.synchronizedMap(new HashMap<>());
private TradeBot() {
EventBus.INSTANCE.addListener(event -> TradeBot.getInstance().listen(event));
}
public static synchronized TradeBot getInstance() {
if (instance == null)
instance = new TradeBot();
return instance;
}
public ACCT getAcctUsingAtData(ATData atData) {
byte[] codeHash = atData.getCodeHash();
if (codeHash == null)
return null;
return SupportedBlockchain.getAcctByCodeHash(codeHash);
}
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ACCT acct = this.getAcctUsingAtData(atData);
if (acct == null)
return null;
return acct.populateTradeData(repository, atData);
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint,
* i.e. OFFERing QORT in exchange for foreign blockchain currency.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* <li>secret(s)</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' public key, public key hash</li>
* <li>hash(es) of secret(s)</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native' (Qortal) 'trade' address - used to MESSAGE AT</li>
* <li>'foreign' public key hash - used by Alice's to allow redeem of currency on foreign blockchain</li>
* <li>hash(es) of secret(s) - used by AT (optional) and foreign blockchain as needed</li>
* <li>QORT amount on offer by Bob</li>
* <li>foreign currency amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
// Fetch latest ACCT version for requested foreign blockchain
ACCT acct = tradeBotCreateRequest.foreignBlockchain.getLatestAcct();
AcctTradeBot acctTradeBot = findTradeBotForAcct(acct);
if (acctTradeBot == null)
return null;
return acctTradeBot.createTrade(repository, tradeBotCreateRequest);
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint,
* i.e. matching foreign blockchain currency to an existing QORT offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a foreign blockchain wallet via <tt>foreignKey</tt>.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param foreignKey foreign blockchain wallet key
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct,
CrossChainTradeData crossChainTradeData, String foreignKey, String receivingAddress) throws DataException {
AcctTradeBot acctTradeBot = findTradeBotForAcct(acct);
if (acctTradeBot == null) {
LOGGER.debug(() -> String.format("Couldn't find ACCT trade-bot for AT %s", atData.getATAddress()));
return ResponseResult.NETWORK_ISSUE;
}
// Check Alice doesn't already have an existing, on-going trade-bot entry for this AT.
if (repository.getCrossChainRepository().existsTradeWithAtExcludingStates(atData.getATAddress(), acctTradeBot.getEndStates()))
return ResponseResult.TRADE_ALREADY_EXISTS;
return acctTradeBot.startResponse(repository, atData, acct, crossChainTradeData, foreignKey, receivingAddress);
}
public boolean deleteEntry(Repository repository, byte[] tradePrivateKey) throws DataException {
TradeBotData tradeBotData = repository.getCrossChainRepository().getTradeBotData(tradePrivateKey);
if (tradeBotData == null)
// Can't delete what we don't have!
return false;
boolean canDelete = false;
ACCT acct = SupportedBlockchain.getAcctByName(tradeBotData.getAcctName());
if (acct == null)
// We can't/no longer support this ACCT
canDelete = true;
else {
AcctTradeBot acctTradeBot = findTradeBotForAcct(acct);
canDelete = acctTradeBot == null || acctTradeBot.canDelete(repository, tradeBotData);
}
if (canDelete) {
repository.getCrossChainRepository().delete(tradePrivateKey);
repository.saveChanges();
}
return canDelete;
}
@Override
public void listen(Event event) {
if (!(event instanceof Controller.NewBlockEvent))
return;
synchronized (this) {
List<TradeBotData> allTradeBotData;
try (final Repository repository = RepositoryManager.getRepository()) {
allTradeBotData = repository.getCrossChainRepository().getAllTradeBotData();
} catch (DataException e) {
LOGGER.error("Couldn't run trade bot due to repository issue", e);
return;
}
for (TradeBotData tradeBotData : allTradeBotData)
try (final Repository repository = RepositoryManager.getRepository()) {
// Find ACCT-specific trade-bot for this entry
ACCT acct = SupportedBlockchain.getAcctByName(tradeBotData.getAcctName());
if (acct == null) {
LOGGER.debug(() -> String.format("Couldn't find ACCT matching name %s", tradeBotData.getAcctName()));
continue;
}
AcctTradeBot acctTradeBot = findTradeBotForAcct(acct);
if (acctTradeBot == null) {
LOGGER.debug(() -> String.format("Couldn't find ACCT trade-bot matching name %s", tradeBotData.getAcctName()));
continue;
}
acctTradeBot.progress(repository, tradeBotData);
} catch (DataException e) {
LOGGER.error("Couldn't run trade bot due to repository issue", e);
} catch (ForeignBlockchainException e) {
LOGGER.warn(() -> String.format("Foreign blockchain issue processing trade-bot entry for AT %s: %s", tradeBotData.getAtAddress(), e.getMessage()));
}
}
}
/*package*/ static byte[] generateTradePrivateKey() {
// The private key is used for both Curve25519 and secp256k1 so needs to be valid for both.
// Curve25519 accepts any seed, so generate a valid secp256k1 key and use that.
return new ECKey().getPrivKeyBytes();
}
/*package*/ static byte[] deriveTradeNativePublicKey(byte[] privateKey) {
return PrivateKeyAccount.toPublicKey(privateKey);
}
/*package*/ static byte[] deriveTradeForeignPublicKey(byte[] privateKey) {
return ECKey.fromPrivate(privateKey).getPubKey();
}
/*package*/ static byte[] generateSecret() {
byte[] secret = new byte[32];
RANDOM.nextBytes(secret);
return secret;
}
/*package*/ static void backupTradeBotData(Repository repository) {
// Attempt to backup the trade bot data. This an optional step and doesn't impact trading, so don't throw an exception on failure
try {
LOGGER.info("About to backup trade bot data...");
repository.exportNodeLocalData();
} catch (DataException e) {
LOGGER.info(String.format("Repository issue when exporting trade bot data: %s", e.getMessage()));
}
}
/** Updates trade-bot entry to new state, with current timestamp, logs message and notifies state-change listeners. */
/*package*/ static void updateTradeBotState(Repository repository, TradeBotData tradeBotData,
String newState, int newStateValue, Supplier<String> logMessageSupplier) throws DataException {
tradeBotData.setState(newState);
tradeBotData.setStateValue(newStateValue);
tradeBotData.setTimestamp(NTP.getTime());
repository.getCrossChainRepository().save(tradeBotData);
repository.saveChanges();
if (Settings.getInstance().isTradebotSystrayEnabled())
SysTray.getInstance().showMessage("Trade-Bot", String.format("%s: %s", tradeBotData.getAtAddress(), newState), MessageType.INFO);
if (logMessageSupplier != null)
LOGGER.info(logMessageSupplier);
LOGGER.debug(() -> String.format("new state for trade-bot entry based on AT %s: %s", tradeBotData.getAtAddress(), newState));
notifyStateChange(tradeBotData);
}
/** Updates trade-bot entry to new state, with current timestamp, logs message and notifies state-change listeners. */
/*package*/ static void updateTradeBotState(Repository repository, TradeBotData tradeBotData, StateNameAndValueSupplier newStateSupplier, Supplier<String> logMessageSupplier) throws DataException {
updateTradeBotState(repository, tradeBotData, newStateSupplier.getState(), newStateSupplier.getStateValue(), logMessageSupplier);
}
/** Updates trade-bot entry to new state, with current timestamp, logs message and notifies state-change listeners. */
/*package*/ static void updateTradeBotState(Repository repository, TradeBotData tradeBotData, Supplier<String> logMessageSupplier) throws DataException {
updateTradeBotState(repository, tradeBotData, tradeBotData.getState(), tradeBotData.getStateValue(), logMessageSupplier);
}
/*package*/ static void notifyStateChange(TradeBotData tradeBotData) {
StateChangeEvent stateChangeEvent = new StateChangeEvent(tradeBotData);
EventBus.INSTANCE.notify(stateChangeEvent);
}
/*package*/ static AcctTradeBot findTradeBotForAcct(ACCT acct) {
Supplier<AcctTradeBot> acctTradeBotSupplier = acctTradeBotSuppliers.get(acct.getClass());
if (acctTradeBotSupplier == null)
return null;
return acctTradeBotSupplier.get();
}
// PRESENCE-related
/*package*/ void updatePresence(Repository repository, TradeBotData tradeBotData, CrossChainTradeData tradeData)
throws DataException {
String atAddress = tradeBotData.getAtAddress();
PrivateKeyAccount tradeNativeAccount = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
String signerAddress = tradeNativeAccount.getAddress();
/*
* There's no point in Alice trying to build a PRESENCE transaction
* for an AT that isn't locked to her, as other peers won't be able
* to validate the PRESENCE transaction as signing public key won't
* be visible.
*/
if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress))
// Signer is neither Bob, nor Alice, or trade not yet locked to Alice
return;
long now = NTP.getTime();
long threshold = now - PresenceType.TRADE_BOT.getLifetime();
long timestamp = presenceTimestampsByAtAddress.compute(atAddress, (k, v) -> (v == null || v < threshold) ? now : v);
// If timestamp hasn't been updated then nothing to do
if (timestamp != now)
return;
int txGroupId = Group.NO_GROUP;
byte[] reference = new byte[TransactionTransformer.SIGNATURE_LENGTH];
byte[] creatorPublicKey = tradeNativeAccount.getPublicKey();
long fee = 0L;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, txGroupId, reference, creatorPublicKey, fee, null);
int nonce = 0;
byte[] timestampSignature = tradeNativeAccount.sign(Longs.toByteArray(timestamp));
PresenceTransactionData transactionData = new PresenceTransactionData(baseTransactionData, nonce, PresenceType.TRADE_BOT, timestampSignature);
PresenceTransaction presenceTransaction = new PresenceTransaction(repository, transactionData);
presenceTransaction.computeNonce();
presenceTransaction.sign(tradeNativeAccount);
ValidationResult result = presenceTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK)
LOGGER.debug(() -> String.format("Unable to build trade-bot PRESENCE transaction for %s: %s", tradeBotData.getAtAddress(), result.name()));
}
}

View File

@@ -0,0 +1,25 @@
package org.qortal.crosschain;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
public interface ACCT {
public byte[] getCodeBytesHash();
public int getModeByteOffset();
public ForeignBlockchain getBlockchain();
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException;
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException;
public byte[] buildCancelMessage(String creatorQortalAddress);
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException;
}

View File

@@ -0,0 +1,21 @@
package org.qortal.crosschain;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
import java.util.Map;
public enum AcctMode {
OFFERING(0), TRADING(1), CANCELLED(2), REFUNDED(3), REDEEMED(4);
public final int value;
private static final Map<Integer, AcctMode> map = stream(AcctMode.values()).collect(toMap(mode -> mode.value, mode -> mode));
AcctMode(int value) {
this.value = value;
}
public static AcctMode valueOf(int value) {
return map.get(value);
}
}

View File

@@ -1,381 +0,0 @@
package org.qortal.crosschain;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.Address;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.ECKey;
import org.bitcoinj.core.InsufficientMoneyException;
import org.bitcoinj.core.LegacyAddress;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.core.Sha256Hash;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.core.TransactionOutput;
import org.bitcoinj.core.UTXO;
import org.bitcoinj.core.UTXOProvider;
import org.bitcoinj.core.UTXOProviderException;
import org.bitcoinj.crypto.DeterministicHierarchy;
import org.bitcoinj.crypto.DeterministicKey;
import org.bitcoinj.params.MainNetParams;
import org.bitcoinj.params.RegTestParams;
import org.bitcoinj.params.TestNet3Params;
import org.bitcoinj.script.Script.ScriptType;
import org.bitcoinj.script.ScriptBuilder;
import org.bitcoinj.utils.MonetaryFormat;
import org.bitcoinj.wallet.DeterministicKeyChain;
import org.bitcoinj.wallet.SendRequest;
import org.bitcoinj.wallet.Wallet;
import org.qortal.crosschain.ElectrumX.UnspentOutput;
import org.qortal.crypto.Crypto;
import org.qortal.settings.Settings;
import org.qortal.utils.BitTwiddling;
import com.google.common.hash.HashCode;
public class BTC {
public static final long NO_LOCKTIME_NO_RBF_SEQUENCE = 0xFFFFFFFFL;
public static final long LOCKTIME_NO_RBF_SEQUENCE = NO_LOCKTIME_NO_RBF_SEQUENCE - 1;
public static final int HASH160_LENGTH = 20;
protected static final Logger LOGGER = LogManager.getLogger(BTC.class);
private static final int TIMESTAMP_OFFSET = 4 + 32 + 32;
private static final MonetaryFormat FORMAT = new MonetaryFormat().minDecimals(8).postfixCode();
public enum BitcoinNet {
MAIN {
@Override
public NetworkParameters getParams() {
return MainNetParams.get();
}
},
TEST3 {
@Override
public NetworkParameters getParams() {
return TestNet3Params.get();
}
},
REGTEST {
@Override
public NetworkParameters getParams() {
return RegTestParams.get();
}
};
public abstract NetworkParameters getParams();
}
private static BTC instance;
private final NetworkParameters params;
private final ElectrumX electrumX;
// Let ECKey.equals() do the hard work
private final Set<ECKey> spentKeys = new HashSet<>();
// Constructors and instance
private BTC() {
BitcoinNet bitcoinNet = Settings.getInstance().getBitcoinNet();
this.params = bitcoinNet.getParams();
LOGGER.info(() -> String.format("Starting Bitcoin support using %s", bitcoinNet.name()));
this.electrumX = ElectrumX.getInstance(bitcoinNet.name());
}
public static synchronized BTC getInstance() {
if (instance == null)
instance = new BTC();
return instance;
}
// Getters & setters
public NetworkParameters getNetworkParameters() {
return this.params;
}
public static synchronized void resetForTesting() {
instance = null;
}
// Actual useful methods for use by other classes
public static String format(Coin amount) {
return BTC.FORMAT.format(amount).toString();
}
public static String format(long amount) {
return format(Coin.valueOf(amount));
}
public boolean isValidXprv(String xprv58) {
try {
DeterministicKey.deserializeB58(null, xprv58, this.params);
return true;
} catch (IllegalArgumentException e) {
return false;
}
}
/** Returns P2PKH Bitcoin address using passed public key hash. */
public String pkhToAddress(byte[] publicKeyHash) {
return LegacyAddress.fromPubKeyHash(this.params, publicKeyHash).toString();
}
public String deriveP2shAddress(byte[] redeemScriptBytes) {
byte[] redeemScriptHash = Crypto.hash160(redeemScriptBytes);
Address p2shAddress = LegacyAddress.fromScriptHash(params, redeemScriptHash);
return p2shAddress.toString();
}
/** Returns median timestamp from latest 11 blocks, in seconds. */
public Integer getMedianBlockTime() {
Integer height = this.electrumX.getCurrentHeight();
if (height == null)
return null;
// Grab latest 11 blocks
List<byte[]> blockHeaders = this.electrumX.getBlockHeaders(height - 11, 11);
if (blockHeaders == null || blockHeaders.size() < 11)
return null;
List<Integer> blockTimestamps = blockHeaders.stream().map(blockHeader -> BitTwiddling.intFromLEBytes(blockHeader, TIMESTAMP_OFFSET)).collect(Collectors.toList());
// Descending order
blockTimestamps.sort((a, b) -> Integer.compare(b, a));
// Pick median
return blockTimestamps.get(5);
}
public Long getBalance(String base58Address) {
return this.electrumX.getBalance(addressToScript(base58Address));
}
public List<TransactionOutput> getUnspentOutputs(String base58Address) {
List<UnspentOutput> unspentOutputs = this.electrumX.getUnspentOutputs(addressToScript(base58Address));
if (unspentOutputs == null)
return null;
List<TransactionOutput> unspentTransactionOutputs = new ArrayList<>();
for (UnspentOutput unspentOutput : unspentOutputs) {
List<TransactionOutput> transactionOutputs = getOutputs(unspentOutput.hash);
if (transactionOutputs == null)
return null;
unspentTransactionOutputs.add(transactionOutputs.get(unspentOutput.index));
}
return unspentTransactionOutputs;
}
public List<TransactionOutput> getOutputs(byte[] txHash) {
byte[] rawTransactionBytes = this.electrumX.getRawTransaction(txHash);
if (rawTransactionBytes == null)
return null;
Transaction transaction = new Transaction(this.params, rawTransactionBytes);
return transaction.getOutputs();
}
/** Returns list of raw transactions spending passed address. */
public List<byte[]> getAddressTransactions(String base58Address) {
return this.electrumX.getAddressTransactions(addressToScript(base58Address));
}
public boolean broadcastTransaction(Transaction transaction) {
return this.electrumX.broadcastTransaction(transaction.bitcoinSerialize());
}
/**
* Returns bitcoinj transaction sending <tt>amount</tt> to <tt>recipient</tt>.
*
* @param xprv58 BIP32 extended Bitcoin private key
* @param recipient P2PKH address
* @param amount unscaled amount
* @return transaction, or null if insufficient funds
*/
public Transaction buildSpend(String xprv58, String recipient, long amount) {
Wallet wallet = Wallet.fromSpendingKeyB58(this.params, xprv58, DeterministicHierarchy.BIP32_STANDARDISATION_TIME_SECS);
wallet.setUTXOProvider(new WalletAwareUTXOProvider(this, wallet, WalletAwareUTXOProvider.KeySearchMode.REQUEST_MORE_IF_ALL_SPENT));
Address destination = Address.fromString(this.params, recipient);
SendRequest sendRequest = SendRequest.to(destination, Coin.valueOf(amount));
if (this.params == TestNet3Params.get())
// Much smaller fee for TestNet3
sendRequest.feePerKb = Coin.valueOf(2000L);
try {
wallet.completeTx(sendRequest);
return sendRequest.tx;
} catch (InsufficientMoneyException e) {
return null;
}
}
/**
* Returns unspent Bitcoin balance given 'm' BIP32 key.
*
* @param xprv58 BIP32 extended Bitcoin private key
* @return unspent BTC balance, or null if unable to determine balance
*/
public Long getWalletBalance(String xprv58) {
Wallet wallet = Wallet.fromSpendingKeyB58(this.params, xprv58, DeterministicHierarchy.BIP32_STANDARDISATION_TIME_SECS);
wallet.setUTXOProvider(new WalletAwareUTXOProvider(this, wallet, WalletAwareUTXOProvider.KeySearchMode.REQUEST_MORE_IF_ANY_SPENT));
Coin balance = wallet.getBalance();
if (balance == null)
return null;
return balance.value;
}
// UTXOProvider support
static class WalletAwareUTXOProvider implements UTXOProvider {
private static final int LOOKAHEAD_INCREMENT = 3;
private final BTC btc;
private final Wallet wallet;
enum KeySearchMode {
REQUEST_MORE_IF_ALL_SPENT, REQUEST_MORE_IF_ANY_SPENT;
}
private final KeySearchMode keySearchMode;
private final DeterministicKeyChain keyChain;
public WalletAwareUTXOProvider(BTC btc, Wallet wallet, KeySearchMode keySearchMode) {
this.btc = btc;
this.wallet = wallet;
this.keySearchMode = keySearchMode;
this.keyChain = this.wallet.getActiveKeyChain();
// Set up wallet's key chain
this.keyChain.setLookaheadSize(LOOKAHEAD_INCREMENT);
this.keyChain.maybeLookAhead();
}
public List<UTXO> getOpenTransactionOutputs(List<ECKey> keys) throws UTXOProviderException {
List<UTXO> allUnspentOutputs = new ArrayList<>();
final boolean coinbase = false;
int ki = 0;
do {
boolean areAllKeysUnspent = true;
boolean areAllKeysSpent = true;
for (; ki < keys.size(); ++ki) {
ECKey key = keys.get(ki);
Address address = Address.fromKey(btc.params, key, ScriptType.P2PKH);
byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
List<UnspentOutput> unspentOutputs = btc.electrumX.getUnspentOutputs(script);
if (unspentOutputs == null)
throw new UTXOProviderException(String.format("Unable to fetch unspent outputs for %s", address));
/*
* If there are no unspent outputs then either:
* a) all the outputs have been spent
* b) address has never been used
*
* For case (a) we want to remember not to check this address (key) again.
*/
if (unspentOutputs.isEmpty()) {
// If this is a known key that has been spent before, then we can skip asking for transaction history
if (btc.spentKeys.contains(key)) {
wallet.getActiveKeyChain().markKeyAsUsed((DeterministicKey) key);
areAllKeysUnspent = false;
continue;
}
// Ask for transaction history - if it's empty then key has never been used
List<byte[]> historicTransactionHashes = btc.electrumX.getAddressTransactions(script);
if (historicTransactionHashes == null)
throw new UTXOProviderException(
String.format("Unable to fetch transaction history for %s", address));
if (!historicTransactionHashes.isEmpty()) {
// Fully spent key - case (a)
btc.spentKeys.add(key);
wallet.getActiveKeyChain().markKeyAsUsed((DeterministicKey) key);
areAllKeysUnspent = false;
} else {
// Key never been used - case (b)
areAllKeysSpent = false;
}
continue;
}
// If we reach here, then there's definitely at least one unspent key
areAllKeysSpent = false;
for (UnspentOutput unspentOutput : unspentOutputs) {
List<TransactionOutput> transactionOutputs = btc.getOutputs(unspentOutput.hash);
if (transactionOutputs == null)
throw new UTXOProviderException(String.format("Unable to fetch outputs for TX %s",
HashCode.fromBytes(unspentOutput.hash)));
TransactionOutput transactionOutput = transactionOutputs.get(unspentOutput.index);
UTXO utxo = new UTXO(Sha256Hash.wrap(unspentOutput.hash), unspentOutput.index,
Coin.valueOf(unspentOutput.value), unspentOutput.height, coinbase,
transactionOutput.getScriptPubKey());
allUnspentOutputs.add(utxo);
}
}
if ((this.keySearchMode == KeySearchMode.REQUEST_MORE_IF_ALL_SPENT && areAllKeysSpent)
|| (this.keySearchMode == KeySearchMode.REQUEST_MORE_IF_ANY_SPENT && !areAllKeysUnspent)) {
// Generate some more keys
this.keyChain.setLookaheadSize(this.keyChain.getLookaheadSize() + LOOKAHEAD_INCREMENT);
this.keyChain.maybeLookAhead();
// This returns all keys, including those already in 'keys'
List<DeterministicKey> allLeafKeys = this.keyChain.getLeafKeys();
// Add only new keys onto our list of keys to search
List<DeterministicKey> newKeys = allLeafKeys.subList(ki, allLeafKeys.size());
keys.addAll(newKeys);
// Fall-through to checking more keys as now 'ki' is smaller than 'keys.size()' again
}
// If we have processed all keys, then we're done
} while (ki < keys.size());
return allUnspentOutputs;
}
public int getChainHeadHeight() throws UTXOProviderException {
Integer height = btc.electrumX.getCurrentHeight();
if (height == null)
throw new UTXOProviderException("Unable to determine Bitcoin chain height");
return height.intValue();
}
public NetworkParameters getParams() {
return btc.params;
}
}
// Utility methods for us
private byte[] addressToScript(String base58Address) {
Address address = Address.fromString(this.params, base58Address);
return ScriptBuilder.createOutputScript(address).getProgram();
}
}

View File

@@ -1,236 +0,0 @@
package org.qortal.crosschain;
import java.util.List;
import java.util.function.Function;
import org.bitcoinj.core.Address;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.ECKey;
import org.bitcoinj.core.LegacyAddress;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.core.Transaction.SigHash;
import org.bitcoinj.core.TransactionInput;
import org.bitcoinj.core.TransactionOutput;
import org.bitcoinj.crypto.TransactionSignature;
import org.bitcoinj.script.Script;
import org.bitcoinj.script.ScriptBuilder;
import org.bitcoinj.script.ScriptChunk;
import org.bitcoinj.script.ScriptOpCodes;
import org.qortal.crypto.Crypto;
import org.qortal.utils.BitTwiddling;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
public class BTCP2SH {
public static final int SECRET_LENGTH = 32;
public static final int MIN_LOCKTIME = 1500000000;
/*
* OP_TUCK (to copy public key to before signature)
* OP_CHECKSIGVERIFY (sig & pubkey must verify or script fails)
* OP_HASH160 (convert public key to PKH)
* OP_DUP (duplicate PKH)
* <push 20 bytes> <refund PKH> OP_EQUAL (does PKH match refund PKH?)
* OP_IF
* OP_DROP (no need for duplicate PKH)
* <push 4 bytes> <locktime>
* OP_CHECKLOCKTIMEVERIFY (if this passes, leftover stack is <locktime> so script passes)
* OP_ELSE
* <push 20 bytes> <redeem PKH> OP_EQUALVERIFY (duplicate PKH must match redeem PKH or script fails)
* OP_HASH160 (hash secret)
* <push 20 bytes> <hash of secret> OP_EQUAL (do hashes of secrets match? if true, script passes else script fails)
* OP_ENDIF
*/
private static final byte[] redeemScript1 = HashCode.fromString("7dada97614").asBytes(); // OP_TUCK OP_CHECKSIGVERIFY OP_HASH160 OP_DUP push(0x14 bytes)
private static final byte[] redeemScript2 = HashCode.fromString("87637504").asBytes(); // OP_EQUAL OP_IF OP_DROP push(0x4 bytes)
private static final byte[] redeemScript3 = HashCode.fromString("b16714").asBytes(); // OP_CHECKLOCKTIMEVERIFY OP_ELSE push(0x14 bytes)
private static final byte[] redeemScript4 = HashCode.fromString("88a914").asBytes(); // OP_EQUALVERIFY OP_HASH160 push(0x14 bytes)
private static final byte[] redeemScript5 = HashCode.fromString("8768").asBytes(); // OP_EQUAL OP_ENDIF
/**
* Returns Bitcoin redeemScript used for cross-chain trading.
* <p>
* See comments in {@link BTCP2SH} for more details.
*
* @param refunderPubKeyHash 20-byte HASH160 of P2SH funder's public key, for refunding purposes
* @param lockTime seconds-since-epoch threshold, after which P2SH funder can claim refund
* @param redeemerPubKeyHash 20-byte HASH160 of P2SH redeemer's public key
* @param secretHash 20-byte HASH160 of secret, used by P2SH redeemer to claim funds
* @return
*/
public static byte[] buildScript(byte[] refunderPubKeyHash, int lockTime, byte[] redeemerPubKeyHash, byte[] secretHash) {
return Bytes.concat(redeemScript1, refunderPubKeyHash, redeemScript2, BitTwiddling.toLEByteArray((int) (lockTime & 0xffffffffL)),
redeemScript3, redeemerPubKeyHash, redeemScript4, secretHash, redeemScript5);
}
/**
* Builds a custom transaction to spend P2SH.
*
* @param amount output amount, should be total of input amounts, less miner fees
* @param spendKey key for signing transaction, and also where funds are 'sent' (output)
* @param fundingOutput output from transaction that funded P2SH address
* @param redeemScriptBytes the redeemScript itself, in byte[] form
* @param lockTime (optional) transaction nLockTime, used in refund scenario
* @param scriptSigBuilder function for building scriptSig using transaction input signature
* @param outputPublicKeyHash PKH used to create P2PKH output
* @return Signed Bitcoin transaction for spending P2SH
*/
public static Transaction buildP2shTransaction(Coin amount, ECKey spendKey, List<TransactionOutput> fundingOutputs, byte[] redeemScriptBytes,
Long lockTime, Function<byte[], Script> scriptSigBuilder, byte[] outputPublicKeyHash) {
NetworkParameters params = BTC.getInstance().getNetworkParameters();
Transaction transaction = new Transaction(params);
transaction.setVersion(2);
// Output is back to P2SH funder
transaction.addOutput(amount, ScriptBuilder.createP2PKHOutputScript(outputPublicKeyHash));
for (int inputIndex = 0; inputIndex < fundingOutputs.size(); ++inputIndex) {
TransactionOutput fundingOutput = fundingOutputs.get(inputIndex);
// Input (without scriptSig prior to signing)
TransactionInput input = new TransactionInput(params, null, redeemScriptBytes, fundingOutput.getOutPointFor());
if (lockTime != null)
input.setSequenceNumber(BTC.LOCKTIME_NO_RBF_SEQUENCE); // Use max-value - 1, so lockTime can be used but not RBF
else
input.setSequenceNumber(BTC.NO_LOCKTIME_NO_RBF_SEQUENCE); // Use max-value, so no lockTime and no RBF
transaction.addInput(input);
}
// Set locktime after inputs added but before input signatures are generated
if (lockTime != null)
transaction.setLockTime(lockTime);
for (int inputIndex = 0; inputIndex < fundingOutputs.size(); ++inputIndex) {
// Generate transaction signature for input
final boolean anyoneCanPay = false;
TransactionSignature txSig = transaction.calculateSignature(inputIndex, spendKey, redeemScriptBytes, SigHash.ALL, anyoneCanPay);
// Calculate transaction signature
byte[] txSigBytes = txSig.encodeToBitcoin();
// Build scriptSig using lambda and tx signature
Script scriptSig = scriptSigBuilder.apply(txSigBytes);
// Set input scriptSig
transaction.getInput(inputIndex).setScriptSig(scriptSig);
}
return transaction;
}
/**
* Returns signed Bitcoin transaction claiming refund from P2SH address.
*
* @param refundAmount refund amount, should be total of input amounts, less miner fees
* @param refundKey key for signing transaction, and also where refund is 'sent' (output)
* @param fundingOutput output from transaction that funded P2SH address
* @param redeemScriptBytes the redeemScript itself, in byte[] form
* @param lockTime transaction nLockTime - must be at least locktime used in redeemScript
* @return Signed Bitcoin transaction for refunding P2SH
*/
public static Transaction buildRefundTransaction(Coin refundAmount, ECKey refundKey, List<TransactionOutput> fundingOutputs, byte[] redeemScriptBytes, long lockTime) {
Function<byte[], Script> refundSigScriptBuilder = (txSigBytes) -> {
// Build scriptSig with...
ScriptBuilder scriptBuilder = new ScriptBuilder();
// transaction signature
scriptBuilder.addChunk(new ScriptChunk(txSigBytes.length, txSigBytes));
// redeem public key
byte[] refundPubKey = refundKey.getPubKey();
scriptBuilder.addChunk(new ScriptChunk(refundPubKey.length, refundPubKey));
// redeem script
scriptBuilder.addChunk(new ScriptChunk(ScriptOpCodes.OP_PUSHDATA1, redeemScriptBytes));
return scriptBuilder.build();
};
// Send funds back to funding address
return buildP2shTransaction(refundAmount, refundKey, fundingOutputs, redeemScriptBytes, lockTime, refundSigScriptBuilder, refundKey.getPubKeyHash());
}
/**
* Returns signed Bitcoin transaction redeeming funds from P2SH address.
*
* @param redeemAmount redeem amount, should be total of input amounts, less miner fees
* @param redeemKey key for signing transaction, and also where funds are 'sent' (output)
* @param fundingOutput output from transaction that funded P2SH address
* @param redeemScriptBytes the redeemScript itself, in byte[] form
* @param secret actual 32-byte secret used when building redeemScript
* @param receivingAccountInfo Bitcoin PKH used for output
* @return Signed Bitcoin transaction for redeeming P2SH
*/
public static Transaction buildRedeemTransaction(Coin redeemAmount, ECKey redeemKey, List<TransactionOutput> fundingOutputs, byte[] redeemScriptBytes, byte[] secret, byte[] receivingAccountInfo) {
Function<byte[], Script> redeemSigScriptBuilder = (txSigBytes) -> {
// Build scriptSig with...
ScriptBuilder scriptBuilder = new ScriptBuilder();
// secret
scriptBuilder.addChunk(new ScriptChunk(secret.length, secret));
// transaction signature
scriptBuilder.addChunk(new ScriptChunk(txSigBytes.length, txSigBytes));
// redeem public key
byte[] redeemPubKey = redeemKey.getPubKey();
scriptBuilder.addChunk(new ScriptChunk(redeemPubKey.length, redeemPubKey));
// redeem script
scriptBuilder.addChunk(new ScriptChunk(ScriptOpCodes.OP_PUSHDATA1, redeemScriptBytes));
return scriptBuilder.build();
};
return buildP2shTransaction(redeemAmount, redeemKey, fundingOutputs, redeemScriptBytes, null, redeemSigScriptBuilder, receivingAccountInfo);
}
/** Returns 'secret', if any, given list of raw bitcoin transactions. */
public static byte[] findP2shSecret(String p2shAddress, List<byte[]> rawTransactions) {
NetworkParameters params = BTC.getInstance().getNetworkParameters();
for (byte[] rawTransaction : rawTransactions) {
Transaction transaction = new Transaction(params, rawTransaction);
// Cycle through inputs, looking for one that spends our P2SH
for (TransactionInput input : transaction.getInputs()) {
Script scriptSig = input.getScriptSig();
List<ScriptChunk> scriptChunks = scriptSig.getChunks();
// Expected number of script chunks for redeem. Refund might not have the same number.
int expectedChunkCount = 1 /*secret*/ + 1 /*sig*/ + 1 /*pubkey*/ + 1 /*redeemScript*/;
if (scriptChunks.size() != expectedChunkCount)
continue;
// We're expecting last chunk to contain the actual redeemScript
ScriptChunk lastChunk = scriptChunks.get(scriptChunks.size() - 1);
byte[] redeemScriptBytes = lastChunk.data;
// If non-push scripts, redeemScript will be null
if (redeemScriptBytes == null)
continue;
byte[] redeemScriptHash = Crypto.hash160(redeemScriptBytes);
Address inputAddress = LegacyAddress.fromScriptHash(params, redeemScriptHash);
if (!inputAddress.toString().equals(p2shAddress))
// Input isn't spending our P2SH
continue;
byte[] secret = scriptChunks.get(0).data;
if (secret.length != BTCP2SH.SECRET_LENGTH)
continue;
return secret;
}
}
return null;
}
}

View File

@@ -0,0 +1,194 @@
package org.qortal.crosschain;
import java.util.Arrays;
import java.util.Collection;
import java.util.EnumMap;
import java.util.Map;
import org.bitcoinj.core.Context;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.params.MainNetParams;
import org.bitcoinj.params.RegTestParams;
import org.bitcoinj.params.TestNet3Params;
import org.qortal.crosschain.ElectrumX.Server;
import org.qortal.crosschain.ElectrumX.Server.ConnectionType;
import org.qortal.settings.Settings;
public class Bitcoin extends Bitcoiny {
public static final String CURRENCY_CODE = "BTC";
// Temporary values until a dynamic fee system is written.
private static final long OLD_FEE_AMOUNT = 4_000L; // Not 5000 so that existing P2SH-B can output 1000, avoiding dust issue, leaving 4000 for fees.
private static final long NEW_FEE_TIMESTAMP = 1598280000000L; // milliseconds since epoch
private static final long NEW_FEE_AMOUNT = 10_000L;
private static final long NON_MAINNET_FEE = 1000L; // enough for TESTNET3 and should be OK for REGTEST
private static final Map<ElectrumX.Server.ConnectionType, Integer> DEFAULT_ELECTRUMX_PORTS = new EnumMap<>(ElectrumX.Server.ConnectionType.class);
static {
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.TCP, 50001);
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.SSL, 50002);
}
public enum BitcoinNet {
MAIN {
@Override
public NetworkParameters getParams() {
return MainNetParams.get();
}
@Override
public Collection<ElectrumX.Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("128.0.190.26", Server.ConnectionType.SSL, 50002),
new Server("hodlers.beer", Server.ConnectionType.SSL, 50002),
new Server("electrumx.erbium.eu", Server.ConnectionType.TCP, 50001),
new Server("electrumx.erbium.eu", Server.ConnectionType.SSL, 50002),
new Server("btc.lastingcoin.net", Server.ConnectionType.SSL, 50002),
new Server("electrum.bitaroo.net", Server.ConnectionType.SSL, 50002),
new Server("bitcoin.grey.pw", Server.ConnectionType.SSL, 50002),
new Server("2electrumx.hopto.me", Server.ConnectionType.SSL, 56022),
new Server("185.64.116.15", Server.ConnectionType.SSL, 50002),
new Server("kirsche.emzy.de", Server.ConnectionType.SSL, 50002),
new Server("alviss.coinjoined.com", Server.ConnectionType.SSL, 50002),
new Server("electrum.emzy.de", Server.ConnectionType.SSL, 50002),
new Server("electrum.emzy.de", Server.ConnectionType.TCP, 50001),
new Server("vmd71287.contaboserver.net", Server.ConnectionType.SSL, 50002),
new Server("btc.litepay.ch", Server.ConnectionType.SSL, 50002),
new Server("electrum.stippy.com", Server.ConnectionType.SSL, 50002),
new Server("xtrum.com", Server.ConnectionType.SSL, 50002),
new Server("electrum.acinq.co", Server.ConnectionType.SSL, 50002),
new Server("electrum2.taborsky.cz", Server.ConnectionType.SSL, 50002),
new Server("vmd63185.contaboserver.net", Server.ConnectionType.SSL, 50002),
new Server("electrum2.privateservers.network", Server.ConnectionType.SSL, 50002),
new Server("electrumx.alexridevski.net", Server.ConnectionType.SSL, 50002),
new Server("192.166.219.200", Server.ConnectionType.SSL, 50002),
new Server("2ex.digitaleveryware.com", Server.ConnectionType.SSL, 50002),
new Server("dxm.no-ip.biz", Server.ConnectionType.SSL, 50002),
new Server("caleb.vegas", Server.ConnectionType.SSL, 50002),
new Server("ecdsa.net", Server.ConnectionType.SSL, 110),
new Server("electrum.hsmiths.com", Server.ConnectionType.SSL, 995),
new Server("elec.luggs.co", Server.ConnectionType.SSL, 443),
new Server("btc.smsys.me", Server.ConnectionType.SSL, 995));
}
@Override
public String getGenesisHash() {
return "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f";
}
@Override
public long getP2shFee(Long timestamp) {
// TODO: This will need to be replaced with something better in the near future!
if (timestamp != null && timestamp < NEW_FEE_TIMESTAMP)
return OLD_FEE_AMOUNT;
return NEW_FEE_AMOUNT;
}
},
TEST3 {
@Override
public NetworkParameters getParams() {
return TestNet3Params.get();
}
@Override
public Collection<ElectrumX.Server> getServers() {
return Arrays.asList(
new Server("tn.not.fyi", Server.ConnectionType.SSL, 55002),
new Server("electrumx-test.1209k.com", Server.ConnectionType.SSL, 50002),
new Server("testnet.qtornado.com", Server.ConnectionType.SSL, 51002),
new Server("testnet.aranguren.org", Server.ConnectionType.TCP, 51001),
new Server("testnet.aranguren.org", Server.ConnectionType.SSL, 51002),
new Server("testnet.hsmiths.com", Server.ConnectionType.SSL, 53012));
}
@Override
public String getGenesisHash() {
return "000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943";
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
},
REGTEST {
@Override
public NetworkParameters getParams() {
return RegTestParams.get();
}
@Override
public Collection<ElectrumX.Server> getServers() {
return Arrays.asList(
new Server("localhost", Server.ConnectionType.TCP, 50001),
new Server("localhost", Server.ConnectionType.SSL, 50002));
}
@Override
public String getGenesisHash() {
// This is unique to each regtest instance
return null;
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
};
public abstract NetworkParameters getParams();
public abstract Collection<ElectrumX.Server> getServers();
public abstract String getGenesisHash();
public abstract long getP2shFee(Long timestamp) throws ForeignBlockchainException;
}
private static Bitcoin instance;
private final BitcoinNet bitcoinNet;
// Constructors and instance
private Bitcoin(BitcoinNet bitcoinNet, BitcoinyBlockchainProvider blockchain, Context bitcoinjContext, String currencyCode) {
super(blockchain, bitcoinjContext, currencyCode);
this.bitcoinNet = bitcoinNet;
LOGGER.info(() -> String.format("Starting Bitcoin support using %s", this.bitcoinNet.name()));
}
public static synchronized Bitcoin getInstance() {
if (instance == null) {
BitcoinNet bitcoinNet = Settings.getInstance().getBitcoinNet();
BitcoinyBlockchainProvider electrumX = new ElectrumX("Bitcoin-" + bitcoinNet.name(), bitcoinNet.getGenesisHash(), bitcoinNet.getServers(), DEFAULT_ELECTRUMX_PORTS);
Context bitcoinjContext = new Context(bitcoinNet.getParams());
instance = new Bitcoin(bitcoinNet, electrumX, bitcoinjContext, CURRENCY_CODE);
}
return instance;
}
// Getters & setters
public static synchronized void resetForTesting() {
instance = null;
}
// Actual useful methods for use by other classes
/**
* Returns estimated BTC fee, in sats per 1000bytes, optionally for historic timestamp.
*
* @param timestamp optional milliseconds since epoch, or null for 'now'
* @return sats per 1000bytes, or throws ForeignBlockchainException if something went wrong
*/
@Override
public long getP2shFee(Long timestamp) throws ForeignBlockchainException {
return this.bitcoinNet.getP2shFee(timestamp);
}
}

View File

@@ -1,13 +1,10 @@
package org.qortal.crosschain;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
import static org.ciyam.at.OpCode.calcOffset;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import org.ciyam.at.API;
import org.ciyam.at.CompilationException;
@@ -101,11 +98,12 @@ import com.google.common.primitives.Bytes;
* </li>
* </ul>
*/
public class BTCACCT {
public class BitcoinACCTv1 implements ACCT {
public static final String NAME = BitcoinACCTv1.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("f7f419522a9aaa3c671149878f8c1374dfc59d4fd86ca43ff2a4d913cfbc9e89").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
public static final int MIN_LOCKTIME = 1500000000;
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("f7f419522a9aaa3c671149878f8c1374dfc59d4fd86ca43ff2a4d913cfbc9e89").asBytes(); // SHA256 of AT code bytes
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 68;
@@ -126,22 +124,31 @@ public class BTCACCT {
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret*/ + 32 /*secret*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
public enum Mode {
OFFERING(0), TRADING(1), CANCELLED(2), REFUNDED(3), REDEEMED(4);
private static BitcoinACCTv1 instance;
public final int value;
private static final Map<Integer, Mode> map = stream(Mode.values()).collect(toMap(mode -> mode.value, mode -> mode));
Mode(int value) {
this.value = value;
}
public static Mode valueOf(int value) {
return map.get(value);
}
private BitcoinACCTv1() {
}
private BTCACCT() {
public static synchronized BitcoinACCTv1 getInstance() {
if (instance == null)
instance = new BitcoinACCTv1();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Bitcoin.getInstance();
}
/**
@@ -156,7 +163,6 @@ public class BTCACCT {
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param bitcoinAmount how much BTC the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
* @return
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] bitcoinPublicKeyHash, byte[] hashOfSecretB, long qortAmount, long bitcoinAmount, int tradeTimeout) {
// Labels for data segment addresses
@@ -419,7 +425,7 @@ public class BTCACCT {
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, Mode.CANCELLED.value));
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
@@ -470,7 +476,7 @@ public class BTCACCT {
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, Mode.TRADING.value));
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
@@ -568,7 +574,7 @@ public class BTCACCT {
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, Mode.REDEEMED.value));
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
@@ -578,7 +584,7 @@ public class BTCACCT {
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, Mode.REFUNDED.value));
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
@@ -591,7 +597,7 @@ public class BTCACCT {
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), BTCACCT.CODE_BYTES_HASH)
assert Arrays.equals(Crypto.digest(codeBytes), BitcoinACCTv1.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
@@ -604,44 +610,37 @@ public class BTCACCT {
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*
* @param repository
* @param atAddress
* @throws DataException
*/
public static CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atStateData);
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*
* @param repository
* @param atAddress
* @throws DataException
*/
public static CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
byte[] creatorPublicKey = repository.getATRepository().getCreatorPublicKey(atStateData.getATAddress());
return populateTradeData(repository, creatorPublicKey, atStateData);
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*
* @param repository
* @param atAddress
* @throws DataException
*/
public static CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, ATStateData atStateData) throws DataException {
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.BITCOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = atStateData.getCreation();
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
@@ -658,9 +657,9 @@ public class BTCACCT {
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Bitcoin/foreign public key hash
tradeData.creatorBitcoinPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorBitcoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorBitcoinPKH.length); // skip to 32 bytes
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// Hash of secret B
tradeData.hashOfSecretB = new byte[20];
@@ -671,7 +670,7 @@ public class BTCACCT {
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected BTC amount
tradeData.expectedBitcoin = dataByteBuffer.getLong();
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
@@ -784,26 +783,28 @@ public class BTCACCT {
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
Mode mode = Mode.valueOf((int) (modeValue & 0xffL));
AcctMode acctMode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != Mode.OFFERING) {
tradeData.mode = mode;
if (acctMode != null && acctMode != AcctMode.OFFERING) {
tradeData.mode = acctMode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerBitcoinPKH = partnerBitcoinPKH;
tradeData.partnerForeignPKH = partnerBitcoinPKH;
tradeData.lockTimeA = lockTimeA;
tradeData.lockTimeB = lockTimeB;
if (mode == Mode.REDEEMED)
if (acctMode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = Mode.OFFERING;
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
@@ -843,7 +844,8 @@ public class BTCACCT {
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
public static byte[] buildCancelMessage(String creatorQortalAddress) {
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
@@ -866,15 +868,17 @@ public class BTCACCT {
/** Returns P2SH-B lockTime (epoch seconds) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcLockTimeB(long offerMessageTimestamp, int lockTimeA) {
// lockTimeB is halfway between offerMessageTimesamp and lockTimeA
// lockTimeB is halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA + (offerMessageTimestamp / 1000L)) / 2L);
}
public static byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
List<MessageTransactionData> messageTransactionsData = repository.getTransactionRepository().getMessagesByRecipient(atAddress, null, null, null);
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;

View File

@@ -0,0 +1,759 @@
package org.qortal.crosschain;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.Address;
import org.bitcoinj.core.AddressFormatException;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.Context;
import org.bitcoinj.core.ECKey;
import org.bitcoinj.core.InsufficientMoneyException;
import org.bitcoinj.core.LegacyAddress;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.core.Sha256Hash;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.core.TransactionOutput;
import org.bitcoinj.core.UTXO;
import org.bitcoinj.core.UTXOProvider;
import org.bitcoinj.core.UTXOProviderException;
import org.bitcoinj.crypto.ChildNumber;
import org.bitcoinj.crypto.DeterministicHierarchy;
import org.bitcoinj.crypto.DeterministicKey;
import org.bitcoinj.script.Script.ScriptType;
import org.bitcoinj.script.ScriptBuilder;
import org.bitcoinj.wallet.DeterministicKeyChain;
import org.bitcoinj.wallet.SendRequest;
import org.bitcoinj.wallet.Wallet;
import org.qortal.api.model.SimpleForeignTransaction;
import org.qortal.crypto.Crypto;
import org.qortal.utils.Amounts;
import org.qortal.utils.BitTwiddling;
import com.google.common.hash.HashCode;
/** Bitcoin-like (Bitcoin, Litecoin, etc.) support */
public abstract class Bitcoiny implements ForeignBlockchain {
protected static final Logger LOGGER = LogManager.getLogger(Bitcoiny.class);
public static final int HASH160_LENGTH = 20;
protected final BitcoinyBlockchainProvider blockchain;
protected final Context bitcoinjContext;
protected final String currencyCode;
protected final NetworkParameters params;
/** Keys that have been previously marked as fully spent,<br>
* i.e. keys with transactions but with no unspent outputs. */
protected final Set<ECKey> spentKeys = Collections.synchronizedSet(new HashSet<>());
/** How many bitcoinj wallet keys to generate in each batch. */
private static final int WALLET_KEY_LOOKAHEAD_INCREMENT = 3;
/** Byte offset into raw block headers to block timestamp. */
private static final int TIMESTAMP_OFFSET = 4 + 32 + 32;
// Constructors and instance
protected Bitcoiny(BitcoinyBlockchainProvider blockchain, Context bitcoinjContext, String currencyCode) {
this.blockchain = blockchain;
this.bitcoinjContext = bitcoinjContext;
this.currencyCode = currencyCode;
this.params = this.bitcoinjContext.getParams();
}
// Getters & setters
public BitcoinyBlockchainProvider getBlockchainProvider() {
return this.blockchain;
}
public Context getBitcoinjContext() {
return this.bitcoinjContext;
}
public String getCurrencyCode() {
return this.currencyCode;
}
public NetworkParameters getNetworkParameters() {
return this.params;
}
// Interface obligations
@Override
public boolean isValidAddress(String address) {
try {
ScriptType addressType = Address.fromString(this.params, address).getOutputScriptType();
return addressType == ScriptType.P2PKH || addressType == ScriptType.P2SH;
} catch (AddressFormatException e) {
return false;
}
}
@Override
public boolean isValidWalletKey(String walletKey) {
return this.isValidDeterministicKey(walletKey);
}
// Actual useful methods for use by other classes
public String format(Coin amount) {
return this.format(amount.value);
}
public String format(long amount) {
return Amounts.prettyAmount(amount) + " " + this.currencyCode;
}
public boolean isValidDeterministicKey(String key58) {
try {
Context.propagate(this.bitcoinjContext);
DeterministicKey.deserializeB58(null, key58, this.params);
return true;
} catch (IllegalArgumentException e) {
return false;
}
}
/** Returns P2PKH address using passed public key hash. */
public String pkhToAddress(byte[] publicKeyHash) {
Context.propagate(this.bitcoinjContext);
return LegacyAddress.fromPubKeyHash(this.params, publicKeyHash).toString();
}
/** Returns P2SH address using passed redeem script. */
public String deriveP2shAddress(byte[] redeemScriptBytes) {
Context.propagate(bitcoinjContext);
byte[] redeemScriptHash = Crypto.hash160(redeemScriptBytes);
return LegacyAddress.fromScriptHash(this.params, redeemScriptHash).toString();
}
/**
* Returns median timestamp from latest 11 blocks, in seconds.
* <p>
* @throws ForeignBlockchainException if error occurs
*/
public int getMedianBlockTime() throws ForeignBlockchainException {
int height = this.blockchain.getCurrentHeight();
// Grab latest 11 blocks
List<byte[]> blockHeaders = this.blockchain.getRawBlockHeaders(height - 11, 11);
if (blockHeaders.size() < 11)
throw new ForeignBlockchainException("Not enough blocks to determine median block time");
List<Integer> blockTimestamps = blockHeaders.stream().map(blockHeader -> BitTwiddling.intFromLEBytes(blockHeader, TIMESTAMP_OFFSET)).collect(Collectors.toList());
// Descending order
blockTimestamps.sort((a, b) -> Integer.compare(b, a));
// Pick median
return blockTimestamps.get(5);
}
/** Returns fee per transaction KB. To be overridden for testnet/regtest. */
public Coin getFeePerKb() {
return this.bitcoinjContext.getFeePerKb();
}
/** Returns minimum order size in sats. To be overridden for coins that need to restrict order size. */
public long getMinimumOrderAmount() {
return 0L;
}
/**
* Returns fixed P2SH spending fee, in sats per 1000bytes, optionally for historic timestamp.
*
* @param timestamp optional milliseconds since epoch, or null for 'now'
* @return sats per 1000bytes
* @throws ForeignBlockchainException if something went wrong
*/
public abstract long getP2shFee(Long timestamp) throws ForeignBlockchainException;
/**
* Returns confirmed balance, based on passed payment script.
* <p>
* @return confirmed balance, or zero if script unknown
* @throws ForeignBlockchainException if there was an error
*/
public long getConfirmedBalance(String base58Address) throws ForeignBlockchainException {
return this.blockchain.getConfirmedBalance(addressToScriptPubKey(base58Address));
}
/**
* Returns list of unspent outputs pertaining to passed address.
* <p>
* @return list of unspent outputs, or empty list if address unknown
* @throws ForeignBlockchainException if there was an error.
*/
// TODO: don't return bitcoinj-based objects like TransactionOutput, use BitcoinyTransaction.Output instead
public List<TransactionOutput> getUnspentOutputs(String base58Address) throws ForeignBlockchainException {
List<UnspentOutput> unspentOutputs = this.blockchain.getUnspentOutputs(addressToScriptPubKey(base58Address), false);
List<TransactionOutput> unspentTransactionOutputs = new ArrayList<>();
for (UnspentOutput unspentOutput : unspentOutputs) {
List<TransactionOutput> transactionOutputs = this.getOutputs(unspentOutput.hash);
unspentTransactionOutputs.add(transactionOutputs.get(unspentOutput.index));
}
return unspentTransactionOutputs;
}
/**
* Returns list of outputs pertaining to passed transaction hash.
* <p>
* @return list of outputs, or empty list if transaction unknown
* @throws ForeignBlockchainException if there was an error.
*/
// TODO: don't return bitcoinj-based objects like TransactionOutput, use BitcoinyTransaction.Output instead
public List<TransactionOutput> getOutputs(byte[] txHash) throws ForeignBlockchainException {
byte[] rawTransactionBytes = this.blockchain.getRawTransaction(txHash);
Context.propagate(bitcoinjContext);
Transaction transaction = new Transaction(this.params, rawTransactionBytes);
return transaction.getOutputs();
}
/**
* Returns list of transaction hashes pertaining to passed address.
* <p>
* @return list of unspent outputs, or empty list if script unknown
* @throws ForeignBlockchainException if there was an error.
*/
public List<TransactionHash> getAddressTransactions(String base58Address, boolean includeUnconfirmed) throws ForeignBlockchainException {
return this.blockchain.getAddressTransactions(addressToScriptPubKey(base58Address), includeUnconfirmed);
}
/**
* Returns list of raw, confirmed transactions involving given address.
* <p>
* @throws ForeignBlockchainException if there was an error
*/
public List<byte[]> getAddressTransactions(String base58Address) throws ForeignBlockchainException {
List<TransactionHash> transactionHashes = this.blockchain.getAddressTransactions(addressToScriptPubKey(base58Address), false);
List<byte[]> rawTransactions = new ArrayList<>();
for (TransactionHash transactionInfo : transactionHashes) {
byte[] rawTransaction = this.blockchain.getRawTransaction(HashCode.fromString(transactionInfo.txHash).asBytes());
rawTransactions.add(rawTransaction);
}
return rawTransactions;
}
/**
* Returns transaction info for passed transaction hash.
* <p>
* @throws ForeignBlockchainException.NotFoundException if transaction unknown
* @throws ForeignBlockchainException if error occurs
*/
public BitcoinyTransaction getTransaction(String txHash) throws ForeignBlockchainException {
return this.blockchain.getTransaction(txHash);
}
/**
* Broadcasts raw transaction to network.
* <p>
* @throws ForeignBlockchainException if error occurs
*/
public void broadcastTransaction(Transaction transaction) throws ForeignBlockchainException {
this.blockchain.broadcastTransaction(transaction.bitcoinSerialize());
}
/**
* Returns bitcoinj transaction sending <tt>amount</tt> to <tt>recipient</tt>.
*
* @param xprv58 BIP32 private key
* @param recipient P2PKH address
* @param amount unscaled amount
* @param feePerByte unscaled fee per byte, or null to use default fees
* @return transaction, or null if insufficient funds
*/
public Transaction buildSpend(String xprv58, String recipient, long amount, Long feePerByte) {
Context.propagate(bitcoinjContext);
Wallet wallet = Wallet.fromSpendingKeyB58(this.params, xprv58, DeterministicHierarchy.BIP32_STANDARDISATION_TIME_SECS);
wallet.setUTXOProvider(new WalletAwareUTXOProvider(this, wallet));
Address destination = Address.fromString(this.params, recipient);
SendRequest sendRequest = SendRequest.to(destination, Coin.valueOf(amount));
if (feePerByte != null)
sendRequest.feePerKb = Coin.valueOf(feePerByte * 1000L); // Note: 1000 not 1024
else
// Allow override of default for TestNet3, etc.
sendRequest.feePerKb = this.getFeePerKb();
try {
wallet.completeTx(sendRequest);
return sendRequest.tx;
} catch (InsufficientMoneyException e) {
return null;
}
}
/**
* Returns bitcoinj transaction sending <tt>amount</tt> to <tt>recipient</tt> using default fees.
*
* @param xprv58 BIP32 private key
* @param recipient P2PKH address
* @param amount unscaled amount
* @return transaction, or null if insufficient funds
*/
public Transaction buildSpend(String xprv58, String recipient, long amount) {
return buildSpend(xprv58, recipient, amount, null);
}
/**
* Returns unspent Bitcoin balance given 'm' BIP32 key.
*
* @param key58 BIP32/HD extended Bitcoin private/public key
* @return unspent BTC balance, or null if unable to determine balance
*/
public Long getWalletBalance(String key58) {
Context.propagate(bitcoinjContext);
Wallet wallet = walletFromDeterministicKey58(key58);
wallet.setUTXOProvider(new WalletAwareUTXOProvider(this, wallet));
Coin balance = wallet.getBalance();
if (balance == null)
return null;
return balance.value;
}
public List<SimpleTransaction> getWalletTransactions(String key58) throws ForeignBlockchainException {
Context.propagate(bitcoinjContext);
Wallet wallet = walletFromDeterministicKey58(key58);
DeterministicKeyChain keyChain = wallet.getActiveKeyChain();
keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
keyChain.maybeLookAhead();
List<DeterministicKey> keys = new ArrayList<>(keyChain.getLeafKeys());
Set<BitcoinyTransaction> walletTransactions = new HashSet<>();
Set<String> keySet = new HashSet<>();
// Set the number of consecutive empty batches required before giving up
final int numberOfAdditionalBatchesToSearch = 5;
int unusedCounter = 0;
int ki = 0;
do {
boolean areAllKeysUnused = true;
for (; ki < keys.size(); ++ki) {
DeterministicKey dKey = keys.get(ki);
// Check for transactions
Address address = Address.fromKey(this.params, dKey, ScriptType.P2PKH);
keySet.add(address.toString());
byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
// Ask for transaction history - if it's empty then key has never been used
List<TransactionHash> historicTransactionHashes = this.blockchain.getAddressTransactions(script, false);
if (!historicTransactionHashes.isEmpty()) {
areAllKeysUnused = false;
for (TransactionHash transactionHash : historicTransactionHashes)
walletTransactions.add(this.getTransaction(transactionHash.txHash));
}
}
if (areAllKeysUnused) {
// No transactions
if (unusedCounter >= numberOfAdditionalBatchesToSearch) {
// ... and we've hit our search limit
break;
}
// We haven't hit our search limit yet so increment the counter and keep looking
unusedCounter++;
}
else {
// Some keys in this batch were used, so reset the counter
unusedCounter = 0;
}
// Generate some more keys
keys.addAll(generateMoreKeys(keyChain));
// Process new keys
} while (true);
Comparator<SimpleTransaction> newestTimestampFirstComparator = Comparator.comparingInt(SimpleTransaction::getTimestamp).reversed();
return walletTransactions.stream().map(t -> convertToSimpleTransaction(t, keySet)).sorted(newestTimestampFirstComparator).collect(Collectors.toList());
}
protected SimpleTransaction convertToSimpleTransaction(BitcoinyTransaction t, Set<String> keySet) {
long amount = 0;
long total = 0L;
for (BitcoinyTransaction.Input input : t.inputs) {
try {
BitcoinyTransaction t2 = getTransaction(input.outputTxHash);
List<String> senders = t2.outputs.get(input.outputVout).addresses;
for (String sender : senders) {
if (keySet.contains(sender)) {
total += t2.outputs.get(input.outputVout).value;
}
}
} catch (ForeignBlockchainException e) {
LOGGER.trace("Failed to retrieve transaction information {}", input.outputTxHash);
}
}
if (t.outputs != null && !t.outputs.isEmpty()) {
for (BitcoinyTransaction.Output output : t.outputs) {
for (String address : output.addresses) {
if (keySet.contains(address)) {
if (total > 0L) {
amount -= (total - output.value);
} else {
amount += output.value;
}
}
}
}
}
return new SimpleTransaction(t.txHash, t.timestamp, amount);
}
/**
* Returns first unused receive address given 'm' BIP32 key.
*
* @param key58 BIP32/HD extended Bitcoin private/public key
* @return P2PKH address
* @throws ForeignBlockchainException if something went wrong
*/
public String getUnusedReceiveAddress(String key58) throws ForeignBlockchainException {
Context.propagate(bitcoinjContext);
Wallet wallet = walletFromDeterministicKey58(key58);
DeterministicKeyChain keyChain = wallet.getActiveKeyChain();
keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
keyChain.maybeLookAhead();
final int keyChainPathSize = keyChain.getAccountPath().size();
List<DeterministicKey> keys = new ArrayList<>(keyChain.getLeafKeys());
int ki = 0;
do {
for (; ki < keys.size(); ++ki) {
DeterministicKey dKey = keys.get(ki);
List<ChildNumber> dKeyPath = dKey.getPath();
// If keyChain is based on 'm', then make sure dKey is m/0/ki - i.e. a 'receive' address, not 'change' (m/1/ki)
if (dKeyPath.size() != keyChainPathSize + 2 || dKeyPath.get(dKeyPath.size() - 2) != ChildNumber.ZERO)
continue;
// Check unspent
Address address = Address.fromKey(this.params, dKey, ScriptType.P2PKH);
byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
List<UnspentOutput> unspentOutputs = this.blockchain.getUnspentOutputs(script, false);
/*
* If there are no unspent outputs then either:
* a) all the outputs have been spent
* b) address has never been used
*
* For case (a) we want to remember not to check this address (key) again.
*/
if (unspentOutputs.isEmpty()) {
// If this is a known key that has been spent before, then we can skip asking for transaction history
if (this.spentKeys.contains(dKey)) {
wallet.getActiveKeyChain().markKeyAsUsed(dKey);
continue;
}
// Ask for transaction history - if it's empty then key has never been used
List<TransactionHash> historicTransactionHashes = this.blockchain.getAddressTransactions(script, false);
if (!historicTransactionHashes.isEmpty()) {
// Fully spent key - case (a)
this.spentKeys.add(dKey);
wallet.getActiveKeyChain().markKeyAsUsed(dKey);
continue;
}
// Key never been used - case (b)
return address.toString();
}
// Key has unspent outputs, hence used, so no good to us
this.spentKeys.remove(dKey);
}
// Generate some more keys
keys.addAll(generateMoreKeys(keyChain));
// Process new keys
} while (true);
}
// UTXOProvider support
static class WalletAwareUTXOProvider implements UTXOProvider {
private final Bitcoiny bitcoiny;
private final Wallet wallet;
private final DeterministicKeyChain keyChain;
public WalletAwareUTXOProvider(Bitcoiny bitcoiny, Wallet wallet) {
this.bitcoiny = bitcoiny;
this.wallet = wallet;
this.keyChain = this.wallet.getActiveKeyChain();
// Set up wallet's key chain
this.keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
this.keyChain.maybeLookAhead();
}
@Override
public List<UTXO> getOpenTransactionOutputs(List<ECKey> keys) throws UTXOProviderException {
List<UTXO> allUnspentOutputs = new ArrayList<>();
final boolean coinbase = false;
int ki = 0;
do {
boolean areAllKeysUnspent = true;
for (; ki < keys.size(); ++ki) {
ECKey key = keys.get(ki);
Address address = Address.fromKey(this.bitcoiny.params, key, ScriptType.P2PKH);
byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
List<UnspentOutput> unspentOutputs;
try {
unspentOutputs = this.bitcoiny.blockchain.getUnspentOutputs(script, false);
} catch (ForeignBlockchainException e) {
throw new UTXOProviderException(String.format("Unable to fetch unspent outputs for %s", address));
}
/*
* If there are no unspent outputs then either:
* a) all the outputs have been spent
* b) address has never been used
*
* For case (a) we want to remember not to check this address (key) again.
*/
if (unspentOutputs.isEmpty()) {
// If this is a known key that has been spent before, then we can skip asking for transaction history
if (this.bitcoiny.spentKeys.contains(key)) {
this.wallet.getActiveKeyChain().markKeyAsUsed((DeterministicKey) key);
areAllKeysUnspent = false;
continue;
}
// Ask for transaction history - if it's empty then key has never been used
List<TransactionHash> historicTransactionHashes;
try {
historicTransactionHashes = this.bitcoiny.blockchain.getAddressTransactions(script, false);
} catch (ForeignBlockchainException e) {
throw new UTXOProviderException(String.format("Unable to fetch transaction history for %s", address));
}
if (!historicTransactionHashes.isEmpty()) {
// Fully spent key - case (a)
this.bitcoiny.spentKeys.add(key);
this.wallet.getActiveKeyChain().markKeyAsUsed((DeterministicKey) key);
areAllKeysUnspent = false;
} else {
// Key never been used - case (b)
}
continue;
}
// If we reach here, then there's definitely at least one unspent key
this.bitcoiny.spentKeys.remove(key);
for (UnspentOutput unspentOutput : unspentOutputs) {
List<TransactionOutput> transactionOutputs;
try {
transactionOutputs = this.bitcoiny.getOutputs(unspentOutput.hash);
} catch (ForeignBlockchainException e) {
throw new UTXOProviderException(String.format("Unable to fetch outputs for TX %s",
HashCode.fromBytes(unspentOutput.hash)));
}
TransactionOutput transactionOutput = transactionOutputs.get(unspentOutput.index);
UTXO utxo = new UTXO(Sha256Hash.wrap(unspentOutput.hash), unspentOutput.index,
Coin.valueOf(unspentOutput.value), unspentOutput.height, coinbase,
transactionOutput.getScriptPubKey());
allUnspentOutputs.add(utxo);
}
}
if (areAllKeysUnspent)
// No transactions for this batch of keys so assume we're done searching.
return allUnspentOutputs;
// Generate some more keys
keys.addAll(Bitcoiny.generateMoreKeys(this.keyChain));
// Process new keys
} while (true);
}
@Override
public int getChainHeadHeight() throws UTXOProviderException {
try {
return this.bitcoiny.blockchain.getCurrentHeight();
} catch (ForeignBlockchainException e) {
throw new UTXOProviderException("Unable to determine Bitcoiny chain height");
}
}
@Override
public NetworkParameters getParams() {
return this.bitcoiny.params;
}
}
// Utility methods for others
public static List<SimpleForeignTransaction> simplifyWalletTransactions(List<BitcoinyTransaction> transactions) {
// Sort by oldest timestamp first
transactions.sort(Comparator.comparingInt(t -> t.timestamp));
// Manual 2nd-level sort same-timestamp transactions so that a transaction's input comes first
int fromIndex = 0;
do {
int timestamp = transactions.get(fromIndex).timestamp;
int toIndex;
for (toIndex = fromIndex + 1; toIndex < transactions.size(); ++toIndex)
if (transactions.get(toIndex).timestamp != timestamp)
break;
// Process same-timestamp sub-list
List<BitcoinyTransaction> subList = transactions.subList(fromIndex, toIndex);
// Only if necessary
if (subList.size() > 1) {
// Quick index lookup
Map<String, Integer> indexByTxHash = subList.stream().collect(Collectors.toMap(t -> t.txHash, t -> t.timestamp));
int restartIndex = 0;
boolean isSorted;
do {
isSorted = true;
for (int ourIndex = restartIndex; ourIndex < subList.size(); ++ourIndex) {
BitcoinyTransaction ourTx = subList.get(ourIndex);
for (BitcoinyTransaction.Input input : ourTx.inputs) {
Integer inputIndex = indexByTxHash.get(input.outputTxHash);
if (inputIndex != null && inputIndex > ourIndex) {
// Input tx is currently after current tx, so swap
BitcoinyTransaction tmpTx = subList.get(inputIndex);
subList.set(inputIndex, ourTx);
subList.set(ourIndex, tmpTx);
// Update index lookup too
indexByTxHash.put(ourTx.txHash, inputIndex);
indexByTxHash.put(tmpTx.txHash, ourIndex);
if (isSorted)
restartIndex = Math.max(restartIndex, ourIndex);
isSorted = false;
break;
}
}
}
} while (!isSorted);
}
fromIndex = toIndex;
} while (fromIndex < transactions.size());
// Simplify
List<SimpleForeignTransaction> simpleTransactions = new ArrayList<>();
// Quick lookup of txs in our wallet
Set<String> walletTxHashes = transactions.stream().map(t -> t.txHash).collect(Collectors.toSet());
for (BitcoinyTransaction transaction : transactions) {
SimpleForeignTransaction.Builder builder = new SimpleForeignTransaction.Builder();
builder.txHash(transaction.txHash);
builder.timestamp(transaction.timestamp);
builder.isSentNotReceived(false);
for (BitcoinyTransaction.Input input : transaction.inputs) {
// TODO: add input via builder
if (walletTxHashes.contains(input.outputTxHash))
builder.isSentNotReceived(true);
}
for (BitcoinyTransaction.Output output : transaction.outputs)
builder.output(output.addresses, output.value);
simpleTransactions.add(builder.build());
}
return simpleTransactions;
}
// Utility methods for us
protected static List<DeterministicKey> generateMoreKeys(DeterministicKeyChain keyChain) {
int existingLeafKeyCount = keyChain.getLeafKeys().size();
// Increase lookahead size...
keyChain.setLookaheadSize(keyChain.getLookaheadSize() + Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
// ...and lookahead threshold (minimum number of keys to generate)...
keyChain.setLookaheadThreshold(0);
// ...so that this call will generate more keys
keyChain.maybeLookAhead();
// This returns *all* keys
List<DeterministicKey> allLeafKeys = keyChain.getLeafKeys();
// Only return newly generated keys
return allLeafKeys.subList(existingLeafKeyCount, allLeafKeys.size());
}
protected byte[] addressToScriptPubKey(String base58Address) {
Context.propagate(this.bitcoinjContext);
Address address = Address.fromString(this.params, base58Address);
return ScriptBuilder.createOutputScript(address).getProgram();
}
protected Wallet walletFromDeterministicKey58(String key58) {
DeterministicKey dKey = DeterministicKey.deserializeB58(null, key58, this.params);
if (dKey.hasPrivKey())
return Wallet.fromSpendingKeyB58(this.params, key58, DeterministicHierarchy.BIP32_STANDARDISATION_TIME_SECS);
else
return Wallet.fromWatchingKeyB58(this.params, key58, DeterministicHierarchy.BIP32_STANDARDISATION_TIME_SECS);
}
}

View File

@@ -0,0 +1,40 @@
package org.qortal.crosschain;
import java.util.List;
public abstract class BitcoinyBlockchainProvider {
public static final boolean INCLUDE_UNCONFIRMED = true;
public static final boolean EXCLUDE_UNCONFIRMED = false;
/** Returns ID unique to bitcoiny network (e.g. "Litecoin-TEST3") */
public abstract String getNetId();
/** Returns current blockchain height. */
public abstract int getCurrentHeight() throws ForeignBlockchainException;
/** Returns a list of raw block headers, starting at <tt>startHeight</tt> (inclusive), up to <tt>count</tt> max. */
public abstract List<byte[]> getRawBlockHeaders(int startHeight, int count) throws ForeignBlockchainException;
/** Returns balance of address represented by <tt>scriptPubKey</tt>. */
public abstract long getConfirmedBalance(byte[] scriptPubKey) throws ForeignBlockchainException;
/** Returns raw, serialized, transaction bytes given <tt>txHash</tt>. */
public abstract byte[] getRawTransaction(String txHash) throws ForeignBlockchainException;
/** Returns raw, serialized, transaction bytes given <tt>txHash</tt>. */
public abstract byte[] getRawTransaction(byte[] txHash) throws ForeignBlockchainException;
/** Returns unpacked transaction given <tt>txHash</tt>. */
public abstract BitcoinyTransaction getTransaction(String txHash) throws ForeignBlockchainException;
/** Returns list of transaction hashes (and heights) for address represented by <tt>scriptPubKey</tt>, optionally including unconfirmed transactions. */
public abstract List<TransactionHash> getAddressTransactions(byte[] scriptPubKey, boolean includeUnconfirmed) throws ForeignBlockchainException;
/** Returns list of unspent transaction outputs for address represented by <tt>scriptPubKey</tt>, optionally including unconfirmed transactions. */
public abstract List<UnspentOutput> getUnspentOutputs(byte[] scriptPubKey, boolean includeUnconfirmed) throws ForeignBlockchainException;
/** Broadcasts raw, serialized, transaction bytes to network, returning success/failure. */
public abstract void broadcastTransaction(byte[] rawTransaction) throws ForeignBlockchainException;
}

View File

@@ -0,0 +1,438 @@
package org.qortal.crosschain;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import org.bitcoinj.core.Address;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.ECKey;
import org.bitcoinj.core.LegacyAddress;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.core.Transaction.SigHash;
import org.bitcoinj.core.TransactionInput;
import org.bitcoinj.core.TransactionOutput;
import org.bitcoinj.crypto.TransactionSignature;
import org.bitcoinj.script.Script;
import org.bitcoinj.script.ScriptBuilder;
import org.bitcoinj.script.ScriptChunk;
import org.bitcoinj.script.ScriptOpCodes;
import org.qortal.crypto.Crypto;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
public class BitcoinyHTLC {
public enum Status {
UNFUNDED, FUNDING_IN_PROGRESS, FUNDED, REDEEM_IN_PROGRESS, REDEEMED, REFUND_IN_PROGRESS, REFUNDED
}
public static final int SECRET_LENGTH = 32;
public static final int MIN_LOCKTIME = 1500000000;
public static final long NO_LOCKTIME_NO_RBF_SEQUENCE = 0xFFFFFFFFL;
public static final long LOCKTIME_NO_RBF_SEQUENCE = NO_LOCKTIME_NO_RBF_SEQUENCE - 1;
// Assuming node's trade-bot has no more than 100 entries?
private static final int MAX_CACHE_ENTRIES = 100;
// Max time-to-live for cache entries (milliseconds)
private static final long CACHE_TIMEOUT = 30_000L;
@SuppressWarnings("serial")
private static final Map<String, byte[]> SECRET_CACHE = new LinkedHashMap<>(MAX_CACHE_ENTRIES + 1, 0.75F, true) {
// This method is called just after a new entry has been added
@Override
public boolean removeEldestEntry(Map.Entry<String, byte[]> eldest) {
return size() > MAX_CACHE_ENTRIES;
}
};
private static final byte[] NO_SECRET_CACHE_ENTRY = new byte[0];
@SuppressWarnings("serial")
private static final Map<String, Status> STATUS_CACHE = new LinkedHashMap<>(MAX_CACHE_ENTRIES + 1, 0.75F, true) {
// This method is called just after a new entry has been added
@Override
public boolean removeEldestEntry(Map.Entry<String, Status> eldest) {
return size() > MAX_CACHE_ENTRIES;
}
};
/*
* OP_TUCK (to copy public key to before signature)
* OP_CHECKSIGVERIFY (sig & pubkey must verify or script fails)
* OP_HASH160 (convert public key to PKH)
* OP_DUP (duplicate PKH)
* <push 20 bytes> <refund PKH> OP_EQUAL (does PKH match refund PKH?)
* OP_IF
* OP_DROP (no need for duplicate PKH)
* <push 4 bytes> <locktime>
* OP_CHECKLOCKTIMEVERIFY (if this passes, leftover stack is <locktime> so script passes)
* OP_ELSE
* <push 20 bytes> <redeem PKH> OP_EQUALVERIFY (duplicate PKH must match redeem PKH or script fails)
* OP_HASH160 (hash secret)
* <push 20 bytes> <hash of secret> OP_EQUAL (do hashes of secrets match? if true, script passes else script fails)
* OP_ENDIF
*/
private static final byte[] redeemScript1 = HashCode.fromString("7dada97614").asBytes(); // OP_TUCK OP_CHECKSIGVERIFY OP_HASH160 OP_DUP push(0x14 bytes)
private static final byte[] redeemScript2 = HashCode.fromString("87637504").asBytes(); // OP_EQUAL OP_IF OP_DROP push(0x4 bytes)
private static final byte[] redeemScript3 = HashCode.fromString("b16714").asBytes(); // OP_CHECKLOCKTIMEVERIFY OP_ELSE push(0x14 bytes)
private static final byte[] redeemScript4 = HashCode.fromString("88a914").asBytes(); // OP_EQUALVERIFY OP_HASH160 push(0x14 bytes)
private static final byte[] redeemScript5 = HashCode.fromString("8768").asBytes(); // OP_EQUAL OP_ENDIF
/**
* Returns redeemScript used for cross-chain trading.
* <p>
* See comments in {@link BitcoinyHTLC} for more details.
*
* @param refunderPubKeyHash 20-byte HASH160 of P2SH funder's public key, for refunding purposes
* @param lockTime seconds-since-epoch threshold, after which P2SH funder can claim refund
* @param redeemerPubKeyHash 20-byte HASH160 of P2SH redeemer's public key
* @param hashOfSecret 20-byte HASH160 of secret, used by P2SH redeemer to claim funds
*/
public static byte[] buildScript(byte[] refunderPubKeyHash, int lockTime, byte[] redeemerPubKeyHash, byte[] hashOfSecret) {
return Bytes.concat(redeemScript1, refunderPubKeyHash, redeemScript2, BitTwiddling.toLEByteArray((int) (lockTime & 0xffffffffL)),
redeemScript3, redeemerPubKeyHash, redeemScript4, hashOfSecret, redeemScript5);
}
/**
* Builds a custom transaction to spend HTLC P2SH.
*
* @param params blockchain network parameters
* @param amount output amount, should be total of input amounts, less miner fees
* @param spendKey key for signing transaction, and also where funds are 'sent' (output)
* @param fundingOutput output from transaction that funded P2SH address
* @param redeemScriptBytes the redeemScript itself, in byte[] form
* @param lockTime (optional) transaction nLockTime, used in refund scenario
* @param scriptSigBuilder function for building scriptSig using transaction input signature
* @param outputPublicKeyHash PKH used to create P2PKH output
* @return Signed transaction for spending P2SH
*/
public static Transaction buildP2shTransaction(NetworkParameters params, Coin amount, ECKey spendKey,
List<TransactionOutput> fundingOutputs, byte[] redeemScriptBytes,
Long lockTime, Function<byte[], Script> scriptSigBuilder, byte[] outputPublicKeyHash) {
Transaction transaction = new Transaction(params);
transaction.setVersion(2);
// Output is back to P2SH funder
transaction.addOutput(amount, ScriptBuilder.createP2PKHOutputScript(outputPublicKeyHash));
for (int inputIndex = 0; inputIndex < fundingOutputs.size(); ++inputIndex) {
TransactionOutput fundingOutput = fundingOutputs.get(inputIndex);
// Input (without scriptSig prior to signing)
TransactionInput input = new TransactionInput(params, null, redeemScriptBytes, fundingOutput.getOutPointFor());
if (lockTime != null)
input.setSequenceNumber(LOCKTIME_NO_RBF_SEQUENCE); // Use max-value - 1, so lockTime can be used but not RBF
else
input.setSequenceNumber(NO_LOCKTIME_NO_RBF_SEQUENCE); // Use max-value, so no lockTime and no RBF
transaction.addInput(input);
}
// Set locktime after inputs added but before input signatures are generated
if (lockTime != null)
transaction.setLockTime(lockTime);
for (int inputIndex = 0; inputIndex < fundingOutputs.size(); ++inputIndex) {
// Generate transaction signature for input
final boolean anyoneCanPay = false;
TransactionSignature txSig = transaction.calculateSignature(inputIndex, spendKey, redeemScriptBytes, SigHash.ALL, anyoneCanPay);
// Calculate transaction signature
byte[] txSigBytes = txSig.encodeToBitcoin();
// Build scriptSig using lambda and tx signature
Script scriptSig = scriptSigBuilder.apply(txSigBytes);
// Set input scriptSig
transaction.getInput(inputIndex).setScriptSig(scriptSig);
}
return transaction;
}
/**
* Returns signed transaction claiming refund from HTLC P2SH.
*
* @param params blockchain network parameters
* @param refundAmount refund amount, should be total of input amounts, less miner fees
* @param refundKey key for signing transaction
* @param fundingOutputs outputs from transaction that funded P2SH address
* @param redeemScriptBytes the redeemScript itself, in byte[] form
* @param lockTime transaction nLockTime - must be at least locktime used in redeemScript
* @param receivingAccountInfo public-key-hash used for P2PKH output
* @return Signed transaction for refunding P2SH
*/
public static Transaction buildRefundTransaction(NetworkParameters params, Coin refundAmount, ECKey refundKey,
List<TransactionOutput> fundingOutputs, byte[] redeemScriptBytes, long lockTime, byte[] receivingAccountInfo) {
Function<byte[], Script> refundSigScriptBuilder = (txSigBytes) -> {
// Build scriptSig with...
ScriptBuilder scriptBuilder = new ScriptBuilder();
// transaction signature
scriptBuilder.addChunk(new ScriptChunk(txSigBytes.length, txSigBytes));
// redeem public key
byte[] refundPubKey = refundKey.getPubKey();
scriptBuilder.addChunk(new ScriptChunk(refundPubKey.length, refundPubKey));
// redeem script
scriptBuilder.addChunk(new ScriptChunk(ScriptOpCodes.OP_PUSHDATA1, redeemScriptBytes));
return scriptBuilder.build();
};
// Send funds back to funding address
return buildP2shTransaction(params, refundAmount, refundKey, fundingOutputs, redeemScriptBytes, lockTime, refundSigScriptBuilder, receivingAccountInfo);
}
/**
* Returns signed transaction redeeming funds from P2SH address.
*
* @param params blockchain network parameters
* @param redeemAmount redeem amount, should be total of input amounts, less miner fees
* @param redeemKey key for signing transaction
* @param fundingOutputs outputs from transaction that funded P2SH address
* @param redeemScriptBytes the redeemScript itself, in byte[] form
* @param secret actual 32-byte secret used when building redeemScript
* @param receivingAccountInfo Bitcoin PKH used for output
* @return Signed transaction for redeeming P2SH
*/
public static Transaction buildRedeemTransaction(NetworkParameters params, Coin redeemAmount, ECKey redeemKey,
List<TransactionOutput> fundingOutputs, byte[] redeemScriptBytes, byte[] secret, byte[] receivingAccountInfo) {
Function<byte[], Script> redeemSigScriptBuilder = (txSigBytes) -> {
// Build scriptSig with...
ScriptBuilder scriptBuilder = new ScriptBuilder();
// secret
scriptBuilder.addChunk(new ScriptChunk(secret.length, secret));
// transaction signature
scriptBuilder.addChunk(new ScriptChunk(txSigBytes.length, txSigBytes));
// redeem public key
byte[] redeemPubKey = redeemKey.getPubKey();
scriptBuilder.addChunk(new ScriptChunk(redeemPubKey.length, redeemPubKey));
// redeem script
scriptBuilder.addChunk(new ScriptChunk(ScriptOpCodes.OP_PUSHDATA1, redeemScriptBytes));
return scriptBuilder.build();
};
return buildP2shTransaction(params, redeemAmount, redeemKey, fundingOutputs, redeemScriptBytes, null, redeemSigScriptBuilder, receivingAccountInfo);
}
/**
* Returns 'secret', if any, given HTLC's P2SH address.
* <p>
* @throws ForeignBlockchainException
*/
public static byte[] findHtlcSecret(Bitcoiny bitcoiny, String p2shAddress) throws ForeignBlockchainException {
NetworkParameters params = bitcoiny.getNetworkParameters();
String compoundKey = String.format("%s-%s-%d", params.getId(), p2shAddress, System.currentTimeMillis() / CACHE_TIMEOUT);
byte[] secret = SECRET_CACHE.getOrDefault(compoundKey, NO_SECRET_CACHE_ENTRY);
if (secret != NO_SECRET_CACHE_ENTRY)
return secret;
List<byte[]> rawTransactions = bitcoiny.getAddressTransactions(p2shAddress);
for (byte[] rawTransaction : rawTransactions) {
Transaction transaction = new Transaction(params, rawTransaction);
// Cycle through inputs, looking for one that spends our HTLC
for (TransactionInput input : transaction.getInputs()) {
Script scriptSig = input.getScriptSig();
List<ScriptChunk> scriptChunks = scriptSig.getChunks();
// Expected number of script chunks for redeem. Refund might not have the same number.
int expectedChunkCount = 1 /*secret*/ + 1 /*sig*/ + 1 /*pubkey*/ + 1 /*redeemScript*/;
if (scriptChunks.size() != expectedChunkCount)
continue;
// We're expecting last chunk to contain the actual redeemScript
ScriptChunk lastChunk = scriptChunks.get(scriptChunks.size() - 1);
byte[] redeemScriptBytes = lastChunk.data;
// If non-push scripts, redeemScript will be null
if (redeemScriptBytes == null)
continue;
byte[] redeemScriptHash = Crypto.hash160(redeemScriptBytes);
Address inputAddress = LegacyAddress.fromScriptHash(params, redeemScriptHash);
if (!inputAddress.toString().equals(p2shAddress))
// Input isn't spending our HTLC
continue;
secret = scriptChunks.get(0).data;
if (secret.length != BitcoinyHTLC.SECRET_LENGTH)
continue;
// Cache secret for a while
SECRET_CACHE.put(compoundKey, secret);
return secret;
}
}
// Cache negative result
SECRET_CACHE.put(compoundKey, null);
return null;
}
/**
* Returns HTLC status, given P2SH address and expected redeem/refund amount
* <p>
* @throws ForeignBlockchainException if error occurs
*/
public static Status determineHtlcStatus(BitcoinyBlockchainProvider blockchain, String p2shAddress, long minimumAmount) throws ForeignBlockchainException {
String compoundKey = String.format("%s-%s-%d", blockchain.getNetId(), p2shAddress, System.currentTimeMillis() / CACHE_TIMEOUT);
Status cachedStatus = STATUS_CACHE.getOrDefault(compoundKey, null);
if (cachedStatus != null)
return cachedStatus;
byte[] ourScriptPubKey = addressToScriptPubKey(p2shAddress);
List<TransactionHash> transactionHashes = blockchain.getAddressTransactions(ourScriptPubKey, BitcoinyBlockchainProvider.INCLUDE_UNCONFIRMED);
// Sort by confirmed first, followed by ascending height
transactionHashes.sort(TransactionHash.CONFIRMED_FIRST.thenComparing(TransactionHash::getHeight));
// Transaction cache
Map<String, BitcoinyTransaction> transactionsByHash = new HashMap<>();
// HASH160(redeem script) for this p2shAddress
byte[] ourRedeemScriptHash = addressToRedeemScriptHash(p2shAddress);
// Check for spends first, caching full transaction info as we progress just in case we don't return in this loop
for (TransactionHash transactionInfo : transactionHashes) {
BitcoinyTransaction bitcoinyTransaction = blockchain.getTransaction(transactionInfo.txHash);
// Cache for possible later reuse
transactionsByHash.put(transactionInfo.txHash, bitcoinyTransaction);
// Acceptable funding is one transaction output, so we're expecting only one input
if (bitcoinyTransaction.inputs.size() != 1)
// Wrong number of inputs
continue;
String scriptSig = bitcoinyTransaction.inputs.get(0).scriptSig;
List<byte[]> scriptSigChunks = extractScriptSigChunks(HashCode.fromString(scriptSig).asBytes());
if (scriptSigChunks.size() < 3 || scriptSigChunks.size() > 4)
// Not valid chunks for our form of HTLC
continue;
// Last chunk is redeem script
byte[] redeemScriptBytes = scriptSigChunks.get(scriptSigChunks.size() - 1);
byte[] redeemScriptHash = Crypto.hash160(redeemScriptBytes);
if (!Arrays.equals(redeemScriptHash, ourRedeemScriptHash))
// Not spending our specific HTLC redeem script
continue;
if (scriptSigChunks.size() == 4)
// If we have 4 chunks, then secret is present, hence redeem
cachedStatus = transactionInfo.height == 0 ? Status.REDEEM_IN_PROGRESS : Status.REDEEMED;
else
cachedStatus = transactionInfo.height == 0 ? Status.REFUND_IN_PROGRESS : Status.REFUNDED;
STATUS_CACHE.put(compoundKey, cachedStatus);
return cachedStatus;
}
String ourScriptPubKeyHex = HashCode.fromBytes(ourScriptPubKey).toString();
// Check for funding
for (TransactionHash transactionInfo : transactionHashes) {
BitcoinyTransaction bitcoinyTransaction = transactionsByHash.get(transactionInfo.txHash);
if (bitcoinyTransaction == null)
// Should be present in map!
throw new ForeignBlockchainException("Cached Bitcoin transaction now missing?");
// Check outputs for our specific P2SH
for (BitcoinyTransaction.Output output : bitcoinyTransaction.outputs) {
// Check amount
if (output.value < minimumAmount)
// Output amount too small (not taking fees into account)
continue;
String scriptPubKeyHex = output.scriptPubKey;
if (!scriptPubKeyHex.equals(ourScriptPubKeyHex))
// Not funding our specific P2SH
continue;
cachedStatus = transactionInfo.height == 0 ? Status.FUNDING_IN_PROGRESS : Status.FUNDED;
STATUS_CACHE.put(compoundKey, cachedStatus);
return cachedStatus;
}
}
cachedStatus = Status.UNFUNDED;
STATUS_CACHE.put(compoundKey, cachedStatus);
return cachedStatus;
}
private static List<byte[]> extractScriptSigChunks(byte[] scriptSigBytes) {
List<byte[]> chunks = new ArrayList<>();
int offset = 0;
int previousOffset = 0;
while (offset < scriptSigBytes.length) {
byte pushOp = scriptSigBytes[offset++];
if (pushOp < 0 || pushOp > 0x4c)
// Unacceptable OP
return Collections.emptyList();
// Special treatment for OP_PUSHDATA1
if (pushOp == 0x4c) {
if (offset >= scriptSigBytes.length)
// Run out of scriptSig bytes?
return Collections.emptyList();
pushOp = scriptSigBytes[offset++];
}
previousOffset = offset;
offset += Byte.toUnsignedInt(pushOp);
byte[] chunk = Arrays.copyOfRange(scriptSigBytes, previousOffset, offset);
chunks.add(chunk);
}
return chunks;
}
private static byte[] addressToScriptPubKey(String p2shAddress) {
// We want the HASH160 part of the P2SH address
byte[] p2shAddressBytes = Base58.decode(p2shAddress);
byte[] scriptPubKey = new byte[1 + 1 + 20 + 1];
scriptPubKey[0x00] = (byte) 0xa9; /* OP_HASH160 */
scriptPubKey[0x01] = (byte) 0x14; /* PUSH 0x14 bytes */
System.arraycopy(p2shAddressBytes, 1, scriptPubKey, 2, 0x14);
scriptPubKey[0x16] = (byte) 0x87; /* OP_EQUAL */
return scriptPubKey;
}
private static byte[] addressToRedeemScriptHash(String p2shAddress) {
// We want the HASH160 part of the P2SH address
byte[] p2shAddressBytes = Base58.decode(p2shAddress);
return Arrays.copyOfRange(p2shAddressBytes, 1, 1 + 20);
}
}

View File

@@ -0,0 +1,146 @@
package org.qortal.crosschain;
import java.util.List;
import java.util.stream.Collectors;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlTransient;
@XmlAccessorType(XmlAccessType.FIELD)
public class BitcoinyTransaction {
public final String txHash;
@XmlTransient
public final int size;
@XmlTransient
public final int locktime;
// Not present if transaction is unconfirmed
public final Integer timestamp;
public static class Input {
@XmlTransient
public final String scriptSig;
@XmlTransient
public final int sequence;
public final String outputTxHash;
public final int outputVout;
// For JAXB
protected Input() {
this.scriptSig = null;
this.sequence = 0;
this.outputTxHash = null;
this.outputVout = 0;
}
public Input(String scriptSig, int sequence, String outputTxHash, int outputVout) {
this.scriptSig = scriptSig;
this.sequence = sequence;
this.outputTxHash = outputTxHash;
this.outputVout = outputVout;
}
public String toString() {
return String.format("{output %s:%d, sequence %d, scriptSig %s}",
this.outputTxHash, this.outputVout, this.sequence, this.scriptSig);
}
}
@XmlTransient
public final List<Input> inputs;
public static class Output {
@XmlTransient
public final String scriptPubKey;
public final long value;
public final List<String> addresses;
// For JAXB
protected Output() {
this.scriptPubKey = null;
this.value = 0;
this.addresses = null;
}
public Output(String scriptPubKey, long value) {
this.scriptPubKey = scriptPubKey;
this.value = value;
this.addresses = null;
}
public Output(String scriptPubKey, long value, List<String> addresses) {
this.scriptPubKey = scriptPubKey;
this.value = value;
this.addresses = addresses;
}
public String toString() {
return String.format("{value %d, scriptPubKey %s}", this.value, this.scriptPubKey);
}
}
public final List<Output> outputs;
public final long totalAmount;
// For JAXB
protected BitcoinyTransaction() {
this.txHash = null;
this.size = 0;
this.locktime = 0;
this.timestamp = 0;
this.inputs = null;
this.outputs = null;
this.totalAmount = 0;
}
public BitcoinyTransaction(String txHash, int size, int locktime, Integer timestamp,
List<Input> inputs, List<Output> outputs) {
this.txHash = txHash;
this.size = size;
this.locktime = locktime;
this.timestamp = timestamp;
this.inputs = inputs;
this.outputs = outputs;
this.totalAmount = outputs.stream().map(output -> output.value).reduce(0L, Long::sum);
}
public String toString() {
return String.format("txHash %s, size %d, locktime %d, timestamp %d\n"
+ "\tinputs: [%s]\n"
+ "\toutputs: [%s]\n",
this.txHash,
this.size,
this.locktime,
this.timestamp,
this.inputs.stream().map(Input::toString).collect(Collectors.joining(",\n\t\t")),
this.outputs.stream().map(Output::toString).collect(Collectors.joining(",\n\t\t")));
}
@Override
public boolean equals(Object other) {
if (other == this)
return true;
if (!(other instanceof BitcoinyTransaction))
return false;
BitcoinyTransaction otherTransaction = (BitcoinyTransaction) other;
return this.txHash.equals(otherTransaction.txHash);
}
@Override
public int hashCode() {
return this.txHash.hashCode();
}
}

View File

@@ -0,0 +1,171 @@
package org.qortal.crosschain;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.Context;
import org.bitcoinj.core.NetworkParameters;
import org.libdohj.params.DogecoinMainNetParams;
//import org.libdohj.params.DogecoinRegTestParams;
import org.libdohj.params.DogecoinTestNet3Params;
import org.qortal.crosschain.ElectrumX.Server;
import org.qortal.crosschain.ElectrumX.Server.ConnectionType;
import org.qortal.settings.Settings;
import java.util.Arrays;
import java.util.Collection;
import java.util.EnumMap;
import java.util.Map;
public class Dogecoin extends Bitcoiny {
public static final String CURRENCY_CODE = "DOGE";
private static final Coin DEFAULT_FEE_PER_KB = Coin.valueOf(500000000); // 5 DOGE per 1000 bytes
private static final long MINIMUM_ORDER_AMOUNT = 300000000L; // 3 DOGE minimum order. The RPC dust threshold is around 2 DOGE
// Temporary values until a dynamic fee system is written.
private static final long MAINNET_FEE = 110000000L;
private static final long NON_MAINNET_FEE = 10000L; // TODO: calibrate this
private static final Map<ConnectionType, Integer> DEFAULT_ELECTRUMX_PORTS = new EnumMap<>(ConnectionType.class);
static {
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.TCP, 50001);
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.SSL, 50002);
}
public enum DogecoinNet {
MAIN {
@Override
public NetworkParameters getParams() {
return DogecoinMainNetParams.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("electrum1.cipig.net", ConnectionType.TCP, 10060),
new Server("electrum2.cipig.net", ConnectionType.TCP, 10060),
new Server("electrum3.cipig.net", ConnectionType.TCP, 10060));
// TODO: add more mainnet servers. It's too centralized.
}
@Override
public String getGenesisHash() {
return "1a91e3dace36e2be3bf030a65679fe821aa1d6ef92e7c9902eb318182c355691";
}
@Override
public long getP2shFee(Long timestamp) {
// TODO: This will need to be replaced with something better in the near future!
return MAINNET_FEE;
}
},
TEST3 {
@Override
public NetworkParameters getParams() {
return DogecoinTestNet3Params.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(); // TODO: find testnet servers
}
@Override
public String getGenesisHash() {
return "4966625a4b2851d9fdee139e56211a0d88575f59ed816ff5e6a63deb4e3e29a0";
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
},
REGTEST {
@Override
public NetworkParameters getParams() {
return null; // TODO: DogecoinRegTestParams.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(
new Server("localhost", ConnectionType.TCP, 50001),
new Server("localhost", ConnectionType.SSL, 50002));
}
@Override
public String getGenesisHash() {
// This is unique to each regtest instance
return null;
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
};
public abstract NetworkParameters getParams();
public abstract Collection<Server> getServers();
public abstract String getGenesisHash();
public abstract long getP2shFee(Long timestamp) throws ForeignBlockchainException;
}
private static Dogecoin instance;
private final DogecoinNet dogecoinNet;
// Constructors and instance
private Dogecoin(DogecoinNet dogecoinNet, BitcoinyBlockchainProvider blockchain, Context bitcoinjContext, String currencyCode) {
super(blockchain, bitcoinjContext, currencyCode);
this.dogecoinNet = dogecoinNet;
LOGGER.info(() -> String.format("Starting Dogecoin support using %s", this.dogecoinNet.name()));
}
public static synchronized Dogecoin getInstance() {
if (instance == null) {
DogecoinNet dogecoinNet = Settings.getInstance().getDogecoinNet();
BitcoinyBlockchainProvider electrumX = new ElectrumX("Dogecoin-" + dogecoinNet.name(), dogecoinNet.getGenesisHash(), dogecoinNet.getServers(), DEFAULT_ELECTRUMX_PORTS);
Context bitcoinjContext = new Context(dogecoinNet.getParams());
instance = new Dogecoin(dogecoinNet, electrumX, bitcoinjContext, CURRENCY_CODE);
}
return instance;
}
// Getters & setters
public static synchronized void resetForTesting() {
instance = null;
}
// Actual useful methods for use by other classes
@Override
public Coin getFeePerKb() {
return DEFAULT_FEE_PER_KB;
}
@Override
public long getMinimumOrderAmount() {
return MINIMUM_ORDER_AMOUNT;
}
/**
* Returns estimated DOGE fee, in sats per 1000bytes, optionally for historic timestamp.
*
* @param timestamp optional milliseconds since epoch, or null for 'now'
* @return sats per 1000bytes, or throws ForeignBlockchainException if something went wrong
*/
@Override
public long getP2shFee(Long timestamp) throws ForeignBlockchainException {
return this.dogecoinNet.getP2shFee(timestamp);
}
}

View File

@@ -0,0 +1,855 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Dogecoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Dogecoin & Qortal 'trade' keys</li>
* <li>Alice funds Dogecoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Dogecoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Dogecoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Dogecoin trade key and secret-A</li>
* <li>P2SH-A DOGE funds end up at Dogecoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class DogecoinACCTv1 implements ACCT {
private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv1.class);
public static final String NAME = DogecoinACCTv1.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("0eb49b0313ff3855a29d860c2a8203faa2ef62e28ea30459321f176079cfa3a5").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerDogecoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerDogecoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Dogecoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static DogecoinACCTv1 instance;
private DogecoinACCTv1() {
}
public static synchronized DogecoinACCTv1 getInstance() {
if (instance == null)
instance = new DogecoinACCTv1();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Dogecoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param dogecoinPublicKeyHash 20-byte HASH160 of creator's trade Dogecoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param dogecoinAmount how much DOGE the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] dogecoinPublicKeyHash, long qortAmount, long dogecoinAmount, int tradeTimeout) {
if (dogecoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Dogecoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrDogecoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrDogecoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerDogecoinPKHOffset = addrCounter++;
final int addrPartnerDogecoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerDogecoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Dogecoin public key hash
assert dataByteBuffer.position() == addrDogecoinPublicKeyHash * MachineState.VALUE_SIZE : "addrDogecoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(dogecoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Dogecoin amount
assert dataByteBuffer.position() == addrDogecoinAmount * MachineState.VALUE_SIZE : "addrDogecoinAmount incorrect";
dataByteBuffer.putLong(dogecoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Dogecoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerDogecoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerDogecoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Dogecoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerDogecoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerDogecoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerDogecoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
/* NOP - to ensure DOGECOIN ACCT is unique */
codeByteBuffer.put(OpCode.NOP.compile());
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Dogecoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerDogecoinPKHOffset));
// Store partner's Dogecoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerDogecoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile DOGE-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), DogecoinACCTv1.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.DOGECOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Dogecoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected DOGE amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Dogecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Dogecoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Dogecoin PKH
byte[] partnerDogecoinPKH = new byte[20];
dataByteBuffer.get(partnerDogecoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerDogecoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerDogecoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerDogecoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -1,21 +1,26 @@
package org.qortal.crosschain;
import java.io.IOException;
import java.math.BigDecimal;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Collection;
import java.util.Collections;
import java.util.EnumMap;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.NoSuchElementException;
import java.util.Random;
import java.util.Scanner;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;
import org.apache.logging.log4j.LogManager;
@@ -28,24 +33,27 @@ import org.qortal.crypto.TrustlessSSLSocketFactory;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.qortal.utils.BitTwiddling;
/** ElectrumX network support for querying Bitcoin-related info like block headers, transaction outputs, etc. */
public class ElectrumX {
/** ElectrumX network support for querying Bitcoiny-related info like block headers, transaction outputs, etc. */
public class ElectrumX extends BitcoinyBlockchainProvider {
private static final Logger LOGGER = LogManager.getLogger(ElectrumX.class);
private static final Random RANDOM = new Random();
private static final int DEFAULT_TCP_PORT = 50001;
private static final int DEFAULT_SSL_PORT = 50002;
private static final double MIN_PROTOCOL_VERSION = 1.2;
private static final int BLOCK_HEADER_LENGTH = 80;
private static final Map<String, ElectrumX> instances = new HashMap<>();
// "message": "daemon error: DaemonError({'code': -5, 'message': 'No such mempool or blockchain transaction. Use gettransaction for wallet transactions.'})"
private static final Pattern DAEMON_ERROR_REGEX = Pattern.compile("DaemonError\\(\\{.*'code': ?(-?[0-9]+).*\\}\\)\\z"); // Capture 'code' inside curly-brace content
static class Server {
/** Error message sent by some ElectrumX servers when they don't support returning verbose transactions. */
private static final String VERBOSE_TRANSACTIONS_UNSUPPORTED_MESSAGE = "verbose transactions are currently unsupported";
public static class Server {
String hostname;
enum ConnectionType { TCP, SSL };
public enum ConnectionType { TCP, SSL }
ConnectionType connectionType;
int port;
@@ -82,144 +90,166 @@ public class ElectrumX {
}
}
private Set<Server> servers = new HashSet<>();
private List<Server> remainingServers = new ArrayList<>();
private Set<Server> uselessServers = Collections.synchronizedSet(new HashSet<>());
private final String netId;
private final String expectedGenesisHash;
private final Map<Server.ConnectionType, Integer> defaultPorts = new EnumMap<>(Server.ConnectionType.class);
private final Object serverLock = new Object();
private Server currentServer;
private Socket socket;
private Scanner scanner;
private int nextId = 1;
private static final int TX_CACHE_SIZE = 200;
@SuppressWarnings("serial")
private final Map<String, BitcoinyTransaction> transactionCache = Collections.synchronizedMap(new LinkedHashMap<>(TX_CACHE_SIZE + 1, 0.75F, true) {
// This method is called just after a new entry has been added
@Override
public boolean removeEldestEntry(Map.Entry<String, BitcoinyTransaction> eldest) {
return size() > TX_CACHE_SIZE;
}
});
// Constructors
private ElectrumX(String bitcoinNetwork) {
switch (bitcoinNetwork) {
case "MAIN":
servers.addAll(Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("tardis.bauerj.eu", Server.ConnectionType.SSL, 50002),
new Server("rbx.curalle.ovh", Server.ConnectionType.SSL, 50002),
new Server("quick.electumx.live", Server.ConnectionType.SSL, 50002),
new Server("enode.duckdns.org", Server.ConnectionType.SSL, 50002),
new Server("electrumx.ddns.net", Server.ConnectionType.SSL, 50002),
new Server("electrumx.ml", Server.ConnectionType.SSL, 50002),
new Server("electrum.eff.ro", Server.ConnectionType.SSL, 50002),
new Server("electrum.bitkoins.nl", Server.ConnectionType.SSL, 50512),
new Server("E-X.not.fyi", Server.ConnectionType.SSL, 50002),
new Server("btc.electroncash.dk", Server.ConnectionType.SSL, 60002),
new Server("electrum.blockstream.info", Server.ConnectionType.TCP, 50001),
new Server("electrum.blockstream.info", Server.ConnectionType.SSL, 50002),
new Server("bitcoin.aranguren.org", Server.ConnectionType.TCP, 50001)));
break;
case "TEST3":
servers.addAll(Arrays.asList(
new Server("tn.not.fyi", Server.ConnectionType.TCP, 55001),
new Server("tn.not.fyi", Server.ConnectionType.SSL, 55002),
new Server("testnet.aranguren.org", Server.ConnectionType.TCP, 51001),
new Server("testnet.aranguren.org", Server.ConnectionType.SSL, 51002),
new Server("testnet.hsmiths.com", Server.ConnectionType.SSL, 53012)));
break;
case "REGTEST":
servers.addAll(Arrays.asList(
new Server("localhost", Server.ConnectionType.TCP, DEFAULT_TCP_PORT),
new Server("localhost", Server.ConnectionType.SSL, DEFAULT_SSL_PORT)));
break;
default:
throw new IllegalArgumentException(String.format("Bitcoin network '%s' unknown", bitcoinNetwork));
}
LOGGER.debug(() -> String.format("Starting ElectrumX support for %s Bitcoin network", bitcoinNetwork));
rpc("server.banner");
}
/** Returns ElectrumX instance linked to passed Bitcoin network, one of "MAIN", "TEST3" or "REGTEST". */
public static synchronized ElectrumX getInstance(String bitcoinNetwork) {
if (!instances.containsKey(bitcoinNetwork))
instances.put(bitcoinNetwork, new ElectrumX(bitcoinNetwork));
return instances.get(bitcoinNetwork);
public ElectrumX(String netId, String genesisHash, Collection<Server> initialServerList, Map<Server.ConnectionType, Integer> defaultPorts) {
this.netId = netId;
this.expectedGenesisHash = genesisHash;
this.servers.addAll(initialServerList);
this.defaultPorts.putAll(defaultPorts);
}
// Methods for use by other classes
public Integer getCurrentHeight() {
Object blockObj = this.rpc("blockchain.headers.subscribe");
if (!(blockObj instanceof JSONObject))
return null;
JSONObject blockJson = (JSONObject) blockObj;
if (!blockJson.containsKey("height"))
return null;
return ((Long) blockJson.get("height")).intValue();
@Override
public String getNetId() {
return this.netId;
}
public List<byte[]> getBlockHeaders(int startHeight, long count) {
Object blockObj = this.rpc("blockchain.block.headers", startHeight, count);
/**
* Returns current blockchain height.
* <p>
* @throws ForeignBlockchainException if error occurs
*/
@Override
public int getCurrentHeight() throws ForeignBlockchainException {
Object blockObj = this.rpc("blockchain.headers.subscribe");
if (!(blockObj instanceof JSONObject))
return null;
throw new ForeignBlockchainException.NetworkException("Unexpected output from ElectrumX blockchain.headers.subscribe RPC");
JSONObject blockJson = (JSONObject) blockObj;
if (!blockJson.containsKey("count") || !blockJson.containsKey("hex"))
return null;
Object heightObj = blockJson.get("height");
Long returnedCount = (Long) blockJson.get("count");
String hex = (String) blockJson.get("hex");
if (!(heightObj instanceof Long))
throw new ForeignBlockchainException.NetworkException("Missing/invalid 'height' in JSON from ElectrumX blockchain.headers.subscribe RPC");
byte[] raw = HashCode.fromString(hex).asBytes();
if (raw.length != returnedCount * BLOCK_HEADER_LENGTH)
return null;
return ((Long) heightObj).intValue();
}
/**
* Returns list of raw block headers, starting from <tt>startHeight</tt> inclusive.
* <p>
* @throws ForeignBlockchainException if error occurs
*/
@Override
public List<byte[]> getRawBlockHeaders(int startHeight, int count) throws ForeignBlockchainException {
Object blockObj = this.rpc("blockchain.block.headers", startHeight, count);
if (!(blockObj instanceof JSONObject))
throw new ForeignBlockchainException.NetworkException("Unexpected output from ElectrumX blockchain.block.headers RPC");
JSONObject blockJson = (JSONObject) blockObj;
Object countObj = blockJson.get("count");
Object hexObj = blockJson.get("hex");
if (!(countObj instanceof Long) || !(hexObj instanceof String))
throw new ForeignBlockchainException.NetworkException("Missing/invalid 'count' or 'hex' entries in JSON from ElectrumX blockchain.block.headers RPC");
Long returnedCount = (Long) countObj;
String hex = (String) hexObj;
List<byte[]> rawBlockHeaders = new ArrayList<>(returnedCount.intValue());
for (int i = 0; i < returnedCount; ++i)
rawBlockHeaders.add(Arrays.copyOfRange(raw, i * BLOCK_HEADER_LENGTH, (i + 1) * BLOCK_HEADER_LENGTH));
byte[] raw = HashCode.fromString(hex).asBytes();
// Most chains use a fixed length 80 byte header, so block headers can be split up by dividing the hex into
// 80-byte segments. However, some chains such as DOGE use variable length headers due to AuxPoW or other
// reasons. In these cases we can identify the start of each block header by the location of the block version
// numbers. Each block starts with a version number, and for DOGE this is easily identifiable (6422788) at the
// time of writing (Jul 2021). If we encounter a chain that is using more generic version numbers (e.g. 1)
// and can't be used to accurately identify block indexes, then there are sufficient checks to ensure an
// exception is thrown.
if (raw.length == returnedCount * BLOCK_HEADER_LENGTH) {
// Fixed-length header (BTC, LTC, etc)
for (int i = 0; i < returnedCount; ++i) {
rawBlockHeaders.add(Arrays.copyOfRange(raw, i * BLOCK_HEADER_LENGTH, (i + 1) * BLOCK_HEADER_LENGTH));
}
}
else if (raw.length > returnedCount * BLOCK_HEADER_LENGTH) {
// Assume AuxPoW variable length header (DOGE)
int referenceVersion = BitTwiddling.intFromLEBytes(raw, 0); // DOGE uses 6422788 at time of commit (Jul 2021)
for (int i = 0; i < raw.length - 4; ++i) {
// Locate the start of each block by its version number
if (BitTwiddling.intFromLEBytes(raw, i) == referenceVersion) {
rawBlockHeaders.add(Arrays.copyOfRange(raw, i, i + BLOCK_HEADER_LENGTH));
}
}
// Ensure that we found the correct number of block headers
if (rawBlockHeaders.size() != count) {
throw new ForeignBlockchainException.NetworkException("Unexpected raw header contents in JSON from ElectrumX blockchain.block.headers RPC.");
}
}
else if (raw.length != returnedCount * BLOCK_HEADER_LENGTH) {
throw new ForeignBlockchainException.NetworkException("Unexpected raw header length in JSON from ElectrumX blockchain.block.headers RPC");
}
return rawBlockHeaders;
}
/** Returns confirmed balance, based on passed payment script, or null if there was an error or no known balance. */
public Long getBalance(byte[] script) {
/**
* Returns confirmed balance, based on passed payment script.
* <p>
* @return confirmed balance, or zero if script unknown
* @throws ForeignBlockchainException if there was an error
*/
@Override
public long getConfirmedBalance(byte[] script) throws ForeignBlockchainException {
byte[] scriptHash = Crypto.digest(script);
Bytes.reverse(scriptHash);
Object balanceObj = this.rpc("blockchain.scripthash.get_balance", HashCode.fromBytes(scriptHash).toString());
if (!(balanceObj instanceof JSONObject))
return null;
throw new ForeignBlockchainException.NetworkException("Unexpected output from ElectrumX blockchain.scripthash.get_balance RPC");
JSONObject balanceJson = (JSONObject) balanceObj;
if (!balanceJson.containsKey("confirmed"))
return null;
Object confirmedBalanceObj = balanceJson.get("confirmed");
if (!(confirmedBalanceObj instanceof Long))
throw new ForeignBlockchainException.NetworkException("Missing confirmed balance from ElectrumX blockchain.scripthash.get_balance RPC");
return (Long) balanceJson.get("confirmed");
}
/** Unspent output info as returned by ElectrumX network. */
public static class UnspentOutput {
public final byte[] hash;
public final int index;
public final int height;
public final long value;
public UnspentOutput(byte[] hash, int index, int height, long value) {
this.hash = hash;
this.index = index;
this.height = height;
this.value = value;
}
}
/** Returns list of unspent outputs pertaining to passed payment script, or null if there was an error. */
public List<UnspentOutput> getUnspentOutputs(byte[] script) {
/**
* Returns list of unspent outputs pertaining to passed payment script.
* <p>
* @return list of unspent outputs, or empty list if script unknown
* @throws ForeignBlockchainException if there was an error.
*/
@Override
public List<UnspentOutput> getUnspentOutputs(byte[] script, boolean includeUnconfirmed) throws ForeignBlockchainException {
byte[] scriptHash = Crypto.digest(script);
Bytes.reverse(scriptHash);
Object unspentJson = this.rpc("blockchain.scripthash.listunspent", HashCode.fromBytes(scriptHash).toString());
if (!(unspentJson instanceof JSONArray))
return null;
throw new ForeignBlockchainException("Expected array output from ElectrumX blockchain.scripthash.listunspent RPC");
List<UnspentOutput> unspentOutputs = new ArrayList<>();
for (Object rawUnspent : (JSONArray) unspentJson) {
@@ -227,7 +257,7 @@ public class ElectrumX {
int height = ((Long) unspent.get("height")).intValue();
// We only want unspent outputs from confirmed transactions (and definitely not mempool duplicates with height 0)
if (height <= 0)
if (!includeUnconfirmed && height <= 0)
continue;
byte[] txHash = HashCode.fromString((String) unspent.get("tx_hash")).asBytes();
@@ -240,68 +270,215 @@ public class ElectrumX {
return unspentOutputs;
}
/** Returns raw transaction for passed transaction hash, or null if not found. */
public byte[] getRawTransaction(byte[] txHash) {
Object rawTransactionHex = this.rpc("blockchain.transaction.get", HashCode.fromBytes(txHash).toString());
/**
* Returns raw transaction for passed transaction hash.
* <p>
* NOTE: Do not mutate returned byte[]!
*
* @throws ForeignBlockchainException.NotFoundException if transaction not found
* @throws ForeignBlockchainException if error occurs
*/
@Override
public byte[] getRawTransaction(String txHash) throws ForeignBlockchainException {
Object rawTransactionHex;
try {
rawTransactionHex = this.rpc("blockchain.transaction.get", txHash, false);
} catch (ForeignBlockchainException.NetworkException e) {
// DaemonError({'code': -5, 'message': 'No such mempool or blockchain transaction. Use gettransaction for wallet transactions.'})
if (Integer.valueOf(-5).equals(e.getDaemonErrorCode()))
throw new ForeignBlockchainException.NotFoundException(e.getMessage());
throw e;
}
if (!(rawTransactionHex instanceof String))
return null;
throw new ForeignBlockchainException.NetworkException("Expected hex string as raw transaction from ElectrumX blockchain.transaction.get RPC");
return HashCode.fromString((String) rawTransactionHex).asBytes();
}
/** Returns list of raw transactions, relating to passed payment script, if null if there's an error. */
public List<byte[]> getAddressTransactions(byte[] script) {
/**
* Returns raw transaction for passed transaction hash.
* <p>
* NOTE: Do not mutate returned byte[]!
*
* @throws ForeignBlockchainException.NotFoundException if transaction not found
* @throws ForeignBlockchainException if error occurs
*/
@Override
public byte[] getRawTransaction(byte[] txHash) throws ForeignBlockchainException {
return getRawTransaction(HashCode.fromBytes(txHash).toString());
}
/**
* Returns transaction info for passed transaction hash.
* <p>
* @throws ForeignBlockchainException.NotFoundException if transaction not found
* @throws ForeignBlockchainException if error occurs
*/
@Override
public BitcoinyTransaction getTransaction(String txHash) throws ForeignBlockchainException {
// Check cache first
BitcoinyTransaction transaction = transactionCache.get(txHash);
if (transaction != null)
return transaction;
Object transactionObj = null;
do {
try {
transactionObj = this.rpc("blockchain.transaction.get", txHash, true);
} catch (ForeignBlockchainException.NetworkException e) {
// DaemonError({'code': -5, 'message': 'No such mempool or blockchain transaction. Use gettransaction for wallet transactions.'})
if (Integer.valueOf(-5).equals(e.getDaemonErrorCode()))
throw new ForeignBlockchainException.NotFoundException(e.getMessage());
// Some servers also return non-standard responses like this:
// {"error":"verbose transactions are currently unsupported","id":3,"jsonrpc":"2.0"}
// We should probably not use this server any more
if (e.getServer() != null && e.getMessage() != null && e.getMessage().contains(VERBOSE_TRANSACTIONS_UNSUPPORTED_MESSAGE)) {
Server uselessServer = (Server) e.getServer();
LOGGER.trace(() -> String.format("Server %s doesn't support verbose transactions - barring use of that server", uselessServer));
this.uselessServers.add(uselessServer);
this.closeServer(uselessServer);
continue;
}
throw e;
}
} while (transactionObj == null);
if (!(transactionObj instanceof JSONObject))
throw new ForeignBlockchainException.NetworkException("Expected JSONObject as response from ElectrumX blockchain.transaction.get RPC");
JSONObject transactionJson = (JSONObject) transactionObj;
Object inputsObj = transactionJson.get("vin");
if (!(inputsObj instanceof JSONArray))
throw new ForeignBlockchainException.NetworkException("Expected JSONArray for 'vin' from ElectrumX blockchain.transaction.get RPC");
Object outputsObj = transactionJson.get("vout");
if (!(outputsObj instanceof JSONArray))
throw new ForeignBlockchainException.NetworkException("Expected JSONArray for 'vout' from ElectrumX blockchain.transaction.get RPC");
try {
int size = ((Long) transactionJson.get("size")).intValue();
int locktime = ((Long) transactionJson.get("locktime")).intValue();
// Timestamp might not be present, e.g. for unconfirmed transaction
Object timeObj = transactionJson.get("time");
Integer timestamp = timeObj != null
? ((Long) timeObj).intValue()
: null;
List<BitcoinyTransaction.Input> inputs = new ArrayList<>();
for (Object inputObj : (JSONArray) inputsObj) {
JSONObject inputJson = (JSONObject) inputObj;
String scriptSig = (String) ((JSONObject) inputJson.get("scriptSig")).get("hex");
int sequence = ((Long) inputJson.get("sequence")).intValue();
String outputTxHash = (String) inputJson.get("txid");
int outputVout = ((Long) inputJson.get("vout")).intValue();
inputs.add(new BitcoinyTransaction.Input(scriptSig, sequence, outputTxHash, outputVout));
}
List<BitcoinyTransaction.Output> outputs = new ArrayList<>();
for (Object outputObj : (JSONArray) outputsObj) {
JSONObject outputJson = (JSONObject) outputObj;
String scriptPubKey = (String) ((JSONObject) outputJson.get("scriptPubKey")).get("hex");
long value = BigDecimal.valueOf((Double) outputJson.get("value")).setScale(8).unscaledValue().longValue();
// address too, if present
List<String> addresses = null;
Object addressesObj = ((JSONObject) outputJson.get("scriptPubKey")).get("addresses");
if (addressesObj instanceof JSONArray) {
addresses = new ArrayList<>();
for (Object addressObj : (JSONArray) addressesObj)
addresses.add((String) addressObj);
}
outputs.add(new BitcoinyTransaction.Output(scriptPubKey, value, addresses));
}
transaction = new BitcoinyTransaction(txHash, size, locktime, timestamp, inputs, outputs);
// Save into cache
transactionCache.put(txHash, transaction);
return transaction;
} catch (NullPointerException | ClassCastException e) {
// Unexpected / invalid response from ElectrumX server
}
throw new ForeignBlockchainException.NetworkException("Unexpected JSON format from ElectrumX blockchain.transaction.get RPC");
}
/**
* Returns list of transactions, relating to passed payment script.
* <p>
* @return list of related transactions, or empty list if script unknown
* @throws ForeignBlockchainException if error occurs
*/
@Override
public List<TransactionHash> getAddressTransactions(byte[] script, boolean includeUnconfirmed) throws ForeignBlockchainException {
byte[] scriptHash = Crypto.digest(script);
Bytes.reverse(scriptHash);
Object transactionsJson = this.rpc("blockchain.scripthash.get_history", HashCode.fromBytes(scriptHash).toString());
if (!(transactionsJson instanceof JSONArray))
return null;
throw new ForeignBlockchainException.NetworkException("Expected array output from ElectrumX blockchain.scripthash.get_history RPC");
List<byte[]> rawTransactions = new ArrayList<>();
List<TransactionHash> transactionHashes = new ArrayList<>();
for (Object rawTransactionInfo : (JSONArray) transactionsJson) {
JSONObject transactionInfo = (JSONObject) rawTransactionInfo;
// We only want confirmed transactions
if (!transactionInfo.containsKey("height"))
Long height = (Long) transactionInfo.get("height");
if (!includeUnconfirmed && (height == null || height == 0))
// We only want confirmed transactions
continue;
String txHash = (String) transactionInfo.get("tx_hash");
String rawTransactionHex = (String) this.rpc("blockchain.transaction.get", txHash);
if (rawTransactionHex == null)
return null;
rawTransactions.add(HashCode.fromString(rawTransactionHex).asBytes());
transactionHashes.add(new TransactionHash(height.intValue(), txHash));
}
return rawTransactions;
return transactionHashes;
}
/** Returns true if raw transaction successfully broadcast. */
public boolean broadcastTransaction(byte[] transactionBytes) {
/**
* Broadcasts raw transaction to network.
* <p>
* @throws ForeignBlockchainException if error occurs
*/
@Override
public void broadcastTransaction(byte[] transactionBytes) throws ForeignBlockchainException {
Object rawBroadcastResult = this.rpc("blockchain.transaction.broadcast", HashCode.fromBytes(transactionBytes).toString());
if (rawBroadcastResult == null)
return false;
// If result is a String, then it is simply transaction hash.
// Otherwise result is JSON and probably contains error info instead.
return rawBroadcastResult instanceof String;
// We're expecting a simple string that is the transaction hash
if (!(rawBroadcastResult instanceof String))
throw new ForeignBlockchainException.NetworkException("Unexpected response from ElectrumX blockchain.transaction.broadcast RPC");
}
// Class-private utility methods
/** Query current server for its list of peer servers, and return those we can parse. */
private Set<Server> serverPeersSubscribe() {
/**
* Query current server for its list of peer servers, and return those we can parse.
* <p>
* @throws ForeignBlockchainException
* @throws ClassCastException to be handled by caller
*/
private Set<Server> serverPeersSubscribe() throws ForeignBlockchainException {
Set<Server> newServers = new HashSet<>();
Object peers = this.connectedRpc("server.peers.subscribe");
if (!(peers instanceof JSONArray))
return newServers;
for (Object rawPeer : (JSONArray) peers) {
JSONArray peer = (JSONArray) rawPeer;
if (peer.size() < 3)
// We're expecting at least 3 fields for each peer entry: IP, hostname, features
continue;
String hostname = (String) peer.get(1);
@@ -310,21 +487,26 @@ public class ElectrumX {
for (Object rawFeature : features) {
String feature = (String) rawFeature;
Server.ConnectionType connectionType = null;
int port = -1;
Integer port = null;
switch (feature.charAt(0)) {
case 's':
connectionType = Server.ConnectionType.SSL;
port = DEFAULT_SSL_PORT;
port = this.defaultPorts.get(connectionType);
break;
case 't':
connectionType = Server.ConnectionType.TCP;
port = DEFAULT_TCP_PORT;
port = this.defaultPorts.get(connectionType);
break;
default:
// e.g. could be 'v' for protocol version, or 'p' for pruning limit
break;
}
if (connectionType == null)
if (connectionType == null || port == null)
// We couldn't extract any peer connection info?
continue;
// Possible non-default port?
@@ -344,34 +526,39 @@ public class ElectrumX {
return newServers;
}
/** Return output from RPC call, with automatic reconnection to different server if needed. */
private synchronized Object rpc(String method, Object...params) {
while (haveConnection()) {
Object response = connectedRpc(method, params);
if (response != null)
return response;
/**
* Performs RPC call, with automatic reconnection to different server if needed.
* <p>
* @return "result" object from within JSON output
* @throws ForeignBlockchainException if server returns error or something goes wrong
*/
private Object rpc(String method, Object...params) throws ForeignBlockchainException {
synchronized (this.serverLock) {
if (this.remainingServers.isEmpty())
this.remainingServers.addAll(this.servers);
this.currentServer = null;
try {
this.socket.close();
} catch (IOException e) {
/* ignore */
while (haveConnection()) {
Object response = connectedRpc(method, params);
if (response != null)
return response;
// Didn't work, try another server...
this.closeServer();
}
this.scanner = null;
}
return null;
// Failed to perform RPC - maybe lack of servers?
LOGGER.info("Error: No connected Electrum servers when trying to make RPC call");
throw new ForeignBlockchainException.NetworkException(String.format("Failed to perform ElectrumX RPC %s", method));
}
}
/** Returns true if we have, or create, a connection to an ElectrumX server. */
private boolean haveConnection() {
private boolean haveConnection() throws ForeignBlockchainException {
if (this.currentServer != null)
return true;
List<Server> remainingServers = new ArrayList<>(this.servers);
while (!remainingServers.isEmpty()) {
Server server = remainingServers.remove(RANDOM.nextInt(remainingServers.size()));
while (!this.remainingServers.isEmpty()) {
Server server = this.remainingServers.remove(RANDOM.nextInt(this.remainingServers.size()));
LOGGER.trace(() -> String.format("Connecting to %s", server));
try {
@@ -384,36 +571,55 @@ public class ElectrumX {
if (server.connectionType == Server.ConnectionType.SSL) {
SSLSocketFactory factory = TrustlessSSLSocketFactory.getSocketFactory();
this.socket = (SSLSocket) factory.createSocket(this.socket, server.hostname, server.port, true);
this.socket = factory.createSocket(this.socket, server.hostname, server.port, true);
}
this.scanner = new Scanner(this.socket.getInputStream());
this.scanner.useDelimiter("\n");
// Check connection works by asking for more servers
// Check connection is suitable by asking for server features, including genesis block hash
JSONObject featuresJson = (JSONObject) this.connectedRpc("server.features");
if (featuresJson == null || Double.valueOf((String) featuresJson.get("protocol_min")) < MIN_PROTOCOL_VERSION)
continue;
if (this.expectedGenesisHash != null && !((String) featuresJson.get("genesis_hash")).equals(this.expectedGenesisHash))
continue;
// Ask for more servers
Set<Server> moreServers = serverPeersSubscribe();
// Discard duplicate servers we already know
moreServers.removeAll(this.servers);
remainingServers.addAll(moreServers);
// Add to both lists
this.remainingServers.addAll(moreServers);
this.servers.addAll(moreServers);
LOGGER.debug(() -> String.format("Connected to %s", server));
this.currentServer = server;
return true;
} catch (IOException e) {
// Try another server...
this.socket = null;
this.scanner = null;
} catch (IOException | ForeignBlockchainException | ClassCastException | NullPointerException e) {
// Didn't work, try another server...
closeServer();
}
}
return false;
}
/**
* Perform RPC using currently connected server.
* <p>
* @param method
* @param params
* @return response Object, or null if server fails to respond
* @throws ForeignBlockchainException if server returns error
*/
@SuppressWarnings("unchecked")
private Object connectedRpc(String method, Object...params) {
private Object connectedRpc(String method, Object...params) throws ForeignBlockchainException {
JSONObject requestJson = new JSONObject();
requestJson.put("id", this.nextId++);
requestJson.put("method", method);
requestJson.put("jsonrpc", "2.0");
JSONArray requestParams = new JSONArray();
requestParams.addAll(Arrays.asList(params));
@@ -428,21 +634,85 @@ public class ElectrumX {
this.socket.getOutputStream().write(request.getBytes());
response = scanner.next();
} catch (IOException | NoSuchElementException e) {
// Unable to send, or receive -- try another server?
return null;
}
LOGGER.trace(() -> String.format("Response: %s", response));
if (response.isEmpty())
// Empty response - try another server?
return null;
Object responseObj = JSONValue.parse(response);
if (!(responseObj instanceof JSONObject))
// Unexpected response - try another server?
return null;
JSONObject responseJson = (JSONObject) responseObj;
Object errorObj = responseJson.get("error");
if (errorObj != null) {
if (errorObj instanceof String)
throw new ForeignBlockchainException.NetworkException(String.format("Unexpected error message from ElectrumX RPC %s: %s", method, (String) errorObj), this.currentServer);
if (!(errorObj instanceof JSONObject))
throw new ForeignBlockchainException.NetworkException(String.format("Unexpected error response from ElectrumX RPC %s", method), this.currentServer);
JSONObject errorJson = (JSONObject) errorObj;
Object messageObj = errorJson.get("message");
if (!(messageObj instanceof String))
throw new ForeignBlockchainException.NetworkException(String.format("Missing/invalid message in error response from ElectrumX RPC %s", method), this.currentServer);
String message = (String) messageObj;
// Some error 'messages' are actually wrapped upstream bitcoind errors:
// "message": "daemon error: DaemonError({'code': -5, 'message': 'No such mempool or blockchain transaction. Use gettransaction for wallet transactions.'})"
// We want to detect these and extract the upstream error code for caller's use
Matcher messageMatcher = DAEMON_ERROR_REGEX.matcher(message);
if (messageMatcher.find())
try {
int daemonErrorCode = Integer.parseInt(messageMatcher.group(1));
throw new ForeignBlockchainException.NetworkException(daemonErrorCode, message, this.currentServer);
} catch (NumberFormatException e) {
// We couldn't parse the error code integer? Fall-through to generic exception...
}
throw new ForeignBlockchainException.NetworkException(message, this.currentServer);
}
return responseJson.get("result");
}
/**
* Closes connection to <tt>server</tt> if it is currently connected server.
* @param server
*/
private void closeServer(Server server) {
synchronized (this.serverLock) {
if (this.currentServer == null || !this.currentServer.equals(server))
return;
if (this.socket != null && !this.socket.isClosed())
try {
this.socket.close();
} catch (IOException e) {
// We did try...
}
this.socket = null;
this.scanner = null;
this.currentServer = null;
}
}
/** Closes connection to currently connected server (if any). */
private void closeServer() {
synchronized (this.serverLock) {
this.closeServer(this.currentServer);
}
}
}

Some files were not shown because too many files have changed in this diff Show More