Compare commits

...

360 Commits

Author SHA1 Message Date
CalDescent
d3b6c5f052 Bump version to 3.3.5 2022-06-14 23:18:46 +01:00
CalDescent
f48eb27f00 Revert "Temporarily limit block minter as first stage of multipart minting fix. This will be reverted almost immediately after release."
This reverts commit 0eebfe4a8c.
2022-06-14 22:42:44 +01:00
CalDescent
b02ac2561f Revert "Safety check - also to be removed shortly."
This reverts commit 8b3f9db497.
2022-06-14 22:42:38 +01:00
CalDescent
1b2f66b201 Bump version to 3.3.4 2022-06-13 23:40:39 +01:00
CalDescent
e992f6b683 Increase column size to allow for approx 10x as many online accounts in each block. 2022-06-13 22:51:48 +01:00
CalDescent
8b3f9db497 Safety check - also to be removed shortly. 2022-06-13 22:41:10 +01:00
CalDescent
0eebfe4a8c Temporarily limit block minter as first stage of multipart minting fix. This will be reverted almost immediately after release. 2022-06-13 22:40:54 +01:00
CalDescent
12b3fc257b Updated AdvancedInstaller project for v3.3.3 2022-06-04 19:50:43 +01:00
CalDescent
66a3322ea6 Bump version to 3.3.3 2022-06-04 19:16:01 +01:00
CalDescent
4965cb7121 Set BlockV2Message min peer version to 3.3.3 2022-06-04 15:50:17 +01:00
CalDescent
b92b1fecb0 disableReferenceTimestamp set to 1655222400000 (Tuesday, 14 June 2022 16:00:00 UTC) 2022-06-04 14:51:37 +01:00
CalDescent
43a75420d0 Merge branch 'disable-reference' 2022-06-04 14:46:51 +01:00
catbref
e85026f866 Initial work on reducing network load for transferring blocks.
Reduced AT state info from per-AT address + state hash + fees to AT count + total AT fees + hash of all AT states.
Modified Block and Controller to support above. Controller needs more work regarding CachedBlockMessages.
Note that blocks fetched from archive are in old V1 format.
Changed Triple<BlockData, List<TransactionData>, List<ATStateData>> to BlockTransformation to support both V1 and V2 forms.

Set min peer version to 3.3.203 in BlockV2Message class.
2022-06-04 14:43:46 +01:00
CalDescent
ba7b9f3ad8 Added feature to allow repository to be kept intact after running certain tests 2022-06-04 14:17:01 +01:00
CalDescent
987446cf7f Updated AdvancedInstaller project for v3.3.2 2022-06-02 11:31:56 +01:00
CalDescent
d2fc705846 Merge remote-tracking branch 'catbref/UnsupportedMessage' 2022-05-31 09:09:08 +02:00
CalDescent
e393150e9c Require references to be the correct length post feature-trigger 2022-05-30 22:40:39 +02:00
CalDescent
43bfd28bcd Revert "Discard unsupported messages instead of disconnecting the peer."
This reverts commit d086ade91f.
2022-05-30 21:46:16 +02:00
catbref
ca8f8a59f4 Better forwards compatibility with newer message types so we don't disconnect newer peers 2022-05-30 20:41:45 +01:00
CalDescent
d086ade91f Discard unsupported messages instead of disconnecting the peer. 2022-05-30 15:27:42 +02:00
CalDescent
64d4c458ec Fixed logging error 2022-05-30 15:16:44 +02:00
CalDescent
2478450694 Revert "Removed "consecutive blocks" limitation in block minter."
This reverts commit f41fbb3b3d.
2022-05-30 13:49:15 +02:00
CalDescent
9f19a042e6 Added test to ensure short (1 byte) references can be imported. 2022-05-30 13:46:00 +02:00
CalDescent
922ffcc0be Modified post-trigger last reference checking, to now require a non-null value
This allows for compatibility with TRANSFER_PRIVS validation in commit 8950bb7, which treats any account with a non-null reference as "existing". It also avoids possible unknown side effects from trying to process and store transactions with a null reference - something that wouldn't have been possible until the validation was removed.
2022-05-30 13:22:15 +02:00
CalDescent
f887fcafe3 Disable last reference validation after feature trigger timestamp (not yet set).
This should prevent the failed transactions that are encountered when issuing two or more in a short space of time. Using a feature trigger (hard fork) to release this, to avoid potential consensus confusion around the time of the update (older versions could consider the main chain invalid until updating).
2022-05-30 13:06:55 +02:00
CalDescent
48b562f71b Auto update check interval slowed from 10s to 30s, to hopefully reduce the chance of encountering "repository in use by another process?" error. 2022-05-28 14:33:37 +02:00
CalDescent
33cffe45fd Bump version to 3.3.2 2022-05-27 17:23:53 +02:00
CalDescent
a0ce75a978 Minimum BTC order amount set to 0.001 BTC. Anything lower than that will result in greater than 10% fees. 2022-05-27 16:59:34 +02:00
QuickMythril
8d168f6ad4 Override default Bitcoin trade fee
Reduced to 20 sats/byte.
2022-05-27 06:57:38 -04:00
CalDescent
0875c5bf3b Fix ConcurrentModificationException in getCachedSigValidTransactions() 2022-05-27 10:34:26 +02:00
QuickMythril
8e71cbd822 Reduce static Bitcoin trade fee 2022-05-26 16:25:07 -04:00
QuickMythril
9896ec2ba6 Fix typo 2022-05-26 15:37:38 -04:00
QuickMythril
9f9a74809e Add Bitcoin ACCTv3
This provides support for restoring BTC in the Trade Portal.
2022-05-26 15:34:51 -04:00
QuickMythril
acce81cdcd Add tray menu item to show Build Version
Core build version in a message dialog for OS which cannot display the entire tooltip.
2022-05-26 15:10:04 -04:00
CalDescent
d72953ae78 Drop expired transactions from the import queue before they are considered "sig valid".
This should prevent expired transactions from being kept alive, adding unnecessary load to the import queue.
2022-05-25 19:06:08 +01:00
CalDescent
551686c2de Updated AdvancedInstaller project for v3.3.1 2022-05-23 21:54:25 +01:00
CalDescent
b73c041cc3 Bump version to 3.3.1 2022-05-23 20:31:36 +01:00
CalDescent
9e8d85285f Removed extra unnecessary digest after writing new data. 2022-05-22 16:13:08 +01:00
CalDescent
f41fbb3b3d Removed "consecutive blocks" limitation in block minter. 2022-05-22 16:13:04 +01:00
CalDescent
fa3a81575a Reduce wasted time that could otherwise be spent validating queued transaction signatures. 2022-05-14 12:53:36 +01:00
CalDescent
6990766f75 Speed up unconfirmed transaction propagation.
Currently, new transactions take a very long time to be included in each block (or reach the intended recipient), because each node has to obtain a repository lock and import the transaction before it notifies its peers. This can take a long time due to the lock being held by the block minter or synchronizer, and this compounds with every peer that the transaction is routed through.

Validating signatures doesn't require a lock, and so can take place very soon after receipt of a new transaction. This change causes each node to broadcast a new transaction to its peers as soon as its signature is validated, rather than waiting until after the import.

When a notified peer then makes a request for the transaction data itself, this can now be loaded from the sig-valid import queue as an alternative to the repository (since they won't be in the repository until after the import, which likely won't have happened yet).

One small downside to this approach is that each unconfirmed transaction is now notified twice - once after the signature is deemed valid, and again in Controller.onNewTransaction(), but this should be an acceptable trade off given the speed improvements it should achieve. Another downside is that it could cause invalid transactions (with valid signatures) to propagate, but these would quickly be added to each peer's invalidUnconfirmedTransactions list after the import failure, and therefore be ignored.
2022-05-14 12:43:54 +01:00
CalDescent
b33afd99a5 Bump invalid transaction import logging from trace to debug. 2022-05-13 13:22:58 +01:00
CalDescent
3c2ba4a0ea Improved logging when importing transactions. 2022-05-13 12:31:48 +01:00
CalDescent
ab0fc07ee9 Refactored transaction importer, to separate signature validation from importing.
Importing has to be single threaded since it requires the database lock, but there's nothing to stop us from validating signatures on multiple threads, as no lock is required. So it makes sense to separate these two functions to allow for possible multi threaded signature validation in the future, to speed up the process.

Everything remains single threaded in this commit. It should be functionally the same as before, to reduce risk.
2022-05-13 12:31:19 +01:00
CalDescent
001650d48e Fixed typo in currently unused getSignaturesInvolvingAddress() method. 2022-05-13 11:28:55 +01:00
CalDescent
659431ebfd Added "txTypes" parameter to GET /transactions/unconfirmed, to allow optional filtering of unconfirmed transactions by one or more types 2022-05-13 11:28:21 +01:00
CalDescent
0a419cb105 Added "creator" parameter to GET /transactions/unconfirmed, to allow filtering unconfirmed transactions by creator's public key 2022-05-13 11:13:39 +01:00
CalDescent
a4d4d17b82 Added /wallets to .gitignore, in preparation for pirate chain support. 2022-05-12 19:34:53 +01:00
CalDescent
0829ff6908 Fixed bugs in tooltip, introduced in lite node branch 2022-05-12 19:33:12 +01:00
CalDescent
2d1b0fd6d0 Merge branch 'lite-node' 2022-05-12 08:51:05 +01:00
CalDescent
122539596d Merge branch 'qdn-direct-connections' 2022-05-12 08:45:45 +01:00
CalDescent
86015e59a1 Updated AdvancedInstaller project for v3.3.0 2022-05-10 08:27:17 +01:00
CalDescent
1ea1e00344 Bump version to 3.3.0 2022-05-09 18:47:19 +01:00
CalDescent
adecb21ada Added mainnet xpub addresses for DGB and RVN tests, as testnet support isn't fully implemented yet. 2022-05-07 17:40:36 +01:00
CalDescent
dac484136f Fixed bug in name rebuilding. 2022-05-07 16:46:10 +01:00
CalDescent
aaa0b25106 Make sure to set Peer.isDataPeer() to false as well as true, to prevent bugs due to object reuse.
Also designate a peer as a "data peer" when making an outbound connection to request data from it.
2022-05-02 10:20:23 +01:00
CalDescent
3409086978 Merge branch 'master' into lite-node 2022-05-02 08:31:43 +01:00
CalDescent
6c201db3dd Merge branch 'EPC-fixes' 2022-05-02 08:28:14 +01:00
CalDescent
da47df0a25 Fixed merge issue. 2022-05-01 16:41:56 +01:00
CalDescent
eea215dacf Merge branch 'ravencoin' into new-coins
# Conflicts:
#	pom.xml
#	src/main/java/org/qortal/controller/tradebot/TradeBot.java
#	src/main/java/org/qortal/crosschain/SupportedBlockchain.java
#	src/main/java/org/qortal/settings/Settings.java
2022-05-01 16:28:22 +01:00
CalDescent
0949271dda Merge branch 'digibyte' into new-coins 2022-05-01 16:25:40 +01:00
CalDescent
6bb9227159 Removed RavencoinACCTv1
Also removed CrossChainRavencoinACCTv1Resource - same as Digibyte.
2022-05-01 16:17:00 +01:00
CalDescent
a95a37277c Removed DigibyteACCTv1 and v2
Also removed CrossChainDigibyteACCTv1Resource, since this is unused, and it seems excessive to maintain support of this for every coin (and potentially every ACCT version).
2022-05-01 16:12:12 +01:00
CalDescent
48b9aa5c18 Allow images to be displayed in QDN websites via data: and blob: 2022-05-01 14:37:57 +01:00
CalDescent
1d7203a6fb Bug fixes found when testing previous commits. 2022-05-01 14:29:24 +01:00
CalDescent
1030b00f0a Keep track of peers requesting data for which we have at least one chunk. Then allow subsequent incoming connections from that peer through, up to a maximum of maxDataPeers.
Direct connections for arbitrary data are currently unlikely to succeed, because those allowing incoming connections generally have their slots maxed out and have reached maxPeers. The idea here is that some connections remain reserved for dedicated arbitrary data transfers, therefore temporarily circumventing the limit (up to a defined maximum number of reserved connections).

Arbitrary data connections will auto disconnect after 2 minutes (we might be able to reduce this at a later date), and it also probably makes sense for the requesting node to disconnect as soon as it has all the chunks that it needs (this part isn't implemented yet).

One downside of this feature is that the listen socket is now going to be accepting connections most of the time, since it is unlikely that we will regularly have 4 data peers connected. This could be improved by modifying the OP_ACCEPT behaviour based on whether we are expecting any data peers to connect. In most cases, this would allow it to remain closed. But for the sake of simplicity I will leave that optimization for a future commit.
2022-05-01 14:02:44 +01:00
CalDescent
0c16d1fc11 Added "maxDataPeerConnectionTime" setting (default 2 mins).
This is used to force a quick disconnect for peers that are only connecting for the purposes of requesting data for a specific arbitrary transaction signature.
2022-05-01 14:02:44 +01:00
CalDescent
ed04375385 Increased default maxPeers from 32 to 36 to compensate - otherwise the network will lose a considerable amount of inbound capacity. 2022-05-01 14:02:44 +01:00
CalDescent
6e49d20383 Added "maxDataPeers" setting to reserve 4 connections by default for direct QDN data requests. 2022-05-01 14:02:44 +01:00
CalDescent
dc34eed203 Include our address when requesting QDN data 2022-05-01 14:02:44 +01:00
CalDescent
fbe4f3fad8 Fixed incorrect minOutboundPeers conditional 2022-05-01 14:01:58 +01:00
CalDescent
e7ee3a06c7 Merge branch 'EPC-fixes' into lite-node 2022-04-30 15:33:07 +01:00
CalDescent
599877195b Merge branch 'master' into EPC-fixes 2022-04-30 15:32:44 +01:00
CalDescent
7f9d267992 Improved lite node response error logging. 2022-04-30 15:32:23 +01:00
CalDescent
52904db413 Migrated new lite node message types to new format. 2022-04-30 15:22:50 +01:00
CalDescent
5e0bde226a Merge branch 'EPC-fixes' into lite-node
# Conflicts:
#	src/main/java/org/qortal/network/message/Message.java
2022-04-30 13:25:02 +01:00
CalDescent
0695039ee3 Fixed long term bug causing last line to be missed out. 2022-04-30 12:08:10 +01:00
CalDescent
a4bcd4451c Added "tail" parameter to GET /admin/logs to allow returning the last X (limit) lines.
This should make it easy to display core logs in the UI.
2022-04-30 12:07:47 +01:00
CalDescent
e5b4b61832 Fixed bugs causing "Hash ... does not match file digest ..." errors 2022-04-30 11:26:05 +01:00
CalDescent
dd55dc277b Updated AdvancedInstaller project for v3.2.5 2022-04-27 08:50:40 +01:00
CalDescent
81ef1ae964 Bump version to 3.2.5 2022-04-26 20:17:57 +01:00
CalDescent
46701e4de7 Revert "Remove peers with unknown height, lower height or same height and same block signature (unless we don't have their block signature)"
This reverts commit 895f02f178.
2022-04-26 19:52:08 +01:00
QuickMythril
0f52ccb433 add Ravencoin ACCTs 2022-04-26 13:51:19 -04:00
QuickMythril
8aed84e6af add Digibyte ACCTs 2022-04-26 11:40:42 -04:00
CalDescent
568497e1c5 Updated AdvancedInstaller project for v3.2.4 2022-04-25 09:03:57 +01:00
CalDescent
f3f8e0013d Bump version to 3.2.4 2022-04-24 17:48:22 +01:00
CalDescent
d03c145189 Added to testRegisterNameFeeIncrease() test to catch the recently detected bug. 2022-04-24 17:41:24 +01:00
CalDescent
682a5fde94 Revert "Attempt to fix core startup problems on some systems (GNOME Desktop?) by adding defensiveness to GUI elements."
This reverts commit 311f41c610.
2022-04-24 15:54:20 +01:00
CalDescent
cca5bac30a Fixed logic bug in name registration fee calculation. 2022-04-24 15:36:36 +01:00
CalDescent
64e102a8c6 Name registration fee reduction to 1.25 QORT set to Sun, 01 May 2022 16:00:00 GMT 2022-04-24 15:27:21 +01:00
CalDescent
f9972f50e0 Updated altcoinj 2022-04-24 15:08:43 +01:00
QuickMythril
05d9a7e820 Switched to Qortal fork of altcoinj, using RavencoinMainNetParams 2022-04-23 08:28:12 -04:00
CalDescent
df290950ea Reduce log spam in BlockMinter 2022-04-23 12:32:06 +01:00
CalDescent
ae64be4802 Retry scheduled repository maintenance up to 5 times, as it's common for it to timeout waiting for the repository. Subsequent retries normally succeed. 2022-04-23 12:31:15 +01:00
CalDescent
348f3c382e Merge branch 'master' into lite-node 2022-04-22 20:40:47 +01:00
CalDescent
d98678fc5f Renamed SECRET_LENGTH to SECRET_SIZE_LENGTH. Thanks to catbref for finding this. 2022-04-22 20:40:13 +01:00
CalDescent
1da157d33f Added separate AT serialization tests, based on generic transaction serialization tests. This allows for testing both MESSAGE-type and PAYMENT-type AT transactions. 2022-04-22 20:36:22 +01:00
CalDescent
de4f004a08 Bump to transaction version 6 at a future undecided timestamp. 2022-04-22 20:35:17 +01:00
CalDescent
522ef282c8 Added support for deserialization of MESSAGE-type AT transactions (requires transaction version 6) 2022-04-22 20:34:42 +01:00
CalDescent
b5522ea260 Added support for PAYMENT-type AT transactions in serialization tests 2022-04-22 20:06:37 +01:00
CalDescent
b1f184c493 Use DigibyteMainNetParams 2022-04-22 16:31:44 +01:00
CalDescent
d66dd51bf6 Switched to Qortal fork of altcoinj, with Digibyte support 2022-04-22 16:31:32 +01:00
QuickMythril
0baed55a44 add DGB wallet 2022-04-21 11:40:17 -04:00
QuickMythril
390b359761 add RVN wallet 2022-04-21 11:38:49 -04:00
CalDescent
311f41c610 Attempt to fix core startup problems on some systems (GNOME Desktop?) by adding defensiveness to GUI elements. 2022-04-20 08:41:37 +01:00
CalDescent
0a156c76a2 Fix for NPE observed on the EPC-fixes branch (but putting the fix on master in case unrelated) 2022-04-20 08:38:59 +01:00
CalDescent
70eaaa9e3b Merge remote-tracking branch 'catbref/EPC-fixes' into EPC-fixes 2022-04-19 20:50:32 +01:00
catbref
3e622f7185 EPC-fixes: catch CancelledKeyExceptions thrown in short window between nextSelectionKey.isValid() and nextSelectionKey.isXXXable() calls 2022-04-18 14:33:05 +01:00
CalDescent
3f12be50ac Merge remote-tracking branch 'catbref/EPC-fixes' into EPC-fixes 2022-04-18 09:20:55 +01:00
catbref
68412b49a1 EPC-fixes: use bindAddress from Settings for outgoing peer connections, not just listen socket 2022-04-17 19:38:50 +01:00
catbref
c9b2620461 EPC-fixes: fix constructing GET_ONLINE_ACCOUNTS_V2 message for case where onlineAccount args is empty list 2022-04-17 19:37:28 +01:00
CalDescent
337b03aa68 Catch java.util.ServiceConfigurationError in Gui.loadImage() 2022-04-17 17:59:29 +01:00
catbref
df3f16ccf1 EPC-fixes: Improve Network shutdown by exiting fast during broadcast and skipping callbacks during peer disconnect. 2022-04-17 14:50:15 +01:00
catbref
22aa5c41b5 WIP: EPC-fixes
BlockMessage was broken because the repository 'connection' associated with the message's Block object was closed between message queuing and message sending.

The fix was to serialize Message subclasses on construction, thus freeing reliance on objects passed into constructor.
The serialized byte[] is held by the message between queuing and sending.
This forces messages into one of two 'modes': outgoing or incoming.
Outgoing messages contain serialized byte[] whereas incoming messages unpack a ByteBuffer into Message subclass fields.
As a result, all network message types have been refactored in this way.
More details in Message's class comment.

A knock-on effect is that incoming messages cannot then be sent out - a new message needs to be constructed.
Some changes needed to Arbitrary controller package classes in this respect.

Bonus: Network no longer needs broadcast threads because 'broadcasting' is now simply the act of queuing a message for many peers.
2022-04-17 14:50:15 +01:00
catbref
8e09567221 EPC-fixed: avoiding some CancelledKeyExceptions 2022-04-17 14:50:15 +01:00
catbref
3505788d42 Another chunk of improvements to networking / EPC.
Instead of synchronizing/blocking in Peer.sendMessage(),
we queue messages in a concurrent blocking TransferQueue, with timeout.

In EPC, ChannelWriteTasks consume from TransferQueue, unblocking callers to Peer.sendMessage().

If a new message is to be sent, or socket output buffer is full,
then OP_WRITE is used to wait for socket to become writable again.

Only one ChannelWriteTask per peer can be active/pending at a time.
Each ChannelWriteTask tries to send as much as it can in one go.

Other minor tidy-ups.
2022-04-17 14:50:15 +01:00
catbref
91e0c9b940 More improvements to networking:
As per work done by szisti in PR#45:
Extracted network 'Tasks' to their own classes.
Network.NetworkProcessor reduced to only producing Tasks.

Improved usage of SocketChannel interest-ops.
Eventually this might lead to reducing task-producing synchronization lock into more granular locks.
Work still needed to convert sending messages to a queue and to make use of OP_WRITE instead of sleeping to wait for socket buffer to empty.

Disabled PeerConnectTask producer from checking against connected peers via DNS as it's too slow.

Swapped Peer's replyQueues from SynchronizedMap(wrapped HashMap) to ConcurrentHashMap.

Other minor changes within networking.
2022-04-17 14:50:15 +01:00
catbref
00996b047f Networking work-in-progress:
As per work done by szisti in PR#45:
Extracted MessageException from inside Message into its own class.
Extracted MessageType from inside Message into its own class.

Converted reflection-based Message.fromByteBuffer method call to non-reflection, functional interface, method-reference.
This should have minor performance improvement but stronger method signature and type enforcement, as well as better IDE integration.

Message.fromByteBuffer method 'contract' tightened up to:
1. throw BufferUnderflowException if there are not enough bytes to deserialize message
2. throw MessageException if bytes contain invalid data
3. should not return null

Message.toData method 'contract' tightened up to:
1. return null if the message has no payload to serialize
2. throw IOException directly - no need to try-catch in each subclass

Several Message-subclass fields now marked 'final' as per IDE suggestion.
Several Message-subclass fromByteBuffer() method signatures have changed 'throws' list.
Several bytes.remaining() != some-value changed to bytes.remaining() < some-value as per new contract.

Some bytes.remaining() checks removed for fixed-length messages because we can rely on ByteBuffer throwing BufferUnderflowException.
Some bytes.remaining() checks retained for variable-length messages, or messages that read a large amount of data, to prevent wasted memory allocations.

Other minor tidying up
2022-04-17 14:50:15 +01:00
catbref
44fc0f367d Networking work-in-progress:
Temporarily increase sleep from 1ms to 100ms when waiting for outgoing socket buffer to empty.

Real fix is to rewrite using an outgoing message queue and OP_WRITE interest op.
2022-04-17 14:50:15 +01:00
catbref
b0e6259073 Networking work-in-progress:
De-register a peer's socket channel OP_READ interest op when producing a ChannelTask for that peer.
This should prevent duplicate ChannelTasks for the same peer.

Re-register OP_READ once node has read from peer's channel.
2022-04-17 14:50:15 +01:00
catbref
6255b2a907 Networking work-in-progress:
When node has reached max connections, Network will ignore pending incoming connections by:
1. not calling accept()
2. de-registering OP_ACCEPT 'interest op' on the listen socket's channel

When a peer disconnects, Network might re-register OP_ACCEPT interest op on listen socket.
2022-04-17 14:50:15 +01:00
catbref
a5fb0be274 Fix Network.disconnectPeer(PeerAddress) to prevent removeIf() on UnmodifiableList throwing UnsupportedOperationException 2022-04-17 14:50:15 +01:00
catbref
e835f6d998 ExecuteProduceConsume:
Slight reworking of EPC to simplify when producer can block
and generally make some of the conditional code more readable.

Improved logging with task class names and logging level editable during runtime!
Use /peer/enginestats?newLoggingLevel=DEBUG (or TRACE or back to INFO) to change.
2022-04-17 14:50:15 +01:00
CalDescent
54ff564bb1 Set name for transaction importer thread 2022-04-16 23:32:42 +01:00
CalDescent
f8a5ded0ba Fix for bug introduced in commit cfe9252 2022-04-16 23:20:49 +01:00
CalDescent
a1be66f02b Temporarily ease the filtering of lite node peers, in order to make development easier. 2022-04-16 20:52:31 +01:00
CalDescent
0815ad2cf0 Added AT transaction deserialization, to all them to be sent in messages for lite nodes.
Note that it is currently not easy to distinguish between MESSAGE-type and PAYMENT-type AT transactions, so PAYMENT-type is currently the only one supported (and used). A hard fork will likely be needed in order to specify the type within each message.
2022-04-16 20:52:31 +01:00
CalDescent
3484047ad4 Added GET /transactions/address/{address} API endpoint
This is a more standardized alternative to using GET /transactions/search?address=xyz. This avoids the need to build full transaction search ability into the lite node protocols right away.
2022-04-16 20:52:31 +01:00
CalDescent
a63fa1cce5 Added GET_ACCOUNT_TRANSACTIONS message, as well as a generic TRANSACTIONS message for responses. 2022-04-16 20:52:31 +01:00
CalDescent
59119ebc3b Added GET_NAME message to allow lookups from name to owner (or any other name data). 2022-04-16 20:52:31 +01:00
CalDescent
276f1b7e68 Fixed small errors in earlier commits. 2022-04-16 20:52:31 +01:00
CalDescent
c482e5b5ca Added GET_ACCOUNT_NAMES message to request names for an address, and a generic NAMES message to return a list of NameData objects. The generic NAMES message can be reused for many other responses, such as requesting the various lists of names that the API supports. 2022-04-16 20:52:31 +01:00
CalDescent
8c3e0adf35 Added message types to fetch account details and account balances, and use these in various APIs.
This should bring in enough data for very basic chat and wallet functionality (using addresses rather than registered names).

Data currently comes from a single random peer, however this can be expanded to request from multiple peers to gain confidence in the accuracy of the data. If bad data is returned from a peer, it's not the end of the world since the transaction would just be considered invalid by full nodes and would be thrown out. But this should be mostly avoidable by taking data from multiple sources to improve confidence in its accuracy.
2022-04-16 20:52:30 +01:00
CalDescent
64ff3ac672 Improved comment 2022-04-16 20:52:30 +01:00
CalDescent
cfe92525ed Disable various core functions when running as a lite node.
Lite nodes can't sync or mint blocks, and they also have a very limited ability to verify unconfirmed transactions due to a lack of contextual information (i.e. the blockchain). For now, most validation is skipped and they simply act as relays to help get transactions around the network. Full and topOnly nodes will disregard any invalid transactions upon receipt as usual, and since the lite nodes aren't signing any blocks, there is little risk to the reduced validation, other than the experience of the lite node itself. This can be tightened up considerably as the lite nodes become more powerful, but the current approach works as a PoC.
2022-04-16 20:52:30 +01:00
CalDescent
0e3a9ee2b2 Return the node "type" (full / topOnly / lite) in GET /admin/info endpoint.
This can used by the UI to hide features that aren't supported on lite nodes.
2022-04-16 20:50:27 +01:00
CalDescent
a921db2cc6 Added "lite" setting to designate the core as a lite node. 2022-04-16 20:50:27 +01:00
CalDescent
3d99f86630 Improved logging 2022-04-16 20:50:00 +01:00
CalDescent
8d1a58ec06 POW_DIFFICULTY_NO_QORT reduced from 14 to 12 (around 4x faster) 2022-04-16 12:36:32 +01:00
CalDescent
2e5a7cb5a1 Adapted Blockchain.java to use lookup table for name registration fees, to more easily support fee adjustments.
This is currently for name registration transactions only, but can be adapted (or duplicated) for other transaction types when needed.

Note: this switches from a greater-than (>) to a greater-than-or-equal (>=) timestamp comparison, as it makes more sense this way. It shouldn't affect the previous transition since there were no REGISTER_NAME transactions at that exact timestamp.
2022-04-16 12:20:03 +01:00
CalDescent
895f02f178 Remove peers with unknown height, lower height or same height and same block signature (unless we don't have their block signature)
Adapted from code originally written by catbref from before genesis, and essentially prevents syncing backwards. This needs significant testing on testnet.
2022-04-16 11:30:07 +01:00
CalDescent
c59869982b Fix for system-wide QDN issues occuring when the metadata file has an empty chunks array.
It is quite likely that existing resources with both metadata and an empty chunks array will need to be republished, because this bug may have led to incorrect file deletions.
2022-04-16 11:25:44 +01:00
CalDescent
3b3368f950 Merge pull request #85 from QuickMythril/member-count
Add member count to each group returned by GET /member/{address}
2022-04-16 11:00:35 +01:00
QuickMythril
3f02c760c2 Add member count to each group returned by GET /member/{address} 2022-04-15 06:23:10 -04:00
CalDescent
fee603e500 Add member count to each group returned by GET /groups (expanded on code written by QuickMythril) 2022-04-15 10:19:43 +01:00
QuickMythril
ad31d8014d get memberCount with Group Data
works for lookup by groupId
2022-04-14 22:08:52 -04:00
CalDescent
58a0ac74d2 Merge pull request #84 from catbref/ByteArray
Improvements to ByteArray to leverage Java 11 'native' Arrays methods
2022-04-14 21:30:59 +01:00
QuickMythril
8388aa9c23 update Russian translation
credit: Alexander45 & malina
2022-04-10 15:50:29 -04:00
catbref
c1894d8c00 Improvements to ByteArray to leverage Java 11 'native' Arrays.hashCode and Arrays.compareUnsigned for speed.
Also modified ambiguous ByteArray::new and ByteArray::of to ByteArray::wrap and ByteArray::copyOf.
Modifications to other classes that use ByteArray.
2022-04-10 16:38:02 +01:00
QuickMythril
f7f9cdc518 Merge pull request #83 from aldum/feature/hungarian_translation
fixup grammar; add missing translations
2022-04-09 00:10:37 -04:00
QuickMythril
850d7f8220 add/update translations
credit: johnnyfg (sv), schizo (it), IsBe (nl), Eduardo9999 (es)
2022-04-08 23:57:54 -04:00
aldum
051043283c fixup grammar; add missing translations 2022-04-06 23:21:49 +02:00
QuickMythril
15bc69de01 Merge pull request #82 from JaymenChou/patch-8
Update SysTray_zh_CN.properties
2022-04-05 13:38:13 -04:00
QuickMythril
ee3cfa4d6d fix typo 2022-04-05 13:26:02 -04:00
QuickMythril
df1f3079a5 Merge pull request #81 from JaymenChou/patch-7
Update SysTray_zh_TW.properties
2022-04-05 13:25:06 -04:00
QuickMythril
d9ae8a5552 Merge branch 'master' into patch-8 2022-04-05 13:23:19 -04:00
QuickMythril
2326c31ee7 Merge branch 'master' into patch-7 2022-04-05 13:11:14 -04:00
QuickMythril
91cb0f30dd Updated TransactionValidity translations
added some missing entries, and sorted alphabetically.
2022-04-05 12:51:49 -04:00
QuickMythril
c0307c352c Updated ApiError translations
removed some duplicate entries, and standardized the order
2022-04-05 11:46:32 -04:00
QuickMythril
8fd7c1b313 formatting fix 2022-04-05 11:09:30 -04:00
QuickMythril
b8147659b1 Updated SysTray translations
added some missing entries, and sorted alphabetically.
2022-04-05 10:48:43 -04:00
JaymenChou
7a1bac682f Update SysTray_zh_TW.properties
Add the missing term "PERFORMING_DB_MAINTENANCE" and translate it to Traditional Chinese
2022-04-04 20:36:48 +08:00
JaymenChou
9fdb7c977f Update SysTray_zh_CN.properties
Translate remaining terms to Simple Chinese
2022-04-04 20:33:59 +08:00
JaymenChou
4f3948323b Update SysTray_zh_TW.properties
Translate the remaining terms to Traditional Chinese
2022-04-04 20:31:19 +08:00
QuickMythril
70fcc1f712 Merge pull request #78 from JaymenChou/patch-4
Create ApiError_zh_CN.properties
2022-04-04 02:49:00 -04:00
JaymenChou
f20fe9199f Update ApiError_zh_CN.properties 2022-04-04 14:36:55 +08:00
QuickMythril
91dee4a3b8 Merge pull request #80 from JaymenChou/patch-6
Create TransactionValidity_zh_CN.properties
2022-04-04 02:17:35 -04:00
QuickMythril
0b89b8084e Merge pull request #79 from JaymenChou/patch-5
Create TransactionValidity_zh_TW.properties
2022-04-04 02:17:24 -04:00
QuickMythril
a5a80302b2 Merge pull request #77 from JaymenChou/patch-3
Create ApiError_zh_TW.properties
2022-04-04 02:17:02 -04:00
QuickMythril
e61a24ee7b removed electrum-ltc.bysh.me
this server often gives a false positive for phishing by some antivirus software.
2022-04-03 22:32:57 -04:00
JaymenChou
55ed342b59 Create TransactionValidity_zh_CN.properties
Add Simple Chinese For better understanding of logs
2022-04-03 13:27:52 +08:00
JaymenChou
3c6f79eec0 Create TransactionValidity_zh_TW.properties
Add Traditional Chinese For TransactionValidity Logs.
2022-04-03 13:25:32 +08:00
JaymenChou
590800ac1d Create ApiError_zh_CN.properties
Add Simple Chinese Support For API Error Message 
Hope it helps in understanding the API !
2022-04-03 12:43:18 +08:00
JaymenChou
95c412b946 Create ApiError_zh_TW.properties
Add Traditional Chinese support to API Responses
2022-04-03 12:40:27 +08:00
CalDescent
a232395750 Merge branch 'master' of github.com:Qortal/qortal 2022-04-01 11:24:56 +01:00
QuickMythril
6edbc8b6a5 add decimal precision to download progress 2022-03-31 13:46:40 -04:00
QuickMythril
f8ffeed302 updated BTC electrumx servers
added new and removed TCP, closed servers, and versions older than 1.16.0
2022-03-31 11:32:55 -04:00
QuickMythril
e2ee68427c removed TCP electrumx servers 2022-03-31 11:29:54 -04:00
QuickMythril
74ff23239d removed TCP electrumx servers 2022-03-31 11:27:56 -04:00
QuickMythril
f1fa2ba2f6 added SSL electrumx servers 2022-03-31 10:02:31 -04:00
QuickMythril
e1522cec94 updated LTC electrumx servers 2022-03-31 09:58:53 -04:00
QuickMythril
8841b3cbb1 add spanish translations 2022-03-31 08:44:33 -04:00
CalDescent
94260bd93f Decreased the number of retries for missing metadata, to reduce broadcast spam. 2022-03-30 08:23:22 +01:00
CalDescent
15ff8af7ac Don't process trade bots or broadcast presence timestamps if our chain is more than 30 minutes old 2022-03-30 08:11:02 +01:00
CalDescent
d420033b36 Revert "Revert "Add Qortal AT FunctionCodes for getting account level / blocks minted + tests""
This reverts commit 59025b8f47.
2022-03-30 08:07:07 +01:00
CalDescent
bda63f0310 Removed hardcoded "qortal-backup/TradeBotStates.json" from POST /admin/repository/data API, as it's no longer needed now that API keys are required. 2022-03-30 08:06:09 +01:00
QuickMythril
54add26ccb fixed typo 2022-03-25 23:39:41 -04:00
CalDescent
089b068362 Updated AdvancedInstaller project for v3.2.3 2022-03-19 22:38:58 +00:00
CalDescent
fe474b4507 Bump version to 3.2.3 2022-03-19 20:44:41 +00:00
CalDescent
bbe15b563c Added unit test to simulate recent issue.
This fails with the 3.2.2 code but now passes when using the latest fixes.
2022-03-19 20:41:38 +00:00
CalDescent
59025b8f47 Revert "Add Qortal AT FunctionCodes for getting account level / blocks minted + tests"
This reverts commit eb9b94b9c6.
2022-03-19 19:52:14 +00:00
CalDescent
1b42c5edb1 Fixed NPE in runIntegrityCheck()
This feature is disabled by default so can be tidied up later. For now, the unhandled scenario is logged and the checking continues on.

One name's transactions are too complex for the current integrity check code to verify (MangoSalsa), but it has been verified manually. All other names pass the automated test.
2022-03-19 19:22:16 +00:00
CalDescent
362335913d Fixed infinite loop in name rebuilding.
If an account is renamed and then at some point renamed back to one of the original names, it confused the names rebuilding code. The current solution is to track the linked names that have already been rebuilt, and then break out of the loop once a name is encountered a second time.
2022-03-19 18:55:19 +00:00
CalDescent
4340dac595 Fixed recently introduced issue in name rebuilding code causing transactions to be unordered.
This is the likely cause of inconsistent name entries across different nodes, as we can't guarantee that every environment will return the same transaction order from the SQL queries.
2022-03-19 18:44:16 +00:00
CalDescent
f3e1fc884c Merge pull request #63 from catbref/master
Add Qortal AT FunctionCodes for getting account level / blocks minted
2022-03-19 11:32:39 +00:00
CalDescent
39c06d8817 Merge pull request #75 from catbref/name-unicode
Unicode / NAME updates.
2022-03-19 11:32:22 +00:00
CalDescent
91cee36c21 Catch and log all exceptions in addStatusToResources()
Some users are seeing 500 errors deriving from this code. This should hopefully allow more info to be obtained, as well as causing it to omit the status for resources that encounter problems.
2022-03-19 11:08:42 +00:00
CalDescent
6bef883942 Removed OpenJDK 11 reference in build-release.sh, as it seems that checksums will not match by default due to timestamps and file orderings.
See: https://dzone.com/articles/reproducible-builds-in-java
2022-03-19 11:05:51 +00:00
CalDescent
25ba2406c0 Updated AdvancedInstaller project for v3.2.2 2022-03-16 19:53:22 +00:00
CalDescent
e4dc8f85a7 Bump version to 3.2.2 2022-03-15 19:57:02 +00:00
CalDescent
12a4a260c8 Handle new sync result case. 2022-03-14 22:04:11 +00:00
CalDescent
268f02b5c3 Added automated test to ensure that the core's default bootstrap hosts are functional.
Whilst not strictly a unit test, this should allow issues with the core's bootstrap servers to be caught quickly.
2022-03-14 21:52:54 +00:00
CalDescent
13eff43b87 Fixed synchronizer issues which caused large re-orgs
Peers without a recent block are removed at the start of the sync process, however, due to the time lag involved in fetching block summaries and comparing the list of peers, some of these could subsequently drop back to a non-recent block and still be chosen as the next peer to sync with. The end result being that nodes could unnecessarily orphan as many as 20 blocks due to syncing with a peer that doesn't have a recent block (but has a couple of high weight blocks after the common block).

This commit adds some additional filtering to avoid this situation.

1) Peers without a recent block are removed as candidates in comparePeers(), allowing for alternate peers to be chosen.
2) After comparePeers() completes, the list is filtered a second time to make sure that all are still recent.
3) Finally, the peer's state is checked one last time in syncToPeerChain(), just before any orphaning takes place.

Whilst just one of the above would probably have been sufficient, the consequences of this bug are so severe that it makes sense to be very thorough.

The only exception to the above is when the node is in "recovery mode", in which case peers without recent blocks are allowed to be included. Items 1 and 3 above do not apply in recovery mode. Item 2 does apply, since the entire comparePeers() functionality is already skipped in a recovery situation due to our chain being out of date.
2022-03-14 21:47:37 +00:00
catbref
e604a19bce Unicode / NAME updates.
Fix UPDATE_NAME not processing empty 'newName' transactions correctly.
Fix some emoji code-points not being processed correctly.
Updated tests.
Now included ICU4J v70.1 - WARNING: this could add around 10MB to JAR size!
Bumped homoglyph to v1.2.1.
2022-03-14 08:45:32 +00:00
CalDescent
e63e39fe9a Updated AdvancedInstaller project for v3.2.1 2022-03-13 19:39:58 +00:00
CalDescent
584c951824 Bump version to 3.2.1 2022-03-13 18:53:54 +00:00
CalDescent
f0d9982ee4 Made arbitraryDataFileHashResponses final, and use .sort() rather than .stream() to avoid new instance creation. 2022-03-12 09:43:56 +00:00
CalDescent
c65de74d13 Revert "Synchronize arbitrary data list removals, as it seems that SynchronizedList and SynchronizedMap aren't overriding removeIf() with a thread-safe version."
This reverts commit e5f88fe2f4.
2022-03-12 09:40:13 +00:00
CalDescent
df0a9701ba Improved logging in onNetworkGetArbitraryDataFileListMessage() 2022-03-11 16:51:19 +00:00
CalDescent
4ec7b1ff1e Removed time estimations that are no longer correct or relevant. 2022-03-11 16:50:34 +00:00
CalDescent
7d3a465386 Including the number of hashes (even if zero) is now required in GetArbitraryDataFileListMessage, to allow for additional fields. Enough peers should have updated by now for this to be ok. 2022-03-11 16:50:11 +00:00
CalDescent
30347900d9 Tidied up one last place that was accessing immutableConnectedPeers directly. This makes no difference, but helps with code consistency. 2022-03-11 15:28:54 +00:00
CalDescent
e5f88fe2f4 Synchronize arbitrary data list removals, as it seems that SynchronizedList and SynchronizedMap aren't overriding removeIf() with a thread-safe version. 2022-03-11 15:22:34 +00:00
CalDescent
0d0ccfd0ac Small refactor for code simplicity. 2022-03-11 15:11:07 +00:00
CalDescent
9013d11d24 Report as 100% synced if the latest block is within the last 30 mins
This should hopefully reduce confusion due to APIs reporting 99% synced even though up to date. The systray should never show this since it already treats blocks in the last 30 mins as synced.
2022-03-11 14:53:10 +00:00
CalDescent
fc5672a161 Use a more tolerant latest block timestamp in the isUpToDate() calls below to reduce misleading systray statuses.
Any block in the last 30 minutes is considered "up to date" for the purposes of displaying statuses.
2022-03-11 14:49:02 +00:00
CalDescent
221c3629e4 Don't refetch the file list if the fileListCache is empty, since an empty list now means that there are likely to be no files available on disk. 2022-03-11 13:08:37 +00:00
CalDescent
76fc56f1c9 Fetch the file list in getFilenameForHeight() if needed. 2022-03-11 13:07:16 +00:00
CalDescent
8e59aa2885 Peer getter methods renamed to include "immutable", for consistency with underlying lists and also to make it clearer to the callers. 2022-03-11 13:00:47 +00:00
CalDescent
0738dbd613 Avoid direct access to this.connectedPeers, as we need to use the immutable copy. 2022-03-11 12:58:11 +00:00
CalDescent
196ecffaf3 Skip calls to this.logger.trace() in ExecuteProduceConsume.run() if trace logging isn't enabled.
This could very slightly reduce load due to skipping the internal filtering inside log4j. Given that this method is causing major problems with CPU at times, I'm trying to make it as optimized as possible.
2022-03-11 11:59:18 +00:00
CalDescent
a0fedbd4b0 Implemented suggestions from catbref to avoid potential thread safety issue in peer arrays. 2022-03-11 11:27:13 +00:00
CalDescent
7c47e22000 Set fileListCache to null when invalidating. 2022-03-11 11:01:29 +00:00
CalDescent
6aad6a1618 fileListCache is now an immutable Map, which is thread safe. Thanks to catbref for this idea. 2022-03-11 10:59:07 +00:00
CalDescent
b764172500 Revert "Hopeful fix for ConcurrentModificationException in BlockArchiveReader.getFilenameForHeight()"
This reverts commit a12ae8ad24.
2022-03-11 10:55:22 +00:00
CalDescent
c185d79672 Loop through all available direct peer connections and try each one in turn.
Also added some extra conditionals to avoid repeated attempts with the same port.
2022-03-09 20:55:27 +00:00
CalDescent
76b8ba91dd Only add an entry to directConnectionInfo if one with this peer-signature combination doesn't already exist. 2022-03-09 20:50:03 +00:00
CalDescent
0418c831e6 Direct connections with peers now prefer those with the highest number of chunks for a resource. Once a connection has been attempted with a peer, remove it from the list so that it isn't attempted again in the same round. 2022-03-09 20:15:26 +00:00
CalDescent
4078f94caa Modified GetArbitraryDataFileListMessage to allow requesting peer's address to be optionally included.
This can ultimately be used to notify the serving peer to expect a direct connection from the requesting peer (to allow it to temporarily bypass maxConnections for long enough for the files to be retrieved). Or it could even possibly be used to trigger a reverse connection (from the serving peer to the requesting peer).
2022-03-09 19:58:02 +00:00
CalDescent
a12ae8ad24 Hopeful fix for ConcurrentModificationException in BlockArchiveReader.getFilenameForHeight() 2022-03-09 19:46:50 +00:00
CalDescent
498ca29aab Wait until a successful connection with a peer before tracking the direct request. 2022-03-08 23:07:08 +00:00
CalDescent
ba70e457b6 Default chunk size reduced from 1MB to 0.5MB 2022-03-08 22:44:43 +00:00
CalDescent
d62808fe1d Don't attempt to create the data directory every time an ArbitraryDataFile instance is instantiated. This was using excessive amounts of CPU and disk I/O. 2022-03-08 22:42:07 +00:00
CalDescent
6c14b79dfb Removed bootstrap host that is no longer functional. 2022-03-08 22:30:01 +00:00
CalDescent
631a253bcc Added support for dark theme in loading screen. 2022-03-08 22:29:37 +00:00
CalDescent
4cb63100d3 Drop the ArbitraryPeers table as it's no longer needed 2022-03-06 13:01:09 +00:00
CalDescent
42fcee0cfd Removed all code that interfaced with the ArbitraryPeers table 2022-03-06 13:00:11 +00:00
CalDescent
829a2e937b Removed all arbitrary signature broadcasts 2022-03-06 12:58:01 +00:00
CalDescent
5d7e5e8e59 Dropped support of ARBITRARY_SIGNATURES message handling, as this feature has been superseded by the peerAddress in file list requests. 2022-03-06 12:46:06 +00:00
CalDescent
6f0a0ef324 Small refactor 2022-03-06 12:42:19 +00:00
CalDescent
f7fe91abeb sendOurOnlineAccountsInfo() moved to its own thread, in preparation for mempow 2022-03-06 12:41:54 +00:00
CalDescent
7252e8d160 Deleted presence tests, as they are no longer relevant, and aren't easily adaptable to the new approach. 2022-03-06 12:03:18 +00:00
CalDescent
2630c35f8c Chunk validation now uses MAX_CHUNK_SIZE rather than CHUNK_SIZE, to allow for a smaller CHUNK_SIZE value to be optionally used, without failing the validation of existing resources. 2022-03-06 11:43:28 +00:00
CalDescent
49f466c073 Added missing break; 2022-03-06 11:21:55 +00:00
CalDescent
c198f785e6 Added significant CPU optimizations to ArbitraryDataManager
- Slow down loops that query the db
- Check for new metadata every 5 minutes instead of constantly
- Check for new data every 1 minute instead of constantly

This could be further improved in the future by having block.process() notify the ArbitraryDataManager that there is new data to process. This would avoid the need for the frequent checks/loops, and only a single complete sweep would be needed on node startup (as long as failures are then retried). But I will avoid this additional complexity for now.
2022-03-06 11:21:39 +00:00
CalDescent
5be093dafc Fix for "Synchronizing null%" systray bug introduced in 3.2.0 2022-03-06 11:00:53 +00:00
CalDescent
2c33d5256c Added code accidentally missed out of commit 1b036b7 2022-03-05 20:44:01 +00:00
CalDescent
4448e2b5df Handle case when metadata isn't returned. 2022-03-05 17:39:13 +00:00
CalDescent
146d234dec Additional defensiveness in ArbitraryDataFile.fromHash() to avoid similar future bugs. 2022-03-05 17:25:48 +00:00
CalDescent
18d5c924e6 Fixed bug cased by fetchAllMetadata() 2022-03-05 17:25:14 +00:00
CalDescent
b520838195 Increased default maxNetworkThreadPoolSize from 20 to 32
This will hopefully offset some of the additional network demands from arbitrary data requests.
2022-03-05 17:24:55 +00:00
CalDescent
1b036b763c Major CPU optimization to block minter
Load sorted list of reward share public keys into memory, so that the indexes can be obtained. This is around 100x faster than querying each index separately (and the savings will increase as more keys are added).

For 4150 reward share keys, it was taking around 5000ms to query individually, vs 50ms using this approach.

The main trade off is that these 4150 keys require around 130kB of additional memory when minting (and this will increase proportionally with more minters). However, this one query was often accounting for 50% of the entire core's CPU usage, so the additional memory usage seems insignificant by comparison.

To gain confidence, I ran both old and new approaches side by side, and confirmed that the indexes matched exactly.
2022-03-05 16:10:43 +00:00
CalDescent
8545a8bf0d Automatically fetch metadata for all resources that have it. 2022-03-05 13:00:49 +00:00
CalDescent
f0136a5018 Include the external port when responding ArbitraryDataFileListRequests 2022-03-05 13:00:17 +00:00
CalDescent
6697b3376b Direct peer connections now use the on-demand data retrieved from file list requests, rather than the stale and incomplete ArbitraryPeerData. 2022-03-05 12:59:13 +00:00
CalDescent
ea785f79b8 Removed unnecessary synchronization 2022-03-04 19:02:30 +00:00
CalDescent
0352a09de7 New online accounts are now verified on the OnlineAccountsManager thread rather than on network threads. This is an attempt to reduce the amount of blocked network threads due to signature verification, and is necessary for the upcoming mempow addition. 2022-03-04 17:58:06 +00:00
CalDescent
5b4f15ab2e Transaction importing code moved to TransactionImporter controller class
As with online accounts, no logic changes other than moving transaction queue processing from the controller thread to its own dedicated thread.
2022-03-04 16:47:21 +00:00
CalDescent
fd37c2b76b Moved all online accounts code to a new OnlineAccountsManager controller class
There are no logic changes here other than moving performOnlineAccountsTasks() onto its own thread, so that it's not subject to anything that might be slowing down the main controller thread.
2022-03-04 16:24:04 +00:00
CalDescent
924aa05681 Optimized peer lists
- Removed synchronization from connectedPeers, and replaced it with an unmodifiableList.
- Added additional immuatable caches: handshakedPeers and outboundHandshakedPeers

This should greatly reduce the amount of time spent waiting around for access to the connectedPeers array, since it is now immediately accessible without needing to obtain a lock. It also removes calls to stream() which were consuming large amounts of CPU to constantly filter the connected peers down to a list of handshaked peers.

Thanks to @catbref for these great suggestions.
2022-03-04 15:14:12 +00:00
CalDescent
84b42210f1 Use ArbitraryDataFileRequestThreads only - instead of reusing file list response threads. 2022-03-04 13:34:16 +00:00
CalDescent
941080c395 Rework of arbitraryDataFileHashResponses to use a list rather than a map (limited to 1000) items. Sort the list by routes with the lowest number of peer hops first, to try and prioritize those which are easiest and quickest to reach. 2022-03-04 13:33:17 +00:00
CalDescent
35d9a10cf4 Avoid logging if there are no remaining transaction signatures to validate. There was too much log spam, none of which was particularly useful. 2022-03-04 12:03:58 +00:00
CalDescent
7c181379b4 Added more granularity to logging, to differentiate between signature validation and general processing/importing, as well as showing counts of the transactions being processed in each round. 2022-03-04 11:12:23 +00:00
CalDescent
f9576d8afb Further optimizations to Controller.processIncomingTransactionsQueue()
- Signature validation is now able to run concurrently with synchronization, to reduce the chances of the queue building up, and to speed up the propagation of new transactions. There's no need to break out of the loop - or avoid looping in the first place - since signatures can be validated without holding the blockchain lock.
- A blockchain lock isn't even attempted if a sync request is pending.
2022-03-04 11:05:58 +00:00
CalDescent
6a8a113fa1 Merge pull request #74 from catbref/presence-txns-removal
PRESENCE transactions changed to always fail signature validation
2022-03-04 10:33:11 +00:00
CalDescent
ef59c34165 Added missing "break" which was causing additional unnecessary debug logging. Originally introduced due to a merge conflict with the metadata branch. 2022-03-04 10:28:44 +00:00
CalDescent
a19e1f06c0 Merge pull request #73 from catbref/incoming-txns-rework
Reworking of Controller.processIncomingTransactionsQueue()
2022-03-04 09:45:29 +00:00
catbref
a9371f0a90 In Controller.processIncomingTransactionsQueue(), don't bother with 2nd-phase of locking blockchain and importing if there are no valid signature transactions to actually import 2022-03-03 20:32:27 +00:00
catbref
a7a94e49e8 PRESENCE transactions changed to always fail signature validation 2022-03-03 20:25:58 +00:00
catbref
affd100298 Reworking of Controller.processIncomingTransactionsQueue()
Main changes are:
* Check transaction signature validity in initial round, without blockchain lock
* Convert List of incoming transactions to Map so we can record whether we have validated transaction signature before to save rechecking effort
* Add invalid signature transactions to invalidUnconfirmedTransactions map with INVALID_TRANSACTION_RECHECK_INTERVAL expiry (~60min)
* Other minor changes related to List->Map change and Java object synchronization
2022-03-03 20:21:04 +00:00
CalDescent
fd6ec301a4 Updated AdvancedInstaller project for v3.2.0 2022-03-03 20:02:30 +00:00
CalDescent
5666e6084b Bump version to 3.2.0 2022-03-02 20:04:49 +00:00
CalDescent
69309c437e Tightened up the content security policy for non HTML files. 2022-03-01 20:36:34 +00:00
CalDescent
e392e4d344 Allow eval(), setTimeout(), etc, to enable various QDN sites to function correctly. The existing sandboxing should be locking this down enough already. Limited to .html and .htm files only. 2022-03-01 20:35:56 +00:00
CalDescent
bd53856927 Disabled auto fetching of metadata. To be re-enabled at a later date. 2022-03-01 20:26:09 +00:00
CalDescent
cbd1018ecf Allow identical data to be published if the metadata differs. 2022-03-01 20:22:47 +00:00
CalDescent
46606152eb /arbitrary/metadata/* endpoint now returns ArbitraryResourceMetadata rather than a raw JSON string. 2022-03-01 20:22:20 +00:00
CalDescent
e6f93e0a08 Added categoryName to ArbitraryResourceMetadata, along with the existing category ID 2022-03-01 20:19:08 +00:00
CalDescent
8d81f1822f Merge branch 'master' into qdn-metadata
# Conflicts:
#	src/main/java/org/qortal/controller/Controller.java
#	src/main/java/org/qortal/network/message/Message.java
2022-02-28 20:10:39 +00:00
CalDescent
5903607363 Merge pull request #72 from catbref/presence-v2
Presence v2
2022-02-27 22:01:59 +00:00
catbref
590a8f52db Remove future work comment from Controller 2022-02-27 16:57:26 +00:00
catbref
ecac47d1bc Also notify TradePresenceWebsocket (using TradePresenceEvent) when bridging old PRESENCE txns 2022-02-27 16:56:17 +00:00
catbref
3b477ef637 Fix JAXB marshalling error (duplicate tradeAddress) in TradePresenceWebSocket. No need to send signature. Make sure publicKey is sent in Base58, not Base64. 2022-02-27 16:56:17 +00:00
catbref
e2ef5b2ef3 Missed change from last commit: incorrect logic in TradePresenceWebSocket 2022-02-27 16:56:17 +00:00
catbref
1d59feeb72 Created /websockets/crosschain/tradepresence to replace /websockets/presence 2022-02-27 16:55:30 +00:00
catbref
c53dd31765 Tidy up of trade presence timestamp generation & checking. Added tests. Renamed "online trades" to "trade presences" 2022-02-27 16:54:42 +00:00
catbref
4c02081992 Tidy up TradeBot presence logging. Decorate API endpoints /crosschain/tradeoffers and /crosschain/trade with presence expiry timestamps 2022-02-27 16:54:42 +00:00
catbref
cb57af3c53 Bugfixes to online trade sigs + bridging from PRESENCE transactions 2022-02-27 16:54:42 +00:00
catbref
01d810fc00 Initial effort at migrating PRESENCE transactions to dedicated network messages 2022-02-27 16:54:42 +00:00
CalDescent
8c2a9279ee Return metadata in various /arbitrary APIs if the "includemetadata" parameter is included.
This is very inefficient and will soon be replaced with dedicated ArbitraryResources / ArbitraryMetadata tables. But this is acceptable in the short term, especially if limit and offset are used.
2022-02-27 09:09:18 +00:00
CalDescent
0d65448f3d Request all metadata automatically. 2022-02-27 08:20:39 +00:00
CalDescent
9da2b3c11a Don't respond to file list requests with just the metadata file.
We have the separate metadata protocol for this now.
2022-02-27 07:28:11 +00:00
CalDescent
95400da977 Fixed typo in various tests (copy and paste error) 2022-02-26 22:10:55 +00:00
CalDescent
dc41dc4c69 Tags now use an array of strings, rather than a single string. 2022-02-26 22:09:07 +00:00
CalDescent
a5c11d4c23 Reduced "Ignoring hash list request" logs from DEBUG to TRACE 2022-02-26 16:10:44 +00:00
CalDescent
878394535e Improvements relating to fetching metadata
- Rate limiter is disabled when using the API
- fetchArbitraryMetadata() returns the actual metadata content rather than a boolean
- Exceptions are thrown on certain errors, rather than returning null
2022-02-26 16:10:26 +00:00
CalDescent
35dba27a55 Fixed issue due to not updating arbitraryMetadataRequests when receiving the metadata file. 2022-02-26 16:07:06 +00:00
CalDescent
f22ad13fa9 Merge branch 'master' into qdn-metadata
This involved a slight rewrite to remove the "includeMetadataOnly" boolean. Metadata is now always excluded, otherwise it complicates the caching too much.

# Conflicts:
#	src/main/java/org/qortal/api/resource/ArbitraryResource.java
#	src/main/java/org/qortal/controller/arbitrary/ArbitraryDataStorageManager.java
2022-02-26 14:39:20 +00:00
CalDescent
aa2e5cb87b Merge branch 'hosted-resources-search' 2022-02-26 14:05:52 +00:00
CalDescent
7740f3da7e Small formatting tweaks, for consistency with existing code. 2022-02-26 14:05:28 +00:00
CalDescent
badb576991 Fixed exception when identifier is null. Also handling null names as this may be a future scenario. 2022-02-26 14:04:35 +00:00
CalDescent
c65a63fc7e Fixed "query" parameter error in swagger documentation 2022-02-26 13:59:53 +00:00
CalDescent
0111747016 Added debug logging of new file list stats. 2022-02-25 13:30:07 +00:00
CalDescent
eac4b0d87b Maintain backwards support for pre-3.2.0 peers by only including new file list message params when sending to newer peers.
These params are optional and the process will function without them, just less efficiently.
2022-02-25 12:24:02 +00:00
CalDescent
3dadce4da4 Renamed a reference 2022-02-25 12:24:02 +00:00
CalDescent
1864468818 Prefer the route with the least number of hops when relaying. 2022-02-25 12:24:02 +00:00
CalDescent
1a59379162 Optionally include requestTime, requestHops, peerAddress, and isRelayPossible flag in ArbitraryDataFileListMessage 2022-02-25 12:24:02 +00:00
CalDescent
31d34c3946 Updated testnet documentation 2022-02-25 11:08:37 +00:00
CalDescent
3cc394f02d Merge pull request #70 from catbref/synchronizer-newchaintipevent
Modify TradeBot to trigger when chain tip changes instead of with every new block
2022-02-24 20:04:26 +00:00
CalDescent
53c4fe9e80 Fixed another ElectrumX issue found in unit tests.
Peers that were thought to be missing output address data may actually have just been using a different key - "address" instead of "addresses". Now reading the addresses from both keys, which may remove the need for the previously added checks.
2022-02-24 20:01:56 +00:00
CalDescent
d5521068b0 Fixed issue in earlier commit, found in unit tests. 2022-02-24 19:45:37 +00:00
CalDescent
a63ef4010d Disabled expired transaction data deletion code for now, as this was often causing data to be incorrectly deleted.
This will need to be re-enabled at some point, but only after it's modified to be much less aggressive.
2022-02-24 19:05:29 +00:00
CalDescent
cec3e86eef Merge pull request #71 from catbref/transferprivs-fix
Very slightly relax validity checks for TRANSFER_PRIVS…
2022-02-24 19:02:20 +00:00
catbref
8950bb7af9 Very slightly relax validity checks for TRANSFER_PRIVS to allow for skeletal account records, e.g. due to CHAT transactions, but account last reference still needs to be null. Example at block height 736196 / 7 2022-02-24 09:13:51 +00:00
catbref
9e6fe7ceb9 Modify TradeBot, some related websockets, to trigger when chain tip changes instead of with every new block 2022-02-24 09:06:21 +00:00
CalDescent
c333d18cd0 Merge branch 'segwit' 2022-02-23 20:07:07 +00:00
CalDescent
0271ef69c9 When submitting a new transaction, treat the chain as "synced" if the latest block is less than 30 minutes old. Increased from around 7.5 minutes. 2022-02-23 20:06:55 +00:00
CalDescent
2d493a4ea2 Added logging when no addresses are returned for a bitcoiny transaction output. 2022-02-23 09:29:16 +00:00
CalDescent
e339ab856f Skip over Electrum servers that don't return any output addresses. Hopeful fix for BTC transactions that report a zero value due to incomplete data being returned from certain ElectrumX peers. 2022-02-23 08:34:38 +00:00
proto
782904a971 improvement to the search on hosted resources
1) use the cached version instead of rescanning all the files
2) separating the loading (which include files scanning) and listing logic
2022-02-22 17:54:08 +01:00
proto
a3753c01bc Add search functionality to hosted resources 2022-02-22 15:50:46 +01:00
CalDescent
d5c3921846 Only show the red "synchronizing" systray icon if the latest block isn't recent.
This should fix issue where the icon unnecessarily jumps between synced and synchronizing.
2022-02-21 22:34:13 +00:00
CalDescent
a2c462b3da Add <meta charset="UTF-8"> tag to websites. Fixed issue rendering emojis and other special characters. 2022-02-21 22:28:59 +00:00
CalDescent
8673c7ef6e Fixed bug in GET /peers/summary API 2022-02-21 22:28:18 +00:00
CalDescent
8d7be7757f Fixed incorrectly named tag. 2022-02-21 22:27:44 +00:00
proto
6b83927048 Persist MintingAccounts.json on minting accounts add/remove
this fix the behavior of the node, After adding or removing a minting account, allowing it to persist it to the backup folder
2022-02-21 16:17:17 +01:00
proto
e07adbd60e online accounts api call, fix level zero accounts
Added online zero level accounts to the response of /addresses/online/levels api endpoints
2022-02-21 15:40:10 +01:00
CalDescent
7798b8dcdc Keep items in arbitraryDataFileHashResponses if they are currently being requested by another thread. This should help to locate the higher numbered chunks from larger resources. 2022-02-20 11:33:09 +00:00
CalDescent
146e7970bf Synchronize this.allKnownPeers and this.connectedPeers in Network.requestDataFromPeer(), to make the method thread-safe.
This could be further improved by taking an immutable copy, but I'll leave this until the same approach is applied to other Network methods.
2022-02-20 11:04:33 +00:00
CalDescent
f4f7cc58e3 Removed unused import. 2022-02-20 10:44:59 +00:00
CalDescent
21b4b494e7 Renamed method. 2022-02-20 10:44:38 +00:00
CalDescent
7307844bee If UPnP is disabled in settings, close the existing external listenPort if a UPnP rule exists. 2022-02-20 10:44:20 +00:00
CalDescent
5d419dd4ec Handle case where funds are sent to and from the same bitcoiny deterministic wallet. 2022-02-19 17:45:24 +00:00
CalDescent
6d0db7cc5e Catch UncheckedIOException in findAllHostedPaths() which was seen when a file was deleted by another thread. 2022-02-19 17:18:51 +00:00
CalDescent
8de606588c Attempt to open the listen port (default 12392) using UPnP, if the local network supports it. 2022-02-18 20:11:00 +00:00
CalDescent
5842b1272d Add WaifUPnP-1.1 jar to project.
For future reference, the command used was:

mvn install:install-file -Dfile=/Users/user/Downloads/waifupnp-1.1/WaifUPnP.jar -DgroupId=com.dosse -DartifactId=WaifUPnP -Dversion=1.1 -Dpackaging=jar -DlocalRepositoryPath=lib
2022-02-18 20:05:14 +00:00
CalDescent
35b0a85818 Increased WALLET_KEY_LOOKAHEAD_INCREMENT_BITCOINJ to 50 2022-02-18 18:42:06 +00:00
CalDescent
fcdd85af6c Try a lookahead size of 20 (instead of 3) when asking Bitcoinj for the balance. 2022-02-18 18:39:47 +00:00
CalDescent
5aac2dc9df ONLINE_ACCOUNTS_V2_PEER_VERSION set to 3.2.0 2022-02-18 18:34:00 +00:00
CalDescent
17a9b4e442 Merge pull request #66 from catbref/network-online-accounts-v2
New network messages ONLINE_ACCOUNTS_V2 and GET_ONLINE_ACCOUNTS_V2.
2022-02-18 18:28:52 +00:00
CalDescent
becb0b37e6 Merge branch 'master' into network-online-accounts-v2 2022-02-18 17:12:44 +00:00
CalDescent
67ca876567 Log if we're unable to process the received file. 2022-02-18 15:35:37 +00:00
CalDescent
464ce66fd5 Moved deletion retry code into ArbitraryDataFile 2022-02-18 15:09:50 +00:00
CalDescent
3e505481fe Default minPeerVersion also increased to 3.1.0 2022-02-18 14:50:46 +00:00
CalDescent
c90c3a183e Block peers below 3.1.0 2022-02-18 14:50:29 +00:00
CalDescent
d1a7e734dc Updated AdvancedInstaller project for v3.1.1 2022-02-15 23:36:38 +00:00
CalDescent
acddf36467 Handle missing includeMetadata parameter. 2022-02-13 19:27:12 +00:00
CalDescent
166d32032a Fixed message IDs. 2022-02-13 19:22:20 +00:00
CalDescent
e4238a62c9 Exclude metadata-only transactions in the data management page (but added an API parameter to allow them to optionally be included).
This ensures that the list will only show resources where there is at least 1 chunk.
2022-02-13 19:21:16 +00:00
CalDescent
ad9c466712 Fall back to UNCATEGORIZED if the parsed category doesn't match any available categories.
This allows for deletion of categories, as the resources will just move into UNCATEGORIZED until they are next updated.
2022-02-13 18:10:56 +00:00
CalDescent
a3d31bbaf1 Category updates based on feedback so far. 2022-02-13 17:56:47 +00:00
CalDescent
4821139501 Merge branch 'master' into qdn-metadata
# Conflicts:
#	src/main/java/org/qortal/controller/arbitrary/ArbitraryDataFileListManager.java
2022-02-13 15:50:12 +00:00
catbref
244d4f78e2 New network messages ONLINE_ACCOUNTS_V2 and GET_ONLINE_ACCOUNTS_V2.
Increased GetOnlineAccountsMessage.MAX_ACCOUNT_COUNT from 1000 to 5000.

The V2 versions are more efficiently encoded and also cache the payload bytes
which reduces CPU when sending to multiple peers.

Serialization / deserialization unit tests included.

Tentative V2 message activation set at core version 3.1.2
see Controller.ONLINE_ACCOUNTS_V2_PEER_VERSION
2022-01-23 16:24:10 +00:00
CalDescent
dedf65bd4b Added initial protocol methods for metadata requests and forwarding. Not tested yet. 2022-01-22 20:24:37 +00:00
CalDescent
a79ed02ccf Added initial (unfinished) category list, as well as the GET /arbitrary/categories API, and converted the category field from a string to an enum 2022-01-22 12:11:16 +00:00
CalDescent
79f87babdf Limit the metadata string lengths 2022-01-21 22:52:31 +00:00
CalDescent
f296d5138b Allow metadata to optionally be included with any arbitrary resource. 2022-01-21 21:14:28 +00:00
catbref
eb9b94b9c6 Add Qortal AT FunctionCodes for getting account level / blocks minted + tests 2021-12-04 16:36:05 +00:00
CalDescent
e505067759 Added support for Native SegWit (Bech32) addresses, which have the prefixes "ltc1" and "bc1". Bitcoinj supports these automatically, as long as fromString() is used instead of fromBase58(), which is already the case. Tested on LTC mainnet, and BTC testnet. 2021-02-21 12:02:26 +00:00
267 changed files with 21194 additions and 4393 deletions

1
.gitignore vendored
View File

@@ -28,6 +28,7 @@
/WindowsInstaller/Install Files/qortal.jar
/*.7z
/tmp
/wallets
/data*
/src/test/resources/arbitrary/*/.qortal/cache
apikey.txt

View File

@@ -41,13 +41,39 @@
- Start up at least as many nodes as `minBlockchainPeers` (or adjust this value instead)
- Probably best to perform API call `DELETE /peers/known`
- Add other nodes via API call `POST /peers <peer-hostname-or-IP>`
- Add minting private key to node(s) via API call `POST /admin/mintingaccounts <minting-private-key>`
This key must have corresponding `REWARD_SHARE` transaction in testnet genesis block
- Add minting private key to nodes via API call `POST /admin/mintingaccounts <minting-private-key>`
The keys must have corresponding `REWARD_SHARE` transactions in testnet genesis block
- You must have at least 2 separate minting keys and two separate nodes. Assign one minting key to each node.
- Alternatively, comment out the `if (mintedLastBlock) { }` conditional in BlockMinter.java to allow for a single node and key.
- Wait for genesis block timestamp to pass
- A node should mint block 2 approximately 60 seconds after genesis block timestamp
- Other testnet nodes will sync *as long as there is at least `minBlockchainPeers` peers with an "up-to-date" chain`
- You can also use API call `POST /admin/forcesync <connected-peer-IP-and-port>` on stuck nodes
## Single-node testnet
A single-node testnet is possible with code modifications, for basic testing, or to more easily start a new testnet.
To do so, follow these steps:
- Comment out the `if (mintedLastBlock) { }` conditional in BlockMinter.java
- Comment out the `minBlockchainPeers` validation in Settings.validate()
- Set `minBlockchainPeers` to 0 in settings.json
- Set `Synchronizer.RECOVERY_MODE_TIMEOUT` to `0`
- All other steps should remain the same. Only a single reward share key is needed.
- Remember to put these values back after introducing other nodes
## Fixed network
To restrict a testnet to a set of private nodes, you can use the "fixed network" feature.
This ensures that the testnet nodes only communicate with each other and not other known peers.
To do this, add the following setting to each testnet node, substituting the IP addresses:
```
"fixedNetwork": [
"192.168.0.101:62392",
"192.168.0.102:62392",
"192.168.0.103:62392"
]
```
## Dealing with stuck chain
Maybe your nodes have been offline and no-one has minted a recent testnet block.

View File

@@ -17,10 +17,10 @@
<ROW Property="Manufacturer" Value="Qortal"/>
<ROW Property="MsiLogging" MultiBuildValue="DefaultBuild:vp"/>
<ROW Property="NTP_GOOD" Value="false"/>
<ROW Property="ProductCode" Value="1033:{8CB28EE4-A407-4CED-A7C5-5138BCBF4B13} 1049:{C416F381-0C39-459C-A571-AC686D57B808} 2052:{A3DBA289-D2F0-4A3F-94DC-B89F5A6465DE} 2057:{893B13A5-45B6-4E49-8690-3FE46BCA50DF} " Type="16"/>
<ROW Property="ProductCode" Value="1033:{4FED6490-EFD9-4670-A497-4E0778B72720} 1049:{059E82DD-E2D3-40EC-AF0C-259FB68F7BEE} 2052:{0571ADAA-B818-4AA9-9320-57267B44E5C7} 2057:{92A305F9-3C4A-4800-9977-10DBB6B8F30A} " Type="16"/>
<ROW Property="ProductLanguage" Value="2057"/>
<ROW Property="ProductName" Value="Qortal"/>
<ROW Property="ProductVersion" Value="3.1.0" Type="32"/>
<ROW Property="ProductVersion" Value="3.3.3" Type="32"/>
<ROW Property="RECONFIG_NTP" Value="true"/>
<ROW Property="REMOVE_BLOCKCHAIN" Value="YES" Type="4"/>
<ROW Property="REPAIR_BLOCKCHAIN" Value="YES" Type="4"/>
@@ -212,7 +212,7 @@
<ROW Component="ADDITIONAL_LICENSE_INFO_71" ComponentId="{12A3ADBE-BB7A-496C-8869-410681E6232F}" Directory_="jdk.zipfs_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_71" Type="0"/>
<ROW Component="ADDITIONAL_LICENSE_INFO_8" ComponentId="{D53AD95E-CF96-4999-80FC-5812277A7456}" Directory_="java.naming_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_8" Type="0"/>
<ROW Component="ADDITIONAL_LICENSE_INFO_9" ComponentId="{6B7EA9B0-5D17-47A8-B78C-FACE86D15E01}" Directory_="java.net.http_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_9" Type="0"/>
<ROW Component="AI_CustomARPName" ComponentId="{551D2FA5-BE83-4DA1-A96E-7D15C517F9FF}" Directory_="APPDIR" Attributes="260" KeyPath="DisplayName" Options="1"/>
<ROW Component="AI_CustomARPName" ComponentId="{28DC06DF-3CBA-4275-A907-3BD3F8E6AD61}" Directory_="APPDIR" Attributes="260" KeyPath="DisplayName" Options="1"/>
<ROW Component="AI_ExePath" ComponentId="{3644948D-AE0B-41BB-9FAF-A79E70490A08}" Directory_="APPDIR" Attributes="260" KeyPath="AI_ExePath"/>
<ROW Component="APPDIR" ComponentId="{680DFDDE-3FB4-47A5-8FF5-934F576C6F91}" Directory_="APPDIR" Attributes="0"/>
<ROW Component="AccessBridgeCallbacks.h" ComponentId="{288055D1-1062-47A3-AA44-5601B4E38AED}" Directory_="bridge_Dir" Attributes="0" KeyPath="AccessBridgeCallbacks.h" Type="0"/>

Binary file not shown.

View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dosse</groupId>
<artifactId>WaifUPnP</artifactId>
<version>1.1</version>
<description>POM was created from install:install-file</description>
</project>

View File

@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>com.dosse</groupId>
<artifactId>WaifUPnP</artifactId>
<versioning>
<release>1.1</release>
<versions>
<version>1.1</version>
</versions>
<lastUpdated>20220218200127</lastUpdated>
</versioning>
</metadata>

28
pom.xml
View File

@@ -3,11 +3,11 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.qortal</groupId>
<artifactId>qortal</artifactId>
<version>3.1.1</version>
<version>3.3.5</version>
<packaging>jar</packaging>
<properties>
<skipTests>true</skipTests>
<altcoinj.version>bf9fb80</altcoinj.version>
<altcoinj.version>6628cfd</altcoinj.version>
<bitcoinj.version>0.15.10</bitcoinj.version>
<bouncycastle.version>1.64</bouncycastle.version>
<build.timestamp>${maven.build.timestamp}</build.timestamp>
@@ -21,6 +21,9 @@
<dagger.version>1.2.2</dagger.version>
<guava.version>28.1-jre</guava.version>
<hsqldb.version>2.5.1</hsqldb.version>
<homoglyph.version>1.2.1</homoglyph.version>
<icu4j.version>70.1</icu4j.version>
<upnp.version>1.1</upnp.version>
<jersey.version>2.29.1</jersey.version>
<jetty.version>9.4.29.v20200521</jetty.version>
<log4j.version>2.17.1</log4j.version>
@@ -427,6 +430,12 @@
<artifactId>AT</artifactId>
<version>${ciyam-at.version}</version>
</dependency>
<!-- UPnP support -->
<dependency>
<groupId>com.dosse</groupId>
<artifactId>WaifUPnP</artifactId>
<version>${upnp.version}</version>
</dependency>
<!-- Bitcoin support -->
<dependency>
<groupId>org.bitcoinj</groupId>
@@ -435,7 +444,7 @@
</dependency>
<!-- For Litecoin, etc. support, requires bitcoinj -->
<dependency>
<groupId>com.github.jjos2372</groupId>
<groupId>com.github.qortal</groupId>
<artifactId>altcoinj</artifactId>
<version>${altcoinj.version}</version>
</dependency>
@@ -561,7 +570,18 @@
<dependency>
<groupId>net.codebox</groupId>
<artifactId>homoglyph</artifactId>
<version>1.2.0</version>
<version>${homoglyph.version}</version>
</dependency>
<!-- Unicode support -->
<dependency>
<groupId>com.ibm.icu</groupId>
<artifactId>icu4j</artifactId>
<version>${icu4j.version}</version>
</dependency>
<dependency>
<groupId>com.ibm.icu</groupId>
<artifactId>icu4j-charset</artifactId>
<version>${icu4j.version}</version>
</dependency>
<!-- Jetty -->
<dependency>

View File

@@ -37,7 +37,7 @@ public class ApplyUpdate {
private static final String JAVA_TOOL_OPTIONS_NAME = "JAVA_TOOL_OPTIONS";
private static final String JAVA_TOOL_OPTIONS_VALUE = "-XX:MaxRAMFraction=4";
private static final long CHECK_INTERVAL = 10 * 1000L; // ms
private static final long CHECK_INTERVAL = 30 * 1000L; // ms
private static final int MAX_ATTEMPTS = 12;
public static void main(String[] args) {

View File

@@ -8,11 +8,13 @@ import javax.xml.bind.annotation.XmlAccessorType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.block.BlockChain;
import org.qortal.controller.LiteNode;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.data.account.AccountData;
import org.qortal.data.account.RewardShareData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.settings.Settings;
import org.qortal.utils.Base58;
@XmlAccessorType(XmlAccessType.NONE) // Stops JAX-RS errors when unmarshalling blockchain config
@@ -59,7 +61,17 @@ public class Account {
// Balance manipulations - assetId is 0 for QORT
public long getConfirmedBalance(long assetId) throws DataException {
AccountBalanceData accountBalanceData = this.repository.getAccountRepository().getBalance(this.address, assetId);
AccountBalanceData accountBalanceData;
if (Settings.getInstance().isLite()) {
// Lite nodes request data from peers instead of the local db
accountBalanceData = LiteNode.getInstance().fetchAccountBalance(this.address, assetId);
}
else {
// All other node types fetch from the local db
accountBalanceData = this.repository.getAccountRepository().getBalance(this.address, assetId);
}
if (accountBalanceData == null)
return 0;
@@ -205,6 +217,12 @@ public class Account {
return false;
}
/** Returns account's blockMinted (0+) or null if account not found in repository. */
public Integer getBlocksMinted() throws DataException {
return this.repository.getAccountRepository().getMintedBlockCount(this.address);
}
/** Returns whether account can build reward-shares.
* <p>
* To be able to create reward-shares, the account needs to pass at least one of these tests:<br>
@@ -272,7 +290,7 @@ public class Account {
/**
* Returns 'effective' minting level, or zero if reward-share does not exist.
* <p>
* For founder accounts, this returns "founderEffectiveMintingLevel" from blockchain config.
* this is being used on src/main/java/org/qortal/api/resource/AddressesResource.java to fulfil the online accounts api call
*
* @param repository
* @param rewardSharePublicKey
@@ -288,5 +306,26 @@ public class Account {
Account rewardShareMinter = new Account(repository, rewardShareData.getMinter());
return rewardShareMinter.getEffectiveMintingLevel();
}
/**
* Returns 'effective' minting level, with a fix for the zero level.
* <p>
* For founder accounts, this returns "founderEffectiveMintingLevel" from blockchain config.
*
* @param repository
* @param rewardSharePublicKey
* @return 0+
* @throws DataException
*/
public static int getRewardShareEffectiveMintingLevelIncludingLevelZero(Repository repository, byte[] rewardSharePublicKey) throws DataException {
// Find actual minter and get their effective minting level
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(rewardSharePublicKey);
if (rewardShareData == null)
return 0;
else if(!rewardShareData.getMinter().equals(rewardShareData.getRecipient()))//the minter is different than the recipient this means sponsorship
return 0;
Account rewardShareMinter = new Account(repository, rewardShareData.getMinter());
return rewardShareMinter.getEffectiveMintingLevel();
}
}

View File

@@ -40,13 +40,7 @@ import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.servlet.ServletContainer;
import org.qortal.api.resource.AnnotationPostProcessor;
import org.qortal.api.resource.ApiDefinition;
import org.qortal.api.websocket.ActiveChatsWebSocket;
import org.qortal.api.websocket.AdminStatusWebSocket;
import org.qortal.api.websocket.BlocksWebSocket;
import org.qortal.api.websocket.ChatMessagesWebSocket;
import org.qortal.api.websocket.PresenceWebSocket;
import org.qortal.api.websocket.TradeBotWebSocket;
import org.qortal.api.websocket.TradeOffersWebSocket;
import org.qortal.api.websocket.*;
import org.qortal.settings.Settings;
public class ApiService {
@@ -212,6 +206,9 @@ public class ApiService {
context.addServlet(ChatMessagesWebSocket.class, "/websockets/chat/messages");
context.addServlet(TradeOffersWebSocket.class, "/websockets/crosschain/tradeoffers");
context.addServlet(TradeBotWebSocket.class, "/websockets/crosschain/tradebot");
context.addServlet(TradePresenceWebSocket.class, "/websockets/crosschain/tradepresence");
// Deprecated
context.addServlet(PresenceWebSocket.class, "/websockets/presence");
// Start server

View File

@@ -28,6 +28,11 @@ public class HTMLParser {
// Add base href tag
String baseElement = String.format("<base href=\"%s\">", baseUrl);
head.get(0).prepend(baseElement);
// Add meta charset tag
String metaCharsetElement = "<meta charset=\"UTF-8\">";
head.get(0).prepend(metaCharsetElement);
}
String html = document.html();
this.data = html.getBytes();

View File

@@ -17,7 +17,7 @@ import java.util.Map;
@Path("/")
@Tag(name = "Gateway")
@Tag(name = "Domain Map")
public class DomainMapResource {
@Context HttpServletRequest request;

View File

@@ -12,6 +12,7 @@ public class NodeInfo {
public long buildTimestamp;
public String nodeId;
public boolean isTestNet;
public String type;
public NodeInfo() {
}

View File

@@ -24,9 +24,9 @@ public class NodeStatus {
this.isMintingPossible = Controller.getInstance().isMintingPossible();
this.syncPercent = Synchronizer.getInstance().getSyncPercent();
this.isSynchronizing = this.syncPercent != null;
this.isSynchronizing = Synchronizer.getInstance().isSynchronizing();
this.numberOfConnections = Network.getInstance().getHandshakedPeers().size();
this.numberOfConnections = Network.getInstance().getImmutableHandshakedPeers().size();
this.height = Controller.getInstance().getChainHeight();
}

View File

@@ -0,0 +1,29 @@
package org.qortal.api.model.crosschain;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class DigibyteSendRequest {
@Schema(description = "Digibyte BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
public String xprv58;
@Schema(description = "Recipient's Digibyte address ('legacy' P2PKH only)", example = "1DigByteEaterAddressDontSendf59kuE")
public String receivingAddress;
@Schema(description = "Amount of DGB to send", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long digibyteAmount;
@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 DGB (100 sats) per byte", example = "0.00000100", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public Long feePerByte;
public DigibyteSendRequest() {
}
}

View File

@@ -0,0 +1,29 @@
package org.qortal.api.model.crosschain;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import io.swagger.v3.oas.annotations.media.Schema;
@XmlAccessorType(XmlAccessType.FIELD)
public class RavencoinSendRequest {
@Schema(description = "Ravencoin BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
public String xprv58;
@Schema(description = "Recipient's Ravencoin address ('legacy' P2PKH only)", example = "1RvnCoinEaterAddressDontSendf59kuE")
public String receivingAddress;
@Schema(description = "Amount of RVN to send", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long ravencoinAmount;
@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 RVN (100 sats) per byte", example = "0.00000100", type = "number")
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public Long feePerByte;
public RavencoinSendRequest() {
}
}

View File

@@ -30,7 +30,8 @@ import org.qortal.api.Security;
import org.qortal.api.model.ApiOnlineAccount;
import org.qortal.api.model.RewardShareKeyRequest;
import org.qortal.asset.Asset;
import org.qortal.controller.Controller;
import org.qortal.controller.LiteNode;
import org.qortal.controller.OnlineAccountsManager;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountData;
import org.qortal.data.account.RewardShareData;
@@ -109,18 +110,26 @@ public class AddressesResource {
if (!Crypto.isValidAddress(address))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
byte[] lastReference = null;
AccountData accountData;
try (final Repository repository = RepositoryManager.getRepository()) {
AccountData accountData = repository.getAccountRepository().getAccount(address);
// Not found?
if (accountData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
lastReference = accountData.getReference();
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
if (Settings.getInstance().isLite()) {
// Lite nodes request data from peers instead of the local db
accountData = LiteNode.getInstance().fetchAccountData(address);
}
else {
// All other node types request data from local db
try (final Repository repository = RepositoryManager.getRepository()) {
accountData = repository.getAccountRepository().getAccount(address);
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
// Not found?
if (accountData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
byte[] lastReference = accountData.getReference();
if (lastReference == null || lastReference.length == 0)
return "false";
@@ -156,7 +165,7 @@ public class AddressesResource {
)
@ApiErrors({ApiError.PUBLIC_KEY_NOT_FOUND, ApiError.REPOSITORY_ISSUE})
public List<ApiOnlineAccount> getOnlineAccounts() {
List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
List<OnlineAccountData> onlineAccounts = OnlineAccountsManager.getInstance().getOnlineAccounts();
// Map OnlineAccountData entries to OnlineAccount via reward-share data
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -191,14 +200,14 @@ public class AddressesResource {
)
@ApiErrors({ApiError.PUBLIC_KEY_NOT_FOUND, ApiError.REPOSITORY_ISSUE})
public List<OnlineAccountLevel> getOnlineAccountsByLevel() {
List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
List<OnlineAccountData> onlineAccounts = OnlineAccountsManager.getInstance().getOnlineAccounts();
try (final Repository repository = RepositoryManager.getRepository()) {
List<OnlineAccountLevel> onlineAccountLevels = new ArrayList<>();
for (OnlineAccountData onlineAccountData : onlineAccounts) {
try {
final int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, onlineAccountData.getPublicKey());
final int minterLevel = Account.getRewardShareEffectiveMintingLevelIncludingLevelZero(repository, onlineAccountData.getPublicKey());
OnlineAccountLevel onlineAccountLevel = onlineAccountLevels.stream()
.filter(a -> a.getLevel() == minterLevel)

View File

@@ -35,7 +35,6 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.appender.RollingFileAppender;
import org.checkerframework.checker.units.qual.A;
import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.api.*;
@@ -120,10 +119,23 @@ public class AdminResource {
nodeInfo.buildTimestamp = Controller.getInstance().getBuildTimestamp();
nodeInfo.nodeId = Network.getInstance().getOurNodeId();
nodeInfo.isTestNet = Settings.getInstance().isTestNet();
nodeInfo.type = getNodeType();
return nodeInfo;
}
private String getNodeType() {
if (Settings.getInstance().isTopOnly()) {
return "topOnly";
}
else if (Settings.getInstance().isLite()) {
return "lite";
}
else {
return "full";
}
}
@GET
@Path("/status")
@Operation(
@@ -315,6 +327,7 @@ public class AdminResource {
repository.getAccountRepository().save(mintingAccountData);
repository.saveChanges();
repository.exportNodeLocalData();//after adding new minting account let's persist it to the backup MintingAccounts.json
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY, e);
} catch (DataException e) {
@@ -355,6 +368,7 @@ public class AdminResource {
return "false";
repository.saveChanges();
repository.exportNodeLocalData();//after removing new minting account let's persist it to the backup MintingAccounts.json
} catch (IllegalArgumentException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY, e);
} catch (DataException e) {
@@ -380,6 +394,10 @@ public class AdminResource {
) @QueryParam("limit") Integer limit, @Parameter(
ref = "offset"
) @QueryParam("offset") Integer offset, @Parameter(
name = "tail",
description = "Fetch most recent log lines",
schema = @Schema(type = "boolean")
) @QueryParam("tail") Boolean tail, @Parameter(
ref = "reverse"
) @QueryParam("reverse") Boolean reverse) {
LoggerContext loggerContext = (LoggerContext) LogManager.getContext();
@@ -395,6 +413,13 @@ public class AdminResource {
if (reverse != null && reverse)
logLines = Lists.reverse(logLines);
// Tail mode - return the last X lines (where X = limit)
if (tail != null && tail) {
if (limit != null && limit > 0) {
offset = logLines.size() - limit;
}
}
// offset out of bounds?
if (offset != null && (offset < 0 || offset >= logLines.size()))
return "";
@@ -415,7 +440,7 @@ public class AdminResource {
limit = Math.min(limit, logLines.size());
logLines.subList(limit - 1, logLines.size()).clear();
logLines.subList(limit, logLines.size()).clear();
return String.join("\n", logLines);
} catch (IOException e) {
@@ -512,7 +537,7 @@ public class AdminResource {
PeerAddress peerAddress = PeerAddress.fromString(targetPeerAddress);
InetSocketAddress resolvedAddress = peerAddress.toSocketAddress();
List<Peer> peers = Network.getInstance().getHandshakedPeers();
List<Peer> peers = Network.getInstance().getImmutableHandshakedPeers();
Peer targetPeer = peers.stream().filter(peer -> peer.getResolvedAddress().equals(resolvedAddress)).findFirst().orElse(null);
if (targetPeer == null)
@@ -546,7 +571,7 @@ public class AdminResource {
@Path("/repository/data")
@Operation(
summary = "Export sensitive/node-local data from repository.",
description = "Exports data to .script files on local machine"
description = "Exports data to .json files on local machine"
)
@ApiErrors({ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE})
@SecurityRequirement(name = "apiKey")
@@ -587,10 +612,6 @@ public class AdminResource {
public String importRepository(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String filename) {
Security.checkApiCallAllowed(request);
// Hard-coded because it's too dangerous to allow user-supplied filenames in weaker security contexts
if (Settings.getInstance().getApiKey() == null)
filename = "qortal-backup/TradeBotStates.json";
try (final Repository repository = RepositoryManager.getRepository()) {
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();

View File

@@ -12,6 +12,7 @@ import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
@@ -33,13 +34,14 @@ import org.qortal.api.resource.TransactionsResource.ConfirmationStatus;
import org.qortal.arbitrary.*;
import org.qortal.arbitrary.ArbitraryDataFile.ResourceIdType;
import org.qortal.arbitrary.exception.MissingDataException;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.arbitrary.misc.Category;
import org.qortal.arbitrary.misc.Service;
import org.qortal.controller.Controller;
import org.qortal.controller.arbitrary.ArbitraryDataStorageManager;
import org.qortal.controller.arbitrary.ArbitraryMetadataManager;
import org.qortal.data.account.AccountData;
import org.qortal.data.arbitrary.ArbitraryResourceInfo;
import org.qortal.data.arbitrary.ArbitraryResourceNameInfo;
import org.qortal.data.arbitrary.ArbitraryResourceStatus;
import org.qortal.data.arbitrary.*;
import org.qortal.data.naming.NameData;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.data.transaction.TransactionData;
@@ -88,7 +90,8 @@ public class ArbitraryResource {
@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse,
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus) {
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata) {
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -110,9 +113,12 @@ public class ArbitraryResource {
return new ArrayList<>();
}
if (includeStatus != null && includeStatus == true) {
if (includeStatus != null && includeStatus) {
resources = this.addStatusToResources(resources);
}
if (includeMetadata != null && includeMetadata) {
resources = this.addMetadataToResources(resources);
}
return resources;
@@ -140,7 +146,8 @@ public class ArbitraryResource {
@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse,
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus) {
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata) {
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -153,9 +160,12 @@ public class ArbitraryResource {
return new ArrayList<>();
}
if (includeStatus != null && includeStatus == true) {
if (includeStatus != null && includeStatus) {
resources = this.addStatusToResources(resources);
}
if (includeMetadata != null && includeMetadata) {
resources = this.addMetadataToResources(resources);
}
return resources;
@@ -182,7 +192,8 @@ public class ArbitraryResource {
@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse,
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus) {
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata) {
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -206,9 +217,13 @@ public class ArbitraryResource {
List<ArbitraryResourceInfo> resources = repository.getArbitraryRepository()
.getArbitraryResources(service, identifier, name, defaultRes, null, null, reverse);
if (includeStatus != null && includeStatus == true) {
if (includeStatus != null && includeStatus) {
resources = this.addStatusToResources(resources);
}
if (includeMetadata != null && includeMetadata) {
resources = this.addMetadataToResources(resources);
}
creatorName.resources = resources;
}
}
@@ -390,6 +405,28 @@ public class ArbitraryResource {
return Settings.getInstance().isRelayModeEnabled();
}
@GET
@Path("/categories")
@Operation(
summary = "List arbitrary transaction categories",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.APPLICATION_JSON, schema = @Schema(implementation = ArbitraryCategoryInfo.class))
)
}
)
@ApiErrors({ApiError.REPOSITORY_ISSUE})
public List<ArbitraryCategoryInfo> getCategories() {
List<ArbitraryCategoryInfo> categories = new ArrayList<>();
for (Category category : Category.values()) {
ArbitraryCategoryInfo arbitraryCategory = new ArbitraryCategoryInfo();
arbitraryCategory.id = category.toString();
arbitraryCategory.name = category.getName();
categories.add(arbitraryCategory);
}
return categories;
}
@GET
@Path("/hosted/transactions")
@Operation(
@@ -431,15 +468,24 @@ public class ArbitraryResource {
public List<ArbitraryResourceInfo> getHostedResources(
@HeaderParam(Security.API_KEY_HEADER) String apiKey,
@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata,
@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
@Parameter(ref = "offset") @QueryParam("offset") Integer offset) {
@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
@QueryParam("query") String query) {
Security.checkApiCallAllowed(request);
List<ArbitraryResourceInfo> resources = new ArrayList<>();
try (final Repository repository = RepositoryManager.getRepository()) {
List<ArbitraryTransactionData> transactionDataList;
if (query == null || query.equals("")) {
transactionDataList = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository, limit, offset);
} else {
transactionDataList = ArbitraryDataStorageManager.getInstance().searchHostedTransactions(repository,query, limit, offset);
}
List<ArbitraryTransactionData> transactionDataList = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository, limit, offset);
for (ArbitraryTransactionData transactionData : transactionDataList) {
ArbitraryResourceInfo arbitraryResourceInfo = new ArbitraryResourceInfo();
arbitraryResourceInfo.name = transactionData.getName();
@@ -450,9 +496,12 @@ public class ArbitraryResource {
}
}
if (includeStatus != null && includeStatus == true) {
if (includeStatus != null && includeStatus) {
resources = this.addStatusToResources(resources);
}
if (includeMetadata != null && includeMetadata) {
resources = this.addMetadataToResources(resources);
}
return resources;
@@ -461,6 +510,8 @@ public class ArbitraryResource {
}
}
@DELETE
@Path("/resource/{service}/{name}/{identifier}")
@Operation(
@@ -624,6 +675,54 @@ public class ArbitraryResource {
}
// Metadata
@GET
@Path("/metadata/{service}/{name}/{identifier}")
@Operation(
summary = "Fetch raw metadata from resource with supplied service, name, identifier, and relative path",
responses = {
@ApiResponse(
description = "Path to file structure containing requested data",
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = ArbitraryDataTransactionMetadata.class
)
)
)
}
)
@SecurityRequirement(name = "apiKey")
public ArbitraryResourceMetadata getMetadata(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
@PathParam("service") Service service,
@PathParam("name") String name,
@PathParam("identifier") String identifier) {
Security.checkApiCallAllowed(request);
ArbitraryDataResource resource = new ArbitraryDataResource(name, ResourceIdType.NAME, service, identifier);
try {
ArbitraryDataTransactionMetadata transactionMetadata = ArbitraryMetadataManager.getInstance().fetchMetadata(resource, false);
if (transactionMetadata != null) {
ArbitraryResourceMetadata resourceMetadata = ArbitraryResourceMetadata.fromTransactionMetadata(transactionMetadata);
if (resourceMetadata != null) {
return resourceMetadata;
}
else {
// The metadata file doesn't contain title, description, category, or tags
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FILE_NOT_FOUND);
}
}
} catch (IllegalArgumentException e) {
// No metadata exists for this resource
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FILE_NOT_FOUND, e.getMessage());
}
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
}
// Upload data at supplied path
@@ -656,6 +755,10 @@ public class ArbitraryResource {
public String post(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
@PathParam("service") String serviceString,
@PathParam("name") String name,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String path) {
Security.checkApiCallAllowed(request);
@@ -663,7 +766,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Path not supplied");
}
return this.upload(Service.valueOf(serviceString), name, null, path, null, null, false);
return this.upload(Service.valueOf(serviceString), name, null, path, null, null, false,
title, description, tags, category);
}
@POST
@@ -696,6 +800,10 @@ public class ArbitraryResource {
@PathParam("service") String serviceString,
@PathParam("name") String name,
@PathParam("identifier") String identifier,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String path) {
Security.checkApiCallAllowed(request);
@@ -703,7 +811,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Path not supplied");
}
return this.upload(Service.valueOf(serviceString), name, identifier, path, null, null, false);
return this.upload(Service.valueOf(serviceString), name, identifier, path, null, null, false,
title, description, tags, category);
}
@@ -737,6 +846,10 @@ public class ArbitraryResource {
public String postBase64EncodedData(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
@PathParam("service") String serviceString,
@PathParam("name") String name,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String base64) {
Security.checkApiCallAllowed(request);
@@ -744,7 +857,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
}
return this.upload(Service.valueOf(serviceString), name, null, null, null, base64, false);
return this.upload(Service.valueOf(serviceString), name, null, null, null, base64, false,
title, description, tags, category);
}
@POST
@@ -775,6 +889,10 @@ public class ArbitraryResource {
@PathParam("service") String serviceString,
@PathParam("name") String name,
@PathParam("identifier") String identifier,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String base64) {
Security.checkApiCallAllowed(request);
@@ -782,7 +900,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
}
return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64, false);
return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64, false,
title, description, tags, category);
}
@@ -815,6 +934,10 @@ public class ArbitraryResource {
public String postZippedData(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
@PathParam("service") String serviceString,
@PathParam("name") String name,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String base64Zip) {
Security.checkApiCallAllowed(request);
@@ -822,7 +945,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
}
return this.upload(Service.valueOf(serviceString), name, null, null, null, base64Zip, true);
return this.upload(Service.valueOf(serviceString), name, null, null, null, base64Zip, true,
title, description, tags, category);
}
@POST
@@ -853,6 +977,10 @@ public class ArbitraryResource {
@PathParam("service") String serviceString,
@PathParam("name") String name,
@PathParam("identifier") String identifier,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String base64Zip) {
Security.checkApiCallAllowed(request);
@@ -860,7 +988,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
}
return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64Zip, true);
return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64Zip, true,
title, description, tags, category);
}
@@ -896,6 +1025,10 @@ public class ArbitraryResource {
public String postString(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
@PathParam("service") String serviceString,
@PathParam("name") String name,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String string) {
Security.checkApiCallAllowed(request);
@@ -903,7 +1036,8 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data string not supplied");
}
return this.upload(Service.valueOf(serviceString), name, null, null, string, null, false);
return this.upload(Service.valueOf(serviceString), name, null, null, string, null, false,
title, description, tags, category);
}
@POST
@@ -936,6 +1070,10 @@ public class ArbitraryResource {
@PathParam("service") String serviceString,
@PathParam("name") String name,
@PathParam("identifier") String identifier,
@QueryParam("title") String title,
@QueryParam("description") String description,
@QueryParam("tags") List<String> tags,
@QueryParam("category") Category category,
String string) {
Security.checkApiCallAllowed(request);
@@ -943,13 +1081,16 @@ public class ArbitraryResource {
throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data string not supplied");
}
return this.upload(Service.valueOf(serviceString), name, identifier, null, string, null, false);
return this.upload(Service.valueOf(serviceString), name, identifier, null, string, null, false,
title, description, tags, category);
}
// Shared methods
private String upload(Service service, String name, String identifier, String path, String string, String base64, boolean zipped) {
private String upload(Service service, String name, String identifier,
String path, String string, String base64, boolean zipped,
String title, String description, List<String> tags, Category category) {
// Fetch public key from registered name
try (final Repository repository = RepositoryManager.getRepository()) {
NameData nameData = repository.getNameRepository().fromName(name);
@@ -1013,7 +1154,8 @@ public class ArbitraryResource {
try {
ArbitraryDataTransactionBuilder transactionBuilder = new ArbitraryDataTransactionBuilder(
repository, publicKey58, Paths.get(path), name, null, service, identifier
repository, publicKey58, Paths.get(path), name, null, service, identifier,
title, description, tags, category
);
transactionBuilder.build();
@@ -1124,12 +1266,34 @@ public class ArbitraryResource {
private List<ArbitraryResourceInfo> addStatusToResources(List<ArbitraryResourceInfo> resources) {
// Determine and add the status of each resource
List<ArbitraryResourceInfo> updatedResources = new ArrayList<>();
for (ArbitraryResourceInfo resourceInfo : resources) {
try {
ArbitraryDataResource resource = new ArbitraryDataResource(resourceInfo.name, ResourceIdType.NAME,
resourceInfo.service, resourceInfo.identifier);
ArbitraryResourceStatus status = resource.getStatus(true);
if (status != null) {
resourceInfo.status = status;
}
updatedResources.add(resourceInfo);
} catch (Exception e) {
// Catch and log all exceptions, since some systems are experiencing 500 errors when including statuses
LOGGER.info("Caught exception when adding status to resource %s: %s", resourceInfo, e.toString());
}
}
return updatedResources;
}
private List<ArbitraryResourceInfo> addMetadataToResources(List<ArbitraryResourceInfo> resources) {
// Add metadata fields to each resource if they exist
List<ArbitraryResourceInfo> updatedResources = new ArrayList<>();
for (ArbitraryResourceInfo resourceInfo : resources) {
ArbitraryDataResource resource = new ArbitraryDataResource(resourceInfo.name, ResourceIdType.NAME,
resourceInfo.service, resourceInfo.identifier);
ArbitraryResourceStatus status = resource.getStatus(true);
if (status != null) {
resourceInfo.status = status;
ArbitraryDataTransactionMetadata transactionMetadata = resource.getLatestTransactionMetadata();
ArbitraryResourceMetadata resourceMetadata = ArbitraryResourceMetadata.fromTransactionMetadata(transactionMetadata);
if (resourceMetadata != null) {
resourceInfo.metadata = resourceMetadata;
}
updatedResources.add(resourceInfo);
}

View File

@@ -122,7 +122,7 @@ public class CrossChainBitcoinResource {
@Path("/send")
@Operation(
summary = "Sends BTC from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently only supports 'legacy' P2PKH Bitcoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
description = "Currently supports 'legacy' P2PKH Bitcoin addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(

View File

@@ -0,0 +1,177 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.HeaderParam;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.bitcoinj.core.Transaction;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.crosschain.DigibyteSendRequest;
import org.qortal.crosschain.Digibyte;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.crosschain.SimpleTransaction;
@Path("/crosschain/dgb")
@Tag(name = "Cross-Chain (Digibyte)")
public class CrossChainDigibyteResource {
@Context
HttpServletRequest request;
@POST
@Path("/walletbalance")
@Operation(
summary = "Returns DGB balance for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
@SecurityRequirement(name = "apiKey")
public String getDigibyteWalletBalance(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
Security.checkApiCallAllowed(request);
Digibyte digibyte = Digibyte.getInstance();
if (!digibyte.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
Long balance = digibyte.getWalletBalanceFromTransactions(key58);
if (balance == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
return balance.toString();
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/wallettransactions")
@Operation(
summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
@SecurityRequirement(name = "apiKey")
public List<SimpleTransaction> getDigibyteWalletTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
Security.checkApiCallAllowed(request);
Digibyte digibyte = Digibyte.getInstance();
if (!digibyte.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
return digibyte.getWalletTransactions(key58);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/send")
@Operation(
summary = "Sends DGB from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently supports 'legacy' P2PKH Digibyte addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = DigibyteSendRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
@SecurityRequirement(name = "apiKey")
public String sendBitcoin(@HeaderParam(Security.API_KEY_HEADER) String apiKey, DigibyteSendRequest digibyteSendRequest) {
Security.checkApiCallAllowed(request);
if (digibyteSendRequest.digibyteAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (digibyteSendRequest.feePerByte != null && digibyteSendRequest.feePerByte <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Digibyte digibyte = Digibyte.getInstance();
if (!digibyte.isValidAddress(digibyteSendRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!digibyte.isValidDeterministicKey(digibyteSendRequest.xprv58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Transaction spendTransaction = digibyte.buildSpend(digibyteSendRequest.xprv58,
digibyteSendRequest.receivingAddress,
digibyteSendRequest.digibyteAmount,
digibyteSendRequest.feePerByte);
if (spendTransaction == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
try {
digibyte.broadcastTransaction(spendTransaction);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
return spendTransaction.getTxId().toString();
}
}

View File

@@ -122,7 +122,7 @@ public class CrossChainLitecoinResource {
@Path("/send")
@Operation(
summary = "Sends LTC from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently only supports 'legacy' P2PKH Litecoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
description = "Currently supports 'legacy' P2PKH Litecoin addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(

View File

@@ -0,0 +1,177 @@
package org.qortal.api.resource;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.media.ArraySchema;
import io.swagger.v3.oas.annotations.media.Content;
import io.swagger.v3.oas.annotations.media.Schema;
import io.swagger.v3.oas.annotations.parameters.RequestBody;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.security.SecurityRequirement;
import io.swagger.v3.oas.annotations.tags.Tag;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.HeaderParam;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.bitcoinj.core.Transaction;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.crosschain.RavencoinSendRequest;
import org.qortal.crosschain.Ravencoin;
import org.qortal.crosschain.ForeignBlockchainException;
import org.qortal.crosschain.SimpleTransaction;
@Path("/crosschain/rvn")
@Tag(name = "Cross-Chain (Ravencoin)")
public class CrossChainRavencoinResource {
@Context
HttpServletRequest request;
@POST
@Path("/walletbalance")
@Operation(
summary = "Returns RVN balance for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
@SecurityRequirement(name = "apiKey")
public String getRavencoinWalletBalance(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
Security.checkApiCallAllowed(request);
Ravencoin ravencoin = Ravencoin.getInstance();
if (!ravencoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
Long balance = ravencoin.getWalletBalanceFromTransactions(key58);
if (balance == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
return balance.toString();
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/wallettransactions")
@Operation(
summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.TEXT_PLAIN,
schema = @Schema(
type = "string",
description = "BIP32 'm' private/public key in base58",
example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
)
)
),
responses = {
@ApiResponse(
content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
@SecurityRequirement(name = "apiKey")
public List<SimpleTransaction> getRavencoinWalletTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
Security.checkApiCallAllowed(request);
Ravencoin ravencoin = Ravencoin.getInstance();
if (!ravencoin.isValidDeterministicKey(key58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
try {
return ravencoin.getWalletTransactions(key58);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
}
@POST
@Path("/send")
@Operation(
summary = "Sends RVN from hierarchical, deterministic BIP32 wallet to specific address",
description = "Currently only supports 'legacy' P2PKH Ravencoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
requestBody = @RequestBody(
required = true,
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
schema = @Schema(
implementation = RavencoinSendRequest.class
)
)
),
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
)
}
)
@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
@SecurityRequirement(name = "apiKey")
public String sendBitcoin(@HeaderParam(Security.API_KEY_HEADER) String apiKey, RavencoinSendRequest ravencoinSendRequest) {
Security.checkApiCallAllowed(request);
if (ravencoinSendRequest.ravencoinAmount <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
if (ravencoinSendRequest.feePerByte != null && ravencoinSendRequest.feePerByte <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
Ravencoin ravencoin = Ravencoin.getInstance();
if (!ravencoin.isValidAddress(ravencoinSendRequest.receivingAddress))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
if (!ravencoin.isValidDeterministicKey(ravencoinSendRequest.xprv58))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
Transaction spendTransaction = ravencoin.buildSpend(ravencoinSendRequest.xprv58,
ravencoinSendRequest.receivingAddress,
ravencoinSendRequest.ravencoinAmount,
ravencoinSendRequest.feePerByte);
if (spendTransaction == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
try {
ravencoin.broadcastTransaction(spendTransaction);
} catch (ForeignBlockchainException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
}
return spendTransaction.getTxId().toString();
}
}

View File

@@ -25,6 +25,7 @@ import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.CrossChainCancelRequest;
import org.qortal.api.model.CrossChainTradeSummary;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.crosschain.SupportedBlockchain;
import org.qortal.crosschain.ACCT;
import org.qortal.crosschain.AcctMode;
@@ -120,6 +121,8 @@ public class CrossChainResource {
crossChainTrades = crossChainTrades.subList(0, upperLimit);
}
crossChainTrades.stream().forEach(CrossChainResource::decorateTradeDataWithPresence);
return crossChainTrades;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
@@ -151,7 +154,11 @@ public class CrossChainResource {
if (acct == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
return acct.populateTradeData(repository, atData);
CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
decorateTradeDataWithPresence(crossChainTradeData);
return crossChainTradeData;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -486,4 +493,7 @@ public class CrossChainResource {
}
}
private static void decorateTradeDataWithPresence(CrossChainTradeData crossChainTradeData) {
TradeBot.getInstance().decorateTradeDataWithPresence(crossChainTradeData);
}
}

View File

@@ -98,7 +98,15 @@ public class GroupsResource {
ref = "reverse"
) @QueryParam("reverse") Boolean reverse) {
try (final Repository repository = RepositoryManager.getRepository()) {
return repository.getGroupRepository().getAllGroups(limit, offset, reverse);
List<GroupData> allGroupData = repository.getGroupRepository().getAllGroups(limit, offset, reverse);
allGroupData.forEach(groupData -> {
try {
groupData.memberCount = repository.getGroupRepository().countGroupMembers(groupData.getGroupId());
} catch (DataException e) {
// Exclude memberCount for this group
}
});
return allGroupData;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -150,7 +158,15 @@ public class GroupsResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
return repository.getGroupRepository().getGroupsWithMember(member);
List<GroupData> allGroupData = repository.getGroupRepository().getGroupsWithMember(member);
allGroupData.forEach(groupData -> {
try {
groupData.memberCount = repository.getGroupRepository().countGroupMembers(groupData.getGroupId());
} catch (DataException e) {
// Exclude memberCount for this group
}
});
return allGroupData;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
@@ -177,6 +193,7 @@ public class GroupsResource {
if (groupData == null)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.GROUP_UNKNOWN);
groupData.memberCount = repository.getGroupRepository().countGroupMembers(groupId);
return groupData;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
@@ -922,4 +939,4 @@ public class GroupsResource {
}
}
}
}

View File

@@ -26,6 +26,7 @@ import org.qortal.api.ApiErrors;
import org.qortal.api.ApiException;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.model.NameSummary;
import org.qortal.controller.LiteNode;
import org.qortal.crypto.Crypto;
import org.qortal.data.naming.NameData;
import org.qortal.data.transaction.BuyNameTransactionData;
@@ -101,7 +102,14 @@ public class NamesResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
try (final Repository repository = RepositoryManager.getRepository()) {
List<NameData> names = repository.getNameRepository().getNamesByOwner(address, limit, offset, reverse);
List<NameData> names;
if (Settings.getInstance().isLite()) {
names = LiteNode.getInstance().fetchAccountNames(address);
}
else {
names = repository.getNameRepository().getNamesByOwner(address, limit, offset, reverse);
}
return names.stream().map(NameSummary::new).collect(Collectors.toList());
} catch (DataException e) {
@@ -126,10 +134,18 @@ public class NamesResource {
@ApiErrors({ApiError.NAME_UNKNOWN, ApiError.REPOSITORY_ISSUE})
public NameData getName(@PathParam("name") String name) {
try (final Repository repository = RepositoryManager.getRepository()) {
NameData nameData = repository.getNameRepository().fromName(name);
NameData nameData;
if (nameData == null)
if (Settings.getInstance().isLite()) {
nameData = LiteNode.getInstance().fetchNameData(name);
}
else {
nameData = repository.getNameRepository().fromName(name);
}
if (nameData == null) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.NAME_UNKNOWN);
}
return nameData;
} catch (ApiException e) {

View File

@@ -20,6 +20,11 @@ import javax.ws.rs.*;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.core.config.Configuration;
import org.apache.logging.log4j.core.config.LoggerConfig;
import org.apache.logging.log4j.core.LoggerContext;
import org.qortal.api.*;
import org.qortal.api.model.ConnectedPeer;
import org.qortal.api.model.PeersSummary;
@@ -61,7 +66,7 @@ public class PeersResource {
}
)
public List<ConnectedPeer> getPeers() {
return Network.getInstance().getConnectedPeers().stream().map(ConnectedPeer::new).collect(Collectors.toList());
return Network.getInstance().getImmutableConnectedPeers().stream().map(ConnectedPeer::new).collect(Collectors.toList());
}
@GET
@@ -127,9 +132,29 @@ public class PeersResource {
}
)
@SecurityRequirement(name = "apiKey")
public ExecuteProduceConsume.StatsSnapshot getEngineStats(@HeaderParam(Security.API_KEY_HEADER) String apiKey) {
public ExecuteProduceConsume.StatsSnapshot getEngineStats(@HeaderParam(Security.API_KEY_HEADER) String apiKey, @QueryParam("newLoggingLevel") Level newLoggingLevel) {
Security.checkApiCallAllowed(request);
if (newLoggingLevel != null) {
final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
final Configuration config = ctx.getConfiguration();
String epcClassName = "org.qortal.network.Network.NetworkProcessor";
LoggerConfig loggerConfig = config.getLoggerConfig(epcClassName);
LoggerConfig specificConfig = loggerConfig;
// We need a specific configuration for this logger,
// otherwise we would change the level of all other loggers
// having the original configuration as parent as well
if (!loggerConfig.getName().equals(epcClassName)) {
specificConfig = new LoggerConfig(epcClassName, newLoggingLevel, true);
specificConfig.setParent(loggerConfig);
config.addLogger(epcClassName, specificConfig);
}
specificConfig.setLevel(newLoggingLevel);
ctx.updateLoggers();
}
return Network.getInstance().getStatsSnapshot();
}
@@ -304,7 +329,7 @@ public class PeersResource {
PeerAddress peerAddress = PeerAddress.fromString(targetPeerAddress);
InetSocketAddress resolvedAddress = peerAddress.toSocketAddress();
List<Peer> peers = Network.getInstance().getHandshakedPeers();
List<Peer> peers = Network.getInstance().getImmutableHandshakedPeers();
Peer targetPeer = peers.stream().filter(peer -> peer.getResolvedAddress().equals(resolvedAddress)).findFirst().orElse(null);
if (targetPeer == null)
@@ -352,9 +377,9 @@ public class PeersResource {
public PeersSummary peersSummary() {
PeersSummary peersSummary = new PeersSummary();
List<Peer> connectedPeers = Network.getInstance().getConnectedPeers().stream().collect(Collectors.toList());
List<Peer> connectedPeers = Network.getInstance().getImmutableConnectedPeers().stream().collect(Collectors.toList());
for (Peer peer : connectedPeers) {
if (peer.isOutbound()) {
if (!peer.isOutbound()) {
peersSummary.inboundConnections++;
}
else {

View File

@@ -74,7 +74,9 @@ public class RenderResource {
Method method = Method.PUT;
Compression compression = Compression.ZIP;
ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(Paths.get(directoryPath), null, Service.WEBSITE, null, method, compression);
ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(Paths.get(directoryPath),
null, Service.WEBSITE, null, method, compression,
null, null, null, null);
try {
arbitraryDataWriter.save();
} catch (IOException | DataException | InterruptedException | MissingDataException e) {
@@ -136,34 +138,38 @@ public class RenderResource {
@GET
@Path("/signature/{signature}")
@SecurityRequirement(name = "apiKey")
public HttpServletResponse getIndexBySignature(@PathParam("signature") String signature) {
public HttpServletResponse getIndexBySignature(@PathParam("signature") String signature,
@QueryParam("theme") String theme) {
Security.requirePriorAuthorization(request, signature, Service.WEBSITE, null);
return this.get(signature, ResourceIdType.SIGNATURE, null, "/", null, "/render/signature", true, true);
return this.get(signature, ResourceIdType.SIGNATURE, null, "/", null, "/render/signature", true, true, theme);
}
@GET
@Path("/signature/{signature}/{path:.*}")
@SecurityRequirement(name = "apiKey")
public HttpServletResponse getPathBySignature(@PathParam("signature") String signature, @PathParam("path") String inPath) {
public HttpServletResponse getPathBySignature(@PathParam("signature") String signature, @PathParam("path") String inPath,
@QueryParam("theme") String theme) {
Security.requirePriorAuthorization(request, signature, Service.WEBSITE, null);
return this.get(signature, ResourceIdType.SIGNATURE, null, inPath,null, "/render/signature", true, true);
return this.get(signature, ResourceIdType.SIGNATURE, null, inPath,null, "/render/signature", true, true, theme);
}
@GET
@Path("/hash/{hash}")
@SecurityRequirement(name = "apiKey")
public HttpServletResponse getIndexByHash(@PathParam("hash") String hash58, @QueryParam("secret") String secret58) {
public HttpServletResponse getIndexByHash(@PathParam("hash") String hash58, @QueryParam("secret") String secret58,
@QueryParam("theme") String theme) {
Security.requirePriorAuthorization(request, hash58, Service.WEBSITE, null);
return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, "/", secret58, "/render/hash", true, false);
return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, "/", secret58, "/render/hash", true, false, theme);
}
@GET
@Path("/hash/{hash}/{path:.*}")
@SecurityRequirement(name = "apiKey")
public HttpServletResponse getPathByHash(@PathParam("hash") String hash58, @PathParam("path") String inPath,
@QueryParam("secret") String secret58) {
@QueryParam("secret") String secret58,
@QueryParam("theme") String theme) {
Security.requirePriorAuthorization(request, hash58, Service.WEBSITE, null);
return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, inPath, secret58, "/render/hash", true, false);
return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, inPath, secret58, "/render/hash", true, false, theme);
}
@GET
@@ -171,29 +177,35 @@ public class RenderResource {
@SecurityRequirement(name = "apiKey")
public HttpServletResponse getPathByName(@PathParam("service") Service service,
@PathParam("name") String name,
@PathParam("path") String inPath) {
@PathParam("path") String inPath,
@QueryParam("theme") String theme) {
Security.requirePriorAuthorization(request, name, service, null);
String prefix = String.format("/render/%s", service);
return this.get(name, ResourceIdType.NAME, service, inPath, null, prefix, true, true);
return this.get(name, ResourceIdType.NAME, service, inPath, null, prefix, true, true, theme);
}
@GET
@Path("{service}/{name}")
@SecurityRequirement(name = "apiKey")
public HttpServletResponse getIndexByName(@PathParam("service") Service service,
@PathParam("name") String name) {
@PathParam("name") String name,
@QueryParam("theme") String theme) {
Security.requirePriorAuthorization(request, name, service, null);
String prefix = String.format("/render/%s", service);
return this.get(name, ResourceIdType.NAME, service, "/", null, prefix, true, true);
return this.get(name, ResourceIdType.NAME, service, "/", null, prefix, true, true, theme);
}
private HttpServletResponse get(String resourceId, ResourceIdType resourceIdType, Service service, String inPath,
String secret58, String prefix, boolean usePrefix, boolean async) {
String secret58, String prefix, boolean usePrefix, boolean async, String theme) {
ArbitraryDataRenderer renderer = new ArbitraryDataRenderer(resourceId, resourceIdType, service, inPath,
secret58, prefix, usePrefix, async, request, response, context);
if (theme != null) {
renderer.setTheme(theme);
}
return renderer.render();
}

View File

@@ -12,6 +12,7 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;
@@ -32,6 +33,8 @@ import org.qortal.api.ApiException;
import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.model.SimpleTransactionSignRequest;
import org.qortal.controller.Controller;
import org.qortal.controller.LiteNode;
import org.qortal.crypto.Crypto;
import org.qortal.data.transaction.TransactionData;
import org.qortal.globalization.Translator;
import org.qortal.repository.DataException;
@@ -250,14 +253,29 @@ public class TransactionsResource {
ApiError.REPOSITORY_ISSUE
})
public List<TransactionData> getUnconfirmedTransactions(@Parameter(
description = "A list of transaction types"
) @QueryParam("txType") List<TransactionType> txTypes, @Parameter(
description = "Transaction creator's base58 encoded public key"
) @QueryParam("creator") String creatorPublicKey58, @Parameter(
ref = "limit"
) @QueryParam("limit") Integer limit, @Parameter(
ref = "offset"
) @QueryParam("offset") Integer offset, @Parameter(
ref = "reverse"
) @QueryParam("reverse") Boolean reverse) {
// Decode public key if supplied
byte[] creatorPublicKey = null;
if (creatorPublicKey58 != null) {
try {
creatorPublicKey = Base58.decode(creatorPublicKey58);
} catch (NumberFormatException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY, e);
}
}
try (final Repository repository = RepositoryManager.getRepository()) {
return repository.getTransactionRepository().getUnconfirmedTransactions(limit, offset, reverse);
return repository.getTransactionRepository().getUnconfirmedTransactions(txTypes, creatorPublicKey, limit, offset, reverse);
} catch (ApiException e) {
throw e;
} catch (DataException e) {
@@ -366,6 +384,73 @@ public class TransactionsResource {
}
}
@GET
@Path("/address/{address}")
@Operation(
summary = "Returns transactions for given address",
responses = {
@ApiResponse(
description = "transactions",
content = @Content(
array = @ArraySchema(
schema = @Schema(
implementation = TransactionData.class
)
)
)
)
}
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.REPOSITORY_ISSUE})
public List<TransactionData> getAddressTransactions(@PathParam("address") String address,
@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse) {
if (!Crypto.isValidAddress(address)) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
}
if (limit == null) {
limit = 0;
}
if (offset == null) {
offset = 0;
}
List<TransactionData> transactions;
if (Settings.getInstance().isLite()) {
// Fetch from network
transactions = LiteNode.getInstance().fetchAccountTransactions(address, limit, offset);
// Sort the data, since we can't guarantee the order that a peer sent it in
if (reverse) {
transactions.sort(Comparator.comparingLong(TransactionData::getTimestamp).reversed());
} else {
transactions.sort(Comparator.comparingLong(TransactionData::getTimestamp));
}
}
else {
// Fetch from local db
try (final Repository repository = RepositoryManager.getRepository()) {
List<byte[]> signatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(null, null, null,
null, null, null, address, TransactionsResource.ConfirmationStatus.CONFIRMED, limit, offset, reverse);
// Expand signatures to transactions
transactions = new ArrayList<>(signatures.size());
for (byte[] signature : signatures) {
transactions.add(repository.getTransactionRepository().fromSignature(signature));
}
} catch (ApiException e) {
throw e;
} catch (DataException e) {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
}
}
return transactions;
}
@GET
@Path("/unitfee")
@Operation(
@@ -638,7 +723,10 @@ public class TransactionsResource {
ApiError.BLOCKCHAIN_NEEDS_SYNC, ApiError.INVALID_SIGNATURE, ApiError.INVALID_DATA, ApiError.TRANSFORMATION_ERROR, ApiError.REPOSITORY_ISSUE
})
public String processTransaction(String rawBytes58) {
if (!Controller.getInstance().isUpToDate())
// Only allow a transaction to be processed if our latest block is less than 30 minutes old
// If older than this, we should first wait until the blockchain is synced
final Long minLatestBlockTimestamp = NTP.getTime() - (30 * 60 * 1000L);
if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
byte[] rawBytes = Base58.decode(rawBytes58);

View File

@@ -20,6 +20,7 @@ import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.controller.Controller;
import org.qortal.controller.Synchronizer;
import org.qortal.crypto.Crypto;
import org.qortal.data.transaction.PresenceTransactionData;
import org.qortal.data.transaction.TransactionData;
@@ -99,13 +100,13 @@ public class PresenceWebSocket extends ApiWebSocket implements Listener {
@Override
public void listen(Event event) {
// We use NewBlockEvent as a proxy for 1-minute timer
if (!(event instanceof Controller.NewTransactionEvent) && !(event instanceof Controller.NewBlockEvent))
// We use Synchronizer.NewChainTipEvent as a proxy for 1-minute timer
if (!(event instanceof Controller.NewTransactionEvent) && !(event instanceof Synchronizer.NewChainTipEvent))
return;
removeOldEntries();
if (event instanceof Controller.NewBlockEvent)
if (event instanceof Synchronizer.NewChainTipEvent)
// We only wanted a chance to cull old entries
return;

View File

@@ -23,6 +23,7 @@ import org.eclipse.jetty.websocket.api.annotations.WebSocket;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.api.model.CrossChainOfferSummary;
import org.qortal.controller.Controller;
import org.qortal.controller.Synchronizer;
import org.qortal.crosschain.SupportedBlockchain;
import org.qortal.crosschain.ACCT;
import org.qortal.crosschain.AcctMode;
@@ -80,10 +81,10 @@ public class TradeOffersWebSocket extends ApiWebSocket implements Listener {
@Override
public void listen(Event event) {
if (!(event instanceof Controller.NewBlockEvent))
if (!(event instanceof Synchronizer.NewChainTipEvent))
return;
BlockData blockData = ((Controller.NewBlockEvent) event).getBlockData();
BlockData blockData = ((Synchronizer.NewChainTipEvent) event).getNewChainTip();
// Process any new info

View File

@@ -0,0 +1,137 @@
package org.qortal.api.websocket;
import org.eclipse.jetty.websocket.api.Session;
import org.eclipse.jetty.websocket.api.annotations.*;
import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
import org.qortal.controller.Controller;
import org.qortal.controller.tradebot.TradeBot;
import org.qortal.data.network.TradePresenceData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.io.IOException;
import java.io.StringWriter;
import java.util.*;
@WebSocket
@SuppressWarnings("serial")
public class TradePresenceWebSocket extends ApiWebSocket implements Listener {
/** Map key is public key in base58, map value is trade presence */
private static final Map<String, TradePresenceData> currentEntries = Collections.synchronizedMap(new HashMap<>());
@Override
public void configure(WebSocketServletFactory factory) {
factory.register(TradePresenceWebSocket.class);
populateCurrentInfo();
EventBus.INSTANCE.addListener(this::listen);
}
@Override
public void listen(Event event) {
// XXX - Suggest we change this to something like Synchronizer.NewChainTipEvent?
// We use NewBlockEvent as a proxy for 1-minute timer
if (!(event instanceof TradeBot.TradePresenceEvent) && !(event instanceof Controller.NewBlockEvent))
return;
removeOldEntries();
if (event instanceof Controller.NewBlockEvent)
// We only wanted a chance to cull old entries
return;
TradePresenceData tradePresence = ((TradeBot.TradePresenceEvent) event).getTradePresenceData();
boolean somethingChanged = mergePresence(tradePresence);
if (!somethingChanged)
// nothing changed
return;
List<TradePresenceData> tradePresences = Collections.singletonList(tradePresence);
// Notify sessions
for (Session session : getSessions()) {
sendTradePresences(session, tradePresences);
}
}
@OnWebSocketConnect
@Override
public void onWebSocketConnect(Session session) {
Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
List<TradePresenceData> tradePresences;
synchronized (currentEntries) {
tradePresences = List.copyOf(currentEntries.values());
}
if (!sendTradePresences(session, tradePresences)) {
session.close(4002, "websocket issue");
return;
}
super.onWebSocketConnect(session);
}
@OnWebSocketClose
@Override
public void onWebSocketClose(Session session, int statusCode, String reason) {
// clean up
super.onWebSocketClose(session, statusCode, reason);
}
@OnWebSocketError
public void onWebSocketError(Session session, Throwable throwable) {
/* ignored */
}
@OnWebSocketMessage
public void onWebSocketMessage(Session session, String message) {
/* ignored */
}
private boolean sendTradePresences(Session session, List<TradePresenceData> tradePresences) {
try {
StringWriter stringWriter = new StringWriter();
marshall(stringWriter, tradePresences);
String output = stringWriter.toString();
session.getRemote().sendStringByFuture(output);
} catch (IOException e) {
// No output this time?
return false;
}
return true;
}
private static void populateCurrentInfo() {
// We want ALL trade presences
TradeBot.getInstance().getAllTradePresences().stream()
.forEach(TradePresenceWebSocket::mergePresence);
}
/** Merge trade presence into cache of current entries, returns true if cache was updated. */
private static boolean mergePresence(TradePresenceData tradePresence) {
// Put/replace for this publickey making sure we keep newest timestamp
String pubKey58 = Base58.encode(tradePresence.getPublicKey());
TradePresenceData newEntry = currentEntries.compute(pubKey58, (k, v) -> v == null || v.getTimestamp() < tradePresence.getTimestamp() ? tradePresence : v);
return newEntry == tradePresence;
}
private static void removeOldEntries() {
long now = NTP.getTime();
currentEntries.values().removeIf(v -> v.getTimestamp() < now);
}
}

View File

@@ -53,7 +53,8 @@ public class ArbitraryDataFile {
private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFile.class);
public static final long MAX_FILE_SIZE = 500 * 1024 * 1024; // 500MiB
public static final int CHUNK_SIZE = 1 * 1024 * 1024; // 1MiB
protected static final int MAX_CHUNK_SIZE = 1 * 1024 * 1024; // 1MiB
public static final int CHUNK_SIZE = 512 * 1024; // 0.5MiB
public static int SHORT_DIGEST_LENGTH = 8;
protected Path filePath;
@@ -72,7 +73,6 @@ public class ArbitraryDataFile {
}
public ArbitraryDataFile(String hash58, byte[] signature) throws DataException {
this.createDataDirectory();
this.filePath = ArbitraryDataFile.getOutputFilePath(hash58, signature, false);
this.chunks = new ArrayList<>();
this.hash58 = hash58;
@@ -94,14 +94,9 @@ public class ArbitraryDataFile {
try (FileOutputStream outputStream = new FileOutputStream(outputFile)) {
outputStream.write(fileContent);
this.filePath = outputFilePath;
// Verify hash
if (!this.hash58.equals(this.digest58())) {
LOGGER.error("Hash {} does not match file digest {}", this.hash58, this.digest58());
this.delete();
throw new DataException("Data file digest validation failed");
}
} catch (IOException e) {
throw new DataException("Unable to write data to file");
this.delete();
throw new DataException(String.format("Unable to write data with hash %s: %s", this.hash58, e.getMessage()));
}
}
@@ -110,6 +105,9 @@ public class ArbitraryDataFile {
}
public static ArbitraryDataFile fromHash(byte[] hash, byte[] signature) throws DataException {
if (hash == null) {
return null;
}
return ArbitraryDataFile.fromHash58(Base58.encode(hash), signature);
}
@@ -146,19 +144,6 @@ public class ArbitraryDataFile {
return ArbitraryDataFile.fromPath(Paths.get(file.getPath()), signature);
}
private boolean createDataDirectory() {
// Create the data directory if it doesn't exist
String dataPath = Settings.getInstance().getDataPath();
Path dataDirectory = Paths.get(dataPath);
try {
Files.createDirectories(dataDirectory);
} catch (IOException e) {
LOGGER.error("Unable to create data directory");
return false;
}
return true;
}
private Path copyToDataDirectory(Path sourcePath, byte[] signature) throws DataException {
if (this.hash58 == null || this.filePath == null) {
return null;
@@ -366,6 +351,21 @@ public class ArbitraryDataFile {
return false;
}
public boolean delete(int attempts) {
// Keep trying to delete the data until it is deleted, or we reach 10 attempts
for (int i=0; i<attempts; i++) {
if (this.delete()) {
return true;
}
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
// Fall through to exit method
}
}
return false;
}
public boolean deleteAllChunks() {
boolean success = false;
@@ -473,6 +473,14 @@ public class ArbitraryDataFile {
// Read the metadata
List<byte[]> chunks = metadata.getChunks();
// If the chunks array is empty, then this resource has no chunks,
// so we must return false to avoid confusing the caller.
if (chunks.isEmpty()) {
return false;
}
// Otherwise, we need to check each chunk individually
for (byte[] chunkHash : chunks) {
ArbitraryDataFileChunk chunk = ArbitraryDataFileChunk.fromHash(chunkHash, this.signature);
if (!chunk.exists()) {
@@ -771,6 +779,10 @@ public class ArbitraryDataFile {
this.loadMetadata();
}
public ArbitraryDataTransactionMetadata getMetadata() {
return this.metadata;
}
@Override
public String toString() {
return this.shortHash58();

View File

@@ -40,8 +40,8 @@ public class ArbitraryDataFileChunk extends ArbitraryDataFile {
try {
// Validate the file size (chunks have stricter limits)
long fileSize = Files.size(this.filePath);
if (fileSize > CHUNK_SIZE) {
LOGGER.error(String.format("DataFileChunk is too large: %d bytes (max chunk size: %d bytes)", fileSize, CHUNK_SIZE));
if (fileSize > MAX_CHUNK_SIZE) {
LOGGER.error(String.format("DataFileChunk is too large: %d bytes (max chunk size: %d bytes)", fileSize, MAX_CHUNK_SIZE));
return ValidationResult.FILE_TOO_LARGE;
}

View File

@@ -34,6 +34,7 @@ public class ArbitraryDataRenderer {
private final String resourceId;
private final ResourceIdType resourceIdType;
private final Service service;
private String theme = "light";
private String inPath;
private final String secret58;
private final String prefix;
@@ -77,7 +78,7 @@ public class ArbitraryDataRenderer {
// If async is requested, show a loading screen whilst build is in progress
if (async) {
arbitraryDataReader.loadAsynchronously(false, 10);
return this.getLoadingResponse(service, resourceId);
return this.getLoadingResponse(service, resourceId, theme);
}
// Otherwise, loop until we have data
@@ -119,7 +120,7 @@ public class ArbitraryDataRenderer {
byte[] data = Files.readAllBytes(Paths.get(filePath)); // TODO: limit file size that can be read into memory
HTMLParser htmlParser = new HTMLParser(resourceId, inPath, prefix, usePrefix, data);
htmlParser.addAdditionalHeaderTags();
response.addHeader("Content-Security-Policy", "default-src 'self' 'unsafe-inline'; media-src 'self' blob:");
response.addHeader("Content-Security-Policy", "default-src 'self' 'unsafe-inline' 'unsafe-eval'; media-src 'self' blob:; img-src 'self' data: blob:;");
response.setContentType(context.getMimeType(filename));
response.setContentLength(htmlParser.getData().length);
response.getOutputStream().write(htmlParser.getData());
@@ -128,7 +129,7 @@ public class ArbitraryDataRenderer {
// Regular file - can be streamed directly
File file = new File(filePath);
FileInputStream inputStream = new FileInputStream(file);
response.addHeader("Content-Security-Policy", "default-src 'self' 'unsafe-inline'; media-src 'self' blob:");
response.addHeader("Content-Security-Policy", "default-src 'self'");
response.setContentType(context.getMimeType(filename));
int bytesRead, length = 0;
byte[] buffer = new byte[10240];
@@ -171,7 +172,7 @@ public class ArbitraryDataRenderer {
return userPath;
}
private HttpServletResponse getLoadingResponse(Service service, String name) {
private HttpServletResponse getLoadingResponse(Service service, String name, String theme) {
String responseString = "";
URL url = Resources.getResource("loading/index.html");
try {
@@ -180,6 +181,7 @@ public class ArbitraryDataRenderer {
// Replace vars
responseString = responseString.replace("%%SERVICE%%", service.toString());
responseString = responseString.replace("%%NAME%%", name);
responseString = responseString.replace("%%THEME%%", theme);
} catch (IOException e) {
LOGGER.info("Unable to show loading screen: {}", e.getMessage());
@@ -210,4 +212,8 @@ public class ArbitraryDataRenderer {
return indexFiles;
}
public void setTheme(String theme) {
this.theme = theme;
}
}

View File

@@ -3,6 +3,7 @@ package org.qortal.arbitrary;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.arbitrary.ArbitraryDataFile.ResourceIdType;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.arbitrary.misc.Service;
import org.qortal.controller.arbitrary.ArbitraryDataBuildManager;
import org.qortal.controller.arbitrary.ArbitraryDataManager;
@@ -37,6 +38,7 @@ public class ArbitraryDataResource {
private List<ArbitraryTransactionData> transactions;
private ArbitraryTransactionData latestPutTransaction;
private ArbitraryTransactionData latestTransaction;
private int layerCount;
private Integer localChunkCount = null;
private Integer totalChunkCount = null;
@@ -105,6 +107,33 @@ public class ArbitraryDataResource {
return new ArbitraryResourceStatus(Status.DOWNLOADED, this.localChunkCount, this.totalChunkCount);
}
public ArbitraryDataTransactionMetadata getLatestTransactionMetadata() {
this.fetchLatestTransaction();
if (latestTransaction != null) {
byte[] signature = latestTransaction.getSignature();
byte[] metadataHash = latestTransaction.getMetadataHash();
if (metadataHash == null) {
// This resource doesn't have metadata
return null;
}
try {
ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
if (metadataFile.exists()) {
ArbitraryDataTransactionMetadata transactionMetadata = new ArbitraryDataTransactionMetadata(metadataFile.getFilePath());
transactionMetadata.read();
return transactionMetadata;
}
} catch (DataException | IOException e) {
// Do nothing
}
}
return null;
}
public boolean delete() {
try {
this.fetchTransactions();
@@ -306,6 +335,32 @@ public class ArbitraryDataResource {
this.transactions = transactionDataList;
this.layerCount = transactionDataList.size();
} catch (DataException e) {
LOGGER.info(String.format("Repository error when fetching transactions for resource %s: %s", this, e.getMessage()));
}
}
private void fetchLatestTransaction() {
if (this.latestTransaction != null) {
// Already fetched
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Get the most recent transaction
ArbitraryTransactionData latestTransaction = repository.getArbitraryRepository()
.getLatestTransaction(this.resourceId, this.service, null, this.identifier);
if (latestTransaction == null) {
String message = String.format("Couldn't find transaction for name %s, service %s and identifier %s",
this.resourceId, this.service, this.identifierString());
throw new DataException(message);
}
this.latestTransaction = latestTransaction;
} catch (DataException e) {
LOGGER.info(String.format("Repository error when fetching latest transaction for resource %s: %s", this, e.getMessage()));
}
}

View File

@@ -6,8 +6,9 @@ import org.qortal.arbitrary.exception.MissingDataException;
import org.qortal.arbitrary.ArbitraryDataFile.ResourceIdType;
import org.qortal.arbitrary.ArbitraryDataDiff.*;
import org.qortal.arbitrary.metadata.ArbitraryDataMetadataPatch;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.arbitrary.misc.Category;
import org.qortal.arbitrary.misc.Service;
import org.qortal.block.BlockChain;
import org.qortal.crypto.Crypto;
import org.qortal.data.PaymentData;
import org.qortal.data.transaction.ArbitraryTransactionData;
@@ -27,6 +28,7 @@ import java.io.IOException;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.Random;
public class ArbitraryDataTransactionBuilder {
@@ -51,13 +53,20 @@ public class ArbitraryDataTransactionBuilder {
private final String identifier;
private final Repository repository;
// Metadata
private final String title;
private final String description;
private final List<String> tags;
private final Category category;
private int chunkSize = ArbitraryDataFile.CHUNK_SIZE;
private ArbitraryTransactionData arbitraryTransactionData;
private ArbitraryDataFile arbitraryDataFile;
public ArbitraryDataTransactionBuilder(Repository repository, String publicKey58, Path path, String name,
Method method, Service service, String identifier) {
Method method, Service service, String identifier,
String title, String description, List<String> tags, Category category) {
this.repository = repository;
this.publicKey58 = publicKey58;
this.path = path;
@@ -70,6 +79,12 @@ public class ArbitraryDataTransactionBuilder {
identifier = null;
}
this.identifier = identifier;
// Metadata (optional)
this.title = ArbitraryDataTransactionMetadata.limitTitle(title);
this.description = ArbitraryDataTransactionMetadata.limitDescription(description);
this.tags = ArbitraryDataTransactionMetadata.limitTags(tags);
this.category = category;
}
public void build() throws DataException {
@@ -108,6 +123,10 @@ public class ArbitraryDataTransactionBuilder {
return Method.PUT;
}
// Get existing metadata and see if it matches the new metadata
ArbitraryDataResource resource = new ArbitraryDataResource(this.name, ResourceIdType.NAME, this.service, this.identifier);
ArbitraryDataTransactionMetadata existingMetadata = resource.getLatestTransactionMetadata();
try {
// Check layer count
int layerCount = reader.getLayerCount();
@@ -118,7 +137,23 @@ public class ArbitraryDataTransactionBuilder {
// Check size of differences between this layer and previous layer
ArbitraryDataCreatePatch patch = new ArbitraryDataCreatePatch(reader.getFilePath(), this.path, reader.getLatestSignature());
patch.create();
try {
patch.create();
}
catch (DataException | IOException e) {
// Handle matching states separately, as it's best to block transactions with duplicate states
if (e.getMessage().equals("Current state matches previous state. Nothing to do.")) {
// Only throw an exception if the metadata is also identical, as well as the data
if (this.isMetadataEqual(existingMetadata)) {
throw new DataException(e.getMessage());
}
}
LOGGER.info("Caught exception when creating patch: {}", e.getMessage());
LOGGER.info("Unable to load existing resource - using PUT to overwrite it.");
return Method.PUT;
}
long diffSize = FilesystemUtils.getDirectorySize(patch.getFinalPath());
long existingStateSize = FilesystemUtils.getDirectorySize(reader.getFilePath());
double difference = (double) diffSize / (double) existingStateSize;
@@ -155,11 +190,8 @@ public class ArbitraryDataTransactionBuilder {
// State is appropriate for a PATCH transaction
return Method.PATCH;
}
catch (IOException | DataException e) {
// Handle matching states separately, as it's best to block transactions with duplicate states
if (e.getMessage().equals("Current state matches previous state. Nothing to do.")) {
throw new DataException(e.getMessage());
}
catch (IOException e) {
// IMPORTANT: Don't catch DataException here, as they must be passed to the caller
LOGGER.info("Caught exception: {}", e.getMessage());
LOGGER.info("Unable to load existing resource - using PUT to overwrite it.");
return Method.PUT;
@@ -200,7 +232,8 @@ public class ArbitraryDataTransactionBuilder {
// FUTURE? Use zip compression for directories, or no compression for single files
// Compression compression = (path.toFile().isDirectory()) ? Compression.ZIP : Compression.NONE;
ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(path, name, service, identifier, method, compression);
ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(path, name, service, identifier, method,
compression, title, description, tags, category);
try {
arbitraryDataWriter.setChunkSize(this.chunkSize);
arbitraryDataWriter.save();
@@ -253,6 +286,22 @@ public class ArbitraryDataTransactionBuilder {
}
private boolean isMetadataEqual(ArbitraryDataTransactionMetadata existingMetadata) {
if (!Objects.equals(existingMetadata.getTitle(), this.title)) {
return false;
}
if (!Objects.equals(existingMetadata.getDescription(), this.description)) {
return false;
}
if (!Objects.equals(existingMetadata.getCategory(), this.category)) {
return false;
}
if (!Objects.equals(existingMetadata.getTags(), this.tags)) {
return false;
}
return true;
}
public void computeNonce() throws DataException {
if (this.arbitraryTransactionData == null) {
throw new DataException("Arbitrary transaction data is required to compute nonce");

View File

@@ -5,6 +5,7 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.arbitrary.exception.MissingDataException;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.arbitrary.misc.Category;
import org.qortal.arbitrary.misc.Service;
import org.qortal.crypto.Crypto;
import org.qortal.data.transaction.ArbitraryTransactionData.*;
@@ -28,6 +29,10 @@ import java.nio.file.Paths;
import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Objects;
public class ArbitraryDataWriter {
@@ -40,6 +45,12 @@ public class ArbitraryDataWriter {
private final Method method;
private final Compression compression;
// Metadata
private final String title;
private final String description;
private final List<String> tags;
private final Category category;
private int chunkSize = ArbitraryDataFile.CHUNK_SIZE;
private SecretKey aesKey;
@@ -50,7 +61,8 @@ public class ArbitraryDataWriter {
private Path compressedPath;
private Path encryptedPath;
public ArbitraryDataWriter(Path filePath, String name, Service service, String identifier, Method method, Compression compression) {
public ArbitraryDataWriter(Path filePath, String name, Service service, String identifier, Method method, Compression compression,
String title, String description, List<String> tags, Category category) {
this.filePath = filePath;
this.name = name;
this.service = service;
@@ -62,6 +74,12 @@ public class ArbitraryDataWriter {
identifier = null;
}
this.identifier = identifier;
// Metadata (optional)
this.title = ArbitraryDataTransactionMetadata.limitTitle(title);
this.description = ArbitraryDataTransactionMetadata.limitDescription(description);
this.tags = ArbitraryDataTransactionMetadata.limitTags(tags);
this.category = category;
}
public void save() throws IOException, DataException, InterruptedException, MissingDataException {
@@ -258,12 +276,16 @@ public class ArbitraryDataWriter {
private void createMetadataFile() throws IOException, DataException {
// If we have at least one chunk, we need to create an index file containing their hashes
if (this.arbitraryDataFile.chunkCount() > 1) {
if (this.needsMetadataFile()) {
// Create the JSON file
Path chunkFilePath = Paths.get(this.workingPath.toString(), "metadata.json");
ArbitraryDataTransactionMetadata chunkMetadata = new ArbitraryDataTransactionMetadata(chunkFilePath);
chunkMetadata.setChunks(this.arbitraryDataFile.chunkHashList());
chunkMetadata.write();
ArbitraryDataTransactionMetadata metadata = new ArbitraryDataTransactionMetadata(chunkFilePath);
metadata.setTitle(this.title);
metadata.setDescription(this.description);
metadata.setTags(this.tags);
metadata.setCategory(this.category);
metadata.setChunks(this.arbitraryDataFile.chunkHashList());
metadata.write();
// Create an ArbitraryDataFile from the JSON file (we don't have a signature yet)
ArbitraryDataFile metadataFile = ArbitraryDataFile.fromPath(chunkFilePath, null);
@@ -308,6 +330,20 @@ public class ArbitraryDataWriter {
throw new DataException(String.format("Missing chunk %s in metadata file", Base58.encode(chunk)));
}
}
// Check that the metadata is correct
if (!Objects.equals(metadata.getTitle(), this.title)) {
throw new DataException("Metadata mismatch: title");
}
if (!Objects.equals(metadata.getDescription(), this.description)) {
throw new DataException("Metadata mismatch: description");
}
if (!Objects.equals(metadata.getTags(), this.tags)) {
throw new DataException("Metadata mismatch: tags");
}
if (!Objects.equals(metadata.getCategory(), this.category)) {
throw new DataException("Metadata mismatch: category");
}
}
}
@@ -330,6 +366,16 @@ public class ArbitraryDataWriter {
}
}
private boolean needsMetadataFile() {
if (this.arbitraryDataFile.chunkCount() > 1) {
return true;
}
if (this.title != null || this.description != null || this.tags != null || this.category != null) {
return true;
}
return false;
}
public ArbitraryDataFile getArbitraryDataFile() {
return this.arbitraryDataFile;

View File

@@ -2,17 +2,28 @@ package org.qortal.arbitrary.metadata;
import org.json.JSONArray;
import org.json.JSONObject;
import org.qortal.arbitrary.misc.Category;
import org.qortal.repository.DataException;
import org.qortal.utils.Base58;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
private List<byte[]> chunks;
private String title;
private String description;
private List<String> tags;
private Category category;
private static int MAX_TITLE_LENGTH = 80;
private static int MAX_DESCRIPTION_LENGTH = 500;
private static int MAX_TAG_LENGTH = 20;
private static int MAX_TAGS_COUNT = 5;
public ArbitraryDataTransactionMetadata(Path filePath) {
super(filePath);
@@ -25,10 +36,37 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
throw new DataException("Transaction metadata JSON string is null");
}
JSONObject metadata = new JSONObject(this.jsonString);
if (metadata.has("title")) {
this.title = metadata.getString("title");
}
if (metadata.has("description")) {
this.description = metadata.getString("description");
}
List<String> tagsList = new ArrayList<>();
if (metadata.has("tags")) {
JSONArray tags = metadata.getJSONArray("tags");
if (tags != null) {
for (int i=0; i<tags.length(); i++) {
String tag = tags.getString(i);
if (tag != null) {
tagsList.add(tag);
}
}
}
this.tags = tagsList;
}
if (metadata.has("category")) {
this.category = Category.uncategorizedValueOf(metadata.getString("category"));
}
List<byte[]> chunksList = new ArrayList<>();
JSONObject cache = new JSONObject(this.jsonString);
if (cache.has("chunks")) {
JSONArray chunks = cache.getJSONArray("chunks");
if (metadata.has("chunks")) {
JSONArray chunks = metadata.getJSONArray("chunks");
if (chunks != null) {
for (int i=0; i<chunks.length(); i++) {
String chunk = chunks.getString(i);
@@ -45,6 +83,26 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
protected void buildJson() {
JSONObject outer = new JSONObject();
if (this.title != null && !this.title.isEmpty()) {
outer.put("title", this.title);
}
if (this.description != null && !this.description.isEmpty()) {
outer.put("description", this.description);
}
JSONArray tags = new JSONArray();
if (this.tags != null) {
for (String tag : this.tags) {
tags.put(tag);
}
outer.put("tags", tags);
}
if (this.category != null) {
outer.put("category", this.category.toString());
}
JSONArray chunks = new JSONArray();
if (this.chunks != null) {
for (byte[] chunk : this.chunks) {
@@ -66,6 +124,38 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
return this.chunks;
}
public void setTitle(String title) {
this.title = title;
}
public String getTitle() {
return this.title;
}
public void setDescription(String description) {
this.description = description;
}
public String getDescription() {
return this.description;
}
public void setTags(List<String> tags) {
this.tags = tags;
}
public List<String> getTags() {
return this.tags;
}
public void setCategory(Category category) {
this.category = category;
}
public Category getCategory() {
return this.category;
}
public boolean containsChunk(byte[] chunk) {
for (byte[] c : this.chunks) {
if (Arrays.equals(c, chunk)) {
@@ -75,4 +165,61 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
return false;
}
// Static helper methods
public static String limitTitle(String title) {
if (title == null) {
return null;
}
if (title.isEmpty()) {
return null;
}
return title.substring(0, Math.min(title.length(), MAX_TITLE_LENGTH));
}
public static String limitDescription(String description) {
if (description == null) {
return null;
}
if (description.isEmpty()) {
return null;
}
return description.substring(0, Math.min(description.length(), MAX_DESCRIPTION_LENGTH));
}
public static List<String> limitTags(List<String> tags) {
if (tags == null) {
return null;
}
// Ensure tags list is mutable
List<String> mutableTags = new ArrayList<>(tags);
int tagCount = mutableTags.size();
if (tagCount == 0) {
return null;
}
// Remove tags over the limit
// This is cleaner than truncating, which results in malformed tags
// Also remove tags that are empty
Iterator iterator = mutableTags.iterator();
while (iterator.hasNext()) {
String tag = (String) iterator.next();
if (tag == null || tag.length() > MAX_TAG_LENGTH || tag.isEmpty()) {
iterator.remove();
}
}
// Limit the total number of tags
if (tagCount > MAX_TAGS_COUNT) {
mutableTags = mutableTags.subList(0, MAX_TAGS_COUNT);
}
return mutableTags;
}
}

View File

@@ -0,0 +1,81 @@
package org.qortal.arbitrary.misc;
public enum Category {
ART("Art and Design"),
AUTOMOTIVE("Automotive"),
BEAUTY("Beauty"),
BOOKS("Books and Reference"),
BUSINESS("Business"),
COMMUNICATIONS("Communications"),
CRYPTOCURRENCY("Cryptocurrency and Blockchain"),
CULTURE("Culture"),
DATING("Dating"),
DESIGN("Design"),
ENTERTAINMENT("Entertainment"),
EVENTS("Events"),
FAITH("Faith and Religion"),
FASHION("Fashion"),
FINANCE("Finance"),
FOOD("Food and Drink"),
GAMING("Gaming"),
GEOGRAPHY("Geography"),
HEALTH("Health"),
HISTORY("History"),
HOME("Home"),
KNOWLEDGE("Knowledge Share"),
LANGUAGE("Language"),
LIFESTYLE("Lifestyle"),
MANUFACTURING("Manufacturing"),
MAPS("Maps and Navigation"),
MUSIC("Music"),
NEWS("News"),
OTHER("Other"),
PETS("Pets"),
PHILOSOPHY("Philosophy"),
PHOTOGRAPHY("Photography"),
POLITICS("Politics"),
PRODUCE("Products and Services"),
PRODUCTIVITY("Productivity"),
PSYCHOLOGY("Psychology"),
QORTAL("Qortal"),
SCIENCE("Science"),
SELF_CARE("Self Care"),
SELF_SUFFICIENCY("Self-Sufficiency and Homesteading"),
SHOPPING("Shopping"),
SOCIAL("Social"),
SOFTWARE("Software"),
SPIRITUALITY("Spirituality"),
SPORTS("Sports"),
STORYTELLING("Storytelling"),
TECHNOLOGY("Technology"),
TOOLS("Tools"),
TRAVEL("Travel"),
UNCATEGORIZED("Uncategorized"),
VIDEO("Video"),
WEATHER("Weather");
private final String name;
Category(String name) {
this.name = name;
}
public String getName() {
return this.name;
}
/**
* Same as valueOf() but with fallback to UNCATEGORIZED if there's no match
* @param name
* @return a Category (using UNCATEGORIZED if no match found)
*/
public static Category uncategorizedValueOf(String name) {
try {
return Category.valueOf(name);
}
catch (IllegalArgumentException e) {
return Category.UNCATEGORIZED;
}
}
}

View File

@@ -551,7 +551,7 @@ public class QortalATAPI extends API {
* <p>
* Otherwise, assume B is a public key.
*/
private Account getAccountFromB(MachineState state) {
/*package*/ Account getAccountFromB(MachineState state) {
byte[] bBytes = this.getB(state);
if ((bBytes[0] == Crypto.ADDRESS_VERSION || bBytes[0] == Crypto.AT_ADDRESS_VERSION)

View File

@@ -10,9 +10,11 @@ import org.ciyam.at.ExecutionException;
import org.ciyam.at.FunctionData;
import org.ciyam.at.IllegalFunctionCodeException;
import org.ciyam.at.MachineState;
import org.qortal.account.Account;
import org.qortal.crosschain.Bitcoin;
import org.qortal.crypto.Crypto;
import org.qortal.data.transaction.TransactionData;
import org.qortal.repository.DataException;
import org.qortal.settings.Settings;
/**
@@ -160,6 +162,68 @@ public enum QortalFunctionCode {
protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
convertAddressInB(Crypto.ADDRESS_VERSION, state);
}
},
/**
* Returns account level of account in B.<br>
* <tt>0x0520</tt><br>
* B should contain either Qortal address or public key,<br>
* e.g. as a result of calling function {@link org.ciyam.at.FunctionCode#PUT_ADDRESS_FROM_TX_IN_A_INTO_B}</code>.
* <p></p>
* Returns account level, or -1 if account unknown.
* <p></p>
* @see QortalATAPI#getAccountFromB(MachineState)
*/
GET_ACCOUNT_LEVEL_FROM_ACCOUNT_IN_B(0x0520, 0, true) {
@Override
protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
QortalATAPI api = (QortalATAPI) state.getAPI();
Account account = api.getAccountFromB(state);
Integer accountLevel = null;
if (account != null) {
try {
accountLevel = account.getLevel();
} catch (DataException e) {
throw new RuntimeException("AT API unable to fetch account level?", e);
}
}
functionData.returnValue = accountLevel != null
? accountLevel.longValue()
: -1;
}
},
/**
* Returns account's minted block count of account in B.<br>
* <tt>0x0521</tt><br>
* B should contain either Qortal address or public key,<br>
* e.g. as a result of calling function {@link org.ciyam.at.FunctionCode#PUT_ADDRESS_FROM_TX_IN_A_INTO_B}</code>.
* <p></p>
* Returns account level, or -1 if account unknown.
* <p></p>
* @see QortalATAPI#getAccountFromB(MachineState)
*/
GET_BLOCKS_MINTED_FROM_ACCOUNT_IN_B(0x0521, 0, true) {
@Override
protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
QortalATAPI api = (QortalATAPI) state.getAPI();
Account account = api.getAccountFromB(state);
Integer blocksMinted = null;
if (account != null) {
try {
blocksMinted = account.getBlocksMinted();
} catch (DataException e) {
throw new RuntimeException("AT API unable to fetch account's minted block count?", e);
}
}
functionData.returnValue = blocksMinted != null
? blocksMinted.longValue()
: -1;
}
};
public final short value;

View File

@@ -3,18 +3,15 @@ package org.qortal.block;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.math.RoundingMode;
import java.nio.charset.StandardCharsets;
import java.text.DecimalFormat;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.*;
import java.util.stream.Collectors;
import org.apache.logging.log4j.Level;
@@ -28,7 +25,7 @@ import org.qortal.asset.Asset;
import org.qortal.at.AT;
import org.qortal.block.BlockChain.BlockTimingByHeight;
import org.qortal.block.BlockChain.AccountLevelShareBin;
import org.qortal.controller.Controller;
import org.qortal.controller.OnlineAccountsManager;
import org.qortal.crypto.Crypto;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.data.account.AccountData;
@@ -124,6 +121,8 @@ public class Block {
/** Remote/imported/loaded AT states */
protected List<ATStateData> atStates;
/** Remote hash of AT states - in lieu of full AT state data in {@code atStates} */
protected byte[] atStatesHash;
/** Locally-generated AT states */
protected List<ATStateData> ourAtStates;
/** Locally-generated AT fees */
@@ -261,7 +260,7 @@ public class Block {
* Constructs new Block using passed transaction and AT states.
* <p>
* This constructor typically used when receiving a serialized block over the network.
*
*
* @param repository
* @param blockData
* @param transactions
@@ -287,6 +286,35 @@ public class Block {
this.blockData.setTotalFees(totalFees);
}
/**
* Constructs new Block using passed transaction and minimal AT state info.
* <p>
* This constructor typically used when receiving a serialized block over the network.
*
* @param repository
* @param blockData
* @param transactions
* @param atStatesHash
*/
public Block(Repository repository, BlockData blockData, List<TransactionData> transactions, byte[] atStatesHash) {
this(repository, blockData);
this.transactions = new ArrayList<>();
long totalFees = 0;
// We have to sum fees too
for (TransactionData transactionData : transactions) {
this.transactions.add(Transaction.fromData(repository, transactionData));
totalFees += transactionData.getFee();
}
this.atStatesHash = atStatesHash;
totalFees += this.blockData.getATFees();
this.blockData.setTotalFees(totalFees);
}
/**
* Constructs new Block with empty transaction list, using passed minter account.
*
@@ -320,7 +348,7 @@ public class Block {
byte[] reference = parentBlockData.getSignature();
// Fetch our list of online accounts
List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
List<OnlineAccountData> onlineAccounts = OnlineAccountsManager.getInstance().getOnlineAccounts();
if (onlineAccounts.isEmpty()) {
LOGGER.error("No online accounts - not even our own?");
return null;
@@ -333,6 +361,11 @@ public class Block {
onlineAccountsTimestamp = onlineAccountData.getTimestamp();
}
// Load sorted list of reward share public keys into memory, so that the indexes can be obtained.
// This is up to 100x faster than querying each index separately. For 4150 reward share keys, it
// was taking around 5000ms to query individually, vs 50ms using this approach.
List<byte[]> allRewardSharePublicKeys = repository.getAccountRepository().getRewardSharePublicKeys();
// Map using index into sorted list of reward-shares as key
Map<Integer, OnlineAccountData> indexedOnlineAccounts = new HashMap<>();
for (OnlineAccountData onlineAccountData : onlineAccounts) {
@@ -340,7 +373,7 @@ public class Block {
if (onlineAccountData.getTimestamp() != onlineAccountsTimestamp)
continue;
Integer accountIndex = repository.getAccountRepository().getRewardShareIndex(onlineAccountData.getPublicKey());
Integer accountIndex = getRewardShareIndex(onlineAccountData.getPublicKey(), allRewardSharePublicKeys);
if (accountIndex == null)
// Online account (reward-share) with current timestamp but reward-share cancelled
continue;
@@ -988,10 +1021,10 @@ public class Block {
byte[] onlineTimestampBytes = Longs.toByteArray(onlineTimestamp);
// If this block is much older than current online timestamp, then there's no point checking current online accounts
List<OnlineAccountData> currentOnlineAccounts = onlineTimestamp < NTP.getTime() - Controller.ONLINE_TIMESTAMP_MODULUS
List<OnlineAccountData> currentOnlineAccounts = onlineTimestamp < NTP.getTime() - OnlineAccountsManager.ONLINE_TIMESTAMP_MODULUS
? null
: Controller.getInstance().getOnlineAccounts();
List<OnlineAccountData> latestBlocksOnlineAccounts = Controller.getInstance().getLatestBlocksOnlineAccounts();
: OnlineAccountsManager.getInstance().getOnlineAccounts();
List<OnlineAccountData> latestBlocksOnlineAccounts = OnlineAccountsManager.getInstance().getLatestBlocksOnlineAccounts();
// Extract online accounts' timestamp signatures from block data
List<byte[]> onlineAccountsSignatures = BlockTransformer.decodeTimestampSignatures(this.blockData.getOnlineAccountsSignatures());
@@ -1195,7 +1228,7 @@ public class Block {
*/
private ValidationResult areAtsValid() throws DataException {
// Locally generated AT states should be valid so no need to re-execute them
if (this.ourAtStates == this.getATStates()) // Note object reference compare
if (this.ourAtStates != null && this.ourAtStates == this.atStates) // Note object reference compare
return ValidationResult.OK;
// Generate local AT states for comparison
@@ -1209,8 +1242,33 @@ public class Block {
if (this.ourAtFees != this.blockData.getATFees())
return ValidationResult.AT_STATES_MISMATCH;
// Note: this.atStates fully loaded thanks to this.getATStates() call above
for (int s = 0; s < this.atStates.size(); ++s) {
// If we have a single AT states hash then compare that in preference
if (this.atStatesHash != null) {
int atBytesLength = blockData.getATCount() * BlockTransformer.AT_ENTRY_LENGTH;
ByteArrayOutputStream atHashBytes = new ByteArrayOutputStream(atBytesLength);
try {
for (ATStateData atStateData : this.ourAtStates) {
atHashBytes.write(atStateData.getATAddress().getBytes(StandardCharsets.UTF_8));
atHashBytes.write(atStateData.getStateHash());
atHashBytes.write(Longs.toByteArray(atStateData.getFees()));
}
} catch (IOException e) {
throw new DataException("Couldn't validate AT states hash due to serialization issue?", e);
}
byte[] ourAtStatesHash = Crypto.digest(atHashBytes.toByteArray());
if (!Arrays.equals(ourAtStatesHash, this.atStatesHash))
return ValidationResult.AT_STATES_MISMATCH;
// Use our AT state data from now on
this.atStates = this.ourAtStates;
return ValidationResult.OK;
}
// Note: this.atStates fully loaded thanks to this.getATStates() call:
this.getATStates();
for (int s = 0; s < this.ourAtStates.size(); ++s) {
ATStateData ourAtState = this.ourAtStates.get(s);
ATStateData theirAtState = this.atStates.get(s);
@@ -1369,7 +1427,7 @@ public class Block {
postBlockTidy();
// Give Controller our cached, valid online accounts data (if any) to help reduce CPU load for next block
Controller.getInstance().pushLatestBlocksOnlineAccounts(this.cachedValidOnlineAccounts);
OnlineAccountsManager.getInstance().pushLatestBlocksOnlineAccounts(this.cachedValidOnlineAccounts);
// Log some debugging info relating to the block weight calculation
this.logDebugInfo();
@@ -1588,7 +1646,7 @@ public class Block {
postBlockTidy();
// Remove any cached, valid online accounts data from Controller
Controller.getInstance().popLatestBlocksOnlineAccounts();
OnlineAccountsManager.getInstance().popLatestBlocksOnlineAccounts();
}
protected void orphanTransactionsFromBlock() throws DataException {
@@ -2029,6 +2087,26 @@ public class Block {
this.repository.getAccountRepository().tidy();
}
// Utils
/**
* Find index of rewardSharePublicKey in list of rewardSharePublicKeys
*
* @param rewardSharePublicKey - the key to query
* @param rewardSharePublicKeys - a sorted list of keys
* @return - the index of the key, or null if not found
*/
private static Integer getRewardShareIndex(byte[] rewardSharePublicKey, List<byte[]> rewardSharePublicKeys) {
int index = 0;
for (byte[] publicKey : rewardSharePublicKeys) {
if (Arrays.equals(rewardSharePublicKey, publicKey)) {
return index;
}
index++;
}
return null;
}
private void logDebugInfo() {
try {
// Avoid calculations if possible. We have to check against INFO here, since Level.isMoreSpecificThan() confusingly uses <= rather than just <

View File

@@ -69,13 +69,19 @@ public class BlockChain {
newBlockSigHeight,
shareBinFix,
calcChainWeightTimestamp,
transactionV5Timestamp;
transactionV5Timestamp,
transactionV6Timestamp,
disableReferenceTimestamp
}
// Custom transaction fees
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
private long nameRegistrationUnitFee;
private long nameRegistrationUnitFeeTimestamp;
/** Unit fees by transaction timestamp */
public static class UnitFeesByTimestamp {
public long timestamp;
@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
public long fee;
}
private List<UnitFeesByTimestamp> nameRegistrationUnitFees;
/** Map of which blockchain features are enabled when (height/timestamp) */
@XmlJavaTypeAdapter(StringLongMapXmlAdapter.class)
@@ -306,16 +312,6 @@ public class BlockChain {
return this.maxBlockSize;
}
// Custom transaction fees
public long getNameRegistrationUnitFee() {
return this.nameRegistrationUnitFee;
}
public long getNameRegistrationUnitFeeTimestamp() {
// FUTURE: we could use a separate structure to indicate fee adjustments for different transaction types
return this.nameRegistrationUnitFeeTimestamp;
}
/** Returns true if approval-needing transaction types require a txGroupId other than NO_GROUP. */
public boolean getRequireGroupForApproval() {
return this.requireGroupForApproval;
@@ -411,6 +407,14 @@ public class BlockChain {
return this.featureTriggers.get(FeatureTrigger.transactionV5Timestamp.name()).longValue();
}
public long getTransactionV6Timestamp() {
return this.featureTriggers.get(FeatureTrigger.transactionV6Timestamp.name()).longValue();
}
public long getDisableReferenceTimestamp() {
return this.featureTriggers.get(FeatureTrigger.disableReferenceTimestamp.name()).longValue();
}
// More complex getters for aspects that change by height or timestamp
public long getRewardAtHeight(int ourHeight) {
@@ -430,6 +434,15 @@ public class BlockChain {
throw new IllegalStateException(String.format("No block timing info available for height %d", ourHeight));
}
public long getNameRegistrationUnitFeeAtTimestamp(long ourTimestamp) {
for (int i = nameRegistrationUnitFees.size() - 1; i >= 0; --i)
if (nameRegistrationUnitFees.get(i).timestamp <= ourTimestamp)
return nameRegistrationUnitFees.get(i).fee;
// Default to system-wide unit fee
return this.getUnitFee();
}
/** Validate blockchain config read from JSON */
private void validateConfig() {
if (this.genesisInfo == null)

View File

@@ -61,6 +61,11 @@ public class BlockMinter extends Thread {
public void run() {
Thread.currentThread().setName("BlockMinter");
if (Settings.getInstance().isLite()) {
// Lite nodes do not mint
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
if (Settings.getInstance().getWipeUnconfirmedOnStart()) {
// Wipe existing unconfirmed transactions
@@ -110,7 +115,7 @@ public class BlockMinter extends Thread {
continue;
// No online accounts? (e.g. during startup)
if (Controller.getInstance().getOnlineAccounts().isEmpty())
if (OnlineAccountsManager.getInstance().getOnlineAccounts().isEmpty())
continue;
List<MintingAccountData> mintingAccountsData = repository.getAccountRepository().getMintingAccounts();
@@ -148,7 +153,8 @@ public class BlockMinter extends Thread {
}
}
List<Peer> peers = Network.getInstance().getHandshakedPeers();
// Needs a mutable copy of the unmodifiableList
List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
BlockData lastBlockData = blockRepository.getLastBlock();
// Disregard peers that have "misbehaved" recently
@@ -218,7 +224,7 @@ public class BlockMinter extends Thread {
// The last iteration found a higher weight block in the network, so sleep for a while
// to allow is to sync the higher weight chain. We are sleeping here rather than when
// detected as we don't want to hold the blockchain lock open.
LOGGER.info("Sleeping for 10 seconds...");
LOGGER.debug("Sleeping for 10 seconds...");
Thread.sleep(10 * 1000L);
}
@@ -327,13 +333,13 @@ public class BlockMinter extends Thread {
// If less than 30 seconds has passed since first detection the higher weight chain,
// we should skip our block submission to give us the opportunity to sync to the better chain
if (NTP.getTime() - timeOfLastLowWeightBlock < 30*1000L) {
LOGGER.info("Higher weight chain found in peers, so not signing a block this round");
LOGGER.info("Time since detected: {}", NTP.getTime() - timeOfLastLowWeightBlock);
LOGGER.debug("Higher weight chain found in peers, so not signing a block this round");
LOGGER.debug("Time since detected: {}ms", NTP.getTime() - timeOfLastLowWeightBlock);
continue;
}
else {
// More than 30 seconds have passed, so we should submit our block candidate anyway.
LOGGER.info("More than 30 seconds passed, so proceeding to submit block candidate...");
LOGGER.debug("More than 30 seconds passed, so proceeding to submit block candidate...");
}
}
else {
@@ -478,7 +484,7 @@ public class BlockMinter extends Thread {
throw new DataException("Ignoring attempt to mint testing block for non-test chain!");
// Ensure mintingAccount is 'online' so blocks can be minted
Controller.getInstance().ensureTestingAccountsOnline(mintingAndOnlineAccounts);
OnlineAccountsManager.getInstance().ensureTestingAccountsOnline(mintingAndOnlineAccounts);
PrivateKeyAccount mintingAccount = mintingAndOnlineAccounts[0];
@@ -544,7 +550,7 @@ public class BlockMinter extends Thread {
}
NumberFormat formatter = new DecimalFormat("0.###E0");
List<Peer> peers = Network.getInstance().getHandshakedPeers();
List<Peer> peers = Network.getInstance().getImmutableHandshakedPeers();
// Loop through handshaked peers and check for any new block candidates
for (Peer peer : peers) {
if (peer.getCommonBlockData() != null && peer.getCommonBlockData().getCommonBlockSummary() != null) {

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,189 @@
package org.qortal.controller;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.data.account.AccountData;
import org.qortal.data.naming.NameData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.*;
import java.security.SecureRandom;
import java.util.*;
import static org.qortal.network.message.MessageType.*;
public class LiteNode {
private static final Logger LOGGER = LogManager.getLogger(LiteNode.class);
private static LiteNode instance;
public Map<Integer, Long> pendingRequests = Collections.synchronizedMap(new HashMap<>());
public int MAX_TRANSACTIONS_PER_MESSAGE = 100;
public LiteNode() {
}
public static synchronized LiteNode getInstance() {
if (instance == null) {
instance = new LiteNode();
}
return instance;
}
/**
* Fetch account data from peers for given QORT address
* @param address - the QORT address to query
* @return accountData - the account data for this address, or null if not retrieved
*/
public AccountData fetchAccountData(String address) {
GetAccountMessage getAccountMessage = new GetAccountMessage(address);
AccountMessage accountMessage = (AccountMessage) this.sendMessage(getAccountMessage, ACCOUNT);
if (accountMessage == null) {
return null;
}
return accountMessage.getAccountData();
}
/**
* Fetch account balance data from peers for given QORT address and asset ID
* @param address - the QORT address to query
* @return balance - the balance for this address and assetId, or null if not retrieved
*/
public AccountBalanceData fetchAccountBalance(String address, long assetId) {
GetAccountBalanceMessage getAccountMessage = new GetAccountBalanceMessage(address, assetId);
AccountBalanceMessage accountMessage = (AccountBalanceMessage) this.sendMessage(getAccountMessage, ACCOUNT_BALANCE);
if (accountMessage == null) {
return null;
}
return accountMessage.getAccountBalanceData();
}
/**
* Fetch list of transactions for given QORT address
* @param address - the QORT address to query
* @param limit - the maximum number of results to return
* @param offset - the starting index
* @return a list of TransactionData objects, or null if not retrieved
*/
public List<TransactionData> fetchAccountTransactions(String address, int limit, int offset) {
List<TransactionData> allTransactions = new ArrayList<>();
if (limit == 0) {
limit = Integer.MAX_VALUE;
}
int batchSize = Math.min(limit, MAX_TRANSACTIONS_PER_MESSAGE);
while (allTransactions.size() < limit) {
GetAccountTransactionsMessage getAccountTransactionsMessage = new GetAccountTransactionsMessage(address, batchSize, offset);
TransactionsMessage transactionsMessage = (TransactionsMessage) this.sendMessage(getAccountTransactionsMessage, TRANSACTIONS);
if (transactionsMessage == null) {
// An error occurred, so give up instead of returning partial results
return null;
}
allTransactions.addAll(transactionsMessage.getTransactions());
if (transactionsMessage.getTransactions().size() < batchSize) {
// No more transactions to fetch
break;
}
offset += batchSize;
}
return allTransactions;
}
/**
* Fetch list of names for given QORT address
* @param address - the QORT address to query
* @return a list of NameData objects, or null if not retrieved
*/
public List<NameData> fetchAccountNames(String address) {
GetAccountNamesMessage getAccountNamesMessage = new GetAccountNamesMessage(address);
NamesMessage namesMessage = (NamesMessage) this.sendMessage(getAccountNamesMessage, NAMES);
if (namesMessage == null) {
return null;
}
return namesMessage.getNameDataList();
}
/**
* Fetch info about a registered name
* @param name - the name to query
* @return a NameData object, or null if not retrieved
*/
public NameData fetchNameData(String name) {
GetNameMessage getNameMessage = new GetNameMessage(name);
NamesMessage namesMessage = (NamesMessage) this.sendMessage(getNameMessage, NAMES);
if (namesMessage == null) {
return null;
}
List<NameData> nameDataList = namesMessage.getNameDataList();
if (nameDataList == null || nameDataList.size() != 1) {
return null;
}
// We are only expecting a single item in the list
return nameDataList.get(0);
}
private Message sendMessage(Message message, MessageType expectedResponseMessageType) {
// This asks a random peer for the data
// TODO: ask multiple peers, and disregard everything if there are any significant differences in the responses
// Needs a mutable copy of the unmodifiableList
List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
// Disregard peers that have "misbehaved" recently
peers.removeIf(Controller.hasMisbehaved);
// Disregard peers that only have genesis block
// TODO: peers.removeIf(Controller.hasOnlyGenesisBlock);
// Disregard peers that are on an old version
peers.removeIf(Controller.hasOldVersion);
// Disregard peers that are on a known inferior chain tip
// TODO: peers.removeIf(Controller.hasInferiorChainTip);
if (peers.isEmpty()) {
LOGGER.info("No peers available to send {} message to", message.getType());
return null;
}
// Pick random peer
int index = new SecureRandom().nextInt(peers.size());
Peer peer = peers.get(index);
LOGGER.info("Sending {} message to peer {}...", message.getType(), peer);
Message responseMessage;
try {
responseMessage = peer.getResponse(message);
} catch (InterruptedException e) {
return null;
}
if (responseMessage == null) {
LOGGER.info("Peer {} didn't respond to {} message", peer, message.getType());
return null;
}
else if (responseMessage.getType() != expectedResponseMessageType) {
LOGGER.info("Peer responded with unexpected message type {} (should be {})", peer, responseMessage.getType(), expectedResponseMessageType);
return null;
}
LOGGER.info("Peer {} responded with {} message", peer, responseMessage.getType());
return responseMessage;
}
}

View File

@@ -0,0 +1,524 @@
package org.qortal.controller;
import com.google.common.primitives.Longs;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.block.BlockChain;
import org.qortal.data.account.MintingAccountData;
import org.qortal.data.account.RewardShareData;
import org.qortal.data.network.OnlineAccountData;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.*;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.*;
import java.util.stream.Collectors;
public class OnlineAccountsManager extends Thread {
private class OurOnlineAccountsThread extends Thread {
public void run() {
try {
while (!isStopping) {
Thread.sleep(10000L);
// Refresh our online accounts signatures?
sendOurOnlineAccountsInfo();
}
} catch (InterruptedException e) {
// Fall through to exit thread
}
}
}
private static final Logger LOGGER = LogManager.getLogger(OnlineAccountsManager.class);
private static OnlineAccountsManager instance;
private volatile boolean isStopping = false;
// To do with online accounts list
private static final long ONLINE_ACCOUNTS_TASKS_INTERVAL = 10 * 1000L; // ms
private static final long ONLINE_ACCOUNTS_BROADCAST_INTERVAL = 1 * 60 * 1000L; // ms
public static final long ONLINE_TIMESTAMP_MODULUS = 5 * 60 * 1000L;
private static final long LAST_SEEN_EXPIRY_PERIOD = (ONLINE_TIMESTAMP_MODULUS * 2) + (1 * 60 * 1000L);
/** How many (latest) blocks' worth of online accounts we cache */
private static final int MAX_BLOCKS_CACHED_ONLINE_ACCOUNTS = 2;
private static final long ONLINE_ACCOUNTS_V2_PEER_VERSION = 0x0300020000L;
private long onlineAccountsTasksTimestamp = Controller.startTime + ONLINE_ACCOUNTS_TASKS_INTERVAL; // ms
private final List<OnlineAccountData> onlineAccountsImportQueue = Collections.synchronizedList(new ArrayList<>());
/** Cache of current 'online accounts' */
List<OnlineAccountData> onlineAccounts = new ArrayList<>();
/** Cache of latest blocks' online accounts */
Deque<List<OnlineAccountData>> latestBlocksOnlineAccounts = new ArrayDeque<>(MAX_BLOCKS_CACHED_ONLINE_ACCOUNTS);
public OnlineAccountsManager() {
}
public static synchronized OnlineAccountsManager getInstance() {
if (instance == null) {
instance = new OnlineAccountsManager();
}
return instance;
}
public void run() {
// Start separate thread to prepare our online accounts
// This could be converted to a thread pool later if more concurrency is needed
OurOnlineAccountsThread ourOnlineAccountsThread = new OurOnlineAccountsThread();
ourOnlineAccountsThread.start();
try {
while (!Controller.isStopping()) {
Thread.sleep(100L);
final Long now = NTP.getTime();
if (now == null) {
continue;
}
// Perform tasks to do with managing online accounts list
if (now >= onlineAccountsTasksTimestamp) {
onlineAccountsTasksTimestamp = now + ONLINE_ACCOUNTS_TASKS_INTERVAL;
performOnlineAccountsTasks();
}
// Process queued online account verifications
this.processOnlineAccountsImportQueue();
}
} catch (InterruptedException e) {
// Fall through to exit thread
}
ourOnlineAccountsThread.interrupt();
}
public void shutdown() {
isStopping = true;
this.interrupt();
}
// Online accounts import queue
private void processOnlineAccountsImportQueue() {
if (this.onlineAccountsImportQueue.isEmpty()) {
// Nothing to do
return;
}
LOGGER.debug("Processing online accounts import queue (size: {})", this.onlineAccountsImportQueue.size());
try (final Repository repository = RepositoryManager.getRepository()) {
List<OnlineAccountData> onlineAccountDataCopy = new ArrayList<>(this.onlineAccountsImportQueue);
for (OnlineAccountData onlineAccountData : onlineAccountDataCopy) {
if (isStopping) {
return;
}
this.verifyAndAddAccount(repository, onlineAccountData);
// Remove from queue
onlineAccountsImportQueue.remove(onlineAccountData);
}
LOGGER.debug("Finished processing online accounts import queue");
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while verifying online accounts"), e);
}
}
// Utilities
private void verifyAndAddAccount(Repository repository, OnlineAccountData onlineAccountData) throws DataException {
final Long now = NTP.getTime();
if (now == null)
return;
PublicKeyAccount otherAccount = new PublicKeyAccount(repository, onlineAccountData.getPublicKey());
// Check timestamp is 'recent' here
if (Math.abs(onlineAccountData.getTimestamp() - now) > ONLINE_TIMESTAMP_MODULUS * 2) {
LOGGER.trace(() -> String.format("Rejecting online account %s with out of range timestamp %d", otherAccount.getAddress(), onlineAccountData.getTimestamp()));
return;
}
// Verify
byte[] data = Longs.toByteArray(onlineAccountData.getTimestamp());
if (!otherAccount.verify(onlineAccountData.getSignature(), data)) {
LOGGER.trace(() -> String.format("Rejecting invalid online account %s", otherAccount.getAddress()));
return;
}
// Qortal: check online account is actually reward-share
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(onlineAccountData.getPublicKey());
if (rewardShareData == null) {
// Reward-share doesn't even exist - probably not a good sign
LOGGER.trace(() -> String.format("Rejecting unknown online reward-share public key %s", Base58.encode(onlineAccountData.getPublicKey())));
return;
}
Account mintingAccount = new Account(repository, rewardShareData.getMinter());
if (!mintingAccount.canMint()) {
// Minting-account component of reward-share can no longer mint - disregard
LOGGER.trace(() -> String.format("Rejecting online reward-share with non-minting account %s", mintingAccount.getAddress()));
return;
}
synchronized (this.onlineAccounts) {
OnlineAccountData existingAccountData = this.onlineAccounts.stream().filter(account -> Arrays.equals(account.getPublicKey(), onlineAccountData.getPublicKey())).findFirst().orElse(null);
if (existingAccountData != null) {
if (existingAccountData.getTimestamp() < onlineAccountData.getTimestamp()) {
this.onlineAccounts.remove(existingAccountData);
LOGGER.trace(() -> String.format("Updated online account %s with timestamp %d (was %d)", otherAccount.getAddress(), onlineAccountData.getTimestamp(), existingAccountData.getTimestamp()));
} else {
LOGGER.trace(() -> String.format("Not updating existing online account %s", otherAccount.getAddress()));
return;
}
} else {
LOGGER.trace(() -> String.format("Added online account %s with timestamp %d", otherAccount.getAddress(), onlineAccountData.getTimestamp()));
}
this.onlineAccounts.add(onlineAccountData);
}
}
public void ensureTestingAccountsOnline(PrivateKeyAccount... onlineAccounts) {
if (!BlockChain.getInstance().isTestChain()) {
LOGGER.warn("Ignoring attempt to ensure test account is online for non-test chain!");
return;
}
final Long now = NTP.getTime();
if (now == null)
return;
final long onlineAccountsTimestamp = toOnlineAccountTimestamp(now);
byte[] timestampBytes = Longs.toByteArray(onlineAccountsTimestamp);
synchronized (this.onlineAccounts) {
this.onlineAccounts.clear();
for (PrivateKeyAccount onlineAccount : onlineAccounts) {
// Check mintingAccount is actually reward-share?
byte[] signature = onlineAccount.sign(timestampBytes);
byte[] publicKey = onlineAccount.getPublicKey();
OnlineAccountData ourOnlineAccountData = new OnlineAccountData(onlineAccountsTimestamp, signature, publicKey);
this.onlineAccounts.add(ourOnlineAccountData);
}
}
}
private void performOnlineAccountsTasks() {
final Long now = NTP.getTime();
if (now == null)
return;
// Expire old entries
final long cutoffThreshold = now - LAST_SEEN_EXPIRY_PERIOD;
synchronized (this.onlineAccounts) {
Iterator<OnlineAccountData> iterator = this.onlineAccounts.iterator();
while (iterator.hasNext()) {
OnlineAccountData onlineAccountData = iterator.next();
if (onlineAccountData.getTimestamp() < cutoffThreshold) {
iterator.remove();
LOGGER.trace(() -> {
PublicKeyAccount otherAccount = new PublicKeyAccount(null, onlineAccountData.getPublicKey());
return String.format("Removed expired online account %s with timestamp %d", otherAccount.getAddress(), onlineAccountData.getTimestamp());
});
}
}
}
// Request data from other peers?
if ((this.onlineAccountsTasksTimestamp % ONLINE_ACCOUNTS_BROADCAST_INTERVAL) < ONLINE_ACCOUNTS_TASKS_INTERVAL) {
List<OnlineAccountData> safeOnlineAccounts;
synchronized (this.onlineAccounts) {
safeOnlineAccounts = new ArrayList<>(this.onlineAccounts);
}
Message messageV1 = new GetOnlineAccountsMessage(safeOnlineAccounts);
Message messageV2 = new GetOnlineAccountsV2Message(safeOnlineAccounts);
Network.getInstance().broadcast(peer ->
peer.getPeersVersion() >= ONLINE_ACCOUNTS_V2_PEER_VERSION ? messageV2 : messageV1
);
}
}
private void sendOurOnlineAccountsInfo() {
final Long now = NTP.getTime();
if (now == null) {
return;
}
List<MintingAccountData> mintingAccounts;
try (final Repository repository = RepositoryManager.getRepository()) {
mintingAccounts = repository.getAccountRepository().getMintingAccounts();
// We have no accounts, but don't reset timestamp
if (mintingAccounts.isEmpty())
return;
// Only reward-share accounts allowed
Iterator<MintingAccountData> iterator = mintingAccounts.iterator();
int i = 0;
while (iterator.hasNext()) {
MintingAccountData mintingAccountData = iterator.next();
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(mintingAccountData.getPublicKey());
if (rewardShareData == null) {
// Reward-share doesn't even exist - probably not a good sign
iterator.remove();
continue;
}
Account mintingAccount = new Account(repository, rewardShareData.getMinter());
if (!mintingAccount.canMint()) {
// Minting-account component of reward-share can no longer mint - disregard
iterator.remove();
continue;
}
if (++i > 1+1) {
iterator.remove();
continue;
}
}
} catch (DataException e) {
LOGGER.warn(String.format("Repository issue trying to fetch minting accounts: %s", e.getMessage()));
return;
}
// 'current' timestamp
final long onlineAccountsTimestamp = toOnlineAccountTimestamp(now);
boolean hasInfoChanged = false;
byte[] timestampBytes = Longs.toByteArray(onlineAccountsTimestamp);
List<OnlineAccountData> ourOnlineAccounts = new ArrayList<>();
MINTING_ACCOUNTS:
for (MintingAccountData mintingAccountData : mintingAccounts) {
PrivateKeyAccount mintingAccount = new PrivateKeyAccount(null, mintingAccountData.getPrivateKey());
byte[] signature = mintingAccount.sign(timestampBytes);
byte[] publicKey = mintingAccount.getPublicKey();
// Our account is online
OnlineAccountData ourOnlineAccountData = new OnlineAccountData(onlineAccountsTimestamp, signature, publicKey);
synchronized (this.onlineAccounts) {
Iterator<OnlineAccountData> iterator = this.onlineAccounts.iterator();
while (iterator.hasNext()) {
OnlineAccountData existingOnlineAccountData = iterator.next();
if (Arrays.equals(existingOnlineAccountData.getPublicKey(), ourOnlineAccountData.getPublicKey())) {
// If our online account is already present, with same timestamp, then move on to next mintingAccount
if (existingOnlineAccountData.getTimestamp() == onlineAccountsTimestamp)
continue MINTING_ACCOUNTS;
// If our online account is already present, but with older timestamp, then remove it
iterator.remove();
break;
}
}
this.onlineAccounts.add(ourOnlineAccountData);
}
LOGGER.trace(() -> String.format("Added our online account %s with timestamp %d", mintingAccount.getAddress(), onlineAccountsTimestamp));
ourOnlineAccounts.add(ourOnlineAccountData);
hasInfoChanged = true;
}
if (!hasInfoChanged)
return;
Message messageV1 = new OnlineAccountsMessage(ourOnlineAccounts);
Message messageV2 = new OnlineAccountsV2Message(ourOnlineAccounts);
Network.getInstance().broadcast(peer ->
peer.getPeersVersion() >= ONLINE_ACCOUNTS_V2_PEER_VERSION ? messageV2 : messageV1
);
LOGGER.trace(() -> String.format("Broadcasted %d online account%s with timestamp %d", ourOnlineAccounts.size(), (ourOnlineAccounts.size() != 1 ? "s" : ""), onlineAccountsTimestamp));
}
public static long toOnlineAccountTimestamp(long timestamp) {
return (timestamp / ONLINE_TIMESTAMP_MODULUS) * ONLINE_TIMESTAMP_MODULUS;
}
/** Returns list of online accounts with timestamp recent enough to be considered currently online. */
public List<OnlineAccountData> getOnlineAccounts() {
final long onlineTimestamp = toOnlineAccountTimestamp(NTP.getTime());
synchronized (this.onlineAccounts) {
return this.onlineAccounts.stream().filter(account -> account.getTimestamp() == onlineTimestamp).collect(Collectors.toList());
}
}
/** Returns cached, unmodifiable list of latest block's online accounts. */
public List<OnlineAccountData> getLatestBlocksOnlineAccounts() {
synchronized (this.latestBlocksOnlineAccounts) {
return this.latestBlocksOnlineAccounts.peekFirst();
}
}
/** Caches list of latest block's online accounts. Typically called by Block.process() */
public void pushLatestBlocksOnlineAccounts(List<OnlineAccountData> latestBlocksOnlineAccounts) {
synchronized (this.latestBlocksOnlineAccounts) {
if (this.latestBlocksOnlineAccounts.size() == MAX_BLOCKS_CACHED_ONLINE_ACCOUNTS)
this.latestBlocksOnlineAccounts.pollLast();
this.latestBlocksOnlineAccounts.addFirst(latestBlocksOnlineAccounts == null
? Collections.emptyList()
: Collections.unmodifiableList(latestBlocksOnlineAccounts));
}
}
/** Reverts list of latest block's online accounts. Typically called by Block.orphan() */
public void popLatestBlocksOnlineAccounts() {
synchronized (this.latestBlocksOnlineAccounts) {
this.latestBlocksOnlineAccounts.pollFirst();
}
}
// Network handlers
public void onNetworkGetOnlineAccountsMessage(Peer peer, Message message) {
GetOnlineAccountsMessage getOnlineAccountsMessage = (GetOnlineAccountsMessage) message;
List<OnlineAccountData> excludeAccounts = getOnlineAccountsMessage.getOnlineAccounts();
// Send online accounts info, excluding entries with matching timestamp & public key from excludeAccounts
List<OnlineAccountData> accountsToSend;
synchronized (this.onlineAccounts) {
accountsToSend = new ArrayList<>(this.onlineAccounts);
}
Iterator<OnlineAccountData> iterator = accountsToSend.iterator();
SEND_ITERATOR:
while (iterator.hasNext()) {
OnlineAccountData onlineAccountData = iterator.next();
for (int i = 0; i < excludeAccounts.size(); ++i) {
OnlineAccountData excludeAccountData = excludeAccounts.get(i);
if (onlineAccountData.getTimestamp() == excludeAccountData.getTimestamp() && Arrays.equals(onlineAccountData.getPublicKey(), excludeAccountData.getPublicKey())) {
iterator.remove();
continue SEND_ITERATOR;
}
}
}
Message onlineAccountsMessage = new OnlineAccountsMessage(accountsToSend);
peer.sendMessage(onlineAccountsMessage);
LOGGER.trace(() -> String.format("Sent %d of our %d online accounts to %s", accountsToSend.size(), this.onlineAccounts.size(), peer));
}
public void onNetworkOnlineAccountsMessage(Peer peer, Message message) {
OnlineAccountsMessage onlineAccountsMessage = (OnlineAccountsMessage) message;
List<OnlineAccountData> peersOnlineAccounts = onlineAccountsMessage.getOnlineAccounts();
LOGGER.trace(() -> String.format("Received %d online accounts from %s", peersOnlineAccounts.size(), peer));
try (final Repository repository = RepositoryManager.getRepository()) {
for (OnlineAccountData onlineAccountData : peersOnlineAccounts)
this.verifyAndAddAccount(repository, onlineAccountData);
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while verifying online accounts from peer %s", peer), e);
}
}
public void onNetworkGetOnlineAccountsV2Message(Peer peer, Message message) {
GetOnlineAccountsV2Message getOnlineAccountsMessage = (GetOnlineAccountsV2Message) message;
List<OnlineAccountData> excludeAccounts = getOnlineAccountsMessage.getOnlineAccounts();
// Send online accounts info, excluding entries with matching timestamp & public key from excludeAccounts
List<OnlineAccountData> accountsToSend;
synchronized (this.onlineAccounts) {
accountsToSend = new ArrayList<>(this.onlineAccounts);
}
Iterator<OnlineAccountData> iterator = accountsToSend.iterator();
SEND_ITERATOR:
while (iterator.hasNext()) {
OnlineAccountData onlineAccountData = iterator.next();
for (int i = 0; i < excludeAccounts.size(); ++i) {
OnlineAccountData excludeAccountData = excludeAccounts.get(i);
if (onlineAccountData.getTimestamp() == excludeAccountData.getTimestamp() && Arrays.equals(onlineAccountData.getPublicKey(), excludeAccountData.getPublicKey())) {
iterator.remove();
continue SEND_ITERATOR;
}
}
}
Message onlineAccountsMessage = new OnlineAccountsV2Message(accountsToSend);
peer.sendMessage(onlineAccountsMessage);
LOGGER.trace(() -> String.format("Sent %d of our %d online accounts to %s", accountsToSend.size(), this.onlineAccounts.size(), peer));
}
public void onNetworkOnlineAccountsV2Message(Peer peer, Message message) {
OnlineAccountsV2Message onlineAccountsMessage = (OnlineAccountsV2Message) message;
List<OnlineAccountData> peersOnlineAccounts = onlineAccountsMessage.getOnlineAccounts();
LOGGER.debug(String.format("Received %d online accounts from %s", peersOnlineAccounts.size(), peer));
int importCount = 0;
// Add any online accounts to the queue that aren't already present
for (OnlineAccountData onlineAccountData : peersOnlineAccounts) {
// Do we already know about this online account data?
if (onlineAccounts.contains(onlineAccountData)) {
continue;
}
// Is it already in the import queue?
if (onlineAccountsImportQueue.contains(onlineAccountData)) {
continue;
}
onlineAccountsImportQueue.add(onlineAccountData);
importCount++;
}
LOGGER.debug(String.format("Added %d online accounts to queue", importCount));
}
}

View File

@@ -22,16 +22,11 @@ import org.qortal.data.block.CommonBlockData;
import org.qortal.data.network.PeerChainTipData;
import org.qortal.data.transaction.RewardShareTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.BlockMessage;
import org.qortal.network.message.BlockSummariesMessage;
import org.qortal.network.message.GetBlockMessage;
import org.qortal.network.message.GetBlockSummariesMessage;
import org.qortal.network.message.GetSignaturesV2Message;
import org.qortal.network.message.Message;
import org.qortal.network.message.SignaturesMessage;
import org.qortal.network.message.Message.MessageType;
import org.qortal.network.message.*;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@@ -93,7 +88,25 @@ public class Synchronizer extends Thread {
private static Synchronizer instance;
public enum SynchronizationResult {
OK, NOTHING_TO_DO, GENESIS_ONLY, NO_COMMON_BLOCK, TOO_DIVERGENT, NO_REPLY, INFERIOR_CHAIN, INVALID_DATA, NO_BLOCKCHAIN_LOCK, REPOSITORY_ISSUE, SHUTTING_DOWN;
OK, NOTHING_TO_DO, GENESIS_ONLY, NO_COMMON_BLOCK, TOO_DIVERGENT, NO_REPLY, INFERIOR_CHAIN, INVALID_DATA, NO_BLOCKCHAIN_LOCK, REPOSITORY_ISSUE, SHUTTING_DOWN, CHAIN_TIP_TOO_OLD;
}
public static class NewChainTipEvent implements Event {
private final BlockData priorChainTip;
private final BlockData newChainTip;
public NewChainTipEvent(BlockData priorChainTip, BlockData newChainTip) {
this.priorChainTip = priorChainTip;
this.newChainTip = newChainTip;
}
public BlockData getPriorChainTip() {
return this.priorChainTip;
}
public BlockData getNewChainTip() {
return this.newChainTip;
}
}
// Constructors
@@ -114,6 +127,11 @@ public class Synchronizer extends Thread {
public void run() {
Thread.currentThread().setName("Synchronizer");
if (Settings.getInstance().isLite()) {
// Lite nodes don't need to sync
return;
}
try {
while (running && !Controller.isStopping()) {
Thread.sleep(1000);
@@ -153,6 +171,12 @@ public class Synchronizer extends Thread {
public Integer getSyncPercent() {
synchronized (this.syncLock) {
// Report as 100% synced if the latest block is within the last 30 mins
final Long minLatestBlockTimestamp = NTP.getTime() - (30 * 60 * 1000L);
if (Controller.getInstance().isUpToDate(minLatestBlockTimestamp)) {
return 100;
}
return this.isSynchronizing ? this.syncPercent : null;
}
}
@@ -175,7 +199,8 @@ public class Synchronizer extends Thread {
if (this.isSynchronizing)
return true;
List<Peer> peers = Network.getInstance().getHandshakedPeers();
// Needs a mutable copy of the unmodifiableList
List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
// Disregard peers that have "misbehaved" recently
peers.removeIf(Controller.hasMisbehaved);
@@ -191,7 +216,8 @@ public class Synchronizer extends Thread {
checkRecoveryModeForPeers(peers);
if (recoveryMode) {
peers = Network.getInstance().getHandshakedPeers();
// Needs a mutable copy of the unmodifiableList
peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
peers.removeIf(Controller.hasOnlyGenesisBlock);
peers.removeIf(Controller.hasMisbehaved);
peers.removeIf(Controller.hasOldVersion);
@@ -218,6 +244,12 @@ public class Synchronizer extends Thread {
// We may have added more inferior chain tips when comparing peers, so remove any peers that are currently on those chains
peers.removeIf(Controller.hasInferiorChainTip);
// Remove any peers that are no longer on a recent block since the last check
// Except for times when we're in recovery mode, in which case we need to keep them
if (!recoveryMode) {
peers.removeIf(Controller.hasNoRecentBlock);
}
final int peersRemoved = peersBeforeComparison - peers.size();
if (peersRemoved > 0 && peers.size() > 0)
LOGGER.debug(String.format("Ignoring %d peers on inferior chains. Peers remaining: %d", peersRemoved, peers.size()));
@@ -280,7 +312,7 @@ public class Synchronizer extends Thread {
case INFERIOR_CHAIN: {
// Update our list of inferior chain tips
ByteArray inferiorChainSignature = new ByteArray(peer.getChainTipData().getLastBlockSignature());
ByteArray inferiorChainSignature = ByteArray.wrap(peer.getChainTipData().getLastBlockSignature());
if (!inferiorChainSignatures.contains(inferiorChainSignature))
inferiorChainSignatures.add(inferiorChainSignature);
@@ -296,6 +328,7 @@ public class Synchronizer extends Thread {
case NO_REPLY:
case NO_BLOCKCHAIN_LOCK:
case REPOSITORY_ISSUE:
case CHAIN_TIP_TOO_OLD:
// These are minor failure results so fine to try again
LOGGER.debug(() -> String.format("Failed to synchronize with peer %s (%s)", peer, syncResult.name()));
break;
@@ -308,7 +341,7 @@ public class Synchronizer extends Thread {
// fall-through...
case NOTHING_TO_DO: {
// Update our list of inferior chain tips
ByteArray inferiorChainSignature = new ByteArray(peer.getChainTipData().getLastBlockSignature());
ByteArray inferiorChainSignature = ByteArray.wrap(peer.getChainTipData().getLastBlockSignature());
if (!inferiorChainSignatures.contains(inferiorChainSignature))
inferiorChainSignatures.add(inferiorChainSignature);
@@ -338,6 +371,8 @@ public class Synchronizer extends Thread {
Network network = Network.getInstance();
network.broadcast(broadcastPeer -> network.buildHeightMessage(broadcastPeer, newChainTip));
EventBus.INSTANCE.notify(new NewChainTipEvent(priorChainTip, newChainTip));
}
return syncResult;
@@ -348,7 +383,7 @@ public class Synchronizer extends Thread {
}
private boolean checkRecoveryModeForPeers(List<Peer> qualifiedPeers) {
List<Peer> handshakedPeers = Network.getInstance().getHandshakedPeers();
List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
if (handshakedPeers.size() > 0) {
// There is at least one handshaked peer
@@ -382,7 +417,7 @@ public class Synchronizer extends Thread {
public void addInferiorChainSignature(byte[] inferiorSignature) {
// Update our list of inferior chain tips
ByteArray inferiorChainSignature = new ByteArray(inferiorSignature);
ByteArray inferiorChainSignature = ByteArray.wrap(inferiorSignature);
if (!inferiorChainSignatures.contains(inferiorChainSignature))
inferiorChainSignatures.add(inferiorChainSignature);
}
@@ -533,7 +568,7 @@ public class Synchronizer extends Thread {
// If our latest block is very old, it's best that we don't try and determine the best peers to sync to.
// This is because it can involve very large chain comparisons, which is too intensive.
// In reality, most forking problems occur near the chain tips, so we will reserve this functionality for those situations.
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (minLatestBlockTimestamp == null)
return peers;
@@ -689,6 +724,7 @@ public class Synchronizer extends Thread {
LOGGER.debug(String.format("Listing peers with common block %.8s...", Base58.encode(commonBlockSummary.getSignature())));
for (Peer peer : peersSharingCommonBlock) {
final int peerHeight = peer.getChainTipData().getLastHeight();
final Long peerLastBlockTimestamp = peer.getChainTipData().getLastBlockTimestamp();
final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
final CommonBlockData peerCommonBlockData = peer.getCommonBlockData();
@@ -699,6 +735,14 @@ public class Synchronizer extends Thread {
continue;
}
// If peer is our of date (since our last check), we should exclude it from this round
minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
if (peerLastBlockTimestamp == null || peerLastBlockTimestamp < minLatestBlockTimestamp) {
LOGGER.debug(String.format("Peer %s is out of date - removing it from this round", peer));
peers.remove(peer);
continue;
}
final List<BlockSummaryData> peerBlockSummariesAfterCommonBlock = peerCommonBlockData.getBlockSummariesAfterCommonBlock();
populateBlockSummariesMinterLevels(repository, peerBlockSummariesAfterCommonBlock);
@@ -1261,6 +1305,16 @@ public class Synchronizer extends Thread {
return SynchronizationResult.INVALID_DATA;
}
// Final check to make sure the peer isn't out of date (except for when we're in recovery mode)
if (!recoveryMode && peer.getChainTipData() != null) {
final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
final Long peerLastBlockTimestamp = peer.getChainTipData().getLastBlockTimestamp();
if (peerLastBlockTimestamp == null || peerLastBlockTimestamp < minLatestBlockTimestamp) {
LOGGER.info(String.format("Peer %s is out of date, so abandoning sync attempt", peer));
return SynchronizationResult.CHAIN_TIP_TOO_OLD;
}
}
byte[] nextPeerSignature = peerBlockSignatures.get(0);
int nextHeight = height + 1;
@@ -1518,12 +1572,23 @@ public class Synchronizer extends Thread {
Message getBlockMessage = new GetBlockMessage(signature);
Message message = peer.getResponse(getBlockMessage);
if (message == null || message.getType() != MessageType.BLOCK)
if (message == null)
return null;
BlockMessage blockMessage = (BlockMessage) message;
switch (message.getType()) {
case BLOCK: {
BlockMessage blockMessage = (BlockMessage) message;
return new Block(repository, blockMessage.getBlockData(), blockMessage.getTransactions(), blockMessage.getAtStates());
}
return new Block(repository, blockMessage.getBlockData(), blockMessage.getTransactions(), blockMessage.getAtStates());
case BLOCK_V2: {
BlockV2Message blockMessage = (BlockV2Message) message;
return new Block(repository, blockMessage.getBlockData(), blockMessage.getTransactions(), blockMessage.getAtStatesHash());
}
default:
return null;
}
}
public void populateBlockSummariesMinterLevels(Repository repository, List<BlockSummaryData> blockSummaries) throws DataException {

View File

@@ -0,0 +1,441 @@
package org.qortal.controller;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.data.block.BlockData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.GetTransactionMessage;
import org.qortal.network.message.Message;
import org.qortal.network.message.TransactionMessage;
import org.qortal.network.message.TransactionSignaturesMessage;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.transaction.Transaction;
import org.qortal.transform.TransformationException;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.*;
import java.util.concurrent.locks.ReentrantLock;
import java.util.stream.Collectors;
public class TransactionImporter extends Thread {
private static final Logger LOGGER = LogManager.getLogger(TransactionImporter.class);
private static TransactionImporter instance;
private volatile boolean isStopping = false;
private static final int MAX_INCOMING_TRANSACTIONS = 5000;
/** Minimum time before considering an invalid unconfirmed transaction as "stale" */
public static final long INVALID_TRANSACTION_STALE_TIMEOUT = 30 * 60 * 1000L; // ms
/** Minimum frequency to re-request stale unconfirmed transactions from peers, to recheck validity */
public static final long INVALID_TRANSACTION_RECHECK_INTERVAL = 60 * 60 * 1000L; // ms\
/** Minimum frequency to re-request expired unconfirmed transactions from peers, to recheck validity
* This mainly exists to stop expired transactions from bloating the list */
public static final long EXPIRED_TRANSACTION_RECHECK_INTERVAL = 10 * 60 * 1000L; // ms
/** Map of incoming transaction that are in the import queue. Key is transaction data, value is whether signature has been validated. */
private final Map<TransactionData, Boolean> incomingTransactions = Collections.synchronizedMap(new HashMap<>());
/** Map of recent invalid unconfirmed transactions. Key is base58 transaction signature, value is do-not-request expiry timestamp. */
private final Map<String, Long> invalidUnconfirmedTransactions = Collections.synchronizedMap(new HashMap<>());
public static synchronized TransactionImporter getInstance() {
if (instance == null) {
instance = new TransactionImporter();
}
return instance;
}
@Override
public void run() {
Thread.currentThread().setName("Transaction Importer");
try {
while (!Controller.isStopping()) {
Thread.sleep(500L);
// Process incoming transactions queue
validateTransactionsInQueue();
importTransactionsInQueue();
// Clean up invalid incoming transactions list
cleanupInvalidTransactionsList(NTP.getTime());
}
} catch (InterruptedException e) {
// Fall through to exit thread
}
}
public void shutdown() {
isStopping = true;
this.interrupt();
}
// Incoming transactions queue
private boolean incomingTransactionQueueContains(byte[] signature) {
synchronized (incomingTransactions) {
return incomingTransactions.keySet().stream().anyMatch(t -> Arrays.equals(t.getSignature(), signature));
}
}
private void removeIncomingTransaction(byte[] signature) {
incomingTransactions.keySet().removeIf(t -> Arrays.equals(t.getSignature(), signature));
}
/**
* Retrieve all pending unconfirmed transactions that have had their signatures validated.
* @return a list of TransactionData objects, with valid signatures.
*/
private List<TransactionData> getCachedSigValidTransactions() {
synchronized (this.incomingTransactions) {
return this.incomingTransactions.entrySet().stream()
.filter(t -> Boolean.TRUE.equals(t.getValue()))
.map(Map.Entry::getKey)
.collect(Collectors.toList());
}
}
/**
* Validate the signatures of any transactions pending import, then update their
* entries in the queue to mark them as valid/invalid.
*
* No database lock is required.
*/
private void validateTransactionsInQueue() {
if (this.incomingTransactions.isEmpty()) {
// Nothing to do?
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Take a snapshot of incomingTransactions, so we don't need to lock it while processing
Map<TransactionData, Boolean> incomingTransactionsCopy = Map.copyOf(this.incomingTransactions);
int unvalidatedCount = Collections.frequency(incomingTransactionsCopy.values(), Boolean.FALSE);
int validatedCount = 0;
if (unvalidatedCount > 0) {
LOGGER.debug("Validating signatures in incoming transactions queue (size {})...", unvalidatedCount);
}
// A list of all currently pending transactions that have valid signatures
List<Transaction> sigValidTransactions = new ArrayList<>();
// A list of signatures that became valid in this round
List<byte[]> newlyValidSignatures = new ArrayList<>();
boolean isLiteNode = Settings.getInstance().isLite();
// We need the latest block in order to check for expired transactions
BlockData latestBlock = Controller.getInstance().getChainTip();
// Signature validation round - does not require blockchain lock
for (Map.Entry<TransactionData, Boolean> transactionEntry : incomingTransactionsCopy.entrySet()) {
// Quick exit?
if (isStopping) {
return;
}
TransactionData transactionData = transactionEntry.getKey();
Transaction transaction = Transaction.fromData(repository, transactionData);
String signature58 = Base58.encode(transactionData.getSignature());
Long now = NTP.getTime();
if (now == null) {
return;
}
// Drop expired transactions before they are considered "sig valid"
if (latestBlock != null && transaction.getDeadline() <= latestBlock.getTimestamp()) {
LOGGER.debug("Removing expired {} transaction {} from import queue", transactionData.getType().name(), signature58);
removeIncomingTransaction(transactionData.getSignature());
invalidUnconfirmedTransactions.put(signature58, (now + EXPIRED_TRANSACTION_RECHECK_INTERVAL));
continue;
}
// Only validate signature if we haven't already done so
Boolean isSigValid = transactionEntry.getValue();
if (!Boolean.TRUE.equals(isSigValid)) {
if (isLiteNode) {
// Lite nodes can't easily validate transactions, so for now we will have to assume that everything is valid
sigValidTransactions.add(transaction);
newlyValidSignatures.add(transactionData.getSignature());
// Add mark signature as valid if transaction still exists in import queue
incomingTransactions.computeIfPresent(transactionData, (k, v) -> Boolean.TRUE);
continue;
}
if (!transaction.isSignatureValid()) {
LOGGER.debug("Ignoring {} transaction {} with invalid signature", transactionData.getType().name(), signature58);
removeIncomingTransaction(transactionData.getSignature());
// Also add to invalidIncomingTransactions map
now = NTP.getTime();
if (now != null) {
Long expiry = now + INVALID_TRANSACTION_RECHECK_INTERVAL;
LOGGER.trace("Adding invalid transaction {} to invalidUnconfirmedTransactions...", signature58);
// Add to invalidUnconfirmedTransactions so that we don't keep requesting it
invalidUnconfirmedTransactions.put(signature58, expiry);
}
// We're done with this transaction
continue;
}
// Count the number that were validated in this round, for logging purposes
validatedCount++;
// Add mark signature as valid if transaction still exists in import queue
incomingTransactions.computeIfPresent(transactionData, (k, v) -> Boolean.TRUE);
// Signature validated in this round
newlyValidSignatures.add(transactionData.getSignature());
} else {
LOGGER.trace(() -> String.format("Transaction %s known to have valid signature", Base58.encode(transactionData.getSignature())));
}
// Signature valid - add to shortlist
sigValidTransactions.add(transaction);
}
if (unvalidatedCount > 0) {
LOGGER.debug("Finished validating signatures in incoming transactions queue (valid this round: {}, total pending import: {})...", validatedCount, sigValidTransactions.size());
}
if (!newlyValidSignatures.isEmpty()) {
LOGGER.debug("Broadcasting {} newly valid signatures ahead of import", newlyValidSignatures.size());
Message newTransactionSignatureMessage = new TransactionSignaturesMessage(newlyValidSignatures);
Network.getInstance().broadcast(broadcastPeer -> newTransactionSignatureMessage);
}
} catch (DataException e) {
LOGGER.error("Repository issue while processing incoming transactions", e);
}
}
/**
* Import any transactions in the queue that have valid signatures.
*
* A database lock is required.
*/
private void importTransactionsInQueue() {
List<TransactionData> sigValidTransactions = this.getCachedSigValidTransactions();
if (sigValidTransactions.isEmpty()) {
// Don't bother locking if there are no new transactions to process
return;
}
if (Synchronizer.getInstance().isSyncRequested() || Synchronizer.getInstance().isSynchronizing()) {
// Prioritize syncing, and don't attempt to lock
return;
}
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
if (!blockchainLock.tryLock()) {
LOGGER.debug("Too busy to import incoming transactions queue");
return;
}
LOGGER.debug("Importing incoming transactions queue (size {})...", sigValidTransactions.size());
int processedCount = 0;
try (final Repository repository = RepositoryManager.getRepository()) {
// Import transactions with valid signatures
try {
for (int i = 0; i < sigValidTransactions.size(); ++i) {
if (isStopping) {
return;
}
if (Synchronizer.getInstance().isSyncRequestPending()) {
LOGGER.debug("Breaking out of transaction importing with {} remaining, because a sync request is pending", sigValidTransactions.size() - i);
return;
}
TransactionData transactionData = sigValidTransactions.get(i);
Transaction transaction = Transaction.fromData(repository, transactionData);
Transaction.ValidationResult validationResult = transaction.importAsUnconfirmed();
processedCount++;
switch (validationResult) {
case TRANSACTION_ALREADY_EXISTS: {
LOGGER.trace(() -> String.format("Ignoring existing transaction %s", Base58.encode(transactionData.getSignature())));
break;
}
case NO_BLOCKCHAIN_LOCK: {
// Is this even possible considering we acquired blockchain lock above?
LOGGER.trace(() -> String.format("Couldn't lock blockchain to import unconfirmed transaction %s", Base58.encode(transactionData.getSignature())));
break;
}
case OK: {
LOGGER.debug(() -> String.format("Imported %s transaction %s", transactionData.getType().name(), Base58.encode(transactionData.getSignature())));
break;
}
// All other invalid cases:
default: {
final String signature58 = Base58.encode(transactionData.getSignature());
LOGGER.debug(() -> String.format("Ignoring invalid (%s) %s transaction %s", validationResult.name(), transactionData.getType().name(), signature58));
Long now = NTP.getTime();
if (now != null && now - transactionData.getTimestamp() > INVALID_TRANSACTION_STALE_TIMEOUT) {
Long expiryLength = INVALID_TRANSACTION_RECHECK_INTERVAL;
if (validationResult == Transaction.ValidationResult.TIMESTAMP_TOO_OLD) {
// Use shorter recheck interval for expired transactions
expiryLength = EXPIRED_TRANSACTION_RECHECK_INTERVAL;
}
Long expiry = now + expiryLength;
LOGGER.trace("Adding stale invalid transaction {} to invalidUnconfirmedTransactions...", signature58);
// Invalid, unconfirmed transaction has become stale - add to invalidUnconfirmedTransactions so that we don't keep requesting it
invalidUnconfirmedTransactions.put(signature58, expiry);
}
}
}
// Transaction has been processed, even if only to reject it
removeIncomingTransaction(transactionData.getSignature());
}
} finally {
LOGGER.debug("Finished importing {} incoming transaction{}", processedCount, (processedCount == 1 ? "" : "s"));
blockchainLock.unlock();
}
} catch (DataException e) {
LOGGER.error("Repository issue while importing incoming transactions", e);
}
}
private void cleanupInvalidTransactionsList(Long now) {
if (now == null) {
return;
}
// Periodically remove invalid unconfirmed transactions from the list, so that they can be fetched again
invalidUnconfirmedTransactions.entrySet().removeIf(entry -> entry.getValue() == null || entry.getValue() < now);
}
// Network handlers
public void onNetworkTransactionMessage(Peer peer, Message message) {
TransactionMessage transactionMessage = (TransactionMessage) message;
TransactionData transactionData = transactionMessage.getTransactionData();
if (this.incomingTransactions.size() < MAX_INCOMING_TRANSACTIONS) {
synchronized (this.incomingTransactions) {
if (!incomingTransactionQueueContains(transactionData.getSignature())) {
this.incomingTransactions.put(transactionData, Boolean.FALSE);
}
}
}
}
public void onNetworkGetTransactionMessage(Peer peer, Message message) {
GetTransactionMessage getTransactionMessage = (GetTransactionMessage) message;
byte[] signature = getTransactionMessage.getSignature();
try (final Repository repository = RepositoryManager.getRepository()) {
// Firstly check the sig-valid transactions that are currently queued for import
TransactionData transactionData = this.getCachedSigValidTransactions().stream()
.filter(t -> Arrays.equals(signature, t.getSignature()))
.findFirst().orElse(null);
if (transactionData == null) {
// Not found in import queue, so try the database
transactionData = repository.getTransactionRepository().fromSignature(signature);
}
if (transactionData == null) {
// Still not found - so we don't have this transaction
LOGGER.debug(() -> String.format("Ignoring GET_TRANSACTION request from peer %s for unknown transaction %s", peer, Base58.encode(signature)));
// Send no response at all???
return;
}
Message transactionMessage = new TransactionMessage(transactionData);
transactionMessage.setId(message.getId());
if (!peer.sendMessage(transactionMessage))
peer.disconnect("failed to send transaction");
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending transaction %s to peer %s", Base58.encode(signature), peer), e);
} catch (TransformationException e) {
LOGGER.error(String.format("Serialization issue while sending transaction %s to peer %s", Base58.encode(signature), peer), e);
}
}
public void onNetworkGetUnconfirmedTransactionsMessage(Peer peer, Message message) {
try (final Repository repository = RepositoryManager.getRepository()) {
List<byte[]> signatures = Collections.emptyList();
// If we're NOT up-to-date then don't send out unconfirmed transactions
// as it's possible they are already included in a later block that we don't have.
if (Controller.getInstance().isUpToDate())
signatures = repository.getTransactionRepository().getUnconfirmedTransactionSignatures();
Message transactionSignaturesMessage = new TransactionSignaturesMessage(signatures);
if (!peer.sendMessage(transactionSignaturesMessage))
peer.disconnect("failed to send unconfirmed transaction signatures");
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while sending unconfirmed transaction signatures to peer %s", peer), e);
}
}
public void onNetworkTransactionSignaturesMessage(Peer peer, Message message) {
TransactionSignaturesMessage transactionSignaturesMessage = (TransactionSignaturesMessage) message;
List<byte[]> signatures = transactionSignaturesMessage.getSignatures();
try (final Repository repository = RepositoryManager.getRepository()) {
for (byte[] signature : signatures) {
String signature58 = Base58.encode(signature);
if (invalidUnconfirmedTransactions.containsKey(signature58)) {
// Previously invalid transaction - don't keep requesting it
// It will be periodically removed from invalidUnconfirmedTransactions to allow for rechecks
continue;
}
// Ignore if this transaction is in the queue
if (incomingTransactionQueueContains(signature)) {
LOGGER.trace(() -> String.format("Ignoring existing queued transaction %s from peer %s", Base58.encode(signature), peer));
continue;
}
// Do we have it already? (Before requesting transaction data itself)
if (repository.getTransactionRepository().exists(signature)) {
LOGGER.trace(() -> String.format("Ignoring existing transaction %s from peer %s", Base58.encode(signature), peer));
continue;
}
// Check isInterrupted() here and exit fast
if (Thread.currentThread().isInterrupted())
return;
// Fetch actual transaction data from peer
Message getTransactionMessage = new GetTransactionMessage(signature);
if (!peer.sendMessage(getTransactionMessage)) {
peer.disconnect("failed to request transaction");
return;
}
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while processing unconfirmed transactions from peer %s", peer), e);
}
}
}

View File

@@ -180,9 +180,6 @@ public class ArbitraryDataCleanupManager extends Thread {
arbitraryTransactionData.getName(), Base58.encode(signature)));
ArbitraryTransactionUtils.deleteCompleteFileAndChunks(arbitraryTransactionData);
// We should also remove peers for this transaction from the lookup table to save space
this.removePeersHostingTransactionData(repository, arbitraryTransactionData);
continue;
}
@@ -222,7 +219,11 @@ public class ArbitraryDataCleanupManager extends Thread {
try (final Repository repository = RepositoryManager.getRepository()) {
// Check if there are any hosted files that don't have matching transactions
this.checkForExpiredTransactions(repository);
// UPDATE: This has been disabled for now as it was deleting valid transactions
// and causing chunks to go missing on the network. If ever re-enabled, we MUST
// ensure that original copies of data aren't deleted, and that sufficient time
// is allowed (ideally several hours) before treating a transaction as missing.
// this.checkForExpiredTransactions(repository);
// Delete additional data at random if we're over our storage limit
// Use the DELETION_THRESHOLD so that we only start deleting once the hard limit is reached
@@ -433,16 +434,6 @@ public class ArbitraryDataCleanupManager extends Thread {
return false;
}
private void removePeersHostingTransactionData(Repository repository, ArbitraryTransactionData transactionData) {
byte[] signature = transactionData.getSignature();
try {
repository.getArbitraryRepository().deleteArbitraryPeersWithSignature(signature);
repository.saveChanges();
} catch (DataException e) {
LOGGER.debug("Unable to delete peers from lookup table for signature: {}", Base58.encode(signature));
}
}
private void cleanupTempDirectory(String folder, long now, long minAge) {
String baseDir = Settings.getInstance().getTempDataPath();
Path tempDir = Paths.get(baseDir, folder);

View File

@@ -5,6 +5,8 @@ import org.apache.logging.log4j.Logger;
import org.qortal.arbitrary.ArbitraryDataFile;
import org.qortal.arbitrary.ArbitraryDataFileChunk;
import org.qortal.controller.Controller;
import org.qortal.data.arbitrary.ArbitraryDirectConnectionInfo;
import org.qortal.data.arbitrary.ArbitraryFileListResponseInfo;
import org.qortal.data.arbitrary.ArbitraryRelayInfo;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.data.transaction.TransactionData;
@@ -23,12 +25,15 @@ import org.qortal.utils.Triple;
import java.util.*;
import static org.qortal.controller.arbitrary.ArbitraryDataFileManager.MAX_FILE_HASH_RESPONSES;
public class ArbitraryDataFileListManager {
private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFileListManager.class);
private static ArbitraryDataFileListManager instance;
private static String MIN_PEER_VERSION_FOR_FILE_LIST_STATS = "3.2.0";
/**
* Map of recent incoming requests for ARBITRARY transaction data file lists.
@@ -58,9 +63,9 @@ public class ArbitraryDataFileListManager {
/** Maximum number of seconds that a file list relay request is able to exist on the network */
private static long RELAY_REQUEST_MAX_DURATION = 5000L;
public static long RELAY_REQUEST_MAX_DURATION = 5000L;
/** Maximum number of hops that a file list relay request is allowed to make */
private static int RELAY_REQUEST_MAX_HOPS = 4;
public static int RELAY_REQUEST_MAX_HOPS = 4;
private ArbitraryDataFileListManager() {
@@ -263,7 +268,7 @@ public class ArbitraryDataFileListManager {
}
this.addToSignatureRequests(signature58, true, false);
List<Peer> handshakedPeers = Network.getInstance().getHandshakedPeers();
List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
List<byte[]> missingHashes = null;
// Find hashes that we are missing
@@ -278,8 +283,11 @@ public class ArbitraryDataFileListManager {
LOGGER.debug(String.format("Sending data file list request for signature %s with %d hashes to %d peers...", signature58, hashCount, handshakedPeers.size()));
// Send our address as requestingPeer, to allow for potential direct connections with seeds/peers
String requestingPeer = Network.getInstance().getOurExternalIpAddressAndPort();
// Build request
Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, missingHashes, now, 0);
Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, missingHashes, now, 0, requestingPeer);
// Save our request into requests map
Triple<String, Peer, Long> requestEntry = new Triple<>(signature58, null, NTP.getTime());
@@ -337,7 +345,7 @@ public class ArbitraryDataFileListManager {
// This could be optimized in the future
long timestamp = now - 60000L;
List<byte[]> hashes = null;
Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, hashes, timestamp, 0);
Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, hashes, timestamp, 0, null);
// Save our request into requests map
Triple<String, Peer, Long> requestEntry = new Triple<>(signature58, null, NTP.getTime());
@@ -403,6 +411,13 @@ public class ArbitraryDataFileListManager {
ArbitraryDataFileListMessage arbitraryDataFileListMessage = (ArbitraryDataFileListMessage) message;
LOGGER.debug("Received hash list from peer {} with {} hashes", peer, arbitraryDataFileListMessage.getHashes().size());
if (LOGGER.isDebugEnabled() && arbitraryDataFileListMessage.getRequestTime() != null) {
long totalRequestTime = NTP.getTime() - arbitraryDataFileListMessage.getRequestTime();
LOGGER.debug("totalRequestTime: {}, requestHops: {}, peerAddress: {}, isRelayPossible: {}",
totalRequestTime, arbitraryDataFileListMessage.getRequestHops(),
arbitraryDataFileListMessage.getPeerAddress(), arbitraryDataFileListMessage.isRelayPossible());
}
// Do we have a pending request for this data?
Triple<String, Peer, Long> request = arbitraryDataFileListRequests.get(message.getId());
if (request == null || request.getA() == null) {
@@ -423,7 +438,6 @@ public class ArbitraryDataFileListManager {
}
ArbitraryTransactionData arbitraryTransactionData = null;
ArbitraryDataFileManager arbitraryDataFileManager = ArbitraryDataFileManager.getInstance();
// Check transaction exists and hashes are correct
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -450,16 +464,28 @@ public class ArbitraryDataFileListManager {
// }
if (!isRelayRequest || !Settings.getInstance().isRelayModeEnabled()) {
// Keep track of the hashes this peer reports to have access to
Long now = NTP.getTime();
for (byte[] hash : hashes) {
String hash58 = Base58.encode(hash);
String sig58 = Base58.encode(signature);
ArbitraryDataFileManager.getInstance().arbitraryDataFileHashResponses.put(hash58, new Triple<>(peer, sig58, now));
if (ArbitraryDataFileManager.getInstance().arbitraryDataFileHashResponses.size() < MAX_FILE_HASH_RESPONSES) {
// Keep track of the hashes this peer reports to have access to
for (byte[] hash : hashes) {
String hash58 = Base58.encode(hash);
// Treat null request hops as 100, so that they are able to be sorted (and put to the end of the list)
int requestHops = arbitraryDataFileListMessage.getRequestHops() != null ? arbitraryDataFileListMessage.getRequestHops() : 100;
ArbitraryFileListResponseInfo responseInfo = new ArbitraryFileListResponseInfo(hash58, signature58,
peer, now, arbitraryDataFileListMessage.getRequestTime(), requestHops);
ArbitraryDataFileManager.getInstance().arbitraryDataFileHashResponses.add(responseInfo);
}
}
// Go and fetch the actual data, since this isn't a relay request
arbitraryDataFileManager.fetchArbitraryDataFiles(repository, peer, signature, arbitraryTransactionData, hashes);
// Keep track of the source peer, for direct connections
if (arbitraryDataFileListMessage.getPeerAddress() != null) {
ArbitraryDataFileManager.getInstance().addDirectConnectionInfoIfUnique(
new ArbitraryDirectConnectionInfo(signature, arbitraryDataFileListMessage.getPeerAddress(), hashes, now));
}
}
} catch (DataException e) {
@@ -472,17 +498,36 @@ public class ArbitraryDataFileListManager {
if (!isBlocked) {
Peer requestingPeer = request.getB();
if (requestingPeer != null) {
Long requestTime = arbitraryDataFileListMessage.getRequestTime();
Integer requestHops = arbitraryDataFileListMessage.getRequestHops();
// Add each hash to our local mapping so we know who to ask later
Long now = NTP.getTime();
for (byte[] hash : hashes) {
String hash58 = Base58.encode(hash);
ArbitraryRelayInfo relayMap = new ArbitraryRelayInfo(hash58, signature58, peer, now);
ArbitraryDataFileManager.getInstance().addToRelayMap(relayMap);
ArbitraryRelayInfo relayInfo = new ArbitraryRelayInfo(hash58, signature58, peer, now, requestTime, requestHops);
ArbitraryDataFileManager.getInstance().addToRelayMap(relayInfo);
}
// Bump requestHops if it exists
if (requestHops != null) {
requestHops++;
}
ArbitraryDataFileListMessage forwardArbitraryDataFileListMessage;
// Remove optional parameters if the requesting peer doesn't support it yet
// A message with less statistical data is better than no message at all
if (!requestingPeer.isAtLeastVersion(MIN_PEER_VERSION_FOR_FILE_LIST_STATS)) {
forwardArbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes);
} else {
forwardArbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes, requestTime, requestHops,
arbitraryDataFileListMessage.getPeerAddress(), arbitraryDataFileListMessage.isRelayPossible());
}
// Forward to requesting peer
LOGGER.debug("Forwarding file list with {} hashes to requesting peer: {}", hashes.size(), requestingPeer);
if (!requestingPeer.sendMessage(arbitraryDataFileListMessage)) {
if (!requestingPeer.sendMessage(forwardArbitraryDataFileListMessage)) {
requestingPeer.disconnect("failed to forward arbitrary data file list");
}
}
@@ -501,21 +546,30 @@ public class ArbitraryDataFileListManager {
GetArbitraryDataFileListMessage getArbitraryDataFileListMessage = (GetArbitraryDataFileListMessage) message;
byte[] signature = getArbitraryDataFileListMessage.getSignature();
String signature58 = Base58.encode(signature);
List<byte[]> requestedHashes = getArbitraryDataFileListMessage.getHashes();
Long now = NTP.getTime();
Triple<String, Peer, Long> newEntry = new Triple<>(signature58, peer, now);
// If we've seen this request recently, then ignore
if (arbitraryDataFileListRequests.putIfAbsent(message.getId(), newEntry) != null) {
LOGGER.debug("Ignoring hash list request from peer {} for signature {}", peer, signature58);
LOGGER.trace("Ignoring hash list request from peer {} for signature {}", peer, signature58);
return;
}
LOGGER.debug("Received hash list request from peer {} for signature {}", peer, signature58);
List<byte[]> requestedHashes = getArbitraryDataFileListMessage.getHashes();
int hashCount = requestedHashes != null ? requestedHashes.size() : 0;
String requestingPeer = getArbitraryDataFileListMessage.getRequestingPeer();
if (requestingPeer != null) {
LOGGER.debug("Received hash list request with {} hashes from peer {} (requesting peer {}) for signature {}", hashCount, peer, requestingPeer, signature58);
}
else {
LOGGER.debug("Received hash list request with {} hashes from peer {} for signature {}", hashCount, peer, signature58);
}
List<byte[]> hashes = new ArrayList<>();
ArbitraryTransactionData transactionData = null;
boolean allChunksExist = false;
boolean hasMetadata = false;
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -540,6 +594,7 @@ public class ArbitraryDataFileListManager {
// Add the metadata file
if (arbitraryDataFile.getMetadataHash() != null) {
requestedHashes.add(arbitraryDataFile.getMetadataHash());
hasMetadata = true;
}
// Add the chunk hashes
@@ -572,9 +627,18 @@ public class ArbitraryDataFileListManager {
LOGGER.error(String.format("Repository issue while fetching arbitrary file list for peer %s", peer), e);
}
// If the only file we have is the metadata then we shouldn't respond. Most nodes will already have that,
// or can use the separate metadata protocol to fetch it. This should greatly reduce network spam.
if (hasMetadata && hashes.size() == 1) {
hashes.clear();
}
// We should only respond if we have at least one hash
if (hashes.size() > 0) {
// Firstly we should keep track of the requesting peer, to allow for potential direct connections later
ArbitraryDataFileManager.getInstance().addRecentDataRequest(requestingPeer);
// We have all the chunks, so update requests map to reflect that we've sent it
// There is no need to keep track of the request, as we can serve all the chunks
if (allChunksExist) {
@@ -582,8 +646,20 @@ public class ArbitraryDataFileListManager {
arbitraryDataFileListRequests.put(message.getId(), newEntry);
}
ArbitraryDataFileListMessage arbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes);
String ourAddress = Network.getInstance().getOurExternalIpAddressAndPort();
ArbitraryDataFileListMessage arbitraryDataFileListMessage;
// Remove optional parameters if the requesting peer doesn't support it yet
// A message with less statistical data is better than no message at all
if (!peer.isAtLeastVersion(MIN_PEER_VERSION_FOR_FILE_LIST_STATS)) {
arbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes);
} else {
arbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature,
hashes, NTP.getTime(), 0, ourAddress, true);
}
arbitraryDataFileListMessage.setId(message.getId());
if (!peer.sendMessage(arbitraryDataFileListMessage)) {
LOGGER.debug("Couldn't send list of hashes");
peer.disconnect("failed to send list of hashes");
@@ -605,8 +681,7 @@ public class ArbitraryDataFileListManager {
// In relay mode - so ask our other peers if they have it
long requestTime = getArbitraryDataFileListMessage.getRequestTime();
int requestHops = getArbitraryDataFileListMessage.getRequestHops();
getArbitraryDataFileListMessage.setRequestHops(++requestHops);
int requestHops = getArbitraryDataFileListMessage.getRequestHops() + 1;
long totalRequestTime = now - requestTime;
if (totalRequestTime < RELAY_REQUEST_MAX_DURATION) {
@@ -614,11 +689,13 @@ public class ArbitraryDataFileListManager {
if (requestHops < RELAY_REQUEST_MAX_HOPS) {
// Relay request hasn't reached the maximum number of hops yet, so can be rebroadcast
Message relayGetArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, hashes, requestTime, requestHops, requestingPeer);
LOGGER.debug("Rebroadcasting hash list request from peer {} for signature {} to our other peers... totalRequestTime: {}, requestHops: {}", peer, Base58.encode(signature), totalRequestTime, requestHops);
Network.getInstance().broadcast(
broadcastPeer -> broadcastPeer == peer ||
Objects.equals(broadcastPeer.getPeerData().getAddress().getHost(), peer.getPeerData().getAddress().getHost())
? null : getArbitraryDataFileListMessage);
? null : relayGetArbitraryDataFileListMessage);
}
else {

View File

@@ -1,11 +1,13 @@
package org.qortal.controller.arbitrary;
import com.google.common.net.InetAddresses;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.arbitrary.ArbitraryDataFile;
import org.qortal.controller.Controller;
import org.qortal.data.arbitrary.ArbitraryDirectConnectionInfo;
import org.qortal.data.arbitrary.ArbitraryFileListResponseInfo;
import org.qortal.data.arbitrary.ArbitraryRelayInfo;
import org.qortal.data.network.ArbitraryPeerData;
import org.qortal.data.network.PeerData;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.network.Network;
@@ -18,7 +20,6 @@ import org.qortal.settings.Settings;
import org.qortal.utils.ArbitraryTransactionUtils;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import org.qortal.utils.Triple;
import java.security.SecureRandom;
import java.util.*;
@@ -37,7 +38,7 @@ public class ArbitraryDataFileManager extends Thread {
/**
* Map to keep track of our in progress (outgoing) arbitrary data file requests
*/
private Map<String, Long> arbitraryDataFileRequests = Collections.synchronizedMap(new HashMap<>());
public Map<String, Long> arbitraryDataFileRequests = Collections.synchronizedMap(new HashMap<>());
/**
* Map to keep track of hashes that we might need to relay
@@ -45,11 +46,24 @@ public class ArbitraryDataFileManager extends Thread {
public List<ArbitraryRelayInfo> arbitraryRelayMap = Collections.synchronizedList(new ArrayList<>());
/**
* Map to keep track of any arbitrary data file hash responses
* Key: string - the hash encoded in base58
* Value: Triple<respondingPeer, signature58, timeResponded>
* List to keep track of any arbitrary data file hash responses
*/
public Map<String, Triple<Peer, String, Long>> arbitraryDataFileHashResponses = Collections.synchronizedMap(new HashMap<>());
public final List<ArbitraryFileListResponseInfo> arbitraryDataFileHashResponses = Collections.synchronizedList(new ArrayList<>());
/**
* List to keep track of peers potentially available for direct connections, based on recent requests
*/
private List<ArbitraryDirectConnectionInfo> directConnectionInfo = Collections.synchronizedList(new ArrayList<>());
/**
* Map to keep track of peers requesting QDN data that we hold.
* Key = peer address string, value = time of last request.
* This allows for additional "burst" connections beyond existing limits.
*/
private Map<String, Long> recentDataRequests = Collections.synchronizedMap(new HashMap<>());
public static int MAX_FILE_HASH_RESPONSES = 1000;
private ArbitraryDataFileManager() {
@@ -98,7 +112,13 @@ public class ArbitraryDataFileManager extends Thread {
final long relayMinimumTimestamp = now - ArbitraryDataManager.getInstance().ARBITRARY_RELAY_TIMEOUT;
arbitraryRelayMap.removeIf(entry -> entry == null || entry.getTimestamp() == null || entry.getTimestamp() < relayMinimumTimestamp);
arbitraryDataFileHashResponses.entrySet().removeIf(entry -> entry.getValue().getC() == null || entry.getValue().getC() < relayMinimumTimestamp);
arbitraryDataFileHashResponses.removeIf(entry -> entry.getTimestamp() < relayMinimumTimestamp);
final long directConnectionInfoMinimumTimestamp = now - ArbitraryDataManager.getInstance().ARBITRARY_DIRECT_CONNECTION_INFO_TIMEOUT;
directConnectionInfo.removeIf(entry -> entry.getTimestamp() < directConnectionInfoMinimumTimestamp);
final long recentDataRequestMinimumTimestamp = now - ArbitraryDataManager.getInstance().ARBITRARY_RECENT_DATA_REQUESTS_TIMEOUT;
recentDataRequests.entrySet().removeIf(entry -> entry.getValue() < recentDataRequestMinimumTimestamp);
}
@@ -130,7 +150,7 @@ public class ArbitraryDataFileManager extends Thread {
Long startTime = NTP.getTime();
ArbitraryDataFileMessage receivedArbitraryDataFileMessage = fetchArbitraryDataFile(peer, null, signature, hash, null);
Long endTime = NTP.getTime();
if (receivedArbitraryDataFileMessage != null) {
if (receivedArbitraryDataFileMessage != null && receivedArbitraryDataFileMessage.getArbitraryDataFile() != null) {
LOGGER.debug("Received data file {} from peer {}. Time taken: {} ms", receivedArbitraryDataFileMessage.getArbitraryDataFile().getHash58(), peer, (endTime-startTime));
receivedAtLeastOneFile = true;
@@ -148,7 +168,7 @@ public class ArbitraryDataFileManager extends Thread {
}
}
else {
LOGGER.trace("Already requesting data file {} for signature {}", arbitraryDataFile, Base58.encode(signature));
LOGGER.trace("Already requesting data file {} for signature {} from peer {}", arbitraryDataFile, Base58.encode(signature), peer);
}
}
else {
@@ -158,16 +178,6 @@ public class ArbitraryDataFileManager extends Thread {
}
if (receivedAtLeastOneFile) {
// Update our lookup table to indicate that this peer holds data for this signature
String peerAddress = peer.getPeerData().getAddress().toString();
ArbitraryPeerData arbitraryPeerData = new ArbitraryPeerData(signature, peer);
repository.discardChanges();
if (arbitraryPeerData.isPeerAddressValid()) {
LOGGER.debug("Adding arbitrary peer: {} for signature {}", peerAddress, Base58.encode(signature));
repository.getArbitraryRepository().save(arbitraryPeerData);
repository.saveChanges();
}
// Invalidate the hosted transactions cache as we are now hosting something new
ArbitraryDataStorageManager.getInstance().invalidateHostedTransactionsCache();
@@ -177,16 +187,7 @@ public class ArbitraryDataFileManager extends Thread {
// We have all the chunks for this transaction, so we should invalidate the transaction's name's
// data cache so that it is rebuilt the next time we serve it
ArbitraryDataManager.getInstance().invalidateCache(arbitraryTransactionData);
// We may also need to broadcast to the network that we are now hosting files for this transaction,
// but only if these files are in accordance with our storage policy
if (ArbitraryDataStorageManager.getInstance().canStoreData(arbitraryTransactionData)) {
// Use a null peer address to indicate our own
Message newArbitrarySignatureMessage = new ArbitrarySignaturesMessage(null, 0, Arrays.asList(signature));
Network.getInstance().broadcast(broadcastPeer -> newArbitrarySignatureMessage);
}
}
}
return receivedAtLeastOneFile;
@@ -196,7 +197,7 @@ public class ArbitraryDataFileManager extends Thread {
ArbitraryDataFile existingFile = ArbitraryDataFile.fromHash(hash, signature);
boolean fileAlreadyExists = existingFile.exists();
String hash58 = Base58.encode(hash);
Message message = null;
ArbitraryDataFileMessage arbitraryDataFileMessage;
// Fetch the file if it doesn't exist locally
if (!fileAlreadyExists) {
@@ -204,10 +205,11 @@ public class ArbitraryDataFileManager extends Thread {
arbitraryDataFileRequests.put(hash58, NTP.getTime());
Message getArbitraryDataFileMessage = new GetArbitraryDataFileMessage(signature, hash);
Message response = null;
try {
message = peer.getResponseWithTimeout(getArbitraryDataFileMessage, (int) ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT);
response = peer.getResponseWithTimeout(getArbitraryDataFileMessage, (int) ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT);
} catch (InterruptedException e) {
// Will return below due to null message
// Will return below due to null response
}
arbitraryDataFileRequests.remove(hash58);
LOGGER.trace(String.format("Removed hash %.8s from arbitraryDataFileRequests", hash58));
@@ -215,22 +217,24 @@ public class ArbitraryDataFileManager extends Thread {
// We may need to remove the file list request, if we have all the files for this transaction
this.handleFileListRequests(signature);
if (message == null) {
LOGGER.debug("Received null message from peer {}", peer);
if (response == null) {
LOGGER.debug("Received null response from peer {}", peer);
return null;
}
if (message.getType() != Message.MessageType.ARBITRARY_DATA_FILE) {
LOGGER.debug("Received message with invalid type: {} from peer {}", message.getType(), peer);
if (response.getType() != MessageType.ARBITRARY_DATA_FILE) {
LOGGER.debug("Received response with invalid type: {} from peer {}", response.getType(), peer);
return null;
}
}
else {
ArbitraryDataFileMessage peersArbitraryDataFileMessage = (ArbitraryDataFileMessage) response;
arbitraryDataFileMessage = new ArbitraryDataFileMessage(signature, peersArbitraryDataFileMessage.getArbitraryDataFile());
} else {
LOGGER.debug(String.format("File hash %s already exists, so skipping the request", hash58));
arbitraryDataFileMessage = new ArbitraryDataFileMessage(signature, existingFile);
}
ArbitraryDataFileMessage arbitraryDataFileMessage = (ArbitraryDataFileMessage) message;
// We might want to forward the request to the peer that originally requested it
this.handleArbitraryDataFileForwarding(requestingPeer, message, originalMessage);
this.handleArbitraryDataFileForwarding(requestingPeer, arbitraryDataFileMessage, originalMessage);
boolean isRelayRequest = (requestingPeer != null);
if (isRelayRequest) {
@@ -240,16 +244,7 @@ public class ArbitraryDataFileManager extends Thread {
ArbitraryDataFile dataFile = arbitraryDataFileMessage.getArbitraryDataFile();
// Keep trying to delete the data until it is deleted, or we reach 10 attempts
for (int i=0; i<10; i++) {
if (dataFile.delete()) {
break;
}
try {
Thread.sleep(1000L);
} catch (InterruptedException e) {
// Fall through to exit method
}
}
dataFile.delete(10);
}
}
@@ -305,89 +300,135 @@ public class ArbitraryDataFileManager extends Thread {
// Fetch data directly from peers
private List<ArbitraryDirectConnectionInfo> getDirectConnectionInfoForSignature(byte[] signature) {
synchronized (directConnectionInfo) {
return directConnectionInfo.stream().filter(i -> Arrays.equals(i.getSignature(), signature)).collect(Collectors.toList());
}
}
/**
* Add an ArbitraryDirectConnectionInfo item, but only if one with this peer-signature combination
* doesn't already exist.
* @param connectionInfo - the direct connection info to add
*/
public void addDirectConnectionInfoIfUnique(ArbitraryDirectConnectionInfo connectionInfo) {
boolean peerAlreadyExists;
synchronized (directConnectionInfo) {
peerAlreadyExists = directConnectionInfo.stream()
.anyMatch(i -> Arrays.equals(i.getSignature(), connectionInfo.getSignature())
&& Objects.equals(i.getPeerAddress(), connectionInfo.getPeerAddress()));
}
if (!peerAlreadyExists) {
directConnectionInfo.add(connectionInfo);
}
}
private void removeDirectConnectionInfo(ArbitraryDirectConnectionInfo connectionInfo) {
this.directConnectionInfo.remove(connectionInfo);
}
public boolean fetchDataFilesFromPeersForSignature(byte[] signature) {
String signature58 = Base58.encode(signature);
ArbitraryDataFileListManager.getInstance().addToSignatureRequests(signature58, false, true);
// Firstly fetch peers that claim to be hosting files for this signature
try (final Repository repository = RepositoryManager.getRepository()) {
boolean success = false;
List<ArbitraryPeerData> peers = repository.getArbitraryRepository().getArbitraryPeerDataForSignature(signature);
if (peers == null || peers.isEmpty()) {
LOGGER.debug("No peers found for signature {}", signature58);
return false;
}
LOGGER.debug("Attempting a direct peer connection for signature {}...", signature58);
// Peers found, so pick a random one and request data from it
int index = new SecureRandom().nextInt(peers.size());
ArbitraryPeerData arbitraryPeerData = peers.get(index);
String peerAddressString = arbitraryPeerData.getPeerAddress();
boolean success = Network.getInstance().requestDataFromPeer(peerAddressString, signature);
// Parse the peer address to find the host and port
String host = null;
int port = -1;
String[] parts = peerAddressString.split(":");
if (parts.length > 1) {
host = parts[0];
port = Integer.parseInt(parts[1]);
}
// If unsuccessful, and using a non-standard port, try a second connection with the default listen port,
// since almost all nodes use that. This is a workaround to account for any ephemeral ports that may
// have made it into the dataset.
if (!success) {
if (host != null && port > 0) {
int defaultPort = Settings.getInstance().getDefaultListenPort();
if (port != defaultPort) {
String newPeerAddressString = String.format("%s:%d", host, defaultPort);
success = Network.getInstance().requestDataFromPeer(newPeerAddressString, signature);
}
try {
while (!success) {
if (isStopping) {
return false;
}
}
Thread.sleep(500L);
// If _still_ unsuccessful, try matching the peer's IP address with some known peers, and then connect
// to each of those in turn until one succeeds.
if (!success) {
if (host != null) {
final String finalHost = host;
List<PeerData> knownPeers = Network.getInstance().getAllKnownPeers().stream()
.filter(knownPeerData -> knownPeerData.getAddress().getHost().equals(finalHost))
.collect(Collectors.toList());
// Loop through each match and attempt a connection
for (PeerData matchingPeer : knownPeers) {
String matchingPeerAddress = matchingPeer.getAddress().toString();
success = Network.getInstance().requestDataFromPeer(matchingPeerAddress, signature);
if (success) {
// Successfully connected, so stop making connections
break;
// Firstly fetch peers that claim to be hosting files for this signature
List<ArbitraryDirectConnectionInfo> connectionInfoList = getDirectConnectionInfoForSignature(signature);
if (connectionInfoList == null || connectionInfoList.isEmpty()) {
LOGGER.debug("No remaining direct connection peers found for signature {}", signature58);
return false;
}
LOGGER.debug("Attempting a direct peer connection for signature {}...", signature58);
// Peers found, so pick one with the highest number of chunks
Comparator<ArbitraryDirectConnectionInfo> highestChunkCountFirstComparator =
Comparator.comparingInt(ArbitraryDirectConnectionInfo::getHashCount).reversed();
ArbitraryDirectConnectionInfo directConnectionInfo = connectionInfoList.stream()
.sorted(highestChunkCountFirstComparator).findFirst().orElse(null);
if (directConnectionInfo == null) {
return false;
}
// Remove from the list so that a different peer is tried next time
removeDirectConnectionInfo(directConnectionInfo);
String peerAddressString = directConnectionInfo.getPeerAddress();
// Parse the peer address to find the host and port
String host = null;
int port = -1;
String[] parts = peerAddressString.split(":");
if (parts.length > 1) {
host = parts[0];
port = Integer.parseInt(parts[1]);
} else {
// Assume no port included
host = peerAddressString;
// Use default listen port
port = Settings.getInstance().getDefaultListenPort();
}
String peerAddressStringWithPort = String.format("%s:%d", host, port);
success = Network.getInstance().requestDataFromPeer(peerAddressStringWithPort, signature);
int defaultPort = Settings.getInstance().getDefaultListenPort();
// If unsuccessful, and using a non-standard port, try a second connection with the default listen port,
// since almost all nodes use that. This is a workaround to account for any ephemeral ports that may
// have made it into the dataset.
if (!success) {
if (host != null && port > 0) {
if (port != defaultPort) {
String newPeerAddressString = String.format("%s:%d", host, defaultPort);
success = Network.getInstance().requestDataFromPeer(newPeerAddressString, signature);
}
}
}
}
// Keep track of the success or failure
arbitraryPeerData.markAsAttempted();
if (success) {
arbitraryPeerData.markAsRetrieved();
arbitraryPeerData.incrementSuccesses();
}
else {
arbitraryPeerData.incrementFailures();
}
repository.discardChanges();
repository.getArbitraryRepository().save(arbitraryPeerData);
repository.saveChanges();
// If _still_ unsuccessful, try matching the peer's IP address with some known peers, and then connect
// to each of those in turn until one succeeds.
if (!success) {
if (host != null) {
final String finalHost = host;
List<PeerData> knownPeers = Network.getInstance().getAllKnownPeers().stream()
.filter(knownPeerData -> knownPeerData.getAddress().getHost().equals(finalHost))
.collect(Collectors.toList());
// Loop through each match and attempt a connection
for (PeerData matchingPeer : knownPeers) {
String matchingPeerAddress = matchingPeer.getAddress().toString();
int matchingPeerPort = matchingPeer.getAddress().getPort();
// Make sure that it's not a port we've already tried
if (matchingPeerPort != port && matchingPeerPort != defaultPort) {
success = Network.getInstance().requestDataFromPeer(matchingPeerAddress, signature);
if (success) {
// Successfully connected, so stop making connections
break;
}
}
}
}
}
return success;
if (success) {
// We were able to connect with a peer, so track the request
ArbitraryDataFileListManager.getInstance().addToSignatureRequests(signature58, false, true);
}
} catch (DataException e) {
LOGGER.debug("Unable to fetch peer list from repository");
}
} catch (InterruptedException e) {
// Do nothing
}
return false;
return success;
}
@@ -401,6 +442,33 @@ public class ArbitraryDataFileManager extends Thread {
}
}
private ArbitraryRelayInfo getOptimalRelayInfoEntryForHash(String hash58) {
LOGGER.trace("Fetching relay info for hash: {}", hash58);
List<ArbitraryRelayInfo> relayInfoList = this.getRelayInfoListForHash(hash58);
if (relayInfoList != null && !relayInfoList.isEmpty()) {
// Remove any with null requestHops
relayInfoList.removeIf(r -> r.getRequestHops() == null);
// If list is now empty, then just return one at random
if (relayInfoList.isEmpty()) {
return this.getRandomRelayInfoEntryForHash(hash58);
}
// Sort by number of hops (lowest first)
relayInfoList.sort(Comparator.comparingInt(ArbitraryRelayInfo::getRequestHops));
// FUTURE: secondary sort by requestTime?
ArbitraryRelayInfo relayInfo = relayInfoList.get(0);
LOGGER.trace("Returning optimal relay info for hash: {} (requestHops {})", hash58, relayInfo.getRequestHops());
return relayInfo;
}
LOGGER.trace("No relay info exists for hash: {}", hash58);
return null;
}
private ArbitraryRelayInfo getRandomRelayInfoEntryForHash(String hash58) {
LOGGER.trace("Fetching random relay info for hash: {}", hash58);
List<ArbitraryRelayInfo> relayInfoList = this.getRelayInfoListForHash(hash58);
@@ -433,6 +501,45 @@ public class ArbitraryDataFileManager extends Thread {
}
// Peers requesting QDN data from us
/**
* Add an address string of a peer that is trying to request data from us.
* @param peerAddress
*/
public void addRecentDataRequest(String peerAddress) {
if (peerAddress == null) {
return;
}
Long now = NTP.getTime();
if (now == null) {
return;
}
// Make sure to remove the port, since it isn't guaranteed to match next time
String[] parts = peerAddress.split(":");
if (parts.length == 0) {
return;
}
String host = parts[0];
if (!InetAddresses.isInetAddress(host)) {
// Invalid host
return;
}
this.recentDataRequests.put(host, now);
}
public boolean isPeerRequestingData(String peerAddressWithoutPort) {
return this.recentDataRequests.containsKey(peerAddressWithoutPort);
}
public boolean hasPendingDataRequest() {
return !this.recentDataRequests.isEmpty();
}
// Network handlers
public void onNetworkGetArbitraryDataFileMessage(Peer peer, Message message) {
@@ -451,7 +558,7 @@ public class ArbitraryDataFileManager extends Thread {
try {
ArbitraryDataFile arbitraryDataFile = ArbitraryDataFile.fromHash(hash, signature);
ArbitraryRelayInfo relayInfo = this.getRandomRelayInfoEntryForHash(hash58);
ArbitraryRelayInfo relayInfo = this.getOptimalRelayInfoEntryForHash(hash58);
if (arbitraryDataFile.exists()) {
LOGGER.trace("Hash {} exists", hash58);

View File

@@ -3,6 +3,7 @@ package org.qortal.controller.arbitrary;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.controller.Controller;
import org.qortal.data.arbitrary.ArbitraryFileListResponseInfo;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.network.Peer;
import org.qortal.repository.DataException;
@@ -11,11 +12,9 @@ import org.qortal.repository.RepositoryManager;
import org.qortal.utils.ArbitraryTransactionUtils;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import org.qortal.utils.Triple;
import java.util.Arrays;
import java.util.Iterator;
import java.util.Map;
import java.util.*;
import java.util.stream.Collectors;
public class ArbitraryDataFileRequestThread implements Runnable {
@@ -51,39 +50,47 @@ public class ArbitraryDataFileRequestThread implements Runnable {
boolean shouldProcess = false;
synchronized (arbitraryDataFileManager.arbitraryDataFileHashResponses) {
Iterator iterator = arbitraryDataFileManager.arbitraryDataFileHashResponses.entrySet().iterator();
while (iterator.hasNext()) {
if (Controller.isStopping()) {
return;
}
if (!arbitraryDataFileManager.arbitraryDataFileHashResponses.isEmpty()) {
Map.Entry entry = (Map.Entry) iterator.next();
if (entry == null || entry.getKey() == null || entry.getValue() == null) {
// Sort by lowest number of node hops first
Comparator<ArbitraryFileListResponseInfo> lowestHopsFirstComparator =
Comparator.comparingInt(ArbitraryFileListResponseInfo::getRequestHops);
arbitraryDataFileManager.arbitraryDataFileHashResponses.sort(lowestHopsFirstComparator);
Iterator iterator = arbitraryDataFileManager.arbitraryDataFileHashResponses.iterator();
while (iterator.hasNext()) {
if (Controller.isStopping()) {
return;
}
ArbitraryFileListResponseInfo responseInfo = (ArbitraryFileListResponseInfo) iterator.next();
if (responseInfo == null) {
iterator.remove();
continue;
}
hash58 = responseInfo.getHash58();
peer = responseInfo.getPeer();
signature58 = responseInfo.getSignature58();
Long timestamp = responseInfo.getTimestamp();
if (now - timestamp >= ArbitraryDataManager.ARBITRARY_RELAY_TIMEOUT || signature58 == null || peer == null) {
// Ignore - to be deleted
iterator.remove();
continue;
}
// Skip if already requesting, but don't remove, as we might want to retry later
if (arbitraryDataFileManager.arbitraryDataFileRequests.containsKey(hash58)) {
// Already requesting - leave this attempt for later
continue;
}
// We want to process this file
shouldProcess = true;
iterator.remove();
continue;
break;
}
hash58 = (String) entry.getKey();
Triple<Peer, String, Long> value = (Triple<Peer, String, Long>) entry.getValue();
if (value == null) {
iterator.remove();
continue;
}
peer = value.getA();
signature58 = value.getB();
Long timestamp = value.getC();
if (now - timestamp >= ArbitraryDataManager.ARBITRARY_RELAY_TIMEOUT || signature58 == null || peer == null) {
// Ignore - to be deleted
iterator.remove();
continue;
}
// We want to process this file
shouldProcess = true;
iterator.remove();
break;
}
}

View File

@@ -1,23 +1,24 @@
package org.qortal.controller.arbitrary;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.*;
import java.util.stream.Collectors;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.api.resource.TransactionsResource.ConfirmationStatus;
import org.qortal.arbitrary.ArbitraryDataFile;
import org.qortal.arbitrary.ArbitraryDataResource;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.arbitrary.misc.Service;
import org.qortal.controller.Controller;
import org.qortal.data.network.ArbitraryPeerData;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.list.ResourceListManager;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.*;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@@ -43,9 +44,21 @@ public class ArbitraryDataManager extends Thread {
/** Maximum time to hold information about an in-progress relay */
public static final long ARBITRARY_RELAY_TIMEOUT = 60 * 1000L; // ms
/** Maximum time to hold direct peer connection information */
public static final long ARBITRARY_DIRECT_CONNECTION_INFO_TIMEOUT = 2 * 60 * 1000L; // ms
/** Maximum time to hold information about recent data requests that we can fulfil */
public static final long ARBITRARY_RECENT_DATA_REQUESTS_TIMEOUT = 2 * 60 * 1000L; // ms
/** Maximum number of hops that an arbitrary signatures request is allowed to make */
private static int ARBITRARY_SIGNATURES_REQUEST_MAX_HOPS = 3;
private long lastMetadataFetchTime = 0L;
private static long METADATA_FETCH_INTERVAL = 5 * 60 * 1000L;
private long lastDataFetchTime = 0L;
private static long DATA_FETCH_INTERVAL = 1 * 60 * 1000L;
private static ArbitraryDataManager instance;
private final Object peerDataLock = new Object();
@@ -79,6 +92,9 @@ public class ArbitraryDataManager extends Thread {
public void run() {
Thread.currentThread().setName("Arbitrary Data Manager");
// Create data directory in case it doesn't exist yet
this.createDataDirectory();
try {
// Wait for node to finish starting up and making connections
Thread.sleep(2 * 60 * 1000L);
@@ -92,7 +108,13 @@ public class ArbitraryDataManager extends Thread {
continue;
}
List<Peer> peers = Network.getInstance().getHandshakedPeers();
Long now = NTP.getTime();
if (now == null) {
continue;
}
// Needs a mutable copy of the unmodifiableList
List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
// Disregard peers that have "misbehaved" recently
peers.removeIf(Controller.hasMisbehaved);
@@ -102,6 +124,21 @@ public class ArbitraryDataManager extends Thread {
continue;
}
// Fetch metadata
if (NTP.getTime() - lastMetadataFetchTime >= METADATA_FETCH_INTERVAL) {
this.fetchAllMetadata();
lastMetadataFetchTime = NTP.getTime();
}
// Check if we need to fetch any data
if (NTP.getTime() - lastDataFetchTime < DATA_FETCH_INTERVAL) {
// Nothing to do yet
continue;
}
// In case the data directory has been deleted...
this.createDataDirectory();
// Fetch data according to storage policy
switch (Settings.getInstance().getStoragePolicy()) {
case FOLLOWED:
@@ -111,6 +148,7 @@ public class ArbitraryDataManager extends Thread {
case ALL:
this.processAll();
break;
case NONE:
case VIEWED:
@@ -119,6 +157,8 @@ public class ArbitraryDataManager extends Thread {
Thread.sleep(60000);
break;
}
lastDataFetchTime = NTP.getTime();
}
} catch (InterruptedException e) {
// Fall-through to exit thread...
@@ -130,7 +170,7 @@ public class ArbitraryDataManager extends Thread {
this.interrupt();
}
private void processNames() {
private void processNames() throws InterruptedException {
// Fetch latest list of followed names
List<String> followedNames = ResourceListManager.getInstance().getStringsInList("followedNames");
if (followedNames == null || followedNames.isEmpty()) {
@@ -143,11 +183,11 @@ public class ArbitraryDataManager extends Thread {
}
}
private void processAll() {
private void processAll() throws InterruptedException {
this.fetchAndProcessTransactions(null);
}
private void fetchAndProcessTransactions(String name) {
private void fetchAndProcessTransactions(String name) throws InterruptedException {
ArbitraryDataStorageManager storageManager = ArbitraryDataStorageManager.getInstance();
// Paginate queries when fetching arbitrary transactions
@@ -155,6 +195,7 @@ public class ArbitraryDataManager extends Thread {
int offset = 0;
while (!isStopping) {
Thread.sleep(1000L);
// Any arbitrary transactions we want to fetch data for?
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -169,6 +210,7 @@ public class ArbitraryDataManager extends Thread {
// Loop through signatures and remove ones we don't need to process
Iterator iterator = signatures.iterator();
while (iterator.hasNext()) {
Thread.sleep(25L); // Reduce CPU usage
byte[] signature = (byte[]) iterator.next();
ArbitraryTransaction arbitraryTransaction = fetchTransaction(repository, signature);
@@ -225,6 +267,85 @@ public class ArbitraryDataManager extends Thread {
}
}
private void fetchAllMetadata() throws InterruptedException {
ArbitraryDataStorageManager storageManager = ArbitraryDataStorageManager.getInstance();
// Paginate queries when fetching arbitrary transactions
final int limit = 100;
int offset = 0;
while (!isStopping) {
Thread.sleep(1000L);
// Any arbitrary transactions we want to fetch data for?
try (final Repository repository = RepositoryManager.getRepository()) {
List<byte[]> signatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(null, null, null, ARBITRARY_TX_TYPE, null, null, null, ConfirmationStatus.BOTH, limit, offset, true);
// LOGGER.trace("Found {} arbitrary transactions at offset: {}, limit: {}", signatures.size(), offset, limit);
if (signatures == null || signatures.isEmpty()) {
offset = 0;
break;
}
offset += limit;
// Loop through signatures and remove ones we don't need to process
Iterator iterator = signatures.iterator();
while (iterator.hasNext()) {
Thread.sleep(25L); // Reduce CPU usage
byte[] signature = (byte[]) iterator.next();
ArbitraryTransaction arbitraryTransaction = fetchTransaction(repository, signature);
if (arbitraryTransaction == null) {
// Best not to process this one
iterator.remove();
continue;
}
ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) arbitraryTransaction.getTransactionData();
// Skip transactions that are blocked
if (storageManager.isBlocked(arbitraryTransactionData)) {
iterator.remove();
continue;
}
// Remove transactions that we already have local data for
if (hasLocalMetadata(arbitraryTransaction)) {
iterator.remove();
continue;
}
}
if (signatures.isEmpty()) {
continue;
}
// Pick one at random
final int index = new Random().nextInt(signatures.size());
byte[] signature = signatures.get(index);
if (signature == null) {
continue;
}
// Check to see if we have had a more recent PUT
ArbitraryTransactionData arbitraryTransactionData = ArbitraryTransactionUtils.fetchTransactionData(repository, signature);
boolean hasMoreRecentPutTransaction = ArbitraryTransactionUtils.hasMoreRecentPutTransaction(repository, arbitraryTransactionData);
if (hasMoreRecentPutTransaction) {
// There is a more recent PUT transaction than the one we are currently processing.
// When a PUT is issued, it replaces any layers that would have been there before.
// Therefore any data relating to this older transaction is no longer needed and we
// shouldn't fetch it from the network.
continue;
}
// Ask our connected peers if they have metadata for this signature
fetchMetadata(arbitraryTransactionData);
} catch (DataException e) {
LOGGER.error("Repository issue when fetching arbitrary transaction data", e);
}
}
}
private ArbitraryTransaction fetchTransaction(final Repository repository, byte[] signature) {
try {
TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
@@ -244,16 +365,48 @@ public class ArbitraryDataManager extends Thread {
} catch (DataException e) {
LOGGER.error("Repository issue when checking arbitrary transaction's data is local", e);
return true;
return true; // Assume true for now, to avoid network spam on error
}
}
private boolean hasLocalMetadata(ArbitraryTransaction arbitraryTransaction) {
try {
ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) arbitraryTransaction.getTransactionData();
byte[] signature = arbitraryTransactionData.getSignature();
byte[] metadataHash = arbitraryTransactionData.getMetadataHash();
if (metadataHash == null) {
// This transaction doesn't have metadata associated with it, so return true to indicate that we have everything
return true;
}
ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
return metadataFile.exists();
} catch (DataException e) {
LOGGER.error("Repository issue when checking arbitrary transaction's metadata is local", e);
return true; // Assume true for now, to avoid network spam on error
}
}
// Entrypoint to request new data from peers
public boolean fetchData(ArbitraryTransactionData arbitraryTransactionData) {
return ArbitraryDataFileListManager.getInstance().fetchArbitraryDataFileList(arbitraryTransactionData);
}
// Entrypoint to request new metadata from peers
public ArbitraryDataTransactionMetadata fetchMetadata(ArbitraryTransactionData arbitraryTransactionData) {
ArbitraryDataResource resource = new ArbitraryDataResource(
arbitraryTransactionData.getName(),
ArbitraryDataFile.ResourceIdType.NAME,
arbitraryTransactionData.getService(),
arbitraryTransactionData.getIdentifier()
);
return ArbitraryMetadataManager.getInstance().fetchMetadata(resource, true);
}
// Useful methods used by other parts of the app
@@ -278,6 +431,9 @@ public class ArbitraryDataManager extends Thread {
// Cleanup file request caches
ArbitraryDataFileManager.getInstance().cleanupRequestCache(now);
// Clean up metadata request caches
ArbitraryMetadataManager.getInstance().cleanupRequestCache(now);
}
public boolean isResourceCached(ArbitraryDataResource resource) {
@@ -365,95 +521,19 @@ public class ArbitraryDataManager extends Thread {
}
}
// Broadcast list of hosted signatures
public void broadcastHostedSignatureList() {
try (final Repository repository = RepositoryManager.getRepository()) {
List<ArbitraryTransactionData> hostedTransactions = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository, null, null);
List<byte[]> hostedSignatures = hostedTransactions.stream().map(ArbitraryTransactionData::getSignature).collect(Collectors.toList());
if (!hostedSignatures.isEmpty()) {
// Broadcast the list, using null to represent our peer address
LOGGER.info("Broadcasting list of hosted signatures...");
Message arbitrarySignatureMessage = new ArbitrarySignaturesMessage(null, 0, hostedSignatures);
Network.getInstance().broadcast(broadcastPeer -> arbitrarySignatureMessage);
}
} catch (DataException e) {
LOGGER.error("Repository issue when fetching arbitrary transaction data for broadcast", e);
private boolean createDataDirectory() {
// Create the data directory if it doesn't exist
String dataPath = Settings.getInstance().getDataPath();
Path dataDirectory = Paths.get(dataPath);
try {
Files.createDirectories(dataDirectory);
} catch (IOException e) {
LOGGER.error("Unable to create data directory");
return false;
}
return true;
}
// Handle incoming arbitrary signatures messages
public void onNetworkArbitrarySignaturesMessage(Peer peer, Message message) {
// Don't process if QDN is disabled
if (!Settings.getInstance().isQdnEnabled()) {
return;
}
LOGGER.debug("Received arbitrary signature list from peer {}", peer);
ArbitrarySignaturesMessage arbitrarySignaturesMessage = (ArbitrarySignaturesMessage) message;
List<byte[]> signatures = arbitrarySignaturesMessage.getSignatures();
String peerAddress = peer.getPeerData().getAddress().toString();
if (arbitrarySignaturesMessage.getPeerAddress() != null && !arbitrarySignaturesMessage.getPeerAddress().isEmpty()) {
// This message is about a different peer than the one that sent it
peerAddress = arbitrarySignaturesMessage.getPeerAddress();
}
boolean containsNewEntry = false;
// Synchronize peer data lookups to make this process thread safe. Otherwise we could broadcast
// the same data multiple times, due to more than one thread processing the same message from different peers
synchronized (this.peerDataLock) {
try (final Repository repository = RepositoryManager.getRepository()) {
for (byte[] signature : signatures) {
// Check if a record already exists for this hash/host combination
// The port is not checked here - only the host/ip - in order to avoid duplicates
// from filling up the db due to dynamic/ephemeral ports
ArbitraryPeerData existingEntry = repository.getArbitraryRepository()
.getArbitraryPeerDataForSignatureAndHost(signature, peer.getPeerData().getAddress().getHost());
if (existingEntry == null) {
// We haven't got a record of this mapping yet, so add it
ArbitraryPeerData arbitraryPeerData = new ArbitraryPeerData(signature, peerAddress);
repository.discardChanges();
if (arbitraryPeerData.isPeerAddressValid()) {
LOGGER.debug("Adding arbitrary peer: {} for signature {}", peerAddress, Base58.encode(signature));
repository.getArbitraryRepository().save(arbitraryPeerData);
repository.saveChanges();
// Remember that this data is new, so that it can be rebroadcast later
containsNewEntry = true;
}
}
}
// If at least one signature in this batch was new to us, we should rebroadcast the message to the
// network in case some peers haven't received it yet
if (containsNewEntry) {
int requestHops = arbitrarySignaturesMessage.getRequestHops();
arbitrarySignaturesMessage.setRequestHops(++requestHops);
if (requestHops < ARBITRARY_SIGNATURES_REQUEST_MAX_HOPS) {
LOGGER.debug("Rebroadcasting arbitrary signature list for peer {}. requestHops: {}", peerAddress, requestHops);
Network.getInstance().broadcast(broadcastPeer -> broadcastPeer == peer ? null : arbitrarySignaturesMessage);
}
} else {
// Don't rebroadcast as otherwise we could get into a loop
}
// If anything needed saving, it would already have called saveChanges() above
repository.discardChanges();
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while processing arbitrary transaction signature list from peer %s", peer), e);
}
}
}
public int getPowDifficulty() {
return this.powDifficulty;
}

View File

@@ -16,6 +16,7 @@ import org.qortal.utils.FilesystemUtils;
import org.qortal.utils.NTP;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
@@ -46,6 +47,9 @@ public class ArbitraryDataStorageManager extends Thread {
private List<ArbitraryTransactionData> hostedTransactions;
private String searchQuery;
private List<ArbitraryTransactionData> searchResultsTransactions;
private static final long DIRECTORY_SIZE_CHECK_INTERVAL = 10 * 60 * 1000L; // 10 minutes
/** Treat storage as full at 90% usage, to reduce risk of going over the limit.
@@ -225,6 +229,16 @@ public class ArbitraryDataStorageManager extends Thread {
}
}
/**
* Check if data relating to a transaction is blocked by this node.
*
* @param arbitraryTransactionData - the transaction
* @return boolean - whether the resource is blocked or not
*/
public boolean isBlocked(ArbitraryTransactionData arbitraryTransactionData) {
return isNameBlocked(arbitraryTransactionData.getName());
}
private boolean isDataTypeAllowed(ArbitraryTransactionData arbitraryTransactionData) {
byte[] secret = arbitraryTransactionData.getSecret();
boolean hasSecret = (secret != null && secret.length == 32);
@@ -257,14 +271,8 @@ public class ArbitraryDataStorageManager extends Thread {
}
// Hosted data
public List<ArbitraryTransactionData> listAllHostedTransactions(Repository repository, Integer limit, Integer offset) {
// Load from cache if we can, to avoid disk reads
if (this.hostedTransactions != null) {
return ArbitraryTransactionUtils.limitOffsetTransactions(this.hostedTransactions, limit, offset);
}
public List<ArbitraryTransactionData> loadAllHostedTransactions(Repository repository) {
List<ArbitraryTransactionData> arbitraryTransactionDataList = new ArrayList<>();
// Find all hosted paths
@@ -285,7 +293,21 @@ public class ArbitraryDataStorageManager extends Thread {
if (transactionData == null || transactionData.getType() != Transaction.TransactionType.ARBITRARY) {
continue;
}
arbitraryTransactionDataList.add((ArbitraryTransactionData) transactionData);
ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) transactionData;
// Make sure to exclude metadata-only resources
if (arbitraryTransactionData.getMetadataHash() != null) {
if (contents.length == 1) {
String metadataHash58 = Base58.encode(arbitraryTransactionData.getMetadataHash());
if (Objects.equals(metadataHash58, contents[0])) {
// We only have the metadata file for this resource, not the actual data, so exclude it
continue;
}
}
}
// Found some data matching a transaction, so add it to the list
arbitraryTransactionDataList.add(arbitraryTransactionData);
} catch (DataException e) {
continue;
@@ -295,10 +317,69 @@ public class ArbitraryDataStorageManager extends Thread {
// Sort by newest first
arbitraryTransactionDataList.sort(Comparator.comparingLong(ArbitraryTransactionData::getTimestamp).reversed());
// Update cache
this.hostedTransactions = arbitraryTransactionDataList;
return arbitraryTransactionDataList;
}
// Hosted data
return ArbitraryTransactionUtils.limitOffsetTransactions(arbitraryTransactionDataList, limit, offset);
public List<ArbitraryTransactionData> listAllHostedTransactions(Repository repository, Integer limit, Integer offset) {
// Load from cache if we can, to avoid disk reads
if (this.hostedTransactions != null) {
return ArbitraryTransactionUtils.limitOffsetTransactions(this.hostedTransactions, limit, offset);
}
this.hostedTransactions = this.loadAllHostedTransactions(repository);
return ArbitraryTransactionUtils.limitOffsetTransactions(this.hostedTransactions, limit, offset);
}
/**
* searchHostedTransactions
* Allow to run a query against hosted data names and return matches if there are any
* @param repository
* @param query
* @param limit
* @param offset
* @return
*/
public List<ArbitraryTransactionData> searchHostedTransactions(Repository repository, String query, Integer limit, Integer offset) {
// Load from results cache if we can (results that exists for the same query), to avoid disk reads
if (this.searchResultsTransactions != null && this.searchQuery.equals(query.toLowerCase())) {
return ArbitraryTransactionUtils.limitOffsetTransactions(this.searchResultsTransactions, limit, offset);
}
// Using cache if we can, to avoid disk reads
if (this.hostedTransactions == null) {
this.hostedTransactions = this.loadAllHostedTransactions(repository);
}
this.searchQuery = query.toLowerCase(); //set the searchQuery so that it can be checked on the next call
List<ArbitraryTransactionData> searchResultsList = new ArrayList<>();
// Loop through cached hostedTransactions
for (ArbitraryTransactionData atd : this.hostedTransactions) {
try {
if (atd.getName() != null && atd.getName().toLowerCase().contains(this.searchQuery)) {
searchResultsList.add(atd);
}
else if (atd.getIdentifier() != null && atd.getIdentifier().toLowerCase().contains(this.searchQuery)) {
searchResultsList.add(atd);
}
} catch (Exception e) {
continue;
}
}
// Sort by newest first
searchResultsList.sort(Comparator.comparingLong(ArbitraryTransactionData::getTimestamp).reversed());
// Update cache
this.searchResultsTransactions = searchResultsList;
return ArbitraryTransactionUtils.limitOffsetTransactions(this.searchResultsTransactions, limit, offset);
}
/**
@@ -322,7 +403,7 @@ public class ArbitraryDataStorageManager extends Thread {
&& path.getFileName().toString().length() > 32)
.collect(Collectors.toList());
}
catch (IOException e) {
catch (IOException | UncheckedIOException e) {
LOGGER.info("Unable to walk through hosted data: {}", e.getMessage());
}

View File

@@ -0,0 +1,455 @@
package org.qortal.controller.arbitrary;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.arbitrary.ArbitraryDataFile;
import org.qortal.arbitrary.ArbitraryDataResource;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.controller.Controller;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.*;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import org.qortal.utils.Triple;
import java.io.IOException;
import java.util.*;
import static org.qortal.controller.arbitrary.ArbitraryDataFileListManager.RELAY_REQUEST_MAX_DURATION;
import static org.qortal.controller.arbitrary.ArbitraryDataFileListManager.RELAY_REQUEST_MAX_HOPS;
public class ArbitraryMetadataManager {
private static final Logger LOGGER = LogManager.getLogger(ArbitraryMetadataManager.class);
private static ArbitraryMetadataManager instance;
/**
* Map of recent incoming requests for ARBITRARY transaction metadata.
* <p>
* Key is original request's message ID<br>
* Value is Triple&lt;transaction signature in base58, first requesting peer, first request's timestamp&gt;
* <p>
* If peer is null then either:<br>
* <ul>
* <li>we are the original requesting peer</li>
* <li>we have already sent data payload to original requesting peer.</li>
* </ul>
* If signature is null then we have already received the file list and either:<br>
* <ul>
* <li>we are the original requesting peer and have processed it</li>
* <li>we have forwarded the metadata</li>
* </ul>
*/
public Map<Integer, Triple<String, Peer, Long>> arbitraryMetadataRequests = Collections.synchronizedMap(new HashMap<>());
/**
* Map to keep track of in progress arbitrary metadata requests
* Key: string - the signature encoded in base58
* Value: Triple<networkBroadcastCount, directPeerRequestCount, lastAttemptTimestamp>
*/
private Map<String, Triple<Integer, Integer, Long>> arbitraryMetadataSignatureRequests = Collections.synchronizedMap(new HashMap<>());
private ArbitraryMetadataManager() {
}
public static ArbitraryMetadataManager getInstance() {
if (instance == null)
instance = new ArbitraryMetadataManager();
return instance;
}
public void cleanupRequestCache(Long now) {
if (now == null) {
return;
}
final long requestMinimumTimestamp = now - ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT;
arbitraryMetadataRequests.entrySet().removeIf(entry -> entry.getValue().getC() == null || entry.getValue().getC() < requestMinimumTimestamp);
}
public ArbitraryDataTransactionMetadata fetchMetadata(ArbitraryDataResource arbitraryDataResource, boolean useRateLimiter) {
try (final Repository repository = RepositoryManager.getRepository()) {
// Find latest transaction
ArbitraryTransactionData latestTransaction = repository.getArbitraryRepository()
.getLatestTransaction(arbitraryDataResource.getResourceId(), arbitraryDataResource.getService(),
null, arbitraryDataResource.getIdentifier());
if (latestTransaction != null) {
byte[] signature = latestTransaction.getSignature();
byte[] metadataHash = latestTransaction.getMetadataHash();
if (metadataHash == null) {
// This resource doesn't have metadata
throw new IllegalArgumentException("This resource doesn't have metadata");
}
ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
if (!metadataFile.exists()) {
// Request from network
this.fetchArbitraryMetadata(latestTransaction, useRateLimiter);
}
// Now check again as it may have been downloaded above
if (metadataFile.exists()) {
// Use local copy
ArbitraryDataTransactionMetadata transactionMetadata = new ArbitraryDataTransactionMetadata(metadataFile.getFilePath());
transactionMetadata.read();
return transactionMetadata;
}
}
} catch (DataException | IOException e) {
LOGGER.error("Repository issue when fetching arbitrary transaction metadata", e);
}
return null;
}
// Request metadata from network
public byte[] fetchArbitraryMetadata(ArbitraryTransactionData arbitraryTransactionData, boolean useRateLimiter) {
byte[] metadataHash = arbitraryTransactionData.getMetadataHash();
if (metadataHash == null) {
return null;
}
byte[] signature = arbitraryTransactionData.getSignature();
String signature58 = Base58.encode(signature);
// Require an NTP sync
Long now = NTP.getTime();
if (now == null) {
return null;
}
// If we've already tried too many times in a short space of time, make sure to give up
if (useRateLimiter && !this.shouldMakeMetadataRequestForSignature(signature58)) {
LOGGER.trace("Skipping metadata request for signature {} due to rate limit", signature58);
return null;
}
this.addToSignatureRequests(signature58, true, false);
List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
LOGGER.debug(String.format("Sending metadata request for signature %s to %d peers...", signature58, handshakedPeers.size()));
// Build request
Message getArbitraryMetadataMessage = new GetArbitraryMetadataMessage(signature, now, 0);
// Save our request into requests map
Triple<String, Peer, Long> requestEntry = new Triple<>(signature58, null, NTP.getTime());
// Assign random ID to this message
int id;
do {
id = new Random().nextInt(Integer.MAX_VALUE - 1) + 1;
// Put queue into map (keyed by message ID) so we can poll for a response
// If putIfAbsent() doesn't return null, then this ID is already taken
} while (arbitraryMetadataRequests.put(id, requestEntry) != null);
getArbitraryMetadataMessage.setId(id);
// Broadcast request
Network.getInstance().broadcast(peer -> getArbitraryMetadataMessage);
// Poll to see if data has arrived
final long singleWait = 100;
long totalWait = 0;
while (totalWait < ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT) {
try {
Thread.sleep(singleWait);
} catch (InterruptedException e) {
break;
}
requestEntry = arbitraryMetadataRequests.get(id);
if (requestEntry == null)
return null;
if (requestEntry.getA() == null)
break;
totalWait += singleWait;
}
try {
ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
if (metadataFile.exists()) {
return metadataFile.getBytes();
}
} catch (DataException e) {
// Do nothing
}
return null;
}
// Track metadata lookups by signature
private boolean shouldMakeMetadataRequestForSignature(String signature58) {
Triple<Integer, Integer, Long> request = arbitraryMetadataSignatureRequests.get(signature58);
if (request == null) {
// Not attempted yet
return true;
}
// Extract the components
Integer networkBroadcastCount = request.getA();
// Integer directPeerRequestCount = request.getB();
Long lastAttemptTimestamp = request.getC();
if (lastAttemptTimestamp == null) {
// Not attempted yet
return true;
}
long timeSinceLastAttempt = NTP.getTime() - lastAttemptTimestamp;
// Allow a second attempt after 60 seconds
if (timeSinceLastAttempt > 60 * 1000L) {
// We haven't tried for at least 60 seconds
if (networkBroadcastCount < 2) {
// We've made less than 2 total attempts
return true;
}
}
// Then allow another attempt after 60 minutes
if (timeSinceLastAttempt > 60 * 60 * 1000L) {
// We haven't tried for at least 60 minutes
if (networkBroadcastCount < 3) {
// We've made less than 3 total attempts
return true;
}
}
return false;
}
public boolean isSignatureRateLimited(byte[] signature) {
String signature58 = Base58.encode(signature);
return !this.shouldMakeMetadataRequestForSignature(signature58);
}
public long lastRequestForSignature(byte[] signature) {
String signature58 = Base58.encode(signature);
Triple<Integer, Integer, Long> request = arbitraryMetadataSignatureRequests.get(signature58);
if (request == null) {
// Not attempted yet
return 0;
}
// Extract the components
Long lastAttemptTimestamp = request.getC();
if (lastAttemptTimestamp != null) {
return lastAttemptTimestamp;
}
return 0;
}
public void addToSignatureRequests(String signature58, boolean incrementNetworkRequests, boolean incrementPeerRequests) {
Triple<Integer, Integer, Long> request = arbitraryMetadataSignatureRequests.get(signature58);
Long now = NTP.getTime();
if (request == null) {
// No entry yet
Triple<Integer, Integer, Long> newRequest = new Triple<>(0, 0, now);
arbitraryMetadataSignatureRequests.put(signature58, newRequest);
}
else {
// There is an existing entry
if (incrementNetworkRequests) {
request.setA(request.getA() + 1);
}
if (incrementPeerRequests) {
request.setB(request.getB() + 1);
}
request.setC(now);
arbitraryMetadataSignatureRequests.put(signature58, request);
}
}
public void removeFromSignatureRequests(String signature58) {
arbitraryMetadataSignatureRequests.remove(signature58);
}
// Network handlers
public void onNetworkArbitraryMetadataMessage(Peer peer, Message message) {
// Don't process if QDN is disabled
if (!Settings.getInstance().isQdnEnabled()) {
return;
}
ArbitraryMetadataMessage arbitraryMetadataMessage = (ArbitraryMetadataMessage) message;
LOGGER.debug("Received metadata from peer {}", peer);
// Do we have a pending request for this data?
Triple<String, Peer, Long> request = arbitraryMetadataRequests.get(message.getId());
if (request == null || request.getA() == null) {
return;
}
boolean isRelayRequest = (request.getB() != null);
// Does this message's signature match what we're expecting?
byte[] signature = arbitraryMetadataMessage.getSignature();
String signature58 = Base58.encode(signature);
if (!request.getA().equals(signature58)) {
return;
}
// Update requests map to reflect that we've received all chunks
Triple<String, Peer, Long> newEntry = new Triple<>(null, null, request.getC());
arbitraryMetadataRequests.put(message.getId(), newEntry);
ArbitraryTransactionData arbitraryTransactionData = null;
// Forwarding
if (isRelayRequest && Settings.getInstance().isRelayModeEnabled()) {
// Get transaction info
try (final Repository repository = RepositoryManager.getRepository()) {
TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
if (!(transactionData instanceof ArbitraryTransactionData))
return;
arbitraryTransactionData = (ArbitraryTransactionData) transactionData;
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while finding arbitrary transaction metadata for peer %s", peer), e);
}
// Check if the name is blocked
boolean isBlocked = (arbitraryTransactionData == null || ArbitraryDataStorageManager.getInstance().isNameBlocked(arbitraryTransactionData.getName()));
if (!isBlocked) {
Peer requestingPeer = request.getB();
if (requestingPeer != null) {
ArbitraryMetadataMessage forwardArbitraryMetadataMessage = new ArbitraryMetadataMessage(signature, arbitraryMetadataMessage.getArbitraryMetadataFile());
// Forward to requesting peer
LOGGER.debug("Forwarding metadata to requesting peer: {}", requestingPeer);
if (!requestingPeer.sendMessage(forwardArbitraryMetadataMessage)) {
requestingPeer.disconnect("failed to forward arbitrary metadata");
}
}
}
}
}
public void onNetworkGetArbitraryMetadataMessage(Peer peer, Message message) {
// Don't respond if QDN is disabled
if (!Settings.getInstance().isQdnEnabled()) {
return;
}
Controller.getInstance().stats.getArbitraryMetadataMessageStats.requests.incrementAndGet();
GetArbitraryMetadataMessage getArbitraryMetadataMessage = (GetArbitraryMetadataMessage) message;
byte[] signature = getArbitraryMetadataMessage.getSignature();
String signature58 = Base58.encode(signature);
Long now = NTP.getTime();
Triple<String, Peer, Long> newEntry = new Triple<>(signature58, peer, now);
// If we've seen this request recently, then ignore
if (arbitraryMetadataRequests.putIfAbsent(message.getId(), newEntry) != null) {
LOGGER.debug("Ignoring metadata request from peer {} for signature {}", peer, signature58);
return;
}
LOGGER.debug("Received metadata request from peer {} for signature {}", peer, signature58);
ArbitraryTransactionData transactionData = null;
ArbitraryDataFile metadataFile = null;
try (final Repository repository = RepositoryManager.getRepository()) {
// Firstly we need to lookup this file on chain to get its metadata hash
transactionData = (ArbitraryTransactionData)repository.getTransactionRepository().fromSignature(signature);
if (transactionData instanceof ArbitraryTransactionData) {
// Check if we're even allowed to serve metadata for this transaction
if (ArbitraryDataStorageManager.getInstance().canStoreData(transactionData)) {
byte[] metadataHash = transactionData.getMetadataHash();
if (metadataHash != null) {
// Load metadata file
metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
}
}
}
} catch (DataException e) {
LOGGER.error(String.format("Repository issue while fetching arbitrary metadata for peer %s", peer), e);
}
// We should only respond if we have the metadata file
if (metadataFile != null && metadataFile.exists()) {
// We have the metadata file, so update requests map to reflect that we've sent it
newEntry = new Triple<>(null, null, now);
arbitraryMetadataRequests.put(message.getId(), newEntry);
ArbitraryMetadataMessage arbitraryMetadataMessage = new ArbitraryMetadataMessage(signature, metadataFile);
arbitraryMetadataMessage.setId(message.getId());
if (!peer.sendMessage(arbitraryMetadataMessage)) {
LOGGER.debug("Couldn't send metadata");
peer.disconnect("failed to send metadata");
return;
}
LOGGER.debug("Sent metadata");
// Nothing left to do, so return to prevent any unnecessary forwarding from occurring
LOGGER.debug("No need for any forwarding because metadata request is fully served");
return;
}
// We may need to forward this request on
boolean isBlocked = (transactionData == null || ArbitraryDataStorageManager.getInstance().isNameBlocked(transactionData.getName()));
if (Settings.getInstance().isRelayModeEnabled() && !isBlocked) {
// In relay mode - so ask our other peers if they have it
long requestTime = getArbitraryMetadataMessage.getRequestTime();
int requestHops = getArbitraryMetadataMessage.getRequestHops() + 1;
long totalRequestTime = now - requestTime;
if (totalRequestTime < RELAY_REQUEST_MAX_DURATION) {
// Relay request hasn't timed out yet, so can potentially be rebroadcast
if (requestHops < RELAY_REQUEST_MAX_HOPS) {
// Relay request hasn't reached the maximum number of hops yet, so can be rebroadcast
Message relayGetArbitraryMetadataMessage = new GetArbitraryMetadataMessage(signature, requestTime, requestHops);
LOGGER.debug("Rebroadcasting metadata request from peer {} for signature {} to our other peers... totalRequestTime: {}, requestHops: {}", peer, Base58.encode(signature), totalRequestTime, requestHops);
Network.getInstance().broadcast(
broadcastPeer -> broadcastPeer == peer ||
Objects.equals(broadcastPeer.getPeerData().getAddress().getHost(), peer.getPeerData().getAddress().getHost())
? null : relayGetArbitraryMetadataMessage);
}
else {
// This relay request has reached the maximum number of allowed hops
}
}
else {
// This relay request has timed out
}
}
}
}

View File

@@ -19,6 +19,11 @@ public class AtStatesPruner implements Runnable {
public void run() {
Thread.currentThread().setName("AT States pruner");
if (Settings.getInstance().isLite()) {
// Nothing to prune in lite mode
return;
}
boolean archiveMode = false;
if (!Settings.getInstance().isTopOnly()) {
// Top-only mode isn't enabled, but we might want to prune for the purposes of archiving

View File

@@ -19,6 +19,11 @@ public class AtStatesTrimmer implements Runnable {
public void run() {
Thread.currentThread().setName("AT States trimmer");
if (Settings.getInstance().isLite()) {
// Nothing to trim in lite mode
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
int trimStartHeight = repository.getATRepository().getAtTrimHeight();

View File

@@ -21,7 +21,7 @@ public class BlockArchiver implements Runnable {
public void run() {
Thread.currentThread().setName("Block archiver");
if (!Settings.getInstance().isArchiveEnabled()) {
if (!Settings.getInstance().isArchiveEnabled() || Settings.getInstance().isLite()) {
return;
}

View File

@@ -19,6 +19,11 @@ public class BlockPruner implements Runnable {
public void run() {
Thread.currentThread().setName("Block pruner");
if (Settings.getInstance().isLite()) {
// Nothing to prune in lite mode
return;
}
boolean archiveMode = false;
if (!Settings.getInstance().isTopOnly()) {
// Top-only mode isn't enabled, but we might want to prune for the purposes of archiving

View File

@@ -29,6 +29,15 @@ public class NamesDatabaseIntegrityCheck {
private List<TransactionData> nameTransactions = new ArrayList<>();
public int rebuildName(String name, Repository repository) {
return this.rebuildName(name, repository, null);
}
public int rebuildName(String name, Repository repository, List<String> referenceNames) {
// "referenceNames" tracks the linked names that have already been rebuilt, to prevent circular dependencies
if (referenceNames == null) {
referenceNames = new ArrayList<>();
}
int modificationCount = 0;
try {
List<TransactionData> transactions = this.fetchAllTransactionsInvolvingName(name, repository);
@@ -56,7 +65,14 @@ public class NamesDatabaseIntegrityCheck {
if (Objects.equals(updateNameTransactionData.getNewName(), name) &&
!Objects.equals(updateNameTransactionData.getName(), updateNameTransactionData.getNewName())) {
// This renames an existing name, so we need to process that instead
this.rebuildName(updateNameTransactionData.getName(), repository);
if (!referenceNames.contains(name)) {
referenceNames.add(name);
this.rebuildName(updateNameTransactionData.getName(), repository, referenceNames);
}
else {
// We've already processed this name so there's nothing more to do
}
}
else {
Name nameObj = new Name(repository, name);
@@ -91,7 +107,7 @@ public class NamesDatabaseIntegrityCheck {
BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) currentTransaction;
Name nameObj = new Name(repository, buyNameTransactionData.getName());
if (nameObj != null && nameObj.getNameData() != null) {
nameObj.buy(buyNameTransactionData);
nameObj.buy(buyNameTransactionData, false);
modificationCount++;
LOGGER.trace("Processed BUY_NAME transaction for name {}", name);
}
@@ -193,7 +209,12 @@ public class NamesDatabaseIntegrityCheck {
newName = registeredName;
}
NameData newNameData = repository.getNameRepository().fromName(newName);
if (!Objects.equals(creator.getAddress(), newNameData.getOwner())) {
if (newNameData == null) {
LOGGER.info("Error: registered name {} has no new name data. This is likely due to account {} " +
"being renamed another time, which is a scenario that is not yet checked automatically.",
updateNameTransactionData.getNewName(), creator.getAddress());
}
else if (!Objects.equals(creator.getAddress(), newNameData.getOwner())) {
LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
updateNameTransactionData.getNewName(), newNameData.getOwner(), creator.getAddress());
integrityCheckFailed = true;
@@ -313,6 +334,10 @@ public class NamesDatabaseIntegrityCheck {
transactions.add(transactionData);
}
}
// Sort by lowest timestamp first
transactions.sort(Comparator.comparingLong(TransactionData::getTimestamp));
return transactions;
}

View File

@@ -21,6 +21,11 @@ public class OnlineAccountsSignaturesTrimmer implements Runnable {
public void run() {
Thread.currentThread().setName("Online Accounts trimmer");
if (Settings.getInstance().isLite()) {
// Nothing to trim in lite mode
return;
}
try (final Repository repository = RepositoryManager.getRepository()) {
// Don't even start trimming until initial rush has ended
Thread.sleep(INITIAL_SLEEP_PERIOD);

View File

@@ -0,0 +1,885 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class BitcoinACCTv3TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(BitcoinACCTv3TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static BitcoinACCTv3TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private BitcoinACCTv3TradeBot() {
}
public static synchronized BitcoinACCTv3TradeBot getInstance() {
if (instance == null)
instance = new BitcoinACCTv3TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for BTC.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Bitcoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Bitcoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>BTC amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Bitcoin receiving address into public key hash (we only support P2PKH at this time)
Address bitcoinReceivingAddress;
try {
bitcoinReceivingAddress = Address.fromString(Bitcoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Bitcoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (bitcoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Bitcoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] bitcoinReceivingAccountInfo = bitcoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/BTC ACCT";
String description = "QORT/BTC cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT BTC";
byte[] creationBytes = BitcoinACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, BitcoinACCTv3.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.BITCOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, bitcoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching BTC to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Bitcoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Bitcoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Bitcoin main-net)
* or 'tprv' for (Bitcoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Bitcoin amount expected by 'Bob'.
* <p>
* If the Bitcoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Bitcoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, BitcoinACCTv3.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.BITCOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Bitcoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Bitcoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Bitcoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Bitcoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Bitcoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = BitcoinACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = BitcoinACCTv3.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Bitcoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Bitcoin bitcoin = Bitcoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Bitcoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
BitcoinACCTv3.OfferMessageData offerMessageData = BitcoinACCTv3.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerBitcoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = BitcoinACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = BitcoinACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the BTC funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Bitcoin bitcoin = Bitcoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = BitcoinACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = BitcoinACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the BTC funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the BTC funds from P2SH-A
* to Bob's 'foreign'/Bitcoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send BTC to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the BTC
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = BitcoinACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Bitcoin bitcoin = Bitcoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = bitcoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(bitcoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
bitcoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = bitcoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Bitcoin bitcoin = Bitcoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = bitcoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = bitcoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = bitcoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(bitcoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(bitcoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
bitcoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,885 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class DigibyteACCTv3TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(DigibyteACCTv3TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static DigibyteACCTv3TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private DigibyteACCTv3TradeBot() {
}
public static synchronized DigibyteACCTv3TradeBot getInstance() {
if (instance == null)
instance = new DigibyteACCTv3TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for DGB.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Digibyte) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Digibyte public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>DGB amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Digibyte receiving address into public key hash (we only support P2PKH at this time)
Address digibyteReceivingAddress;
try {
digibyteReceivingAddress = Address.fromString(Digibyte.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Digibyte receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (digibyteReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Digibyte receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] digibyteReceivingAccountInfo = digibyteReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/DGB ACCT";
String description = "QORT/DGB cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT DGB";
byte[] creationBytes = DigibyteACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DigibyteACCTv3.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.DIGIBYTE.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, digibyteReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching DGB to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Digibyte wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Digibyte BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Digibyte main-net)
* or 'tprv' for (Digibyte test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Digibyte amount expected by 'Bob'.
* <p>
* If the Digibyte transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Digibyte network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, DigibyteACCTv3.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.DIGIBYTE.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Digibyte.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Digibyte fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Digibyte.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Digibyte.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Digibyte.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = DigibyteACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = DigibyteACCTv3.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Digibyte balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Digibyte digibyte = Digibyte.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Digibyte pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
DigibyteACCTv3.OfferMessageData offerMessageData = DigibyteACCTv3.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerDigibytePKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = DigibyteACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = DigibyteACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the DGB funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Digibyte digibyte = Digibyte.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = DigibyteACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = DigibyteACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the DGB funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the DGB funds from P2SH-A
* to Bob's 'foreign'/Digibyte trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send DGB to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the DGB
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = DigibyteACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Digibyte digibyte = Digibyte.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = digibyte.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(digibyte.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
digibyte.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = digibyte.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Digibyte digibyte = Digibyte.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = digibyte.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = digibyte.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = digibyte.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(digibyte.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(digibyte.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
digibyte.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -0,0 +1,885 @@
package org.qortal.controller.tradebot;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bitcoinj.core.*;
import org.bitcoinj.script.Script.ScriptType;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.asset.Asset;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.DeployAtTransactionData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.group.Group;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.transaction.DeployAtTransaction;
import org.qortal.transaction.MessageTransaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.TransformationException;
import org.qortal.transform.transaction.DeployAtTransactionTransformer;
import org.qortal.utils.Base58;
import org.qortal.utils.NTP;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import static java.util.Arrays.stream;
import static java.util.stream.Collectors.toMap;
/**
* Performing cross-chain trading steps on behalf of user.
* <p>
* We deal with three different independent state-spaces here:
* <ul>
* <li>Qortal blockchain</li>
* <li>Foreign blockchain</li>
* <li>Trade-bot entries</li>
* </ul>
*/
public class RavencoinACCTv3TradeBot implements AcctTradeBot {
private static final Logger LOGGER = LogManager.getLogger(RavencoinACCTv3TradeBot.class);
public enum State implements TradeBot.StateNameAndValueSupplier {
BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
BOB_WAITING_FOR_MESSAGE(15, true, true),
BOB_WAITING_FOR_AT_REDEEM(25, true, true),
BOB_DONE(30, false, false),
BOB_REFUNDED(35, false, false),
ALICE_WAITING_FOR_AT_LOCK(85, true, true),
ALICE_DONE(95, false, false),
ALICE_REFUNDING_A(105, true, true),
ALICE_REFUNDED(110, false, false);
private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
public final int value;
public final boolean requiresAtData;
public final boolean requiresTradeData;
State(int value, boolean requiresAtData, boolean requiresTradeData) {
this.value = value;
this.requiresAtData = requiresAtData;
this.requiresTradeData = requiresTradeData;
}
public static State valueOf(int value) {
return map.get(value);
}
@Override
public String getState() {
return this.name();
}
@Override
public int getStateValue() {
return this.value;
}
}
/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
private static RavencoinACCTv3TradeBot instance;
private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
.map(State::name)
.collect(Collectors.toUnmodifiableList());
private RavencoinACCTv3TradeBot() {
}
public static synchronized RavencoinACCTv3TradeBot getInstance() {
if (instance == null)
instance = new RavencoinACCTv3TradeBot();
return instance;
}
@Override
public List<String> getEndStates() {
return this.endStates;
}
/**
* Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for RVN.
* <p>
* Generates:
* <ul>
* <li>new 'trade' private key</li>
* </ul>
* Derives:
* <ul>
* <li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
* <li>'foreign' (as in Ravencoin) public key, public key hash</li>
* </ul>
* A Qortal AT is then constructed including the following as constants in the 'data segment':
* <ul>
* <li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
* <li>'foreign'/Ravencoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
* <li>QORT amount on offer by Bob</li>
* <li>RVN amount expected in return by Bob (from Alice)</li>
* <li>trading timeout, in case things go wrong and everyone needs to refund</li>
* </ul>
* Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
* <p>
* Trade-bot will wait for Bob's AT to be deployed before taking next step.
* <p>
* @param repository
* @param tradeBotCreateRequest
* @return raw, unsigned DEPLOY_AT transaction
* @throws DataException
*/
public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
// Convert Ravencoin receiving address into public key hash (we only support P2PKH at this time)
Address ravencoinReceivingAddress;
try {
ravencoinReceivingAddress = Address.fromString(Ravencoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
} catch (AddressFormatException e) {
throw new DataException("Unsupported Ravencoin receiving address: " + tradeBotCreateRequest.receivingAddress);
}
if (ravencoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
throw new DataException("Unsupported Ravencoin receiving address: " + tradeBotCreateRequest.receivingAddress);
byte[] ravencoinReceivingAccountInfo = ravencoinReceivingAddress.getHash();
PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
// Deploy AT
long timestamp = NTP.getTime();
byte[] reference = creator.getLastReference();
long fee = 0L;
byte[] signature = null;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
String name = "QORT/RVN ACCT";
String description = "QORT/RVN cross-chain trade";
String aTType = "ACCT";
String tags = "ACCT QORT RVN";
byte[] creationBytes = RavencoinACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
long amount = tradeBotCreateRequest.fundingQortAmount;
DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
fee = deployAtTransaction.calcRecommendedFee();
deployAtTransactionData.setFee(fee);
DeployAtTransaction.ensureATAddress(deployAtTransactionData);
String atAddress = deployAtTransactionData.getAtAddress();
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, RavencoinACCTv3.NAME,
State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
null, null,
SupportedBlockchain.RAVENCOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
tradeBotCreateRequest.foreignAmount, null, null, null, ravencoinReceivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
// Attempt to backup the trade bot data
TradeBot.backupTradeBotData(repository, null);
// Return to user for signing and broadcast as we don't have their Qortal private key
try {
return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
} catch (TransformationException e) {
throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
}
}
/**
* Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching RVN to an existing offer.
* <p>
* Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
* and access to a Ravencoin wallet via <tt>xprv58</tt>.
* <p>
* The <tt>crossChainTradeData</tt> contains the current trade offer state
* as extracted from the AT's data segment.
* <p>
* Access to a funded wallet is via a Ravencoin BIP32 hierarchical deterministic key,
* passed via <tt>xprv58</tt>.
* <b>This key will be stored in your node's database</b>
* to allow trade-bot to create/fund the necessary P2SH transactions!
* However, due to the nature of BIP32 keys, it is possible to give the trade-bot
* only a subset of wallet access (see BIP32 for more details).
* <p>
* As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
* Electrum wallet by going to the console tab and entering:<br>
* <tt>wallet.keystore.xprv</tt><br>
* which should result in a base58 string starting with either 'xprv' (for Ravencoin main-net)
* or 'tprv' for (Ravencoin test-net).
* <p>
* It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
* <p>
* If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
* with the Ravencoin amount expected by 'Bob'.
* <p>
* If the Ravencoin transaction is successfully broadcast to the network then
* we also send a MESSAGE to Bob's trade-bot to let them know.
* <p>
* The trade-bot entry is saved to the repository and the cross-chain trading process commences.
* <p>
* @param repository
* @param crossChainTradeData chosen trade OFFER that Alice wants to match
* @param xprv58 funded wallet xprv in base58
* @return true if P2SH-A funding transaction successfully broadcast to Ravencoin network, false otherwise
* @throws DataException
*/
public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
byte[] secretA = TradeBot.generateSecret();
byte[] hashOfSecretA = Crypto.hash160(secretA);
byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
// We need to generate lockTime-A: add tradeTimeout to now
long now = NTP.getTime();
int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
TradeBotData tradeBotData = new TradeBotData(tradePrivateKey, RavencoinACCTv3.NAME,
State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
secretA, hashOfSecretA,
SupportedBlockchain.RAVENCOIN.name(),
tradeForeignPublicKey, tradeForeignPublicKeyHash,
crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
// Attempt to backup the trade bot data
// Include tradeBotData as an additional parameter, since it's not in the repository yet
TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
long p2shFee;
try {
p2shFee = Ravencoin.getInstance().getP2shFee(now);
} catch (ForeignBlockchainException e) {
LOGGER.debug("Couldn't estimate Ravencoin fees?");
return ResponseResult.NETWORK_ISSUE;
}
// Fee for redeem/refund is subtracted from P2SH-A balance.
// Do not include fee for funding transaction as this is covered by buildSpend()
long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
// P2SH-A to be funded
byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
String p2shAddress = Ravencoin.getInstance().deriveP2shAddress(redeemScriptBytes);
// Build transaction for funding P2SH-A
Transaction p2shFundingTransaction = Ravencoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
if (p2shFundingTransaction == null) {
LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
return ResponseResult.BALANCE_ISSUE;
}
try {
Ravencoin.getInstance().broadcastTransaction(p2shFundingTransaction);
} catch (ForeignBlockchainException e) {
// We couldn't fund P2SH-A at this time
LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
return ResponseResult.NETWORK_ISSUE;
}
// Attempt to send MESSAGE to Bob's Qortal trade address
byte[] messageData = RavencoinACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
return ResponseResult.NETWORK_ISSUE;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
return ResponseResult.OK;
}
@Override
public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null)
return true;
// If the AT doesn't exist then we might as well let the user tidy up
if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
return true;
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
case ALICE_DONE:
case BOB_DONE:
case ALICE_REFUNDED:
case BOB_REFUNDED:
case ALICE_REFUNDING_A:
return true;
default:
return false;
}
}
@Override
public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
State tradeBotState = State.valueOf(tradeBotData.getStateValue());
if (tradeBotState == null) {
LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
return;
}
ATData atData = null;
CrossChainTradeData tradeData = null;
if (tradeBotState.requiresAtData) {
// Attempt to fetch AT data
atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
if (atData == null) {
LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
return;
}
if (tradeBotState.requiresTradeData) {
tradeData = RavencoinACCTv3.getInstance().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
return;
}
}
}
switch (tradeBotState) {
case BOB_WAITING_FOR_AT_CONFIRM:
handleBobWaitingForAtConfirm(repository, tradeBotData);
break;
case BOB_WAITING_FOR_MESSAGE:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
break;
case ALICE_WAITING_FOR_AT_LOCK:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
break;
case BOB_WAITING_FOR_AT_REDEEM:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
break;
case ALICE_DONE:
case BOB_DONE:
break;
case ALICE_REFUNDING_A:
TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
break;
case ALICE_REFUNDED:
case BOB_REFUNDED:
break;
}
}
/**
* Trade-bot is waiting for Bob's AT to deploy.
* <p>
* If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
*/
private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
return;
// We've waited ages for AT to be confirmed into a block but something has gone awry.
// After this long we assume transaction loss so give up with trade-bot entry too.
tradeBotData.setState(State.BOB_REFUNDED.name());
tradeBotData.setStateValue(State.BOB_REFUNDED.value);
tradeBotData.setTimestamp(NTP.getTime());
// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
repository.saveChanges();
LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
TradeBot.notifyStateChange(tradeBotData);
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
}
/**
* Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
* <p>
* It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
* in which case trade-bot is done with this specific trade and finalizes on refunded state.
* <p>
* Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
* <p>
* Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
* <p>
* Assuming P2SH-A has at least expected Ravencoin balance,
* Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
* <p>
* On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
* <p>
* Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
* extract secret-A needed to redeem Alice's P2SH.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// If AT has finished then Bob likely cancelled his trade offer
if (atData.getIsFinished()) {
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
return;
}
Ravencoin ravencoin = Ravencoin.getInstance();
String address = tradeBotData.getTradeNativeAddress();
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
if (messageTransactionData.isText())
continue;
// We're expecting: HASH160(secret-A), Alice's Ravencoin pubkeyhash and lockTime-A
byte[] messageData = messageTransactionData.getData();
RavencoinACCTv3.OfferMessageData offerMessageData = RavencoinACCTv3.extractOfferMessageData(messageData);
if (offerMessageData == null)
continue;
byte[] aliceForeignPublicKeyHash = offerMessageData.partnerRavencoinPKH;
byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
int lockTimeA = (int) offerMessageData.lockTimeA;
long messageTimestamp = messageTransactionData.getTimestamp();
int refundTimeout = RavencoinACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
// Determine P2SH-A address and confirm funded
byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// There might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// We've already redeemed this?
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
continue;
case FUNDED:
// Fall-through out of switch...
break;
}
// Good to go - send MESSAGE to AT
String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
byte[] outgoingMessageData = RavencoinACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
outgoingMessageTransaction.computeNonce();
outgoingMessageTransaction.sign(sender);
// reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
return;
}
}
/**
* Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
* <p>
* It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
* this process has taken so long that we've reached P2SH-A's locktime, or that someone else
* has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
* <p>
* Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
* <p>
* If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
* <p>
* In revealing a valid secret-A, Bob can then redeem the RVN funds from P2SH-A.
* <p>
* @throws ForeignBlockchainException
*/
private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
return;
Ravencoin ravencoin = Ravencoin.getInstance();
int lockTimeA = tradeBotData.getLockTimeA();
// Refund P2SH-A if we've passed lockTime-A
if (NTP.getTime() >= lockTimeA * 1000L) {
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
case FUNDED:
break;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Already redeemed?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
return;
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> atData.getIsFinished()
? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
return;
}
// We're waiting for AT to be in TRADE mode
if (crossChainTradeData.mode != AcctMode.TRADING)
return;
// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
// Find our MESSAGE to AT from previous state
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
return;
}
long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
int refundTimeout = RavencoinACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
// Our calculated refundTimeout should match AT's refundTimeout
if (refundTimeout != crossChainTradeData.refundTimeout) {
LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
// We'll eventually refund
return;
}
// We're good to redeem AT
// Send 'redeem' MESSAGE to AT using both secret
byte[] secretA = tradeBotData.getSecret();
String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
byte[] messageData = RavencoinACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
String messageRecipient = tradeBotData.getAtAddress();
boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
if (!isMessageAlreadySent) {
PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
messageTransaction.computeNonce();
messageTransaction.sign(sender);
// Reset repository state to prevent deadlock
repository.discardChanges();
ValidationResult result = messageTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK) {
LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
return;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("Redeeming AT %s. Funds should arrive at %s",
tradeBotData.getAtAddress(), qortalReceivingAddress));
}
/**
* Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the RVN funds from P2SH-A.
* <p>
* It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
* trade-bot is done with this specific trade and finalizes in refunded state.
* <p>
* Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the RVN funds from P2SH-A
* to Bob's 'foreign'/Ravencoin trade legacy-format address, as derived from trade private key.
* <p>
* (This could potentially be 'improved' to send RVN to any address of Bob's choosing by changing the transaction output).
* <p>
* If trade-bot successfully broadcasts the transaction, then this specific trade is done.
* @throws ForeignBlockchainException
*/
private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// AT should be 'finished' once Alice has redeemed QORT funds
if (!atData.getIsFinished())
// Not finished yet
return;
// If AT is REFUNDED or CANCELLED then something has gone wrong
if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the RVN
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
return;
}
byte[] secretA = RavencoinACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
if (secretA == null) {
LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
return;
}
// Use secret-A to redeem P2SH-A
Ravencoin ravencoin = Ravencoin.getInstance();
byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
int lockTimeA = crossChainTradeData.lockTimeA;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Double-check that we have redeemed P2SH-A...
break;
case REFUND_IN_PROGRESS:
case REFUNDED:
// Wait for AT to auto-refund
return;
case FUNDED: {
Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = ravencoin.getUnspentOutputs(p2shAddressA);
Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(ravencoin.getNetworkParameters(), redeemAmount, redeemKey,
fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
ravencoin.broadcastTransaction(p2shRedeemTransaction);
break;
}
}
String receivingAddress = ravencoin.pkhToAddress(receivingAccountInfo);
TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
}
/**
* Trade-bot is attempting to refund P2SH-A.
* @throws ForeignBlockchainException
*/
private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
int lockTimeA = tradeBotData.getLockTimeA();
// We can't refund P2SH-A until lockTime-A has passed
if (NTP.getTime() <= lockTimeA * 1000L)
return;
Ravencoin ravencoin = Ravencoin.getInstance();
// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
int medianBlockTime = ravencoin.getMedianBlockTime();
if (medianBlockTime <= lockTimeA)
return;
byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
// Fee for redeem/refund is subtracted from P2SH-A balance.
long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
switch (htlcStatusA) {
case UNFUNDED:
case FUNDING_IN_PROGRESS:
// Still waiting for P2SH-A to be funded...
return;
case REDEEM_IN_PROGRESS:
case REDEEMED:
// Too late!
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("P2SH-A %s already spent!", p2shAddressA));
return;
case REFUND_IN_PROGRESS:
case REFUNDED:
break;
case FUNDED:{
Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
List<TransactionOutput> fundingOutputs = ravencoin.getUnspentOutputs(p2shAddressA);
// Determine receive address for refund
String receiveAddress = ravencoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
Address receiving = Address.fromString(ravencoin.getNetworkParameters(), receiveAddress);
Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(ravencoin.getNetworkParameters(), refundAmount, refundKey,
fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
ravencoin.broadcastTransaction(p2shRefundTransaction);
break;
}
}
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
}
/**
* Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
* <p>
* Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
*
* @throws DataException
* @throws ForeignBlockchainException
*/
private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
// This is OK
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
return false;
boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
if (isAtLockedToUs) {
// AT is trading with us - OK
return false;
} else {
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
return true;
}
if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
// We've redeemed already?
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
} else {
// Any other state is not good, so start defensive refund
TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
}
return true;
}
private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
return (lockTimeA - tradeTimeout * 60) * 1000L;
}
}

View File

@@ -2,41 +2,38 @@ package org.qortal.controller.tradebot;
import java.awt.TrayIcon.MessageType;
import java.security.SecureRandom;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.concurrent.locks.ReentrantLock;
import java.util.*;
import java.util.function.Supplier;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.util.Supplier;
import org.bitcoinj.core.ECKey;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.api.model.crosschain.TradeBotCreateRequest;
import org.qortal.controller.Controller;
import org.qortal.controller.Synchronizer;
import org.qortal.controller.tradebot.AcctTradeBot.ResponseResult;
import org.qortal.crosschain.*;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.crosschain.TradeBotData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.PresenceTransactionData;
import org.qortal.data.network.TradePresenceData;
import org.qortal.event.Event;
import org.qortal.event.EventBus;
import org.qortal.event.Listener;
import org.qortal.group.Group;
import org.qortal.gui.SysTray;
import org.qortal.network.Network;
import org.qortal.network.Peer;
import org.qortal.network.message.GetTradePresencesMessage;
import org.qortal.network.message.Message;
import org.qortal.network.message.TradePresencesMessage;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.repository.hsqldb.HSQLDBImportExport;
import org.qortal.settings.Settings;
import org.qortal.transaction.PresenceTransaction;
import org.qortal.transaction.PresenceTransaction.PresenceType;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transform.transaction.TransactionTransformer;
import org.qortal.utils.ByteArray;
import org.qortal.utils.NTP;
import com.google.common.primitives.Longs;
@@ -56,6 +53,15 @@ public class TradeBot implements Listener {
private static final Logger LOGGER = LogManager.getLogger(TradeBot.class);
private static final Random RANDOM = new SecureRandom();
/** Maximum lifetime of trade presence timestamp. 30 mins in ms. */
private static final long PRESENCE_LIFETIME = 30 * 60 * 1000L;
/** How soon before expiry of our own trade presence timestamp that we want to trigger renewal. 5 mins in ms. */
private static final long EARLY_RENEWAL_PERIOD = 5 * 60 * 1000L;
/** Trade presence timestamps are rounded up to this nearest interval. Bigger values improve grouping of entries in [GET_]TRADE_PRESENCES network messages. 15 mins in ms. */
private static final long EXPIRY_ROUNDING = 15 * 60 * 1000L;
/** How often we want to broadcast our list of all known trade presences to peers. 5 mins in ms. */
private static final long PRESENCE_BROADCAST_INTERVAL = 5 * 60 * 1000L;
public interface StateNameAndValueSupplier {
public String getState();
public int getStateValue();
@@ -73,20 +79,40 @@ public class TradeBot implements Listener {
}
}
public static class TradePresenceEvent implements Event {
private final TradePresenceData tradePresenceData;
public TradePresenceEvent(TradePresenceData tradePresenceData) {
this.tradePresenceData = tradePresenceData;
}
public TradePresenceData getTradePresenceData() {
return this.tradePresenceData;
}
}
private static final Map<Class<? extends ACCT>, Supplier<AcctTradeBot>> acctTradeBotSuppliers = new HashMap<>();
static {
acctTradeBotSuppliers.put(BitcoinACCTv1.class, BitcoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(BitcoinACCTv3.class, BitcoinACCTv3TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv1.class, LitecoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv2.class, LitecoinACCTv2TradeBot::getInstance);
acctTradeBotSuppliers.put(LitecoinACCTv3.class, LitecoinACCTv3TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv1.class, DogecoinACCTv1TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv2.class, DogecoinACCTv2TradeBot::getInstance);
acctTradeBotSuppliers.put(DogecoinACCTv3.class, DogecoinACCTv3TradeBot::getInstance);
acctTradeBotSuppliers.put(DigibyteACCTv3.class, DigibyteACCTv3TradeBot::getInstance);
acctTradeBotSuppliers.put(RavencoinACCTv3.class, RavencoinACCTv3TradeBot::getInstance);
}
private static TradeBot instance;
private final Map<String, Long> presenceTimestampsByAtAddress = Collections.synchronizedMap(new HashMap<>());
private final Map<ByteArray, Long> ourTradePresenceTimestampsByPubkey = Collections.synchronizedMap(new HashMap<>());
private final List<TradePresenceData> pendingTradePresences = Collections.synchronizedList(new ArrayList<>());
private final Map<ByteArray, TradePresenceData> allTradePresencesByPubkey = Collections.synchronizedMap(new HashMap<>());
private Map<ByteArray, TradePresenceData> safeAllTradePresencesByPubkey = Collections.emptyMap();
private long nextTradePresenceBroadcastTimestamp = 0L;
private TradeBot() {
EventBus.INSTANCE.addListener(event -> TradeBot.getInstance().listen(event));
@@ -213,10 +239,17 @@ public class TradeBot implements Listener {
@Override
public void listen(Event event) {
if (!(event instanceof Controller.NewBlockEvent))
if (!(event instanceof Synchronizer.NewChainTipEvent))
return;
// Don't process trade bots or broadcast presence timestamps if our chain is more than 30 minutes old
final Long minLatestBlockTimestamp = NTP.getTime() - (30 * 60 * 1000L);
if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp))
return;
synchronized (this) {
expireOldPresenceTimestamps();
List<TradeBotData> allTradeBotData;
try (final Repository repository = RepositoryManager.getRepository()) {
@@ -247,6 +280,8 @@ public class TradeBot implements Listener {
} catch (ForeignBlockchainException e) {
LOGGER.warn(() -> String.format("Foreign blockchain issue processing trade-bot entry for AT %s: %s", tradeBotData.getAtAddress(), e.getMessage()));
}
broadcastPresenceTimestamps();
}
}
@@ -324,6 +359,33 @@ public class TradeBot implements Listener {
}
// PRESENCE-related
public Collection<TradePresenceData> getAllTradePresences() {
return this.safeAllTradePresencesByPubkey.values();
}
/** Trade presence timestamps expire in the 'future' so any that reach 'now' have expired and are removed. */
private void expireOldPresenceTimestamps() {
long now = NTP.getTime();
int allRemovedCount = 0;
synchronized (this.allTradePresencesByPubkey) {
int preRemoveCount = this.allTradePresencesByPubkey.size();
this.allTradePresencesByPubkey.values().removeIf(tradePresenceData -> tradePresenceData.getTimestamp() <= now);
allRemovedCount = this.allTradePresencesByPubkey.size() - preRemoveCount;
}
int ourRemovedCount = 0;
synchronized (this.ourTradePresenceTimestampsByPubkey) {
int preRemoveCount = this.ourTradePresenceTimestampsByPubkey.size();
this.ourTradePresenceTimestampsByPubkey.values().removeIf(timestamp -> timestamp < now);
ourRemovedCount = this.ourTradePresenceTimestampsByPubkey.size() - preRemoveCount;
}
if (allRemovedCount > 0)
LOGGER.debug("Removed {} expired trade presences, of which {} ours", allRemovedCount, ourRemovedCount);
}
/*package*/ void updatePresence(Repository repository, TradeBotData tradeBotData, CrossChainTradeData tradeData)
throws DataException {
String atAddress = tradeBotData.getAtAddress();
@@ -332,44 +394,292 @@ public class TradeBot implements Listener {
String signerAddress = tradeNativeAccount.getAddress();
/*
* There's no point in Alice trying to build a PRESENCE transaction
* for an AT that isn't locked to her, as other peers won't be able
* to validate the PRESENCE transaction as signing public key won't
* be visible.
*/
if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress))
// Signer is neither Bob, nor Alice, or trade not yet locked to Alice
* There's no point in Alice trying to broadcast presence for an AT that isn't locked to her,
* as other peers won't be able to verify as signing public key isn't yet in the AT's data segment.
*/
if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress)) {
// Signer is neither Bob, nor trade locked to Alice
LOGGER.trace("Can't provide trade presence for our AT {} as it's not yet locked to Alice", atAddress);
return;
}
long now = NTP.getTime();
long threshold = now - PresenceType.TRADE_BOT.getLifetime();
long newExpiry = generateExpiry(now);
ByteArray pubkeyByteArray = ByteArray.wrap(tradeNativeAccount.getPublicKey());
long timestamp = presenceTimestampsByAtAddress.compute(atAddress, (k, v) -> (v == null || v < threshold) ? now : v);
// If map entry's timestamp is missing, or within early renewal period, use the new expiry - otherwise use existing timestamp.
synchronized (this.ourTradePresenceTimestampsByPubkey) {
Long currentTimestamp = this.ourTradePresenceTimestampsByPubkey.get(pubkeyByteArray);
// If timestamp hasn't been updated then nothing to do
if (timestamp != now)
if (currentTimestamp != null && currentTimestamp - now > EARLY_RENEWAL_PERIOD) {
// timestamp still good
LOGGER.trace("Current trade presence timestamp {} still good for our trade {}", currentTimestamp, atAddress);
return;
}
this.ourTradePresenceTimestampsByPubkey.put(pubkeyByteArray, newExpiry);
}
// Create signature
byte[] signature = tradeNativeAccount.sign(Longs.toByteArray(newExpiry));
// Add new trade presence to queue to be broadcast around network
TradePresenceData tradePresenceData = new TradePresenceData(newExpiry, tradeNativeAccount.getPublicKey(), signature, atAddress);
this.pendingTradePresences.add(tradePresenceData);
this.allTradePresencesByPubkey.put(pubkeyByteArray, tradePresenceData);
rebuildSafeAllTradePresences();
LOGGER.trace("New trade presence timestamp {} for our trade {}", newExpiry, atAddress);
EventBus.INSTANCE.notify(new TradePresenceEvent(tradePresenceData));
}
private void rebuildSafeAllTradePresences() {
synchronized (this.allTradePresencesByPubkey) {
// Collect into a *new* unmodifiable map.
this.safeAllTradePresencesByPubkey = Map.copyOf(this.allTradePresencesByPubkey);
}
}
private void broadcastPresenceTimestamps() {
// If we have new trade presences that are pending broadcast, send those as a priority
if (!this.pendingTradePresences.isEmpty()) {
// Create a copy for Network to safely use in another thread
List<TradePresenceData> safeTradePresences;
synchronized (this.pendingTradePresences) {
safeTradePresences = List.copyOf(this.pendingTradePresences);
this.pendingTradePresences.clear();
}
LOGGER.debug("Broadcasting {} new trade presences", safeTradePresences.size());
TradePresencesMessage tradePresencesMessage = new TradePresencesMessage(safeTradePresences);
Network.getInstance().broadcast(peer -> tradePresencesMessage);
return;
}
// As we have no new trade presences, check whether it's time to do a general broadcast
Long now = NTP.getTime();
if (now == null || now < nextTradePresenceBroadcastTimestamp)
return;
int txGroupId = Group.NO_GROUP;
byte[] reference = new byte[TransactionTransformer.SIGNATURE_LENGTH];
byte[] creatorPublicKey = tradeNativeAccount.getPublicKey();
long fee = 0L;
nextTradePresenceBroadcastTimestamp = now + PRESENCE_BROADCAST_INTERVAL;
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, txGroupId, reference, creatorPublicKey, fee, null);
List<TradePresenceData> safeTradePresences = List.copyOf(this.safeAllTradePresencesByPubkey.values());
int nonce = 0;
byte[] timestampSignature = tradeNativeAccount.sign(Longs.toByteArray(timestamp));
if (safeTradePresences.isEmpty())
return;
PresenceTransactionData transactionData = new PresenceTransactionData(baseTransactionData, nonce, PresenceType.TRADE_BOT, timestampSignature);
LOGGER.debug("Broadcasting all {} known trade presences. Next broadcast timestamp: {}",
safeTradePresences.size(), nextTradePresenceBroadcastTimestamp
);
PresenceTransaction presenceTransaction = new PresenceTransaction(repository, transactionData);
presenceTransaction.computeNonce();
GetTradePresencesMessage getTradePresencesMessage = new GetTradePresencesMessage(safeTradePresences);
Network.getInstance().broadcast(peer -> getTradePresencesMessage);
}
presenceTransaction.sign(tradeNativeAccount);
// Network message processing
ValidationResult result = presenceTransaction.importAsUnconfirmed();
if (result != ValidationResult.OK)
LOGGER.debug(() -> String.format("Unable to build trade-bot PRESENCE transaction for %s: %s", tradeBotData.getAtAddress(), result.name()));
public void onGetTradePresencesMessage(Peer peer, Message message) {
GetTradePresencesMessage getTradePresencesMessage = (GetTradePresencesMessage) message;
List<TradePresenceData> peersTradePresences = getTradePresencesMessage.getTradePresences();
// Create mutable copy from safe snapshot
Map<ByteArray, TradePresenceData> entriesUnknownToPeer = new HashMap<>(this.safeAllTradePresencesByPubkey);
int knownCount = entriesUnknownToPeer.size();
for (TradePresenceData peersTradePresence : peersTradePresences) {
ByteArray pubkeyByteArray = ByteArray.wrap(peersTradePresence.getPublicKey());
TradePresenceData ourEntry = entriesUnknownToPeer.get(pubkeyByteArray);
if (ourEntry != null && ourEntry.getTimestamp() == peersTradePresence.getTimestamp())
entriesUnknownToPeer.remove(pubkeyByteArray);
}
if (entriesUnknownToPeer.isEmpty())
return;
LOGGER.debug("Sending {} trade presences to peer {} after excluding their {} from known {}",
entriesUnknownToPeer.size(), peer, peersTradePresences.size(), knownCount
);
// Send complement to peer
List<TradePresenceData> safeTradePresences = List.copyOf(entriesUnknownToPeer.values());
Message responseMessage = new TradePresencesMessage(safeTradePresences);
if (!peer.sendMessage(responseMessage)) {
peer.disconnect("failed to send TRADE_PRESENCES response");
return;
}
}
public void onTradePresencesMessage(Peer peer, Message message) {
TradePresencesMessage tradePresencesMessage = (TradePresencesMessage) message;
List<TradePresenceData> peersTradePresences = tradePresencesMessage.getTradePresences();
long now = NTP.getTime();
// Timestamps before this are too far into the past
long pastThreshold = now;
// Timestamps after this are too far into the future
long futureThreshold = now + PRESENCE_LIFETIME;
Map<ByteArray, Supplier<ACCT>> acctSuppliersByCodeHash = SupportedBlockchain.getAcctMap();
int newCount = 0;
try (final Repository repository = RepositoryManager.getRepository()) {
for (TradePresenceData peersTradePresence : peersTradePresences) {
long timestamp = peersTradePresence.getTimestamp();
// Ignore if timestamp is out of bounds
if (timestamp < pastThreshold || timestamp > futureThreshold) {
if (timestamp < pastThreshold)
LOGGER.trace("Ignoring trade presence {} from peer {} as timestamp {} is too old vs {}",
peersTradePresence.getAtAddress(), peer, timestamp, pastThreshold
);
else
LOGGER.trace("Ignoring trade presence {} from peer {} as timestamp {} is too new vs {}",
peersTradePresence.getAtAddress(), peer, timestamp, pastThreshold
);
continue;
}
ByteArray pubkeyByteArray = ByteArray.wrap(peersTradePresence.getPublicKey());
// Ignore if we've previously verified this timestamp+publickey combo or sent timestamp is older
TradePresenceData existingTradeData = this.safeAllTradePresencesByPubkey.get(pubkeyByteArray);
if (existingTradeData != null && timestamp <= existingTradeData.getTimestamp()) {
if (timestamp == existingTradeData.getTimestamp())
LOGGER.trace("Ignoring trade presence {} from peer {} as we have verified timestamp {} before",
peersTradePresence.getAtAddress(), peer, timestamp
);
else
LOGGER.trace("Ignoring trade presence {} from peer {} as timestamp {} is older than latest {}",
peersTradePresence.getAtAddress(), peer, timestamp, existingTradeData.getTimestamp()
);
continue;
}
// Check timestamp signature
byte[] timestampSignature = peersTradePresence.getSignature();
byte[] timestampBytes = Longs.toByteArray(timestamp);
byte[] publicKey = peersTradePresence.getPublicKey();
if (!Crypto.verify(publicKey, timestampSignature, timestampBytes)) {
LOGGER.trace("Ignoring trade presence {} from peer {} as signature failed to verify",
peersTradePresence.getAtAddress(), peer
);
continue;
}
ATData atData = repository.getATRepository().fromATAddress(peersTradePresence.getAtAddress());
if (atData == null || atData.getIsFrozen() || atData.getIsFinished()) {
if (atData == null)
LOGGER.trace("Ignoring trade presence {} from peer {} as AT doesn't exist",
peersTradePresence.getAtAddress(), peer
);
else
LOGGER.trace("Ignoring trade presence {} from peer {} as AT is frozen or finished",
peersTradePresence.getAtAddress(), peer
);
continue;
}
ByteArray atCodeHash = ByteArray.wrap(atData.getCodeHash());
Supplier<ACCT> acctSupplier = acctSuppliersByCodeHash.get(atCodeHash);
if (acctSupplier == null) {
LOGGER.trace("Ignoring trade presence {} from peer {} as AT isn't a known ACCT?",
peersTradePresence.getAtAddress(), peer
);
continue;
}
CrossChainTradeData tradeData = acctSupplier.get().populateTradeData(repository, atData);
if (tradeData == null) {
LOGGER.trace("Ignoring trade presence {} from peer {} as trade data not found?",
peersTradePresence.getAtAddress(), peer
);
continue;
}
// Convert signer's public key to address form
String signerAddress = peersTradePresence.getTradeAddress();
// Signer's public key (in address form) must match Bob's / Alice's trade public key (in address form)
if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress)) {
LOGGER.trace("Ignoring trade presence {} from peer {} as signer isn't Alice or Bob?",
peersTradePresence.getAtAddress(), peer
);
continue;
}
// This is new to us
this.allTradePresencesByPubkey.put(pubkeyByteArray, peersTradePresence);
++newCount;
LOGGER.trace("Added trade presence {} from peer {} with timestamp {}",
peersTradePresence.getAtAddress(), peer, timestamp
);
EventBus.INSTANCE.notify(new TradePresenceEvent(peersTradePresence));
}
} catch (DataException e) {
LOGGER.error("Couldn't process TRADE_PRESENCES message due to repository issue", e);
}
if (newCount > 0) {
LOGGER.debug("New trade presences: {}", newCount);
rebuildSafeAllTradePresences();
}
}
public void bridgePresence(long timestamp, byte[] publicKey, byte[] signature, String atAddress) {
long expiry = generateExpiry(timestamp);
ByteArray pubkeyByteArray = ByteArray.wrap(publicKey);
TradePresenceData fakeTradePresenceData = new TradePresenceData(expiry, publicKey, signature, atAddress);
// Only bridge if trade presence expiry timestamp is newer
TradePresenceData computedTradePresenceData = this.allTradePresencesByPubkey.compute(pubkeyByteArray, (k, v) ->
v == null || v.getTimestamp() < expiry ? fakeTradePresenceData : v
);
if (computedTradePresenceData == fakeTradePresenceData) {
LOGGER.trace("Bridged PRESENCE transaction for trade {} with timestamp {}", atAddress, expiry);
rebuildSafeAllTradePresences();
EventBus.INSTANCE.notify(new TradePresenceEvent(fakeTradePresenceData));
}
}
/** Decorates a CrossChainTradeData object with Alice / Bob trade-bot presence timestamp, if available. */
public void decorateTradeDataWithPresence(CrossChainTradeData crossChainTradeData) {
// Match by AT address, then check for Bob vs Alice
this.safeAllTradePresencesByPubkey.values().stream()
.filter(tradePresenceData -> tradePresenceData.getAtAddress().equals(crossChainTradeData.qortalAtAddress))
.forEach(tradePresenceData -> {
String signerAddress = tradePresenceData.getTradeAddress();
// Signer's public key (in address form) must match Bob's / Alice's trade public key (in address form)
if (signerAddress.equals(crossChainTradeData.qortalCreatorTradeAddress))
crossChainTradeData.creatorPresenceExpiry = tradePresenceData.getTimestamp();
else if (signerAddress.equals(crossChainTradeData.qortalPartnerAddress))
crossChainTradeData.partnerPresenceExpiry = tradePresenceData.getTimestamp();
});
}
private long generateExpiry(long timestamp) {
return ((timestamp - 1) / EXPIRY_ROUNDING) * EXPIRY_ROUNDING + PRESENCE_LIFETIME;
}
}

View File

@@ -7,6 +7,7 @@ import java.util.Map;
import org.bitcoinj.core.Context;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.core.Transaction;
import org.bitcoinj.params.MainNetParams;
import org.bitcoinj.params.RegTestParams;
import org.bitcoinj.params.TestNet3Params;
@@ -18,10 +19,12 @@ public class Bitcoin extends Bitcoiny {
public static final String CURRENCY_CODE = "BTC";
private static final long MINIMUM_ORDER_AMOUNT = 100000; // 0.001 BTC minimum order, due to high fees
// Temporary values until a dynamic fee system is written.
private static final long OLD_FEE_AMOUNT = 4_000L; // Not 5000 so that existing P2SH-B can output 1000, avoiding dust issue, leaving 4000 for fees.
private static final long NEW_FEE_TIMESTAMP = 1598280000000L; // milliseconds since epoch
private static final long NEW_FEE_AMOUNT = 10_000L;
private static final long NEW_FEE_AMOUNT = 6_000L;
private static final long NON_MAINNET_FEE = 1000L; // enough for TESTNET3 and should be OK for REGTEST
@@ -42,30 +45,40 @@ public class Bitcoin extends Bitcoiny {
public Collection<ElectrumX.Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("hodlers.beer", Server.ConnectionType.SSL, 50002),
new Server("btc.lastingcoin.net", Server.ConnectionType.SSL, 50002),
new Server("electrum.bitaroo.net", Server.ConnectionType.SSL, 50002),
new Server("bitcoin.grey.pw", Server.ConnectionType.SSL, 50002),
// Status verified at https://1209k.com/bitcoin-eye/ele.php?chain=btc
//CLOSED new Server("bitcoin.grey.pw", Server.ConnectionType.SSL, 50002),
//CLOSED new Server("btc.litepay.ch", Server.ConnectionType.SSL, 50002),
//CLOSED new Server("electrum.pabu.io", Server.ConnectionType.SSL, 50002),
//CLOSED new Server("electrumx.hodlwallet.com", Server.ConnectionType.SSL, 50002),
//CLOSED new Server("gd42.org", Server.ConnectionType.SSL, 50002),
//CLOSED new Server("korea.electrum-server.com", Server.ConnectionType.SSL, 50002),
//CLOSED new Server("prospero.bitsrc.net", Server.ConnectionType.SSL, 50002),
//1.15.0 new Server("alviss.coinjoined.com", Server.ConnectionType.SSL, 50002),
//1.15.0 new Server("electrum.acinq.co", Server.ConnectionType.SSL, 50002),
//1.14.0 new Server("electrum.coinext.com.br", Server.ConnectionType.SSL, 50002),
new Server("104.248.139.211", Server.ConnectionType.SSL, 50002),
new Server("142.93.6.38", Server.ConnectionType.SSL, 50002),
new Server("157.245.172.236", Server.ConnectionType.SSL, 50002),
new Server("167.172.226.175", Server.ConnectionType.SSL, 50002),
new Server("167.172.42.31", Server.ConnectionType.SSL, 50002),
new Server("178.62.80.20", Server.ConnectionType.SSL, 50002),
new Server("185.64.116.15", Server.ConnectionType.SSL, 50002),
new Server("alviss.coinjoined.com", Server.ConnectionType.SSL, 50002),
new Server("btc.litepay.ch", Server.ConnectionType.SSL, 50002),
new Server("xtrum.com", Server.ConnectionType.SSL, 50002),
new Server("electrum.acinq.co", Server.ConnectionType.SSL, 50002),
new Server("caleb.vegas", Server.ConnectionType.SSL, 50002),
new Server("electrum.coinext.com.br", Server.ConnectionType.TCP, 50001),
new Server("korea.electrum-server.com", Server.ConnectionType.TCP, 50001),
new Server("eai.coincited.net", Server.ConnectionType.TCP, 50001),
new Server("electrum.coinext.com.br", Server.ConnectionType.SSL, 50002),
new Server("node1.btccuracao.com", Server.ConnectionType.SSL, 50002),
new Server("korea.electrum-server.com", Server.ConnectionType.SSL, 50002),
new Server("btce.iiiiiii.biz", Server.ConnectionType.SSL, 50002),
new Server("68.183.188.105", Server.ConnectionType.SSL, 50002),
new Server("bitcoin.lukechilds.co", Server.ConnectionType.SSL, 50002),
new Server("guichet.centure.cc", Server.ConnectionType.SSL, 50002),
new Server("electrumx.hodlwallet.com", Server.ConnectionType.SSL, 50002),
new Server("blkhub.net", Server.ConnectionType.SSL, 50002),
new Server("btc.lastingcoin.net", Server.ConnectionType.SSL, 50002),
new Server("btce.iiiiiii.biz", Server.ConnectionType.SSL, 50002),
new Server("caleb.vegas", Server.ConnectionType.SSL, 50002),
new Server("eai.coincited.net", Server.ConnectionType.SSL, 50002),
new Server("prospero.bitsrc.net", Server.ConnectionType.SSL, 50002),
new Server("gd42.org", Server.ConnectionType.SSL, 50002),
new Server("electrum.pabu.io", Server.ConnectionType.SSL, 50002));
new Server("electrum.bitaroo.net", Server.ConnectionType.SSL, 50002),
new Server("electrumx.dev", Server.ConnectionType.SSL, 50002),
new Server("elx.bitske.com", Server.ConnectionType.SSL, 50002),
new Server("fortress.qtornado.com", Server.ConnectionType.SSL, 50002),
new Server("guichet.centure.cc", Server.ConnectionType.SSL, 50002),
new Server("kareoke.qoppa.org", Server.ConnectionType.SSL, 50002),
new Server("hodlers.beer", Server.ConnectionType.SSL, 50002),
new Server("node1.btccuracao.com", Server.ConnectionType.SSL, 50002),
new Server("xtrum.com", Server.ConnectionType.SSL, 50002));
}
@Override
@@ -172,6 +185,11 @@ public class Bitcoin extends Bitcoiny {
instance = null;
}
@Override
public long getMinimumOrderAmount() {
return MINIMUM_ORDER_AMOUNT;
}
// Actual useful methods for use by other classes
/**
@@ -185,4 +203,17 @@ public class Bitcoin extends Bitcoiny {
return this.bitcoinNet.getP2shFee(timestamp);
}
/**
* Returns bitcoinj transaction sending <tt>amount</tt> to <tt>recipient</tt> using 20 sat/byte fee.
*
* @param xprv58 BIP32 private key
* @param recipient P2PKH address
* @param amount unscaled amount
* @return transaction, or null if insufficient funds
*/
@Override
public Transaction buildSpend(String xprv58, String recipient, long amount) {
return buildSpend(xprv58, recipient, amount, 20L);
}
}

View File

@@ -0,0 +1,858 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Bitcoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Bitcoin & Qortal 'trade' keys</li>
* <li>Alice funds Bitcoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Bitcoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Bitcoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Bitcoin trade key and secret-A</li>
* <li>P2SH-A BTC funds end up at Bitcoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class BitcoinACCTv3 implements ACCT {
private static final Logger LOGGER = LogManager.getLogger(BitcoinACCTv3.class);
public static final String NAME = BitcoinACCTv3.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("676fb9350708dafa054eb0262d655039e393c1eb4918ec582f8d45524c9b4860").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerBitcoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerBitcoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Bitcoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static BitcoinACCTv3 instance;
private BitcoinACCTv3() {
}
public static synchronized BitcoinACCTv3 getInstance() {
if (instance == null)
instance = new BitcoinACCTv3();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Bitcoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param bitcoinPublicKeyHash 20-byte HASH160 of creator's trade Bitcoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param bitcoinAmount how much BTC the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] bitcoinPublicKeyHash, long qortAmount, long bitcoinAmount, int tradeTimeout) {
if (bitcoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Bitcoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrBitcoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrBitcoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerBitcoinPKHOffset = addrCounter++;
final int addrPartnerBitcoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerBitcoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Bitcoin public key hash
assert dataByteBuffer.position() == addrBitcoinPublicKeyHash * MachineState.VALUE_SIZE : "addrBitcoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(bitcoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Bitcoin amount
assert dataByteBuffer.position() == addrBitcoinAmount * MachineState.VALUE_SIZE : "addrBitcoinAmount incorrect";
dataByteBuffer.putLong(bitcoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Bitcoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerBitcoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerBitcoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Bitcoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerBitcoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerBitcoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerBitcoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Bitcoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerBitcoinPKHOffset));
// Store partner's Bitcoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerBitcoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
/* NOP - to ensure BITCOIN ACCT is unique */
codeByteBuffer.put(OpCode.NOP.compile());
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile BTC-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), BitcoinACCTv3.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.BITCOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Bitcoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected BTC amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Bitcoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Bitcoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Bitcoin PKH
byte[] partnerBitcoinPKH = new byte[20];
dataByteBuffer.get(partnerBitcoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerBitcoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerBitcoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerBitcoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -58,9 +58,14 @@ public abstract class Bitcoiny implements ForeignBlockchain {
* i.e. keys with transactions but with no unspent outputs. */
protected final Set<ECKey> spentKeys = Collections.synchronizedSet(new HashSet<>());
/** How many bitcoinj wallet keys to generate in each batch. */
/** How many wallet keys to generate in each batch. */
private static final int WALLET_KEY_LOOKAHEAD_INCREMENT = 3;
/** How many wallet keys to generate when using bitcoinj as the data provider.
* We must use a higher value here since we are unable to request multiple batches of keys.
* Without this, the bitcoinj state can be missing transactions, causing errors such as "insufficient balance". */
private static final int WALLET_KEY_LOOKAHEAD_INCREMENT_BITCOINJ = 50;
/** Byte offset into raw block headers to block timestamp. */
private static final int TIMESTAMP_OFFSET = 4 + 32 + 32;
@@ -99,8 +104,9 @@ public abstract class Bitcoiny implements ForeignBlockchain {
try {
ScriptType addressType = Address.fromString(this.params, address).getOutputScriptType();
return addressType == ScriptType.P2PKH || addressType == ScriptType.P2SH;
return addressType == ScriptType.P2PKH || addressType == ScriptType.P2SH || addressType == ScriptType.P2WPKH;
} catch (AddressFormatException e) {
LOGGER.error(String.format("Unrecognised address format: %s", address));
return false;
}
}
@@ -471,6 +477,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
List<SimpleTransaction.Output> outputs = new ArrayList<>();
boolean anyOutputAddressInWallet = false;
boolean transactionInvolvesExternalWallet = false;
for (BitcoinyTransaction.Input input : t.inputs) {
try {
@@ -485,6 +492,9 @@ public abstract class Bitcoiny implements ForeignBlockchain {
total += inputAmount;
addressInWallet = true;
}
else {
transactionInvolvesExternalWallet = true;
}
inputs.add(new SimpleTransaction.Input(sender, inputAmount, addressInWallet));
}
}
@@ -498,14 +508,17 @@ public abstract class Bitcoiny implements ForeignBlockchain {
for (String address : output.addresses) {
boolean addressInWallet = false;
if (keySet.contains(address)) {
if (total > 0L) {
if (total > 0L) { // Change returned from sent amount
amount -= (total - output.value);
} else {
} else { // Amount received
amount += output.value;
}
addressInWallet = true;
anyOutputAddressInWallet = true;
}
else {
transactionInvolvesExternalWallet = true;
}
outputs.add(new SimpleTransaction.Output(address, output.value, addressInWallet));
}
}
@@ -520,6 +533,10 @@ public abstract class Bitcoiny implements ForeignBlockchain {
amount = total * -1;
}
}
else if (!transactionInvolvesExternalWallet) {
// All inputs and outputs relate to this wallet, so the balance should be unaffected
amount = 0;
}
return new SimpleTransaction(t.txHash, t.timestamp, amount, fee, inputs, outputs);
}
@@ -612,7 +629,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
this.keyChain = this.wallet.getActiveKeyChain();
// Set up wallet's key chain
this.keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
this.keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT_BITCOINJ);
this.keyChain.maybeLookAhead();
}

View File

@@ -0,0 +1,171 @@
package org.qortal.crosschain;
import java.util.Arrays;
import java.util.Collection;
import java.util.EnumMap;
import java.util.Map;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.Context;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.params.RegTestParams;
import org.bitcoinj.params.TestNet3Params;
import org.libdohj.params.DigibyteMainNetParams;
import org.qortal.crosschain.ElectrumX.Server;
import org.qortal.crosschain.ElectrumX.Server.ConnectionType;
import org.qortal.settings.Settings;
public class Digibyte extends Bitcoiny {
public static final String CURRENCY_CODE = "DGB";
private static final Coin DEFAULT_FEE_PER_KB = Coin.valueOf(100000); // 0.001 DGB per 1000 bytes
private static final long MINIMUM_ORDER_AMOUNT = 1000000; // 0.01 DGB minimum order, to avoid dust errors
// Temporary values until a dynamic fee system is written.
private static final long MAINNET_FEE = 10000L;
private static final long NON_MAINNET_FEE = 10000L; // enough for TESTNET3 and should be OK for REGTEST
private static final Map<ConnectionType, Integer> DEFAULT_ELECTRUMX_PORTS = new EnumMap<>(ConnectionType.class);
static {
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.TCP, 50001);
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.SSL, 50002);
}
public enum DigibyteNet {
MAIN {
@Override
public NetworkParameters getParams() {
return DigibyteMainNetParams.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
// Status verified at https://1209k.com/bitcoin-eye/ele.php?chain=dgb
new Server("electrum1.cipig.net", ConnectionType.SSL, 20059),
new Server("electrum2.cipig.net", ConnectionType.SSL, 20059),
new Server("electrum3.cipig.net", ConnectionType.SSL, 20059));
}
@Override
public String getGenesisHash() {
return "7497ea1b465eb39f1c8f507bc877078fe016d6fcb6dfad3a64c98dcc6e1e8496";
}
@Override
public long getP2shFee(Long timestamp) {
// TODO: This will need to be replaced with something better in the near future!
return MAINNET_FEE;
}
},
TEST3 {
@Override
public NetworkParameters getParams() {
return TestNet3Params.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(); // TODO: find testnet servers
}
@Override
public String getGenesisHash() {
return "308ea0711d5763be2995670dd9ca9872753561285a84da1d58be58acaa822252";
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
},
REGTEST {
@Override
public NetworkParameters getParams() {
return RegTestParams.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(
new Server("localhost", ConnectionType.TCP, 50001),
new Server("localhost", ConnectionType.SSL, 50002));
}
@Override
public String getGenesisHash() {
// This is unique to each regtest instance
return null;
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
};
public abstract NetworkParameters getParams();
public abstract Collection<Server> getServers();
public abstract String getGenesisHash();
public abstract long getP2shFee(Long timestamp) throws ForeignBlockchainException;
}
private static Digibyte instance;
private final DigibyteNet digibyteNet;
// Constructors and instance
private Digibyte(DigibyteNet digibyteNet, BitcoinyBlockchainProvider blockchain, Context bitcoinjContext, String currencyCode) {
super(blockchain, bitcoinjContext, currencyCode);
this.digibyteNet = digibyteNet;
LOGGER.info(() -> String.format("Starting Digibyte support using %s", this.digibyteNet.name()));
}
public static synchronized Digibyte getInstance() {
if (instance == null) {
DigibyteNet digibyteNet = Settings.getInstance().getDigibyteNet();
BitcoinyBlockchainProvider electrumX = new ElectrumX("Digibyte-" + digibyteNet.name(), digibyteNet.getGenesisHash(), digibyteNet.getServers(), DEFAULT_ELECTRUMX_PORTS);
Context bitcoinjContext = new Context(digibyteNet.getParams());
instance = new Digibyte(digibyteNet, electrumX, bitcoinjContext, CURRENCY_CODE);
}
return instance;
}
// Getters & setters
public static synchronized void resetForTesting() {
instance = null;
}
// Actual useful methods for use by other classes
@Override
public Coin getFeePerKb() {
return DEFAULT_FEE_PER_KB;
}
@Override
public long getMinimumOrderAmount() {
return MINIMUM_ORDER_AMOUNT;
}
/**
* Returns estimated DGB fee, in sats per 1000bytes, optionally for historic timestamp.
*
* @param timestamp optional milliseconds since epoch, or null for 'now'
* @return sats per 1000bytes, or throws ForeignBlockchainException if something went wrong
*/
@Override
public long getP2shFee(Long timestamp) throws ForeignBlockchainException {
return this.digibyteNet.getP2shFee(timestamp);
}
}

View File

@@ -0,0 +1,858 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Digibyte & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Digibyte & Qortal 'trade' keys</li>
* <li>Alice funds Digibyte P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Digibyte PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Digibyte PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Digibyte trade key and secret-A</li>
* <li>P2SH-A DGB funds end up at Digibyte address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class DigibyteACCTv3 implements ACCT {
private static final Logger LOGGER = LogManager.getLogger(DigibyteACCTv3.class);
public static final String NAME = DigibyteACCTv3.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("e6a7dcd87296fae3ce7d80183bf7660c8e2cb4f8746c6a0421a17148f87a0e1d").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerDigibytePKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerDigibytePKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Digibyte PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static DigibyteACCTv3 instance;
private DigibyteACCTv3() {
}
public static synchronized DigibyteACCTv3 getInstance() {
if (instance == null)
instance = new DigibyteACCTv3();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Digibyte.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param digibytePublicKeyHash 20-byte HASH160 of creator's trade Digibyte public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param digibyteAmount how much DGB the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] digibytePublicKeyHash, long qortAmount, long digibyteAmount, int tradeTimeout) {
if (digibytePublicKeyHash.length != 20)
throw new IllegalArgumentException("Digibyte public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrDigibytePublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrDigibyteAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerDigibytePKHOffset = addrCounter++;
final int addrPartnerDigibytePKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerDigibytePKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Digibyte public key hash
assert dataByteBuffer.position() == addrDigibytePublicKeyHash * MachineState.VALUE_SIZE : "addrDigibytePublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(digibytePublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Digibyte amount
assert dataByteBuffer.position() == addrDigibyteAmount * MachineState.VALUE_SIZE : "addrDigibyteAmount incorrect";
dataByteBuffer.putLong(digibyteAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Digibyte PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerDigibytePKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerDigibytePKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Digibyte PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerDigibytePKHPointer * MachineState.VALUE_SIZE : "addrPartnerDigibytePKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerDigibytePKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
/* NOP - to ensure DIGIBYTE ACCT is unique */
codeByteBuffer.put(OpCode.NOP.compile());
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Digibyte public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerDigibytePKHOffset));
// Store partner's Digibyte PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerDigibytePKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile DGB-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), DigibyteACCTv3.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.DIGIBYTE.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Digibyte/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected DGB amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Digibyte PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Digibyte PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Digibyte PKH
byte[] partnerDigibytePKH = new byte[20];
dataByteBuffer.get(partnerDigibytePKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerDigibytePKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerDigibytePKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerDigibytePKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -45,9 +45,9 @@ public class Dogecoin extends Bitcoiny {
public Collection<Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("electrum1.cipig.net", ConnectionType.TCP, 10060),
new Server("electrum2.cipig.net", ConnectionType.TCP, 10060),
new Server("electrum3.cipig.net", ConnectionType.TCP, 10060));
new Server("electrum1.cipig.net", ConnectionType.SSL, 20060),
new Server("electrum2.cipig.net", ConnectionType.SSL, 20060),
new Server("electrum3.cipig.net", ConnectionType.SSL, 20060));
// TODO: add more mainnet servers. It's too centralized.
}

View File

@@ -401,13 +401,36 @@ public class ElectrumX extends BitcoinyBlockchainProvider {
String scriptPubKey = (String) ((JSONObject) outputJson.get("scriptPubKey")).get("hex");
long value = BigDecimal.valueOf((Double) outputJson.get("value")).setScale(8).unscaledValue().longValue();
// address too, if present
// address too, if present in the "addresses" array
List<String> addresses = null;
Object addressesObj = ((JSONObject) outputJson.get("scriptPubKey")).get("addresses");
if (addressesObj instanceof JSONArray) {
addresses = new ArrayList<>();
for (Object addressObj : (JSONArray) addressesObj)
for (Object addressObj : (JSONArray) addressesObj) {
addresses.add((String) addressObj);
}
}
// some peers return a single "address" string
Object addressObj = ((JSONObject) outputJson.get("scriptPubKey")).get("address");
if (addressObj instanceof String) {
if (addresses == null) {
addresses = new ArrayList<>();
}
addresses.add((String) addressObj);
}
// For the purposes of Qortal we require all outputs to contain addresses
// Some servers omit this info, causing problems down the line with balance calculations
// Update: it turns out that they were just using a different key - "address" instead of "addresses"
// The code below can remain in place, just in case a peer returns a missing address in the future
if (addresses == null || addresses.isEmpty()) {
if (this.currentServer != null) {
this.uselessServers.add(this.currentServer);
this.closeServer(this.currentServer);
}
LOGGER.info("No output addresses returned for transaction {}", txHash);
throw new ForeignBlockchainException(String.format("No output addresses returned for transaction %s", txHash));
}
outputs.add(new BitcoinyTransaction.Output(scriptPubKey, value, addresses));

View File

@@ -44,23 +44,17 @@ public class Litecoin extends Bitcoiny {
public Collection<ElectrumX.Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
new Server("electrum-ltc.someguy123.net", Server.ConnectionType.SSL, 50002),
new Server("backup.electrum-ltc.org", Server.ConnectionType.TCP, 50001),
// Status verified at https://1209k.com/bitcoin-eye/ele.php?chain=ltc
//CLOSED new Server("electrum-ltc.petrkr.net", Server.ConnectionType.SSL, 60002),
//CLOSED new Server("electrum-ltc.someguy123.net", Server.ConnectionType.SSL, 50002),
//PHISHY new Server("electrum-ltc.bysh.me", Server.ConnectionType.SSL, 50002),
new Server("backup.electrum-ltc.org", Server.ConnectionType.SSL, 443),
new Server("electrum.ltc.xurious.com", Server.ConnectionType.TCP, 50001),
new Server("electrum.ltc.xurious.com", Server.ConnectionType.SSL, 50002),
new Server("electrum-ltc.bysh.me", Server.ConnectionType.SSL, 50002),
new Server("electrum1.cipig.net", Server.ConnectionType.SSL, 20063),
new Server("electrum2.cipig.net", Server.ConnectionType.SSL, 20063),
new Server("electrum3.cipig.net", Server.ConnectionType.SSL, 20063),
new Server("electrum3.cipig.net", ConnectionType.TCP, 10063),
new Server("electrum2.cipig.net", Server.ConnectionType.TCP, 10063),
new Server("electrum1.cipig.net", Server.ConnectionType.SSL, 20063),
new Server("electrum1.cipig.net", Server.ConnectionType.TCP, 10063),
new Server("electrum-ltc.petrkr.net", Server.ConnectionType.SSL, 60002),
new Server("ltc.litepay.ch", Server.ConnectionType.SSL, 50022),
new Server("electrum-ltc-bysh.me", Server.ConnectionType.TCP, 50002),
new Server("electrum.jochen-hoenicke.de", Server.ConnectionType.TCP, 50005),
new Server("node.ispol.sk", Server.ConnectionType.TCP, 50004));
new Server("ltc.rentonrisk.com", Server.ConnectionType.SSL, 50002));
}
@Override

View File

@@ -0,0 +1,175 @@
package org.qortal.crosschain;
import java.util.Arrays;
import java.util.Collection;
import java.util.EnumMap;
import java.util.Map;
import org.bitcoinj.core.Coin;
import org.bitcoinj.core.Context;
import org.bitcoinj.core.NetworkParameters;
import org.bitcoinj.params.RegTestParams;
import org.bitcoinj.params.TestNet3Params;
import org.libdohj.params.RavencoinMainNetParams;
import org.qortal.crosschain.ElectrumX.Server;
import org.qortal.crosschain.ElectrumX.Server.ConnectionType;
import org.qortal.settings.Settings;
public class Ravencoin extends Bitcoiny {
public static final String CURRENCY_CODE = "RVN";
private static final Coin DEFAULT_FEE_PER_KB = Coin.valueOf(1125000); // 0.01125 RVN per 1000 bytes
private static final long MINIMUM_ORDER_AMOUNT = 1000000; // 0.01 RVN minimum order, to avoid dust errors
// Temporary values until a dynamic fee system is written.
private static final long MAINNET_FEE = 1000000L;
private static final long NON_MAINNET_FEE = 1000000L; // enough for TESTNET3 and should be OK for REGTEST
private static final Map<ConnectionType, Integer> DEFAULT_ELECTRUMX_PORTS = new EnumMap<>(ConnectionType.class);
static {
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.TCP, 50001);
DEFAULT_ELECTRUMX_PORTS.put(ConnectionType.SSL, 50002);
}
public enum RavencoinNet {
MAIN {
@Override
public NetworkParameters getParams() {
return RavencoinMainNetParams.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(
// Servers chosen on NO BASIS WHATSOEVER from various sources!
// Status verified at https://1209k.com/bitcoin-eye/ele.php?chain=rvn
new Server("aethyn.com", ConnectionType.SSL, 50002),
new Server("electrum2.rvn.rocks", ConnectionType.SSL, 50002),
new Server("rvn-dashboard.com", ConnectionType.SSL, 50002),
new Server("rvn4lyfe.com", ConnectionType.SSL, 50002),
new Server("electrum1.cipig.net", ConnectionType.SSL, 20051),
new Server("electrum2.cipig.net", ConnectionType.SSL, 20051),
new Server("electrum3.cipig.net", ConnectionType.SSL, 20051));
}
@Override
public String getGenesisHash() {
return "0000006b444bc2f2ffe627be9d9e7e7a0730000870ef6eb6da46c8eae389df90";
}
@Override
public long getP2shFee(Long timestamp) {
// TODO: This will need to be replaced with something better in the near future!
return MAINNET_FEE;
}
},
TEST3 {
@Override
public NetworkParameters getParams() {
return TestNet3Params.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(); // TODO: find testnet servers
}
@Override
public String getGenesisHash() {
return "000000ecfc5e6324a079542221d00e10362bdc894d56500c414060eea8a3ad5a";
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
},
REGTEST {
@Override
public NetworkParameters getParams() {
return RegTestParams.get();
}
@Override
public Collection<Server> getServers() {
return Arrays.asList(
new Server("localhost", ConnectionType.TCP, 50001),
new Server("localhost", ConnectionType.SSL, 50002));
}
@Override
public String getGenesisHash() {
// This is unique to each regtest instance
return null;
}
@Override
public long getP2shFee(Long timestamp) {
return NON_MAINNET_FEE;
}
};
public abstract NetworkParameters getParams();
public abstract Collection<Server> getServers();
public abstract String getGenesisHash();
public abstract long getP2shFee(Long timestamp) throws ForeignBlockchainException;
}
private static Ravencoin instance;
private final RavencoinNet ravencoinNet;
// Constructors and instance
private Ravencoin(RavencoinNet ravencoinNet, BitcoinyBlockchainProvider blockchain, Context bitcoinjContext, String currencyCode) {
super(blockchain, bitcoinjContext, currencyCode);
this.ravencoinNet = ravencoinNet;
LOGGER.info(() -> String.format("Starting Ravencoin support using %s", this.ravencoinNet.name()));
}
public static synchronized Ravencoin getInstance() {
if (instance == null) {
RavencoinNet ravencoinNet = Settings.getInstance().getRavencoinNet();
BitcoinyBlockchainProvider electrumX = new ElectrumX("Ravencoin-" + ravencoinNet.name(), ravencoinNet.getGenesisHash(), ravencoinNet.getServers(), DEFAULT_ELECTRUMX_PORTS);
Context bitcoinjContext = new Context(ravencoinNet.getParams());
instance = new Ravencoin(ravencoinNet, electrumX, bitcoinjContext, CURRENCY_CODE);
}
return instance;
}
// Getters & setters
public static synchronized void resetForTesting() {
instance = null;
}
// Actual useful methods for use by other classes
@Override
public Coin getFeePerKb() {
return DEFAULT_FEE_PER_KB;
}
@Override
public long getMinimumOrderAmount() {
return MINIMUM_ORDER_AMOUNT;
}
/**
* Returns estimated RVN fee, in sats per 1000bytes, optionally for historic timestamp.
*
* @param timestamp optional milliseconds since epoch, or null for 'now'
* @return sats per 1000bytes, or throws ForeignBlockchainException if something went wrong
*/
@Override
public long getP2shFee(Long timestamp) throws ForeignBlockchainException {
return this.ravencoinNet.getP2shFee(timestamp);
}
}

View File

@@ -0,0 +1,858 @@
package org.qortal.crosschain;
import com.google.common.hash.HashCode;
import com.google.common.primitives.Bytes;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.ciyam.at.*;
import org.qortal.account.Account;
import org.qortal.asset.Asset;
import org.qortal.at.QortalFunctionCode;
import org.qortal.crypto.Crypto;
import org.qortal.data.at.ATData;
import org.qortal.data.at.ATStateData;
import org.qortal.data.crosschain.CrossChainTradeData;
import org.qortal.data.transaction.MessageTransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.utils.Base58;
import org.qortal.utils.BitTwiddling;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import static org.ciyam.at.OpCode.calcOffset;
/**
* Cross-chain trade AT
*
* <p>
* <ul>
* <li>Bob generates Ravencoin & Qortal 'trade' keys
* <ul>
* <li>private key required to sign P2SH redeem tx</li>
* <li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
* <li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
* </ul>
* </li>
* <li>Bob deploys Qortal AT
* <ul>
* </ul>
* </li>
* <li>Alice finds Qortal AT and wants to trade
* <ul>
* <li>Alice generates Ravencoin & Qortal 'trade' keys</li>
* <li>Alice funds Ravencoin P2SH-A</li>
* <li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
* <ul>
* <li>hash-of-secret-A</li>
* <li>her 'trade' Ravencoin PKH</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Bob receives "offer" MESSAGE
* <ul>
* <li>Checks Alice's P2SH-A</li>
* <li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
* <ul>
* <li>Alice's trade Qortal address</li>
* <li>Alice's trade Ravencoin PKH</li>
* <li>hash-of-secret-A</li>
* </ul>
* </li>
* </ul>
* </li>
* <li>Alice checks Qortal AT to confirm it's locked to her
* <ul>
* <li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
* <ul>
* <li>secret-A</li>
* <li>Qortal receiving address of her chosing</li>
* </ul>
* </li>
* <li>AT's QORT funds are sent to Qortal receiving address</li>
* </ul>
* </li>
* <li>Bob checks AT, extracts secret-A
* <ul>
* <li>Bob redeems P2SH-A using his Ravencoin trade key and secret-A</li>
* <li>P2SH-A RVN funds end up at Ravencoin address determined by redeem transaction output(s)</li>
* </ul>
* </li>
* </ul>
*/
public class RavencoinACCTv3 implements ACCT {
private static final Logger LOGGER = LogManager.getLogger(RavencoinACCTv3.class);
public static final String NAME = RavencoinACCTv3.class.getSimpleName();
public static final byte[] CODE_BYTES_HASH = HashCode.fromString("91395fa1ec0dfa35beddb0a7f4cc0a1bede157c38787ddb0af0cf03dfdc10f77").asBytes(); // SHA256 of AT code bytes
public static final int SECRET_LENGTH = 32;
/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
private static final int MODE_VALUE_OFFSET = 61;
/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
public static class OfferMessageData {
public byte[] partnerRavencoinPKH;
public byte[] hashOfSecretA;
public long lockTimeA;
}
public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerRavencoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+ 24 /*partner's Ravencoin PKH (padded from 20 to 24)*/
+ 8 /*AT trade timeout (minutes)*/
+ 24 /*hash of secret-A (padded from 20 to 24)*/
+ 8 /*lockTimeA*/;
public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
private static RavencoinACCTv3 instance;
private RavencoinACCTv3() {
}
public static synchronized RavencoinACCTv3 getInstance() {
if (instance == null)
instance = new RavencoinACCTv3();
return instance;
}
@Override
public byte[] getCodeBytesHash() {
return CODE_BYTES_HASH;
}
@Override
public int getModeByteOffset() {
return MODE_BYTE_OFFSET;
}
@Override
public ForeignBlockchain getBlockchain() {
return Ravencoin.getInstance();
}
/**
* Returns Qortal AT creation bytes for cross-chain trading AT.
* <p>
* <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
* 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
*
* @param creatorTradeAddress AT creator's trade Qortal address
* @param ravencoinPublicKeyHash 20-byte HASH160 of creator's trade Ravencoin public key
* @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
* @param ravencoinAmount how much RVN the AT creator is expecting to trade
* @param tradeTimeout suggested timeout for entire trade
*/
public static byte[] buildQortalAT(String creatorTradeAddress, byte[] ravencoinPublicKeyHash, long qortAmount, long ravencoinAmount, int tradeTimeout) {
if (ravencoinPublicKeyHash.length != 20)
throw new IllegalArgumentException("Ravencoin public key hash should be 20 bytes");
// Labels for data segment addresses
int addrCounter = 0;
// Constants (with corresponding dataByteBuffer.put*() calls below)
final int addrCreatorTradeAddress1 = addrCounter++;
final int addrCreatorTradeAddress2 = addrCounter++;
final int addrCreatorTradeAddress3 = addrCounter++;
final int addrCreatorTradeAddress4 = addrCounter++;
final int addrRavencoinPublicKeyHash = addrCounter;
addrCounter += 4;
final int addrQortAmount = addrCounter++;
final int addrRavencoinAmount = addrCounter++;
final int addrTradeTimeout = addrCounter++;
final int addrMessageTxnType = addrCounter++;
final int addrExpectedTradeMessageLength = addrCounter++;
final int addrExpectedRedeemMessageLength = addrCounter++;
final int addrCreatorAddressPointer = addrCounter++;
final int addrQortalPartnerAddressPointer = addrCounter++;
final int addrMessageSenderPointer = addrCounter++;
final int addrTradeMessagePartnerRavencoinPKHOffset = addrCounter++;
final int addrPartnerRavencoinPKHPointer = addrCounter++;
final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
final int addrHashOfSecretAPointer = addrCounter++;
final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
final int addrMessageDataPointer = addrCounter++;
final int addrMessageDataLength = addrCounter++;
final int addrPartnerReceivingAddressPointer = addrCounter++;
final int addrEndOfConstants = addrCounter;
// Variables
final int addrCreatorAddress1 = addrCounter++;
final int addrCreatorAddress2 = addrCounter++;
final int addrCreatorAddress3 = addrCounter++;
final int addrCreatorAddress4 = addrCounter++;
final int addrQortalPartnerAddress1 = addrCounter++;
final int addrQortalPartnerAddress2 = addrCounter++;
final int addrQortalPartnerAddress3 = addrCounter++;
final int addrQortalPartnerAddress4 = addrCounter++;
final int addrLockTimeA = addrCounter++;
final int addrRefundTimeout = addrCounter++;
final int addrRefundTimestamp = addrCounter++;
final int addrLastTxnTimestamp = addrCounter++;
final int addrBlockTimestamp = addrCounter++;
final int addrTxnType = addrCounter++;
final int addrResult = addrCounter++;
final int addrMessageSender1 = addrCounter++;
final int addrMessageSender2 = addrCounter++;
final int addrMessageSender3 = addrCounter++;
final int addrMessageSender4 = addrCounter++;
final int addrMessageLength = addrCounter++;
final int addrMessageData = addrCounter;
addrCounter += 4;
final int addrHashOfSecretA = addrCounter;
addrCounter += 4;
final int addrPartnerRavencoinPKH = addrCounter;
addrCounter += 4;
final int addrPartnerReceivingAddress = addrCounter;
addrCounter += 4;
final int addrMode = addrCounter++;
assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
// Data segment
ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
// AT creator's trade Qortal address, decoded from Base58
assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
// Ravencoin public key hash
assert dataByteBuffer.position() == addrRavencoinPublicKeyHash * MachineState.VALUE_SIZE : "addrRavencoinPublicKeyHash incorrect";
dataByteBuffer.put(Bytes.ensureCapacity(ravencoinPublicKeyHash, 32, 0));
// Redeem Qort amount
assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
dataByteBuffer.putLong(qortAmount);
// Expected Ravencoin amount
assert dataByteBuffer.position() == addrRavencoinAmount * MachineState.VALUE_SIZE : "addrRavencoinAmount incorrect";
dataByteBuffer.putLong(ravencoinAmount);
// Suggested trade timeout (minutes)
assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
dataByteBuffer.putLong(tradeTimeout);
// We're only interested in MESSAGE transactions
assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
// Expected length of 'trade' MESSAGE data from AT creator
assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
// Expected length of 'redeem' MESSAGE data from trade partner
assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
// Index into data segment of AT creator's address, used by GET_B_IND
assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
dataByteBuffer.putLong(addrCreatorAddress1);
// Index into data segment of partner's Qortal address, used by SET_B_IND
assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
dataByteBuffer.putLong(addrQortalPartnerAddress1);
// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
dataByteBuffer.putLong(addrMessageSender1);
// Offset into 'trade' MESSAGE data payload for extracting partner's Ravencoin PKH
assert dataByteBuffer.position() == addrTradeMessagePartnerRavencoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerRavencoinPKHOffset incorrect";
dataByteBuffer.putLong(32L);
// Index into data segment of partner's Ravencoin PKH, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerRavencoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerRavencoinPKHPointer incorrect";
dataByteBuffer.putLong(addrPartnerRavencoinPKH);
// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
dataByteBuffer.putLong(64L);
// Index into data segment to hash of secret A, used by GET_B_IND
assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
dataByteBuffer.putLong(addrHashOfSecretA);
// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
dataByteBuffer.putLong(32L);
// Source location and length for hashing any passed secret
assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
dataByteBuffer.putLong(addrMessageData);
assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
dataByteBuffer.putLong(32L);
// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
dataByteBuffer.putLong(addrPartnerReceivingAddress);
assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
// Code labels
Integer labelRefund = null;
Integer labelTradeTxnLoop = null;
Integer labelCheckTradeTxn = null;
Integer labelCheckCancelTxn = null;
Integer labelNotTradeNorCancelTxn = null;
Integer labelCheckNonRefundTradeTxn = null;
Integer labelTradeTxnExtract = null;
Integer labelRedeemTxnLoop = null;
Integer labelCheckRedeemTxn = null;
Integer labelCheckRedeemTxnSender = null;
Integer labelPayout = null;
ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
// Two-pass version
for (int pass = 0; pass < 2; ++pass) {
codeByteBuffer.clear();
try {
/* Initialization */
// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
/* NOP - to ensure RAVENCOIN ACCT is unique */
codeByteBuffer.put(OpCode.NOP.compile());
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
/* Transaction processing loop */
labelTradeTxnLoop = codeByteBuffer.position();
/* Sleep until message arrives */
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckTradeTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
// Message sender's address matches AT creator's trade address so go process 'trade' message
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
/* Checking message sender for possible cancel message */
labelCheckCancelTxn = codeByteBuffer.position();
// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
// Partner address is AT creator's address, so cancel offer and finish.
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
/* Not trade nor cancel message */
labelNotTradeNorCancelTxn = codeByteBuffer.position();
// Loop to find another transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Possible switch-to-trade-mode message */
labelCheckNonRefundTradeTxn = codeByteBuffer.position();
// Check 'trade' message we received has expected number of message bytes
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to info extraction code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
/* Extracting info from 'trade' MESSAGE transaction */
labelTradeTxnExtract = codeByteBuffer.position();
// Extract message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
// Extract trade partner's Ravencoin public key hash (PKH) from message into B
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerRavencoinPKHOffset));
// Store partner's Ravencoin PKH (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerRavencoinPKHPointer));
// Extract AT trade timeout (minutes) (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
// Grab next 32 bytes
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
// Extract hash-of-secret-A (we only really use values from B1-B3)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
// Extract lockTime-A (from B4)
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
/* We are in 'trade mode' */
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
// Set restart position to after this opcode
codeByteBuffer.put(OpCode.SET_PCS.compile());
/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
// Fetch current block 'timestamp'
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
// If we're not past refund 'timestamp' then look for next transaction
codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
// We're past refund 'timestamp' so go refund everything back to AT creator
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
/* Transaction processing loop */
labelRedeemTxnLoop = codeByteBuffer.position();
// Find next transaction to this AT since the last one (if any)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
// Stop and wait for next block
codeByteBuffer.put(OpCode.STP_IMD.compile());
/* Check transaction */
labelCheckRedeemTxn = codeByteBuffer.position();
// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
// Extract transaction type (message/payment) from transaction and save type in addrTxnType
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
// If transaction type is not MESSAGE type then go look for another transaction
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check message payload length */
codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
// If message length matches, branch to sender checking code
codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Check transaction's sender */
labelCheckRedeemTxnSender = codeByteBuffer.position();
// Extract sender address from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
/* Check 'secret-A' in transaction's message */
// Extract secret-A from first 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
// Save the equality result (1 if they match, 0 otherwise) into addrResult.
codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
// If hashes don't match, addrResult will be zero so go find another transaction
codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
/* Success! Pay arranged amount to receiving address */
labelPayout = codeByteBuffer.position();
// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
// Pay AT's balance to receiving address
codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
// Set redeemed mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
// Fall-through to refunding any remaining balance back to AT creator
/* Refund balance back to AT creator */
labelRefund = codeByteBuffer.position();
// Set refunded mode
codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
// We're finished forever (finishing auto-refunds remaining balance to AT creator)
codeByteBuffer.put(OpCode.FIN_IMD.compile());
} catch (CompilationException e) {
throw new IllegalStateException("Unable to compile RVN-QORT ACCT?", e);
}
}
codeByteBuffer.flip();
byte[] codeBytes = new byte[codeByteBuffer.limit()];
codeByteBuffer.get(codeBytes);
assert Arrays.equals(Crypto.digest(codeBytes), RavencoinACCTv3.CODE_BYTES_HASH)
: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
final short ciyamAtVersion = 2;
final short numCallStackPages = 0;
final short numUserStackPages = 0;
final long minActivationAmount = 0L;
return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
@Override
public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
}
/**
* Returns CrossChainTradeData with useful info extracted from AT.
*/
public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
byte[] addressBytes = new byte[25]; // for general use
String atAddress = atStateData.getATAddress();
CrossChainTradeData tradeData = new CrossChainTradeData();
tradeData.foreignBlockchain = SupportedBlockchain.RAVENCOIN.name();
tradeData.acctName = NAME;
tradeData.qortalAtAddress = atAddress;
tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
tradeData.creationTimestamp = creationTimestamp;
Account atAccount = new Account(repository, atAddress);
tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
byte[] stateData = atStateData.getStateData();
ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
dataByteBuffer.position(MachineState.HEADER_LENGTH);
/* Constants */
// Skip creator's trade address
dataByteBuffer.get(addressBytes);
tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Creator's Ravencoin/foreign public key hash
tradeData.creatorForeignPKH = new byte[20];
dataByteBuffer.get(tradeData.creatorForeignPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
// We don't use secret-B
tradeData.hashOfSecretB = null;
// Redeem payout
tradeData.qortAmount = dataByteBuffer.getLong();
// Expected RVN amount
tradeData.expectedForeignAmount = dataByteBuffer.getLong();
// Trade timeout
tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
// Skip MESSAGE transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'trade' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip expected 'redeem' message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Qortal trade address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message sender
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for partner's Ravencoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's Ravencoin PKH
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'trade' message data offset for hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to hash-of-secret-A
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip 'redeem' message data offset for partner's Qortal receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to message data
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip message data length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip pointer to partner's receiving address
dataByteBuffer.position(dataByteBuffer.position() + 8);
/* End of constants / begin variables */
// Skip AT creator's address
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Partner's trade address (if present)
dataByteBuffer.get(addressBytes);
String qortalRecipient = Base58.encode(addressBytes);
dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
// Potential lockTimeA (if in trade mode)
int lockTimeA = (int) dataByteBuffer.getLong();
// AT refund timeout (probably only useful for debugging)
int refundTimeout = (int) dataByteBuffer.getLong();
// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
long tradeRefundTimestamp = dataByteBuffer.getLong();
// Skip last transaction timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip block timestamp
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip transaction type
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary result
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message sender
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Skip message length
dataByteBuffer.position(dataByteBuffer.position() + 8);
// Skip temporary message data
dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
// Potential hash160 of secret A
byte[] hashOfSecretA = new byte[20];
dataByteBuffer.get(hashOfSecretA);
dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
// Potential partner's Ravencoin PKH
byte[] partnerRavencoinPKH = new byte[20];
dataByteBuffer.get(partnerRavencoinPKH);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerRavencoinPKH.length); // skip to 32 bytes
// Partner's receiving address (if present)
byte[] partnerReceivingAddress = new byte[25];
dataByteBuffer.get(partnerReceivingAddress);
dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
// Trade AT's 'mode'
long modeValue = dataByteBuffer.getLong();
AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
/* End of variables */
if (mode != null && mode != AcctMode.OFFERING) {
tradeData.mode = mode;
tradeData.refundTimeout = refundTimeout;
tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
tradeData.qortalPartnerAddress = qortalRecipient;
tradeData.hashOfSecretA = hashOfSecretA;
tradeData.partnerForeignPKH = partnerRavencoinPKH;
tradeData.lockTimeA = lockTimeA;
if (mode == AcctMode.REDEEMED)
tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
} else {
tradeData.mode = AcctMode.OFFERING;
}
tradeData.duplicateDeprecated();
return tradeData;
}
/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
}
/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
public static OfferMessageData extractOfferMessageData(byte[] messageData) {
if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
return null;
OfferMessageData offerMessageData = new OfferMessageData();
offerMessageData.partnerRavencoinPKH = Arrays.copyOfRange(messageData, 0, 20);
offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
return offerMessageData;
}
/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
byte[] data = new byte[TRADE_MESSAGE_LENGTH];
byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
return data;
}
/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
@Override
public byte[] buildCancelMessage(String creatorQortalAddress) {
byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
return data;
}
/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
System.arraycopy(secretA, 0, data, 0, secretA.length);
System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
return data;
}
/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
}
@Override
public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
String atAddress = crossChainTradeData.qortalAtAddress;
String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
// We don't have partner's public key so we check every message to AT
List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
if (messageTransactionsData == null)
return null;
// Find 'redeem' message
for (MessageTransactionData messageTransactionData : messageTransactionsData) {
// Check message payload type/encryption
if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
continue;
// Check message payload size
byte[] messageData = messageTransactionData.getData();
if (messageData.length != REDEEM_MESSAGE_LENGTH)
// Wrong payload length
continue;
// Check sender
if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
// Wrong sender;
continue;
// Extract secretA
byte[] secretA = new byte[32];
System.arraycopy(messageData, 0, secretA, 0, secretA.length);
byte[] hashOfSecretA = Crypto.hash160(secretA);
if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
continue;
return secretA;
}
return null;
}
}

View File

@@ -13,8 +13,8 @@ import org.qortal.utils.Triple;
public enum SupportedBlockchain {
BITCOIN(Arrays.asList(
Triple.valueOf(BitcoinACCTv1.NAME, BitcoinACCTv1.CODE_BYTES_HASH, BitcoinACCTv1::getInstance)
// Could add improved BitcoinACCTv2 here in the future
Triple.valueOf(BitcoinACCTv1.NAME, BitcoinACCTv1.CODE_BYTES_HASH, BitcoinACCTv1::getInstance),
Triple.valueOf(BitcoinACCTv3.NAME, BitcoinACCTv3.CODE_BYTES_HASH, BitcoinACCTv3::getInstance)
)) {
@Override
public ForeignBlockchain getInstance() {
@@ -23,7 +23,7 @@ public enum SupportedBlockchain {
@Override
public ACCT getLatestAcct() {
return BitcoinACCTv1.getInstance();
return BitcoinACCTv3.getInstance();
}
},
@@ -57,12 +57,40 @@ public enum SupportedBlockchain {
public ACCT getLatestAcct() {
return DogecoinACCTv3.getInstance();
}
},
DIGIBYTE(Arrays.asList(
Triple.valueOf(DigibyteACCTv3.NAME, DigibyteACCTv3.CODE_BYTES_HASH, DigibyteACCTv3::getInstance)
)) {
@Override
public ForeignBlockchain getInstance() {
return Digibyte.getInstance();
}
@Override
public ACCT getLatestAcct() {
return DigibyteACCTv3.getInstance();
}
},
RAVENCOIN(Arrays.asList(
Triple.valueOf(RavencoinACCTv3.NAME, RavencoinACCTv3.CODE_BYTES_HASH, RavencoinACCTv3::getInstance)
)) {
@Override
public ForeignBlockchain getInstance() {
return Ravencoin.getInstance();
}
@Override
public ACCT getLatestAcct() {
return RavencoinACCTv3.getInstance();
}
};
private static final Map<ByteArray, Supplier<ACCT>> supportedAcctsByCodeHash = Arrays.stream(SupportedBlockchain.values())
.map(supportedBlockchain -> supportedBlockchain.supportedAccts)
.flatMap(List::stream)
.collect(Collectors.toUnmodifiableMap(triple -> new ByteArray(triple.getB()), Triple::getC));
.collect(Collectors.toUnmodifiableMap(triple -> ByteArray.wrap(triple.getB()), Triple::getC));
private static final Map<String, Supplier<ACCT>> supportedAcctsByName = Arrays.stream(SupportedBlockchain.values())
.map(supportedBlockchain -> supportedBlockchain.supportedAccts)
@@ -94,7 +122,7 @@ public enum SupportedBlockchain {
return getAcctMap();
return blockchain.supportedAccts.stream()
.collect(Collectors.toUnmodifiableMap(triple -> new ByteArray(triple.getB()), Triple::getC));
.collect(Collectors.toUnmodifiableMap(triple -> ByteArray.wrap(triple.getB()), Triple::getC));
}
public static Map<ByteArray, Supplier<ACCT>> getFilteredAcctMap(String specificBlockchain) {
@@ -109,7 +137,7 @@ public enum SupportedBlockchain {
}
public static ACCT getAcctByCodeHash(byte[] codeHash) {
ByteArray wrappedCodeHash = new ByteArray(codeHash);
ByteArray wrappedCodeHash = ByteArray.wrap(codeHash);
Supplier<ACCT> acctInstanceSupplier = supportedAcctsByCodeHash.get(wrappedCodeHash);

View File

@@ -0,0 +1,12 @@
package org.qortal.data.arbitrary;
public class ArbitraryCategoryInfo {
public String id;
public String name;
public ArbitraryCategoryInfo() {
}
}

View File

@@ -0,0 +1,59 @@
package org.qortal.data.arbitrary;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
public class ArbitraryDirectConnectionInfo {
private final byte[] signature;
private final String peerAddress;
private final List<byte[]> hashes;
private final long timestamp;
public ArbitraryDirectConnectionInfo(byte[] signature, String peerAddress, List<byte[]> hashes, long timestamp) {
this.signature = signature;
this.peerAddress = peerAddress;
this.hashes = hashes;
this.timestamp = timestamp;
}
public byte[] getSignature() {
return this.signature;
}
public String getPeerAddress() {
return this.peerAddress;
}
public List<byte[]> getHashes() {
return this.hashes;
}
public long getTimestamp() {
return this.timestamp;
}
public int getHashCount() {
if (this.hashes == null) {
return 0;
}
return this.hashes.size();
}
@Override
public boolean equals(Object other) {
if (other == this)
return true;
if (!(other instanceof ArbitraryDirectConnectionInfo))
return false;
ArbitraryDirectConnectionInfo otherDirectConnectionInfo = (ArbitraryDirectConnectionInfo) other;
return Arrays.equals(this.signature, otherDirectConnectionInfo.getSignature())
&& Objects.equals(this.peerAddress, otherDirectConnectionInfo.getPeerAddress())
&& Objects.equals(this.hashes, otherDirectConnectionInfo.getHashes())
&& Objects.equals(this.timestamp, otherDirectConnectionInfo.getTimestamp());
}
}

View File

@@ -0,0 +1,11 @@
package org.qortal.data.arbitrary;
import org.qortal.network.Peer;
public class ArbitraryFileListResponseInfo extends ArbitraryRelayInfo {
public ArbitraryFileListResponseInfo(String hash58, String signature58, Peer peer, Long timestamp, Long requestTime, Integer requestHops) {
super(hash58, signature58, peer, timestamp, requestTime, requestHops);
}
}

View File

@@ -9,12 +9,16 @@ public class ArbitraryRelayInfo {
private final String signature58;
private final Peer peer;
private final Long timestamp;
private final Long requestTime;
private final Integer requestHops;
public ArbitraryRelayInfo(String hash58, String signature58, Peer peer, Long timestamp) {
public ArbitraryRelayInfo(String hash58, String signature58, Peer peer, Long timestamp, Long requestTime, Integer requestHops) {
this.hash58 = hash58;
this.signature58 = signature58;
this.peer = peer;
this.timestamp = timestamp;
this.requestTime = requestTime;
this.requestHops = requestHops;
}
public boolean isValid() {
@@ -38,6 +42,14 @@ public class ArbitraryRelayInfo {
return timestamp;
}
public Long getRequestTime() {
return this.requestTime;
}
public Integer getRequestHops() {
return this.requestHops;
}
@Override
public String toString() {
return String.format("%s = %s, %s, %d", this.hash58, this.signature58, this.peer, this.timestamp);

View File

@@ -13,6 +13,7 @@ public class ArbitraryResourceInfo {
public Service service;
public String identifier;
public ArbitraryResourceStatus status;
public ArbitraryResourceMetadata metadata;
public Long size;

View File

@@ -0,0 +1,45 @@
package org.qortal.data.arbitrary;
import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
import org.qortal.arbitrary.misc.Category;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import java.util.List;
@XmlAccessorType(XmlAccessType.FIELD)
public class ArbitraryResourceMetadata {
private String title;
private String description;
private List<String> tags;
private Category category;
private String categoryName;
public ArbitraryResourceMetadata() {
}
public ArbitraryResourceMetadata(String title, String description, List<String> tags, Category category) {
this.title = title;
this.description = description;
this.tags = tags;
this.category = category;
this.categoryName = category.getName();
}
public static ArbitraryResourceMetadata fromTransactionMetadata(ArbitraryDataTransactionMetadata transactionMetadata) {
if (transactionMetadata == null) {
return null;
}
String title = transactionMetadata.getTitle();
String description = transactionMetadata.getDescription();
List<String> tags = transactionMetadata.getTags();
Category category = transactionMetadata.getCategory();
if (title == null && description == null && tags == null && category == null) {
return null;
}
return new ArbitraryResourceMetadata(title, description, tags, category);
}
}

View File

@@ -94,6 +94,12 @@ public class CrossChainTradeData {
public String acctName;
@Schema(description = "Timestamp when AT creator's trade-bot presence expires")
public Long creatorPresenceExpiry;
@Schema(description = "Timestamp when trade partner's trade-bot presence expires")
public Long partnerPresenceExpiry;
// Constructors
// Necessary for JAXB

View File

@@ -23,6 +23,7 @@ public class GroupData {
private ApprovalThreshold approvalThreshold;
private int minimumBlockDelay;
private int maximumBlockDelay;
public int memberCount;
/** Reference to CREATE_GROUP or UPDATE_GROUP transaction, used to rebuild group during orphaning. */
// No need to ever expose this via API

View File

@@ -0,0 +1,114 @@
package org.qortal.data.network;
import org.qortal.crypto.Crypto;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlTransient;
import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import java.util.Arrays;
// All properties to be converted to JSON via JAXB
@XmlAccessorType(XmlAccessType.FIELD)
public class TradePresenceData {
protected long timestamp;
@XmlJavaTypeAdapter(
type = byte[].class,
value = org.qortal.api.Base58TypeAdapter.class
)
protected byte[] publicKey; // Could be BOB's or ALICE's
// No need to send this via websocket / API
@XmlTransient
protected byte[] signature; // Not always present
protected String atAddress; // Not always present
// Have JAXB use getter instead
@XmlTransient
protected String tradeAddress; // Lazily instantiated
// Constructors
// necessary for JAXB serialization
protected TradePresenceData() {
}
public TradePresenceData(long timestamp, byte[] publicKey, byte[] signature, String atAddress) {
this.timestamp = timestamp;
this.publicKey = publicKey;
this.signature = signature;
this.atAddress = atAddress;
}
public TradePresenceData(long timestamp, byte[] publicKey) {
this(timestamp, publicKey, null, null);
}
public long getTimestamp() {
return this.timestamp;
}
public byte[] getPublicKey() {
return this.publicKey;
}
public byte[] getSignature() {
return this.signature;
}
public String getAtAddress() {
return this.atAddress;
}
// Probably doesn't need synchronization
@XmlElement
public String getTradeAddress() {
if (tradeAddress != null)
return tradeAddress;
tradeAddress = Crypto.toAddress(this.publicKey);
return tradeAddress;
}
// Comparison
@Override
public boolean equals(Object other) {
if (other == this)
return true;
if (!(other instanceof TradePresenceData))
return false;
TradePresenceData otherTradePresenceData = (TradePresenceData) other;
// Very quick comparison
if (otherTradePresenceData.timestamp != this.timestamp)
return false;
if (!Arrays.equals(otherTradePresenceData.publicKey, this.publicKey))
return false;
if (otherTradePresenceData.atAddress != null && !otherTradePresenceData.atAddress.equals(this.atAddress))
return false;
if (this.atAddress != null && !this.atAddress.equals(otherTradePresenceData.atAddress))
return false;
if (!Arrays.equals(otherTradePresenceData.signature, this.signature))
return false;
return true;
}
@Override
public int hashCode() {
// Pretty lazy implementation
return (int) this.timestamp;
}
}

View File

@@ -48,6 +48,7 @@ public class UpdateNameTransactionData extends TransactionData {
public void afterUnmarshal(Unmarshaller u, Object parent) {
this.creatorPublicKey = this.ownerPublicKey;
this.reducedNewName = this.newName != null ? Unicode.sanitize(this.newName) : null;
}
/** From repository */
@@ -62,7 +63,7 @@ public class UpdateNameTransactionData extends TransactionData {
this.nameReference = nameReference;
}
/** From network/API */
/** From network */
public UpdateNameTransactionData(BaseTransactionData baseTransactionData, String name, String newName, String newData) {
this(baseTransactionData, name, newName, newData, Unicode.sanitize(newName), null);
}

View File

@@ -4,6 +4,7 @@ import java.awt.GraphicsEnvironment;
import java.awt.image.BufferedImage;
import java.io.IOException;
import java.io.InputStream;
import java.util.ServiceConfigurationError;
import javax.imageio.ImageIO;
import javax.swing.JOptionPane;
@@ -49,7 +50,7 @@ public class Gui {
protected static BufferedImage loadImage(String resourceName) {
try (InputStream in = Gui.class.getResourceAsStream("/images/" + resourceName)) {
return ImageIO.read(in);
} catch (IllegalArgumentException | IOException e) {
} catch (IllegalArgumentException | IOException | ServiceConfigurationError e) {
LOGGER.warn(String.format("Couldn't locate image resource \"images/%s\"", resourceName));
return null;
}

View File

@@ -23,6 +23,7 @@ import java.util.List;
import javax.swing.JDialog;
import javax.swing.JMenuItem;
import javax.swing.JOptionPane;
import javax.swing.JPopupMenu;
import javax.swing.SwingWorker;
import javax.swing.event.PopupMenuEvent;
@@ -178,6 +179,14 @@ public class SysTray {
menu.add(syncTime);
}
JMenuItem about = new JMenuItem(Translator.INSTANCE.translate("SysTray", "BUILD_VERSION"));
about.addActionListener(actionEvent -> {
destroyHiddenDialog();
JOptionPane.showMessageDialog(null,"Qortal Core\n" + Translator.INSTANCE.translate("SysTray", "BUILD_VERSION") + ":\n" + Controller.getInstance().getVersionStringWithoutPrefix(),"Qortal Core",1);
});
menu.add(about);
JMenuItem exit = new JMenuItem(Translator.INSTANCE.translate("SysTray", "EXIT"));
exit.addActionListener(actionEvent -> {
destroyHiddenDialog();

View File

@@ -195,7 +195,7 @@ public class Name {
this.repository.getNameRepository().save(this.nameData);
}
public void buy(BuyNameTransactionData buyNameTransactionData) throws DataException {
public void buy(BuyNameTransactionData buyNameTransactionData, boolean modifyBalances) throws DataException {
// Save previous name-changing reference in this transaction's data
// Caller is expected to save
buyNameTransactionData.setNameReference(this.nameData.getReference());
@@ -203,15 +203,20 @@ public class Name {
// Mark not for-sale but leave price in case we want to orphan
this.nameData.setIsForSale(false);
// Update seller's balance
Account seller = new Account(this.repository, this.nameData.getOwner());
seller.modifyAssetBalance(Asset.QORT, buyNameTransactionData.getAmount());
if (modifyBalances) {
// Update seller's balance
Account seller = new Account(this.repository, this.nameData.getOwner());
seller.modifyAssetBalance(Asset.QORT, buyNameTransactionData.getAmount());
}
// Set new owner
Account buyer = new PublicKeyAccount(this.repository, buyNameTransactionData.getBuyerPublicKey());
this.nameData.setOwner(buyer.getAddress());
// Update buyer's balance
buyer.modifyAssetBalance(Asset.QORT, - buyNameTransactionData.getAmount());
if (modifyBalances) {
// Update buyer's balance
buyer.modifyAssetBalance(Asset.QORT, -buyNameTransactionData.getAmount());
}
// Set name-changing reference to this transaction
this.nameData.setReference(buyNameTransactionData.getSignature());

View File

@@ -13,7 +13,7 @@ import org.qortal.crypto.MemoryPoW;
import org.qortal.network.message.ChallengeMessage;
import org.qortal.network.message.HelloMessage;
import org.qortal.network.message.Message;
import org.qortal.network.message.Message.MessageType;
import org.qortal.network.message.MessageType;
import org.qortal.settings.Settings;
import org.qortal.network.message.ResponseMessage;
import org.qortal.utils.DaemonThreadFactory;
@@ -74,6 +74,12 @@ public enum Handshake {
peer.setPeersConnectionTimestamp(peersConnectionTimestamp);
peer.setPeersVersion(versionString, version);
// Ensure the peer is running at least the version specified in MIN_PEER_VERSION
if (peer.isAtLeastVersion(MIN_PEER_VERSION) == false) {
LOGGER.debug(String.format("Ignoring peer %s because it is on an old version (%s)", peer, versionString));
return null;
}
if (Settings.getInstance().getAllowConnectionsWithOlderPeerVersions() == false) {
// Ensure the peer is running at least the minimum version allowed for connections
final String minPeerVersion = Settings.getInstance().getMinPeerVersion();
@@ -258,6 +264,9 @@ public enum Handshake {
private static final long PEER_VERSION_131 = 0x0100030001L;
/** Minimum peer version that we are allowed to communicate with */
private static final String MIN_PEER_VERSION = "3.1.0";
private static final int POW_BUFFER_SIZE_PRE_131 = 8 * 1024 * 1024; // bytes
private static final int POW_DIFFICULTY_PRE_131 = 8; // leading zero bits
// Can always be made harder in the future...

View File

@@ -1,5 +1,6 @@
package org.qortal.network;
import com.dosse.upnp.UPnP;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.bouncycastle.crypto.params.Ed25519PrivateKeyParameters;
@@ -13,6 +14,7 @@ import org.qortal.data.block.BlockData;
import org.qortal.data.network.PeerData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.network.message.*;
import org.qortal.network.task.*;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@@ -32,6 +34,7 @@ import java.nio.channels.*;
import java.security.SecureRandom;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Function;
@@ -41,9 +44,8 @@ import java.util.stream.Collectors;
// For managing peers
public class Network {
private static final Logger LOGGER = LogManager.getLogger(Network.class);
private static Network instance;
private static final int LISTEN_BACKLOG = 10;
private static final int LISTEN_BACKLOG = 5;
/**
* How long before retrying after a connection failure, in milliseconds.
*/
@@ -100,25 +102,38 @@ public class Network {
private long nextDisconnectionCheck = 0L;
private final List<PeerData> allKnownPeers = new ArrayList<>();
private final List<Peer> connectedPeers = new ArrayList<>();
/**
* Maintain two lists for each subset of peers:
* - A synchronizedList, to be modified when peers are added/removed
* - An immutable List, which is rebuilt automatically to mirror the synchronized list, and is then served to consumers
* This allows for thread safety without having to synchronize every time a thread requests a peer list
*/
private final List<Peer> connectedPeers = Collections.synchronizedList(new ArrayList<>());
private List<Peer> immutableConnectedPeers = Collections.emptyList(); // always rebuilt from mutable, synced list above
private final List<Peer> handshakedPeers = Collections.synchronizedList(new ArrayList<>());
private List<Peer> immutableHandshakedPeers = Collections.emptyList(); // always rebuilt from mutable, synced list above
private final List<Peer> outboundHandshakedPeers = Collections.synchronizedList(new ArrayList<>());
private List<Peer> immutableOutboundHandshakedPeers = Collections.emptyList(); // always rebuilt from mutable, synced list above
private final List<PeerAddress> selfPeers = new ArrayList<>();
private final ExecuteProduceConsume networkEPC;
private Selector channelSelector;
private ServerSocketChannel serverChannel;
private Iterator<SelectionKey> channelIterator = null;
// volatile because value is updated inside any one of the EPC threads
private volatile long nextConnectTaskTimestamp = 0L; // ms - try first connect once NTP syncs
private final ExecutorService broadcastExecutor = Executors.newCachedThreadPool();
// volatile because value is updated inside any one of the EPC threads
private volatile long nextBroadcastTimestamp = 0L; // ms - try first broadcast once NTP syncs
private SelectionKey serverSelectionKey;
private final Set<SelectableChannel> channelsPendingWrite = ConcurrentHashMap.newKeySet();
private final Lock mergePeersLock = new ReentrantLock();
private List<String> ourExternalIpAddressHistory = new ArrayList<>();
private String ourExternalIpAddress = null;
private int ourExternalPort = Settings.getInstance().getListenPort();
private volatile boolean isShuttingDown = false;
// Constructors
@@ -153,7 +168,7 @@ public class Network {
serverChannel.configureBlocking(false);
serverChannel.setOption(StandardSocketOptions.SO_REUSEADDR, true);
serverChannel.bind(endpoint, LISTEN_BACKLOG);
serverChannel.register(channelSelector, SelectionKey.OP_ACCEPT);
serverSelectionKey = serverChannel.register(channelSelector, SelectionKey.OP_ACCEPT);
} catch (UnknownHostException e) {
LOGGER.error("Can't bind listen socket to address {}", Settings.getInstance().getBindAddress());
throw new IOException("Can't bind listen socket to address", e);
@@ -163,7 +178,8 @@ public class Network {
}
// Load all known peers from repository
synchronized (this.allKnownPeers) { List<String> fixedNetwork = Settings.getInstance().getFixedNetwork();
synchronized (this.allKnownPeers) {
List<String> fixedNetwork = Settings.getInstance().getFixedNetwork();
if (fixedNetwork != null && !fixedNetwork.isEmpty()) {
Long addedWhen = NTP.getTime();
String addedBy = "fixedNetwork";
@@ -183,18 +199,30 @@ public class Network {
}
}
// Attempt to set up UPnP. All errors are ignored.
if (Settings.getInstance().isUPnPEnabled()) {
UPnP.openPortTCP(Settings.getInstance().getListenPort());
}
else {
UPnP.closePortTCP(Settings.getInstance().getListenPort());
}
// Start up first networking thread
networkEPC.start();
}
// Getters / setters
public static synchronized Network getInstance() {
if (instance == null) {
instance = new Network();
}
private static class SingletonContainer {
private static final Network INSTANCE = new Network();
}
return instance;
public static Network getInstance() {
return SingletonContainer.INSTANCE;
}
public int getMaxPeers() {
return this.maxPeers;
}
public byte[] getMessageMagic() {
@@ -228,10 +256,33 @@ public class Network {
}
}
public List<Peer> getConnectedPeers() {
synchronized (this.connectedPeers) {
return new ArrayList<>(this.connectedPeers);
}
public List<Peer> getImmutableConnectedPeers() {
return this.immutableConnectedPeers;
}
public List<Peer> getImmutableConnectedDataPeers() {
return this.getImmutableConnectedPeers().stream()
.filter(p -> p.isDataPeer())
.collect(Collectors.toList());
}
public List<Peer> getImmutableConnectedNonDataPeers() {
return this.getImmutableConnectedPeers().stream()
.filter(p -> !p.isDataPeer())
.collect(Collectors.toList());
}
public void addConnectedPeer(Peer peer) {
this.connectedPeers.add(peer); // thread safe thanks to synchronized list
this.immutableConnectedPeers = List.copyOf(this.connectedPeers); // also thread safe thanks to synchronized collection's toArray() being fed to List.of(array)
}
public void removeConnectedPeer(Peer peer) {
// Firstly remove from handshaked peers
this.removeHandshakedPeer(peer);
this.connectedPeers.remove(peer); // thread safe thanks to synchronized list
this.immutableConnectedPeers = List.copyOf(this.connectedPeers); // also thread safe thanks to synchronized collection's toArray() being fed to List.of(array)
}
public List<PeerAddress> getSelfPeers() {
@@ -243,12 +294,15 @@ public class Network {
public boolean requestDataFromPeer(String peerAddressString, byte[] signature) {
if (peerAddressString != null) {
PeerAddress peerAddress = PeerAddress.fromString(peerAddressString);
PeerData peerData = null;
// Reuse an existing PeerData instance if it's already in the known peers list
PeerData peerData = this.allKnownPeers.stream()
.filter(knownPeerData -> knownPeerData.getAddress().equals(peerAddress))
.findFirst()
.orElse(null);
synchronized (this.allKnownPeers) {
peerData = this.allKnownPeers.stream()
.filter(knownPeerData -> knownPeerData.getAddress().equals(peerAddress))
.findFirst()
.orElse(null);
}
if (peerData == null) {
// Not a known peer, so we need to create one
@@ -263,13 +317,14 @@ public class Network {
}
// Check if we're already connected to and handshaked with this peer
Peer connectedPeer = this.connectedPeers.stream()
.filter(p -> p.getPeerData().getAddress().equals(peerAddress))
.findFirst()
.orElse(null);
Peer connectedPeer = this.getImmutableConnectedPeers().stream()
.filter(p -> p.getPeerData().getAddress().equals(peerAddress))
.findFirst()
.orElse(null);
boolean isConnected = (connectedPeer != null);
boolean isHandshaked = this.getHandshakedPeers().stream()
boolean isHandshaked = this.getImmutableHandshakedPeers().stream()
.anyMatch(p -> p.getPeerData().getAddress().equals(peerAddress));
if (isConnected && isHandshaked) {
@@ -283,6 +338,7 @@ public class Network {
// Add this signature to the list of pending requests for this peer
LOGGER.info("Making connection to peer {} to request files for signature {}...", peerAddressString, Base58.encode(signature));
Peer peer = new Peer(peerData);
peer.setIsDataPeer(true);
peer.addPendingSignatureRequest(signature);
return this.connectPeer(peer);
// If connection (and handshake) is successful, data will automatically be requested
@@ -313,35 +369,61 @@ public class Network {
/**
* Returns list of connected peers that have completed handshaking.
*/
public List<Peer> getHandshakedPeers() {
synchronized (this.connectedPeers) {
return this.connectedPeers.stream()
.filter(peer -> peer.getHandshakeStatus() == Handshake.COMPLETED)
.collect(Collectors.toList());
public List<Peer> getImmutableHandshakedPeers() {
return this.immutableHandshakedPeers;
}
public void addHandshakedPeer(Peer peer) {
this.handshakedPeers.add(peer); // thread safe thanks to synchronized list
this.immutableHandshakedPeers = List.copyOf(this.handshakedPeers); // also thread safe thanks to synchronized collection's toArray() being fed to List.of(array)
// Also add to outbound handshaked peers cache
if (peer.isOutbound()) {
this.addOutboundHandshakedPeer(peer);
}
}
public void removeHandshakedPeer(Peer peer) {
this.handshakedPeers.remove(peer); // thread safe thanks to synchronized list
this.immutableHandshakedPeers = List.copyOf(this.handshakedPeers); // also thread safe thanks to synchronized collection's toArray() being fed to List.of(array)
// Also remove from outbound handshaked peers cache
if (peer.isOutbound()) {
this.removeOutboundHandshakedPeer(peer);
}
}
/**
* Returns list of peers we connected to that have completed handshaking.
*/
public List<Peer> getOutboundHandshakedPeers() {
synchronized (this.connectedPeers) {
return this.connectedPeers.stream()
.filter(peer -> peer.isOutbound() && peer.getHandshakeStatus() == Handshake.COMPLETED)
.collect(Collectors.toList());
public List<Peer> getImmutableOutboundHandshakedPeers() {
return this.immutableOutboundHandshakedPeers;
}
public void addOutboundHandshakedPeer(Peer peer) {
if (!peer.isOutbound()) {
return;
}
this.outboundHandshakedPeers.add(peer); // thread safe thanks to synchronized list
this.immutableOutboundHandshakedPeers = List.copyOf(this.outboundHandshakedPeers); // also thread safe thanks to synchronized collection's toArray() being fed to List.of(array)
}
public void removeOutboundHandshakedPeer(Peer peer) {
if (!peer.isOutbound()) {
return;
}
this.outboundHandshakedPeers.remove(peer); // thread safe thanks to synchronized list
this.immutableOutboundHandshakedPeers = List.copyOf(this.outboundHandshakedPeers); // also thread safe thanks to synchronized collection's toArray() being fed to List.of(array)
}
/**
* Returns first peer that has completed handshaking and has matching public key.
*/
public Peer getHandshakedPeerWithPublicKey(byte[] publicKey) {
synchronized (this.connectedPeers) {
return this.connectedPeers.stream()
.filter(peer -> peer.getHandshakeStatus() == Handshake.COMPLETED
&& Arrays.equals(peer.getPeersPublicKey(), publicKey))
.findFirst().orElse(null);
}
return this.getImmutableConnectedPeers().stream()
.filter(peer -> peer.getHandshakeStatus() == Handshake.COMPLETED
&& Arrays.equals(peer.getPeersPublicKey(), publicKey))
.findFirst().orElse(null);
}
// Peer list filters
@@ -354,21 +436,15 @@ public class Network {
return this.selfPeers.stream().anyMatch(selfPeer -> selfPeer.equals(peerAddress));
};
/**
* Must be inside <tt>synchronized (this.connectedPeers) {...}</tt>
*/
private final Predicate<PeerData> isConnectedPeer = peerData -> {
PeerAddress peerAddress = peerData.getAddress();
return this.connectedPeers.stream().anyMatch(peer -> peer.getPeerData().getAddress().equals(peerAddress));
return this.getImmutableConnectedPeers().stream().anyMatch(peer -> peer.getPeerData().getAddress().equals(peerAddress));
};
/**
* Must be inside <tt>synchronized (this.connectedPeers) {...}</tt>
*/
private final Predicate<PeerData> isResolvedAsConnectedPeer = peerData -> {
try {
InetSocketAddress resolvedSocketAddress = peerData.getAddress().toSocketAddress();
return this.connectedPeers.stream()
return this.getImmutableConnectedPeers().stream()
.anyMatch(peer -> peer.getResolvedAddress().equals(resolvedSocketAddress));
} catch (UnknownHostException e) {
// Can't resolve - no point even trying to connect
@@ -393,6 +469,11 @@ public class Network {
class NetworkProcessor extends ExecuteProduceConsume {
private final AtomicLong nextConnectTaskTimestamp = new AtomicLong(0L); // ms - try first connect once NTP syncs
private final AtomicLong nextBroadcastTimestamp = new AtomicLong(0L); // ms - try first broadcast once NTP syncs
private Iterator<SelectionKey> channelIterator = null;
NetworkProcessor(ExecutorService executor) {
super(executor);
}
@@ -434,51 +515,31 @@ public class Network {
}
private Task maybeProducePeerMessageTask() {
for (Peer peer : getConnectedPeers()) {
Task peerTask = peer.getMessageTask();
if (peerTask != null) {
return peerTask;
}
}
return null;
return getImmutableConnectedPeers().stream()
.map(Peer::getMessageTask)
.filter(Objects::nonNull)
.findFirst()
.orElse(null);
}
private Task maybeProducePeerPingTask(Long now) {
// Ask connected peers whether they need a ping
for (Peer peer : getHandshakedPeers()) {
Task peerTask = peer.getPingTask(now);
if (peerTask != null) {
return peerTask;
}
}
return null;
}
class PeerConnectTask implements ExecuteProduceConsume.Task {
private final Peer peer;
PeerConnectTask(Peer peer) {
this.peer = peer;
}
@Override
public void perform() throws InterruptedException {
connectPeer(peer);
}
return getImmutableHandshakedPeers().stream()
.map(peer -> peer.getPingTask(now))
.filter(Objects::nonNull)
.findFirst()
.orElse(null);
}
private Task maybeProduceConnectPeerTask(Long now) throws InterruptedException {
if (now == null || now < nextConnectTaskTimestamp) {
if (now == null || now < nextConnectTaskTimestamp.get()) {
return null;
}
if (getOutboundHandshakedPeers().size() >= minOutboundPeers) {
if (getImmutableOutboundHandshakedPeers().size() >= minOutboundPeers) {
return null;
}
nextConnectTaskTimestamp = now + 1000L;
nextConnectTaskTimestamp.set(now + 1000L);
Peer targetPeer = getConnectablePeer(now);
if (targetPeer == null) {
@@ -490,66 +551,15 @@ public class Network {
}
private Task maybeProduceBroadcastTask(Long now) {
if (now == null || now < nextBroadcastTimestamp) {
if (now == null || now < nextBroadcastTimestamp.get()) {
return null;
}
nextBroadcastTimestamp = now + BROADCAST_INTERVAL;
return () -> Controller.getInstance().doNetworkBroadcast();
}
class ChannelTask implements ExecuteProduceConsume.Task {
private final SelectionKey selectionKey;
ChannelTask(SelectionKey selectionKey) {
this.selectionKey = selectionKey;
}
@Override
public void perform() throws InterruptedException {
try {
LOGGER.trace("Thread {} has pending channel: {}, with ops {}",
Thread.currentThread().getId(), selectionKey.channel(), selectionKey.readyOps());
// process pending channel task
if (selectionKey.isReadable()) {
connectionRead((SocketChannel) selectionKey.channel());
} else if (selectionKey.isAcceptable()) {
acceptConnection((ServerSocketChannel) selectionKey.channel());
}
LOGGER.trace("Thread {} processed channel: {}",
Thread.currentThread().getId(), selectionKey.channel());
} catch (CancelledKeyException e) {
LOGGER.trace("Thread {} encountered cancelled channel: {}",
Thread.currentThread().getId(), selectionKey.channel());
}
}
private void connectionRead(SocketChannel socketChannel) {
Peer peer = getPeerFromChannel(socketChannel);
if (peer == null) {
return;
}
try {
peer.readChannel();
} catch (IOException e) {
if (e.getMessage() != null && e.getMessage().toLowerCase().contains("connection reset")) {
peer.disconnect("Connection reset");
return;
}
LOGGER.trace("[{}] Network thread {} encountered I/O error: {}", peer.getPeerConnectionId(),
Thread.currentThread().getId(), e.getMessage(), e);
peer.disconnect("I/O error");
}
}
nextBroadcastTimestamp.set(now + BROADCAST_INTERVAL);
return new BroadcastTask();
}
private Task maybeProduceChannelTask(boolean canBlock) throws InterruptedException {
final SelectionKey nextSelectionKey;
// Synchronization here to enforce thread-safety on channelIterator
synchronized (channelSelector) {
// anything to do?
@@ -570,92 +580,73 @@ public class Network {
}
channelIterator = channelSelector.selectedKeys().iterator();
LOGGER.trace("Thread {}, after {} select, channelIterator now {}",
Thread.currentThread().getId(),
canBlock ? "blocking": "non-blocking",
channelIterator);
}
if (channelIterator.hasNext()) {
nextSelectionKey = channelIterator.next();
channelIterator.remove();
} else {
nextSelectionKey = null;
if (!channelIterator.hasNext()) {
channelIterator = null; // Nothing to do so reset iterator to cause new select
LOGGER.trace("Thread {}, channelIterator now null", Thread.currentThread().getId());
return null;
}
LOGGER.trace("Thread {}, nextSelectionKey {}, channelIterator now {}",
Thread.currentThread().getId(), nextSelectionKey, channelIterator);
}
final SelectionKey nextSelectionKey = channelIterator.next();
channelIterator.remove();
if (nextSelectionKey == null) {
return null;
}
// Just in case underlying socket channel already closed elsewhere, etc.
if (!nextSelectionKey.isValid())
return null;
return new ChannelTask(nextSelectionKey);
}
}
LOGGER.trace("Thread {}, nextSelectionKey {}", Thread.currentThread().getId(), nextSelectionKey);
private void acceptConnection(ServerSocketChannel serverSocketChannel) throws InterruptedException {
SocketChannel socketChannel;
SelectableChannel socketChannel = nextSelectionKey.channel();
try {
socketChannel = serverSocketChannel.accept();
} catch (IOException e) {
return;
}
// No connection actually accepted?
if (socketChannel == null) {
return;
}
PeerAddress address = PeerAddress.fromSocket(socketChannel.socket());
List<String> fixedNetwork = Settings.getInstance().getFixedNetwork();
if (fixedNetwork != null && !fixedNetwork.isEmpty() && ipNotInFixedList(address, fixedNetwork)) {
try {
LOGGER.debug("Connection discarded from peer {} as not in the fixed network list", address);
socketChannel.close();
} catch (IOException e) {
// IGNORE
}
return;
}
final Long now = NTP.getTime();
Peer newPeer;
try {
if (now == null) {
LOGGER.debug("Connection discarded from peer {} due to lack of NTP sync", address);
socketChannel.close();
return;
}
synchronized (this.connectedPeers) {
if (connectedPeers.size() >= maxPeers) {
// We have enough peers
LOGGER.debug("Connection discarded from peer {} because the server is full", address);
socketChannel.close();
return;
}
LOGGER.debug("Connection accepted from peer {}", address);
newPeer = new Peer(socketChannel, channelSelector);
this.connectedPeers.add(newPeer);
}
} catch (IOException e) {
if (socketChannel.isOpen()) {
try {
LOGGER.debug("Connection failed from peer {} while connecting/closing", address);
socketChannel.close();
} catch (IOException ce) {
// Couldn't close?
if (nextSelectionKey.isReadable()) {
clearInterestOps(nextSelectionKey, SelectionKey.OP_READ);
Peer peer = getPeerFromChannel((SocketChannel) socketChannel);
if (peer == null)
return null;
return new ChannelReadTask((SocketChannel) socketChannel, peer);
}
if (nextSelectionKey.isWritable()) {
clearInterestOps(nextSelectionKey, SelectionKey.OP_WRITE);
Peer peer = getPeerFromChannel((SocketChannel) socketChannel);
if (peer == null)
return null;
// Any thread that queues a message to send can set OP_WRITE,
// but we only allow one pending/active ChannelWriteTask per Peer
if (!channelsPendingWrite.add(socketChannel))
return null;
return new ChannelWriteTask((SocketChannel) socketChannel, peer);
}
if (nextSelectionKey.isAcceptable()) {
clearInterestOps(nextSelectionKey, SelectionKey.OP_ACCEPT);
return new ChannelAcceptTask((ServerSocketChannel) socketChannel);
}
} catch (CancelledKeyException e) {
/*
* Sometimes nextSelectionKey is cancelled / becomes invalid between the isValid() test at line 586
* and later calls to isReadable() / isWritable() / isAcceptable() which themselves call isValid()!
* Those isXXXable() calls could throw CancelledKeyException, so we catch it here and return null.
*/
return null;
}
}
return;
}
this.onPeerReady(newPeer);
return null;
}
}
private boolean ipNotInFixedList(PeerAddress address, List<String> fixedNetwork) {
public boolean ipNotInFixedList(PeerAddress address, List<String> fixedNetwork) {
for (String ipAddress : fixedNetwork) {
String[] bits = ipAddress.split(":");
if (bits.length >= 1 && bits.length <= 2 && address.getHost().equals(bits[0])) {
@@ -687,16 +678,15 @@ public class Network {
peers.removeIf(isSelfPeer);
}
synchronized (this.connectedPeers) {
// Don't consider already connected peers (simple address match)
peers.removeIf(isConnectedPeer);
// Don't consider already connected peers (simple address match)
peers.removeIf(isConnectedPeer);
// Don't consider already connected peers (resolved address match)
// XXX This might be too slow if we end up waiting a long time for hostnames to resolve via DNS
peers.removeIf(isResolvedAsConnectedPeer);
// Don't consider already connected peers (resolved address match)
// Disabled because this might be too slow if we end up waiting a long time for hostnames to resolve via DNS
// Which is ok because duplicate connections to the same peer are handled during handshaking
// peers.removeIf(isResolvedAsConnectedPeer);
this.checkLongestConnection(now);
}
this.checkLongestConnection(now);
// Any left?
if (peers.isEmpty()) {
@@ -709,6 +699,7 @@ public class Network {
// Pick candidate
PeerData peerData = peers.get(peerIndex);
Peer newPeer = new Peer(peerData);
newPeer.setIsDataPeer(false);
// Update connection attempt info
peerData.setLastAttempted(now);
@@ -724,8 +715,12 @@ public class Network {
}
}
private boolean connectPeer(Peer newPeer) throws InterruptedException {
SocketChannel socketChannel = newPeer.connect(this.channelSelector);
public boolean connectPeer(Peer newPeer) throws InterruptedException {
// Also checked before creating PeerConnectTask
if (getImmutableOutboundHandshakedPeers().size() >= minOutboundPeers)
return false;
SocketChannel socketChannel = newPeer.connect();
if (socketChannel == null) {
return false;
}
@@ -734,21 +729,16 @@ public class Network {
return false;
}
synchronized (this.connectedPeers) {
this.connectedPeers.add(newPeer);
}
this.addConnectedPeer(newPeer);
this.onPeerReady(newPeer);
return true;
}
private Peer getPeerFromChannel(SocketChannel socketChannel) {
synchronized (this.connectedPeers) {
for (Peer peer : this.connectedPeers) {
if (peer.getSocketChannel() == socketChannel) {
return peer;
}
public Peer getPeerFromChannel(SocketChannel socketChannel) {
for (Peer peer : this.getImmutableConnectedPeers()) {
if (peer.getSocketChannel() == socketChannel) {
return peer;
}
}
@@ -761,7 +751,7 @@ public class Network {
}
// Find peers that have reached their maximum connection age, and disconnect them
List<Peer> peersToDisconnect = this.connectedPeers.stream()
List<Peer> peersToDisconnect = this.getImmutableConnectedPeers().stream()
.filter(peer -> !peer.isSyncInProgress())
.filter(peer -> peer.hasReachedMaxConnectionAge())
.collect(Collectors.toList());
@@ -778,7 +768,74 @@ public class Network {
nextDisconnectionCheck = now + DISCONNECTION_CHECK_INTERVAL;
}
// Peer callbacks
// SocketChannel interest-ops manipulations
private static final String[] OP_NAMES = new String[SelectionKey.OP_ACCEPT * 2];
static {
for (int i = 0; i < OP_NAMES.length; i++) {
StringJoiner joiner = new StringJoiner(",");
if ((i & SelectionKey.OP_READ) != 0) joiner.add("OP_READ");
if ((i & SelectionKey.OP_WRITE) != 0) joiner.add("OP_WRITE");
if ((i & SelectionKey.OP_CONNECT) != 0) joiner.add("OP_CONNECT");
if ((i & SelectionKey.OP_ACCEPT) != 0) joiner.add("OP_ACCEPT");
OP_NAMES[i] = joiner.toString();
}
}
public void clearInterestOps(SelectableChannel socketChannel, int interestOps) {
SelectionKey selectionKey = socketChannel.keyFor(channelSelector);
if (selectionKey == null)
return;
clearInterestOps(selectionKey, interestOps);
}
private void clearInterestOps(SelectionKey selectionKey, int interestOps) {
if (!selectionKey.channel().isOpen())
return;
LOGGER.trace("Thread {} clearing {} interest-ops on channel: {}",
Thread.currentThread().getId(),
OP_NAMES[interestOps],
selectionKey.channel());
selectionKey.interestOpsAnd(~interestOps);
}
public void setInterestOps(SelectableChannel socketChannel, int interestOps) {
SelectionKey selectionKey = socketChannel.keyFor(channelSelector);
if (selectionKey == null) {
try {
selectionKey = socketChannel.register(this.channelSelector, interestOps);
} catch (ClosedChannelException e) {
// Channel already closed so ignore
return;
}
// Fall-through to allow logging
}
setInterestOps(selectionKey, interestOps);
}
private void setInterestOps(SelectionKey selectionKey, int interestOps) {
if (!selectionKey.channel().isOpen())
return;
LOGGER.trace("Thread {} setting {} interest-ops on channel: {}",
Thread.currentThread().getId(),
OP_NAMES[interestOps],
selectionKey.channel());
selectionKey.interestOpsOr(interestOps);
}
// Peer / Task callbacks
public void notifyChannelNotWriting(SelectableChannel socketChannel) {
this.channelsPendingWrite.remove(socketChannel);
}
protected void wakeupChannelSelector() {
this.channelSelector.wakeup();
@@ -804,17 +861,32 @@ public class Network {
}
public void onDisconnect(Peer peer) {
// Notify Controller
Controller.getInstance().onPeerDisconnect(peer);
if (peer.getConnectionEstablishedTime() > 0L) {
LOGGER.debug("[{}] Disconnected from peer {}", peer.getPeerConnectionId(), peer);
} else {
LOGGER.debug("[{}] Failed to connect to peer {}", peer.getPeerConnectionId(), peer);
}
synchronized (this.connectedPeers) {
this.connectedPeers.remove(peer);
this.removeConnectedPeer(peer);
this.channelsPendingWrite.remove(peer.getSocketChannel());
if (this.isShuttingDown)
// No need to do any further processing, like re-enabling listen socket or notifying Controller
return;
if (getImmutableConnectedPeers().size() < maxPeers - 1
&& serverSelectionKey.isValid()
&& (serverSelectionKey.interestOps() & SelectionKey.OP_ACCEPT) == 0) {
try {
LOGGER.debug("Re-enabling accepting incoming connections because the server is not longer full");
setInterestOps(serverSelectionKey, SelectionKey.OP_ACCEPT);
} catch (CancelledKeyException e) {
LOGGER.error("Failed to re-enable accepting of incoming connections: {}", e.getMessage());
}
}
// Notify Controller
Controller.getInstance().onPeerDisconnect(peer);
}
public void peerMisbehaved(Peer peer) {
@@ -975,6 +1047,9 @@ public class Network {
return;
}
// Add to handshaked peers cache
this.addHandshakedPeer(peer);
// Make a note that we've successfully completed handshake (and when)
peer.getPeerData().setLastConnected(NTP.getTime());
@@ -1009,11 +1084,13 @@ public class Network {
// (If inbound sent anything here, it's possible it could be processed out-of-order with handshake message).
if (peer.isOutbound()) {
// Send our height
Message heightMessage = buildHeightMessage(peer, Controller.getInstance().getChainTip());
if (!peer.sendMessage(heightMessage)) {
peer.disconnect("failed to send height/info");
return;
if (!Settings.getInstance().isLite()) {
// Send our height
Message heightMessage = buildHeightMessage(peer, Controller.getInstance().getChainTip());
if (!peer.sendMessage(heightMessage)) {
peer.disconnect("failed to send height/info");
return;
}
}
// Send our peers list
@@ -1114,6 +1191,7 @@ public class Network {
return;
}
String host = parts[0];
try {
InetAddress addr = InetAddress.getByName(host);
if (addr.isAnyLocalAddress() || addr.isSiteLocalAddress()) {
@@ -1124,6 +1202,9 @@ public class Network {
return;
}
// Keep track of the port
this.ourExternalPort = Integer.parseInt(parts[1]);
// Add to the list
this.ourExternalIpAddressHistory.add(host);
@@ -1177,8 +1258,19 @@ public class Network {
public void onExternalIpUpdate(String ipAddress) {
LOGGER.info("External IP address updated to {}", ipAddress);
}
//ArbitraryDataManager.getInstance().broadcastHostedSignatureList();
public String getOurExternalIpAddress() {
// FUTURE: replace port if UPnP is active, as it will be more accurate
return this.ourExternalIpAddress;
}
public String getOurExternalIpAddressAndPort() {
String ipAddress = this.getOurExternalIpAddress();
if (ipAddress == null) {
return null;
}
return String.format("%s:%d", ipAddress, this.ourExternalPort);
}
@@ -1222,7 +1314,7 @@ public class Network {
}
}
for (Peer peer : this.getConnectedPeers()) {
for (Peer peer : this.getImmutableConnectedPeers()) {
peer.disconnect("to be forgotten");
}
@@ -1234,8 +1326,9 @@ public class Network {
try {
InetSocketAddress knownAddress = peerAddress.toSocketAddress();
List<Peer> peers = this.getConnectedPeers();
peers.removeIf(peer -> !Peer.addressEquals(knownAddress, peer.getResolvedAddress()));
List<Peer> peers = this.getImmutableConnectedPeers().stream()
.filter(peer -> Peer.addressEquals(knownAddress, peer.getResolvedAddress()))
.collect(Collectors.toList());
for (Peer peer : peers) {
peer.disconnect("to be forgotten");
@@ -1254,7 +1347,8 @@ public class Network {
}
// Disconnect peers that are stuck during handshake
List<Peer> handshakePeers = this.getConnectedPeers();
// Needs a mutable copy of the unmodifiableList
List<Peer> handshakePeers = new ArrayList<>(this.getImmutableConnectedPeers());
// Disregard peers that have completed handshake or only connected recently
handshakePeers.removeIf(peer -> peer.getHandshakeStatus() == Handshake.COMPLETED
@@ -1296,9 +1390,7 @@ public class Network {
peers.removeIf(isNotOldPeer);
// Don't consider already connected peers (simple address match)
synchronized (this.connectedPeers) {
peers.removeIf(isConnectedPeer);
}
peers.removeIf(isConnectedPeer);
for (PeerData peerData : peers) {
LOGGER.debug("Deleting old peer {} from repository", peerData.getAddress().toString());
@@ -1394,54 +1486,27 @@ public class Network {
}
public void broadcast(Function<Peer, Message> peerMessageBuilder) {
class Broadcaster implements Runnable {
private final Random random = new Random();
for (Peer peer : getImmutableHandshakedPeers()) {
if (this.isShuttingDown)
return;
private List<Peer> targetPeers;
private Function<Peer, Message> peerMessageBuilder;
Message message = peerMessageBuilder.apply(peer);
Broadcaster(List<Peer> targetPeers, Function<Peer, Message> peerMessageBuilder) {
this.targetPeers = targetPeers;
this.peerMessageBuilder = peerMessageBuilder;
if (message == null) {
continue;
}
@Override
public void run() {
Thread.currentThread().setName("Network Broadcast");
for (Peer peer : targetPeers) {
// Very short sleep to reduce strain, improve multi-threading and catch interrupts
try {
Thread.sleep(random.nextInt(20) + 20L);
} catch (InterruptedException e) {
break;
}
Message message = peerMessageBuilder.apply(peer);
if (message == null) {
continue;
}
if (!peer.sendMessage(message)) {
peer.disconnect("failed to broadcast message");
}
}
Thread.currentThread().setName("Network Broadcast (dormant)");
if (!peer.sendMessage(message)) {
peer.disconnect("failed to broadcast message");
}
}
try {
broadcastExecutor.execute(new Broadcaster(this.getHandshakedPeers(), peerMessageBuilder));
} catch (RejectedExecutionException e) {
// Can't execute - probably because we're shutting down, so ignore
}
}
// Shutdown
public void shutdown() {
this.isShuttingDown = true;
// Close listen socket to prevent more incoming connections
if (this.serverChannel.isOpen()) {
try {
@@ -1460,18 +1525,8 @@ public class Network {
LOGGER.warn("Interrupted while waiting for networking threads to terminate");
}
// Stop broadcasts
this.broadcastExecutor.shutdownNow();
try {
if (!this.broadcastExecutor.awaitTermination(1000, TimeUnit.MILLISECONDS)) {
LOGGER.warn("Broadcast threads failed to terminate");
}
} catch (InterruptedException e) {
LOGGER.warn("Interrupted while waiting for broadcast threads failed to terminate");
}
// Close all peer connections
for (Peer peer : this.getConnectedPeers()) {
for (Peer peer : this.getImmutableConnectedPeers()) {
peer.shutdown();
}
}

View File

@@ -11,25 +11,22 @@ import org.qortal.data.network.PeerChainTipData;
import org.qortal.data.network.PeerData;
import org.qortal.network.message.ChallengeMessage;
import org.qortal.network.message.Message;
import org.qortal.network.message.Message.MessageException;
import org.qortal.network.message.Message.MessageType;
import org.qortal.network.message.PingMessage;
import org.qortal.network.message.MessageException;
import org.qortal.network.message.MessageType;
import org.qortal.network.task.MessageTask;
import org.qortal.network.task.PingTask;
import org.qortal.settings.Settings;
import org.qortal.utils.ExecuteProduceConsume;
import org.qortal.utils.ExecuteProduceConsume.Task;
import org.qortal.utils.NTP;
import java.io.IOException;
import java.net.*;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.security.SecureRandom;
import java.util.*;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@@ -48,9 +45,9 @@ public class Peer {
private static final int RESPONSE_TIMEOUT = 3000; // ms
/**
* Maximum time to wait for a peer to respond with blocks (ms)
* Maximum time to wait for a message to be added to sendQueue (ms)
*/
public static final int FETCH_BLOCKS_TIMEOUT = 10000;
private static final int QUEUE_TIMEOUT = 1000; // ms
/**
* Interval between PING messages to a peer. (ms)
@@ -68,13 +65,22 @@ public class Peer {
*/
private boolean isLocal;
/**
* True if connected for the purposes of transfering specific QDN data
*/
private boolean isDataPeer;
private final UUID peerConnectionId = UUID.randomUUID();
private final Object byteBufferLock = new Object();
private ByteBuffer byteBuffer;
private Map<Integer, BlockingQueue<Message>> replyQueues;
private LinkedBlockingQueue<Message> pendingMessages;
private TransferQueue<Message> sendQueue;
private ByteBuffer outputBuffer;
private String outputMessageType;
private int outputMessageId;
/**
* True if we created connection to peer, false if we accepted incoming connection from peer.
*/
@@ -98,7 +104,7 @@ public class Peer {
/**
* When last PING message was sent, or null if pings not started yet.
*/
private Long lastPingSent;
private Long lastPingSent = null;
byte[] ourChallenge;
@@ -160,10 +166,10 @@ public class Peer {
/**
* Construct Peer using existing, connected socket
*/
public Peer(SocketChannel socketChannel, Selector channelSelector) throws IOException {
public Peer(SocketChannel socketChannel) throws IOException {
this.isOutbound = false;
this.socketChannel = socketChannel;
sharedSetup(channelSelector);
sharedSetup();
this.resolvedAddress = ((InetSocketAddress) socketChannel.socket().getRemoteSocketAddress());
this.isLocal = isAddressLocal(this.resolvedAddress.getAddress());
@@ -194,6 +200,14 @@ public class Peer {
return this.isOutbound;
}
public boolean isDataPeer() {
return isDataPeer;
}
public void setIsDataPeer(boolean isDataPeer) {
this.isDataPeer = isDataPeer;
}
public Handshake getHandshakeStatus() {
synchronized (this.handshakingLock) {
return this.handshakeStatus;
@@ -211,6 +225,11 @@ public class Peer {
}
private void generateRandomMaxConnectionAge() {
if (this.maxConnectionAge > 0L) {
// Already generated, so we don't want to overwrite the existing value
return;
}
// Retrieve the min and max connection time from the settings, and calculate the range
final int minPeerConnectionTime = Settings.getInstance().getMinPeerConnectionTime();
final int maxPeerConnectionTime = Settings.getInstance().getMaxPeerConnectionTime();
@@ -276,7 +295,7 @@ public class Peer {
}
}
protected void setLastPing(long lastPing) {
public void setLastPing(long lastPing) {
synchronized (this.peerInfoLock) {
this.lastPing = lastPing;
}
@@ -346,12 +365,6 @@ public class Peer {
}
}
protected void queueMessage(Message message) {
if (!this.pendingMessages.offer(message)) {
LOGGER.info("[{}] No room to queue message from peer {} - discarding", this.peerConnectionId, this);
}
}
public boolean isSyncInProgress() {
return this.syncInProgress;
}
@@ -396,13 +409,14 @@ public class Peer {
// Processing
private void sharedSetup(Selector channelSelector) throws IOException {
private void sharedSetup() throws IOException {
this.connectionTimestamp = NTP.getTime();
this.socketChannel.setOption(StandardSocketOptions.TCP_NODELAY, true);
this.socketChannel.configureBlocking(false);
this.socketChannel.register(channelSelector, SelectionKey.OP_READ);
Network.getInstance().setInterestOps(this.socketChannel, SelectionKey.OP_READ);
this.byteBuffer = null; // Defer allocation to when we need it, to save memory. Sorry GC!
this.replyQueues = Collections.synchronizedMap(new HashMap<Integer, BlockingQueue<Message>>());
this.sendQueue = new LinkedTransferQueue<>();
this.replyQueues = new ConcurrentHashMap<>();
this.pendingMessages = new LinkedBlockingQueue<>();
Random random = new SecureRandom();
@@ -410,7 +424,7 @@ public class Peer {
random.nextBytes(this.ourChallenge);
}
public SocketChannel connect(Selector channelSelector) {
public SocketChannel connect() {
LOGGER.trace("[{}] Connecting to peer {}", this.peerConnectionId, this);
try {
@@ -418,6 +432,8 @@ public class Peer {
this.isLocal = isAddressLocal(this.resolvedAddress.getAddress());
this.socketChannel = SocketChannel.open();
InetAddress bindAddr = InetAddress.getByName(Settings.getInstance().getBindAddress());
this.socketChannel.socket().bind(new InetSocketAddress(bindAddr, 0));
this.socketChannel.socket().connect(resolvedAddress, CONNECT_TIMEOUT);
} catch (SocketTimeoutException e) {
LOGGER.trace("[{}] Connection timed out to peer {}", this.peerConnectionId, this);
@@ -432,7 +448,7 @@ public class Peer {
try {
LOGGER.debug("[{}] Connected to peer {}", this.peerConnectionId, this);
sharedSetup(channelSelector);
sharedSetup();
return socketChannel;
} catch (IOException e) {
LOGGER.trace("[{}] Post-connection setup failed, peer {}", this.peerConnectionId, this);
@@ -450,7 +466,7 @@ public class Peer {
*
* @throws IOException If this channel is not yet connected
*/
protected void readChannel() throws IOException {
public void readChannel() throws IOException {
synchronized (this.byteBufferLock) {
while (true) {
if (!this.socketChannel.isOpen() || this.socketChannel.socket().isClosed()) {
@@ -531,6 +547,10 @@ public class Peer {
// adjusting position accordingly, reset limit to capacity
this.byteBuffer.compact();
// Unsupported message type? Discard with no further processing
if (message.getType() == MessageType.UNSUPPORTED)
continue;
BlockingQueue<Message> queue = this.replyQueues.get(message.getId());
if (queue != null) {
// Adding message to queue will unblock thread waiting for response
@@ -556,7 +576,67 @@ public class Peer {
}
}
protected ExecuteProduceConsume.Task getMessageTask() {
/** Maybe send some pending outgoing messages.
*
* @return true if more data is pending to be sent
*/
public boolean writeChannel() throws IOException {
// It is the responsibility of ChannelWriteTask's producer to produce only one call to writeChannel() at a time
while (true) {
// If output byte buffer is null, fetch next message from queue (if any)
while (this.outputBuffer == null) {
Message message;
try {
// Allow other thread time to add message to queue having raised OP_WRITE.
// Timeout is overkill but not excessive enough to clog up networking / EPC.
// This is to avoid race condition in sendMessageWithTimeout() below.
message = this.sendQueue.poll(QUEUE_TIMEOUT, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
// Shutdown situation
return false;
}
// No message? No further work to be done
if (message == null)
return false;
try {
this.outputBuffer = ByteBuffer.wrap(message.toBytes());
this.outputMessageType = message.getType().name();
this.outputMessageId = message.getId();
LOGGER.trace("[{}] Sending {} message with ID {} to peer {}",
this.peerConnectionId, this.outputMessageType, this.outputMessageId, this);
} catch (MessageException e) {
// Something went wrong converting message to bytes, so discard but allow another round
LOGGER.warn("[{}] Failed to send {} message with ID {} to peer {}: {}", this.peerConnectionId,
message.getType().name(), message.getId(), this, e.getMessage());
}
}
// If output byte buffer is not null, send from that
int bytesWritten = this.socketChannel.write(outputBuffer);
LOGGER.trace("[{}] Sent {} bytes of {} message with ID {} to peer {} ({} total)", this.peerConnectionId,
bytesWritten, this.outputMessageType, this.outputMessageId, this, outputBuffer.limit());
// If we've sent 0 bytes then socket buffer is full so we need to wait until it's empty again
if (bytesWritten == 0) {
return true;
}
// If we then exhaust the byte buffer, set it to null (otherwise loop and try to send more)
if (!this.outputBuffer.hasRemaining()) {
this.outputMessageType = null;
this.outputMessageId = 0;
this.outputBuffer = null;
}
}
}
protected Task getMessageTask() {
/*
* If we are still handshaking and there is a message yet to be processed then
* don't produce another message task. This allows us to process handshake
@@ -580,7 +660,7 @@ public class Peer {
}
// Return a task to process message in queue
return () -> Network.getInstance().onMessage(this, nextMessage);
return new MessageTask(this, nextMessage);
}
/**
@@ -605,54 +685,25 @@ public class Peer {
}
try {
// Send message
LOGGER.trace("[{}] Sending {} message with ID {} to peer {}", this.peerConnectionId,
// Queue message, to be picked up by ChannelWriteTask and then peer.writeChannel()
LOGGER.trace("[{}] Queuing {} message with ID {} to peer {}", this.peerConnectionId,
message.getType().name(), message.getId(), this);
ByteBuffer outputBuffer = ByteBuffer.wrap(message.toBytes());
// Check message properly constructed
message.checkValidOutgoing();
synchronized (this.socketChannel) {
final long sendStart = System.currentTimeMillis();
long totalBytes = 0;
while (outputBuffer.hasRemaining()) {
int bytesWritten = this.socketChannel.write(outputBuffer);
totalBytes += bytesWritten;
LOGGER.trace("[{}] Sent {} bytes of {} message with ID {} to peer {} ({} total)", this.peerConnectionId,
bytesWritten, message.getType().name(), message.getId(), this, totalBytes);
if (bytesWritten == 0) {
// Underlying socket's internal buffer probably full,
// so wait a short while for bytes to actually be transmitted over the wire
/*
* NOSONAR squid:S2276 - we don't want to use this.socketChannel.wait()
* as this releases the lock held by synchronized() above
* and would allow another thread to send another message,
* potentially interleaving them on-the-wire, causing checksum failures
* and connection loss.
*/
Thread.sleep(1L); //NOSONAR squid:S2276
if (System.currentTimeMillis() - sendStart > timeout) {
// We've taken too long to send this message
return false;
}
}
}
}
} catch (MessageException e) {
LOGGER.warn("[{}] Failed to send {} message with ID {} to peer {}: {}", this.peerConnectionId,
message.getType().name(), message.getId(), this, e.getMessage());
return false;
} catch (IOException | InterruptedException e) {
// Possible race condition:
// We set OP_WRITE, EPC creates ChannelWriteTask which calls Peer.writeChannel, writeChannel's poll() finds no message to send
// Avoided by poll-with-timeout in writeChannel() above.
Network.getInstance().setInterestOps(this.socketChannel, SelectionKey.OP_WRITE);
return this.sendQueue.tryTransfer(message, timeout, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
// Send failure
return false;
} catch (MessageException e) {
LOGGER.error(e.getMessage(), e);
return false;
}
// Sent OK
return true;
}
/**
@@ -720,7 +771,7 @@ public class Peer {
this.lastPingSent = NTP.getTime();
}
protected ExecuteProduceConsume.Task getPingTask(Long now) {
protected Task getPingTask(Long now) {
// Pings not enabled yet?
if (now == null || this.lastPingSent == null) {
return null;
@@ -734,19 +785,7 @@ public class Peer {
// Not strictly true, but prevents this peer from being immediately chosen again
this.lastPingSent = now;
return () -> {
PingMessage pingMessage = new PingMessage();
Message message = this.getResponse(pingMessage);
if (message == null || message.getType() != MessageType.PING) {
LOGGER.debug("[{}] Didn't receive reply from {} for PING ID {}", this.peerConnectionId, this,
pingMessage.getId());
this.disconnect("no ping received");
return;
}
this.setLastPing(NTP.getTime() - now);
};
return new PingTask(this, now);
}
public void disconnect(String reason) {
@@ -877,6 +916,10 @@ public class Peer {
return maxConnectionAge;
}
public void setMaxConnectionAge(long maxConnectionAge) {
this.maxConnectionAge = maxConnectionAge;
}
public boolean hasReachedMaxConnectionAge() {
return this.getConnectionAge() > this.getMaxConnectionAge();
}

View File

@@ -0,0 +1,70 @@
package org.qortal.network.message;
import com.google.common.primitives.Longs;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.transform.Transformer;
import org.qortal.utils.Base58;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
public class AccountBalanceMessage extends Message {
private static final int ADDRESS_LENGTH = Transformer.ADDRESS_LENGTH;
private AccountBalanceData accountBalanceData;
public AccountBalanceMessage(AccountBalanceData accountBalanceData) {
super(MessageType.ACCOUNT_BALANCE);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try {
// Send raw address instead of base58 encoded
byte[] address = Base58.decode(accountBalanceData.getAddress());
bytes.write(address);
bytes.write(Longs.toByteArray(accountBalanceData.getAssetId()));
bytes.write(Longs.toByteArray(accountBalanceData.getBalance()));
} catch (IOException e) {
throw new AssertionError("IOException shouldn't occur with ByteArrayOutputStream");
}
this.dataBytes = bytes.toByteArray();
this.checksumBytes = Message.generateChecksum(this.dataBytes);
}
public AccountBalanceMessage(int id, AccountBalanceData accountBalanceData) {
super(id, MessageType.ACCOUNT_BALANCE);
this.accountBalanceData = accountBalanceData;
}
public AccountBalanceData getAccountBalanceData() {
return this.accountBalanceData;
}
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) {
byte[] addressBytes = new byte[ADDRESS_LENGTH];
byteBuffer.get(addressBytes);
String address = Base58.encode(addressBytes);
long assetId = byteBuffer.getLong();
long balance = byteBuffer.getLong();
AccountBalanceData accountBalanceData = new AccountBalanceData(address, assetId, balance);
return new AccountBalanceMessage(id, accountBalanceData);
}
public AccountBalanceMessage cloneWithNewId(int newId) {
AccountBalanceMessage clone = new AccountBalanceMessage(this.accountBalanceData);
clone.setId(newId);
return clone;
}
}

View File

@@ -0,0 +1,93 @@
package org.qortal.network.message;
import com.google.common.primitives.Ints;
import org.qortal.data.account.AccountData;
import org.qortal.transform.Transformer;
import org.qortal.utils.Base58;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
public class AccountMessage extends Message {
private static final int ADDRESS_LENGTH = Transformer.ADDRESS_LENGTH;
private static final int REFERENCE_LENGTH = Transformer.SIGNATURE_LENGTH;
private static final int PUBLIC_KEY_LENGTH = Transformer.PUBLIC_KEY_LENGTH;
private AccountData accountData;
public AccountMessage(AccountData accountData) {
super(MessageType.ACCOUNT);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try {
// Send raw address instead of base58 encoded
byte[] address = Base58.decode(accountData.getAddress());
bytes.write(address);
bytes.write(accountData.getReference());
bytes.write(accountData.getPublicKey());
bytes.write(Ints.toByteArray(accountData.getDefaultGroupId()));
bytes.write(Ints.toByteArray(accountData.getFlags()));
bytes.write(Ints.toByteArray(accountData.getLevel()));
bytes.write(Ints.toByteArray(accountData.getBlocksMinted()));
bytes.write(Ints.toByteArray(accountData.getBlocksMintedAdjustment()));
} catch (IOException e) {
throw new AssertionError("IOException shouldn't occur with ByteArrayOutputStream");
}
this.dataBytes = bytes.toByteArray();
this.checksumBytes = Message.generateChecksum(this.dataBytes);
}
public AccountMessage(int id, AccountData accountData) {
super(id, MessageType.ACCOUNT);
this.accountData = accountData;
}
public AccountData getAccountData() {
return this.accountData;
}
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) {
byte[] addressBytes = new byte[ADDRESS_LENGTH];
byteBuffer.get(addressBytes);
String address = Base58.encode(addressBytes);
byte[] reference = new byte[REFERENCE_LENGTH];
byteBuffer.get(reference);
byte[] publicKey = new byte[PUBLIC_KEY_LENGTH];
byteBuffer.get(publicKey);
int defaultGroupId = byteBuffer.getInt();
int flags = byteBuffer.getInt();
int level = byteBuffer.getInt();
int blocksMinted = byteBuffer.getInt();
int blocksMintedAdjustment = byteBuffer.getInt();
AccountData accountData = new AccountData(address, reference, publicKey, defaultGroupId, flags, level, blocksMinted, blocksMintedAdjustment);
return new AccountMessage(id, accountData);
}
public AccountMessage cloneWithNewId(int newId) {
AccountMessage clone = new AccountMessage(this.accountData);
clone.setId(newId);
return clone;
}
}

View File

@@ -1,90 +1,136 @@
package org.qortal.network.message;
import com.google.common.primitives.Ints;
import com.google.common.primitives.Longs;
import org.qortal.data.network.PeerData;
import org.qortal.transform.TransformationException;
import org.qortal.transform.Transformer;
import org.qortal.utils.Serialization;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.List;
public class ArbitraryDataFileListMessage extends Message {
private static final int SIGNATURE_LENGTH = Transformer.SIGNATURE_LENGTH;
private static final int HASH_LENGTH = Transformer.SHA256_LENGTH;
private byte[] signature;
private List<byte[]> hashes;
private Long requestTime;
private Integer requestHops;
private String peerAddress;
private Boolean isRelayPossible;
private final byte[] signature;
private final List<byte[]> hashes;
public ArbitraryDataFileListMessage(byte[] signature, List<byte[]> hashes) {
public ArbitraryDataFileListMessage(byte[] signature, List<byte[]> hashes, Long requestTime,
Integer requestHops, String peerAddress, Boolean isRelayPossible) {
super(MessageType.ARBITRARY_DATA_FILE_LIST);
this.signature = signature;
this.hashes = hashes;
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try {
bytes.write(signature);
bytes.write(Ints.toByteArray(hashes.size()));
for (byte[] hash : hashes) {
bytes.write(hash);
}
if (requestTime != null) {
// The remaining fields are optional
bytes.write(Longs.toByteArray(requestTime));
bytes.write(Ints.toByteArray(requestHops));
Serialization.serializeSizedStringV2(bytes, peerAddress);
bytes.write(Ints.toByteArray(Boolean.TRUE.equals(isRelayPossible) ? 1 : 0));
}
} catch (IOException e) {
throw new AssertionError("IOException shouldn't occur with ByteArrayOutputStream");
}
this.dataBytes = bytes.toByteArray();
this.checksumBytes = Message.generateChecksum(this.dataBytes);
}
public ArbitraryDataFileListMessage(int id, byte[] signature, List<byte[]> hashes) {
/** Legacy version */
public ArbitraryDataFileListMessage(byte[] signature, List<byte[]> hashes) {
this(signature, hashes, null, null, null, null);
}
private ArbitraryDataFileListMessage(int id, byte[] signature, List<byte[]> hashes, Long requestTime,
Integer requestHops, String peerAddress, boolean isRelayPossible) {
super(id, MessageType.ARBITRARY_DATA_FILE_LIST);
this.signature = signature;
this.hashes = hashes;
}
public List<byte[]> getHashes() {
return this.hashes;
this.requestTime = requestTime;
this.requestHops = requestHops;
this.peerAddress = peerAddress;
this.isRelayPossible = isRelayPossible;
}
public byte[] getSignature() {
return this.signature;
}
public static Message fromByteBuffer(int id, ByteBuffer bytes) throws UnsupportedEncodingException, TransformationException {
byte[] signature = new byte[SIGNATURE_LENGTH];
public List<byte[]> getHashes() {
return this.hashes;
}
public Long getRequestTime() {
return this.requestTime;
}
public Integer getRequestHops() {
return this.requestHops;
}
public String getPeerAddress() {
return this.peerAddress;
}
public Boolean isRelayPossible() {
return this.isRelayPossible;
}
public static Message fromByteBuffer(int id, ByteBuffer bytes) throws MessageException {
byte[] signature = new byte[Transformer.SIGNATURE_LENGTH];
bytes.get(signature);
int count = bytes.getInt();
if (bytes.remaining() != count * HASH_LENGTH)
return null;
List<byte[]> hashes = new ArrayList<>();
for (int i = 0; i < count; ++i) {
byte[] hash = new byte[HASH_LENGTH];
byte[] hash = new byte[Transformer.SHA256_LENGTH];
bytes.get(hash);
hashes.add(hash);
}
return new ArbitraryDataFileListMessage(id, signature, hashes);
}
Long requestTime = null;
Integer requestHops = null;
String peerAddress = null;
boolean isRelayPossible = true; // Legacy versions only send this message when relaying is possible
@Override
protected byte[] toData() {
try {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
// The remaining fields are optional
if (bytes.hasRemaining()) {
try {
requestTime = bytes.getLong();
bytes.write(this.signature);
requestHops = bytes.getInt();
bytes.write(Ints.toByteArray(this.hashes.size()));
peerAddress = Serialization.deserializeSizedStringV2(bytes, PeerData.MAX_PEER_ADDRESS_SIZE);
for (byte[] hash : this.hashes) {
bytes.write(hash);
isRelayPossible = bytes.getInt() > 0;
} catch (TransformationException e) {
throw new MessageException(e.getMessage(), e);
}
return bytes.toByteArray();
} catch (IOException e) {
return null;
}
}
public ArbitraryDataFileListMessage cloneWithNewId(int newId) {
ArbitraryDataFileListMessage clone = new ArbitraryDataFileListMessage(this.signature, this.hashes);
clone.setId(newId);
return clone;
return new ArbitraryDataFileListMessage(id, signature, hashes, requestTime, requestHops, peerAddress, isRelayPossible);
}
}

View File

@@ -1,48 +1,68 @@
package org.qortal.network.message;
import com.google.common.primitives.Ints;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.arbitrary.ArbitraryDataFile;
import org.qortal.repository.DataException;
import org.qortal.transform.Transformer;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.nio.BufferUnderflowException;
import java.nio.ByteBuffer;
public class ArbitraryDataFileMessage extends Message {
private static final int SIGNATURE_LENGTH = Transformer.SIGNATURE_LENGTH;
private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFileMessage.class);
private final byte[] signature;
private final ArbitraryDataFile arbitraryDataFile;
private byte[] signature;
private ArbitraryDataFile arbitraryDataFile;
public ArbitraryDataFileMessage(byte[] signature, ArbitraryDataFile arbitraryDataFile) {
super(MessageType.ARBITRARY_DATA_FILE);
this.signature = signature;
this.arbitraryDataFile = arbitraryDataFile;
byte[] data = arbitraryDataFile.getBytes();
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try {
bytes.write(signature);
bytes.write(Ints.toByteArray(data.length));
bytes.write(data);
} catch (IOException e) {
throw new AssertionError("IOException shouldn't occur with ByteArrayOutputStream");
}
this.dataBytes = bytes.toByteArray();
this.checksumBytes = Message.generateChecksum(this.dataBytes);
}
public ArbitraryDataFileMessage(int id, byte[] signature, ArbitraryDataFile arbitraryDataFile) {
private ArbitraryDataFileMessage(int id, byte[] signature, ArbitraryDataFile arbitraryDataFile) {
super(id, MessageType.ARBITRARY_DATA_FILE);
this.signature = signature;
this.arbitraryDataFile = arbitraryDataFile;
}
public byte[] getSignature() {
return this.signature;
}
public ArbitraryDataFile getArbitraryDataFile() {
return this.arbitraryDataFile;
}
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) throws UnsupportedEncodingException {
byte[] signature = new byte[SIGNATURE_LENGTH];
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) throws MessageException {
byte[] signature = new byte[Transformer.SIGNATURE_LENGTH];
byteBuffer.get(signature);
int dataLength = byteBuffer.getInt();
if (byteBuffer.remaining() != dataLength)
return null;
if (byteBuffer.remaining() < dataLength)
throw new BufferUnderflowException();
byte[] data = new byte[dataLength];
byteBuffer.get(data);
@@ -50,42 +70,10 @@ public class ArbitraryDataFileMessage extends Message {
try {
ArbitraryDataFile arbitraryDataFile = new ArbitraryDataFile(data, signature);
return new ArbitraryDataFileMessage(id, signature, arbitraryDataFile);
} catch (DataException e) {
LOGGER.info("Unable to process received file: {}", e.getMessage());
throw new MessageException("Unable to process received file: " + e.getMessage(), e);
}
catch (DataException e) {
return null;
}
}
@Override
protected byte[] toData() {
if (this.arbitraryDataFile == null) {
return null;
}
byte[] data = this.arbitraryDataFile.getBytes();
if (data == null) {
return null;
}
try {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
bytes.write(signature);
bytes.write(Ints.toByteArray(data.length));
bytes.write(data);
return bytes.toByteArray();
} catch (IOException e) {
return null;
}
}
public ArbitraryDataFileMessage cloneWithNewId(int newId) {
ArbitraryDataFileMessage clone = new ArbitraryDataFileMessage(this.signature, this.arbitraryDataFile);
clone.setId(newId);
return clone;
}
}

View File

@@ -2,7 +2,7 @@ package org.qortal.network.message;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.nio.BufferUnderflowException;
import java.nio.ByteBuffer;
import org.qortal.transform.Transformer;
@@ -11,13 +11,26 @@ import com.google.common.primitives.Ints;
public class ArbitraryDataMessage extends Message {
private static final int SIGNATURE_LENGTH = Transformer.SIGNATURE_LENGTH;
private byte[] signature;
private byte[] data;
public ArbitraryDataMessage(byte[] signature, byte[] data) {
this(-1, signature, data);
super(MessageType.ARBITRARY_DATA);
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try {
bytes.write(signature);
bytes.write(Ints.toByteArray(data.length));
bytes.write(data);
} catch (IOException e) {
throw new AssertionError("IOException shouldn't occur with ByteArrayOutputStream");
}
this.dataBytes = bytes.toByteArray();
this.checksumBytes = Message.generateChecksum(this.dataBytes);
}
private ArbitraryDataMessage(int id, byte[] signature, byte[] data) {
@@ -35,14 +48,14 @@ public class ArbitraryDataMessage extends Message {
return this.data;
}
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) throws UnsupportedEncodingException {
byte[] signature = new byte[SIGNATURE_LENGTH];
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) {
byte[] signature = new byte[Transformer.SIGNATURE_LENGTH];
byteBuffer.get(signature);
int dataLength = byteBuffer.getInt();
if (byteBuffer.remaining() != dataLength)
return null;
if (byteBuffer.remaining() < dataLength)
throw new BufferUnderflowException();
byte[] data = new byte[dataLength];
byteBuffer.get(data);
@@ -50,24 +63,4 @@ public class ArbitraryDataMessage extends Message {
return new ArbitraryDataMessage(id, signature, data);
}
@Override
protected byte[] toData() {
if (this.data == null)
return null;
try {
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
bytes.write(this.signature);
bytes.write(Ints.toByteArray(this.data.length));
bytes.write(this.data);
return bytes.toByteArray();
} catch (IOException e) {
return null;
}
}
}

View File

@@ -0,0 +1,74 @@
package org.qortal.network.message;
import com.google.common.primitives.Ints;
import org.qortal.arbitrary.ArbitraryDataFile;
import org.qortal.repository.DataException;
import org.qortal.transform.Transformer;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.BufferUnderflowException;
import java.nio.ByteBuffer;
public class ArbitraryMetadataMessage extends Message {
private byte[] signature;
private ArbitraryDataFile arbitraryMetadataFile;
public ArbitraryMetadataMessage(byte[] signature, ArbitraryDataFile arbitraryMetadataFile) {
super(MessageType.ARBITRARY_METADATA);
byte[] data = arbitraryMetadataFile.getBytes();
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
try {
bytes.write(signature);
bytes.write(Ints.toByteArray(data.length));
bytes.write(data);
} catch (IOException e) {
throw new AssertionError("IOException shouldn't occur with ByteArrayOutputStream");
}
this.dataBytes = bytes.toByteArray();
this.checksumBytes = Message.generateChecksum(this.dataBytes);
}
private ArbitraryMetadataMessage(int id, byte[] signature, ArbitraryDataFile arbitraryMetadataFile) {
super(id, MessageType.ARBITRARY_METADATA);
this.signature = signature;
this.arbitraryMetadataFile = arbitraryMetadataFile;
}
public byte[] getSignature() {
return this.signature;
}
public ArbitraryDataFile getArbitraryMetadataFile() {
return this.arbitraryMetadataFile;
}
public static Message fromByteBuffer(int id, ByteBuffer byteBuffer) throws MessageException {
byte[] signature = new byte[Transformer.SIGNATURE_LENGTH];
byteBuffer.get(signature);
int dataLength = byteBuffer.getInt();
if (byteBuffer.remaining() < dataLength)
throw new BufferUnderflowException();
byte[] data = new byte[dataLength];
byteBuffer.get(data);
try {
ArbitraryDataFile arbitraryMetadataFile = new ArbitraryDataFile(data, signature);
return new ArbitraryMetadataMessage(id, signature, arbitraryMetadataFile);
} catch (DataException e) {
throw new MessageException("Unable to process arbitrary metadata message: " + e.getMessage(), e);
}
}
}

Some files were not shown because too many files have changed in this diff Show More