Commit Graph

796 Commits

Author SHA1 Message Date
catbref
76485010ad
Merge pull request #16 from tcallahan14/feature/electrum_nodes
Updated Electrum nodes list
2020-08-21 13:34:49 +01:00
catbref
b8ac128d5c Improve comparing chains where some blocks signed with cancelled reward-share
Symptoms include this in logs:

Unexpected zero effective minter level for reward-share %s - using 1 instead!

This occurs when Synchronizer compares two sub-chains from a common block,
and one of the blocks is signed by a reward-share key that has
subsequently been cancelled.

Although this is catered for, excessive log-spam is emited.

So in addition to demoting the log level from WARN to DEBUG,
more code has been added to try harder to find the actual data needed,
thus preventing the logging in the first place.

New repository transaction search method added to support above,
along with corresponding tests.
2020-08-21 12:27:06 +01:00
CalDescent
06c75310a1 Updated Electrum nodes list. All nodes have been tested to ensure they respond to jsonrpc calls. 2020-08-15 16:53:09 +01:00
catbref
b9d819220d Bumped HSQLDB to v2.5.1 and AT/cross-chain SQL speed-ups! 2020-08-15 11:12:10 +01:00
catbref
7a569f342f Reduce confusing BlockMinter log spam - issue #9 2020-08-15 10:52:48 +01:00
catbref
f1efae79c8 Speed-ups for some AT-related SQL queries 2020-08-14 11:54:33 +01:00
catbref
1cd4bbc078 Refactored various websockets to event bus from old BlockNotifier/StatusNotifier 2020-08-14 10:03:51 +01:00
catbref
0b5e5832c4 Added another repository deadlock test while investigating a deadlock case 2020-08-14 09:57:08 +01:00
catbref
f8725d6313 Modify ApplyUpdate to pass JVM options to Windows launcher EXE
ApplyUpdate is the 2nd-stage of the auto-update system, called
after core has downloaded the update.

As old versions of the Windows launcher EXE selects a 'client'
JVM mode, heap memory could be limited to only 256MB.

Until users upgrade via Windows installer, which replaces the EXE
with 'server' JVM mode baked-in, then a work-around is to
pass -XX:MaxRAMFraction=4 to the new JVM in order to emulate
heap size in 'server' JVM mode.
2020-08-13 14:08:47 +01:00
catbref
2165c87b9d Fix race condition between Network.start() and Controller calling Network.prunePeers()
Modified synchronized Lists to be final.
Moved some initializers out of constructor.
2020-08-13 13:45:06 +01:00
catbref
f61e320230 Fix API call GET /crosschain/trades (get completed trades) due to poorly performing SQL query.
Added "minimumTimestamp" param to same API call to allow fetching results for scenarios like:
* completed trades since midnight
* completed trades within last 24 hours

Added corresponding tests for above API call, including checking call response times.
2020-08-13 11:56:08 +01:00
catbref
f6216b9745 Respect repositoryBackupInterval when deciding whether to perform backup during auto-update. Issue #10 2020-08-12 14:22:36 +01:00
catbref
91e82d1e3c Add HSQLDB CHECKPOINT statements between database reshaping statements to reduce failures 2020-08-12 14:20:23 +01:00
catbref
50e2bda020 Quicken blockchain validity check at start-up by only checking most recent 1440 blocks 2020-08-12 14:19:14 +01:00
catbref
ab1de1aafa Improve HSQLDB query for finding latest AT state data 2020-08-12 14:18:05 +01:00
catbref
d4ac87f91d Update to more efficient CIYAM AT v1.3.7 2020-08-12 14:17:09 +01:00
catbref
d8dd71ff50 Fix off-by-one in some sync cases 2020-08-10 15:22:01 +01:00
catbref
02966bf39a Update CIYAM-AT to v1.3.6 to make use of lambda-based logging 2020-08-10 14:05:42 +01:00
catbref
a83d8bf1d5 Split Synchronizer into two strategies depending on whether swapping chains or simply adding new blocks 2020-08-10 14:00:45 +01:00
catbref
1e4432b1f3 Convert LOBs to VARBINARY in DB, reducing sizes at the same time 2020-08-10 13:59:56 +01:00
catbref
d50c979d9f Tighten limit on AT sizes 2020-08-10 13:59:20 +01:00
catbref
4e60ec5192 Disable peer buffer dealloc for now to reduce GC pressure 2020-08-10 13:58:12 +01:00
catbref
31c4e3b1be Reduce memory PoW during network handshake 2020-08-10 13:57:16 +01:00
catbref
b97fbd3171 Bug-fix for cached online accounts 2020-08-07 16:19:32 +01:00
catbref
43fb5d9332 Cache top 2 blocks' worth of online account data to avoid unnecessary Ed25519 verifications 2020-08-07 13:33:31 +01:00
catbref
ea3f1a8eff Actually respond to peers requesting unknown block instead of letting them timeout on their side 2020-08-07 13:32:19 +01:00
catbref
a1ab0b7c31 Added more nodes to initial list 2020-08-06 15:38:52 +01:00
catbref
fae2afd010 Remove obsolete github repo from potential auto-update sites 2020-08-06 15:23:13 +01:00
catbref
76c0a5a4fa Increase default Bitcoin transaction fee to 5000 sats 2020-08-06 13:12:54 +01:00
catbref
cdb65657b6 Added qortal.ru nodes to initial nodes list used when creating DB 2020-08-06 09:51:14 +01:00
catbref
9007dfe779 Added API call POST /crosschain/btc/send for sending Bitcoin 2020-08-06 09:46:08 +01:00
catbref
99d09a9877 Change HTTP response codes for BTC_BALANCE_ISSUE and BTC_TOO_SOON from 422 to 402 & 408 2020-08-06 09:45:38 +01:00
catbref
afcf51399e Minor post-merge fix-up 2020-08-06 09:18:08 +01:00
catbref
47679b7f6c Merge branch 'trade-bot' 2020-08-06 09:00:45 +01:00
catbref
8f2985862d Update BTC-ACCT 'cancel' API call to expect AT creator's as sender 2020-08-06 08:52:51 +01:00
catbref
23a524b464 BTC-ACCT: change AT so 'cancel' MESSAGE needs to come from AT creator's address (not trade address) so fee can be used instead of PoW for faster cancels 2020-08-06 08:23:49 +01:00
catbref
ce8992867d Include last 24 hours of CANCELLED & REFUNDED trade offers in first message 2020-08-05 20:58:25 +01:00
catbref
c89de7adfb Add creatorAddress, qortAmount and (last updated) timestamp to trade-bot entries 2020-08-05 16:00:40 +01:00
catbref
cac68ccc14 Added trade-bot websocket 2020-08-05 13:23:24 +01:00
catbref
d507383487 Rework ApiWebSocket so it can manage sessions and in readiness to conversion from "notifiers" to event-bus 2020-08-05 13:23:07 +01:00
catbref
ce5cf87094 Added unified, simple event bus to eventually replace controller "notifiers" 2020-08-05 13:20:19 +01:00
catbref
ec2c9d2a44 Improve /crosschain/tradebot/respond with varied API errors such as BTC_BALANCE_ISSUE, BTC_NETWORK_ISSUE, etc. instead of just "false" 2020-08-05 10:05:09 +01:00
catbref
36d0abe635 WIP: trade-bot: log warning when we can't fund P2SH-B for some reason 2020-08-05 10:04:25 +01:00
catbref
615381ca5a Fix BTC spend txn building to be less aggressive about caching/checking spent keys 2020-08-05 10:03:08 +01:00
catbref
6b83499216 WIP: trade-bot: add support for showing trade partner's Qortal receiving address in trade offer summaries 2020-08-04 20:35:22 +01:00
catbref
faa2e9502b WIP: trade-bot: include creation/latest timestamp (as appropriate) in trade offer summaries via websocket 2020-08-04 17:00:36 +01:00
catbref
cd07240ce7 Add BTC.getWalletBalance(xprv) and add API call to access that.
Also improved BTC.WalletAwareUTXOProvider to derive more keys itself
instead of throwing and relying on caller to do the work.
Added benefit of cleaning up caller code and being more efficient.
Needed because not all receiving/change addresses were being picked up.
2020-08-04 16:37:44 +01:00
catbref
91518464c2 WIP: trade-bot: fix empty bitcoin wallet edge case when finding UTXOs 2020-08-04 12:26:09 +01:00
catbref
25bf315e23 WIP: trade-bot: tradeoffers websocket initial message with OFFERING/REDEEMED and fixed subsequent messages 2020-08-04 11:21:57 +01:00
catbref
a8743b1bd3 ElectrumX network main-net servers 2020-08-03 19:28:55 +01:00
catbref
f90bd6ee45 WIP: trade-bot: added WS for streaming existing/new trades in OFFERING state 2020-08-03 17:57:22 +01:00
catbref
a351756883 WIP: trade-bot: add missing JavaTypeAdapter to TradeBotData.bitcoinAmount
Also: unify "receiveAddress" to "receivingAddress"

Fix missed changes from "recipient" to "partner" in BTCACCT, etc.
2020-08-03 15:52:17 +01:00
catbref
ea9b0d4588 WIP: trade-bot: initial API call for listing completed trades
Also: renamed trade bot field/column "receiving_public_key_hash"
to "receiving_account_info" as Alice's trade bot uses it to
store Alice's Qortal address, not PKH.

Added some extra simplistic repository calls to support above,
like BlockRepository.getTimestampFromHeight,
ATRepository.getCreatorPublicKey(atAddress)
2020-08-03 14:54:45 +01:00
catbref
e9c85c946e WIP: trade-bot: two more unit tests to cover some edge cases 2020-08-03 10:49:47 +01:00
catbref
876bfb525b WIP: trade-bot: more receive address support, some terminology clarification
Bitcoin receive address no longer stored in AT but dealt with by trade-bot.
This allows 'Bob' to have his BTC sent anywhere he likes when redeeming P2SH-A
thus saving a step, typically incurred by UI. DB shape change due to this.

Similarly, AT code has been updated to expect a Qortal receiving address when
Alice sends MESSAGE to redeem AT.

This means both trade-bot entries (Alice/Bob) can be safely wiped once trade completes.

Some terms were confusing like "trade recipient" which actually referred to
Alice and so have been unified as "trade partner" as to not be confused with
(say) "recipient address"

The MESSAGEs sent from Alice to Bob, from Bob to AT and from Alice to AT have been
given more useful names: 'offer', 'trade' and 'redeem'. There is also a cancel
MESSAGE sent from Bob to AT to cancel AT before trading occurs.

Some API calls have been renamed in light of above.

AT's 'mode' has been expanded from simply OFFER/TRADE to:
OFFERING, TRADING, REFUNDED, REDEEMED, CANCELLED

Tests updated, but MORE TESTING REQUIRED BEFORE RELEASE
2020-08-03 09:36:46 +01:00
catbref
6be67d0d92 WIP: trade-bot: make sure the "trade" private key is valid for both Curve25519 and secp256k1 2020-07-30 08:12:45 +01:00
catbref
16581766c6 WIP: trade-bot: detect and remove mempool entries from ElectrumX "listunspent" results 2020-07-29 20:48:06 +01:00
catbref
7fd7104f46 WIP: trade-bot: add flag to be set by AT if redeem happens so trade-bot detects redeem instead of refund 2020-07-29 20:25:28 +01:00
catbref
d2cae7c8b5 WIP: trade-bot: use correct Bob Bitcoin receive address in log entry 2020-07-29 18:14:47 +01:00
catbref
83955acd22 WIP: trade-bot: allow trade-bot entries to be deleted if in BOB_WAITING_FOR_AT_CONFIRM state. Also, return false (instead of throwing internal error) if trade-bot entry does not exist 2020-07-29 18:13:27 +01:00
catbref
d85b746021 WIP: trade-bot: add xprv validation method to BTC class and use that for API call /crosschain/tradebot/respond instead of vague byte-length check 2020-07-29 18:11:47 +01:00
catbref
e2dc91c1ea Fix API call DELETE /crosschain/tradeoffer regarding PoW MESSAGE reference 2020-07-29 10:38:32 +01:00
catbref
098e2623d6 WIP: cross-chain AT now stores bitcoin receiving PKH 2020-07-28 17:21:54 +01:00
catbref
6c182a3567 Allow minting accounts to be removed from node using public key as well as private key 2020-07-28 10:45:06 +01:00
catbref
340d6dfc8d Add websocket error handler support 2020-07-27 10:16:21 +01:00
catbref
eb27b0d3e2 Blocks websocket now returns simpler block info 2020-07-24 14:52:14 +01:00
catbref
7377893050 WebSocket improvements, inc. bump Jetty to v9.4.29-20200521
Various issues in Jetty v9.4.22 (and some later versions too)
cause websockets to use up all available threads.

Bumped Jetty to v9.4.29 to resolve some of these issues.

Changed some Qortal-side websocket code to minimize
locking on websocket notifiers. Websocket messages now
sent async, although the returned Futures are discarded,
as it's up to the remote end to consume fast enough.

Changed Controller to only request a SysTray update before
synchronization if there's a chance node might change height.
Similarly, Controller only requests SysTray update after
synchronization if chain tip has actually changed.
Both of the above together should reduce the number of
messages sent out via the admin status websockets.
2020-07-24 10:34:42 +01:00
catbref
21d7a4eed1 Improved AT PUT_TX_AFTER_TIMESTAMP_INTO_A function
Previous version fetched all the blocks from previous 'timestamp'
to current height, checking each transaction. (very slow)

New implementation leverages repository to do the heavy lifting.

Could potentially benefit from some DB indexes in the future?

Added unit test to cover.
2020-07-23 08:43:27 +01:00
catbref
fb2c2b1d09 Added API call GET /blocks/summaries
Returns summary info about a range of blocks.

(Not to be confused with network-related BlockSummaries)
2020-07-20 13:05:43 +01:00
catbref
dea2f34c52 Trade-bot: more comments, more documentation, more ElectrumX servers.
Bitcoin main-net ElectrumX server list added to ElectrumX class,
albeit commented out at this point until it is decided that trade-bot
is ready for production use. (Simply remove the leading //s)

More comments and documentation has been added to TradeBot class
to further describe the actions taken.

It is important to note that:

Bitcoin wallet access is required by trade-bot

and so:

A Bitcoin WALLET PRIVATE KEY is stored in the database by trade-bot

and hence, if you use trade-bot:

DO NOT DISTRIBUTE YOUR DB FILES TO ANYONE ELSE!

Furthermore it should be obvious that this functionality is provided on
a 'best effort", not guaranteed, basis, therefore:

YOUR FUNDS ARE AT RISK!

If you are unsure about any aspect, or cannot afford to lose your funds,
or it's possible that unexpected outcomes occur, then DO NOT USE.

To use trade-bot on Bitcoin TESTNET then this to your settings JSON file:
  "bitcoinNet": "TEST3",

See Settings.java line 100, and BTC class for more info.
2020-07-17 11:28:22 +01:00
catbref
b294f5e333 WIP: More defensive ElectrumX calls. Bring non-trade-bot API calls up to date 2020-07-17 08:39:45 +01:00
catbref
f9b726a75d WIP: TradeBot - added refunding code 2020-07-17 08:39:45 +01:00
catbref
579645d6b7 WIP: trade-bot now does complete end-to-end trade (more work needed)
bitcoinj now uses ElectrumX as an UTXO provider in order to keep track
of coins in BIP32 deterministic wallet.

Trade responder (Alice) needs to pass a BIP32 extended private key to API
so trade-bot can create unattended spends.

Both Alice and Bob can find their final funds in accounts using the
ephemeral 'tradePrivateKey' from trade-bot state data.

Most cross-chain API calls are now only allowed from localhost.

Most Bitcoin fees pegged at 0.00001000 BTC.

More work needed to handle refunds in case of trade failures.
(See XXX comment tags in TradeBot.java)
2020-07-17 08:39:45 +01:00
catbref
e729571a21 WIP: trade-bot: do not run trade-bot if not up-to-date 2020-07-17 08:39:45 +01:00
catbref
f179139967 WIP: trade-bot: Alice P2SH_a progress
Qortal AT now includes suggested tradeTimeout again as a constant so trade partner/recipient can use that to calculate a suitable lockTimeA. CODE_HASH changed!

Renamed some secret_hash to hash_of_secret.

Changed TradeBotStates.trade_state back to TINYINT and adjusted values in TradeBotData.State enum to suit.
Added lockTimeA to TradeBotData & repository.

Added JAXB-only extra representations of Bitcoin PKHs as addresses.

Fixed incorrect expected length in BTCACCT.extractOfferMessageData().

CrossChainTradeData.refundTimeout now only present in TRADE mode.

Added BTC.pkhToAddress().

Added initial TradeBot.handleAliceWaitingForP2shA().

Enforce only one TradeBot thread running using 'activeFlag' atomic boolean.

Replace incorrect SHA256 with HASH160 for hashOfSecretA in TradeBot.startResponse().
2020-07-17 08:39:45 +01:00
catbref
ee5119e4dd WIP: trade-bot. move trade-bot hook, fix bugs, etc.
Controller now calls TradeBot.onChainTipChange() inside thread
started by Controller.onNewBlock(), instead of blocking
Controller.setChainTip().

DB TradeBotStates has trade_foreign_public_key changed to VARBINARY(33)
as Bitcoin pubkeys aren't uniformly 32 bytes!
Also, trade_state changed from TINYINT to SMALLINT to cover enum value range.

TradeBot.createTrade() incorrectly used Crypto.digest() to create hash-of-secret
instead of Crypto.hash160(). Also corrected tradeState to
BOB_WAITING_FOR_AT_CONFIRM. Also added missing fee calculation.

Added missing repository.saveChanges() to TradeBot methods.

Added balance check to API POST /crosschain/tradebot before passing
request to TradeBot.createTrade(), which also ensures there's a
usable account last-reference too.
2020-07-17 08:39:45 +01:00
catbref
11bf5ac6fc WIP: remove trade_timeout from DB TradeBotStates & TradeBotCreateRequest 2020-07-17 08:39:45 +01:00
catbref
c3eb385066 WIP: cross-chain trading with new lockTimes, requires AT v1.3.5 2020-07-17 08:39:45 +01:00
catbref
886c9156a5 WIP: cross-chain trading AT passes AtTests now 2020-07-17 08:39:45 +01:00
catbref
23062c59cd WIP: trade-bot, particularly the new two-P2SH Qortal AT code 2020-07-17 08:39:45 +01:00
catbref
da254058c5 WIP: split P2SH from BTCACCT, add more fields to TradeBotData, remove initial QORT payout 2020-07-17 08:39:45 +01:00
catbref
a6fa4fc613 WIP: trade-bot MESSAGE support 2020-07-17 08:39:45 +01:00
catbref
593b61ea4b Reduce bitcoinj exposure to classes outside of org.qortal.crosschain package.
BTC.getBalance() now returns Long instead of Coin.

BTC.FORMAT.format(Coin) changed to BTC.format(Coin or long).

Added BTC.deriveP2shAddress(byte[] redeemScriptBytes).
2020-07-17 08:39:45 +01:00
catbref
04d691991a WIP: more work on trade-bot 2020-07-17 08:39:45 +01:00
catbref
faa6e82bef WIP on trade-bot 2020-07-17 08:39:45 +01:00
catbref
65ccb80aa4 AT-related changes: new Qortal functions, tests, etc.
Added GET_MESSAGE_LENGTH_FROM_TX_IN_A
and PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.

Replaced AT-1.3.4 with version including bug-fix for off-by-one
data address bounds checking.

Moved long-from-bytes method to BitTwiddling class.

Renamed some methods to make it more obvious they work with
little/big endian data.
2020-07-17 08:39:45 +01:00
catbref
cc13d1d0f1 WIP commit 2020-07-17 08:39:45 +01:00
catbref
ead84d70d1 Initial skeleton code for Trade Bot 2020-07-17 08:39:45 +01:00
catbref
275146fb55 Return REWARD_SHARE_UNKNOWN when trying to cancel non-existent reward-share 2020-07-14 09:53:14 +01:00
catbref
e74a249388 Collate network PoW computes into a fixed-sized pool, with dead peer detection.
Also added Named/DaemonThreadFactory classes.

Network EPC now uses NamedThreadFactory for easier debugging.

Added settings field "networkPoWComputePoolSize", default 2, which
seems to work with both low-power ARM boards and high-power desktops.
2020-07-03 09:31:46 +01:00
catbref
d8c5e557d8 Update uiLocalServers, autoUpdateRepos and bump to v1.2.1 2020-06-30 15:41:53 +01:00
catbref
984e8b5227 Network optimizations: if we're not up to date then don't request, or send, unconfirmed transaction lists 2020-06-30 14:26:12 +01:00
catbref
469bf2a63e Improve inbound peer handshaking
If a node accepts a connection from an inbound peer
then remote peer will send RESPONSE first
and local node would previously change handshaking state
to COMPLETED while computing their own RESPONSE.

This meant that the local node would sometimes also start
sending post-handshake messages to the remote peer,
e.g. TRANSACTION_SIGNATURES.

Remote peer is only expecting a RESPONSE message, so would
close connection.

So we introduce an extra handshaking state "RESPONDING" for use
by local node while they compute RESPONSE in a separate thread.
Once the RESPONSE has been sent, local node moves to COMPLETED
state and called onHandshakeCompleted() as per usual.

Note that the code path when connecting outbound to a remote peer
is not changed, and the RESPONDING state is not used.

Also in this commit:

Network.onPeerReady now bypasses call to onMessage and instead
calls onHandshakingMessage() directly to avoid race condition
where peer's handshake status could change between
onPeerReady's caller and onMessage() calling peer.getHandshakeStatus()
2020-06-30 13:37:14 +01:00
catbref
e05fcd6655 Final genesis block & bump to v1.2.0 2020-06-29 15:45:10 +01:00
catbref
3a7751910e Fix /websockets/chat/messages so it disregards group-membership change notifications 2020-06-29 08:38:19 +01:00
catbref
3c139f3e53 Minor fix-up to allow go-live:
Re-add (for now) Ed25519 HSQLDB conversion.
Catch DataException in BlockChain.isGenesisBlockValid and return false.
Remove duplicate bug-fix for LeaveGroupTransactions DB table.

Interim, near-final blockchain.json
2020-06-26 15:06:04 +01:00
catbref
2c14a12464 Check for unknown Qortal-only AT function codes & add safety to AT running in general 2020-06-25 14:22:28 +01:00
catbref
faa6405d5f Reference fixes for MESSAGE transactions & tests to cover 2020-06-24 17:15:17 +01:00
catbref
e2e4555009 Added /websockets/admin/status and improved GET version.
NodeStatus contructor now fills in fields, which themselves are now 'final'.
NodeStatus also includes numberOfConnections and height as per systray.

AdminResource.status() unified with websocket version.
2020-06-24 11:54:06 +01:00
catbref
448e984995 Fix some minor group-related bugs
Incorrect column names when saving a group ban.

Missing column in LeaveGroupTransactions.

More stringent validity checks in group-kick, group-ban and remove-group-admin.

Added loads more tests to cover group actions.
2020-06-24 11:24:19 +01:00