Commit Graph

548 Commits

Author SHA1 Message Date
catbref
d50f16b8a9 PUBLICIZE transaction for on-chain record of public key 2020-05-25 15:20:21 +01:00
catbref
59de22883b Use CHAT, not MESSAGE, MAX_DATA_SIZE in ChatTransactionTransformer 2020-05-25 08:30:06 +01:00
catbref
db73afaf88 Remove Block.orphan() forced repository debugging 2020-05-25 07:27:42 +01:00
catbref
3afbd7aa51 Recheck for duplicate connection after handshaking to cover race condition with simultaneous bi-directional connections 2020-05-25 07:24:19 +01:00
catbref
0c32afa07f New network handshaking. NOT backwards compatible!
Old Qora v1 message types removed.
Message type values changed.

Network handshaking reworked to fix multiple-connections issue.
Instead of using some random peerID, we now use proper keypairs and a challenge-response handshake to prevent doppelgangers/ID-theft.
This results in simpler handshaking code as we don't have to perform some arcane doppelganger resolution.

Handshaking still uses proof-of-work for challenge-response, but switched to newer MemoryPoW.

API call GET /peers no longer has 'buildTimestamp' field, but does now have 'nodeId' field.

Network no longer has a whole raft of getXXXpeers() due to simplified handshaking.
Quite a few method calls changed to simply Network.getHandshakedPeers(), which is also faster.
2020-05-22 17:16:45 +01:00
catbref
bd543a526b Update uses of old Public/PrivateKeyAccount static methods to Crypto 2020-05-22 17:13:55 +01:00
catbref
b262044a52 Move some crypto methods from Public/PrivateKeyAccount to Crypto for reuse by new network handshaking 2020-05-22 17:06:48 +01:00
catbref
200a97184c Include transaction reference in chat messages returned by API call GET /chat/messages 2020-05-22 08:00:43 +01:00
catbref
5f4b66e5b0 Save public keys from CHAT transactions so they can be fetched via API. 2020-05-20 15:53:43 +01:00
catbref
9c48343581 Potential fix for rare HSQLDB "serialization failure" in Transaction.importAsUnconfirmed() 2020-05-20 07:33:21 +01:00
catbref
219f82f562 Modify API call GET /chats/active/{address} to return info on all joined groups.
Previously GET /chats/active/{address} would only return an active group chat
entry where 'address' was a member AND there was an existing CHAT
transaction with the same tx_group_id (and no recipient).

Now the response contains entries for ALL groups where 'address' is a member,
regardless of an existing CHAT transactions, omitting the 'timestamp' entry
if there are none.
2020-05-19 17:12:41 +01:00
catbref
51bfd49e25 Re-add missing senderName & recipientName to output of API call GET /chat/messages 2020-05-19 15:29:12 +01:00
catbref
7102f4a727 Fix for incorrect amounts reported by API 2020-05-19 15:20:20 +01:00
catbref
28991a926f Fix incorrect getDataLength() in RegisterNameTransactionTransformer 2020-05-19 14:55:26 +01:00
catbref
74f89af841 Enforce version 2+ for DEPLOY_AT 2020-05-19 08:35:25 +01:00
catbref
b4284515e7 Unify transaction NAME_NOT_LOWER_CASE checks to Unicode NAME_NOT_NORMALIZED version 2020-05-19 08:31:36 +01:00
catbref
032c5d0d07 Add missing fee check to TRANSFER_PRIVS 2020-05-19 08:25:53 +01:00
catbref
72100fe1d8 Refactor Unicode 'reduced' name code from side-effects into 'data' objects.
CREATE_GROUP, ISSUE_ASSET, REGISTER_NAME and UPDATE_NAME transactions affected.

The code to actually generate 'reduced' name was called inside isValid() and
relied on setting the corresponding transaction data object field so that it would
be saved by isValid()'s caller. Although this worked, it wasn't a very clean
solution.

Now the 'reduced' name is generated by transaction data object's constructors so
it is always present.

Also removed name/group/asset reduceName(String) methods as they were all the
same single-line call to Unicode.sanitize().
2020-05-19 08:08:21 +01:00
catbref
ed178e744d Merge branch 'asset-unicode' into launch 2020-05-19 07:57:06 +01:00
catbref
94f7079c2e Unicode homoglyph support to Assets 2020-05-19 07:56:17 +01:00
catbref
f1638aa9d9 Removed "owner" from CREATE_GROUP and added Unicode homoglyph support.
Group owner now derived from CREATE_GROUP transaction creator's public key.

Added 'reduced' group name to GroupData, with corresponding change to DB.
Renamed GroupData.getIsOpen() to simply isOpen().

Tidied up CreateGroupTransactionData, adding 'reduced' group name.
Renamed getIsOpen() to simply isOpen().
Added code to generated reduced group name when building genesis block.

Added Group.MIN_NAME_SIZE of 3.

DB tables changed to add reduced_group_name where appropriate,
removing owner where necessary.

Added GroupRepository.reducedGroupNameExists(String).

Fixed up test blockchain configs in src/test/resources/test-chain-v2*.json.
2020-05-18 17:27:32 +01:00
catbref
a7b9215ace Merge branch 'message-wo-recipient' into launch 2020-05-18 10:12:54 +01:00
catbref
956ad7bfa8 Merge branch 'asset-fixes' into launch 2020-05-18 10:12:42 +01:00
catbref
4baf442cb8 Merge branch 'name-fixes' into launch 2020-05-18 10:12:31 +01:00
catbref
24eb7c6933 Allow MESSAGE transactions to have no recipient.
This allows on-chain messages to a group, including NO_GROUP / groupID zero.

No-recipient messages cannot have an amount - where would it go?

Changed MESSAGE serialization layout to add boolean indicating
whether recipient is present.

Changed MESSAGE serialization layout so assetID is after amount,
and only present if amount is non-zero.

Changed DB table structures to cover above.

Added unit tests to cover above.
2020-05-18 09:09:35 +01:00
catbref
38a2af8cd5 Tidy up Assets by removing 'owner' from ISSUE_ASSET.
Owner now derived from issuer's public key.
Maximum asset name length reduced to 40 characters.

Repository table changes.

"owner" removed from test blockchain configs and "issuerPublicKey" used instead
where applicable.

Some getters in the form of "getIs___()" renamed to simply "is____()".
2020-05-15 16:22:13 +01:00
catbref
7447ab20a9 Add index for finding Registered Names using 'reduced' form 2020-05-15 14:18:51 +01:00
catbref
197c742ce7 Major work on Registered Names
Changes include:

* Allowing renaming
* Tracking last-updated timestamps
* More stringent Unicode processing
* Way more unit tests
* Max name length reduction to 40 chars

Note: HSQLDB repository table changes
2020-05-15 14:08:46 +01:00
catbref
f6ed3388a4 BTC tidy-up 2020-05-15 07:45:24 +01:00
catbref
c61690f3e6 BTC class does not need to extend Thread! 2020-05-14 13:18:44 +01:00
catbref
9a94873d0e Fix broken Long vs Long comparison in Block.areAtsValid() 2020-05-14 13:18:17 +01:00
catbref
5c8bda37d1 Rework BTC class for better startup & shutdown.
Controller no longer starts up BTC support during main startup.
This does mean that BTC startup is deferred until first BTC-related
action, and that the first BTC-related action will take much longer
to complete.

Added tests to cover startup/shutdown.

This also fixes splash logo stuck on-screen and broken Controller
shutdown when using REGTEST bitcoin network AND there is no
local regtest bitcoin server running.
2020-05-14 12:52:26 +01:00
catbref
fa08041696 BlockTransformer should skip AT transactions when calculating block length 2020-05-14 12:51:44 +01:00
catbref
f01a34a461 Throw an API error for inappropriate calls on OFFER-state cross-chain ATs.
P2SH-related API calls under /crosschain/ aren't applicable for
cross-chain ATs that are still in OFFER state, only TRADE state.
2020-05-14 12:48:41 +01:00
catbref
ef790a8cb1 Fix missing groupId 0 entry in output from API call GET /chat/active/{address} 2020-05-13 16:30:28 +01:00
catbref
cea0cee9a8 Names: fixes to allow name change and tests to cover 2020-05-13 15:07:36 +01:00
catbref
d9f784ed2b Registered names: changing 'owner' and allowing renaming.
REGISTER_NAME has an "owner" field which can be different from the actual
registrant (transaction creator's public key, used for signing transaction).

This allowed people to register names to be owned by someone else, thus breaking
the whole "one name per account" aspect.

So now "owner" is removed from REGISTER_NAME, and the actual owner address is
derived from transaction creator's public key, as you would expect.

Similarly, UPDATE_NAME has a corresponding "newOwner" field which has been removed.

In addition, UPDATE_NAME now allows users to change their registered name using a new
"newName" field.

Various changes made to DB, Name class, etc. to accomodate above, along with some minor
bug-fixes and comment improvements/corrections.

Needs new unit tests to cover both new functionality and old!
2020-05-13 10:19:56 +01:00
catbref
f29ae656b9 More work on CHAT
Always add group 0 info to output of API call GET /chats/active/{address}.
No groupName entry as it's "no group" or "group-less" or "not group related".
Timestamp also might be omitted if no message found.

Fix output of POST /chats/compute so it doesn't include zeroed 64-byte signature.
2020-05-12 20:27:09 +01:00
catbref
a9852e5305 More work on CHAT API / support.
Renamed GET /chats/search to /chats/messages.

Added GET /chats/active/{address} to return lists of group chats
and direct chats involving {address}, where a chat message exists.
2020-05-12 14:28:41 +01:00
catbref
32470fa641 Improve CHAT API and repository support.
Change CHAT API call GET /chat/search to better support the two
main scenarios of:

group-based chatting: supply txGroupId only
private chatting: supply 2 'involving' addresses only

Added some DB indexes to cater for above.

GET /chat/search now returns specialized ChatMessage objects
instead of ChatTransactions. This is to reduce unnecessary fetching
of data from repository, and onward sending to API client.
2020-05-12 10:02:41 +01:00
catbref
0d1c08bf96 Add index on DB Names.owner to help find names by owner 2020-05-12 10:02:26 +01:00
catbref
026c904ce4 Change processing of network TRANSACTION_SIGNATURES message.
Previously Controller would loop through the transaction signatures,
discard those already known, and then requesting the full transaction
via peer.getResponse(). This would tie up a networking thread for some
time and also potentially cause repository deadlocks, although the latter
could have been fixed another way.

However, the code after peer.getResponse() was identical to the code
processing an incoming TRANSACTION message. Now instead of requesting
and waiting for then processing each transaction, Controller simply
sends the peer a GET_TRANSACTION for each unknown transaction signature.

As the peer responds with corresponding TRANSACTION messages, these can
be processed individually with shorter period of locking.
2020-05-12 08:03:11 +01:00
catbref
59ae070c83 Fix API call POST /peers so it returns "false" for existing peer, instead of throwing / Internal Server Error 2020-05-11 12:59:56 +01:00
catbref
2ab695f308 NTP: don't call shutdownNow() on null instanceExecutor
When using fixed NTP offset, e.g. via "testNtpoffset" in settings.json,
Controller calls NTP.shutdownNow() which throws a NPE because
NTP.instanceExecutor is null.
2020-05-11 12:59:45 +01:00
catbref
f0ff77cd31 Fix ChatTransaction w.r.t. txGroupId meaning. Relax no-QORT PoW difficulty. 2020-05-11 12:58:46 +01:00
catbref
e241d9fa67 Split CHAT compute into separate API call to help UI 2020-05-11 12:58:05 +01:00
catbref
5e9b0cd03c Fix GroupInvites.expires_when in DB to re-allow NULL 2020-05-11 12:55:58 +01:00
catbref
a5c437913f Transaction.isValidTxGroupId() changed from private to protected - needed so ChatTransaction can override 2020-05-11 12:55:22 +01:00
catbref
3fa7da5115 Fix for incorrect blocksMinted count. Added test to cover 2020-05-08 08:51:56 +01:00
catbref
6d8f41ab05 Fix long overflow in Block.distributeBlockRewardToQoraHolders()
Sadly no native 128bit integer support in Java 11 so resorting to using
BigInteger.

Added/improved unit tests to cover.
2020-05-07 16:37:40 +01:00
catbref
3094ec3c26 Massive clean-up of DB & conversion to long for timestamps
Collated all development changes to DB so now we build
initial DB structure directly with final layout.
i.e. no ALTER TABLE, etc.

Reordered HSQLDB 'CREATE TYPE' statements into alphabetical order
for easier maintainability.

Replaced TIMESTAMP WITH TIME ZONE with simple BIGINT ("EpochMillis").
Timezone conversion is now a presentation task, rather than having
pretty values in database.

Removed associated conversion methods, like toOffsetDateTime(),
fromOffsetDateTime() and getZonedTimestampMilli().

Renamed some DB columns to make them more obviously timestamps, like:
Names.registered is now Names.registered_when.

Removed IFNULL(balance, 0) from HSQLDBAccountRepository as balances
are never null, or actually never 0 either.

Added more tests to increase API call, and hence repository, coverage.

Removed unused "milestone block" from Transactions.
2020-05-07 12:16:22 +01:00
catbref
359a35931e Fix broken Transaction.getUnconfirmedTransactions() and getInvalidTransactions() 2020-05-07 12:15:09 +01:00
catbref
9e0001c4f6 Improved comments, variable names, etc. for some repository interfaces 2020-05-07 10:22:44 +01:00
catbref
53112709fe Improve comment & tidy annotation in GroupData 2020-05-07 10:11:35 +01:00
catbref
74b5401e84 Merge chain-stall, blocksMinted and other fixes 2020-05-06 08:01:51 +01:00
catbref
d2559f36ce Fix for Block not correctly adjusting accounts' blocksMinted values.
Added BlocksMintedCountTests to cover above.
2020-05-05 16:10:54 +01:00
catbref
0cc9cd728e Fix for chain-stall relating to freshly cancelled reward-shares.
In some cases, a freshly cancelled reward-share could still have
an associated signed timestamp. Block.mint() failed to spot this
and used an incorrect "online account" index when building the
to-be-minted block.

Block.mint() now checks that AccountRepository.getRewardShareIndex()
doesn't return null, i.e. indicating that the associated reward-share
for that "online account" no longer exists.

In turn, AccountRepository.getRewardShareIndex() didn't fulfill its
contract of returning null when the passed public key wasn't present
in the repository. So this method has been corrected also.

AccountRepository.rewardShareExists(byte[] publicKey) : boolean added.

BlockMinter had another bug where it didn't check the return from
Block.remint() for null properly. This has been fixed.

BlockMinter now has additional logging, with cool-off to prevent log
spam, for situations where minting could not happen.

Unit test (DisagreementTests) added to cover cancelled reward-share
case above. BlockMinter testing support slightly modified to help.
2020-05-05 11:09:46 +01:00
catbref
e5cf76f3e0 Replace throwing IllegalStateException with more defensive log & null in Block/BlockMinter 2020-05-04 15:50:10 +01:00
catbref
44e8b3e6e7 Log when BlockMinter fails to acquire blockchain lock after waiting 2020-05-04 14:33:10 +01:00
catbref
1bca152d9c Reduce minBlockchainPeers for now 2020-05-04 14:32:42 +01:00
catbref
4edc3ee121 Fix AT transaction reference lookup/generation in light of new last-ref scheme 2020-05-04 09:32:18 +01:00
catbref
e9f29767c8 Change Transaction.countUnconfirmedByCreator() to disregard CHAT transactions.
This is because CHAT transactions have intrinsic anti-spam/DoS prevention
by requiring proof of work.
2020-05-04 09:29:07 +01:00
catbref
e2916b130b No need to store repository handle in Block$ExpandedAccount 2020-05-04 09:19:17 +01:00
catbref
538e117abd Clean up Transaction.isStillValidUnconfirmed()
Brought more into line with isValidUnconfirmed().
No need to update creator's lastReference under new last-ref scheme.

Correspondingly, no need to acquire blockchain lock or repository
shenanigans in getUnconfirmedTransactions() and getInvalidTransactions()
for the same reason.

getInvalidTransactions() seems to be unused and may well be cleaned up
in a future commit.
2020-05-04 09:14:52 +01:00
catbref
71e80bd02f Convert to Account.modifyAssetBalance()
Change code of the form (assetId aspect not shown):

account.setConfirmedBalance( account.getConfirmedBalance(), amount )

to:

account.modifyAssetBalance( amount )

Also tidied "0 - value" to use unary negate: "- value"
2020-05-04 08:45:31 +01:00
catbref
800103225b Remove pointless "return" in DeployAtTransaction 2020-05-04 08:18:59 +01:00
catbref
cfb7a3cc4c Minor terminology correction in GenesisBlock 2020-05-04 08:18:33 +01:00
catbref
3185cf23df Replace throwing IllegalStateException with more defensive log & null in Block/BlockMinter 2020-05-04 08:18:15 +01:00
catbref
3ac1b36549 Restrict API call POST /chat to prevent CPU abuse 2020-05-04 08:17:05 +01:00
catbref
55e99062ca CHAT PoW difficulty now much greater if sender has no QORT balance 2020-05-01 11:01:52 +01:00
catbref
edb56b74da Merge branch 'chat' into launch 2020-05-01 10:51:38 +01:00
catbref
233ace23de AT transactions now either have null message or null amount&assetId.
AT transaction transformer changed to refuse to deserialize AT transactions,
as they should never appear on the wire.

Ditto for GENESIS transactions.
2020-05-01 10:42:32 +01:00
catbref
e1f3b9a7a3 Update QortalATAPI.putTransactionAfterTimestampIntoA() to use Transaction.getRecipientAddresses 2020-05-01 10:20:25 +01:00
catbref
6be88ac86e In BTCACCT, use Account to fetch balance instead of direct from DB 2020-05-01 10:19:28 +01:00
catbref
d03cca2e76 Merge branch 'BTC-ACCT' into launch 2020-05-01 10:09:54 +01:00
catbref
e86143426b Fix potentially overflowing multiply in Block reward processing.
Change BlockChain config to use AmountTypeAdapter instead of
creating duplicated long versions of BigDecimal values.

Some tidying to Amounts class.
2020-05-01 08:57:15 +01:00
catbref
a309f8de9e Fix dead code warning in Account 2020-05-01 08:56:49 +01:00
catbref
df15f81b9f Fix whitespace 2020-05-01 08:56:27 +01:00
catbref
6ab50e4dff Fix some SonarLint complaints 2020-05-01 08:56:03 +01:00
catbref
476d9e4c95 Converted tests from BigDecimal to long
Moved Asset.MULTIPLIER, etc. to Amounts class.

Had to reintroduce BigInteger for asset trading code.
Various helper methods added to Amounts class.

Payment.process/orphan no longer needs unused transaction
signature or reference.

Added post block process/orphan tidying, which currently deletes zero account balances to satisfy post-orphan checks in unit tests.

Fix for possible bug when orphaning TRANSFER_PRIVS.

Added RewardSharePercentTypeAdapter like AmountTypeAdapter.

Replaced a whole load of JAXB-special getters with type-adapters.

Tests looking good!
2020-05-01 08:41:35 +01:00
catbref
9eaf31707a Massive conversion from BigDecimal to long.
Now possible thanks to removing Qora v1 support.

Maximum asset quantities now unified to 10_000_000_000,
to 8 decimal places, removing prior 10 billion billion
indivisible maximum.

All values can now fit into a 64bit long.
(Except maybe when processing asset trades).

Added a general-use JAXB AmountTypeAdapter for converting
amounts to/from String/long.

Asset trading engine split into more methods for easier
readability.

Switched to using FIXED founder block reward distribution code,
ready for launch.

In HSQLDBDatabaseUpdates,
QortalAmount changed from DECIMAL(27, 0) to BIGINT
RewardSharePercent added to replace DECIMAL(5,2) with INT

Ripped out unused Transaction.isInvolved and Transaction.getAmount
in all subclasses.

Changed
  Transaction.getRecipientAccounts() : List<Account>
to
  Transaction.getRecipientAddresses() : List<String>
as only addresses are ever used.

Corrected returned values for above getRecipientAddresses() for
some transaction subclasses.

Added some account caching to some transactions to reduce repeated
loads during validation and then processing.

Transaction transformers:

Changed serialization of asset amounts from using 12 bytes to
now standard 8 byte long.

Updated transaction 'layouts' to reflect new sizes.

RewardShareTransactionTransformer still uses 8byte long to represent
reward share percent.

Updated some unit tests - more work needed!
2020-05-01 08:40:32 +01:00
catbref
e0007269b9 Initial attempt at transient CHAT transaction type.
CHAT transactions don't ever get included into a block.
They use a memory-intensive proof-of-work instead of a fee.
Reference field isn't checked but must be present.
Recipient is optional.
isText/isEncrypted as per MESSAGE, basically indicative flags only.

Some API support.

Memory PoW takes roughly 800ms on Ryzen 3600, maybe 2400ms on QORTector?
2020-04-28 16:45:09 +01:00
catbref
0006911e0a Account lastReference cache, now with Block support.
As this changes how lastReferences are checked and updated,
this is not suitable for rolling into current chain without a
"feature trigger", or chain restart!

Added unit tests.
2020-04-27 16:07:00 +01:00
catbref
e141e98ecc Interim commit with new AccountRefCache, but no tests and no Block support 2020-04-27 16:07:00 +01:00
catbref
40531284dd Convert old "genesis account" addresses in blockchain configs to new "null account" address 2020-04-27 16:06:47 +01:00
catbref
9e2663b11b Fixed old Qora v1 "GenesisAccount" by replacing with NullAccount
NullAccount has 'empty' public key (32 bytes of zeros) compared
with GenesisAccount's vague sometimes 8 bytes, sometimes 32 bytes
public key.

NullAccount has static public key and address, plus overridden
methods to speed up pointless calls like verify().

Genesis Block also tidied up, dropping old Qora v1 compatibility
and using proper block signature and public key to generate
minter's block signature.

Genesis Block transaction processing also simplified, with no need
to access repository to handle fake references, due to new
last-reference code (which will need to be merged).

Dropped support for old, broken RMD160 code.
2020-04-27 16:06:47 +01:00
catbref
2602bb01e1 Limit both divisible & indivisible asset quantites to 1e10 (with 8dp) to fit into 8byte long 2020-04-27 16:04:19 +01:00
catbref
ac15dfe789 Removed broken MD160 support in Crypto 2020-04-27 15:33:23 +01:00
catbref
cd066cf357 Remove more old Qora v1 compatibility code 2020-04-27 15:33:23 +01:00
catbref
bd521baade Removed code for providing compatibility with Qora v1
Qortal is never going to continue off the old Qora blockchain,
so removed all code regarding compatibility.

Removals include:
* various blockchain "feature triggers"
* special Qora-only broken code for various transaction signatures
* "old" asset pricing / trading
* pre-group txGroupId field in transactions
* compatibility unit tests

Possibly safe for roll-out on pre-genesis blockchain?
2020-04-27 15:33:23 +01:00
catbref
136188339d Combined account balance fixes needed for unit tests 2020-04-27 15:33:09 +01:00
catbref
48de33fe24 More informative error messages when parsing blockchain config 2020-04-27 15:26:55 +01:00
catbref
df4798e2a1 Networking improvements: cached known peers, fewer DB accesses, EPC spawn-failure hook 2020-04-24 16:57:20 +01:00
catbref
edb842f0d1 Correct field ordering in layout docs for TRANSFER_ASSET 2020-04-24 15:31:41 +01:00
catbref
b563fe567d Add missing field (poll owner's address) to layout docs for CREATE_POLL 2020-04-24 15:31:02 +01:00
catbref
b3dd0d89df Add missing field (group owner's address) to layout docs for CREATE_GROUP 2020-04-24 15:07:19 +01:00
catbref
f1e4528581 Fix Transaction.calcRecommendedFee() 2020-04-24 09:40:43 +01:00
catbref
1375372380 Added tests to cover validity checks on group min/max block delay values 2020-04-23 17:06:23 +01:00
catbref
a7d0ad27b1 Legacy QORA block reward fix
If there's no more unrewarded legacy QORA held,
then quickly return from Block.distributeBlockRewardToQoraHolders()
instead of causing divide-by-zero.
2020-04-23 17:00:08 +01:00
catbref
833a785996 More work on Bitcoin-side of cross-chain trading.
Tidied up duplicated cross-chain API code that
fetched Qortal AT info.

Added Bitcoin-related cross-chain API calls
for building, checking, refunding and redeeming
P2SH.

Added new Bitcoin-related API error codes.

Controller now starts up, and shuts down, bitcoinj.

Speed-up in BTC class so bitcoinj doesn't have
to throw away all peers and rediscover & reconnect
to them with every chain-related call.
2020-04-23 09:13:32 +01:00