Commit Graph

64 Commits

Author SHA1 Message Date
CalDescent
a320bea68a Limit the amount of data that can be stored per name.
This is calculated by the total capacity divided by the number of names the node follows. The idea here is that a single content creator can't upload terabytes of data and consume all the space on their followers' nodes. They can only use a proportion, with equal space given to each followed name. And since the limit is dynamic, following more names reduces the allocation to existing names.
2021-12-04 13:33:45 +00:00
CalDescent
b7ee00fb22 Fixed errors in Admin API tests due to failing authentication. 2021-11-27 20:08:59 +00:00
CalDescent
4b8bcd265b Make sure unit test use a different lists directory, and delete it before and after each test. 2021-11-27 18:05:25 +00:00
CalDescent
ae0f01d326 Added storage policy unit tests 2021-11-24 11:02:54 +00:00
CalDescent
caf163f98c Include "tempDataPath" in test settings so that tests don't put files in the main temp directory. 2021-11-12 17:46:48 +00:00
CalDescent
8630f3be96 Added first end-to-end test of data storage 2021-10-24 19:08:09 +01:00
CalDescent
04aabe0921 Include the original file instead of a patch if the patch is larger than the original file.
This saves processing and disk space, as there is no point in applying a patch when the original file is smaller and can be included in its entirety.
2021-10-24 12:02:09 +01:00
CalDescent
8dd4d71d75 Significant rework of patches
- The "diff type" is now specified per file, allowing for different diff methods in each modified file.
- Patches will only be created when both the before and after files are less than 100kiB in size.
- Patches are validated after creation, and if invalid it will fall back to including the entire file.

This has identified a bug where patching fails for files without trailing newline characters, which still needs to be fixed. Until then, it will fall back to including the entire file in these cases.
2021-10-24 10:47:47 +01:00
CalDescent
12b3267d5c Added arbitrary data merge tests. 2021-10-22 11:49:15 +01:00
CalDescent
f3ef112297 Merge remote-tracking branch 'qortal/master'
# Conflicts:
#	.gitignore
#	pom.xml
#	src/main/java/org/qortal/controller/Controller.java
#	src/main/java/org/qortal/gui/SysTray.java
#	src/main/java/org/qortal/settings/Settings.java
#	src/main/resources/i18n/ApiError_en.properties
#	src/test/java/org/qortal/test/CryptoTests.java
#	src/test/resources/test-settings-v2.json
2021-10-15 09:03:28 +01:00
CalDescent
7375357b11 Added bootstrap tests
This involved adding a feature to the test suite in include the option of using a repository located on disk rather than in memory. Also moved the bootstrap compression/extraction working directories to temporary folders.
2021-10-01 07:44:33 +01:00
CalDescent
0a4479fe9e Initial implementation of automatic bootstrapping
Currently supports block archive (i.e. full) bootstraps only. Still need to add support for top-only bootstraps.
2021-09-28 20:17:19 +01:00
CalDescent
8926d2a73c Rework of import/export process.
- Adds support for minting accounts as well as trade bot states
- Includes automatic import of both types on node startup, and automatic export on node shutdown
- Retains legacy trade bot states in a separate "TradeBotStatesArchive.json" file, whilst keeping the current active ones in "TradeBotStates.json". This prevents states being re-imported after they have been removed, but still keeps a copy of the data in case a key is ever needed.
- Uses indentation in the JSON files for easier readability.
2021-09-28 09:24:21 +01:00
CalDescent
74ea2a847d Added unit tests for trimming, pruning, and archiving. 2021-09-24 08:37:36 +01:00
CalDescent
e1feb46de9 Put test data in a separate folder to real data. 2021-08-17 09:08:03 +01:00
CalDescent
0f1927b4b1 Take a hash of the previous state's directory structure and file contents, and then include that hash in the patch metadata (when creating a new patch). This allows the integrity of the layers to be validated as each one is applied. 2021-08-17 08:43:08 +01:00
CalDescent
241e2bef85 Merge branch 'master' into chain-weight-consensus
# Conflicts:
#	src/main/java/org/qortal/block/BlockChain.java
#	src/main/resources/blockchain.json
#	src/test/resources/test-chain-v2-founder-rewards.json
#	src/test/resources/test-chain-v2-leftover-reward.json
#	src/test/resources/test-chain-v2-minting.json
#	src/test/resources/test-chain-v2-qora-holder-extremes.json
#	src/test/resources/test-chain-v2-qora-holder.json
#	src/test/resources/test-chain-v2-reward-scaling.json
#	src/test/resources/test-chain-v2.json
2021-05-02 18:18:20 +01:00
CalDescent
fde68dc598 Added unit test to test level 1 and 2 rewards.
1. Assign 3 minters (one founder, one level 1, one level 2)
2. Mint a block after the shareBinFix, ensuring that level 1 and 2 are being rewarded evenly from the same share bin.
3. Orphan the block and ensure the rewards are reversed.
4. Orphan two more blocks, each time checking that the balances are being reduced in accordance with the pre-shareBinFix mapping.
2021-03-16 09:11:49 +00:00
CalDescent
847e81e95c Fixed a mapping issue in Block->getShareBins(), to take effect at some future (undecided) height.
Post trigger, account levels will map correctly to share bins, subtracting 1 to account for the 0th element of the shareBinsByLevel array.
Pre-trigger, the legacy mapping will remain in effect.
2021-03-12 19:48:49 +00:00
catbref
91925cf931 Change block "minter" signature code, to take effect at some future (undecided) height.
Post trigger, this change will use all 128 bytes of previous block's signature when
calculating/validating next block's "minter" signature (itself the first 64 bytes of a block signature).

Prior to trigger, current behaviour is to only use first 64 bytes of previous block's
signature, which doesn't encompass transactions signature.

New block sig code should help reduce forking and help improve transactional
security.

Added "newBlockSigHeight" to blockchain.json but initially set to block 999999
pending decision on when to merge, auto-update, go-live, etc.
2021-02-20 12:08:51 +00:00
catbref
6cd86d86a6 Add "atFindNextTransactionFix" feature-trigger to all test blockchain configs 2021-02-13 11:24:26 +00:00
catbref
9b0e88ca87 Only compare same number of blocks when comparing peer chains 2021-02-06 11:40:29 +00:00
catbref
2ffd0770c6 Initial Litecoin support
Added altcoinj library as Maven dependency.

Added new Litecoin subclass of Bitcoiny,
with mainnet and testnet ElectrumX server lists.

Added litecoinNet settings variable and getter.

Added LitecoinTests.
Most tests work but testFindHtclSecret()
needs a redeemed HTLC on chain (not yet done).

Added litecoinNet to some test settings files
in resources.

Added Litecoin BuildHTLC, Refund test apps.

Added SendLTC app as Electrum-LTC seems a bit flaky?

So far managed to build HTLC P2SH, fund it and then
refund it!

---

As Bitcoin and Litecoin are both subclasses of Bitcoiny,
could unify some test apps with added Bitcoin/Litecoin
switch as first arg?
2020-09-15 17:53:54 +01:00
catbref
5ffddd0169 Changes to block reward distribution
Any reward leftover from ditributing to legacy QORA holders is reallocated to either:
founders if any online
or
account-level-based reward candidates, if no founders online

We should get pretty close to 100% block reward distribution, barring rounding artifacts.

More documentation and tests.

Removed BlockChain's founderShare as it is calculated in Block on a per-block basis instead.
2020-06-02 10:42:45 +01:00
catbref
2493d5f7a8 Fix wrong test blockchain config being used for legacy qora holder testing 2020-06-01 16:44:56 +01:00
catbref
f1638aa9d9 Removed "owner" from CREATE_GROUP and added Unicode homoglyph support.
Group owner now derived from CREATE_GROUP transaction creator's public key.

Added 'reduced' group name to GroupData, with corresponding change to DB.
Renamed GroupData.getIsOpen() to simply isOpen().

Tidied up CreateGroupTransactionData, adding 'reduced' group name.
Renamed getIsOpen() to simply isOpen().
Added code to generated reduced group name when building genesis block.

Added Group.MIN_NAME_SIZE of 3.

DB tables changed to add reduced_group_name where appropriate,
removing owner where necessary.

Added GroupRepository.reducedGroupNameExists(String).

Fixed up test blockchain configs in src/test/resources/test-chain-v2*.json.
2020-05-18 17:27:32 +01:00
catbref
38a2af8cd5 Tidy up Assets by removing 'owner' from ISSUE_ASSET.
Owner now derived from issuer's public key.
Maximum asset name length reduced to 40 characters.

Repository table changes.

"owner" removed from test blockchain configs and "issuerPublicKey" used instead
where applicable.

Some getters in the form of "getIs___()" renamed to simply "is____()".
2020-05-15 16:22:13 +01:00
catbref
5c8bda37d1 Rework BTC class for better startup & shutdown.
Controller no longer starts up BTC support during main startup.
This does mean that BTC startup is deferred until first BTC-related
action, and that the first BTC-related action will take much longer
to complete.

Added tests to cover startup/shutdown.

This also fixes splash logo stuck on-screen and broken Controller
shutdown when using REGTEST bitcoin network AND there is no
local regtest bitcoin server running.
2020-05-14 12:52:26 +01:00
catbref
6d8f41ab05 Fix long overflow in Block.distributeBlockRewardToQoraHolders()
Sadly no native 128bit integer support in Java 11 so resorting to using
BigInteger.

Added/improved unit tests to cover.
2020-05-07 16:37:40 +01:00
catbref
d03cca2e76 Merge branch 'BTC-ACCT' into launch 2020-05-01 10:09:54 +01:00
catbref
476d9e4c95 Converted tests from BigDecimal to long
Moved Asset.MULTIPLIER, etc. to Amounts class.

Had to reintroduce BigInteger for asset trading code.
Various helper methods added to Amounts class.

Payment.process/orphan no longer needs unused transaction
signature or reference.

Added post block process/orphan tidying, which currently deletes zero account balances to satisfy post-orphan checks in unit tests.

Fix for possible bug when orphaning TRANSFER_PRIVS.

Added RewardSharePercentTypeAdapter like AmountTypeAdapter.

Replaced a whole load of JAXB-special getters with type-adapters.

Tests looking good!
2020-05-01 08:41:35 +01:00
catbref
40531284dd Convert old "genesis account" addresses in blockchain configs to new "null account" address 2020-04-27 16:06:47 +01:00
catbref
bd521baade Removed code for providing compatibility with Qora v1
Qortal is never going to continue off the old Qora blockchain,
so removed all code regarding compatibility.

Removals include:
* various blockchain "feature triggers"
* special Qora-only broken code for various transaction signatures
* "old" asset pricing / trading
* pre-group txGroupId field in transactions
* compatibility unit tests

Possibly safe for roll-out on pre-genesis blockchain?
2020-04-27 15:33:23 +01:00
catbref
1375372380 Added tests to cover validity checks on group min/max block delay values 2020-04-23 17:06:23 +01:00
catbref
833a785996 More work on Bitcoin-side of cross-chain trading.
Tidied up duplicated cross-chain API code that
fetched Qortal AT info.

Added Bitcoin-related cross-chain API calls
for building, checking, refunding and redeeming
P2SH.

Added new Bitcoin-related API error codes.

Controller now starts up, and shuts down, bitcoinj.

Speed-up in BTC class so bitcoinj doesn't have
to throw away all peers and rediscover & reconnect
to them with every chain-related call.
2020-04-23 09:13:32 +01:00
catbref
98506a038b Loads of work on CIYAM AT support, including BTC-QORT cross-chain trading.
We require AT v1.3.4 now!

Updated AT-related logging.

Added "isInitial" flag to AT state data so that state data created at
deployment is not added to serialized block data.

Updated BTC-QORT AT code and tests to cover various scenarios.

Added missing 'testNtpOffset' to various test versions of 'settings.json'.
Added missing 'ciyamAtSettings' to various test blockchain configs.

Loads of AT-related additions/fixes/etc. to core code, e.g Block
2020-04-23 09:13:32 +01:00
catbref
f7e2ee383e More complete testing of block reward distribution to founders 2020-03-18 18:03:56 +00:00
catbref
e0e9673837 Massive refactor to change 'qora' references to 'qortal'.
Blockchain configs will need "v2Timestamp" feature trigger renaming to "qortalTimestamp"!

Due to Controller.VERSION_PREFIX changing, peer-to-peer protocol version detection has
been changed. Was previous a substring match, now we test peers's buildTimestamp is at
least Peer.V2_PROTOCOL_TIMESTAMP_THRESHOLD. Changes in Peer.java.

Also added comment and throw() to QortalATAPI.getNextTransactionTimestamp()
as this needs given the change to Qortal's block timestamps from legacy Qora.

Changes to HSQLDB data types, e.g. QoraAddress to QortalAddress, means existing
database will need to be thrown away after this commit!
2020-02-04 12:11:37 +00:00
catbref
5cd35e07d0 Fix off-by-one error when reducing account level during block orphaning.
Added test to cover above.

Modified test-chain-v2.json so Dilbert starts with level 5, not 8,
to reduce number of blocks minted during tests.
2020-01-21 13:34:46 +00:00
catbref
30df320e7f Redoing account balances with block height.
*** WARNING ***
Possible block reward bug in this commit. Further investigation needed.

Reverted AccountBalances back to height-less form.
Added HistoricAccountBalances table that is populated via trigger on AccountBalances.
This saves work when performing common requests for latest/confirmed balances,
shunting the extra work to when requesting height-related account balances.

Unified API call GET /addresses/balance/{address} by having address/assetId/height as
query params.

Simpler call for fetching legacy QORA holders during block rewarding.

Improved SQL for fetching asset balances, in all conditions,
e.g. with/without filtering addresses, with/without filtering assetIds,
etc.

Unit test for above to make sure query execution is fast enough.
(At one point, some SQL query was taking 6 seconds!)

Added optional 'height' Integer to AccountBalanceData, but this
is not populated/used very often.

HSQLDBAccountRepository.save(AccountBalanceData) now checks zero balance saves
to see if the row can be deleted instead. This fixes a lot of unhappy tests
that complain that there are residual account balance rows left after
post-test orphaning back to genesis block.

Yet more tests.
Removed very old 'TransactionTests' which are mostly covered in more specific tests elsewhere.
Added cancel-sell-name test from above.

Fixed AssetsApiTests to check for QORT not QORA!

Changed hard-coded assetIDs in test.common.AssetUtils in light of new LEGACY_QORA & QORT_FROM_QORA genesis assets.

Some test blockchain config changes.
2019-11-08 17:30:09 +00:00
catbref
31cbc1f15b Account assets balances now height-dependant. QORT-from-QORA block reward fixes. 2019-11-07 15:53:37 +00:00
catbref
00aee1458e Higher account levels more likely to win blocks
Also:

RewardShareKeys app now supports only one arg (minter private key)
in self-reward-share mode, where recipient public key is derived
from minter private key.

Added methods to Account for returning 'effective' minting level
where minting level for founders is read from blockchain config.
(Or returns zero if unable to mint).

Changed two Block constructors into static methods that return
a new Block as there was way too much work being done to really
be called a constructor, especially with all the opportunities
to throw an exception too.

Main blockchain config updated to reflect near-launch version.

Added/changed blockchain weight tests to check block winning
based on higher account levels.
2019-11-06 09:47:10 +00:00
catbref
ebc2ee6ea9 Rework distributing block reward to legacy qora holders
Fix related unit tests.

Fix assetID support in GENESIS transactions.
2019-11-01 10:01:31 +00:00
catbref
fef16e7620 Self-reward-share tests and fixes.
Added Transaction.isFeeValid() to allow transaction subclasses to override and allow zero fees, etc.

Added tests to cover self-reward-shares, including zero fee scenario.

Set 'dilbert' test account to level 8 in test genesis block.

Removed leftover mention of "previous_level" from HSQLDBAccountLevelTransactions,
and AccountLevelTransactionData. (Previous level makes no sense as ACCOUNT_LEVEL
transactions are genesis-block only).

Fixed some incorrect uses of PrivateKeyAccount.getSharedSecret() to
PrivateKeyAccount.getRewardSharePrivateKey() in tests.
2019-10-30 10:23:59 +00:00
catbref
491e79b8e6 ProxyForging->RewardShare massive refactor & more...
Unified terminology from block "generator", "forger", etc. to "minter"

Unified terminology "proxy forger" to "reward share" as it was
incorrect, or at least highly ambiguous, which account had which role.

AccountRepository.findRewardShares() has different arg order!

Account.canMint() now returns true if account has 'founder' flag set.
Added Account.canRewardShare() which returns whether acocunt can create
a reward-share (e.g. level 5+ or founder).

Fixed HSQLDBAssetRepository.getAllAssets() which had incorrect
resultSet column indexes.

Removed old traces of EnableForging transaction.

ACCOUNT_LEVEL and ACCOUNT_FLAGS (genesis-block-only transaction types)
now set target account's last-reference. This is allow later REWARD_SHARE
transactions in genesis block and post-genesis transactions by that account.

REWARD_SHARE transactions are now FREE, but only if minter is also recipient.
If a self-reward-share already exists, then unless share-percent is zero (to
terminate reward-share), any subsequent self-reward-share is invalid.

Updated SysTray i18n properties file.

BlockChain config file requires 'minAccountLevelToRewardShare' and optional
'minAccountLevelToMint'.

Added potential, but currently unused, memory-hard PoW algorithm.

Fixed/removed/disabled some unit tests.
BlockMinter.generateTestingBlock asks Controller to pretend mintingAccount is 'online'.
More testing needed!
2019-10-29 17:46:55 +00:00
catbref
4330782bb7 Added Ed25519 signature verify test JavaScript
JS uses tonyg/js-nacl libsodium port.

First sig shows disagreement with WhisperSystems-based legacy crypto.
2019-08-16 12:33:21 +01:00
catbref
2889d04633 Fix incorrect orphaning of BUY_NAME transactions.
Orphaning a BUY_NAME transaction would not reinstate the
sale price. Sale price could be nullified by (e.g.) orphaning
a SELL_NAME transaction.

Also added test case to cover above and other test-related support,
e.g. test-mode in NTP.
2019-08-16 11:48:30 +01:00
catbref
915eebb8e5 BlockChain.isTestNet now BlockChain.isTestChain.
Added Settings.isTestNet.

Disabled ArbitraryDataManager for now.
2019-08-02 14:42:10 +01:00
catbref
9af18aad34 Setting proxy forging reward share to 0% removes proxy forging relationship.
Included unit test to cover change.
Modified test blockchain config "test-chain-v2.json" to add maxProxyRelationships.

Added comments to some proxy-related methods in AccountRepository class.
2019-08-02 13:05:56 +01:00
catbref
e42c3e60bc Added mock-up of node management UI 2019-07-23 15:24:36 +01:00