Commit Graph

335 Commits

Author SHA1 Message Date
catbref
07e8e01865 Add check to PROXY_FORGING so that proxy public key always maps to same forger-recipient combo 2019-07-23 10:30:17 +01:00
catbref
d0b45cb696 Proxy forging share to recipient can now be zero 2019-07-23 10:29:53 +01:00
catbref
0259702df2 Fix generating X25519 shared secret.
X25519 shared secrets now match those generated by libsodium.

New tests show that shared secrets are the same using either set
of private+public key combinations.

Changed proxy private key generation from using simple SHA256
of shared secret to using SHA256(shared secret + both public keys).

Added a temporary "BouncyCastle25519" shim class to provide missing
key conversion from Ed25519 to X25519.
2019-07-23 10:29:18 +01:00
catbref
4279ad0673 Fix Javascript shared-secret & proxy forging private key code 2019-07-23 10:20:20 +01:00
catbref
748dddcc32 Add checks to API call POST /admin/forgingaccounts.
Now only accepts private keys for accounts with minting rights
or derives to known proxy forging public key.
2019-07-23 10:15:17 +01:00
catbref
7409c024f6 Include proxy-forged blocks in BlockRepository.countForgedBlocks.
As used during validity check for ENABLE_FORGING transaction.
2019-07-23 10:11:51 +01:00
catbref
ae6b41a893 Improve API call showing block forger summaries to include proxy info 2019-07-23 10:11:21 +01:00
catbref
92e60e2531 Add method to prune old peers from database 2019-07-23 10:10:21 +01:00
catbref
bf8a12d422 Delete old, basic block-explorer 2019-07-23 10:07:55 +01:00
catbref
14686e1e0d Corrected wrong Synchronization enum result 2019-07-23 10:06:16 +01:00
catbref
6a8a0f25c1 Synchronization improvements
Added a finer-grained result (enum) to synchronization to replace
stark boolean result. This allows Controller to decide whether
peer can be retried in the next round (e.g. network issue) or
cooled off for an hour (e.g. peer way too far behind)

Fixed bug with wrong UPDATE_GROUP serialized transaction length.

Added logging to BlockMessage when deserialization to help catch
future bugs.
2019-07-22 18:35:30 +01:00
catbref
1f81784bd6 Improved synchronization logging 2019-07-22 18:27:48 +01:00
catbref
c05fe58676 Add static method to extract generatorSignature from block reference 2019-07-22 18:24:45 +01:00
catbref
ce2dc9b8ad Reduce synchronization attempts from every 10s to every 60s 2019-07-22 18:20:29 +01:00
catbref
1b58d12005 Fix peer data bug where lastConnected was swapped with lastAttempted. 2019-07-22 18:20:04 +01:00
catbref
ad9b42dc5d Install initial peers when creating repository 2019-07-22 18:15:07 +01:00
catbref
6feca1f4d6 Synchronizer improvements & minor fix
Restrict synchronizer to only consider peers at most 60 blocks behind.

Synchronizer will only process a batch of 200 blocks at a time.

Improve debugging message in Peer.
2019-07-22 18:08:06 +01:00
catbref
4b02b7a14f Fix unit tests 2019-07-22 18:05:41 +01:00
catbref
ecdf8212ea Fix PROXY_FORGING transaction length issue 2019-07-22 18:03:17 +01:00
catbref
af22922025 AutoUpdate: added comment about mandatory use of OpenJDK JRE, not Java SE 2019-07-22 18:01:56 +01:00
catbref
c0ae99161b Auto-Update changes
Checking interval increased from 5 seconds to 5 minutes.
(Reduces repeated fails).

System tray icon should show message just prior to applying downloaded update.
2019-07-22 18:01:29 +01:00
catbref
6be9bf9c3c Add mnemonic phrase support to VanityGen 2019-07-22 18:00:47 +01:00
catbref
7a318c9fc7 Proxy forging improvements + account flags fixes
Proxy forging recipient no longer needs a public key on the blockchain
at the point PROXY_FORGING transaction is submitted.

Proxy forging recipient is given a last-reference, if they don't have one,
when they receive their first block rewards.

Split block fees in proxy forging scenario, using same share proportion.

100% proxy sharing is now allowed.

Fixed account flags processing for accounts in genesis block.
2019-07-22 18:00:12 +01:00
catbref
3ffcf50d7c Added TransactionParticipants index 2019-07-22 17:54:16 +01:00
catbref
29428e9450 Add extra entries to blockchain.json, remove HttpsTest, add Vanity Gen
+ minor whitespace / comment / JAXB mods

Was commit e14381d:
working on genesis block
2019-07-22 17:05:10 +01:00
catbref
57be191814 Splash startup & sys tray 2019-07-18 18:46:18 +01:00
catbref
e5b3166df4 Improve synchronization 2019-07-18 18:40:23 +01:00
catbref
4744232a1c Don't try to sync with a peer that only has genesis block
Was 85bc70b:
new testchain timestamp + don't try to sync with a peer that only has genesis block
2019-07-18 18:36:19 +01:00
catbref
675596ed6e auto update debugging 2019-07-18 18:30:30 +01:00
catbref
52ac881db0 auto update checks download against hash in tx + checks against build timestamp 2019-07-18 18:29:58 +01:00
catbref
a49e3f7a4e auto update 2019-07-18 18:28:56 +01:00
catbref
20aa49a1f1 Another attempt at auto-update 2019-07-18 15:50:05 +01:00
catbref
63a36073ec Work on auto-update + repository path + Settings bugfixes
AutoUpdate needs separate logfiles for its process as
log4j2's rolling appender doesn't support locking.
So we create AU_LOGGER in AutoUpdate and set it up
using static { } block.

Added support for starting/querying/stopping auto-update
Windows service.

Changed Controller's public connectionUrl field to
getRepositoryUrl() public static method so it
can incorporate repositoryPath from Settings.

Controller has added support for auto-update on startup.

Fix bugs with Settings not using userPath properly.

Removed obsolete standalone binary classes:
* blockgenerator
* txhex
* v1feeder
2019-07-18 15:26:41 +01:00
catbref
e4482c5ade Initial auto-update support, API improvements, arbitrary tx improvements
Removed all @Produces from API resources as response content type is
sorted by Swagger.

Added API /admin/info for generic node info.

Added API /arbitrary/ endpoints.

Moved arbitrary data storage from ArbitraryTransaction to ArbitraryRepository.

V4 arbitrary transaction signature is based on data's hash.

Original commit was d02f282, and commit message was:
Initial auto-update support, network MAGIC change, arbitrary tx improvements
2019-07-18 14:53:19 +01:00
catbref
06e6802d97 More work on synchronization
Various fixes to synchronization

Added missing code for processing incoming block summaries in Network.

Fixed block summaries serialization and removed references to BlockData.

Fixed bug in transaction transformation where base transaction length
didn't include reference or fee lengths.

Original commit was ebbab7b
2019-07-18 13:59:49 +01:00
catbref
57b982d2fb Block summaries (repository/data/message/synchronizer) + BlockGenerator
Also refactored some tests.

Original commit was 06fe8fc, with commit message:
Initial implementation of random block generator, etc.
2019-07-18 12:15:11 +01:00
catbref
747f5e41cf Bump Jetty to 9.4.17.v20190418 due to CVE-2019-10247
+ remove older version of package-info-maven-plugin
2019-04-30 09:00:30 +01:00
catbref
882d910631 Remove unneeded HSQLDB debugging and tidy some comments 2019-04-30 08:33:33 +01:00
catbref
a316b8a810 Potential HSQLDB deadlock fix
After opening repository connection with RepositoryManager.getRepostory(),
any 'read' from repository (e.g. SELECT) starts the transaction
even though HSQLDB documentation states there are no shared/read locks
in MVCC concurrency model.

The work-around for this is to 'reset' HSQLDB's in-transaction flag
by performing a ROLLBACK (Repository.discardChanges) immediately
after acquiring the blockchain lock (which is used to ringfence
changes that might collide like these).

Also adding an extra check to prevent payments to nonexistent AT
addresses as it touches Transaction.
2019-04-29 15:18:44 +01:00
catbref
b21ef18533 More HSQLDB deadlock debugging 2019-04-29 12:54:34 +01:00
catbref
db77901026 grab latest block directly instead of via height 2019-04-26 15:01:57 +01:00
catbref
b4d0f9ab68 Allow multiple txType in API GET /transactions/search 2019-04-26 09:23:44 +01:00
catbref
b0b74eb48d Fix JAXB annotations for transactions with creatorPublicKey 2019-04-26 09:04:58 +01:00
catbref
d4ce797241 More HSQLDB debugging 2019-04-26 08:31:10 +01:00
catbref
b7687bf326 More HSQLDB debugging 2019-04-26 08:07:57 +01:00
catbref
0296c4bbb1 HSQLDB debugging & optimization for Transaction.isValidUnconfirmed 2019-04-25 18:29:01 +01:00
catbref
51fd029e22 Access to log entries from API & checks for some transaction-based API calls. 2019-04-24 15:02:37 +01:00
catbref
126e651f27 Change sync consensus to favour lower-value block sigs + other changes
API /addresses/{address} now returns lastReference taking unconfirmed into account.

Added DELETE /peers/known to remove all known peers from repository.

Added blockchain locking around Transaction methods like isValidUnconfirmed
as they (temporarily) update account lastReference.

Ditto getInvalidTransactions, etc.
2019-04-24 12:46:50 +01:00
catbref
d33ffee3ba Work on granting forging rights
Move hard-coded forging tiers to blockchain config.
Tests for granting forging rights.

Added API call to list top block forgers.

Fixed typo with Reward[s]ByHeight class name.
2019-04-19 09:55:04 +01:00
catbref
93230e9704 Add API call to list blocks with given generator. +more tests +pad genesis public key 2019-04-17 18:11:16 +01:00
catbref
d1c547f24a Refactor to use BouncyCastle Ed25519/X25519, and more...
Remove old whispersystems, etc. *25519 and use new v1.61 BouncyCastle.

Fix proxy forging private key derivation from X25519 shared secret.
Also include Javascript test version for comparison.

Fix block rewards for proxy forging.

Add extra useful info to API call GET /admin/forgingaccounts.
Fix API response to POST/DELETE /admin/forgingaccounts when
passed invalid private keys.

Added block rewards and account flags to testchain config.

Tests to cover changes above.
2019-04-17 12:32:03 +01:00
catbref
8e74884536 Fixes after rebase from master 2019-04-12 14:41:59 +01:00
catbref
2f6ef32f35 Replace settings-based generator private keys with DB table and API calls 2019-04-12 13:01:26 +01:00
catbref
c9035edd2c Interim proxy minting commit
Added /addresses/proxying to find proxy forging mappings.

Added /addresses/proxykey/{genprivkey}/{recipientpubkey} to calculate proxy private key.

New Block.regenerate factory method to create new Blocks
 but without having to reprocess ATs, etc.

Added support for proxied generator in Block.calcGeneratorsTarget

BlockGenerator now generates and checks new blocks for various generators,
 including proxy generators.

BlockGenerator now uses generator private keys supplied by Settings.
Corresponding changes to Settings to load base58-encoded private keys.

+ minor stuff
2019-04-12 12:57:21 +01:00
catbref
9b859f3efd Interim minting commit with block rewards (untested)
+ minting long-term simulator
+ Maven pom.xml changes for Eclipse IDE
2019-04-12 12:56:12 +01:00
catbref
3c06d358b7 interim commit with proxy forging repository/transaction support
no block validity/generator support yet
2019-04-12 12:53:44 +01:00
catbref
2dc1720af8 Initial support for account flags + tx (genesis account use only atm) 2019-04-12 12:50:10 +01:00
catbref
85acc4d9df Fix incorrect refunds when cancelling asset orders
Also improved asset tests in that they don't use QORA any more
which takes fees/block rewards out of the picture.

More test assets are issued in the genesis block to
accomplish this:

1m TEST issued to Alice
1m OTHER issued to Bob
1m GOLD issued to Alice
2019-04-12 10:38:25 +01:00
catbref
2f51ced5c0 Fix incorrectly applied price improvement refund.
Also fix broken tests which contributed to this bug slipping by.
2019-04-12 08:44:13 +01:00
catbref
c23f55e6a6 Asset trading: refund saving due to price improvement back to initiator
Added test case to cover the above.

Also improve test harness to properly check balances after orphan back to genesis.
2019-04-11 17:47:12 +01:00
catbref
16dab6972c Fix some asset orders incorrectly matching worse prices. 2019-04-10 15:25:16 +01:00
catbref
cfbf5c12bf Supply extra information fields to various asset-related API calls.
e.g. supply "assetName" in JSON for TRANSFER_ASSET transactions

Also supply have/want asset names, amount asset ID/name, price-pair
and creator address in asset orders.

Show CREATE_ASSET_ORDER amount ID/name & price-pair in correct
format depending on whether transaction was placed before/after
'new' asset pricing took effect. (Orders are always in 'new' form).

Change API call /assets/transfers/{assetid}/{address} to
/assets/transfers/{assetid} with optional "address" query param.
2019-04-10 13:09:30 +01:00
catbref
a5e963911d New asset pricing scheme (take 2)
Orders are back to having "amount" and "price".
(No more "unitPrice" or "wantAmount").

Order "amount" is expressed in terms of asset with highest
assetID.
"price" is expressed in (lowest-assetID)/(highest-assetID).

Given an order with two assets, e.g. QORA (0) and GOLD (31),
"amount" is in GOLD (31), "price" is in QORA/GOLD (0/31).

Order's "fulfilled" is in the same asset as "amount".

Yet more tests and debugging.

For simplicity's sake, the change to HSQLDB repository is
assumed to take place when 'new' pricing switch also
occurs.

Don't forget to change "newAssetPricingTimestamp" in
blockchain config JSON file.
2019-04-10 07:18:50 +01:00
catbref
1b45ee85e7 Fix handling of CORS preflight OPTIONS requests.
Typical log entry would be:

2019-04-05 09:47:08 ERROR WadlGeneratorJAXBGrammarGenerator:401 - Failed to generate the schema for the JAX-B elements
com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 4 counts of IllegalAnnotationExceptions
        at com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:91) ~[?:1.8.0_191]
2019-04-05 10:56:38 +01:00
catbref
7026c35e73 For API "GET /assets" add optional query param "includeData" 2019-04-04 12:33:12 +01:00
catbref
97142fdde8 Return HTTP 400 code with useful message when a ParamException occurs from API call. 2019-04-04 12:30:28 +01:00
catbref
315ebff61d Add optional API request logging via "apiLoggingEnabled" entry in Settings 2019-04-04 11:33:13 +01:00
catbref
ed94c3c5b3 Fix Crypto.isValidAddress to return false when passed empty string 2019-04-04 10:41:04 +01:00
catbref
26e3adb92b Completing work on new asset trading changes
Changed API call GET /assets to NOT return asset "data" fields
as they can be huge. If need be, call GET /assets/info to fetch
a specific asset's data field.

Improve asset trade amount granularity, especially for indivisible
assets, under "new" pricing scheme only.

Added corresponding tests for granularity adjustments.

Fix/unify asset order logging text under "old" and "new"
pricing schemes.

Change asset order related API data models so that old "price" is
now "unitPrice" and add new "return" as in amount of want-asset
to receive if have-asset "amount" was fully matched.
(Affects OrderData, CreateAssetOrderTransactionData)

Some changes to the HSQLDB tables.

Don't forget to add "newAssetPricingTimestamp" to your blockchain config's
"featureTriggers" map.
2019-04-03 18:00:20 +01:00
catbref
60e562566e Interim commit on new asset trading schema
Better order matching, especially in situations
where inexact fractional representations (e.g. 1/12)
or rounding issues might occur. Also better matching
with indivisible assets.

Essentially change ordering from have-amount & price
to have-amount and want-return, leaving unit price
to be calculated internally to a finer degree (in
some cases to 48 decimal points).

Corresponding unit tests to cover both legacy and new
scenarios. Support for tests to switch between
blockchain configs.

"New" pricing schema is its own 'feature trigger'
independent from general qorav2 switch.

Safety checks added during trading process.

HSQLDB schema changes (will probably need
careful conflict resolution on merge).

Still to do:

API changes
etc.
2019-04-02 21:10:16 +01:00
catbref
031657878e Added safety feature to prevent negative balances 2019-03-29 10:56:46 +00:00
catbref
789b311984 Interim commit with newer asset order "price" arg
+ unit test
+ newer unit test harness

but still needs:

BlockChain config support for activating newer "price" arg
New unit test to check old "price" arg usage

Rework existing asset-related unit tests

Check API inputs/output pre/post "price" arg crossover
2019-03-28 16:28:31 +00:00
catbref
d3c1602d9b Adjust Asset data max size to reflect HSQLDB CLOB size
+ fix incorrect logic when calculating issue-asset transaction
data length
2019-03-26 14:36:47 +00:00
catbref
d741580ccf Restore missing minimum fee checks & ValidationResult 2019-03-26 14:36:06 +00:00
catbref
c69f84afab Added ApiExceptionManager to try to trace IllegalAnnotationExceptions 2019-03-26 14:34:45 +00:00
catbref
870646fec8 Correct min/max block times for testing blockchain config 2019-03-26 14:32:51 +00:00
catbref
c5e5020369 In API call /assets/balances, change "ordering" to be optional but with default "ASSET_BALANCE_ACCOUNT" 2019-03-25 15:21:10 +00:00
catbref
a4d28d52cc Add variety of results ordering options to API GET /assets/balances 2019-03-25 12:24:22 +00:00
catbref
0fd8ee3547 Add API GET /assets/transfers/{assetid}/{address} 2019-03-25 11:55:03 +00:00
catbref
2a45e27048 Important FIX to payment-related isValid()!
Non-native-coin payments (e.g. transfer-asset) didn't build map
for checking whether sender had enough balance.

Also fixed raw layout description for TRANSFER_ASSET.

Also fixed ISSUE_ASSET deserialization to use empty string (instead
of null) for v1 transactions.
2019-03-25 11:23:18 +00:00
catbref
eb038b8f31 Controller debugging, including logging build version/timestamp 2019-03-25 11:22:57 +00:00
catbref
315a3c33cf Add API error handler to log unexpected errors/exceptions
Fixed /admin/uptime to return text/plain instead of application/json
 as response is only a simple long.
Ditto /admin/stop.
2019-03-19 10:09:01 +00:00
catbref
048c54fc0a Add API call to support TRANSFER_ASSET + activity summary API call 2019-03-18 11:34:46 +00:00
catbref
91ee505ba9 Fix GET /assets/trades/recent regression + silence Jetty timeout warnings 2019-03-17 13:35:24 +00:00
catbref
76277ae9c3 Increase asset "data" size and make description/data optional in UPDATE_ASSET
Bumped Asset data size from 4K to 400K
2019-03-15 11:22:53 +00:00
catbref
1b3fa5c806 Automatically set/unset account's defaultGroupId + bugfix
FIX: somehow GET /groups/member/{address} HSQLDB's query broke
2019-03-14 13:12:20 +00:00
catbref
d53777f461 Added "data" field to assets, added UPDATE_ASSET tx + fixes 2019-03-13 14:06:52 +00:00
catbref
8f72d9d423 Don't return empty balances when requesting all asset holders. 2019-03-11 13:03:57 +00:00
catbref
cdd1f5e966 API modifications, remove blockchain defaultGroupId, Eclipse/git
GET /assets/trades/recent expanded to allow multiple otherAssetIds

When GET /assets/balances is called with address(es) but no assetIDs
then it will return balances for all assets, including zero balances
for assets the addresses don't own.

GET /addresses/{address} no long fakes a default groupID - in fact
defaultGroupId now removed from blockchain config.

Some Eclipse IDE files now hidden/removed from git repo.
2019-03-11 11:17:34 +00:00
catbref
43eec116b5 Reorder handshake to increase compat with v1 nodes + request remote peers 2019-03-06 10:40:39 +00:00
catbref
bf3f78e718 Add AssetOrders is_closed fix-up to DB updates + minor 2019-03-05 11:12:00 +00:00
catbref
4d69242cdb Added/improved assets-related API calls
Imported Block/BlockChain fixes from "minting" branch to do
with block timestamps.

GET /assets/holders/{assetid}
and
GET /assets/address/{address}
and
GET /assets/balance/{assetid}/{address}
all combined into
GET /assets/balances?address=...&address=...&assetid=...&assetid=...

New GET /assets/trades/recent?assetid=...&assetid=...
that returns most recent two trades for each asset-pair.

GET /assets/orders/{address}/{assetid}/{otherassetid} has
includeClosed and includeFulfilled repurposed as
isClosed (true/false/omitted) and isFulfilled (true/false/omitted).

ALSO, Order.isClosed is now set to true when isFulfilled is set to true
during processing (and correspondingly set to false during orphaning).

AccountBalanceData now includes optional assetName field for use with API
but generally not set for internal use.
2019-03-04 18:53:54 +00:00
catbref
752361ecff Performance fix & other minor fixes
Fixed bug in GenesisBlock which wasn't stripping
out ISSUE_ASSET transactions for v1 blockchains.

Updated blockchain.json for v1 qora blockchain to
fall into line with new BlockChain config
unmarshalling code.

Improved TransactionData.equals when comparing
exact same object.

Improved HSQLDBBlockRepository.getHeightFromTimestamp to use OffsetDateTime
object, which includes time zone info, to fix incorrect SQL full-scan
so that DB now does index-scan instead. Also converted use of MAX(column)
to ORDER BY column DESC as MAX() not optimized when MVCC in effect.

Added corresponding INDEX to Blocks on columns (generation, height);

Similar MAX(column) to ORDER BY column DESC improvement for getBlockchainHeight().

Bumped size of HSQLDB TYPE ATTags from VARCHAR(32) to VARCHAR(80)

Improved reflection-based method calls to rethrow underlying exceptions
like DataException, TransformationException, etc. instead of losing
them in general InvocationTargetException.

Re-added atTransaction.toBytes() method so v1 transactions can be verified.

Fixed extraneous additional copy of voter's public key when serializing
in VoteOnPollTransactionTransformer.toBytes()

Fix-up of v1feeder
2019-03-04 11:41:30 +00:00
catbref
edcbf4f318 Add support for "only one registered name per account" + de-static brokenMD160 blockchain flag 2019-02-27 10:34:25 +00:00
catbref
ec5eba9c60 Fix up some group-approval related issues
UPDATE_NAME requires txGroupId to match original txGroupId used in REGISTER_NAME
UPDATE_GROUP requires txGroupId to match original txGroupId used in CREATE_GROUP

Extraneous JAXB unmarshal code removed from various transaction constructors.
Incorrect, old, groupID vs txGroupId checks removed from various transactions.

Fix bug in UPDATE_GROUP fromBase() method using mismatched column indexes.

Fix bug in API's pending transaction fetch not checking admin is of the same
group as the txGroupId. (Fix actually in HSQLDBTransactionRepository).
Similar fix in GroupApprovalTransaction.
2019-02-26 16:52:22 +00:00
catbref
c7123df79d Better logging for settings/blockchain JSON file parsing 2019-02-26 11:20:44 +00:00
catbref
deae03f1a4 Fix blockchain config logic error 2019-02-26 11:07:44 +00:00
catbref
c80ac9e321 Asset API additions, txGroupId minor fix, testnet blockgen fix
GET /assets/orderbook/{assetid}/{otherassetid} renamed to
GET /assets/openorders/{assetid}/{otherassetid}

Replacement /assets/orderbook/{assetid}/{otherassetid} now
returns aggregated orders, with entries containing only
"price" and "unfulfilled" (amount).

Added /assets/orders/{assetid}/{otherassetid}/{address} to return
orders by specific account, for a specific asset-pair.

Block timestamp validity extracted to separate method so that
BlockGenerator can test timestamp and generate blocks at the usual
rate, even for testnets. This still allows testnets to a way to
generate blocks on demand as Block's isValid skips some timestamp
validity checks if testnet.

txGroupId was sometimes incorrectedly checked for approval-less tx types.
2019-02-26 10:56:19 +00:00
catbref
16c1b13ab2 Proper JSON unmarshalling for settings, blockchain config, genesis block
GenesisBlock (v4) now supports various transaction types (issue-asset, etc.)
 with generated signatures (like genesis transaction signature) and
 missing references inserted.

JUnit reverted back to v4 for Eclipse support (for now).
2019-02-25 13:31:05 +00:00
catbref
86a35c3b71 Work on groups
Some dev/testing API calls are now turned off by default in production mode,
 see "restrictApi" settings entry, returning NON_PRODUCTION API error.

Corrections to how account's defaultGroupId works, removing "effective groupID"
 which overly complicated matters.
In relation to above, DEFAULT_GROUP (0) no longer exists and NO_GROUP(-1) now has
 the value 0 instead.
So transactions can no longer have txGroupId of DEFAULT_GROUP, which in turn
 required all the erroneous "effective groupID" code.

API call /addresses/{address} now supplies blockchain-wide defaultGroupId if
 account doesn't exist or if account's default not set and NO-GROUP not allowed.

API /transactions/pending now offloaded to repository instead of Java-based
 processing and filtering.

Transaction approval checks added to Block.isValid

Groups now have min/max approval block delays.
 Checks added to incoming unconfirmed, block generator, block.isValid, etc.

'needing approval' and 'meets approval threshold' now split into separate calls.

NB: settings.json no longer part of git repo
2019-02-20 12:25:30 +00:00
catbref
00656f6724 Interim safety commit due to large number of changes!
log4j2.properties now has debugging entries removed.
log4j2-test.properties (not in repo) takes priority
 so using that in development instead.

Unconfirmed transactions no longer wiped on start-up
 by default - see Settings

Reworking of {Public,Private,Genesis}Accounts as it seemed
 possible to silently lose public key in repository.
The use of AccountData didn't work and so field-specific
 repository calls have been made instead
 (e.g. setLastReference) that try to opportunistically
 store public key too, if available (i.e. caller is
 PublicKeyAccount subclass, or better).

Added API call GET /addresses/{address} to return
 general account info in one go. (Essentially the
 AccountData object as fetched from repository).

Initial work on adding default groupID to accounts,
 along with corresponding SET_GROUP transaction type.
In additional, added blockchain-wide default groupID
 and flag to allow/disallow no-group/groupless
 transactions.

Initial work on group-admin approval of transactions
 tied to a specific group via txGroupId.

More work needed on transaction's "effective txGroupId"!

API call /transactions/pending to list transactions
 pending group-admin approval. However, this needs more
 work (see effective txGroupId above) and potentially
 offloading to HSQLDB repository if possible.

Minor CIYAM AT renames to help static reflection initializers.

Block.orphan() no longer adds orphaned transactions back to
 unconfirmed pile as they are themselves deleted during
 Transaction.orphan(). Maybe the answer is to NOT delete
 them during Transaction.orphan() but to add them to
 unconfirmed pile at that point? Very old transactions
 leftover from major resync would simply expire, whereas
 recently transactions leftover from minor resync could
 still make it into a new block on synced chain fork.

Changes/tidying/improvements to block generator regarding
 removing invalid transactions and dealing with transactions
 pending group approval.

Approval threshold added to groups.

Mass refactoring of transaction-related classes to unify
 constructors, particularly field ordering, to fall in line
 with raw transaction layout.
e.g. constructors now reflect that raw transactions mostly
 start with type, timestamp, txGroupId, publicKey, reference
e.g. JAXB afterUnmarshal methods added where needed and corresponding
 nasty code in Transaction subclass constructors ripped out.
e.g. TransactionTransformer subclasses contain less duplicated code.

Fixed bug with repository save points thanks to swapping to Deque.

Some fixes to do with missing transaction types being passed to JAXB
 TransactionData subclass constructors.

Ripped out obsolete toJSON in TransactionTransformers as this
 is all nicely taken care of by Swagger/OpenAPI (thanks @Kc)
2019-02-18 19:00:37 +00:00
catbref
82e9e1e7dc Improved peer addresses.
Added short sleep() to GET /admin/stop to allow time for HTTP response body to be sent back.

Improved documentation for /peers API resources. Added examples, tidied API output models.
Fixed issue where IPv6 literals with port couldn't be parsed. Now uses RFC5952/RFC3986 style
literal IPv6 addresses with ports, e.g. [::1]:9084

Fixed NPE in Controller.potentiallySynchronize() where peer might not have sent height yet.

Improved Handshake to discard inbound connections if we already have an outbound connection
to a peer with that ID. This prevents us from having two connections to the same peer, one
in each direction.

Network.mergePeers() now runs in a separate thread as acquiring the lock might block.

Network.creationConnection() exits fast based on number of outbound connections, instead of
number of total connections.

Network no longer sends 'local' peer addresses to non-local peers.
e.g. it won't send localhost:9084 to node4.qora.org:9084

Added try-catch to Network.broadcast for when we try to broadcast while shutting down.

Added PeerAddress class to deal with the whole hostname/IPv4/IPv6 address situation.

Reworked PEERS_V2 message type to only send sized-strings instead of separate port,
and potentially IPv6 byte arrays.

Change to HSQLDB database shape.
Corresponding changes to HSQLDBNetworkRepository.
2019-02-12 18:50:49 +00:00
catbref
174a1a5909 Re-assert HSQLDB concurrency control mode 2019-02-11 18:36:29 +00:00
catbref
7a53ac17a6 Synchronization, peer management + fixes
Peers now broadcast height after successful synchronization.

Added support for sending unconfirmed transactions to other peers.
This is done on connect and also after a new unconfirmed transaction is submitted via API.

Fixed synchronizer to handle blocks with transactions correctly.

Fixed network-related PoW to not use class-global SHA256 message digester!
(It was being corrupted by simulataneous access by different threads - whoops)

Surrounded Network.mergePeers with a lock to prevent HSQLDB deadlocks.
Also changed HSQLDB concurrency model to MVCC (only takes effect if DB rebuilt).

Added support for logging other HSQLDB sessions in the event of exception.
(Currently only used by HSQLDBSaver)

Transaction transformer modifications to help deserialize TransactionMessages.
2019-02-11 17:37:52 +00:00
catbref
7f4511cb7b Networking and repository
Some pom.xml changes to reduce maven-shade-plugin conflicting classes warnings.

Repository now supports SAVEPOINT and ROLLBACK TO SAVEPOINT.
HSQLDB concurrency/transaction model changed from LOCKS to MVCC to help with
 transaction deadlocks/rollbacks.

More XXXs and TODOs added to Block.java for investigation/fix/improvements.
Also used new repository SAVEPOINT feature when validating transactions
 instead of rolling back entire transaction. This fixes problem during
 synchronization where the rollback would undo previously synchronized,
 but not yet committed, blocks!

Transactions orphaned by Block.orphan ARE now added to unconfirmed pile,
 unlike before.

Concurrent lock now prevents simultaneous block generation and synchronization,
 including synchronization via multiple peers.

Lots of new networking code: peer lists, block signatures, blocks,
 blockchain synchronization.

PEERS_V2 message now supports hostnames, IPv6 and port numbers.

Fixed bug with block serialization for transport over network.
2019-02-01 14:03:06 +00:00
catbref
0db43451d4 Interim networking code commit in case of dev catastrophe!
DB shape change from v29 to add peer info.
New NetworkRepository to handle above.

Peer handshaking with v2 anti-DoS PoW code.
Handshaking refactored into a state-machine-like enum.
Some peer-related API calls added.

Peers exchange pings, heights, peers.

No actual peer sync yet.

Other changes:

Peer version info taken from Maven build properties/resource file.
AnnotationPostProcessor more resilient when fetching PathItems.
Per-repository session debugging flag that can be toggled at will.
HSQLDBRepository.delete() now returns int so callers can detect
 whether anything was actually deleted.
Some renaming to settings.
2019-01-30 18:24:10 +00:00
catbref
79b3074d01 Reworked reflection code in Transaction static init to appease Maven compiler!? 2019-01-25 15:56:17 +00:00
catbref
6eb3520295 Asset-related transactions API + performance improvements + fix
Moved as much reflection out to class-static initializers as possible.

Renamed some classes to fall in line with transaction type name
to class name conversion, e.g. DEPLOY_AT -> DeployAt
2019-01-25 15:22:56 +00:00
catbref
4be58514c0 Unify API calls that return lists + offload pagination to repository
API calls that return lists now take limit, offset and reverse params.

API calls that used to return data & optional list (e.g. blockWithTransactions)
now only return base data. The optional lists can be fetched via
a different API call.

Also: SLF4J now routes logging to log4j2 so start up output cleaned up.
Suppressed extraneous Jersey warning about Providers during start-up injection.
2019-01-24 16:42:55 +00:00
catbref
782bc2000f Added raw transaction layout API call to help build raw transactions.
Converted unwieldy tx-type switch statements to use reflection.
2019-01-17 09:01:05 +00:00
catbref
6a10df85ab Hide internal-use references from some objects emitted by API 2019-01-16 15:46:35 +00:00
catbref
c81a3838fc Added API call to list registered names that are for sale 2019-01-16 15:37:18 +00:00
catbref
e04f9df0dc Groups API: get invites by group ID or invitee
+ fixed API response models that were incorrectly NOT arrays
2019-01-16 15:22:15 +00:00
catbref
bc4de8f063 Groups: refactored to use group ID, not group name, plus fixes, etc. 2019-01-16 15:00:25 +00:00
catbref
3b4c12da56 Group ban/unban & pre-conversion to groupID & pre-tidy/refactor 2019-01-15 14:41:36 +00:00
catbref
90f1676c7c Account group invite/cancel-invite and kick
Account group join for a group that is closed/invite-only
and has no corresponding invite is now turned into a "join request".
This can be accepted by an admin sending a corresponding invite tx.
2019-01-14 14:36:43 +00:00
catbref
df730d9fb9 Add/remove account group admins (by owner) 2019-01-11 14:07:16 +00:00
catbref
02a620d57b Join/Leave account groups + API calls 2019-01-11 10:42:42 +00:00
catbref
760cb6cd37 JOIN Group + improved specific group info from API 2019-01-10 17:28:13 +00:00
catbref
83abede8ab Account groups: create group + update group + API calls
(Also minor fix for orphan.java).

Note use of afterUnmarshal() in TransactionData-subclass to replace trash code in Transaction-subclass constructor.
See UpdateGroupTransactionData.afterUnmarshal() compared to RegisterNameTransaction constructor.
2019-01-10 15:45:00 +00:00
catbref
74bf930698 API: added support for sell, cancel-sell and buy name transaction builders 2019-01-09 16:03:07 +00:00
catbref
22c87a6e08 API + fix for names in HSQLDB
Added POST /names/update for building an UPDATE-NAME transaction.

BlockGenerator now tries to validate new block after adding each
unconfirmed transaction in turn. If block becomes invalid then
that transaction is removed/skipped. This should further prevent
block jams. Skipped transactions might be deleted as the next block
is forged when unconfirmed transactions are collated/filtered/expired.

Add Block.deleteTransaction() for use during block generation above.

Block.addTransaction() and Block.deleteTransaction() use transaction
signatures to test for presence in Block's existing transactions.

Names shouldn't have stored registrant's public key!
"registrantPublicKey" removed from NameData Java object/bean.
Corresponding column removed from HSQLDB using ALTER TABLE but
also from the original CREATE TABLE definition. Remove the ALTER
TABLE statement just prior to rebuilding database!

(This needs to be applied to Polls too as some point).

Also, UpdateNameTransactions and BuyNameTransactions tables now
allow name_reference to be NULL as this column value isn't set
until the corresponding transactions are processed/added to a
block. (name_reference is a link to previous name-related
transaction that altered Name data like "owner" or "data" so
that name-related transactions can be orphaned/undone).
2019-01-09 14:41:49 +00:00
catbref
95d640cc8c API + new tx restrictions
Added GET /names to list all registered name.
Added GET /names/{name} for more info on a specific name.
Added GET /names/address/{address} for names owned by address.

Renamed GET /assets/all to GET /assets in line with above.

Fixed edge cases with AnnotationPostProcessor.

Fixed incorrectly exposed "blockHeight" in API UI examples/values.
Changed example transaction timestamp.

Added checks on building/signing/processing new transactions via API
so that they are not too old (older than latest block's timestamp),
too new (more than 24 hours in the future) or the tx creator doesn't
already have a lot of existing unconfirmed transactions (default 100).

Configurable via settings.json properties maxUnconfirmedPerAccount
and maxTransactionTimestampFuture.

Improved /transactions/search to not return unconfirmed transactions
and to order by timestamp.

Transaction.getCreator() now returns PublicKeyAccount, not Account.
2019-01-08 17:30:23 +00:00
catbref
7998166c0a API, switchable MD160,
Now uses working RIPE-MD160 by default but can be switched to broken MD160 using flag in blockchain config,
e.g. for Qora v1 blockchain.

Replaced API signature/reference examples with descriptive text as they weren't very useful.
Replaced API address examples with ones generated using working MD160.

Added GET /transactions/signature/{signature}/raw that returns raw transaction in base58 encoding.

Added "ignoreValidityChecks" query param to POST /transactions/decode to bypass INVALID_REFERENCE errors
if supplying an old/speculative transaction that can't be added to unconfirmed transaction pile.

Finally fixed creating inital assets in BlockChain.

Controller now inserts BouncyCastle as highest priority Security Provider.

TransactionData & transaction repository now tries to return transaction's block height in data when possible.
2019-01-07 13:12:42 +00:00
catbref
b2ca63ce88 Add cancel order and get orders by address API support.
Added getAccountsOrders asset repository call to facilitate the above.
2019-01-07 09:31:27 +00:00
catbref
2497ac256c Added API call for creating an asset bid/ask order
Also added test for missing creator public key on API-submitted
transaction creation calls, like /payments/pay or /asset/issue.
(Needs to be an OpenAPI validator added at some point).
2019-01-04 19:10:50 +00:00
catbref
17f3958ad6 Fix ApiService to use new org.qora.* package names
Also add missing default for blockchain config file to Settings
2019-01-04 16:00:37 +00:00
catbref
5c6e239d76 initial work towards OSGi
refactored packages so they all start with org.qora

added some attempt at OSGi mega bundle using Maven (doesn't work)
2019-01-04 10:19:33 +00:00
catbref
9e425d3877 Clean unconfirmed transactions
Invalid/expired unconfirmed transactions are cleaned during various calls,
e.g. requesting list of unconfirmed transactions,
or requesting account's last reference (including considering unconfirmed),
or generating a new block.

BlockGenerator now calls repository.discardChanges before sleep to
release any repository-level locks.

Added settings.json toggle "wipeUnconfirmedOnStart" (default: true)
to aid testing.

REMOVED API call /addresses/lastreference/{address}/unconfirmed as
/addresses/lastreference/{address} now considers unconfirmed
transactions regardless.

Added useful error to /transactions/sign if an invalid private key
is supplied.

Improved API "invalid transaction" error to include actual apsect
that caused validity check to fail (e.g. invalid reference)
2018-12-28 16:37:59 +00:00
catbref
783edb3447 Wipe unconfirmed transactions on startup
Fix issues with last reference and unconfirmed transactions.
2018-12-21 15:27:38 +00:00
catbref
c4ed4b378c Refactoring, new translations, cleaning up warnings.
Refactored to standard Maven layout:
src/main/java
src/main/resources
src/test/java
etc.

New translation code that uses locale-specific ResourceBundles
to load translations on demand.

Reworked API error/exceptions code to a shorter, simpler
@ApiErrors annotation. Processing of @ApiErrors annotations
produces an example for each possible API error and includes
API error string in HTTP response code, e.g.
400 INVALID_SIGNATURE
Missing API error cases added to each API call.

Translation of openAPI.json removed (for now).

block-explorer.html and BIP39 wordlists now read as resources
instead of direct from disk.

Java compile warnings fixed.
Some runtime warnings remain:

WARNING: A provider api.resource.ApiDefinition registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime.
WARNING: A provider api.resource.AnnotationPostProcessor registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime.
WARN org.reflections.Reflections - given scan urls are empty. set urls in the configuration
2018-12-21 11:14:16 +00:00
catbref
aab6b69da1 API: payments, issue asset, transaction decode, etc.
Added slow query check to HSQLDB repository to help
isolate cases where transaction searching takes too long.

Added BigDecimalTypeAdapter for normalizing API inputs
but doesn't seem to get reliably called so also added
.setScale(8) to BigDecimal serialization method.

API-built transactions are now validated before emitting
base58 raw transaction to help callers.

API's transaction decoder accepts signed/unsigned raw transactions.
2018-12-19 09:52:42 +00:00
catbref
107ef93b37 added API support for signing and processing transactions 2018-12-17 17:22:05 +00:00
catbref
963e4c5d35 spawn block generator thread from controller 2018-12-17 14:53:46 +00:00
catbref
999e6b7289 fix incorrect checksum test on bip39 mnemonic decoding 2018-12-17 14:33:33 +00:00
catbref
3045841151 added block timestamp too-soon check and relaxed PoS for too-old blocks 2018-12-17 14:18:23 +00:00
catbref
f50b53a407 Blockchain config (ncluding genesis block) moved out into config file 2018-12-17 13:28:37 +00:00
catbref
68f99cfc11 API: terminology corrections and more utility calls 2018-12-14 13:39:17 +00:00
catbref
e1dbaa5597 BIP39 seed phrase support 2018-12-14 11:26:38 +00:00
catbref
3ec3c69447 API: wholesale conversion back to Base58
XmlJavaTypeAdapter api.Base58TypeAdapter converts byte[] to Base58.
This XmlAdapter is applied at package-level to all packages inside data and api.models.
So no need to annotate every byte[] property!
Added package-info-maven-plugin to pom.xml to do this.

block-explorer.html fixed to show/use base58 again

Some data objects (e.g. TransactionData) have added XmlElements that
convert public keys to addresses, for convenience.

Several API calls updated to return specifically text/plain instead
of ambiguous application/json and/or text/plain. (Typically
API calls that return a single value, e.g. an integer, like /blocks/height).
2018-12-13 17:25:47 +00:00
catbref
034cf5dee3 API: assets & tidying
Cleaned up responses from /addresses/* endpoints
in that some return text/plain instead of application/json.

Removed need for class-local copy of ApiErrorFactory in
AddressesResource - using getInstance() instead.

Some work still needs to be done on annotating API errors.
API error examples in API UI rendered incorrectly - swagger-ui issue?

Removed repository-accessing code from api.models.*

Added /assets/order/{orderId} for fetching info on specific asset order.

NOTE: AssetRepository.getOrdersTrades() now returns trades where order is
initiating or target. (Previously was initiating order only).
qora.assets.Order.orphan() updated to reflect above change.

block-explorer.html fixed to use new API output.
2018-12-13 12:22:46 +00:00
catbref
dcd19f8e42 Transaction-specific info in API results 2018-12-13 08:44:50 +00:00
catbref
cfd8b53fc1 API, HSQLDB
Added more global parameters to /admin/unused API endpoint (formally /admin/dud)
and also managed to remove /admin/unused from API documentation UI.

Added results slicing to /assets/all

Added /assets/orderbook API call that returns open asset orders

Added /assets/trades that returns successful asset trades

Added POST /assets/issue stub

Unified HSQLDB connectionUrl to public variable inside Controller class.

Can't deploy v1 ATs with isFinished=true flag as that prevents later
transactions sending messages (during import of v1 chain).
Some future hard-fork code will need to set all v1 ATs to "finished".

Changed DB's "TransactionRecipients" to "TransactionParticipants" to
properly support API call to find all transactions 'involving' a
specific address. Support code needed in Block and Transaction with
some transaction-specific overrides for Genesis and AT transactions.

Removed old, deprecated calls from Transaction/TransactionRepository

Moved HSQLDB database properties from connection URL to explicit
SQL statements in HSQLDBDatabaseUpdates. They didn't work in
connection URL during DB creation anyway.

Retrofitted HSQLDB Accounts table with public_key column instead of
rebuilding it later.

Fixed incorrect comments in IssueAssetTransactionTransformer regarding
v1 serialization for signing.

Re-imported v1 chain to test latest changes.
2018-12-12 12:13:06 +00:00
catbref
2aaa199c86 API: basic asset info
Added repository support for asset API calls

Added /utils/seed for returning server-generated 32-byte seed
2018-12-11 13:48:10 +00:00
catbref
3829630b29 API: transaction searching
Converted AddressesResource to full base64, removing base58.
Narrowed range of API errors returnable while there.
Added support for looking up public key of address.
Added support for converting public key TO address.

Added API endpoint for returning a range of block signatures,
to aid block explorers.

Added API support for fetching unconfirmed transactions.
Added API endpoint for searching transactions to meet criteria like:
- participating address (only recipients supported ATM)
- block height range
- transaction type
- result count limit/offset

---

Added storage of account's public key in repository
along with supporting code in AccountData and Account
business object to save public key where possible.
2018-12-10 13:27:41 +00:00
catbref
b5c02f49ce Work on API
Rejigged pom.xml, extracting common dependency versions as properties.
Removed extraneous HSQLDB dependency (v2.4.1) as we're using svn r5836 for now.

Removed calls to Security.checkApiCallAllowed() for all API calls EXCEPT /admin/stop.
Throws error if remote IP is not localhost.

Added 'global' OpenAPI parameters to fake /admin/dud endpoint to save copy&pasting.
This will need more tidying in the future, or at least future support from swagger-core.
Code added in AnnotationPostProcessor to insert global parameters in top-level
  OpenAPI components section.

/block-explorer.html hidden from API UI

BlocksResource now expects Base64 block signatures instead of Base58.
Endpoints that return block data also accept optional "includeTransactions"
    query param which does exactly that.
BlockWithTransactions API model added for above.

Some attempt to get transaction-specific data returned but no luck as yet.
(TransactionData, GenesisTransactionData, PaymentTransactionData touched).
See https://github.com/swagger-api/swagger-core/issues/3046

TransactionsResource now has support for optional query params "limit" and "offset"
    so that only a subset of large results can be requested.

UtilsResource added to provide convenient Base64<->Base58 conversions.

/admin/uptime fixed to return uptime from application launch instead of
    instantiation of AdminResource class!

Controller improved to detect repository and API startup failures.

HSQLDBRepositoryFactory now detects when it can't open database and throws.
(Before it would simply hang).

Removed extraneous import from qora.account.Account
2018-12-07 17:42:31 +00:00
catbref
28c2cdaf5b FATJAR packaging + block explorer changes
Switched from maven-assembly-plugin to maven-shade-plugin for
building FATJAR.
When running from FATJAR, class-path is ". .." to help find
log4j2.properties file.

Swagger-UI can now be served direct from inside FATJAR instead
of requiring resources in filesystem.

Default package Start now controller/Controller

block-explorer.html now served via Jetty and modified to use
relative URLs instead of absolute http://localhost:9085/... style

Improved shutdown code in controller

/admin/stop API call disabled for now

Highly permissive settings.json added
2018-12-05 11:32:55 +00:00
catbref
ad9fa9bf9d More work on API plus basic block explorer
Added FATJAR packaging support to pom.xml

Added some "summary" fields to API calls but more need doing.
Corrected path clash from having unnecessary @OpenAPIDefinition annotations.
Added API "tags" to group similar calls (address-based, block-related, etc.)
Fixed addresses/lastreference/{address}
Implemented addresses/lastreference/{address}/unconfirmed
Implemented addresses/assets/{address}
Added /admin/stop and /admin/uptime API calls.
Moved general API info into new src/api/ApiDefinition.java
Added CORS support to ApiService
Added /transactions/address/{address} and /transactions/block/{signature}

Replaced references to test.Common.* to do with repository factory.
 This fixes issues with building FATJAR due to references to test classes
 that are omitted from FATJAR.

Changes to AccountBalanceData, BlockData and TransactionData
 to support JAX-RS rendering to JSON.

Added getUnconfirmedLastReference() to Account.

Added getAllBalances(address) to account repository
 - returns all asset balances for that address.

Added getAllSignaturesInvolvingAddress(address) to account repository
 but currently only uses TransactionRecipients HSQLDB table.
 (And even that wasn't automatically populated).

Included: very basic block explorer to be opened in browser as a file:
block-explorer.html
2018-12-04 16:34:55 +00:00
catbref
90094be95a CIYAM-ATv2 can be used with any asset, not just QORA.
Some initial BTC cross-chain support. (Needs more work).

Unified timestamp for V2 switchover to block version 4,
applicable to several transaction types.

Qora-specific interface to CIYAM ATv2 library.

Beware: some areas still work-in-progress!
2018-12-03 13:05:38 +00:00
catbref
eaad565765 Moved tests to outside of src/ path
This allows them to be excluded from final built package.

pom.xml and .classpath updated

Unable to test due to lack of JUnit 5 in Eclipse Neon
2018-11-02 15:52:09 +00:00